The Shenzhen Innovation Light Source Facility (SILF), as a 4th light source, is an accelerator-based multidiscipline user facility planned to be constructed in Shenzhen, Guangdong, China. The accelerator complex is composed of a 200 MeV linac, a booster with ramping energy from 0.2 GeV to 3.0 GeV, and a 3.0 GeV storage ring, and two e-beam transport lines for injection and extraction among accelerators. The circumference of the storage ring is 696 m, which includes 28 hybrid seven-bend achromat (H7BA) lattice periodic units to achieve the emittance below 100 pm·rad. SILF needs to control the beam orbit change within 10% of the cluster size within a certain frequency. In order to meet the beam orbit stability requirements, it is necessary to establish a fast orbit feedback (FOFB) system with field programmable gate arrays (FPGA) to reduce feedback latency and increase bandwidth. The FOFB system adopt 28 sub-stations with the same hardware and software function to obtain bias-data from the beam position monitors (BPMs) data using 2.38Gbps in the SFPs and send correct-data to the fast corrector power supplies using a serial point-to-point link around the storage ring, and each substation share BPMs data with daisy-chained using of 10Gbps in the SFPs. This paper introduces the FOFB system design outline and progress. Some technical plans and schedules are also discussed.