A 29-layer int8 convolutional network resides entirely on a single mid-to-low-capacity FPGA without external DDR, measured at about 19.6 inferences per second. This deployment on the Jingwei Qili HME-P3 suggests that for always-on access-control sensing, the cost structure may deserve more attention than peak compute.
The system consists of three nodes: a PC, an FPGA, and a Jetson. The camera outputs 320×240 YUYV; the PC takes only the Y plane, without scaling or grayscale conversion, splits each frame into 60 UDP packets, and sends them to the FPGA at 20 fps. The FPGA independently performs 96×96 preprocessing, 29-layer int8 inference, and result return, while forwarding the same frame unchanged to the Jetson; only when a person is detected does it wake the Jetson to run YOLOv8-pose and return 17 keypoints. The FPGA stays always on for low-cost detection, the Jetson starts on demand, and the PC does not participate in inference. The essence of this division is to keep always-on power on the FPGA side and trade event triggering for GPU duty cycle.
The network is MobileNetV1 (α=0.25, 96×96 grayscale), with 27 convolutional layers plus GAP and FC. Weights are per-channel symmetric int8, activations are per-tensor symmetric int8, convolution multiplies int8 by int8 and accumulates to int32, and BN is folded into bias. The whole network has about 214,000 parameters; the INT8 weights, about 214 KB, reside permanently in BRAM, the input RAM is 1152×64-bit, and no external DDR is needed. Official resource usage is BRAM 112/180 blocks (62.2%), LUT 12.1%, REG 8.4%, and DSP 3.9%. The bottleneck falls on storage rather than compute: adding layers or channels later, or doing full-frame double-bank buffering, will hit BRAM first; on the compute side there is still room for parallelism.
The input RAM has a single write port and writes are prohibited during inference, so frame control works on a whole-frame basis: it starts only after all 60 packets and 76,800 pixels have been received; if a packet or the last packet is lost, the entire frame is dropped and the accumulator is reset; if a new frame header arrives during inference, drop_next is set and the entire frame is dropped. On the engine side, one DSP56V4 is configured as an 8×10×10 multiply-accumulate array, performing 8 int8 MACs per cycle, for about 7.16 million MACs per frame across the whole network; simulation gives 2.87 million cycles at 125 MHz, about 23 ms, and on board, after dividing ce by two, about 46 ms per frame, corresponding to an engine ceiling of 21 fps. Measured detection is about 19.6 times per second, and the person and empty golden vectors match the PC bit-true engine at board level.
For board engineers building always-on vision front ends, the reusable part of this approach is not the model itself but the frame control and storage budget: whole-frame gating, single-write-port input RAM, and weights resident in BRAM are all structures that can be carried directly into one's own pipeline. In terms of device selection, mid-to-low-capacity parts such as the P3 suit networks with weights in the couple-hundred-KB range and resolutions on the order of 96×96; once channels are added or full-frame buffering is required, BRAM tops out first, and one must move to a larger device or switch to block-based reuse. The toolchain side is not covered in the source material, so evaluation must independently confirm the degree of support for bit-exact alignment between the quantization flow and the reference model.
Daily Briefing · Weekly Digest · Monthly Deep Dive