Episode 08 stops measuring one device and puts the two memory levels this CVE2 can actually reach on one table: a 32 KiB on-chip scratch BRAM and 256 MiB of board DDR3. A CPU word read of the scratch reaches only 25.0 MB/s; the same 8 KiB copy through a hardware DMA reaches 199.8 MB/s, about 9x; DDR3 through the DMA reaches 1,422 MB/s read and 733 MB/s write, with a single-beat latency of 241 ns. Two independent board runs end in EP08 RESULT: PASS.
The conclusion first: the SRAM nearest the CPU is not "fast". With no cache and no burst, a CVE2 moving 32-bit data with ordinary loads and stores sees only 22–25 MB/s. Hand the same copy to a hardware DMA and it jumps to 199.8 MB/s. DDR3 at 1,422 MB/s is another order entirely, but only the DMA can reach it; the CPU has no DDR window. These are not three engines in a fair race but a vertical cut through the real memory hierarchy of this SoC.
One firmware pass runs everything: four BRAM measurements, then DDR3 write, read-compare, CRC and ping. The block below is the raw uart_ep08_run1.txt; bytes, cycles, beats and crc_data are read back from RTL counters, not recomputed in a script.
EP08 storage hierarchy on ALINX PI-AR035-F1-1
EP08 VERSION ddr=0x45503038 bram=0x45503038 DDR_SIZE=268435456 CPU_HZ=50000000 UI_HZ=100000000
EP08 BRAM CPU_READ bytes=32768 cycles=65541 MBps=24 sum=0x281ffc00
EP08 BRAM LATENCY loads=32768 cycles=81924 cyc_per_load_x100=250
EP08 BRAM CPU_COPY bytes=8192 cycles=18437 MBps=22 bad=0
EP08 BRAM DMA_COPY bytes=8192 cycles=2050 MBps=199 bad=0
EP08 CALIB: PASS
EP08 ROUND 1 BEGIN
EP08 DDR WRITE bytes=67108864 status=0x000000f5 wr_cycles=9148314 wr_MBps=733 beats=0 err=0
EP08 DDR READ bytes=67108864 status=0x000000f5 rd_cycles=4718705 rd_MBps=1422 beats=4194304 err=0
EP08 DDR CRC bytes=67108864 status=0x000000f5 rd_cycles=4718612 rd_MBps=1422 crc_cycles=167772193 crc_MBps=39 crc_data=0x07e96c24 crc_exp=0x07e96c24 beats=4194304 err=0
EP08 DDR PING n=16 min_ui_cycles=23 max_ui_cycles=36 avg_x100=2412 avg_ns=241 last_addr=0x07800000
EP08 ROUND 1 PASS
EP08 ROUND 2 BEGIN
EP08 DDR WRITE bytes=67108864 status=0x000000f5 wr_cycles=9148282 wr_MBps=733 beats=0 err=0
EP08 DDR READ bytes=67108864 status=0x000000f5 rd_cycles=4718749 rd_MBps=1422 beats=4194304 err=0
EP08 DDR CRC bytes=67108864 status=0x000000f5 rd_cycles=4718625 rd_MBps=1422 crc_cycles=167772164 crc_MBps=39 crc_data=0x07e96c24 crc_exp=0x07e96c24 beats=4194304 err=0
EP08 ROUND 2 PASS
EP08 RESULT: PASS| Path | Engine | Volume | Cycles | Bandwidth | Latency / note |
|---|---|---|---|---|---|
| CPU read BRAM | CVE2 load | 32 KiB / 8,192 words | 65,541 CPU | 25.0 MB/s | 8.0 cycles/word |
| CPU copy BRAM | CVE2 load + store | 8 KiB / 2,048 words | 18,437 CPU | 22.2 MB/s | 9.0 cycles/word, bad=0 |
| DMA copy BRAM | bram_dma | 8 KiB / 2,048 words | 2,050 CPU | 199.8 MB/s | 1.0 word/cycle, bad=0 |
| BRAM dependent load | CVE2 pointer chain | 32,768 loads | 81,924 CPU | — | 2.5 cycles/load = 50 ns |
| DDR3 write | ddr3_dma | 64 MiB | 9,148,314 UI | 733 MB/s | 45.8% of peak |
| DDR3 read-compare | ddr3_dma | 64 MiB / 4,194,304 beats | 4,718,705 UI | 1,422 MB/s | 88.9% of peak, err=0 |
| DDR3 CRC-32 | ddr3_dma | 64 MiB | 167,772,193 UI | 39 MB/s | byte-wise second check |
| DDR3 single-beat ping | ddr3_dma | 16 beats, 8 MiB step | 24.12 UI average | — | min 23 / max 36 = 241 ns |
MBps field is integer-truncated: 24 / 22 / 199. The table recomputes 25.0 / 22.2 / 199.8 from the same cycle counts, so a reader does not divide 24 by 22 and get 1.09. The two independent runs match on every stable field (bytes, beats, CRC, err, PASS); only cycle counts and ping min/max jitter. Run 2 ping is min 23 / max 26 / avg 23.31.Two facts matter. First, the CPU has only a BRAM window (0x2002_0000..0x2002_7FFF) and no DDR3 window. DDR3 addresses, data and CRC are all handled by ddr3_dma; the CPU writes CTRL and reads STATUS and counters. So "CPU memcpy vs DMA" can only be measured on BRAM, and "DDR3 bandwidth" can only be measured through the DMA. The page never merges the two into one curve.
Second, the scratch is a real dual-port BRAM, but only one port is exposed to the CPU. Port A serves CPU loads and stores, Port B serves DMA reads and writes. They never access at the same time: while dma_active is high, CPU scratch requests are gated. The cost is that the CPU cannot touch the scratch during a DMA; the benefit is that 32 KiB infer as 40 BRAM blocks instead of dissolving into LUTRAM.
CPU read. 8,192 sequential word reads take 65,541 CPU cycles, or 8.0 cycles/word. CVE2 has no cache; each load is followed by address increment, accumulation, compare and branch. The load handshake itself is one cycle, the loop is not. 25.0 MB/s is exactly "8 cycles per word × 50 MHz × 4 B".
CPU copy. 2,048 words move from word 0 to word 2,048 in 18,437 cycles, 9.0 cycles/word, 22.2 MB/s. One cycle more than the read because a store is added. Both runs end bad=0, so source and destination do not alias.
DMA copy. The same 8 KiB range is copied by bram_dma in 2,050 CPU cycles, 1.0 word/cycle, 199.8 MB/s. It does not fetch instructions, compute addresses or pass data through CPU registers: source, destination and length go into CSRs, then the state machine issues one read and one write per cycle. The 9.0x gap is not a faster clock — both run at 50 MHz — it is the cost of instruction fetch and loop overhead per word.
Dependent load. A 1,024-node pointer ring runs 32,768 dependent loads in 81,924 cycles, 2.5 cycles/load, 50 ns. This measures the true round trip when the next address depends on the previous data, not streaming throughput. It gives the on-chip baseline for the DDR3 ping: one dependent access is 50 ns in BRAM and 233–241 ns in DDR3, about 4.7x apart.
dma_active is high, CPU scratch accesses are gated, so the firmware only polls CSRs and never touches the scratch during a copy. Before reusing this as a general memory pool, add byte enables or split the ports, otherwise byte and half-word stores silently lose data.DDR3 reuses the Episode 07 MIG native UI and the 128-bit DMA, but fixes the test window at 64 MiB and runs three separate passes: write only, read-compare only, read plus CRC. The pattern still depends on the address (tag={4'hA, addr} expanded to 128 bits), so address decode errors, bank aliasing or stuck data lanes show up on readback; all 4,194,304 beats of 64 MiB compare with err=0.
CRC reads the range again, splits every 128-bit beat into 16 bytes and updates the reflected 0xEDB88320 polynomial byte by byte, with 0xFFFFFFFF as both initial and final XOR. The 64 MiB pass takes 167,772,193 UI cycles (about 1.68 s) and returns 0x07e96c24, equal to the expected value. It issues one read at a time and computes serially, hence 39 MB/s; its value is a second check independent of the pattern compare, not CRC throughput.
Single-beat ping. Sixteen reads step the address by 8 MiB to cross banks and rows; the RTL records the UI-cycle difference from an accepted app_en to app_rd_data_valid. Run 1 gives min 23 / max 36 / avg 24.12 UI cycles, or 241 ns; run 2 gives min 23 / max 26 / avg 23.31. The 36 is one long queueing event; the stable lower bound is 23 cycles, 230 ns.
| Mode | CTRL | Coverage | Key result |
|---|---|---|---|
| Write only | wr=1, crc=0, rd=0 | 64 MiB linear | 9,148,314 UI cycles / 733 MB/s |
| Read-compare | rd=1, wr=0, crc=0 | 64 MiB linear | 4,718,705 UI cycles / 1,422 MB/s / 4,194,304 beats / err=0 |
| Read + CRC | rd=1, crc=1 | 64 MiB linear | 167,772,193 UI cycles / 39 MB/s / 0x07e96c24 |
| Ping | ping toggle | 16 × single beat, 8 MiB step | min 23 / max 36 / avg 24.12 UI cycles |
| Comparison | Ratio | How to read it |
|---|---|---|
| DMA copy ÷ CPU copy (both 8 KiB) | 199.8 / 22.2 = 9.0x | Same 50 MHz, same BRAM; the difference is whether data passes through instruction fetch and a loop. This is the cleanest comparison in this episode. |
| DDR3 read ÷ DMA BRAM copy | 1,422 / 199.8 = 7.1x | DDR3 is 100 MHz 128-bit burst; BRAM DMA is 50 MHz 32-bit single word. Width and clock both differ. |
| DDR3 read ÷ CPU BRAM read | 1,422 / 25.0 = 56.9x | One side is a DMA 128-bit burst, the other a CPU word load. It shows the hierarchy gap, not that "DDR is faster than SRAM". |
| DDR3 write ÷ DMA BRAM copy | 733 / 199.8 = 3.7x | The write path still pays the BL8 rhythm of data one cycle before command. |
| DDR3 ping ÷ BRAM dependent load | 241 / 50 = 4.8x | Ping includes command transfer and cross-bank/row queueing; BRAM is an on-chip round trip. |
DDR3 peak is still 16-bit × 800 MT/s = 1.6 GB/s. Read reaches 1,422 MB/s, about 88.9%; write 733 MB/s, about 45.8%; CRC 39 MB/s, about 2.4%. Read is fast because addresses and commands stream and 32 outstanding reads hide latency. Write is slower because this version keeps the factory self-test rhythm of data one cycle before command, so every BL8 occupies two UI cycles and the ceiling sits near half of peak. Issuing data and command together is the next direction, but it is not verified here.
| Item | Result | Note |
|---|---|---|
| LUT | 9,581 / 20,800 · 46.06% | 248 more than Episode 07 for scratch and bram_dma control |
| FF | 9,212 / 41,600 · 22.14% | Two DMA state machines and ping timing |
| BRAM | 40 / 50 · 80.00% | 32 KiB scratch (32 blocks) + CVE2 RAM + FIFO |
| DSP | 11 / 90 · 12.22% | Used inside MIG |
| IOB | 73 / 250 · 29.20% | 48 DDR3 signals plus board peripherals |
| WNS / WHS | 0.853 / 0.036 ns | TNS / THS 0, 26,767 endpoints, 0 failing |
| DRC | 0 error / 0 warning | Final implementation report |
The final bit is 2,192,137 B, SHA-256 f07d96c23ce78ee835b19ea932099f03df99e7d6d25449de5c0600c6de1a868c. BRAM rises from 32 blocks in Episode 07 to 40, exactly the price of the 32 KiB scratch; LUT grows by only 248 because the memory did not dissolve into LUTRAM.
Timing did not converge on the first try: the first final implementation had WHS −0.040 ns and 2 failing hold endpoints, and a single phys_opt_design -hold_fix inside impl_cdc_ep08.tcl was not enough. The flow switched to launch_runs impl_1, waited for the managed run, then applied two phys_opt_design -hold_fix passes and route_design, reaching 0.853 / +0.036 / 0 failing. The −0.040 bit was discarded; only the final version ships.
dpram_scratch.sv used byte enables and a two-read/one-write inference, producing 17,252 LUTRAM and a place_design UTLZ-1 failure. Adding ram_style="block" alone did nothing; only a classic true-dual-port template (Port A write/read, Port B read, no byte enables, 32-bit full-word writes) inferred 40 BRAM blocks and cut LUT from 26k to 9.5k.cpu_rvalid only responded to reads, so a store never saw rvalid. Fixed with cpu_rvalid <= cpu_en, returning a response for writes too.{bram_done,bram_busy} put done on bit1 and busy on bit0, so firmware polling bit1 for busy waited forever. Changed to {bram_busy,bram_done}.!ping_pulse to the start branch lets both coexist.crc_cycles was 0. Raising the wait_ddr guard to 20 s returned the full result.mbps formula halved the DDR number; it was split into mbps_ui (×100) and mbps (×50).rd_start. A leftover start from the previous pass added an extra state visit. rd_start <= 0 was added.phys_opt_design -hold_fix; a managed run plus two passes and a re-route converged.# build machine
ssh alinx@192.168.8.237
cd /media/alinx/nvme4t/fpga-love/pi_ar035_f1_1_cve2_ep08
# 1) synthesis + implementation + reports; final timing needs the managed run
vivado -mode batch -source build_ep08.tcl
vivado -mode batch -source impl_run_ep08.tcl
# 2) firmware (the HEX is embedded in the bit, so a C change needs re-synthesis)
cd sw && make && python3 bin2hex.py && cd ..
# 3) start capture first, then program; trim at the last EP08 storage hierarchy line
python3 cap_serial.py /dev/ttyUSB1 115200 /tmp/ep08.log &
vivado -mode batch -source program_ep08.tclThe second independent run means a full power cycle and reconfiguration, another program_ep08.tcl, and a fresh UART capture. Compare bytes, beats, CRC, err and the final PASS; cycle counts and ping min/max may jitter, stable fields must match. Restarting only the CPU without resetting the FPGA does not count as a second independent download.
EP08 storage hierarchy line; otherwise old output contaminates the new evidence.RTL, XDC, Tcl, firmware sources and prebuilt outputs, the final bit, three reports, two independent UART logs, build and programming logs, the MIG configuration and the DDR3 pin table are all below. Checksums are in SHA256SUMS.txt; the files/ trees on the Chinese and English pages are identical.
Daily Briefing · Weekly Digest · Monthly Deep Dive