EP08 - Storage hierarchy: BRAM vs DDR3 on ALINX PI-AR035-F1-1 ============================================================ Board : ALINX PI-AR035-F1-1, XC7A35T-FGG484-2 DDR3 : MT41J128M16XX-125, 16-bit, 400 MHz DDR, 256 MiB CPU : OpenHW CVE2 / CV32E20, RV32IM, 50 MHz UI : MIG 7-series native app interface, 4:1, 128-bit, ui_clk = 100 MHz BRAM : 32 KiB scratch, 8,192 x 32-bit, true dual port What is measured ---------------- 1. A CPU loop reads all 8,192 words of the scratch BRAM (32 KiB). 2. A CPU loop copies 2,048 words (8 KiB) from word 0 to word 2,048. 3. The same 8 KiB copy is done by a word-granularity DMA, CPU only polls CSR. 4. A dependent-load pointer chase runs over a 1,024-node ring (32,768 loads). 5. The DDR3 DMA writes a 64 MiB pattern, reads it back beat by beat, then reads it again while computing a byte-wise CRC-32. 6. The DDR3 DMA performs 16 single-beat ping reads, stepping 8 MiB per sample across banks/rows, and records the ui_clk cycles from command accept to app_rd_data_valid. The CPU has no direct DDR3 window. DDR3 is reachable only through the DMA; the comparison is therefore "CPU + on-chip SRAM" versus "DMA + board DRAM", which is the real hierarchy of this SoC. CSR map ------- DDR3 DMA, base 0x2001_0000 +0x00 CTRL bit0 start toggle, bits[3:1] size (0=1 MiB, 1=16 MiB, 2=256 MiB, 3=64 MiB), bit4 CRC enable, bit5 write enable, bit6 read enable, bit7 ping toggle +0x04 STATUS bit0 calib, bit1 busy, bit2 done, bit3 failed, bit4 crc_ok, bits[8:5] state +0x08 SIZE 0x1000_0000 (256 MiB device) +0x0c WR_CYCLES ui_clk cycles, write phase +0x10 RD_CYCLES ui_clk cycles, compare phase +0x14 CRC_CYCLES ui_clk cycles, CRC phase +0x18 BEATS returned 128-bit beats +0x1c ERRORS mismatched beats +0x20 FIRST_ERR first mismatched app_addr +0x24 CRC_DATA computed CRC-32 +0x28 CRC_EXP expected CRC-32 +0x2c VERSION 0x4550_3038 +0x30 PING_LAT last ping latency, ui_clk cycles +0x34 PING_ADDR last ping address Scratch BRAM, base 0x2002_0000 .. 0x2002_7FFF (32 KiB) Word index = address[14:2]. CPU word loads/stores only; the test firmware does not perform byte or half-word stores to this region. BRAM DMA, base 0x2003_0000 +0x00 CTRL bit0 start toggle +0x04 SRC source word index +0x08 DST destination word index +0x0c LEN words to copy +0x10 STATUS bit1 busy, bit0 done +0x14 CYCLES CPU-clock cycles from start to last write +0x18 VERSION 0x4550_3038 Results (run 1; run 2 repeats the same stable values) ------------------------------------------------------ BRAM CPU read 32,768 B 65,541 cycles 25.0 MB/s BRAM CPU copy 8,192 B 18,437 cycles 22.2 MB/s BRAM DMA copy 8,192 B 2,050 cycles 199.8 MB/s BRAM dependent load 32,768 loads 81,924 cycles 2.50 cycles/load (50 ns) DDR3 write 64 MiB 9,148,314 ui_clk 733 MB/s DDR3 read 64 MiB 4,718,705 ui_clk 1,422 MB/s DDR3 CRC 64 MiB 167,772,193 ui_clk 39 MB/s DDR3 beats 4,194,304, errors 0 DDR3 CRC-32 data = expected = 0x07e96c24 DDR3 ping 16 samples, min 23 / max 36 / avg 24.12 ui_clk = 241 ns Ratio highlights: DMA BRAM copy is 9.0x the CPU word copy; DDR3 read is 7.1x the DMA BRAM copy and 56.9x the CPU scratch read. DDR3 write is 3.7x the DMA BRAM copy. BRAM single dependent load is 50 ns; DDR3 single-beat ping is 233-241 ns including command transfer and return. Two independent FPGA configurations were captured. The UART logs have the same 19 lines and identical stable fields (bytes, beats, CRC, MB/s, errors, RESULT). Only the cycle counters and ping min/max differ by normal jitter. Timing / resources (final bitstream) ------------------------------------ WNS 0.853 ns, WHS 0.036 ns, TNS/THS 0, 0 failing endpoints, 26,767 endpoints. Two post-route phys_opt_design -hold_fix passes were needed; an earlier route had WHS -0.040 ns and 2 failing hold endpoints and was discarded. LUT 9,581 (46.06%), FF 9,212 (22.14%), BRAM 40/50 (80%), DSP 11, IOB 73. DRC 0 errors, 0 warnings. Bitstream 2,192,137 B, SHA-256 f07d96c2...de1a868c. Boundaries ---------- - Native MIG UI, not AXI4; no AXI interconnect or AXI-to-native adapter. - No CPU-visible DDR3 window: CPU only programs CSRs and polls status for DDR. - Scratch BRAM accepts full 32-bit words only in this test; byte enables are not wired, and CPU scratch accesses are blocked while the BRAM DMA is busy. The firmware respects this by polling CSRs and not touching the scratch RAM during a DMA copy. - CRC is byte-wise and deliberately throttled; it measures correctness, not peak CRC throughput. - Ping samples one beat per address; it measures command-to-data latency, not sustained random-access throughput. - No ECC, no DDR3 margin sweep, no voltage/temperature corner, no long soak. - Throughput is derived from cycle counters at fixed 50 MHz CPU / 100 MHz UI. Build ----- Vivado 2025.2, part xc7a35t-2fgg484. vivado -mode batch -source build_ep08.tcl Final timing used impl_run_ep08.tcl (managed impl_1 plus two hold-fix passes). Firmware: cd sw && make The bitstream embeds sw/firmware.hex through two simple_ram INIT_FILE parameters, so a firmware change requires re-synthesis.