User Space Networking on RISC-V: F-Stack & Custom PMD

Project Status: Handover Ready
Target Architecture: RISC-V (COMPAS Lab FPGA)
Report Reference: Final Report - User Space Networking.pdf

This repository contains the software artifacts developed to implement a User-Space Kernel Bypass network stack on the COMPAS Lab RISC-V FPGA platform. It serves as the technical companion to the "Final Report - User Space Networking", containing the source code for the custom drivers, architecture ports, and configuration tuning described in Chapters 5, 6, and 7.

---

📂 Repository to Report Mapping

This codebase aligns directly with the architectural methodology detailed in the report. Use this table to navigate the files based on the report chapters:

Component	Directory / File	Description	Report Ref
Custom Driver	COMPAS_PMD/	The Poll Mode Driver (PMD) for the proprietary COMPAS NIC.	Chapter 7.2
F-Stack Port	FSTACK_RISCV_PORT/	Patches to port libfstack and FreeBSD 11.0 to RISC-V.	Chapter 6.2
Build System	riscv_cross.txt	Meson cross-compilation profile for riscv64-linux-gnu.	Chapter 7.3
Runtime Config	f-stack.conf	Tuned configuration (lcore mask 0x4000, correct PCI whitelist).	Chapter 6.4
App Config	nginx.conf	Nginx configuration tuned for single-process F-Stack execution.	Chapter 5.4

---

🔧 Technical Components

1. The Custom Poll Mode Driver (COMPAS_PMD)

Context (Report Ch 7.2): The COMPAS NIC lacks a native DPDK driver. We implemented a custom PMD to map the hardware's MMIO registers directly into user space using vfio-platform.

• Key Logic:
  • Burst Functions: eth_rx_burst and eth_tx_burst replace kernel interrupt handlers with a tight polling loop (Ch 7.2.2).
  • Memory Ordering: Injected fence ow,ow assembly instructions to solve the RISC-V "Weak Memory Ordering" bug where the NIC read stale descriptors (Ch 7.2.3).
  • Doorbell: Implements the logic to trigger the FPGA DMA engine.

2. F-Stack RISC-V Port (FSTACK_RISCV_PORT)

Context (Report Ch 6.2): F-Stack assumes x86 architecture and a multi-process environment. We ported it to RISC-V and patched it for single-process debugging.

• Key Patches:
  • arch_ctx.S: Custom assembly for low-level context switching (saving ra, sp, s0-s11) required by F-Stack's micro-threading on RISC-V.
  • Initialization Fixes: Patched ff_dpdk_if.c to handle uninitialized proc_id and nb_procs variables that caused crashes in single-process mode (The "Single-Process Validation Failure" - Ch 6.2.1).
  • FreeBSD Headers: Helper script setup_freebsd_riscv.sh to fetch the necessary "glue" headers from the FreeBSD source tree.

3. Tuning & Configuration

Context (Report Ch 5 & 6): Running a user-space stack requires precise system tuning.

• f-stack.conf: Pre-configured with the specific bitmask (lcore_mask=0x4000) and PCI address (0000:81:00.0) required for the murphys FPGA environment.
• nginx.conf: Includes the fstack_conf directive and process affinity settings validated during the "Phase I: x86 Baseline" testing.

---

🚀 Getting Started (Handover Guide)

Follow this deployment roadmap to replicate the environment (as described in Chapter 7.3).

Phase 1: Integrate the Driver

1. Copy the driver source into your DPDK tree:

   cp -r COMPAS_PMD/drivers/net/compas /path/to/dpdk/drivers/net/

2. Rebuild DPDK using the provided cross-file:

   meson setup build --cross-file riscv_cross.txt -Ddrivers=net/compas
   ninja -C build

Phase 2: Patch F-Stack

1. Overlay the port files onto your F-Stack source:

   cp -r FSTACK_RISCV_PORT/* /path/to/f-stack/

2. Use the helper script to fetch missing FreeBSD RISC-V headers:

   cd /path/to/f-stack/lib
   ./setup_freebsd_riscv.sh

3. Build libfstack using your RISC-V toolchain.

Phase 3: Runtime

1. Boot the FPGA: Follow the instructions in RISCV_FLASHING.md to program the bitstream and connect to the console.
2. Deploy Configs:
   • Place f-stack.conf in the same directory as your binary.
   • Ensure nginx.conf is in the Nginx conf/ directory.
3. Run:

   # Example for starting Nginx with F-Stack support
   ./nginx -c nginx.conf

Note on "Ghost Library" Errors (Ch 6.3): When building on a shared machine, ensure FF_PATH is exported to point to your local patched F-Stack directory to avoid linking against stale system libraries.