Language-model hardware shaped by real inference behavior.
A proposed RISC-V-controlled accelerator tile for quantized transformer inference, with vector and matrix execution designed alongside memory movement and software.
Architecture and exploration phase
Begin with a narrow, measurable inference target.
The initial direction focuses on dense decoder-only GGUF models, batch-one execution, and an approximate 0.5B–1.5B parameter envelope. A 3B model remains a stretch target.
- 01Prefill and decode separated
- 022K–4K context baseline study
- 03Approximately 8K stretch direction
Scalar control coordinates specialized compute.
A small RISC-V control processor manages queues, firmware, and execution. An RVV-family vector engine handles elementwise work, reductions, and support operators alongside the proposed matrix engine.
An 8 × 8 fabric is a study point, not a frozen answer.
The current physical candidate uses eight activation lanes and eight weight lanes. GEMM and GEMV dataflows, reduction behavior, utilization, and complete quantization handling remain under evaluation.
Candidate: four-bit weights, eight-bit activations, INT32 accumulation.
Scratchpad capacity and DMA behavior are first-class architecture choices.
Explicitly managed local memory keeps hot tiles and intermediate data close to compute. Asynchronous DMA connects that local working set to external DDR or LPDDR through the host or SoC fabric.
- 01128 KB study
- 02256 KB working baseline
- 03512 KB study
The unresolved work is visible by design.
Matrix dataflow, scratchpad banking, command granularity, memory bandwidth, quantization coverage, and prefill/decode balance must be resolved through modeling, implementation, and verification.