About This Research Area

We design the arithmetic units and accelerator architectures that make deep learning cheaper to compute — from the multipliers inside a processor up to full FPGA and ASIC accelerators.

Multiple-precision fused multiply-add unit

About the Project

Bar chart comparing silicon area across eight fused multiply-add designs from half to quadruple precision.
Silicon area across the FMA designs.

The fused multiply-add unit is the workhorse of every AI chip. We built one that supports quadruple, double, single and half precision in a single four-stage pipeline, plus mixed-precision modes — low-precision multiply with high-precision accumulate.

  • Four precisions in one four-stage pipeline
  • Mixed-precision modes: low-precision multiply, high-precision accumulate
  • Only 38% more area than a quadruple-precision unit alone

Zhang, Chen and Ko, IEEE Transactions on Computers, 2019

Approximate multipliers and dividers

About the Project

Block diagram of a Booth multiplier using two Booth decoders and a ripple carry adder.
The proposed BD8.2 approximate Booth multiplier.

Many AI operations tolerate small arithmetic errors. We exploit that, trading exactness for large savings in power and area where the loss in accuracy is negligible.

  • Approximate multipliers built from OR gates and simplified compressors
  • Approximate Booth multipliers with reduced decoders
  • Approximate restoring dividers

Haider, Zhang and Ko, IEEE Transactions on Computers, 2023; Venkatachalam and Ko, IEEE TVLSI, 2017.

Deep neural network accelerators

About the Project

Block diagram of an accelerator processing element with input, filter and partial-sum scratchpads.
Processing element with zero detection.

GPUs are fast but expensive and power-hungry. We design dedicated accelerators that do less work for the same result.

  • Local data reuse through shared scratchpads between processing elements
  • Early detection of negative output features so they can be skipped
  • Run-time pruning of ineffectual computations to keep utilisation high

Asadikouhanjani and Ko, IEEE TCAS-II, 2020; IEEE TCAS-I, 2021.

FlexPWL: multiplier-free activation functions

About the Project

Two dataflow diagrams comparing a multiplier-based activation unit with a multiplier-free design.
A multiplier-based activation unit beside the multiplier-free FlexPWL design.

Activation functions are awkward in hardware because they need multipliers. FlexPWL replaces the multiplier with a shift table and a barrel shifter, approximating any activation function as a flexible piecewise-linear curve.

  • No multiplier: a shift table and barrel shifter do the work
  • Approximates any activation function, not one fixed shape
  • Scalable, and far cheaper on FPGA

Fard, Arias-Garcia, Zhang and Ko, IEEE Transactions on Computers, 2026

Sparse data structures and microarchitecture

About the Project

Table and vector diagram mapping sparse matrix elements to row, column and non-zero index vectors.
The CSM sparse matrix representation.

Real neural networks are full of zeros. CSM is a memory-efficient sparse matrix format that executes in parallel on accelerators, outperforming COO and CSR on sparse matrix-vector multiplication. Alongside it we design a processor microarchitecture built specifically for sparse networks.

  • CSM: a parallel-friendly sparse format that beats COO and CSR
  • A power-efficient pipeline designed around sparsity rather than retrofitted
Five-stage processor pipeline with instruction memory, registers, ALU and data memory.

Power-efficient microarchitecture for sparse deep neural networks.

Spiking neural network processing

About the Project

Table comparing frequency, LUTs, registers, power and energy for three number formats.
Resource and energy comparison across number formats.

Spiking networks are the third generation of neural networks and are naturally energy-efficient. Our work asks how little precision they can get away with.

  • A 7-bit fixed-point format matches single-precision accuracy
  • Variable-precision processing elements
  • Approximate adders to cut resource use further

Also in this area

Further work

  • Posit-format and variable-precision approximate floating-point multipliers
  • A binary transformer accelerator for edge devices on Xilinx Zynq UltraScale+ (Du, Ko and Zhang, ISCAS 2024)
  • TF2RISC-V, an end-to-end framework that cuts the cost of simulating and validating inference testbenches on RISC-V