About This Research Area

Large models do not fit on edge hardware. We shrink them — through pruning, quantization and architecture search — while holding on to accuracy.

Transformer compression with a short training proxy

About the Project

Diagram comparing a twelve-layer BERT model with a compressed six-layer version.
Twelve-layer BERT beside its compressed six-layer counterpart.

Deciding which layers of a transformer to prune is expensive: for BERT there are 924 ways to choose six layers from twelve, and evaluating each one normally means a full training run.

  • The relative ranking of pruning configurations is already visible after a small fraction of training
  • That short proxy lets us explore 100% of the search space at a fraction of the cost
  • Applied to BERT layer selection, where exhaustive search is otherwise infeasible

Haider, Valarezo-Plaza, Muhsin, Zhang and Ko, IEEE ISCAS, 2024.

Energy-efficient CNNs through pruned ternary quantization

About the Project

Training dataflow converting full-precision weights through pruning and sign extraction into ternary weights.
The pruned ternary quantization training dataflow.

Ternary quantization reduces every weight to one of three values. Existing methods pick the pruning threshold from instantaneous statistics, which is unstable from batch to batch.

  • We take an exponential moving average across training instead
  • The threshold becomes far more reliable
  • Better accuracy-to-energy trade-off on image classification

Also in this area

Further work

  • LYRA: low-frequency rank adaptation using factored DCT coefficients for parameter-efficient fine-tuning of transformers (Muhsin and Ko, IEEE Signal Processing Letters, 2026)
  • Reconfigurable multi-precision multipliers for CNN acceleration
  • Surveys of neuromorphic computing for power-efficient neural networks