Neural Network Quantization for Microcontrollers

Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications

Hamza A. Abushahla* Dara Varam* Ariel Justine N. Panopio* Mohamed I. AlHajri†

Department of Computer Science and Engineering, American University of Sharjah, Sharjah, UAE

*Equal contribution  ·  †Corresponding author: mialhajri@aus.edu

Read the survey GitHub BibTeX

Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications

This survey provides a hardware-oriented perspective on neural network quantization, systematically reviewing the quantization methods most relevant to MCUs and extreme-edge devices.

1 Introduction

1.1 Existing Surveys

Table 1: Scope comparison of surveys on quantization, TinyML, and edge AI.
Paper Primary Quantization Advanced Quantization Numeric Representations Hardware Landscape Software Frameworks Applications
Gholami et al. FullFullARM RISC-V
Orășan et al. ARMFullFull
Giordano et al. ARM RISC-V NPU
Saha et al. FullPartialFullFull
Ray FullPartialARM RISC-VFullFull
Rokh et al. FullFull
Akkad et al. RISC-VFull
Liu et al. FullFullFull
Liu et al. RISC-V
Capogrosso et al. FullARM RISC-VFull
Liu et al. FullFullFull
Heydari and Mahmoud Full
Wang and Jia FullPartialFullFull
Somvanshi et al. FullPartialPartialARM RISC-V NPUFullFull
Lê et al. FullPartialPartialARM RISC-VFullPartial
Ours (2026)FullFullFullARM RISC-V NPUFullFull
surveyedpartially surveyednot coveredHardware Landscape lists the architectures each survey covers

2 Preliminaries

2.1 Numerical Representations

FP32 (IEEE 754 single precision)Sign bit S — 1 means negative1Exponent e = 129, biased by 127, so the true exponent is 210000001Mantissa m = 0.5625, with an implicit leading 1 giving 24 bits of effective precision10010000000000000000000313023220sign bit S (1 bit)exponent e (8 bits)mantissa m (23 bits)dynamic range ≈ 1.2×10−38 … 3.4×1038the bits above encode n = −6.25Signed INT8Sign bit — INT8 is stored in two’s complement1Seven magnitude bits1111010Unsigned INT8Eight magnitude bits — no sign bit111110107070range [−128, 127]range [0, 255]these bits = −6the same bits = 2504× smaller than FP32 · integer MAC units only · no floating-point unit required
Hover a field to see what it contributes. The bits shown are one worked example: FP32 −6.25, and the eight INT8 bits that read as −6 signed or 250 unsigned.
Figure 2: (Top) Representation of the FP32 format, (bottom left) signed INT8 format and (bottom right) unsigned INT8 format.

2.3 Quantization Design Choices for Deployment

2.3.1 Uniform and Non-Uniform Quantization

xxintEqual steps — this interval is 10% of the range, like every otherEqual steps — this interval is 10% of the range, like every otherEqual steps — this interval is 10% of the range, like every otherEqual steps — this interval is 10% of the range, like every otherEqual steps — this interval is 10% of the range, like every otherEqual steps — this interval is 10% of the range, like every otherEqual steps — this interval is 10% of the range, like every otherEqual steps — this interval is 10% of the range, like every otherEqual steps — this interval is 10% of the range, like every otherEqual steps — this interval is 10% of the range, like every otherUniformxxintUnequal steps — this interval is 27% of the range; levels are dense near zero and sparse in the tailsUnequal steps — this interval is 11% of the range; levels are dense near zero and sparse in the tailsUnequal steps — this interval is 6% of the range; levels are dense near zero and sparse in the tailsUnequal steps — this interval is 4% of the range; levels are dense near zero and sparse in the tailsUnequal steps — this interval is 2% of the range; levels are dense near zero and sparse in the tailsUnequal steps — this interval is 2% of the range; levels are dense near zero and sparse in the tailsUnequal steps — this interval is 4% of the range; levels are dense near zero and sparse in the tailsUnequal steps — this interval is 6% of the range; levels are dense near zero and sparse in the tailsUnequal steps — this interval is 11% of the range; levels are dense near zero and sparse in the tailsUnequal steps — this interval is 27% of the range; levels are dense near zero and sparse in the tailsNon-uniform
Hover a step to see how wide its interval is relative to the full range.
Figure 3: Uniform quantization (left) vs. Non-uniform quantization (right). Real values \(\mathbf{x}\) are mapped into lower precision integer values \(\mathbf{x}_{\text{int}}\). The distances between the quantized values (quantization levels) are the same in uniform quantization, whereas they can vary in non-uniform quantization.

2.3.2 Symmetric and Asymmetric Quantization

Symmetric Signeds · xint-8Quantization level 1 of 7 — consecutive levels are one scale s apartQuantization level 2 of 7 — consecutive levels are one scale s apartQuantization level 3 of 7 — consecutive levels are one scale s apartQuantization level 4 of 7 — consecutive levels are one scale s apartQuantization level 5 of 7 — consecutive levels are one scale s apartQuantization level 6 of 7 — consecutive levels are one scale s apartQuantization level 7 of 7 — consecutive levels are one scale s apart−128127Real zero and the integer zero-point align00−max|x|max|x|s is the scale: the real-valued width of one quantization stepss = max|x| / 127, z = 0Symmetric Unsigneds · xuint-8Quantization level 1 of 6 — consecutive levels are one scale s apartQuantization level 2 of 6 — consecutive levels are one scale s apartQuantization level 3 of 6 — consecutive levels are one scale s apartQuantization level 4 of 6 — consecutive levels are one scale s apartQuantization level 5 of 6 — consecutive levels are one scale s apartQuantization level 6 of 6 — consecutive levels are one scale s apart255Real zero and the integer zero-point align00max(x)s is the scale: the real-valued width of one quantization stepss = max(x) / 255, z = 0Asymmetrics · (xuint-8 − z)Quantization level 1 of 7 — consecutive levels are one scale s apartQuantization level 2 of 7 — consecutive levels are one scale s apartQuantization level 3 of 7 — consecutive levels are one scale s apartQuantization level 4 of 7 — consecutive levels are one scale s apartQuantization level 5 of 7 — consecutive levels are one scale s apartQuantization level 6 of 7 — consecutive levels are one scale s apartQuantization level 7 of 7 — consecutive levels are one scale s apart0255Real zero does not sit at integer 0: the zero-point is shifted00The grid is shifted by −sz so that real zero is exactly representable−szmin(x)max(x)s is the scale: the real-valued width of one quantization stepss = (max−min) / 255, z = ⌊−min(x)/s⌉
Hover a quantization level, the brace, or a zero marker to see what each one stands for.
Figure 4: Illustration of different quantization schemes: symmetric signed (left), symmetric unsigned (center), and asymmetric (right). Red markers denote the INT8 quantization range. The black labeled “0” on the real axis indicates the true zero in floating-point space, while the red zero shows its corresponding integer location after quantization. In symmetric quantization, the true zero and the mapped zero-point align, whereas in the asymmetric case, the zero-point is shifted.

2.3.5 Quantization Granularity

Input: xLayer 1Layer 2⋯Layer Loutput: ŷGlobal quantization shares one scale and one zero-point across every weight tensor in the network, so every layer uses the same gridW1W2WL…Extract a single s, z from W1, W2, …, WL
Hover the shaded region: every layer’s weights fall under the same single set of quantization parameters.
Figure 5: Simplified illustration of global quantization, where the weight tensors \(W_1, W_2, \ldots, W_L\) of all \(L\) layers share the same quantization parameters.
⋮Layer lInput: xMore GranularityPer Tensor(1 scale, (s, z))Per Channel(1 scale per channel)Per Group(1 scale per blockwithin a channel)
Hover the cubes: each highlighted region shares one set of quantization parameters.
Figure 6: Simplified illustration of quantization granularity for the weight tensor \(W_l\) of a given layer \(l\). Quantization can be done per-tensor, per channel, or per group.

3 Primary Quantization Techniques

The full-precision model QAT starts fromInitial /Pre-trained ModelQAT needs the full training set, because it retrains the modelFull DatasetQuantization is simulated in the forward pass while approximate gradients flow backward, so the weights learn to tolerate it. Weights are still stored in FP32 at this point.Training / Finetuning with QuantizationThe trained FP32 weights are finally converted to the integer grid: training alone does not produce a quantized modelQuantizationHigher accuracy, at the cost of a training runQAT Quantized ModelQATThe same full-precision starting pointPre-trained ModelPTQ needs only a small unlabelled sample, enough to estimate activation rangesCalibration DatasetActivation ranges are measured on the calibration set to fix the scale and zero-pointCalibrationWeights and activations are mapped to the integer gridQuantizationCheaper and faster to obtain; no retrainingPTQ Quantized ModelPTQ
Hover any stage for what happens there and what it costs.
Figure 7: Overview of the two primary quantization paths: quantization-aware training (QAT) and post-training quantization (PTQ).

3.2 Quantization-Aware Training

1Forward Pass23Backward Pass4+Full Precision Weights1.12.2⋯6.48.9⋮⋱⋮7.5⋯1.2W (FP32)Quantized Weights12⋯69⋮⋱⋮8⋯1Q (INT)Gradients (w.r.t FP32Weights)0.10.3⋯-0.10.1⋮⋱⋮-0.1⋯0.2dLdW(FP32)Gradients (w.r.tQuantized Weights)0.10.3⋯-0.10.1⋮⋱⋮-0.1⋯0.2dLdQ(FP32)QuantizerSTE
The marker travels one training step around the loop; each stage lights up as it is reached. Hover or tap the upper half to isolate the forward pass, the lower half for the backward pass.
Figure 8: QAT workflow showing one training step in clockwise order: (1) the top-left FP32 weights are quantized during the forward pass, (2) the quantized weights are used to evaluate the loss, (3) gradients are propagated from the bottom-right through the straight-through estimator (STE), and (4) the resulting FP32 gradients update the top-left weights.
loss ℒ(w)integer levelstrained weight wquantized weight wq−10120.20.40.60.81.0lossw(a) After PTQ−10120.20.40.60.81.0lossw−∂ℒ∂wq(b) After QAT
Sweep either panel: orange is the trained weight, blue is the quantized weight it rounds to, and the readout gives the loss that rounding costs.
Figure 9: Illustration of quantization effects on the loss landscape. The parameter \(w\) is quantized with a scale factor of 1. (a) PTQ: the model converges to a narrow minimum (orange), which is then quantized (blue), resulting in a significant increase in loss. (b) QAT: the model converges to a flatter region of the loss surface, where quantization introduces only a minor loss increase, demonstrating improved robustness to quantization.

4 Advanced Quantization Techniques

4.3 Mixed Precision Quantization

InputPreprocessed dataProcessed dataOutputNetwork⋮⋮⋯⋮Bit selector2-bit3-bitb-bit⋮2-bit3-bitb-bit⋮2-bit3-bitb-bit⋮⋯⋯⋯⋯Layer 1Layer 2Layer n↻ reset
Click a bit-width in any layer column: the selection path and the network’s layer colors update to match.
Figure 11: General structure of mixed precision-based frameworks. The model is optimized such that different layers are assigned different bit-widths, typically through sensitivity-, hardware-, or search-based allocation strategies.

4.3.2 Adaptive Allocation

MNIST sample “3” — a simple digit: the controller can run every layer at 2 bitsMNIST sample “7” — slightly harder: the middle layer is given 3 bitsMNIST sample “4” — the hardest of the four: the controller raises precision throughoutMNIST sample “0” — simple, but the last layer is kept at 3 bitsBit Controller (a network, or read directly from the input data)⋯Input2-bit3-bitb-bit⋮Layer 12-bit3-bitb-bit⋮Layer 22-bit3-bitb-bit⋮Layer n⋯Network
Pick one of the MNIST samples: the controller chooses a bit-width for every layer, and a simpler input gets a cheaper policy.
Figure 12: In adaptive allocation, the bits allocated to each layer are determined by a Bit-Controller, which can either be directly through the data itself, or through some degree of meta-training, where an additional network is trained on top of the existing network to “learn” the best bit-widths.

4.5 Redistribution

wminwmax
(a) Original weight distribution of a model layer. The values follow a zero-centered, Gaussian-like distribution, ranging between \(w_{\min}\) and \(w_{\max}\).
-128127
(b) Weight distribution after uniformization. The weights are transformed to follow a flat uniform distribution over the target quantization range, from \(-128\) to \(127\) in this case for 8-bit signed integers.
Hover a bar to see the share of weight values falling in that bin.
Figure 13: Illustration of weight uniformization for quantization. (a) Original weights typically follow a Gaussian distribution. (b) After uniformization, the weights are redistributed uniformly to reduce quantization error.

4.6 Data-Agnostic Quantization

Pre-TrainedModelCalibrationDataCalibrationQuantizationINT-8INT-4INT-b⋯Deployable PTQ Model
(a) Calibrated PTQ: quantization is guided by a small representative calibration dataset used to collect activation statistics.
Pre-TrainedModelCalibrationDataCalibrationQuantizationNetworkStatisticsError / BiasCorrectionINT-8INT-4INT-b⋯Deployable PTQ Model
(b) Data-Agnostic PTQ: quantization is performed solely on network weight and activation statistics, without any data guiding the scheme.
Figure 14: Illustration of Calibrated PTQ vs. Data-Agnostic PTQ.

6 Hardware and Software Deployment Stack

6.1 Hardware Platforms

Table 2: Representative MCU-class platforms for quantized neural network deployment.
MCU FamilyPlatformCPU(s)AcceleratorClockFlashRAMSupported Formats
ARM-basedArduino Nano 33 BLE SenseCortex-M4F–64 MHz1 MB256 KB RAMINT8, INT16, FP32
SparkFun EdgeCortex-M4F–48 MHz1 MB384 KB RAMINT8, INT16, FP32
Sony Spresense6× Cortex-M4F–156 MHz8 MB1.5 MB RAMINT8, INT16, FP32
OpenMV Cam H7Cortex-M7–480 MHz2 MB1 MB SRAMINT8, INT16, FP32
RISC-V-basedESP32-C3Single-core RISC-V–160 MHz4 MB400 KB SRAMINT8
ESP32-C6Single-core RISC-V–160 MHz8 MB512 KB SRAMINT8
ESP32-P4Dual-core RISC-V–400 MHz16 MB768 KB SRAM,
32 MB PSRAM		INT8, FP32
NPU-IntegratedMSPM0G5187Cortex-M0+TinyEngine NPU80 MHz128 KB32 KB SRAMINT2, INT4, INT8
MAX78002Cortex-M4F,
RISC-V controller		CNN accelerator120 MHz,
60 MHz		2.5 MB1.3 MB CNN SRAM,
384 KB SRAM		INT1, INT2, INT4, INT8
GAP88-core RISC-VHWCE250 MHz20 MB512 KB L2 SRAM,
8 MB L3 RAM		INT4, INT8, INT16
GAP99-core RISC-VNE16270–370 MHz2 MB1.6 MB L2 SRAMINT2–INT8
HX6538-WE2Cortex-M55Ethos-U55400 MHz16 MB2 MB SRAMINT8, INT16
STM32N6Cortex-M55Neural-ART800 MHzExternal4.2 MB SRAMINT8

7 Applications

TrainingDesignCompressionModel OptimizationQuantizedModelSoftwareOptimizationHardwareOptimizationSystem OptimizationDeployment
Hover any stage for a one-sentence description of what happens there.
Figure 16: Overview of quantized model deployment on MCU-class edge devices. The pipeline spans model training and optimization, quantization, hardware-aware conversion, and final deployment on the target MCU or accelerator-backed MCU.

7.1 ARM-based MCUs

Table 3: Selected works using quantization for deployment on ARM-based MCUs.
StudyCategoryQuantizationDevice(s)Framework(s)PerformancePower / EnergyLatency (ms)Memory (KB)
SpeechINT8 QATARM Cortex-M4FPyTorch96.30% Acc.11.2 mJ905419.8
Object DetectionINT8 QATSTM32H7A3,
STM32L4R9,
Apollo4b		TFLM43.50–75.40% mAP41.8 mJ,
102 mJ, 6.08 mJ		359,
996, 540		350–422
HARINT8 PTQSTM32F756ZGPyTorch91.04–98.29% Acc.4.50–8.77 mJ20.70–39.7637.8–44.7
NetworkingINT8 PTQSTM32F746G,
Nucleo-F401RE		TFLM,
STM32Cube.AI		96.59% Acc.7.86 mJ,
29.10 mJ		31.43,
115.40		353
EnvironmentINT8 PTQSTM32H747XI (M7),
STM32H747XI (M4),
nRF52840		ONNX Runtime,
NeuralCasting,
TFLM		98.5–99.1% Acc.600 mW,
600 mW,
100 mW		0.034–0.246,
0.034–0.252,
0.308–2.001		70–410
EducationINT8 QATSony Spresense,
OpenMV Cam H7,
H7 Plus		TFLM93.60–98.73% Acc.494–884 mW,
1238–1609 mW		0.63, 0.08,
2.88–12.84		2900, 340
Spectrum SensingINT8 QATSony SpresenseTFLM92.63–99.94% F1,
70.55–99.09% F1		52–54 mW,
44–52 mW		5.54–37.37,
1.18–5.20		23.8–72.6,
12.8–19.6
HARUINT8 PTQArduino Nano 33
BLE Sense Rev2		TFLM,
EdgeImpulse		97.09% Acc.21 mW21189.6
EnvironmentINT8 QATSTM32H7,
Arduino Nano 33
BLE Sense Rev2		TFLM,
STM32Cube.AI		99.63% Acc.227.86 mW,
1.43 mW, 9 mW		30.25,
522.15, 1.45		–
HealthcareINT8 QAT,
PQAT, PTQ		STM32H743iit6,
STM32H750vbt6		TFLM84.78–87.76% Acc.840 mW,
945 mW		6230,
6300		860

7.2 RISC-V-based MCUs

Table 4: Selected works using quantization for deployment on RISC-V MCUs. A dash indicates that the study did not report the corresponding value.
StudyCategoryQuantizationDevice(s)Framework(s)PerformancePower / EnergyLatency (ms)Memory (KB)
HealthcareINT8 PTQESP32-C6TFLM88.64 ± 1.56% F1,
90.05 ± 1.60% Sens.,
87.29 ± 1.54% Prec.		179.8 mW630310
IndustrialINT8 PTQESP32-C3TFLM95.00% macro-F16.77 mJ82±780
RoboticsINT8 PTQESP32-P4TFLM,
ESP-NN		40 laps collision-free,
99.84% Steer Acc.,
99.79% Throttle Acc.		–30∼23

7.3 NPU-Integrated MCUs

Table 5: Selected papers using quantization for deployment on NPU-integrated MCUs.
StudyCategoryQuantizationDevice(s)Framework(s)PerformancePower / EnergyLatency (ms)Memory (KB)
NetworkingINT8 PTQMAX78000PyTorch,
ai8x-tools		94.87% Acc.0.005 mJ0.104<432
Keyword SpottingINT8 QATNXP MCXN94TFLite97.06% Acc.–3.84730.6
Image ClassificationINT8 QATMAX78000,
MAX78002		PyTorch,
ai8x-tools		19.80–62.00% Acc.0.14–0.4 mJ2–13171.2–1330.7
Face RecognitionINT16 PTQGAP8PyTorch,
GAP8 Autotiler		93.00–96.33% Acc.–<10001500
Object DetectionINT8 QATMAX78000PyTorch,
ai8x-tools		43.50–75.40% mAP0.19 mJ5.5350–422
Object DetectionINT8 PTQGAP9PyTorch14.00–49.00% mAP54 mW56.45<1000
HARINT8 PTQGAP9PyTorch91.04–98.29% Acc.0.035–0.063 mJ1.11–1.9337.8–44.7
HealthcareINT8 QATMAX78002PyTorch,
ai8x-tools		94.60% AUC18 mW0.24825.156
HealthcareINT8 PTQGAP9PyTorch98.5% Acc.30.6 mW61750
HealthcareINT8 QATEthos-U55PyTorch94.25% Acc.0.0119 mW (est.)5<32
Anomaly DetectionINT8 QATMAX78000PyTorch87.19–99.95% Acc.15 mW2.55655.9
EnvironmentINT8 PTQMAX78000PyTorch,
ai8x-tools		79.67–86.53% F10.885–4.275 mJ4–27.5100–460
DronesINT8 PTQGAP8TFLite,
GAP8 Autotiler		98.80% Acc.,
57.20% Succ. Rate		130 mW35.71292
DronesINT8,
FP16 PTQ		GAP9nntool79.00% mAP,
80.00% mAP		34–41 mW147–4621800–3600
AgricultureINT8 PTQGAP8TFLM95.00% Acc.378 mW37.649.6

7.4 Cross-Case Recommendations

Memory Latency Energy Accuracy
1Application
SpeechKeyword spottingObject detectionImage classificationFace recognitionSegmentationSurveillanceVisual question answeringHuman activity recognitionHealthcareWearablesNetworkingEnvironmental sensingAgricultureIndustrial monitoringAnomaly detectionSpectrum sensingDSPRoboticsDronesEducation
8.18.2
2Quantization
PathPTQQATQT (quantized training)
Precision strategyUniform bit-widthMixed-precision allocationExtreme low-bit
RefinementsHardware-awareRedistributionData-agnostic
Design choicesUniform gridNon-uniform gridSymmetricAsymmetricPer-tensorPer-channelPer-groupStaticDynamicCalibration & clipping
3Numerical Representations
Integer & fixed-pointINT16INT8INT4INT2Binary & ternaryMixed precisionAdaptive fixed-point
Floating-pointFP16BF16FP8Block floating-point (MSFP)
Tapered-precisionPositTakum
8.38.4
4Software Stack
Develop & trainTensorFlowPyTorchEdge Impulseai8x-training
Convert & generate codeTensorFlow Lite / LiteRTONNXSTM32Cube.AIai8x-synthesisGAPflow · nntool · AutotilerEthos-U toolchain
Run on deviceTFLMCube.AI runtimeAccelerator runtimesAriel-MLONNX RuntimeMinimal C runtimesCMSIS-NNESP-NNPULP-NNXpulpNN (ISA extension)
8.58.6
5Hardware
ARM-basedSTM32 F4 · F7 · H7 · L4Arduino Nano 33 BLE SenseSony SpresenseOpenMV Cam H7SparkFun EdgeApollo4
RISC-V-basedESP32-C3ESP32-C6ESP32-P4PULP · PULPissimo (research)
NPU-integratedMAX78000MAX78002GAP8GAP9Ethos-U55 (HX6538-WE2)STM32N6 (Neural-ART)NXP MCXN94MSPM0G5187
8.7 Co-design 8.8 Sustainability
Start from your application, then read down. Clicking any entry narrows the question and keeps it; Clear starts over · hover a layer, a rail, or a numbered pin for its role.
Figure 17: A practitioner’s map of quantized inference on MCUs. Start from the application you need to deploy and read down: the figure lists the routes that carry it, best first, and draws the one it recommends, from the quantization you apply to the part it runs on. Selecting anything narrows the question rather than restarting it, so a method, a format, a tool or a platform keeps only the routes that support it, and any combination the tools and the silicon allow can be traced. A chip is either a step on the drawn route, a branch still open to you, or a setting the route fixes rather than a choice. Numbered pins mark where the challenges of Section 8 arise between layers, the left rail carries the constraints that act on every layer, and the right rail the challenges that cut across all of them.
For attribution in academic contexts, please cite this work as 
H. A. Abushahla, D. Varam, A. J. N. Panopio, and M. I. AlHajri, "Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications," arXiv preprint arXiv:2508.15008, 2026.
BibTeX citation 
@misc{abushahla2026neuralnetworkquantizationmicrocontrollers,
      title={Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications}, 
      author={Hamza A. Abushahla and Dara Varam and Ariel Justine N. Panopio and Mohamed I. AlHajri},
      year={2026},
      eprint={2508.15008},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2508.15008}, 
}