Skip to content
MACT.ai
All work

Edge AI Module

A drop-in compute module with known latency, power and cost for on-device inference.

2026Edge AI · Hardware · Embedded Linux
Edge AI Module

Project Overview

A system-on-module and carrier board designed so product teams can add on-device inference without rebuilding their compute platform. The module ships with characterised numbers: latency, sustained throughput, peak memory and thermal behaviour under load.

Challenge

Every customer arrived with a model already chosen and a power budget already promised. The two rarely matched, and the mismatch was usually discovered during integration rather than during architecture.

Solution

We inverted the order. The workload gets benchmarked on candidate silicon first, and only then is the module specified. Quantization runs against a fixed evaluation set with accuracy gates at each conversion step, so a regression is visible the moment it appears.

Architecture

01

Hardware

A SoM with an NPU sized to the measured workload, on a carrier board that exposes camera, audio, Ethernet and PCIe.

  • NPU options from 2 to 20 TOPS on one footprint
  • Brushed aluminium heat spreader, characterised to 15 W
  • Castellated module edge for volume integration
  • Carrier exposes MIPI, I²S, GbE and PCIe
02

AI

A conversion and deployment pipeline with an operator coverage map, so unsupported layers surface during benchmarking instead of integration.

  • INT8 and INT4 quantization with per-channel calibration
  • Operator coverage map produced before silicon commitment
  • Graph fusion and memory-ceiling profiling at peak load
  • Continuous on-target regression benchmarking
03

Software

A Linux BSP with the vendor NPU runtime, an inference service behind a stable local API, and a model update channel independent of firmware.

  • Linux BSP with vendor NPU runtime integrated
  • Local inference API and integrator SDK
  • Model channel separate from firmware releases
  • Health, thermal and performance telemetry

Result

  • Published latency, power and memory figures per configuration
  • Model updates shipped without a firmware release
  • Second silicon candidate kept benchmarked against supply risk
  • Integrators reached working inference in days, not months

Technology Stack

Hardware

SoM + CarrierLPDDR5PCIe

AI

ONNXTensorRTRKNNINT4 / INT8

Embedded

Linux BSPNPU RuntimeModel Channel

Cloud

Model RegistryFleet Telemetry