ANE

Overview

A research project that demonstrates training neural networks on the Apple Neural Engine using reverse-engineered private APIs. Includes benchmarks, transformer training, and analysis of ANE capabilities. Not a production framework and has limitations like low utilization.

README Preview

# ANE Training — Backpropagation on Apple Neural Engine\r\n\r\nTraining neural networks directly on Apple's Neural Engine (ANE) via reverse-engineered private APIs. No CoreML training APIs, no Metal, no GPU — pure ANE compute.\r\n\r\n## Project Scope & Intent\r\n\r\nI'm genuinely grateful for all the attention this project has received — I never expected a weekend research hack to blow up like this. Thank you to everyone who starred, forked, ran benchmarks on their own hardware, and shared the work. It means a lot.\r\n\r\nThat said, I want to set clear expectations about what this project is and isn't.\r\n\r\nThis is a **research project**, not a production framework.\r\n\r\nThe goal was to demonstrate that **training on the Apple Neural Engine — and potentially other NPUs — is possible**, and that the barrier has always been software support, not hardware capability. The ANE is a remarkably capable piece of silicon that Apple restricts to inference-only use through CoreML. This project bypasses that restriction using reverse-engineered private APIs to show what's possible when you give the hardware a chance.\r\n\r\n### What This Project Is\r\n\r\n- A proof of concept for ANE training via `_ANEClient` and `_ANECompiler` private APIs\r\n- A set of benchmarks documenting real ANE performance characteristics (throughput, power, SRAM behavior)\r\n- A reference for anyone exploring direct ANE access outside CoreML\r\n- Research code that I update when I find something interesting\r\n\r\n### What This Project Is Not\r\n\r\n- A maintained framework or library\r\n- A replacement for CoreML, MLX, llama.cpp, or any production inference stack\r\n- A path to training large models on consumer hardware (yet)\r\n\r\n### On The Hype\r\n\r\nSome coverage of this project has overstated its implications. To be clear:\r\n\r\n- Training works, but utilization is low (~5-9% of peak) with significant engineering challenges remaining\r\n- Many element-wise operations still fall back to CPU