Documentation, FAQ, Release Notes, Roadmap, MACE Model Zoo, Demo, Join Us, 中文
Mobile AI Compute Engine (or MACE for short) is a deep learning inference framework optimized for
mobile heterogeneous computing on Android, iOS, Linux and Windows devices. The design focuses on the following
targets:
- Performance
- Runtime is optimized with NEON, OpenCL and Hexagon, and
Winograd algorithm is introduced to
speed up convolution operations. The initialization is also optimized to be faster.
- Runtime is optimized with NEON, OpenCL and Hexagon, and
- Power consumption
- Chip dependent power options like big.LITTLE scheduling, Adreno GPU hints are
included as advanced APIs.
- Chip dependent power options like big.LITTLE scheduling, Adreno GPU hints are
- Responsiveness
- UI responsiveness guarantee is sometimes obligatory when running a model.
Mechanism like automatically breaking OpenCL kernel into small units is
introduced to allow better preemption for the UI rendering task.
- UI responsiveness guarantee is sometimes obligatory when running a model.
- Memory usage and library footprint
- Graph level memory allocation optimization and buffer reuse are supported.
The core library tries to keep minimum external dependencies to keep the
library footprint small.
- Graph level memory allocation optimization and buffer reuse are supported.
- Model protection
- Model protection has been the highest priority since the beginning of
the design. Various techniques are introduced like converting models to C++
code and literal obfuscations.
- Model protection has been the highest priority since the beginning of
- Platform coverage
- Good coverage of recent Qualcomm, MediaTek, Pinecone and other ARM based
chips. CPU runtime supports Android, iOS and Linux.
- Good coverage of recent Qualcomm, MediaTek, Pinecone and other ARM based
- Rich model formats support
- TensorFlow,
Caffe and
ONNX model formats are supported.
- TensorFlow,
Getting Started
Performance
MACE Model Zoo contains
several common neural networks and models which will be built daily against a list of mobile
phones. The benchmark results can be found in the CI result page
(choose the latest passed pipeline, click release step and you will see the benchmark results).
To get the comparison results with other frameworks, you can take a look at
MobileAIBench project.
Communication
- GitHub issues: bug reports, usage issues, feature requests
- Slack: mace-users.slack.com
- QQ群: 756046893
Contributing
Any kind of contribution is welcome. For bug reports, feature requests,
please just open an issue without any hesitation. For code contributions, it's
strongly suggested to open an issue for discussion first. For more details,
please refer to the contribution guide.
License
Acknowledgement
MACE depends on several open source projects located in the
third_party directory. Particularly, we learned a lot from
the following projects during the development:
- Qualcomm Hexagon NN Offload Framework: the Hexagon DSP runtime
depends on this library. - TensorFlow,
Caffe,
SNPE,
ARM ComputeLibrary,
ncnn,
ONNX and many others: we learned many best
practices from these projects.
Finally, we also thank the Qualcomm, Pinecone and MediaTek engineering teams for
their help.
