onnxruntime
SHA-256ONNX Runtime is a cross-platform, high-performance ML inference and training accelerator supporting PyTorch, TensorFlow, scikit-learn, and more, with hardware acceleration and graph optimizations.
Smart Download
Download Download Version
v1.29.0 · 9.6 MB
A cross-platform accelerator that makes model inference and training faster and more cost-effective.
Core Features
- Supports models from PyTorch, TensorFlow, scikit-learn, LightGBM, XGBoost, etc.
- Cross-platform with broad hardware compatibility and accelerators
- Automatic graph optimizations and hardware acceleration boost inference performance
- One-line training acceleration for multi-node NVIDIA GPUs with transformer models
- Extensive documentation, sample repositories, and an active community
What It Can't Do
- •Custom or very new operators may require conversion to supported ONNX opsets; hardware accelerators like CUDA/TensorRT need separate runtime libraries; training acceleration is optimized for specific model types (e.g., transformers) on NVIDIA GPUs.
Use Cases
- Deploy deep learning models (vision, NLP) in production
- Convert models from PyTorch/TensorFlow to ONNX format for unified, accelerated inference
Detailed Introduction
ONNX Runtime is a cross-platform inference and training machine-learning accelerator developed by Microsoft. It enables faster customer experiences and lower costs by supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras, as well as classical machine learning libraries like scikit-learn, LightGBM, and XGBoost. Compatible with diverse hardware, drivers, and operating systems, ONNX Runtime delivers optimal performance through hardware accelerators and advanced graph optimizations. For training, it accelerates model training time on multi-node NVIDIA GPUs for transformer models with a one-line addition to existing PyTorch training scripts. Compared to running inference directly in PyTorch or TensorFlow, ONNX Runtime provides a unified, optimized runtime with broader hardware support and lower latency, making it a preferred choice for production deployment. This project is MIT-licensed and actively maintained with a large community.
Tags
Getting Started
Download installer
Click the button above to download the installer for your system
Install the software
Install the appropriate package for your distro (dpkg / rpm / AppImage)
Install ONNX Runtime (pip install onnxruntime or download pre-built packages from the website)
Export your model to ONNX format (e.g., torch.onnx.export for PyTorch)
Import onnxruntime, create an InferenceSession, and run inference
- Install ONNX Runtime (pip install onnxruntime or download pre-built packages from the website)
- Export your model to ONNX format (e.g., torch.onnx.export for PyTorch)
- Import onnxruntime, create an InferenceSession, and run inference
SHA-256 checksum verified
Checksum extracted from GitHub official Release page
SHA256 Checksum
4ca594a0da83927befbd73fe020d7f569be151d70bb4fe9741ad405f4882e2adThis checksum is extracted from the GitHub Release page. Verify file integrity after download.
All SHA-256 checksums on this platform are extracted from the project's official GitHub Release page, without any modification. You can independently verify them on the GitHub Releases page.
Open Source Transparency
View GitHub SourceUninstall Info
If installed via pip, run 'pip uninstall onnxruntime'. For custom builds, remove the installation directory and any PATH entries.
No Extra Dependencies
Ready to use after download. No additional runtime required.
Similar Projects
llama.cpp
High-performance LLM inference engine in C/C++ with minimal dependencies, supporting quantized models (1.5–8 bit) and diverse hardware (Apple Silicon, CUDA, Vulkan, etc.).
opencv
OpenCV is an open-source computer vision and machine learning library with over 2500 optimized algorithms for real-time image and video analysis.
ollama
Ollama lets you download, run, and manage large language models locally. One command, multiple platforms, endless possibilities.