Microsoft Open Source Blog On-Device Training: Efficient training on the edge with ONNX Runtime
Introducing On-Device Training, a new capability in ONNX Runtime which enables training models on edge devices without the data ever leaving the device. Read more
ONNX Runtime
High-performance deep learning in Oracle Cloud with ONNX Runtime
In this blog post, we’ll share challenges our team faced, and how ONNX Runtime solves these as the backbone of success for high-performance inferencing. Read more
Performant on-device inferencing with ONNX Runtime
The team at Pieces shares the problems and solutions evaluated for their on-device model serving stack and how ONNX Runtime enables their success. Read more
Improve BERT inference speed by combining the power of Optimum, OpenVINO™, ONNX Runtime, and Azure
Make large models smaller and faster with OpenVino Execution Provider, NNCF and ONNX Runtime leveraging Azure Machine Learning. Read more
Hugging Face Transformers now enabled in Apache OpenNLP by ONNX Runtime
We’re excited to share the recent integration of ONNX Runtime in Apache OpenNLP! Apache OpenNLP is a Java machine learning library for natural language processing (NLP) tasks. Read more
Live demos of machine learning models with ONNX and Hugging Face Spaces
Choosing which machine learning model to use, sharing a model with a colleague, and quickly trying out a model are all reasons why you may find yourself wanting to quickly run inference on a model. You can configure your environment and download Jupyter notebooks, but it would be nicer if there was a way to Read more
Optimizing and deploying transformer INT8 inference with ONNX Runtime-TensorRT on NVIDIA GPUs
Mohit Ayani, Solutions Architect, NVIDIA Shang Zhang, Senior AI Developer Technology Engineer, NVIDIA Jay Rodge, Product Marketing Manager-AI, NVIDIA Transformer-based models have revolutionized the natural language processing (NLP) domain. Ever since its inception, transformer architecture has been integrated into models like Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT) for performing tasks Read more
Scaling-up PyTorch inference: Serving billions of daily NLP inferences with ONNX Runtime
Scale, performance, and efficient deployment of state-of-the-art Deep Learning models are ubiquitous challenges as applied machine learning grows across the industry. We’re happy to see that the ONNX Runtime Machine Learning model inferencing solution we’ve built and use in high-volume Microsoft products and services also resonates with our open source community, enabling new capabilities that Read more
Supporting efficient large model training on AMD Instinct™ GPUs with DeepSpeed
This post was co-authored by Jithun Nair and Aswin Mathews, members of technical staff at AMD. In recent years, large-scale deep learning models have demonstrated impressive capabilities, excelling at tasks across natural language processing, computer vision, and speech domains. Companies now use these models to power novel AI-driven user experiences across a whole spectrum of Read more
Add AI to mobile applications with Xamarin and ONNX Runtime
ONNX Runtime now supports building mobile applications in C# with Xamarin. Support for Android and iOS is included in the ONNX Runtime release 1.10 NuGet package. This enables C# developers to build AI applications for Android and iOS to execute ONNX models on mobile devices with ONNX Runtime. ONNX Runtime is the open source project Read more
ONNX Runtime Web—running your machine learning model in browser
We are introducing ONNX Runtime Web (ORT Web), a new feature in ONNX Runtime to enable JavaScript developers to run and deploy machine learning models in browsers. It also helps enable new classes of on-device computation. ORT Web will be replacing the soon to be deprecated onnx.js, with improvements such as a more consistent developer Read more
Accelerate PyTorch training with torch-ort
With a simple change to your PyTorch training script, you can now speed up training large language models with torch_ort.ORTModule, running on the target hardware of your choice. Training deep learning models requires ever-increasing compute and memory resources. Today we release torch_ort.ORTModule, to accelerate distributed training of PyTorch models, reducing the time and resources needed Read more