Microsoft Open Source Blog High-performance deep learning in Oracle Cloud with ONNX Runtime
In this blog post, we’ll share challenges our team faced, and how ONNX Runtime solves these as the backbone of success for high-performance inferencing. Read more
Posts by Faith Xu, Principal Program Manager, Machine Learning Platform
Performant on-device inferencing with ONNX Runtime
The team at Pieces shares the problems and solutions evaluated for their on-device model serving stack and how ONNX Runtime enables their success. Read more
Hugging Face Transformers now enabled in Apache OpenNLP by ONNX Runtime
We’re excited to share the recent integration of ONNX Runtime in Apache OpenNLP! Apache OpenNLP is a Java machine learning library for natural language processing (NLP) tasks. Read more
Scaling-up PyTorch inference: Serving billions of daily NLP inferences with ONNX Runtime
Scale, performance, and efficient deployment of state-of-the-art Deep Learning models are ubiquitous challenges as applied machine learning grows across the industry. We’re happy to see that the ONNX Runtime Machine Learning model inferencing solution we’ve built and use in high-volume Microsoft products and services also resonates with our open source community, enabling new capabilities that Read more
ONNX Runtime 1.8: mobile, web, and accelerated training
The V1.8 release of ONNX Runtime includes many exciting new features. This release launches ONNX Runtime machine learning model inferencing acceleration for Android and iOS mobile ecosystems (previously in preview) and introduces ONNX Runtime Web. Additionally, the release also debuts official packages for accelerating model training workloads in PyTorch. ONNX Runtime is a cross-platform runtime Read more
ONNX Runtime scenario highlight: Vespa.ai integration
Since its open source debut two years ago, ONNX Runtime has seen strong growth with performance improvements, expanded platform and device compatibility, hardware accelerator support, an extension to training acceleration, and more. We are excited by its broad usage in production, powering more than a hundred models across Microsoft products and services and bringing concrete Read more
Accelerate traditional machine learning models on GPU with ONNX Runtime
With the growing trend towards deep learning techniques in AI, there are many investments in accelerating neural network models using GPUs and other specialized hardware. However, many models used in production are still based on traditional machine learning libraries or sometimes a combination of traditional machine learning (ML) and DNNs. We’ve previously shared the performance Read more
Announcing ONNX Runtime 1.0
One year after ONNX Runtime’s initial preview release, we’re excited to announce v1.0 of the high-performance machine learning model inferencing engine. This release marks our commitment to API stability for the cross-platform, multi-language APIs, and introduces a breadth of performance optimizations, broad operator coverage, and pluggable accelerators to take advantage of new and exciting hardware Read more
Now available: ONNX Runtime 0.5 with support for edge hardware acceleration
ONNX Runtime 0.5, the latest update to the open source high performance inference engine for ONNX models, is now available. This release improves the customer experience and supports inferencing optimizations across hardware platforms. Since the last release in May, Microsoft teams have deployed an additional 45+ models that leverage ONNX Runtime for inferencing. These models Read more