Jason Knight is co-founder and CPO at OctoML working to enable automated, cross platform machine learning deployment systems leveraging Apache TVM. He continues to leverage his experience from running Software Product at Intel’s AI Product Group and a PhD in machine learning for computational biology where he applied machine learning and data engineering to large scale biological data.
LinkedIn - Twitter
ML powered specialization is the key to making machine learning more portable, more efficient, and easier.
Introduction to Jason and presentation topic: OctoML and Apache TVM for MAchine Learning Deployment - 0:00
The Problem: Machine Learning is hungry (in terms of engineering costs, cloud costs, and silicon costs) - 0:20
Walking through the deployment challenge and visualizing - 1:27
An exploding ecosystem makes deployment painful. Rapidly evolving MML software ecosystem, and Cambrian explosion of HW backends that you can run models on. - 2:32
Apache TVM: Bridging the gap as a DL compiler and runtime. Enables you to take a model and compile it down into backends, optimized for performance across different backends, and achieve high-performance in cross-platform way. Allows you to reduce model time-to-market, build model once and run anywhere, and cut capital/operational ML costs. - 2:57
AutoTVM Overview - using machine learning to automatically adapt to hardware types - 4:36
Performance benchmarks at OctoML in 2020. 2.1x average performance improvement over baselines (with 2.5x average performance improvement on non-public models, due to TVM’s agnostic approach) - 6:29
TVM can also accelerate classical ML models - 9:30
TVM for TinyML (see TinyML Summit talk and ARM Webinar. Supporting devices that don’t have access to operating system stack. - 10:08
TVM is an emerging industry standard ML stack - 10:55
OctoML: Powered by TVM. Offering benefits of TVM, building on it, and offering it to a broader audience. - 11:45
Octomizer: automated platform for ML ops - 13:15
Octomizer Demo - 13:51
Easy-to-use Octomize API - 19:41
Octomizer Early Access is open (apply at https://octoml.ai) - 19:54
OctoML Offerings: Octomizer SaaS Licensing, Sponsored OSS Development, Expert support and training - 20:30
Selected Customer Engagements and demonstrated results - 21:11
Concluding remarks, preview of 13x Apple TensforFlow performance, and contacts - 22:20
Share your questions and comments below!
Top comments (0)