OCS 2023: Liu Liu (Draw Things)

Liu Liu is the founder of Draw Things.

Introduction to Liu, Draw Things, and presentation. Draw Things implementing open-source AI models across diverse range of Apple devices. — 0:00

What do you mean by Edge Deployment? - 1:16

Don’t we already do Edge Deployment? — 2:30

Open Model Weights — 4:15

“New” Generation of frameworks to handle larger models — 4:56

Why do we need the new generation of models? — 5:44

Difficulty of Edge Deployment with existing frameworks — 10:55

Dynamic Execution and Optimizations by Hand — 11:40

Integration Work on Metal Flash Attention — 13:32

Careful Memory Management, Quantization, and MMAP’d Weights — 15:06

Real-World Memory Usage — 17:34

Let’s Talk about Future (or Now?) - 17:58