Ember – UC Berkeley Sky Computing Lab

An Inference-Time Scaling Architecture Framework

Ember is a compositional framework for building and deploying large inference-time scaling architectures and strategies. We call these architectures Networks of Networks, NONs for shorthand. These architectures can be employed to eclipse the quality or reliability frontier available via today’s frontier LLMs or to achieve comparable quality at 1/1000th or less the cost.

Blog Post

GitHub

Twitter

Contributors

Jared Quincy Davis^F,S, Marquita Ellis^I, Diana Arroyo^I, Pravein Govindan Kannan^I, Paul Castro^I, Siddharth Sharma^F,S, Parth Asawa^B, Alan Zhu^B, Connor Chow^B, Jason Lee^B, Jay Adityanag Tipirneni^B, Chad Ferguson^B, Kathleen Ge^B, Kunal Agrawal^B, Rishab Bhatia^B, Rohan Penmatcha^B, Sai Kolasani^B,Théo Jaffrelot Inizan^B, Lingjiao Chen^MS, Omar Khattab^D,MT, Deepak Narayanan^N, Long Fei^F, Aparajit Raghavan^F, Eyal Cidon^F, Jacob Schein^F, Prasanth Somasundar^F, Boris Hanin^F,P, James Zou^S, Alex Dimakis^B, Joey Gonzalez^B, Peter Bailis^G,S, Ion Stoica^A,B,D, Matei Zaharia^D,B

Foundry (MLFoundry)^F, Databricks^D, IBM Research^I, Stanford University^S, UC Berkeley^B, MIT^MT, NVIDIA^N, Microsoft^MS, Anyscale^A, Google^G, Princeton^P

An Inference-Time Scaling Architecture Framework

Contributors

Publications