PrismML released Bonsai 2 27B on Thursday, a compressed large language model small enough to run on personal computers and high-end smartphones without cloud infrastructure. The model compresses Alibaba’s widely-used Qwen3.8 27B model down to 5.9 gigabytes while retaining 98% of the original’s benchmark performance. The development signals that reasoning-capable AI no longer requires massive data centers for deployment.
The startup employs “ternary” weight compression, converting standard 16-bit model parameters down to three values: positive one, negative one, or zero. Reducing the information density at each weight position creates dramatically smaller model files. The approach maintains reasoning capability because larger models contain redundant parameters that compression can eliminate without meaningful performance loss.
This represents a significant improvement from Bonsai, released in March, which retained 95% of benchmark performance. The original Bonsai model has been downloaded over 11 million times according to PrismML, indicating substantial developer adoption. Additional smaller models have accumulated 2.6 million downloads, demonstrating market demand for device-based AI.
| Capability | Ternary Bonsai 2 27B |
Qwen3.8 27B |
Qwen3.6 27B |
|---|---|---|---|
| Agentic & Tool Callingτ²-bench, BFCLv3 | 77.57 | 79.74 | 80.05 |
| CodingHumanEval+, LiveCodeBench v6, MBPP+, BigCodeBench | 81.58 | 82.17 | 82.57 |
| Instruction FollowingIFBench, IFEval | 82.66 | 81.25 | 74.53 |
| Knowledge & ReasoningMMLU-Redux, GPQA Diamond, AA-LCR | 83.95 | 86.66 | 84.71 |
| MathAIME 2026, AIME 2025, GSM8K, MATH-500 | 96.57 | 97.06 | 94.64 |
| VisionCharXiv, A-OKVQA, OmniDocBench v1.6, RealWorldQA, OCRBench v2 | 78.59 | 81.64 | 79.82 |
| Overall | 83.9 | 85.4 | 83.6 |
PrismML was founded by Caltech researchers and is led by Babak Hassibi, a Caltech professor specializing in compression technologies. The startup raised a $22.25 million seed round backed by Khosla Ventures, Cerberus Capital, and Caltech. Ion Stoica, Databricks co-founder and director of UC Berkeley’s Sky Computing Lab, serves as an adviser.
CEO Hassibi declined to comment on reports that PrismML is in talks with Apple, though the timing suggests potential device-side AI integration. Running reasoning models locally addresses privacy concerns associated with sending user queries to cloud servers. Processing remains private because data never leaves the device, and operation costs nothing beyond the hardware already purchased.
The company’s next goal targets compressing hundreds-of-billions-parameter models within the next couple of months. Hassibi indicated that larger models offer more redundancy, making compression easier while maintaining intelligence. As model size increases, he suggested reaching 100% benchmark performance parity becomes achievable.
Ion Stoica emphasized that device-based AI transforms user experience by providing “intelligence at your fingertips” without cloud dependency. Users gain privacy from local processing, eliminate cloud transmission latency, and avoid subscription costs. The capability represents fundamental shift from cloud-dependent AI toward distributed device-based reasoning.
Bonsai 2 27B runs on NVIDIA GPUs via CUDA and on Apple devices (Mac, iPhone, iPad) via MLX, through custom low-bit kernels. Model weights are available today under the Apache 2.0 License.
