Steep 4B
Chat
Answers questions, writes and explains. The model you talk to. It can call the other installed families for you.
| Version | Memory, to scale | Download | Quality | Speed | Picked on |
|---|---|---|---|---|---|
| Smallest q3 | 3.3 GB | 2.5 GB | 86.9% | 62.5 tok/s | under the 96% bar |
| Small q4 | 3.7 GB | 3 GB | 93.2% | 66.3 tok/s | under the 96% bar |
| Balanced q5 | 4.1 GB | 3.3 GB | 96.4% | 54.6 tok/s | Laptop, 8 GBMac, 16 GBGaming PCMac, 48 GB |
| Better q6 | 4.4 GB | 3.7 GB | 97.3% | 50.4 tok/s | |
| Best quality q8 | 5.4 GB | 4.6 GB | 99.1% | 52.4 tok/s |
Same next token as the full-precision original (wikitext-2 test set, 32 × 512 tokens). Speeds from a MacBook Pro M5 Max that was busy with other work; compare them with each other.