_ache_2 hours ago
From my own test. It's not faster than the unsloth model.
Disclarer: I'm unsing Vulkan on an AMD GC.
sheo2 hours ago
Aged like milk
Mashimoan hour ago
In the comments it reads like bonsei falls apart on longer running tasks.
txrx0000an hour ago
Not really. The largest IQ4_XS quant here is still worth it because Bonsai doesn't offer larger quants. They could beat it if they made a quaternary variant though, I don't know why they're stopping at ternary.