yjftsjthsd-h23 days ago
So is this meant to be used by end users to run models and provide an API like ollama? I don't entirely follow the readme, though I think that's a context problem.
Tostino23 days ago
Pretty much, other than they just came up with their own incompatible API rather than implementing the OpenAI endpoints.