I suggest creating a service that monitors all the LLMs/AI model deployment services on Azure, AWS, Google Cloud, Groq, Krutrim, and other cloud deployments and routes your traffic based on availability, latency, rate-limiting, and other parameters. Manage everything at the backend, just pay a single unified bill, with no deployments or accounts on any other service. Sounds cool?
yorwba23 days ago
Sounds similar to SkyPilot: https://docs.skypilot.co
But you still need to create your own accounts for each service.