Hands-free LLM optimization
New language models come out every week. Your product isn't keeping up.
Auto integrates with your LLM calls in under 5 lines of code to continuously evaluate models for each of your use cases.
You get a customized leaderboard of models by cost & quality - that includes finetuned models.
Schedule a callFeatures
- Just works: Integration takes minutes - and then Auto sends you new LLM optimization opportunities with no additional work from your team.
- Routing: Use any of 200+ LLMs from 30+ providers with a single string swap, with customizable fallback, retry and load-balancing policies.
- Automatic evals: Powerful reasoning models evaluate outputs out-of-the-box, with optional custom rubrics.
- Internal model leaderboards: Compare leading models by cost, latency and quality across your tasks
- New model reports: Stay on the bleeding edge with reports of how new models perform on your tasks within as little as 24h of model release.
- Automatic fine-tuning: Auto fine-tunes models in the background - including fine-tuning for OpenAI, Gemini & open source models.
- Rich feature support: Auto supports streaming, prompt-caching, multimodal inputs, and more.
Roadmap
- Automatic prompting: Your prompts are automatically optimized via libraries such as DSPy, Sylph.ai as well as other custom techniques.
- Bring your own evals: Auto runs new hypotheses, you bring the evals to test outputs against. Define your own evals on our platform, your custom platform, or providers like Braintrust and Langfuse