Valyr

Valyr, also known as Helicone, is a powerful open-source platform designed for developers building applications with Large Language Models (LLMs). It serves as both an AI Gateway and an LLM Observability tool, addressing the critical challenges of monitoring, debugging, and scaling AI-powered products. In a complex landscape of multiple LLM providers and APIs, Valyr provides a unified layer of control and insight, making it an essential component of the modern AI stack. Trusted by fast-growing AI companies, it empowers teams to build reliable, cost-effective, and high-performance applications with confidence.

The platform is built on the principle of simplicity and power. By acting as a proxy between your application and various LLM providers (like OpenAI, Anthropic, Google, etc.), Valyr captures every request and response. This allows for in-depth analysis without requiring complex changes to your existing codebase. Whether you're a solo developer debugging a new feature or a large enterprise managing costs across multiple teams, Valyr provides the tools you need to succeed.

How to use Valyr

Integrating Valyr into your project is designed to be straightforward, typically requiring only a single line of code change. The process is as follows:

1. Sign Up: Create an account on the Valyr (Helicone) platform to get your API key.
2. Change the API Base URL: In your application's code, instead of calling the LLM provider's API directly (e.g., `api.openai.com`), you point it to the Valyr gateway URL.
3. Configure Your Client: Add your Valyr API key to the request headers. This authenticates your application and links the data to your account.
4. Monitor and Analyze: Once integrated, all your LLM traffic will flow through Valyr. You can then log into the Valyr dashboard to view real-time logs, analyze performance metrics, debug errors, and track costs.
5. Enable Advanced Features: Features like caching, rate limiting, and user tracking can be easily configured by adding specific properties to your API request headers, allowing for granular control without code deployments.

Core Features of Valyr

• Unified AI Gateway: Access over 100 different LLM models from various providers like OpenAI, Anthropic, Gemini, and more through a single, consistent API endpoint. This simplifies model switching and experimentation.
• Advanced LLM Observability: Get a complete picture of your application's interactions with LLMs. This includes real-time logging of all requests, responses, latencies, token counts, and status codes.
• Cost Management & Analytics: Track your spending with incredible detail. Break down costs by model, API key, user, or any custom property you define. Set up alerts to prevent budget overruns.
• Powerful Debugging Tools: Quickly identify and resolve issues. Inspect the full payload of any request, trace conversations, and filter logs to pinpoint the source of errors or unexpected model behavior.
• Performance Optimization: Reduce latency and save costs with intelligent caching. Valyr supports both exact-match and semantic caching. It also allows you to set custom rate limits to protect your application from abuse.
• User-Centric Monitoring: Associate API requests with your end-users to understand individual usage patterns, calculate per-user costs, and debug user-specific issues.
• Open Source and Self-Hostable: As an open-source platform, Valyr offers maximum transparency and flexibility. Enterprises can choose to self-host the entire platform for complete data control and security.

Use Cases for Valyr

Valyr is versatile and valuable for anyone working with LLMs:

• AI Application Developers: For day-to-day monitoring of application health, debugging new features, and optimizing API calls for speed and cost.
• MLOps and DevOps Teams: For establishing a robust, production-grade monitoring and logging pipeline for all AI services, ensuring reliability and uptime.
• Product Managers: For gaining insights into how users are interacting with AI features, tracking the costs associated with those features, and making data-driven decisions.
• Startups and Enterprises: For rapidly prototyping, scaling, and managing AI products while maintaining strict control over operational costs and system performance.

Advantages of Valyr

The key advantages of using Valyr include:

• Simplicity and Ease of Use: Integration is non-intrusive and takes minutes, providing immediate value.
• Vendor-Agnostic: Avoid vendor lock-in and seamlessly switch between LLM providers to find the best model for your needs.
• Significant Cost Savings: Features like smart caching and detailed cost analytics directly contribute to reducing your LLM spend.
• Enhanced Reliability: Automatic retries with exponential backoff and granular rate limiting make your application more resilient.
• Deep, Actionable Insights: Move beyond basic logs to understand the 'why' behind your application's performance and user behavior.
• Flexibility and Control: The open-source nature allows for customization and self-hosting, giving you full ownership of your observability stack.

Pricing and Plans

Valyr (Helicone) operates on a freemium model, making it accessible for projects of all sizes.

• Hobby Plan: A generous free tier that includes up to 100,000 requests per month, perfect for personal projects and early-stage startups.
• Pro Plan: A paid plan designed for growing applications, offering a higher request limit (e.g., 1 million requests/month), longer data retention, and advanced features like user-level metrics.
• Growth Plan: For scaling businesses with even higher traffic, providing millions of requests per month and more robust analytics.
• Enterprise Plan: A custom plan for large-scale deployments, offering features like self-hosting, single sign-on (SSO), dedicated support, and unlimited requests.

A 7-day free trial of the paid plans is typically available, with no credit card required to get started.