Build Daily
Tinley Park · August 23, 2026Keep model costs in check with LangSmith LLM Gateway
This video introduces the LangSmith LLM Gateway as a centralized endpoint to manage multi-provider model access, enforce spend limits, and provide audit trails for organizational AI costs.
The outline
Tap a timestamp to jump there in the video.
The problem of uncontrolled token spend
Teams using various API keys and models across the organization lead to unexpected spikes in token spending that are difficult to trace.
Gateway architecture and control
The LangSmith LLM Gateway acts as a single endpoint between the organization and model providers, allowing centralized enforcement of spend and rate limits before requests are sent.
Getting started with API keys
Users can create gateway-specific API keys and add upstream provider credentials to authenticate requests through the gateway.
Integration via base URL change
Existing applications can route through the gateway by simply changing the base URL and API key without modifying request or response handling logic.
Live request demonstration
A live call is made from the browser to an LLM via the gateway, demonstrating successful routing and trace generation in LangSmith.
Centralized spend limit configuration
Spend limits can be set centrally at the organization level or granularly by API key, user, or workspace to prevent individual teams from exceeding budgets.
Testing and enforcing limits
An artificially low spend limit is applied to demonstrate how requests are blocked with clear errors once the cap is reached, preventing unauthorized charges.
Usage tracking and audit trails
The usage page provides a live audit trail of which models, keys, and applications consumed tokens and at what cost, simplifying financial reporting.
A single endpoint that tracks spend across providers helps a solo builder avoid the scramble of unexplained token bills. For someone shipping multiple features, knowing which agent or app burns through credits is as important as the model choice itself.