Snowflake makes AI Gateway tracing and cost controls generally available

Snowflake made Cortex AI Gateway inference, observability and cost management generally available on November 3. The service brings requests from external AI agents and coding tools into a common path where companies can account for model use. Those three capabilities have moved beyond the gateway preview announced on September 15.

For application teams, the gateway supplies a shared endpoint for model requests. Platform teams manage access and inspect activity through the same account infrastructure. Each Snowflake account receives one gateway automatically. Administrators can revoke access and assign separate privileges for viewing every user's records; ordinary usage access gives users visibility into their own activity. Gateway use requires the underlying model permissions, and the public role receives usage access by default. Inference is billed in AI Credits by processed tokens at each model’s rate.

The inference documentation supports OpenAI Chat Completions and Responses formats alongside Anthropic Messages. Clients that accept a custom endpoint and bearer credential can connect. Supported features include streaming, tool calling and structured outputs, allowing applications that already use those request formats to retain familiar model interactions while routing traffic through the company's gateway.

For each request, traces capture the model, timing, token consumption and failures. Tokens are the units used to process model inputs and outputs. Records enter customer-owned event tables under Snowflake's role-based access controls. The default captures metadata. Recording prompt and response payloads requires explicitly enabling content capture.

An agent turn can contain several model calls. Shared trace identifiers let operators assemble their individual records into an execution. Clients configured to export OpenTelemetry data and propagate trace context can add their own intervening steps, bringing the client's view together with requests observed by the gateway.

Usage accounting is collected separately from traces. Request-level history identifies the user, model and token counts for attributing spending to applications. For shared team budgets, user tags define a department or cost center. The same AI feature can therefore contribute spending to a budget for selected users across an organization. Scope can match any of the selected tag-value pairs or require all of them, for example a department and a project together. Spending by other users of the same resource falls outside that budget.

Team budgets and individual blocking work differently

Shared budgets pool tagged users' spending and trigger notifications or administrator-written procedures. Per-user quotas evaluate people individually and can block usage. Budget evaluation can take up to 6.5 hours, or one hour with the low-latency option, according to Snowflake's cost guide. Quota blocking applies within minutes of usage being counted.

An individual spending limit and enabled enforcement are required for blocking. At the gateway, a block rejects new requests while work already running finishes and is billed. Accounting and enforcement delays can also allow overspending. The control consequently has a delay between consumption and denial, rather than checking a remaining balance synchronously at every request.

Daily, weekly and monthly individual quotas follow the UTC calendar. Daily limits reset at midnight, weekly limits on Monday at 00:00 and monthly limits on the first day of the month. Each user is evaluated independently.

The November 3 general-availability release covers inference, tracing and cost management. The visual trace-analytics interface and external tool integration remain in preview. Business Ontology, the business-definition framework announced alongside the gateway updates, is also awaiting public preview. Snowflake has not supplied a specific date for that availability.