For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Vertex AI
Verified : Code examples on this page have been automatically tested and verified.Route agentgateway LLM traffic to models on Google Cloud Vertex AI.
Configure Google Cloud Vertex AI as an LLM provider in agentgateway.
Authentication
Before you can use Vertex AI as an LLM provider, you must authenticate by using Google Cloud’s Application Default Credentials. Choose from one of the three methods:
GOOGLE_APPLICATION_CREDENTIALSapplication_default_credentials.json- metadata server
Configuration
You can add the model in the UI or in your configuration file.
To add the model in the UI, follow these steps. The steps create the same model as the Config file tab.
Set up GCP Application Default Credentials (ADC) in the terminal where you run agentgateway.
Create a configuration file with an empty
llmsection, and start agentgateway in the same terminal. Without anllmsection, the UI does not show the LLM > Models page.cat > config.yaml <<'EOF' # yaml-language-server: $schema=https://agentgateway.dev/schema/config llm: models: [] EOF agentgateway -f config.yamlOpen LLM > Models, such as http://localhost:15000/ui/llm/models, and click Add model.
From the Provider dropdown, select Gemini Enterprise (GCP Vertex).
In the Incoming model match field, replace the default
vertex/*withgemini-2.5-flash.Keep ADC for the Google credentials. To use a service account key file instead, select File, and enter the path to the file.
Enter
my-project-idfor the Vertex project.For the Vertex region, enter or select
us-west2. If you leave the field empty, Vertex usesglobal.For the Outgoing model, select Explicit, and enter
google/gemini-2.5-flash-lite-preview-06-17.
Click Save model.
Whether the UI can save depends on your storage mode. For more information, see Save changes from the UI.
Send file content by URI
In an OpenAI-compatible Chat Completions request, you can put a gs://, http://, or https:// URI in a file content part’s file_data or file_id. The request to Vertex AI uses fileData.
For HTTP(S) signed URLs, MIME type detection uses the extension before the query string or fragment. If the URI does not have a recognized extension, set filename, mime_type, or content_type on the file object.
{
"model": "gemini-2.5-flash",
"messages": [
{
"role": "user",
"content": [
{ "type": "text", "text": "Compare these documents." },
{
"type": "file",
"file": {
"file_id": "https://storage.googleapis.com/my-bucket/report.pdf?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Signature=abc"
}
},
{
"type": "file",
"file": {
"filename": "summary.pdf",
"file_data": "https://example.com/download/123"
}
}
]
}
]
}