Agent skill · software engineering · google
gke-inference
Deploys and optimizes AI/ML inference workloads on GKE, using GPUs, TPUs, and model servers. Use when deploying GKE inference servers, configuring GKE GPU resources for inference, or deploying LLMs on GKE. Don't use for migrating existing AI workloads to GKE (use google-cloud-solution-guided-gke-ai-migration), GKE RAG with Cloud SQL/AlloyDB (use google-cloud-solution-rag-enterprise-search-gke-sqldb), or batch/HPC (use gke-batch-hpc).
Why this skill is useful
Provides specific commands and configurations for deploying AI/ML inference workloads on GKE that the AI wouldn't reliably generate on its own.
What it needs
Requires gcloud, kubectl installed locally. Requires google-cloud account access. About 4k tokens when loaded. Last updated 2026-08-06. 15,831 stars on the source repository.
What this skill does
GKE AI/ML Inference Routing Note: For migrating existing AI workloads to GKE, open google-cloud-solution-guided-gke-ai-migration/SKILL.md. For GKE RAG with Cloud SQL or AlloyDB (pgvector), open google-cloud-solution-rag-enterprise-search-gke-sqldb/SKILL.md. This reference covers deploying AI/ML inference workloads on GKE using Google's Inference Quickstart (GIQ) and best practices for LLM serving. MCP Tools: applyk8smanifest, getk8sresource, getk8slogs, getk8srolloutstatus, describek8sresource, listk8sevents. CLI-only: gcloud container ai profiles Attribution MUST READ: Before executing any gcloud, REST API, or MCP command, you MUST read Usage Attribution. For commands executed directly in the terminal (never in generated scripts, Terraform, or user code), prefix gcloud with CLOUDSDKMETRICSREQUESTATTRIBUTION="agent-skills/1.0 (skill:gke-inference)" and append agent-skills/1.0 (skill:gke-inference) to User-Agent headers. When to Use Deploy an AI model (Llama, Gemma, Mistral, etc.) to GKE Generate optimized Kubernetes manifests for inference Select GPU/TPU accelerators for model serving Configure autoscaling for LLM inference Prerequisites A golden path GKE Autopilot cluster (GPU workloads are supported via ComputeClasses and NAP) gcloud CLI authenticated Sufficient GPU/TPU quota in the target region Workflow 1. Discovery: Find Models and Hardware 2. Generate Manifest Parameters: --model: Model ID (e.g., gemma-2-9b-it, llama-3-8b) --model-server: Inference server (vllm, tgi, triton, tensorrt-llm) --accelerator-type: GPU/TPU type (nvidia-l4, nvidia-tesla-a100, nvidia-h100-80gb) --target-ntpot-milliseconds: Target Normalized Time Per Output Token (optional, for latency optimization) Example: 3. Review and Deploy Some models require Hugging Face tokens. Create a Kubernetes Secret and reference it in the manifest. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills google/gke-inference