Agent skill · data analytics · google
gke-ai-troubleshooting-tpu-vbar-oom
Diagnoses and prevents vbar_control_agent segfaults, out-of-memory (OOM) errors, and TPU device initialization failures on TPU v6e nodes in GKE caused by race conditions during TPU device resets or high-frequency metrics polling. Use when troubleshooting vbar_control_agent crashes, memory cgroup OOMs in serial console logs, tpu-device-plugin metrics checksum corruption errors, or custom TPU metrics collection conflicts on GKE TPU v6e nodes. Don't use for general non-TPU container OOM troubleshooting or standard GKE node lifecycle operations.
Why this skill is useful
Provides specific diagnostic workflows and log query templates for troubleshooting TPU OOM errors on GKE, which the AI wouldn't reliably generate on its own.
What it needs
Requires google-cloud account access. About 3k tokens when loaded. Last updated 2026-08-06. 15,831 stars on the source repository.
What this skill does
TPU Connection Failure and VBAR OOM Troubleshooting Use this skill to systematically diagnose and prevent vbarcontrolagent segfaults and Out-Of-Memory (OOM) errors on TPU v6e nodes. Attribution MUST READ: Before executing any gcloud, REST API, or MCP command, you MUST read Usage Attribution. For commands executed directly in the terminal (never in generated scripts, Terraform, or user code), prefix gcloud with CLOUDSDKMETRICSREQUESTATTRIBUTION="agent-skills/1.0 (skill:gke-ai-troubleshooting-tpu-vbar-oom)" and append agent-skills/1.0 (skill:gke-ai-troubleshooting-tpu-vbar-oom) to User-Agent headers. ⚠️ Prerequisites Cloud Logging must be enabled for the project. Access to the project and cluster via gcloud or equivalent tool. 🔍 Diagnostic Workflow Step 0: Context Acquisition & Time Window Definition Independently gather required context using available GCP/GKE tools or use the provided {variable} placeholders: {projectid}: The GCP Project ID (e.g., customer-ai-project-123). {clustername}: The GKE Cluster Name (e.g., tpu-cluster-prod). {nodename}: The Node Name or Instance ID (e.g., tpu-node-1). {workloadname}: The Workload Name / JobSet Name (e.g., my-training-job-456). {namespace}: The Workload Namespace. {issuetime}: The timestamp of the issue (e.g., 2026-04-14T20:00:00Z). Time Handling & Execution Rules 1. Window Calculation: If an issue timestamp {issuetime} is provided, calculate the query time window as [{issuetime} - 30m] to [{issuetime} + 30m]. Let {starttime} = {issuetime} - 30m Let {endtime} = {issuetime} + 30m 2. Informational vs. Live Execution: If the user request is informational or query-formulation (e.g. "How can I check...", "How do I determine..."), or if live GCP project resources are not actively targetable, directly output the calculated time window, log names, and Cloud Logging filter templates without attempting live log execution commands. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills google/gke-ai-troubleshooting-tpu-vbar-oom