Agent skill · software engineering · microsoft
agent-host-e2e-tests
Use when writing, recording, updating, validating, or troubleshooting Agent Host end-to-end and real-provider integration tests under src/vs/platform/agentHost/test/node/e2e and providerIntegration. Covers adding a cross-provider test, re-recording fixtures after an SDK bump, required cross-platform Azure validation, gating non-deterministic or platform-specific tests, and diagnosing replay cache misses.
Why this skill is useful
Adds detailed workflows and executable scripts for managing end-to-end tests with specific protocols and fixture handling that the AI wouldn't generate on its own.
What it needs
Requires git installed locally. Requires github account access. About 6k tokens when loaded. Last updated 2026-08-07. 188,455 stars on the source repository.
What this skill does
Agent host end-to-end tests These tests run the whole agent host end-to-end (real server, real bundled provider SDK/CLI, real AHP protocol) while replaying recorded model traffic from committed YAML fixtures — deterministic and tokenless. Before doing anything, read the architecture + troubleshooting reference: src/vs/platform/agentHost/test/node/e2e/README.md It documents the mental model, the fixture format, every config flag, and a symptom→cause→fix troubleshooting table. This skill is only the workflows; the README is the source of truth for how it works. When validating an upstream provider/runtime fix, use ci-artifact-testing to check for a compatible CI artifact before building locally. Run the same strict replay against the bundled baseline and candidate; preserve the test assertions and existing cross-platform validation requirements. Core invariants 1. Replay is default and strict. No env var serves committed fixtures without a token or network. An unrecorded request is a hard cache miss. 2. Fixture names derive from test titles. Renaming a model-backed test orphans ${provider}-${slug}.yaml; re-record after a rename. Tests registered with conformanceTest(...) or providerHostOnlyTest(...) share captures/empty.yaml. 3. Recording is intentional and credentialed. AGENTHOSTREPLAYRECORD=1 talks to real CAPI and needs GITHUBTOKEN or gh auth token. Keep prompts trivial and read-only in temporary directories. 4. Fixtures are generated, never hand-edited. Fix normalization or redaction and re-record if a capture contains unstable or sensitive data. 5. Disabled variants stay accountable. Record every gate in e2e/KNOWNISSUES.md with its scope, expected and observed behavior, and focused reproduction command. 6. Azure validation is required. Every new Agent Host E2E or real-provider integration test must pass a focused build from VS Code pipeline definition 111 before merge. Basic workflow: write and evaluate an E2E test 1. Choose the boundary and tier. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills microsoft/agent-host-e2e-tests