Agent skill · software engineering · borghei
computer-use-automation
This skill should be used when the user asks to "build a computer-use agent", "automate a GUI with an AI agent", "when to use computer use vs an API", "make browser automation reliable", or "design screenshot-driven agent actions".
Why this skill is useful
Adds 2 executable Python scripts for decision-making and safety linting in GUI automation that aren't commonly found in public resources.
What it needs
Requires python installed locally. About 3k tokens when loaded. Last updated 2026-07-21. 462 stars on the source repository.
What this skill does
Computer Use Automation Category: Engineering Domain: AI Agents Overview The Computer Use Automation skill helps you design AI agents that operate a graphical interface the way a person does — take a screenshot, reason about what is on screen, then click, type, scroll, or navigate, and repeat. It covers the core perception→reason→action loop, the decision of when computer-use is the right tool versus a structured API/MCP tool (prefer a real API whenever one exists; reach for computer-use only for GUIs with no programmatic surface), reliability patterns (grounding every action in the current screenshot, verifying after each step, recovering from misclicks), safety guardrails (confirmation gates for destructive actions, sandboxing, avoiding blocking dialogs), and how to evaluate a computer-use agent. It is model-agnostic — the patterns apply to any computer-use-capable model and any GUI tool surface. Clarify First Before designing or auditing a computer-use agent, confirm these inputs. If any is unknown or vague, ASK — do not assume: [ ] Does a real API/MCP tool exist? — whether the target exposes an API, SDK, CLI, or MCP server, or is GUI-only (the single biggest factor; if a real API exists, prefer it and skip computer-use) [ ] Task & risk — what the agent must accomplish and whether any step is destructive or irreversible (delete, send, pay, submit), which sets the confirmation gates and sandboxing [ ] Which tool — advise on tool choice for a target, or lint a planned action sequence for safety (selects toolchoiceadvisor.py vs actionsafetylinter.py) Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills borghei/computer-use-automation