---
name: research-scout
description: "Mines academic papers, research blogs, and curator newsletters for stealable methods and frameworks. Use when scanning research for applicable techniques across AI/ML/SWE."
compatibility: Portable core. Works on Claude Code and Codex.
version: "1.1"
last_validated: 2026-07-11
---

# Dev Research Scout

Scans high-signal research sources for **methods, frameworks, and ideas worth applying** to your own work, and converts the top finds into idea cards with how-to-apply recipes, evidence quality grades, and reproducibility notes.

**Supported sources:** arXiv, Hugging Face Papers, Semantic Scholar, Papers with Code (archive only — shut down Jul 2025), conference proceedings (NeurIPS / ICML / ICLR / ACL / EMNLP / KDD), industry research blogs (Anthropic / OpenAI / DeepMind / Google Research / Meta AI / Microsoft Research / Apple ML), and curator newsletters (Lilian Weng, Sebastian Raschka, Eugene Yan, Latent Space, Simon Willison, The Batch, Import AI, Interconnects / Nathan Lambert, Davis Summarizes Papers / Davis Blalock).

**Output is a generative toolkit, not a landscape report:**
- pattern catalog (methods worth stealing, with how-to-apply)
- anti-pattern catalog (research traps — irreproducibility, benchmark gaming, hype)
- recipes (extraction, validation-before-adoption, kill criteria)

**Key distinction from sibling scouts:**
- **This skill** = research-grade idea mining (papers + research blogs + curated synthesis)
- **`research-painpoint-scanner`** = community-pain mining (Reddit / HN / GitHub Issues / G2 / Stack Overflow)
- **`research-arxiv-scout`** = arXiv-only deep triage with category taxonomy and attribution; specialist downstream
- **`research-git`** = public GitHub repo research for skills, practices, and code patterns (separate concern)

Use this skill when the question is "what methods or frameworks are worth stealing from recent research?" — escalate to `research-arxiv-scout` for arXiv-only work where category taxonomy and attribution matter most.

---

## Quick Reference

| Need | Go to |
|------|-------|
| Pick the source mix | `## Source Selection Guide` |
| Run the end-to-end scan | `## Workflow` |
| Reject hype / irreproducible / benchmark-gamed work | [known-traps.md](references/known-traps.md) |
| Pattern-match a paper to a known method shape | [idea-extraction-framework.md](references/idea-extraction-framework.md) |
| How to actually apply a stolen idea | [recipes.md](references/recipes.md) |
| Source-specific query and credibility guidance | `## Navigation` |
| Package the idea cards | `## Templates & Assets` |
| Mine industry/eng blogs + HCI papers for killer-feature attribution (bundle handoff) | `## Killer-Feature Mode (Feature-Precedent Mining)` |

## When to Use

Invoke when users ask for:
- "What methods are people using for {{topic}} that I haven't tried?"
- "Find recent {{AI/ML/SWE}} ideas worth stealing for {{project}}"
- "Mine arXiv + research blogs for {{topic}} in the last {{N}} days"
- "What's worth stealing from NeurIPS / ICML / ICLR {{year}}?"
- "Show me frameworks for {{evaluating LLM agents / RAG eval / inference scaling / etc.}}"
- "Update {{skill name}}'s knowledge base with recent research"

## When NOT to Use

| Situation | Use instead |
|-----------|------------|
| arXiv-only deep triage with attribution | `research-arxiv-scout` |
| Community pain points, not research methods | `research-painpoint-scanner` |
| Mining public GitHub repos for skills, practices, or code patterns | `research-git` |
| Validated Q&A answers or known-error solutions (the Stack Overflow corpus / Stack Overflow for Agents exchange) | `qa-debugging` — that is solved-answer lookup, not research-method mining |
| Production deep-research synthesis (verified citations + reasoning trace) | `ai-deep-research` |
| Single-paper summary for a known arXiv ID | `research-arxiv-scout` step 3 |
| End-user career positioning, company interview reviews, recruiter pitches, or CV tailoring | `career-jobhunt`; this skill may still mine research methods to improve that skill |

---

## Source Selection Guide

| Source | Best for | Query method | Idea quality |
|--------|----------|-------------|--------------|
| **arXiv** | Bleeding-edge methods (preprints, no peer review) | `export.arxiv.org/api/query` | High volume, mixed signal — needs trap filter |
| **Hugging Face Papers** | Community-curated daily highlights | `huggingface.co/papers` + RSS | Pre-filtered, signal-rich, biased to LLM/VLM |
| **Semantic Scholar** | Citation graphs, prior work, influential papers | Semantic Scholar API | Best for "what built on this?" |
| **Papers with Code** | DEAD (Meta shutdown Jul 2025) — historical archive only | `github.com/paperswithcode/paperswithcode-data` (frozen) | None live; reconstruct via HF Papers + GitHub (`research-git`) — see [papers-with-code-strategy.md](references/papers-with-code-strategy.md) |
| **Conference proceedings** | Peer-reviewed, vetted methods | NeurIPS / ICML / ICLR / ACL / EMNLP / KDD sites | Lagged but high-credibility |
| **Industry research blogs** | Production-tested methods at scale | RSS or direct site (Anthropic / OpenAI / DeepMind / Google / Meta / MSR / Apple) | High signal but PR-tinged |
| **Curator newsletters** | Pre-synthesized, opinionated, applied | Substack / blog RSS | Highest applicability, reflects curator bias |

**Default mix:**
- Fast scan (1-2 hr): HF Papers + 1 curator newsletter (Lilian Weng or Eugene Yan) + GitHub repo signal (via `research-git`) for the target task
- Standard scan (1 day): arXiv + HF Papers + Semantic Scholar + 2 industry blogs + 2 curator newsletters
- Deep scan (multi-day): all live source types (arXiv, HF Papers, Semantic Scholar, conferences, industry blogs, curator newsletters; Papers with Code is dead — archive only), time windows 7d/30d/90d, full trap filter, full extraction recipes

---

## Quick Start

> **Semantic Scholar API key:** New keys are no longer approved for free email domains (gmail, outlook, etc.). Use an institutional email to apply, or fall back to [OpenAlex](https://openalex.org/) — same free-key-required model but no email-domain restriction; register at openalex.org/settings/api (OpenAlex has required a key for every request since 2026-02-13). See `references/semantic-scholar-strategy.md` for detail.

**Required inputs:**
- `topic` — Research topic or method family (e.g., "LLM agent tool use", "RAG eval", "inference batching", "distillation")
- `target` — Where the stolen ideas will be applied (e.g., "ai-rag skill", "production RAG service", "agent evals")

**Optional inputs:**
- `sources` — Which source families to scan (default: arxiv, hf_papers, semantic_scholar, curator_newsletters)
- `windows` — Time windows (default: 30d, 90d, 365d)
- `min_evidence_grade` — Minimum evidence grade (`A`/`B`/`C`/`D`/`F`, default `C`; `F` is the floor used by the scoring engine and validator)
- Source-specific: `--arxiv-categories`, `--conference`, `--blog-domains`, `--curators`

---

## Workflow

### ASCII Flow

```text
research idea-mining request
  -> Frame topic, target application, source mix, and time windows
  -> Search academic, code-linked, conference, blog, and curator sources
  -> Normalize findings into the TSV schema
  -> Extract stealable methods, evidence, transfer limits, and kill criteria
  -> Score ideas and apply trap filters
  -> Match method shapes and package idea cards
  -> Produce scan report or sources-json updates with verified claims
```

### Step 1: SCOPE — Frame the idea-hunt

1. State the **target application**: "ideas for {{X}} that I'll apply in {{Y}}".
2. State the **method family/families**: e.g., "agent planning + tool selection", "retrieval reranking", "test-time compute scaling".
3. Pick sources from the [Source Selection Guide](#source-selection-guide). For AI/ML, default to arXiv + HF Papers + Semantic Scholar + ≥1 curator. For SWE, prefer conference proceedings (ICSE/FSE/PLDI) + GitHub repo signal (via `research-git`) + industry blogs. (Papers with Code is dead — do not include it as a live source.)
4. Confirm time windows. Methods aging faster (LLM agents) → 30d/90d. Slower (compilers, type systems) → 1y/3y.

### Step 2: SEARCH — Generate and execute queries

Run the source-specific query generator(s):

```bash
# arXiv
python3 scripts/generate_arxiv_queries.py --topic "{{topic}}" --categories cs.AI cs.CL cs.LG --windows 30d 90d 365d

# Hugging Face Papers
python3 scripts/generate_hf_papers_queries.py --topic "{{topic}}" --windows 30d 90d

# Semantic Scholar
python3 scripts/generate_semantic_scholar_queries.py --topic "{{topic}}" --min-citations 5 --windows 365d 1095d

# Papers with Code — DEAD SOURCE (Meta shutdown Jul 2025). The script is now a
# fail-loud shim that emits HF Papers + GitHub (research-git) replacement URLs.
python3 scripts/generate_papers_with_code_queries.py --task "{{task slug}}"

# Conference proceedings (manual seed list, scripts emit URLs)
python3 scripts/generate_conference_queries.py --conference neurips --year 2025 --topic "{{topic}}"

# Research blogs and curator newsletters (RSS/site map seeds)
python3 scripts/generate_blog_queries.py --domains anthropic.com openai.com deepmind.google research.google ai.meta.com --topic "{{topic}}"
```

For each result, extract into TSV format matching [research-findings.tsv](assets/research-findings.tsv). Required fields:

- `source_url` — Stable URL (arXiv abs page, blog post, paper landing)
- `source_type` — `arxiv`, `hf_papers`, `semantic_scholar`, `papers_with_code`, `conference`, `industry_blog`, `curator_newsletter`
- `source_context` — Source identifier (e.g., "arxiv:cs.AI", "hf_papers", "ss:semanticscholar.org", "neurips/2025", "anthropic.com/research", "lilianweng.github.io")
- `paper_id` — arXiv ID, DOI, conference paper ID, or canonical URL hash when no ID exists
- `title`, `authors`, `posted_at`, `observed_at`
- `method_family` — From the [idea-extraction-framework](references/idea-extraction-framework.md) taxonomy
- `idea_summary` — 1-2 sentence statement of the *method/framework/idea*, not the paper
- `evidence_grade` — A/B/C/D/F using [grading rubric](references/idea-extraction-framework.md#evidence-grades)
- `reproducibility` — `code+benchmarks`, `code_only`, `paper_only`, `proprietary`
- `lift` — `low` (1-3 days), `medium` (1-2 weeks), `high` (>2 weeks)
- `trap_tags`, `shape_tags`, `quote`, `window`
- `claim_type` — `absolute-performance` | `relative-gain` | `efficiency` | `robustness`; see [idea-extraction-framework.md](references/idea-extraction-framework.md#claim-types) — efficiency/robustness claims transfer best regardless of evidence grade
- `cluster_id` — stable method-identity key shared by every finding about the *same method* across different source types. **This is what drives cross-source corroboration** (≥2 distinct `source_type` sharing one `cluster_id` = corroborated). Assign a short slug per method (e.g., `reflexion-critique-retry`); reuse it across the arXiv preprint, the curator mention, and the GitHub repo. If blank, the aggregator falls back to `paper_id` and emits a loud "corroboration unreliable" warning.

Validate before aggregation:

```bash
python3 scripts/validate_findings_tsv.py findings.tsv
```

### Step 3: EXTRACT — Convert papers to ideas

For each surviving entry, extract the **stealable unit** using [idea-extraction-framework.md](references/idea-extraction-framework.md):

1. **Method or framework name** (or invent a clean one if the paper buries it)
2. **What it actually does** in 1-2 sentences (no jargon shield)
3. **Inputs / outputs / preconditions** — what you need to use it
4. **Evidence behind it** — empirical claim + benchmark + N + baselines
5. **Why it might transfer** to your target — and why it might not
6. **Lift estimate** — days to a working prototype against your stack
7. **Kill criteria** — when you'd stop pursuing it

Discard entries where the method can't be described without the original phrasing — that's a strong "no actual idea" signal.

### Step 4: SCORE — Rank ideas

```bash
python3 scripts/aggregate_research_ideas.py findings.tsv --output scored.tsv --target "{{target}}"
```

**The gate is rule-decided; the score only ranks.** A deterministic rule ladder
sets `gate_status`; the numeric score never changes a gate decision — it only
orders rows *within* a bucket. This removes the old failure mode where a
subjective `applicability` guess (default 3) flipped promote/kill.

**Rule ladder (first match wins for the gate):**

1. trap 11 or 12 present → `kill`
2. ≥3 trap tags → `kill`
3. `evidence_grade == F` → `kill`
4. `shape == negative-result` → `background` (exempt from low-score kill — a falsified method you considered is *information*, not noise)
5. corroboration < 2 distinct `source_type` sharing one `cluster_id` → cap at `validate` (enforces the Evidence Quality Gates promote precondition)
6. `reproducibility == proprietary` → cap at `validate`
7. `evidence_grade == D` → cap at `validate`
8. any of traps {1,5,6,8} present → cap at `validate`
9. else → `promote`

**Ranking score (ordering only, never gates):** `(applicability × evidence_strength × reproducibility) / (lift × trap_penalty)`, with per-trap numeric adjustments from [known-traps.md](references/known-traps.md#scoring-effect) (`evidence -1` for trap 2, `applicability -1/-2` for traps 3/9, `lift +1 tier` for trap 4). Weights: `applicability` 1-5 (default 3); `evidence` A=5 B=4 C=3 D=2 F=1; `reproducibility` code+benchmarks=5 code_only=4 paper_only=2 proprietary=1; `lift` inverse low=1 medium=3 high=5; `trap_penalty` 1.0 +0.5 per non-hard trap.

The aggregator emits `gate_status` (`promote` / `validate` / `kill` / `background`), `gate_reason`, `score` (rank-only), and `corroboration` (`yes` / `no` / `unreliable-no-cluster_id`). Do not promote `kill` rows; `background` rows go in the report's Background section, not the shortlist.

### Step 5: COMPARE WINDOWS — Detect emerging vs. mature methods

Use **citations-per-month-since-publication** rather than raw counts to avoid penalising recent papers. Operational thresholds (Semantic Scholar `influentialCitationCount`):

- **Emerging** — first influential citations within 90 days of publication with an accelerating monthly rate (month-over-month increase ≥ 1 influential citation); sparse in 365d window
- **Cresting** — > 10 influential citations in the last 60 days; mentions accelerating across arXiv, HF Papers, and curator sources — adopt now or be late
- **Mature** — stable influential-citation rate over 90d–365d, ≥ 2 independent implementations; safest to adopt
- **Declining** — influential-citation rate falling for 2+ consecutive 30-day windows; likely superseded — investigate the successor

**Cross-source corroboration:** Methods cited in 2+ source families (e.g., arXiv paper + curator newsletter mention + Papers with Code implementation) are high-confidence steal candidates.

### Step 5b: APPLY TRAP FILTER — Reject false positives

Run each top idea through [known-traps.md](references/known-traps.md):

1. Tag each surviving idea with applicable traps (multi-tag allowed).
2. Apply each trap's counter-recipe; downgrade or kill per the scoring-effect table.
3. Trap 11 (`proprietary-component`) and Trap 12 (`benchmark-gaming`) are hard kills unless an alternative exists.
4. Log discarded/downgraded ideas with one-line reason in the scan report.

### Step 5c: MATCH SHAPES — Pattern-match surviving ideas

Match each surviving idea against shape catalog in [idea-extraction-framework.md](references/idea-extraction-framework.md#method-shapes):

1. Identify the shape(s): `prompting-pattern`, `architecture-tweak`, `training-recipe`, `evaluation-method`, `data-construction-recipe`, `inference-time-method`, `system-design-pattern`, `theoretical-bound`, `negative-result`, `survey-or-taxonomy`.
2. Multi-shape methods often signal generality.
3. `negative-result` is high-value when it falsifies a method you considered (saves time). The aggregator assigns it `gate_status = background` (rule 4) so it is never killed for lacking a benchmark gain — it lands in the report's Background section.
4. `survey-or-taxonomy` is *not* a stealable idea — also `background`, list as context only.

### Step 6: PACKAGE — Generate idea cards

1. Fill in one [idea-card.md](assets/idea-card.md) per surviving idea (use [recipes.md](references/recipes.md) to populate the "How to apply" section).
2. Compile into [research-scan-report.md](assets/research-scan-report.md).
3. If updating skill `data/sources.json` files, follow the format in `../research-arxiv-scout/assets/sources-json-template.md`.

---

## Killer-Feature Mode (Feature-Precedent Mining)

Specialized mode for contributing the **`industry_blog_attribution`** and **`hci_retention_paper`** signals to the bundle's [Killer-Feature Convergence Protocol](../research-review-mining/references/killer-feature-convergence.md) owned by `research-review-mining`.

**Premise.** Engineering and PM blog post-mortems and HCI retention papers periodically attribute retention, conversion, or revenue to a specific feature with named metrics. These are the highest-credibility single signals in the bundle (when they exist).

**When to use:** bundle handoff from `research-review-mining` Killer-Feature Mode KF3, OR you want a published metric-backed attribution claim for a candidate feature.

**Workflow:**

```text
KF-PREC-1. SCOPE — commercial product + candidate feature_id
KF-PREC-2. SCAN  — generate_blog_queries.py with engineering-blog domain list
                   biased toward netflixtechblog/stripe/figma/linear/notion/eng.uber/etc.;
                   generate_conference_queries.py for CHI / CSCW / UIST / IUI
KF-PREC-3. EXTRACT — classify attribution as explicit / strong / implicit / reject;
                     extract the feature noun (must be testable) and the WTP quote
KF-PREC-4. APPEND — to ../research-review-mining/assets/pay-trigger-ledger.tsv
                    signal_type = industry_blog_attribution (blog posts)
                                 | hci_retention_paper (CHI/CSCW/UIST/IUI)
KF-PREC-5. HAND OFF — run ../research-review-mining/scripts/converge_killer_features.py
```

**New method shape.** This mode adds `monetizable-feature-pattern` to the [idea-extraction-framework](references/idea-extraction-framework.md#method-shapes) catalog. It uses different scoring gates than the research-method shapes (Trap 11 and Trap 12 do not auto-kill; instead it kills on marketing/PR authorship and promotes on quantitative metric + internal authority).

**References:**
- [references/feature-precedent-mining.md](references/feature-precedent-mining.md) — full extraction protocol, source mix, anti-patterns, precision honesty
- [../research-review-mining/references/killer-feature-convergence.md](../research-review-mining/references/killer-feature-convergence.md) — bundle Convergence Rule
- [../research-review-mining/references/llm-extraction-prompts.md](../research-review-mining/references/llm-extraction-prompts.md) §7 — engineering post-mortem attribution prompt

---

## Templates & Assets

| Template | Purpose |
|----------|---------|
| [research-scan-report.md](assets/research-scan-report.md) | Primary output — full scan with rankings, ideas, traps caught |
| [idea-card.md](assets/idea-card.md) | Per-idea card: method, evidence, lift, how-to-apply, kill criteria |
| [research-findings.tsv](assets/research-findings.tsv) | Input format for `aggregate_research_ideas.py` (header + example) |

## Scripts

| Script | Source | Purpose |
|--------|--------|---------|
| [generate_arxiv_queries.py](scripts/generate_arxiv_queries.py) | arXiv | `export.arxiv.org/api/query` URLs |
| [generate_hf_papers_queries.py](scripts/generate_hf_papers_queries.py) | HF Papers | `huggingface.co/papers` URLs + JSON endpoints |
| [generate_semantic_scholar_queries.py](scripts/generate_semantic_scholar_queries.py) | Semantic Scholar | API UR