---
name: asr-transcribe-to-text
description: >-
  Transcribes audio or video to speaker-labeled, timestamped text, locally with MLX on Apple Silicon
  or remotely. Use for 转录 / 录音转文字 / 说话人分离 / 字幕, and also for preparing audio for ASR without
  transcribing: 转格式, 降采样到 16kHz, merging recorder segments, or compressing and speeding up audio
  before 飞书妙记 — even when it looks like a one-line ffmpeg job.
argument-hint: "[audio-or-video-file-path-or-url ...]"
---

# ASR Transcribe to Text

Transcribe audio/video to **speaker-labeled** text. Local execution has two
explicit routes. Long or unattended recordings use checkpointed whisper.cpp +
Silero VAD blocks, then late-fuse pyannote speakers. Short/medium recordings may
use the Qwen3-ASR + mlx-whisper alignment route. Neither route cuts ASR input at
diarization turns; speaker attribution happens after continuous-context ASR.

## Route before ASR: the transcript is the result, not the run

**Preserve course videos before choosing an ASR route.** When uploading a course
video, or when the user requires the picture or original media, upload the
original video file with its picture, audio and original speed. Skip Step 2 and
the preprocessing section unless the user explicitly requests conversion.
An existing transcript can satisfy the text request but does not satisfy an
outstanding video upload. Slow transfers or upload failures do not authorize
extracting audio, transcoding or speeding up the video.

**Apply original-speed requirements to audio as well.** For an original-file or
original-speed upload request, upload a compatible original directly. If merging
or format conversion is needed and permitted, keep 1.0x speed and the original
source-time mapping. Use 1.0x for playback/speaker review against the original
media unless the user permits a time-mapped derivative. Metered billing alone
does not authorize speedup; follow the preprocessing conditions below.

Before starting transcription, check the owning project's transcript catalog,
external source index, and declared prior-work carriers for an existing
canonical transcript using source ID, date, title, and entity terms. A verified
human-reviewed/current transcript ends transcription unless the user explicitly
asked for a new independent comparison. Raw audio existing is not a reason to
regenerate text that already exists.

When no canonical transcript exists:

1. For ordinary meeting/DJI recordings where cloud processing is allowed, use
   Feishu Minutes as the normal primary route (preprocess to a small M4A first).
2. Use local ASR when the user requires offline/privacy handling, Feishu is
   unavailable or failed, or the task explicitly needs an independent quality
   comparison.
2b. **For a second, independent speaker timeline**, use StepFun's async file
   endpoint via the `stepfun-asr` skill's `asr_file.py`. It is the only vendor
   diarization route here that does not share code or models with local pyannote, so
   agreement between the two is real corroboration rather than one tool agreeing
   with itself. Measured 2026-09-19 on a 10-minute film: both split the key turn
   identically, while Feishu Minutes mis-attributed it. The constraint is that it
   fetches a public URL — base64 is rejected — so it needs staging (see
   `aliyun-multi-account.md`); reach for it when an attribution is load-bearing,
   not for routine transcripts.
3. For non-meeting media or an explicit local/remote ASR request, choose the
   execution location by the audio-location rule below.

Do not run local ASR merely to make a two-route process look complete.

Before choosing local versus remote execution, read
[execution location and comparison](references/execution_location_and_comparison.md).
Use existing text and validated checkpoints first, then compare source placement,
processing permission, actual GPU/model identity and end-to-end quality evidence.
When measurements are missing, prefer an eligible route next to the audio and
state that this avoids a transfer; do not claim it is faster.

Configuration persists in `${CLAUDE_PLUGIN_DATA}/config.json`.

> **Speaker labels are the default.** Every run produces `[start-end] SPEAKER_xx: text`
> + CSV. Plain-text-only output is the opt-out (`--no-diarization`) for monologues,
> podcasts, or when you just want a summary — see Step 3. It also drops every
> timestamp: the plain-text path returns the session text and nothing else. A
> monologue that needs line-level timestamps (subtitles, jump-to-moment archives)
> runs the full pipeline and ignores the `SPEAKER_00` labels.
>
> **One-time setup for diarization:** pyannote is a gated HuggingFace model — it
> needs a token once (`## Speaker Diarization & Identification` below). First run
> without it FAILS with setup steps; after setup, full capability is permanent
> and auto-detected.

## Step 0: Detect Platform and Load Config

```bash
cat "${CLAUDE_PLUGIN_DATA}/config.json" 2>/dev/null
```

**If config exists**, read values and proceed to Step 1.

**If config does not exist**, auto-detect platform first:

```bash
python3 -c "
import sys, platform
is_mac_arm = sys.platform == 'darwin' and platform.machine() in ('arm64', 'aarch64')
print(f'Platform: {sys.platform} {platform.machine()}')
print(f'Apple Silicon: {is_mac_arm}')
if is_mac_arm:
    print('RECOMMEND: local-mlx')
else:
    print('RECOMMEND: remote-api')
"
```

Then use **AskUserQuestion** with platform-aware defaults:

For **macOS Apple Silicon** (recommended: local):
```
ASR setup — your Mac supports local MLX; choose from data location and the execution SOP.

Q1: Transcription mode?
  A) Local MLX — runs on your Mac's GPU, no API key needed
  B) Remote API — send audio to a server (vLLM, Tailscale workstation, etc.)

Q2: Does your network have an HTTP proxy that might intercept traffic?
  A) Yes — bypass proxy for ASR traffic (Recommended if using Shadowrocket/Clash)
  B) No — direct connection
```

For **other platforms** (recommended: remote):
```
ASR setup — local MLX requires macOS Apple Silicon. Using remote API mode.

Q1: ASR Endpoint URL?
  A) https://asr.example.com/v1/audio/transcriptions (Self-hosted remote ASR)
  B) http://localhost:8002/v1/audio/transcriptions (Local ASR server)
  C) Custom URL

Q2: Proxy bypass needed?
  A) Yes (Recommended for Shadowrocket/Clash/corporate proxy)
  B) No
```

Save config:
```bash
mkdir -p "${CLAUDE_PLUGIN_DATA}"
python3 -c "
import json
config = {
    'mode': 'MODE',           # 'local-mlx' or 'remote-api'
    'model': 'MODEL_ID',      # local: 'mlx-community/Qwen3-ASR-1.7B-8bit', remote: 'Qwen/Qwen3-ASR-1.7B'
    'max_tokens': 8192,       # local only; PER 20-minute chunk, not per recording
    'endpoint': 'URL',        # remote only
    'noproxy': True,
    'max_timeout': 900        # remote only
    # 'diarization_declined': True  # set only after the user explicitly declines
    #   the pyannote setup in Step 3 — every run then warns + goes plain-text
    #   until an HF token appears (auto-detected)
}
with open('${CLAUDE_PLUGIN_DATA}/config.json', 'w') as f:
    json.dump(config, f, indent=2)
print('Config saved.')
"
```

## Step 1: Resolve Input

Accept local files, direct media URLs, or web/podcast episode pages.

- **Web or podcast page URL**: inspect the page for an existing transcript first. Use an official/platform transcript only when it is directly accessible to the user's account. If the transcript endpoint requires a login token and none is available, say that clearly and fall back to ASR from the audio URL.
- **Local file, direct media URL, or page URL fallback**: run the bundled resolver. It extracts media from common page metadata (`og:audio`, media tags, JSON-LD, RSS-style enclosure links), downloads URLs with atomic temp-file replacement, verifies remote `Content-Length` when present, computes SHA-256, and validates the result with `ffprobe`.

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/resolve_media_input.py \
  INPUT_FILE_OR_URL [INPUT_FILE_OR_URL2 ...] \
  --output-dir OUTPUT_DIR \
  --manifest OUTPUT_DIR/media_manifest.json
```

For suspicious or high-value downloads, add `--decode-check` to make `ffmpeg` decode the whole file before transcription:

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/resolve_media_input.py \
  "https://www.xiaoyuzhoufm.com/episode/EPISODE_ID" \
  --output-dir OUTPUT_DIR \
  --manifest OUTPUT_DIR/media_manifest.json \
  --decode-check
```

Expected output:

```text
Downloaded ... bytes in ...s -> OUTPUT_DIR/episode-title.m4a
OUTPUT_DIR/episode-title.m4a
```

Use the printed local path as `INPUT_AUDIO` in later steps. If your runtime shows the literal `${CLAUDE_SKILL_DIR}` instead of a substituted path, resolve the skill directory per the Troubleshooting entry at the bottom of this document.

For third-party public podcasts or copyrighted media, save the transcript as a local file for the user's personal analysis. Do not paste a full long transcript into chat; provide a path, previews, summaries, or short excerpts instead.

## Step 2: Extract Audio (if input is video)

Apply this step to text-only ASR; skip it for the original-video upload route above.

For video files (mp4, mov, mkv, avi, webm), extract as 16kHz mono WAV:

```bash
ffmpeg -i INPUT_VIDEO -vn -acodec pcm_s16le -ar 16000 -ac 1 OUTPUT.wav -y
```

Audio files (wav, mp3, m4a, flac, ogg) can be used directly. Get duration:
```bash
ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 INPUT_FILE
```

**Cleanup**: After transcription succeeds, delete extracted WAV files to save disk space.

## Preprocess: Merge Segments & Shrink Metered Uploads (optional)

Skip this section for original-video uploads; retain the selected original file.
For original-file or original-speed audio requests, follow the route above;
neither this section nor its destination table overrides those constraints.

Run this BEFORE transcription when either applies:

- **The recording is a multi-segment dump** — body mics and field recorders split
  sessions into fixed-length files (e.g. `TX02_MIC024_....wav`, `TX02_MIC025_....wav`;
  `TX01/TX02` = DJI MIC MINI 2S internal recording — device roster and the
  recorder→Feishu-Minutes paths: the meeting-ingest skill's `meeting-ingest/references/architecture.md` §①-L0).
  Merge them once to preserve session order; the explicit long-audio runner
  (Step 3 Path L) then owns stable source-time blocks. Transcribing device
  segments separately throws away cross-segment context inside those chunks.
- **The audio goes to a metered ASR and the user permits speedup** (Feishu Minutes,
  any per-minute quota) — a
  pitch-PRESERVED speedup cuts billed duration directly, and modern ASR does not care:
  1.3x was user-verified on Feishu Minutes (2026-07-16) with no perceptible recognition
  difference, and public Whisper benchmarks show no sharp WER drop until 2.0x
  (≤1.5x = safe zone, ~3% WER increase at 1.5x; >2x unusable).

Use the bundled script — it merges, normalizes to 16 kHz mono, optionally speeds up,
and verifies its own output instead of trusting the ffmpeg exit code:

```bash
uv run ${CLAUDE_SKILL_DIR}/scripts/prepare_asr_input.py SEG1.wav SEG2.wav -o merged.wav   # merge only
uv run ${CLAUDE_SKILL_DIR}/scripts/prepare_asr_input.py SEG*.wav -o upload.m4a --speed 1.3  # only with speedup permission
```

Expected output:

```text
Merge order:
  1. SEG1.wav  [pcm_s24le 48000Hz ch=1 1800.14s]
  2. SEG2.wav  [pcm_s24le 48000Hz ch=1 1800.15s]
[OK] duration: 4946.19s vs expected 4946.18s (delta +0.00s)
[OK] boundary 1 @ 1384.7s: max_volume -15.5 dB
[info] overall: mean_volume -38.3 dB, max_volume 0.0 dB
Wrote upload.m4a
```

- Segments sort by the `YYYYMMDD_HHMMSS` timestamp embedded in their filenames when
  every file has one (recorder dumps do); otherwise the given order is kept with a note —
  eyeball the printed merge order before transcribing.
- Self-verification: output duration must equal Σsegments ÷ speed (±1.5 s, hard FAIL
  otherwise); each splice gets a 10 s volume spot-check (dead air at a boundary = wrong
  order or a missing segment); overall loudness prints for comparison with the source.
- Speedup must be `atempo`-style pitch-preserved stretch — never sample-rate trickery,
  which shifts pitch and breaks both ASR accuracy and diarization voiceprints.
- For authorized speedup, retain the originals, speed factor and segment boundaries.
  Map timestamps back to the original media before source-time playback or speaker
  review. If the downstream path cannot preserve that mapping, keep 1.0x speed;
  sped-up upload timestamps are not original-media timestamps.
- **Pick the output format by destination** — codec follows the file extension:

  | Destination | Format | Why |
  |---|---|---|
  | Local MLX pipeline (Path A) | `.wav` or `.m4a` | Both feed the pipeline directly (m4a verified 2026-07-18: a 3-min slice transcribed cleanly). M4A is ~5x smaller — 324 MB WAV → 63 MB M4A on a 2h49m merge, duration identical to the second |
  | Metered upload (Feishu Minutes, per-minute quota) | `.m4a` at 1.0x; `--speed 1.3` only with speedup permission and source-time mapping | AAC 48k is speech-transparent for ASR, ~30% smaller than mp3 at equal speech quality; permitted speedup cuts billed duration ~23% |
  | Self-hosted vLLM endpoint (Path B) | `.ogg` | Accepted where MP3 is refused, and ~8× smaller than WAV — which is what keeps a long recording under the server's 25 MB request cap. See Path B's limits section |
  | Lossless archive | `.flac` | ~50% of WAV, bit-perfect |
  | Only when the target rejects the above | `.mp3` | Compatibility fallback |
- Keep the originals until the transcript passes Step 4 verification.

## Option: Upload to Feishu Minutes for transcription

Use this path with the selected original video or preprocessed audio when the
user wants **Feishu Minutes**, or
when an ordinary meeting/DJI recording has no canonical transcript and cloud
processing is allowed. It is the normal meeting-audio route, not a fallback
that requires a failed local run first.

**Trigger phrases**: 传到妙记 / 上传到飞书妙记 / 让妙记转写 / create a minute from this audio / upload to Feishu minutes.

**Track the requested outcome before uploading**:

- **Upload-only** — the user explicitly says they only want a Minute link or
  only want the media uploaded. Creating `minute_url` is the terminal state.
- **Transcript-only** — the user wants the transcript/summary but no project
  filing. After upload, follow the current version-matched `lark-minutes`
  instructions through the ready transcript.
- **Project delivery** — the user wants correction, routing, a knowledge-base
  file, project indexes, or Git handoff. `meeting-ingest` owns the overall job;
  this skill is only its preprocessing participant. Switch to that orchestrator
  before upload and keep running until its delivery receipt is verified.
- **Downstream unspecified** — the request says to preprocess/upload (for
  example, “上传到飞书妙记，先转成适合 ASR 的格式”) but does not say what should
  happen after the Minute exists. Honor the requested upload without inventing
  transcript or project scope, then emit `outcome_pending` with the durable
  token/URL. A later transcript or knowledge-base request resumes that same
  Minute; it never starts over or depends on the original guess.

Do not infer upload-only merely because the user's first clause says “上传到妙记”.
An explicit downstream outcome in the same request or project context wins.

**Constraints**:
- **No proxy**: all `lark-cli` calls must use `LARK_CLI_NO_PROXY=1`.
- **Single profile**: use the active Feishu profile only. Do not iterate tenant
  profiles or invoke tenant routing.
- **No duplicate local ASR by default**: Feishu owns transcription on this
  branch. Local ASR runs only for an explicit offline/comparison purpose.
- **Load current Feishu guidance**: use `lark-cli-router`, then read the
  version-matched `lark-minutes` upload/detail instructions. Do not copy a stale
  stop condition from this skill over the live CLI contract.

**Step-by-step**:

1. **Select the media required by the outcome**: use the original video for a
   course or other original-video upload; use already-preprocessed audio for the
   ordinary meeting-audio route. Feishu supports MP4/MOV videos and audio formats
   including `.m4a`, `.mp3`, `.wav` and `.aac`; the preprocessor's "Metered upload"
   row applies to the meeting-audio route. Keep the file
   under 6 GB and under 6 hours — those are Feishu upload hard limits.

2. **Upload to Drive as the user**:
   ```bash
   LARK_CLI_NO_PROXY=1 lark-cli drive +upload \
     --file '<selected-media-path>' \
     --name '<basename>' \
     --as user \
     --format json
   ```
   From the result, record `file_token`. If the command errors with a path
   validation or multipart failure, do not retry blindly. For preprocessed audio,
   switch format or size strategy and try once more, then report the exact failure.
   For original video, diagnose the path or transfer failure and preserve the
   selected media; do not change the video to audio to make it upload.

3. **Create the minute from that Drive file**:
   ```bash
   LARK_CLI_NO_PROXY=1 lark-cli minutes +upload \
     --file-token '<file_token>' \
     --as user \
     --format json
   ```
   From the result, record `minute_token` and `minute_url`.

4. **Branch on the previously fixed terminal outcome**:

   For an original-video upload, independently read back the cloud file name and
   size, then verify the ready Minute contains a video track and the original
   duration. An audio-only Minute or a queued upload does not satisfy this outcome.

   - Upload-only: return `minute_url` and stop.
   - Transcript-only: use the current `lark-minutes` detail command with its
     ready-wait behavior and `--transcript`. A created URL is intermediate;
     success requires a readable transcript artifact.
   - Project delivery: return this handoff tuple to `meeting-ingest` and continue
     in the same run: `prepared_media`, `file_token`, `minute_token`,
     `minute_url`, `next_required_phase=minute_ready`. The orchestrator waits for
     the cloud transcript, invokes `sync-feishu-minutes` for token-scoped
     ingest/routing/delegation, runs full `transcript-fixer`, updates every
     project-owned index, completes Git handoff, and records delivery.
   - Downstream unspecified: return `minute_token`, `minute_url`,
     `outcome_pending`, and `next_required_phase=outcome_decision`. Do not wait
     for or file a transcript until the user supplies that downstream outcome.

If readiness times out or a downstream stage blocks, report the exact last
completed phase plus the durable `minute_token`/`minute_url`. Resume that same
token; never re-upload just to obtain a fresh URL.

**Expected output**:
- Upload-only success: a single `minute_url` the user can open.
- Downstream-unspecified success: the requested upload is complete, while the
  run remains explicitly resumable at `outcome_pending`; do not call it project
  delivery.
- Transcript-only success: `minute_url` plus a readable transcript artifact.
- Project-delivery success: only the `meeting-ingest` pushed delivery receipt;
  preprocessing, URL creation, and transcript download are intermediate phases.
- Failure: exact API error from `drive +upload` or `minutes +upload`, plus one
  suggested next action.

**Wrong-skill recovery**: if this request lands while you are inside
`sync-feishu-minutes`, choose by outcome: upload-only routes here; project
delivery routes to `meeting-ingest`, which calls this preprocessing branch and
then returns to token-scoped Feishu ingestion. Never silently downgrade project
delivery to a Minute URL.

## Step 3: Transcribe (speaker labels by default)

### Path L: Long local recording — whisper.cpp + Silero VAD (default for >30 min)

Use this path for recordings longer than 30 minutes, unattended batch work, or
any source likely to contain long silence / post-meeting a