Agent skill · vasilyu1983
foundations-information-theory
Information-theory primitives for AI systems, entropy, mutual information, KL, compression, channel limits, MDL, bottlenecks, and signal quality. Use when quantifying information.
What it needs
About 11k tokens when loaded.
What this skill does
Information Theory Foundations When to Apply Apply information-theory when: Compressing prompts, retrieval contexts, logs, or feature sets Drift detection — distribution shift from baseline (KL, JS divergence) Feature selection by mutual information with target Retrieval re-ranking, MMR, or diversity-aware candidate selection Prompt-quality diagnosis via output-conditional entropy / Fano bound Hallucination / abstention gating via semantic entropy over meaning-clustered samples (#1) RL post-training diagnostics — policy-entropy collapse is the dominant failure mode in RLVR (#1) Agent-to-agent message budgets and KV-cache handoff sizing, framed as a bottleneck/rate problem (#6, #8) Skip and use simpler alternatives when: Question is about causation, not information — use foundations-causal-inference Single-feature linear correlation is sufficient — Pearson r is cheaper than MI for monotonic continuous data Streaming data with hard latency budget — full MI/KL is too slow; use sketches or sampled approximations N samples too small for stable entropy estimate (rule of thumb n > 5 × #bins per variable) Problem is system-stability or feedback control — use foundations-control-theory Bits/nats unit doesn't map to a business decision — risk of treating it as decoration, not signal --- 11 applied information-theory primitives for quantifying uncertainty, signal, and compression, backed by a formal theory map. Each primitive solves a specific measurement problem. Primitives are domain-agnostic: the same entropy calculation that budgets a context window also bounds a lossless compressor; mutual information that scores retrieval also measures feature relevance in ML. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills vasilyu1983/foundations-information-theory