Agent skill · data analytics · jeremylongshore
glean-performance-tuning
Optimize Glean search relevance and indexing throughput with batch sizing,
Why this skill is useful
Adds executable scripts for optimizing Glean search performance and indexing that are not commonly found in public documentation.
What it needs
Requires p-queue installed locally. Requires glean account access. About 3k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
Glean Performance Tuning Overview Glean's enterprise search API handles search queries across multiple connectors, bulk document indexing, and connector sync throughput. Search latency compounds when querying across dozens of datasources simultaneously. Large indexing jobs (10K+ documents) require careful batching to avoid rate limits and maintain connector sync schedules. Optimizing batch sizes, caching frequent search results, and tuning connector configurations reduces search P95 latency and keeps indexing pipelines within SLA windows. Caching Strategy Batch Operations Connection Pooling Rate Limit Management Monitoring Performance Checklist [ ] Batch indexing calls at 100 docs per request with 3 concurrent workers [ ] Use incremental indexing for real-time updates (< 100 docs) [ ] Switch to bulkindexdocuments for daily full refreshes (> 1K docs) [ ] Cache repeated search queries with 1-min TTL [ ] Set descriptive document titles and full body text for relevance [ ] Keep connector sync schedules staggered to avoid burst load [ ] Monitor P95 search latency and indexing throughput [ ] Enable keep-alive connections with high socket count for parallel ops Error Handling Issue Cause Fix ------- ------- ----- Slow cross-datasource search Too many connectors queried in parallel Prioritize datasources, set query scope 429 on bulk indexing Batch size or concurrency too high Reduce to 100/batch, 3 concurrent, 500ms interval Stale search results Index lag after document updates Use incremental indexing with webhooks on change Connector sync timeout Large datasource with no checkpointing Enable incremental sync with cursor tracking Missing documents in results Incomplete metadata during indexing Include title, body, author, and updatedat fields Prerequisites A baseline for latency percentiles, error rate, freshness lag, queue depth, and authorized-search coverage by datasource. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/glean-performance-tuning