Agent skill · software engineering · jeremylongshore
palantir-core-workflow-a
Design and validate a Foundry Python transform pipeline with explicit datasets, compute choice, expectations, and incremental semantics. Use when building or changing batch data pipelines. Trigger with "Foundry transform" or "Palantir data pipeline".
Why this skill is useful
Adds executable scripts and specific patterns for building Palantir Foundry data pipelines that are not widely documented.
What it needs
Requires palantir installed locally. Requires palantir account access. About 3k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
Palantir Python Transform Pipeline Overview Create a pipeline whose inputs, outputs, engine, write behavior, and quality checks are explicit before the first production build. Prefer the simplest supported compute engine that satisfies data scale and feature requirements, then prove the choice with Foundry metrics. Prerequisites Identify the owning project, Code Repository, input and output datasets, schema contract, data classification, and build schedule. Confirm whether the workload requires Spark or can use a single-node engine such as Polars, pandas, or DuckDB. Read references/official-docs.md and inspect current input transaction history before selecting incremental semantics. Develop on a sandbox branch with representative but appropriately protected data. Current Contract Python transforms support batch and incremental pipelines, reusable libraries, expectations, and single-node or distributed engines. Input and output datasets must differ; using the same dataset creates a cyclic dependency. Incremental input modes and output write modes have precise transaction semantics; modify and replace are not interchangeable. A snapshot build may be needed when incremental transaction history becomes progressively slow or invalid. Instructions 1. Write the pipeline contract: owner, inputs, output, primary key or deduplication rule, schema, data-quality expectations, and recovery objective. 2. Choose the engine from required features and observed scale; document why single-node or Spark is appropriate. 3. Implement the transform with explicit input and output declarations and keep pure business logic separately testable. 4. If incremental processing is justified, define input read modes, output write mode, late-arrival behavior, and snapshot recovery. 5. Preview representative cases, run repository checks, build on the branch, inspect metrics and output transactions, then request review. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/palantir-core-workflow-a