What is Poiesis
Poiesis is a GA4GH Task Execution Service (TES v1.1.0) for Kubernetes. Workflow engines send it tasks over a REST API; it runs each task as one Pod with its own volume. One container image (plus one for migrations), one PostgreSQL schema, no broker.
- Your image and command run as submitted. Nothing is wrapped or injected (except a
/bin/shwrapper for executors that setignore_error). - Inputs are staged onto the task's volume, outputs are uploaded from it, and the volume is deleted with the task.
- Callers authenticate with tokens from your identity provider, or with API keys for long workflow runs.
Components
| Component | Role |
|---|---|
| API | FastAPI service exposing TES under /ga4gh/tes/v1. No scheduler, no reconciler. |
| TaskPod | One Pod per task: trec (sidecar), then tif, exec-0..N, tof, ack. tif and tof only if the task has inputs or outputs. |
| TRec | In-pod recorder. Watches the surrounding Pod, marks the task INITIALIZING, then RUNNING when the first executor starts, and writes executor logs. |
| TCtl | Leader-elected Deployment. Writes every final state from the Pod's phase, including OOMKill, eviction, node loss, pending timeouts and vanished Pods. |
| Postgres | Relational store. Conditional UPDATEs make concurrent state writes (TRec, TCtl, a cancel) race-correct. |
See Tasks and TaskPods and Architecture for how they fit together.
Storage protocols
Inputs: s3://, http://, https:// and inline content. Outputs: s3://, http:// and https://. file:// works for both when the operator allows it. See Storage and staging.
What's yours and what's ours
| You | Poiesis |
|---|---|
| Executor image and command | Pod composition, scheduling, lifecycle |
| Input and output URLs | Staging and upload |
TesResources | Volume sizing, container resources |
| Kubernetes 1.29+, Postgres, object storage, an OIDC issuer | API, recorder, controller, schema |
Pick a client
curl: the fastest way to learn the API. The Quickstart has working examples.- A workflow engine: Nextflow, Snakemake, Sprocket and others. See Workflow engines.
- Your own code: the TES API is the contract; py-tes is a ready Python client.
What Poiesis doesn't do
- Schedule across clusters. One TaskPod, one cluster.
- Mutate your executor image. Commands run exactly as submitted.
- Cache anything. Each submission is independent.
- Run its own login flow. Poiesis is a resource server, not an OAuth client. It validates tokens minted by an external identity provider (Keycloak, Auth0, Dex, …). See Authentication.
