Skip to content

What is Poiesis ​

Poiesis is a GA4GH Task Execution Service (TES v1.1.0) for Kubernetes. Workflow engines send it tasks over a REST API; it runs each task as one Pod with its own volume. One container image (plus one for migrations), one PostgreSQL schema, no broker.

  • Your image and command run as submitted. Nothing is wrapped or injected (except a /bin/sh wrapper for executors that set ignore_error).
  • Inputs are staged onto the task's volume, outputs are uploaded from it, and the volume is deleted with the task.
  • Callers authenticate with tokens from your identity provider, or with API keys for long workflow runs.
TES clientPoiesis APITaskPod · one per tasktifstage inexec-0…Nyour imagestofstage outackmaintrec · native sidecarmarks INITIALIZING, RUNNING; writes logsPostgreSQLTCtlleader-electedHTTPsubmits Jobreads / writesstates + logspolls Podsterminal state

Components ​

ComponentRole
APIFastAPI service exposing TES under /ga4gh/tes/v1. No scheduler, no reconciler.
TaskPodOne Pod per task: trec (sidecar), then tif, exec-0..N, tof, ack. tif and tof only if the task has inputs or outputs.
TRecIn-pod recorder. Watches the surrounding Pod, marks the task INITIALIZING, then RUNNING when the first executor starts, and writes executor logs.
TCtlLeader-elected Deployment. Writes every final state from the Pod's phase, including OOMKill, eviction, node loss, pending timeouts and vanished Pods.
PostgresRelational store. Conditional UPDATEs make concurrent state writes (TRec, TCtl, a cancel) race-correct.

See Tasks and TaskPods and Architecture for how they fit together.

Storage protocols ​

Inputs: s3://, http://, https:// and inline content. Outputs: s3://, http:// and https://. file:// works for both when the operator allows it. See Storage and staging.

What's yours and what's ours ​

YouPoiesis
Executor image and commandPod composition, scheduling, lifecycle
Input and output URLsStaging and upload
TesResourcesVolume sizing, container resources
Kubernetes 1.29+, Postgres, object storage, an OIDC issuerAPI, recorder, controller, schema

Pick a client ​

  • curl: the fastest way to learn the API. The Quickstart has working examples.
  • A workflow engine: Nextflow, Snakemake, Sprocket and others. See Workflow engines.
  • Your own code: the TES API is the contract; py-tes is a ready Python client.

What Poiesis doesn't do ​

  • Schedule across clusters. One TaskPod, one cluster.
  • Mutate your executor image. Commands run exactly as submitted.
  • Cache anything. Each submission is independent.
  • Run its own login flow. Poiesis is a resource server, not an OAuth client. It validates tokens minted by an external identity provider (Keycloak, Auth0, Dex, …). See Authentication.