MIT licensed · v3.2 · Python 3.10+

Composable data pipelines that fit in your head

Halyard is a small set of packages for defining, testing, and scheduling pipelines as plain Python functions. No DSL, no scheduler lock-in, no YAML pyramid.

Install in 30 seconds

The packages

halyard-core

halyard-core

stable

v3.2.1 · 1.2 MB

Task graph, typed inputs and outputs, retries, and the local runner. The only package most projects need.

halyard-io

halyard-io

stable

v3.2.1 · 640 kB

Readers and writers for Parquet, Postgres, S3, BigQuery, and Delta, all sharing one connection abstraction.

halyard-test

halyard-test

stable

v3.1.4 · 210 kB

Golden-file fixtures and a fake clock so a pipeline test runs in milliseconds without touching a warehouse.

halyard-scheduler

halyard-scheduler

beta

v0.9.0 · 780 kB

A single-binary scheduler with cron and event triggers. Optional — Airflow and Dagster adapters ship too.

halyard-viz

halyard-viz

experimental

v0.4.2 · 1.8 MB

Renders a pipeline graph to SVG or a local web view, with run history overlaid on each node.

Quick start

From an empty directory to a running pipeline.

  1. Install

    pip install halyard-core halyard-io — no compiler, no system dependencies, wheels for Linux, macOS, and Windows.

  2. Write a task

    Decorate a plain function with @task. Type hints become the contract; Halyard checks them before anything runs.

  3. Compose a pipeline

    Call tasks from a @pipeline function. The graph is inferred from the calls — there is nothing else to declare.

  4. Run and test it

    halyard run daily_sales runs it locally; pytest with halyard-test runs the same graph against fixtures in under a second.

The project, in numbers

18.4k

GitHub stars

312

Contributors

1.1M

Monthly downloads

6

Median days to first review

Maintainers

Halyard is volunteer-run. Three of us are funded part-time by the Open Data Fund.

Sanne de Vries

Sanne de Vries

Lead maintainer · Utrecht

Wrote the original task graph in 2021. Reviews most core PRs.

GitHub
Lucas Amorim

Lucas Amorim

halyard-io · São Paulo

Owns the connector layer and the compatibility matrix.

GitHub
Wei Tan

Wei Tan

Docs & scheduler · Singapore

Writes the guides and is slowly getting the scheduler to 1.0.

GitHub

Release notes

3.2 — Incremental runs
minor2 June 2026

3.2 — Incremental runs

Tasks can now declare a watermark, so a rerun only processes what changed. Plus a 40% faster graph resolver.

3.1 — Typed connection pools
minor14 April 2026

3.1 — Typed connection pools

halyard-io connections are now typed and pooled per pipeline run, which fixes the long-standing Postgres leak on retries.

3.0 — Goodbye YAML
major3 February 2026

3.0 — Goodbye YAML

The declarative config layer is gone. Pipelines are Python functions and nothing else. Migration guide included.

Contributing

Where do I start?

Issues tagged "good first issue" are kept stocked and each one names a maintainer who will review it. Say hello in the issue before you start so nobody duplicates work.

What is the review turnaround?

Median six days. If a pull request has had no reply after two weeks, ping the weekly triage thread and we will pick it up.

Do you accept new connectors?

Yes, into halyard-io, provided the connector has an integration test that runs against a container and a named person willing to maintain it.

How is the project funded?

Open Data Fund grants and GitHub Sponsors. Nobody is employed by a company that sells a hosted Halyard, and we intend to keep it that way.

Star it, or better, break it

Bug reports from real pipelines are worth more to us than stars.

Open an issue