Course page
Log in

You are watching a free preview. Enroll to open all 27 lessons, track progress and ask the course tutor.

Enroll for $69.99

Section 1: Airflow 3 essentials

Your first DAG with the TaskFlow API

Article 12 min Lesson 2 of 27 Preview

In Airflow 3 the recommended way to write DAGs is the TaskFlow API, imported from the Task SDK. Decorators turn plain Python functions into tasks, and return values are passed between tasks automatically using XCom.

from datetime import datetime
from airflow.sdk import dag, task

@dag(schedule="@daily", start_date=datetime(2026, 1, 1), catchup=False, tags=["demo"])
def daily_rates():

    @task
    def extract() -> list[dict]:
        # In the real pipeline this calls an HTTP API
        return [{"currency": "EUR", "rate": 1.0}, {"currency": "USD", "rate": 1.09}]

    @task
    def transform(rows: list[dict]) -> list[dict]:
        return [r for r in rows if r["rate"] > 0]

    @task
    def load(rows: list[dict]) -> None:
        print(f"Loading {len(rows)} rows")

    load(transform(extract()))

daily_rates()

Read it from the bottom. Calling load(transform(extract())) does not run anything immediately. It builds the graph: extract, then transform, then load. Airflow runs the functions later, on workers, when the schedule says so.

Things to notice

  • The DAG id comes from the function name, daily_rates. Keep ids stable, because history is stored against them.
  • start_date is a fixed date, never datetime.now(). A moving start date confuses the scheduler.
  • schedule accepts presets such as @daily, cron strings such as "0 6 * * *", timedeltas, or assets, which we cover later.
  • The function at the bottom must be called so Airflow can discover the DAG.

Keep XCom small

Return values are stored as XCom. That is fine for a list of file paths or a row count. It is the wrong place for a 2 GB data frame. A good rule: pass references (paths, table names, partition keys) between tasks, and keep the data itself in storage built for it.

Drop this file into the dags/ folder of the local project and it appears in the UI after the DAG processor parses it.

← → switch lessons