Apache Airflow 3 Explained: What Changed From Airflow 2 and What to Learn
Airflow 2 reached end of life on 22 April 2026, so Airflow 3 is the version to learn and run in 2027. This guide explains what changed in Airflow 3.0 and the 3.1, 3.2 and 3.3 releases, from the Task SDK and DAG versioning to assets, human-in-the-loop tasks and asset partitioning, with a migration checklist.
GloryTecks Team
Training & Placement Faculty
Airflow 3 is the version of Apache Airflow to learn and run in 2027. Airflow 2 entered limited maintenance in October 2025 and reached end of life on 22 April 2026, which means it no longer receives fixes of any kind. Airflow 3.0, released in April 2025, was the biggest change in the project's history: a new user interface, DAG versioning, scheduler-managed backfills, a client-server architecture through the Task SDK, and data assets with event-driven scheduling. Releases 3.1 to 3.3 then added human-in-the-loop tasks, asset partitioning, task state and pluggable retry policies. Your Airflow 2 knowledge still helps, but several core ideas now work differently.
Why the move matters now
The Airflow project's supported versions page lists 2.11.2 as the last 2.x release and 22 April 2026 as its end-of-life date. End-of-life versions get no security patches or bug fixes. Some managed providers offer longer support for their own customers, but for open-source deployments the message is clear: plan the migration if you have not already, and learn Airflow 3 if you are starting out.
What changed in Airflow 3.0
The project's Airflow 3 announcement describes the main changes:
| Change | What it means in practice |
|---|---|
| New React-based UI | A redesigned interface that moves between task-oriented and asset-oriented views |
| DAG versioning | A run completes using the DAG version it started with, even if a new version is deployed mid-run, and history shows which version ran |
| Scheduler-managed backfills | Backfills run inside the scheduler, with better control and visibility than the old command-line approach |
| Task Execution Interface and Task SDK | Tasks talk to an API server instead of directly to the metadata database, a client-server design with a stronger security model and execution in other environments |
| Edge Executor | Runs tasks on edge devices and remote locations, built on the Task Execution Interface |
| Data assets | Datasets evolved into assets, the basis for data-aware scheduling |
| External event-driven scheduling | DAGs can react to events from messaging systems and to assets updated outside Airflow |
The architectural change is the one that matters most for operators. Because tasks no longer reach into the metadata database, workers can run in more isolated environments, and Airflow can eventually run tasks written in other languages.
What 3.1, 3.2 and 3.3 added
- Airflow 3.1 (September 2025): human-in-the-loop tasks that pause and wait for a person's input in the UI, a React plugin system, deadline alerts (experimental) and a UI available in 17 languages, according to the 3.1.0 announcement.
- Airflow 3.2 (April 2026): asset partitioning, so only the downstream work for the changed partition runs, plus multi-team deployments (experimental), synchronous deadline alert callbacks and async callables in the PythonOperator, as described in the 3.2.0 announcement.
- Airflow 3.3 (July 2026): new partition mappers for rolling up and fanning out partitions, a timetable that assigns partition keys at run time, a task and asset state store that survives retries and runs, pluggable retry policies, and an experimental coordinator layer with Java and Go task SDKs, according to the 3.3.0 release notes.
Airflow 2 concepts and their Airflow 3 equivalents
| Airflow 2 | Airflow 3 | What to do |
|---|---|---|
| Datasets | Assets | Rename imports and references; use assets for data-aware scheduling |
| Tasks reading the metadata database directly | Tasks use the Task SDK and the API server | Replace direct database access with supported APIs |
| Backfills from the command line | Scheduler-managed backfills, visible in the UI | Run and monitor backfills through the new mechanism |
| A DAG run picking up mid-run code changes | DAG versioning, runs finish on the version they started with | Deploy freely; review version history after incidents |
| Ad hoc approval steps built with sensors | Human-in-the-loop tasks (from 3.1) | Use built-in approval tasks where people must decide |
Practising Airflow 3 locally
The quickest way to practise is the official Docker Compose setup described in the Airflow documentation, which starts the scheduler, API server, workers and a metadata database on your laptop. Build two or three DAGs that mirror real work: an ingestion DAG that publishes an asset, a transformation DAG scheduled on that asset, and a monthly report DAG you can backfill. Then break things on purpose, such as a failing task, a slow API or a bad deployment, and practise recovering with retries, alerts and the UI. That experience is exactly what interviewers probe.
Migrating from Airflow 2: a checklist
- Upgrade to the last 2.x release first. Its deprecation warnings point to most of the changes you need.
- Inventory your DAGs, plugins and providers. Note anything that reads the metadata database directly; that pattern no longer works for tasks in Airflow 3.
- Replace removed or changed features, such as old dataset code that now uses assets.
- Update provider packages to versions that support Airflow 3.
- Test in a staging environment with real DAGs, including backfills and failure scenarios.
- Migrate in waves, starting with low-risk DAGs, and keep the old environment until critical pipelines have run cleanly for a few cycles.
What to learn in Airflow 3 for 2027
| Order | Topic | Why it matters |
|---|---|---|
| 1 | DAGs, tasks, operators and the TaskFlow API | The everyday building blocks |
| 2 | Scheduling, timetables, retries and alerts | Reliable unattended pipelines |
| 3 | Assets and data-aware scheduling | Pipelines that run when data arrives, not on a guess |
| 4 | DAG versioning and backfills | Safe changes and historical reprocessing |
| 5 | Task SDK and deployment architecture | How tasks run, securely and at scale |
| 6 | Human-in-the-loop, partitioning and deadline alerts | The newer patterns teams are adopting |
Keep one principle from Airflow 2: Airflow orchestrates work, it should not do the heavy lifting. Push transformations into Spark, SQL or your warehouse and keep tasks thin. Our comparison of Airflow, Dagster and Prefect covers the alternatives, and the Airflow interview questions show what interviewers ask.
Learning Airflow at GloryTecks
The two-month Apache Airflow course covers orchestration concepts, environment setup with Docker, Airflow's architecture, writing DAGs, scheduling and backfills, passing data, dynamic workflows, sensors and deferrable operators, assets, integrations with databases, cloud services, Spark and dbt, and reliability through retries, alerts and monitoring. The Data Engineering course places Airflow inside a complete pipeline. Both run in classroom batches in Ameerpet and live online.
Is Airflow 2 still supported?
What is the biggest change in Airflow 3?
What are assets in Airflow 3?
What is the latest version of Airflow?
How hard is it to migrate from Airflow 2 to Airflow 3?
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master Data Engineering?
Learn Data Engineering at GloryTecks in Hyderabad, with trainers who have industry experience, real-time projects and placement assistance.
Related articles
Databricks vs Snowflake vs Microsoft Fabric: Which to Learn for 2027
Databricks, Snowflake and Microsoft Fabric are the data platforms data engineers in India meet most often. They now overlap heavily but come from different starting points and suit different teams. Here is how each works, where each fits, their certifications and which to learn first for 2027.
Apache Iceberg v3: What Data Engineers Should Know for 2027
Apache Iceberg's format version 3 adds deletion vectors, row lineage, default column values and new types such as variant, geospatial and nanosecond timestamps, and in 2026 Snowflake and AWS made it generally available. What v3 changes, why it matters, which engines support it and how to upgrade a table safely.
Data Engineering Roadmap 2027: Lakehouse, Streaming and AI-Ready Data
A data engineering roadmap for 2027 in seven stages: SQL and modelling, Python, one cloud, Spark and the lakehouse, Airflow 3, streaming and change data capture, and data quality, governance and AI-ready data. Includes the 2026 platform changes, a nine-month plan and three portfolio projects.