Apache Iceberg v3: What Data Engineers Should Know for 2027
Apache Iceberg's format version 3 adds deletion vectors, row lineage, default column values and new types such as variant, geospatial and nanosecond timestamps, and in 2026 Snowflake and AWS made it generally available. What v3 changes, why it matters, which engines support it and how to upgrade a table safely.
GloryTecks Team
Training & Placement Faculty
Apache Iceberg's format version 3 is the table format version most data engineers will work with in 2027. It adds four things that matter in daily work: deletion vectors, which make row-level updates and deletes far cheaper; row lineage, which tracks individual rows across changes and makes incremental processing easier; default column values, which make schema changes smoother; and new data types, including variant for semi-structured data, geometry and geography, and nanosecond timestamps. During 2026 major platforms made v3 generally available, so it is no longer an experiment. Upgrading a table is simple but one-way, which is why it needs a little planning.
What a table format does, briefly
A data lake stores files, usually Parquet, in object storage. A table format such as Iceberg adds a metadata layer on top: which files belong to the table, the schema, partitions, snapshots for time travel, and the changes made by each write. That metadata is what lets many engines, such as Spark, Trino, Flink, Snowflake and cloud query services, read and write the same table safely. Our comparison of Iceberg, Delta Lake and Hudi explains how the main formats differ.
What v3 adds
The Iceberg table specification describes version 3 as extending data types and metadata structures with new capabilities:
| Feature | What it does | Why it matters |
|---|---|---|
| Deletion vectors | Record deleted rows of a data file in a compact bitmap | Updates and deletes no longer pile up many small delete files |
| Row lineage | Gives rows identifiers and tracks when they were last changed | Incremental reads and change tracking become simpler and portable across engines |
| Default values | A column can have an initial default for old rows and a write default for new ones | Adding columns no longer means rewriting or leaving awkward nulls |
| Variant type | Stores semi-structured data such as JSON without a fixed schema | Event and API data can land without constant schema changes |
| Geometry and geography types | Native geospatial columns | Location data without workarounds |
| Nanosecond timestamps | Higher-precision time values | Financial, telemetry and logging data keep full precision |
Deletion vectors
In version 2, row-level deletes were usually written as positional delete files listing which rows to ignore. Tables with frequent updates, such as those fed by change data capture, could accumulate many delete files, slowing reads until compaction ran. In v3 a data file's deleted rows are tracked in a single compact bitmap, the deletion vector. Reads stay fast and maintenance is lighter.
Row lineage
Row lineage records an identifier for each row and the snapshot in which it last changed. Downstream jobs can then ask which rows changed since their last run, without each engine inventing its own mechanism. For change data capture and incremental pipelines this is a significant simplification.
Who supports v3 in 2026
- Snowflake announced general availability of Iceberg v3 support on 7 May 2026, and its release notes later added generally available v3 table upgrades and expression defaults in October 2026.
- AWS documents Iceberg v3 support for deletion vectors, row lineage and column default values with Apache Spark on Amazon EMR, AWS Glue ETL, SageMaker Unified Studio notebooks and the Glue Data Catalog, and says Amazon S3 Tables support all the data types v3 introduces.
- Apache Spark reads and writes v3 tables through the Iceberg runtime; the latest Iceberg Java release listed on the Iceberg releases page is 1.12.0, with runtimes for Spark 4.0 and 4.1.
Other engines are adding support at different speeds, and support for a feature can differ between reading and writing. Check each engine's documentation for the specific features you plan to use.
Variant data in practice
The variant type is likely to be the most visible change for analysts. AWS's documentation explains that with variant you can write semi-structured data such as JSON directly into Iceberg tables without defining a schema in advance, and that v3-compatible engines shred the data into hidden columns as they write, so queries on common fields stay efficient. For event data, API responses and logs, that removes a whole category of schema-change work.
Upgrading a table from v2 to v3
AWS's documentation states two facts that shape any upgrade: existing v2 tables can be upgraded to v3 without rewriting data, and the upgrade is one-way, so a v3 table cannot be downgraded to v2 through standard operations. A safe approach:
- List every engine and job that writes to the table. All writers must support v3 before you upgrade.
- Check readers too, including BI tools and query services, for the v3 features you plan to use.
- Upgrade a non-critical table first and run your normal workloads against it for a few cycles.
- Expect no history backfill for row lineage: lineage tracking starts from the upgrade.
- Upgrade the rest in waves, and record the date of each upgrade in your data catalog.
What about version 4?
The specification already contains a section on version 4, which restructures metadata for performance. It notes that version 4 is under active development and has not been formally adopted. For 2027 planning, v3 is the version to build on.
A worked example: a CDC table on v2 and v3
Picture a customer table fed by change data capture from an operational database, with thousands of updates an hour. On a v2 table, each batch of updates writes new data files plus positional delete files, and the delete files pile up until a compaction job rewrites the table; readers pay for every uncompacted delete file in between. On a v3 table, each data file carries a single deletion vector that is replaced as rows change, so readers check one compact bitmap per file. Row lineage then lets the downstream job that builds a customer dimension read only the rows that changed since its last run, instead of comparing the whole table.
Skills to build for 2027
- Create, query and maintain Iceberg tables with Spark: partition evolution, snapshots, time travel and compaction.
- Understand catalogs, because table formats are only useful with a catalog that engines share.
- Practise change data capture into v3 tables and incremental reads.
- Learn the cost side: file sizes, compaction schedules and how deletes affect reads.
For the wider picture, see our guide to data lakes, warehouses and lakehouses.
Learning open table formats at GloryTecks
The five-month Multi-Cloud Data Engineering course covers object storage and data lake design, Spark and PySpark, and table formats including Delta Lake and Apache Iceberg across Azure, AWS and Google Cloud. The two-month Snowflake course covers Snowflake's architecture, loading, semi-structured data, time travel and streams. Both run in classroom batches in Ameerpet and live online.
What is new in Apache Iceberg v3?
Can I upgrade an Iceberg table from v2 to v3?
Which platforms support Iceberg v3?
What are deletion vectors in Iceberg?
Should I learn Iceberg or Delta Lake in 2027?
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master Data Engineering?
Learn Data Engineering at GloryTecks in Hyderabad, with trainers who have industry experience, real-time projects and placement assistance.
Related articles
Databricks vs Snowflake vs Microsoft Fabric: Which to Learn for 2027
Databricks, Snowflake and Microsoft Fabric are the data platforms data engineers in India meet most often. They now overlap heavily but come from different starting points and suit different teams. Here is how each works, where each fits, their certifications and which to learn first for 2027.
Data Engineering Roadmap 2027: Lakehouse, Streaming and AI-Ready Data
A data engineering roadmap for 2027 in seven stages: SQL and modelling, Python, one cloud, Spark and the lakehouse, Airflow 3, streaming and change data capture, and data quality, governance and AI-ready data. Includes the 2026 platform changes, a nine-month plan and three portfolio projects.
Data Lake vs Data Warehouse vs Lakehouse
A clear comparison of data lakes, data warehouses and lakehouses: what each stores and when structure is applied, strengths and failure modes such as the data swamp, what a lakehouse adds through table formats and governance, a side-by-side table, and practical guidance on choosing or combining them.