Iceberg Tables

Overview

Apache Iceberg is an open-source table format for massive analytic datasets. Snowflake supports Iceberg Tables, allowing users to query data stored externally in their own cloud storage using the Iceberg format, but with the performance and query semantics of native Snowflake tables. This is a new and important topic for the COF-C03 exam.

Key Concepts

The core benefit of Iceberg is interoperability. It prevents vendor lock-in by allowing multiple compute engines (Snowflake, Apache Spark, Trino, etc.) to securely read and write to the same single copy of data in a data lake.

Why Iceberg over standard External Tables?

While standard Snowflake External Tables are read-only and often suffer from slower performance, Iceberg Tables offer:

• Read & Write Capabilities (depending on the catalog setup).
• Better Performance: Iceberg's metadata allows Snowflake to perform micro-partition-like query pruning on external data.
• Time Travel & Schema Evolution: Native Iceberg features that map well to Snowflake's capabilities.

Catalog Integrations (The Two Modes)

To use Iceberg tables, Snowflake needs to know where the metadata (the catalog) is managed. There are two primary modes you need to know for the exam:

#### 1. Snowflake-Managed Catalog

• Snowflake manages the Iceberg metadata.
• Snowflake has full read and write access.
• Snowflake handles automated storage maintenance (compaction, snapshot expiration).
• Use Case: You want Snowflake's performance and ease of use, but you mandate that the physical Parquet files reside in your own AWS S3/Azure Blob/GCS bucket rather than Snowflake's internal storage.

#### 2. Externally Managed Catalog

• Another system manages the metadata (e.g., Polaris Catalog, AWS Glue, or a REST catalog). Polaris Catalog is Snowflake's open-source catalog for Apache Iceberg.
• Snowflake connects to this external catalog to read the table.
• Use Case: You have existing data pipelines writing to Iceberg via Spark or AWS Glue, and you want Snowflake to query that data seamlessly without moving it.

External Volume

To create an Iceberg table, you must first create an External Volume. This is an account-level object that securely connects Snowflake to your external cloud storage bucket.

SQL Examples

-- 1. Create an External Volume (points to your AWS S3 bucket)
CREATE EXTERNAL VOLUME my_s3_vol
   STORAGE_LOCATIONS = (
      (
         NAME = 'my-s3-us-west',
         STORAGE_PROVIDER = 'S3',
         STORAGE_BASE_URL = 's3://my-company-bucket/iceberg-data/',
         STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::123456789:role/my-role'
      )
   );

-- 2. Create an Iceberg Table (Snowflake-managed catalog)
CREATE ICEBERG TABLE my_iceberg_table (
    id INT,
    customer_name VARCHAR
)
    CATALOG = 'SNOWFLAKE'
    EXTERNAL_VOLUME = 'my_s3_vol'
    BASE_LOCATION = 'my_table_dir';

Exam Tips

⚠️ Crucial for COF-C03:

• Interoperability: This is the keyword for Iceberg. It allows multiple engines to hit the same storage.
• External Volume: Know that an External Volume is required to bridge Snowflake to the cloud storage where the Iceberg files live.
• Snowflake vs. External Managed: Understand the difference between Snowflake managing the catalog (read/write/maintenance) vs integrating with an external catalog like AWS Glue.

Key Takeaways

• Iceberg tables use the open-source Apache Iceberg format.
• Data resides in customer-owned cloud storage, not Snowflake's internal storage.
• Provides better performance and features (ACID, Time Travel) compared to legacy external tables.
• Requires an External Volume to configure storage access.