Skip to content

Latest commit

 

History

History
25 lines (17 loc) · 722 Bytes

README.md

File metadata and controls

25 lines (17 loc) · 722 Bytes

Iceberg Spark demo

Demo of how to use Apache Iceberg table format from a Spark application.

Requirements

  • Docker Compose
  • Java >= 11
  • IDE like IntelliJ

Steps

Start both MinIO object storage and PostgreSQL server with docker-compose:

docker-compose up -d
  • Minio admin console will be accessible at http://127.0.0.1:9001. login: minioadmin. password: minioadmin.
  • PostgreSQL server will be available at 127.0.0.1:5432 (postgres/postgres). Database is iceberg_db.

Import the project in an IDE (project is based on Maven and Java >= 11) and run IcebergWithSpark.scala. You can edit and launch methods you want in this file.