README.md

Sample

This sample is for experimenting with the Airbyte Module (ABM) locally, without Kubernetes.

The server is configured to obtain a CSV file from https://people.sc.fsu.edu/~jburkardt/data/csv/letter_frequency.csv

The Airbyte server is both an HTTP server (on port 8080) and an Arrow Flight server (on port 8081)

Steps

1. Install python dependencies

   pipenv install

2. Run the server with

   pipenv run server --config sample/sample.yaml  --workdir /tmp

3. Read the letter_frequency dataset from HTTP server:

   curl localhost:8080/letter_frequency

4. Read the letter_frequency dataset from the arrow-flight server:

   pipenv run python sample/sample.py