sq

Convert and query JSON, JSONL, CSV, and SQLite with ease!

sq is a simple yet powerful command line tool for query and converting from and to SQLite, CSV, JSON, and JSONL files, heavily inspired by jq .

Features

• Convert between SQLite, CSV, JSON, and JSONL files.
• Query CSV, JSON, and JSONL using real SQLite queries.
• Allows for rapid scripting and conversion of data.
• Pipe in data or read from files.

Installation

Download a prebuilt binary from the releases page and add it to your PATH, or use go install to install the latest version.

go install github.com/chand1012/sq@latest

Examples

Query some orders from a CSV file. Column names for CSV files are converted to lower case and spaces are replaced with underscores.

$ sq -r orders.csv 'SELECT country FROM sq WHERE seller_amount > 20 NOT NULL;'
United States of America
United States of America
United States of America
United States of America
United States of America
United States of America
United Kingdom
Canada
United States of America
United States of America
United States of America
United States of America
United States of America
Canada
United States of America
Canada
Canada
Australia
United States of America
...

Download and query some JSONL datasets.

$ curl https://raw.githubusercontent.com/TimeSurgeLabs/llm-finetuning/4e934ce602f34f62f4d803c40cd1e7825d216192/data/fingpt-sentiment-1k.jsonl | sq 'SELECT * FROM sq WHERE output = "positive";' -f jsonl > positive.jsonl

You can even use it with jq !

$ curl https://api.gogopool.com/stakers | jq '.stakers' | sq -t stakers 'SELECT stakerAddr,avaxValidating FROM stakers WHERE avaxValidating > 0;' -f json > stakers.json