Skip to content
This repository was archived by the owner on Oct 18, 2023. It is now read-only.
This repository was archived by the owner on Oct 18, 2023. It is now read-only.

Unexpected disk usage amplification when DB created from a seed file #583

@avinassh

Description

@avinassh

For databases created from seed files, I have noticed higher disk usage than expected. The disk space usage is 4x for primary and 3x for replicas. (the usage peaked at 4.5x and 3.5x respectively)

My seed file is 1.7GB. The disk space used in primary was 6.8GB (while peak at 8.2GB).

To recreate, you may run following commands to generate a test database:

$ sqlite3 big.db

PRAGMA journal_mode=WAL;

CREATE TABLE test (
  id INTEGER PRIMARY KEY NOT NULL, 
  x REAL NOT NULL,
  y REAL NOT NULL,
  z REAL NOT NULL
);

INSERT INTO test
    WITH RECURSIVE
      cnt( id, x, y, z) AS (
      VALUES(1 , random(), random(), random()) UNION ALL 
      SELECT id+1,random(),random(), random() FROM cnt WHERE ID<50000000)
    select * from cnt;

related: This slack thread has some internal details tested on Turso cloud

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions