Skip to content

data-layout.md

Latest commit

 

History

History
42 lines (29 loc) · 2.48 KB

data-layout.md

File metadata and controls

42 lines (29 loc) · 2.48 KB

Data layout

All persistent data lives under a single data root. By default that is data/ in the current working directory. You can override it with the MASS_DATA_DIR environment variable or the global CLI option -d, --data-dir <path>.

Paths under the data root

Path Description
personas/<id>/<id>.json One JSON file per persona (metadata, details, and optional persona narrative in details.persona).
personas/<id>/<id>_memories.json Persona memories (extracted from conversations). Optional; created when memory extraction runs.
personas/<id>/<id>_state.json Persona runtime state. Optional; used for simulation and stateful chat.
cohorts/<id>.json One JSON file per cohort. The CLI discovers cohorts by listing this directory.
workspaces/<id>.json One JSON file per workspace (conversation state).
reports/<id>.json One JSON file per report.
report_responses/ Report response data (e.g. per workspace).
tasks/ Background task state.
executions/ Execution state.

Optional index files

At the data root you may also see:

  • index.json – Persona index (optional; used for name deduplication and lookups in some flows).
  • index_extended.json – Extended persona index (optional).
  • connections.json – Persona connections (optional).

These are not required for basic CLI use (cohort create, persona create/list/delete, chat, report generate/list/show/delete). The app will create and use them as needed for certain operations.

Stats files (optional)

You may see files such as stats.json, stats_extended.json, stats.australia.json, stats_coherence.australia.json, and similar. These are not used by the Mass CLI; they are aggregate summaries (e.g. demographic counts, validation averages) produced by external or one-off tooling for human review.

  • You do not need to keep all of them. Keep only what you use for analysis.
  • The same information can be shown in markdown for easier reading in the repo. See Data stats example for a markdown format; if your stats are generated by a script, consider outputting .md as well as or instead of .json.

Summary

  • Persona and cohort data: personas/, cohorts/.
  • Conversation and reports: workspaces/, reports/, report_responses/.
  • Internal state: tasks/, executions/.

See Creating cohorts and personas for the structure of persona files and optional sidecars.