working with script files is here

Author: pavanjava

README.md

@@ -35,6 +35,7 @@ qql> SEARCH notes SIMILAR TO 'vector databases' LIMIT 5 USING HYBRID RERANK
 - [The QQL Shell](#the-qql-shell)
 - [All QQL Operations](#all-qql-operations)
   - [INSERT — add a point](#insert--add-a-point)
+  - [INSERT BULK — batch insert](#insert-bulk--batch-insert-multiple-points)
   - [SEARCH — find similar points](#search--find-similar-points)
   - [Query-Time Search Params (`EXACT`, `WITH`)](#query-time-search-params-exact-with)
   - [WHERE Clause Filters](#where-clause-filters)
@@ -44,6 +45,9 @@ qql> SEARCH notes SIMILAR TO 'vector databases' LIMIT 5 USING HYBRID RERANK
   - [CREATE COLLECTION — create a collection](#create-collection--create-a-collection)
   - [DROP COLLECTION — delete a collection](#drop-collection--delete-a-collection)
   - [DELETE — remove a point](#delete--remove-a-point)
+- [Script Files](#script-files)
+  - [EXECUTE — run a script file](#execute--run-a-qql-script-file)
+  - [DUMP COLLECTION — export to script](#dump-collection--export-collection-to-a-qql-script-file)
 - [Embedding Models](#embedding-models)
 - [Value Types in Dictionaries](#value-types-in-dictionaries)
 - [Configuration File](#configuration-file)
@@ -838,6 +842,158 @@ To find a point's ID, run a SEARCH first and copy the ID from the results table.
 
 ---
 
+## Script Files
+
+QQL supports reading from and writing to `.qql` script files, making it easy to automate bulk operations, seed databases, and back up collections.
+
+---
+
+### EXECUTE — run a .qql script file
+
+Execute a file containing multiple QQL statements in sequence. Each statement is parsed and executed in order. `--` comments are stripped before parsing.
+
+**CLI usage:**
+```bash
+qql execute /path/to/script.qql
+
+# Stop on first error instead of continuing through all statements
+qql execute /path/to/script.qql --stop-on-error
+```
+
+**In-shell usage (inside the QQL REPL):**
+```
+qql> EXECUTE /path/to/script.qql
+qql> \e /path/to/script.qql
+```
+
+**Script format:**
+
+```sql
+-- This is a comment — the entire line is ignored
+-- ============================================================
+--  QQL Script — populate articles collection
+-- ============================================================
+
+-- Step 1: create the collection
+CREATE COLLECTION articles
+
+-- Step 2: bulk insert records
+INSERT BULK INTO COLLECTION articles VALUES [
+  {'text': 'Neural networks learn representations', 'year': 2023},
+  {'text': 'Attention mechanisms in transformers',  'year': 2024}
+]
+
+-- Step 3: verify
+SHOW COLLECTIONS
+```
+
+**Rules:**
+- `--` to end-of-line is a comment and is ignored (inline or full-line)
+- Statements can span multiple lines (e.g. `INSERT BULK ... VALUES [...]`)
+- Blank lines between statements are ignored
+- By default all statements run even if one fails; use `--stop-on-error` to halt early
+
+**Example output:**
+```
+Executing: /path/to/script.qql
+
+[1/3] CREATE COLLECTION articles
+  ✓ Collection 'articles' created (384-dimensional vectors, cosine distance)
+[2/3] INSERT BULK INTO COLLECTION articles VALUES [ …
+  ✓ Inserted 2 points
+[3/3] SHOW COLLECTIONS
+  ✓ 1 collection(s) found
+
+Done. 3/3 statement(s) succeeded.
+```
+
+---
+
+### DUMP COLLECTION — export collection to a .qql script file
+
+Export every point in a collection to a `.qql` script file. The generated file is valid QQL — it can be re-imported with `qql execute` to restore or migrate the collection. Points are written in batches of 50 as `INSERT BULK` statements.
+
+**CLI usage:**
+```bash
+qql dump <collection_name> <output.qql>
+```
+
+**In-shell usage (inside the QQL REPL):**
+```
+qql> DUMP COLLECTION <name> <output.qql>
+```
+
+**Example:**
+```bash
+qql dump medical_records /tmp/medical_records.qql
+```
+
+```
+Dumping: 'medical_records'  →  /tmp/medical_records.qql
+
+  Collection type : hybrid (dense + sparse)
+  Points          : 41
+  Batches         : 1  (50 points/batch)
+
+  [1/1] wrote 41 point(s)
+
+Done. 41 point(s) written.
+```
+
+**Generated file structure:**
+```sql
+-- ============================================================
+-- QQL Dump — collection: medical_records
+-- Generated : 2026-04-19 14:32:11
+-- Points    : 41
+-- Type      : hybrid (dense + sparse)
+-- Note      : Re-importing re-embeds all text using the
+--             configured model (see: qql connect).
+-- ============================================================
+
+CREATE COLLECTION medical_records HYBRID
+
+-- Batch 1 / 1  (records 1–41)
+INSERT BULK INTO COLLECTION medical_records VALUES [
+  {
+    'text': 'Alzheimers disease is characterized by...',
+    'title': 'Alzheimers Disease Overview',
+    'department': 'neurology',
+    'year': 2023,
+    'peer_reviewed': true
+  },
+  ...
+] USING HYBRID
+
+-- ============================================================
+-- End of dump
+-- Written : 41
+-- Skipped : 0  (no 'text' field)
+-- ============================================================
+```
+
+**Round-trip workflow — backup and restore:**
+```bash
+# 1. Dump the collection
+qql dump medical_records backup.qql
+
+# 2. Drop it
+qql> DROP COLLECTION medical_records
+
+# 3. Restore from the dump
+qql execute backup.qql
+```
+
+**Rules and notes:**
+- Points without a `'text'` payload field are **skipped** (counted in the footer comment).
+- Hybrid collections produce `CREATE COLLECTION <name> HYBRID` and `INSERT BULK ... USING HYBRID` statements.
+- Dense collections produce plain `CREATE COLLECTION <name>` and `INSERT BULK` statements.
+- All payload value types are preserved: strings, integers, floats, booleans (`true`/`false`), `null`, lists, and nested dicts.
+- Re-importing re-embeds all text using your currently configured model — use the same model as the original collection to preserve semantic accuracy.
+- Parent directories of the output path are created automatically.
+
+---
+
 ## Embedding Models
 
 QQL uses [Fastembed](https://github.com/qdrant/fastembed) to convert text into vectors locally — no external API call is needed.

src/qql/cli.py

@@ -56,6 +56,15 @@
   [yellow]DELETE FROM[/yellow] <name> [yellow]WHERE id =[/yellow] '<id>'
       Delete a point by its ID.
 
+Script files (in-shell):
+  [yellow]EXECUTE[/yellow] <path>   or   [yellow]\\e[/yellow] <path>
+      Run a .qql script file. Statements are executed in order.
+      Lines starting with [yellow]--[/yellow] are treated as comments and ignored.
+
+  [yellow]DUMP[/yellow] <name> <output.qql>   or   [yellow]DUMP COLLECTION[/yellow] <name> <output.qql>
+      Export all points in a collection to a .qql script file.
+      The file can be re-imported with EXECUTE.
+
 Keyboard shortcuts:
   ← → arrows   move cursor within the current line
   ↑ ↓ arrows   scroll through command history
@@ -119,6 +128,109 @@ def disconnect() -> None:
     console.print("Disconnected. Config removed.")
 
 
+# ── execute ────────────────────────────────────────────────────────────────────
+
+@main.command()
+@click.argument("file", type=click.Path(exists=True, readable=True))
+@click.option(
+    "--stop-on-error",
+    is_flag=True,
+    default=False,
+    help="Halt execution on the first statement error (default: continue all).",
+)
+def execute(file: str, stop_on_error: bool) -> None:
+    """Execute a .qql script file against the connected Qdrant instance.
+
+    Lines beginning with -- are treated as comments and skipped.
+    Each QQL statement is executed in order and its result is printed.
+    """
+    from qdrant_client import QdrantClient
+
+    cfg = load_config()
+    if cfg is None:
+        err_console.print(
+            "[bold red]Not connected.[/bold red] "
+            "Run: [bold]qql connect --url <url>[/bold]"
+        )
+        sys.exit(1)
+
+    try:
+        client = QdrantClient(url=cfg.url, api_key=cfg.secret)
+        client.get_collections()
+    except Exception as e:
+        err_console.print(f"[bold red]Connection failed:[/bold red] {e}")
+        sys.exit(1)
+
+    from .executor import Executor
+    from .script import run_script
+
+    executor = Executor(client, cfg)
+    console.print(f"[bold cyan]Executing:[/bold cyan] {file}\n")
+
+    ok, fail = run_script(file, executor, console, err_console, stop_on_error)
+    total = ok + fail
+
+    if fail == 0:
+        console.print(
+            f"\n[bold green]Done.[/bold green] "
+            f"{total}/{total} statement(s) succeeded."
+        )
+    else:
+        console.print(
+            f"\n[bold yellow]Done.[/bold yellow] "
+            f"{ok}/{total} succeeded, [bold red]{fail} failed[/bold red]."
+        )
+        sys.exit(1)
+
+
+# ── dump ───────────────────────────────────────────────────────────────────────
+
+@main.command()
+@click.argument("collection")
+@click.argument("output", type=click.Path())
+def dump(collection: str, output: str) -> None:
+    """Dump a collection to a .qql script file.
+
+    OUTPUT is the path for the generated .qql file.
+    The file contains CREATE COLLECTION + INSERT BULK statements and can be
+    re-imported with: qql execute <output>
+    """
+    from qdrant_client import QdrantClient
+
+    cfg = load_config()
+    if cfg is None:
+        err_console.print(
+            "[bold red]Not connected.[/bold red] "
+            "Run: [bold]qql connect --url <url>[/bold]"
+        )
+        sys.exit(1)
+
+    try:
+        client = QdrantClient(url=cfg.url, api_key=cfg.secret)
+        client.get_collections()
+    except Exception as e:
+        err_console.print(f"[bold red]Connection failed:[/bold red] {e}")
+        sys.exit(1)
+
+    from .dumper import dump_collection
+
+    console.print(
+        f"[bold cyan]Dumping:[/bold cyan] '{collection}'  →  {output}\n"
+    )
+    written, skipped = dump_collection(collection, output, client, console, err_console)
+
+    if written == 0 and skipped == 0:
+        # collection not found — error already printed by dump_collection
+        sys.exit(1)
+
+    console.print(
+        f"\n[bold green]Done.[/bold green] "
+        f"{written} point(s) written"
+        + (f", [yellow]{skipped} skipped[/yellow] (no 'text' field)" if skipped else "")
+        + f"."
+    )
+
+
 # ── REPL ───────────────────────────────────────────────────────────────────────
 
 def _launch_repl(cfg: QQLConfig) -> None:
@@ -161,6 +273,62 @@ def _launch_repl(cfg: QQLConfig) -> None:
             console.print(HELP_TEXT)
             continue
 
+        # ── EXECUTE <path> / \e <path> — run a .qql script file ──────────
+        if low.startswith("execute ") or low.startswith("\\e "):
+            script_path = query.split(None, 1)[1].strip()
+            from .script import run_script
+            ok, fail = run_script(script_path, executor, console, err_console)
+            total = ok + fail
+            if fail == 0:
+                console.print(
+                    f"[bold green]Done.[/bold green] "
+                    f"{total}/{total} statement(s) succeeded."
+                )
+            else:
+                console.print(
+                    f"[bold yellow]Done.[/bold yellow] "
+                    f"{ok}/{total} succeeded, [bold red]{fail} failed[/bold red]."
+                )
+            continue
+
+        # ── DUMP [COLLECTION] <name> <file> — export collection to .qql ──
+        # Accepts both:
+        #   DUMP COLLECTION <name> <output.qql>
+        #   DUMP <name> <output.qql>
+        if low.startswith("dump "):
+            parts = query.split(None, 3)  # up to 4 tokens
+            if len(parts) >= 2 and parts[1].lower() == "collection":
+                # DUMP COLLECTION <name> <file>
+                if len(parts) < 4:
+                    err_console.print(
+                        "[bold red]Usage:[/bold red] DUMP COLLECTION <name> <output.qql>"
+                    )
+                    continue
+                coll_name, out_path = parts[2], parts[3]
+            else:
+                # DUMP <name> <file>
+                if len(parts) < 3:
+                    err_console.print(
+                        "[bold red]Usage:[/bold red] DUMP <name> <output.qql>"
+                    )
+                    continue
+                coll_name, out_path = parts[1], parts[2]
+            from .dumper import dump_collection
+            console.print(
+                f"[bold cyan]Dumping:[/bold cyan] '{coll_name}'  →  {out_path}\n"
+            )
+            written, skipped = dump_collection(
+                coll_name, out_path, client, console, err_console
+            )
+            if written > 0 or skipped == 0:
+                console.print(
+                    f"[bold green]Done.[/bold green] "
+                    f"{written} point(s) written"
+                    + (f", [yellow]{skipped} skipped[/yellow] (no 'text' field)" if skipped else "")
+                    + "."
+                )
+            continue
+
         _run_and_print(executor, query)