Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/python-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: python-ci

on:
push:
pull_request:

jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13"]
defaults:
run:
working-directory: packages/core-python
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: pip
cache-dependency-path: packages/core-python/pyproject.toml
- run: python -m pip install -e ".[dev]"
- run: python -m pytest
- run: python -m build
- run: python -m twine check dist/*
46 changes: 46 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: python-publish

on:
release:
types: [published]

jobs:
build:
runs-on: ubuntu-latest
defaults:
run:
working-directory: packages/core-python
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
cache: pip
cache-dependency-path: packages/core-python/pyproject.toml
- run: python -m pip install -e ".[dev]"
- run: python -m build
- run: python -m twine check dist/*
- uses: actions/upload-artifact@v4
with:
name: python-distributions
path: packages/core-python/dist/*
if-no-files-found: error

publish:
runs-on: ubuntu-latest
needs: build
permissions:
contents: read
id-token: write
environment:
name: pypi
steps:
- uses: actions/download-artifact@v4
with:
name: python-distributions
path: dist
- uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist
24 changes: 24 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ scripts/*
!scripts/test.mjs
!scripts/smoke.mjs
!scripts/e2e-icp-local.sh
!scripts/regenerate-python-fixture.mjs
npm-debug.log*
yarn-debug.log*
yarn-error.log*
Expand All @@ -14,14 +15,37 @@ pnpm-lock.yaml
# Build outputs
dist/
tmpdist/
build/
coverage/
.nyc_output/

# TypeScript
*.tsbuildinfo

# Python
__pycache__/
*.py[cod]
.pytest_cache/
.python-user-base/
.mypy_cache/
.ruff_cache/
.tox/
.nox/
.eggs/
*.egg-info/
pip-wheel-metadata/
htmlcov/
.coverage
.coverage.*
.venv/
venv/
env/

# Env / system files
.env
.env.local
.env.*.local
.agents/
.DS_Store
Thumbs.db

Expand Down
26 changes: 26 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,32 @@ cargo test

---

# 🐍 Python Runtime Support (Phase 2)

Knolo also ships a pure-Python runtime in `packages/core-python` for mounting existing `.knolo` packs and running deterministic lexical queries locally.

It stays local-first, requires no vector database, and does not use embeddings on the default query path.

Install locally:

```bash
cd packages/core-python
python -m pip install -e ".[dev]"
```

Use it from Python:

```python
from knolo import mount_pack, query

pack = mount_pack("tests/fixtures/simple.knolo")
hits = query(pack, "alpha beta", top_k=5)
```

For the release checklist and publishing notes, see [`packages/core-python/README.md`](packages/core-python/README.md) and [`packages/core-python/RELEASE.md`](packages/core-python/RELEASE.md).

---

# 🌐 ICP Canister Adapter (New)

Knolo now ships a local-first ICP path that keeps retrieval lexical-first and talks to the canister directly, with no middleware and no vector database.
Expand Down
2 changes: 1 addition & 1 deletion package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions packages/core-python/MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
include README.md
include src/knolo/py.typed

108 changes: 108 additions & 0 deletions packages/core-python/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# `knolo`

`knolo` is the pure-Python runtime for mounting existing `.knolo` packs and running deterministic lexical queries locally.

It is intentionally release-scoped for Phase 2:

- local-first retrieval
- deterministic lexical retrieval
- no vector database
- no embeddings on the default query path
- no Python pack builder
- no LangChain or LlamaIndex integration
- no Node.js runtime dependency for mount/query

Packs are still built with `@knolo/core` in TypeScript, then mounted and queried from Python.

## Install

From this package directory:

```bash
python -m pip install -e ".[dev]"
```

For a normal install, omit the extra:

```bash
python -m pip install .
```

## Query

```python
from knolo import mount_pack, query

pack = mount_pack("tests/fixtures/simple.knolo")
hits = query(pack, "alpha beta", top_k=5)

for hit in hits:
print(hit.block_id, hit.score, hit.text)
```

You can also mount bytes directly:

```python
from pathlib import Path
from knolo import mount_pack_from_bytes

pack = mount_pack_from_bytes(Path("tests/fixtures/simple.knolo").read_bytes())
```

## Release Readiness

The package publishes from GitHub release events via Trusted Publishing. No secret-based PyPI credentials are required in CI.

Before a release, run:

```bash
python -m pytest
python -m build
python -m twine check dist/*
```

A manual upload fallback is still available when needed:

```bash
python -m twine upload dist/*
```

See [`RELEASE.md`](./RELEASE.md) for the release checklist.

## Fixture Regeneration

The committed fixture at `tests/fixtures/simple.knolo` is what tests use, so the test suite does not need Node.js at runtime.

To regenerate the fixture from the checked-in corpus, run the root helper script from the repo root:

```bash
node scripts/regenerate-python-fixture.mjs
```

The script reads `tests/fixtures/corpus/intro.md`, `runtime.md`, and `other.md`, then rewrites the committed binary fixture. Pass `--check` to verify that the committed bytes match the corpus without rewriting.

## API

The public package exports:

- `mount_pack(source)`
- `mount_pack_from_bytes(data)`
- `query(pack, q, ...)`
- `KnoloError`
- `InvalidPackError`
- `PackStats`
- `PackMeta`
- `Pack`
- `QueryOptions`
- `Hit`
- `tokenize()`
- `normalize()`
- `__version__`

## Current Scope

- No Python pack builder yet
- No semantic reranking
- No embeddings or vector database integration on the default path
- No Node.js runtime dependency at query time
- No LangChain or LlamaIndex adapters in this package
13 changes: 13 additions & 0 deletions packages/core-python/RELEASE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Release Checklist

- [ ] Confirm the `knolo` distribution name is still available on PyPI, or choose a fallback package name before release if it is not.
- [ ] `cd packages/core-python && python -m pip install -e ".[dev]"`
- [ ] `cd packages/core-python && python -m pytest`
- [ ] `cd packages/core-python && python -m build`
- [ ] `cd packages/core-python && python -m twine check dist/*`
- [ ] Verify the wheel contents with `python -m zipfile -l dist/knolo-*.whl`.
- [ ] Verify the sdist contents with `tar -tzf dist/knolo-*.tar.gz`.
- [ ] Confirm the Python CI workflow passes on Python 3.10, 3.11, 3.12, and 3.13.
- [ ] Confirm the publish workflow only runs on GitHub release publication and uses Trusted Publishing with no hardcoded secrets.
- [ ] Smoke install the built wheel in a clean environment and run a basic `mount_pack` / `query` check.
- [ ] Yank a bad PyPI release instead of republishing the same tag if a release needs to be rolled back.
35 changes: 35 additions & 0 deletions packages/core-python/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
[build-system]
requires = []
build-backend = "setuptools.build_meta"
backend-path = ["."]

[project]
name = "knolo"
version = "0.1.0"
description = "Pure-Python runtime for mounting and querying .knolo packs."
readme = { file = "README.md", content-type = "text/markdown" }
requires-python = ">=3.10"
license = { text = "Apache-2.0" }
authors = [{ name = "Knolo" }]
dependencies = []

[project.optional-dependencies]
dev = [
"build>=1.2",
"pytest>=8",
"twine>=5",
]

[tool.setuptools]
package-dir = { "" = "src" }
include-package-data = true

[tool.setuptools.packages.find]
where = ["src"]

[tool.setuptools.package-data]
knolo = ["py.typed"]

[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["src"]
2 changes: 2 additions & 0 deletions packages/core-python/setuptools/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
"""Local build backend shim for the knolo Python package."""

Loading
Loading