Skip to content

Commit 8f02792

Browse files
srimon12srimon12
committed
docs: add roadmap and contributor guides
1 parent ff356b8 commit 8f02792

19 files changed

Lines changed: 1298 additions & 0 deletions

‎.github/RFC_TEMPLATE.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# RFC: <Short Title>
2+
3+
- **Status:** Draft / Proposed / Accepted / Rejected / Implemented
4+
- **Author:** @github-username
5+
- **Target:** Python / Go / Both
6+
- **Created:** YYYY-MM-DD
7+
8+
## Summary
9+
10+
Describe the proposed change in one paragraph.
11+
12+
## Motivation
13+
14+
What problem does this solve?
15+
16+
- Current pain:
17+
- Who is affected:
18+
- Why QQL should expose this:
19+
20+
## Proposed Syntax
21+
22+
```sql
23+
-- Minimal example
24+
NEW SYNTAX ...
25+
26+
-- Example with optional clauses
27+
NEW SYNTAX ... WHERE ... WITH { ... }
28+
```
29+
30+
## Qdrant Mapping
31+
32+
| QQL syntax | Qdrant API/model |
33+
|---|---|
34+
| `...` | `...` |
35+
36+
If Qdrant does not directly support the behavior, explain why QQL should still add it.
37+
38+
## Output
39+
40+
Human-readable output:
41+
42+
```text
43+
...
44+
```
45+
46+
JSON output:
47+
48+
```json
49+
{
50+
"success": true,
51+
"message": "...",
52+
"data": {}
53+
}
54+
```
55+
56+
## Compatibility
57+
58+
- Does this break existing QQL scripts?
59+
- Does this affect JSON output contracts?
60+
- Should Python and Go match?
61+
- Can one implementation ship first?
62+
63+
## Implementation Plan
64+
65+
1. Lexer/parser/AST changes
66+
2. Executor changes
67+
3. Tests
68+
4. Documentation
69+
70+
## Alternatives
71+
72+
List simpler or competing designs and why they were not chosen.
73+
74+
## Open Questions
75+
76+
- Question 1
77+
- Question 2
78+
79+
## References
80+
81+
- Qdrant docs:
82+
- Related issues:
83+
- Related PRs:

‎ROADMAP.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# QQL Roadmap
2+
3+
> **Status:** Draft for maintainer and community discussion
4+
> **Scope:** Public direction for the Python `qql-cli` project, with companion notes for the Go implementation where parity matters
5+
> **Principle:** Keep QQL small, readable, and close to Qdrant's real API surface
6+
7+
QQL is a SQL-like language and CLI for common Qdrant workflows. The near-term goal is not to cover every Qdrant feature. The goal is to make everyday vector database work easier to read, script, test, and share.
8+
9+
## Current Position
10+
11+
The Python implementation already supports the core workflow:
12+
13+
| Area | Status |
14+
|---|---|
15+
| Collection create/drop/list | Supported |
16+
| Payload indexes | Supported |
17+
| Insert and bulk insert | Supported |
18+
| Dense search | Supported |
19+
| Hybrid dense+sparse search | Supported |
20+
| Sparse-only search | Supported |
21+
| WHERE filters | Supported |
22+
| Recommend by example IDs | Supported |
23+
| Query-time search params | Supported |
24+
| Reranking | Supported in Python |
25+
| Delete by ID or filter | Supported |
26+
| Script execution and dump/restore | Supported |
27+
| Programmatic Python API | Supported through `run_query()` |
28+
29+
The Go implementation is developed separately. It should aim for language and behavior parity where practical, but this Python repository should not block on Go work before improving its own CLI and documentation.
30+
31+
## Near-Term Priorities
32+
33+
These are the best candidates for small, useful contributions. Each one should have tests and documentation before being considered complete.
34+
35+
| Priority | Feature | Why it matters | Suggested syntax |
36+
|---|---|---|---|
37+
| P0 | Get point by ID | Basic inspection is currently missing | `GET FROM <collection> WHERE id = '<id>'` |
38+
| P0 | Scroll points | Needed for real datasets and exports | `SCROLL <collection> LIMIT 100` |
39+
| P0 | Search pagination | Needed for browsing result sets | `SEARCH ... LIMIT 10 OFFSET 20` |
40+
| P1 | Count points | Useful for validation and scripts | `COUNT <collection> WHERE <filter>` |
41+
| P1 | Describe collection | Improves introspection and debugging | `DESCRIBE COLLECTION <name>` |
42+
| P1 | Update payload | Avoids full reinsert for metadata changes | `UPDATE <collection> SET {...} WHERE id = '<id>'` |
43+
| P1 | Delete payload keys | Removes fields without deleting points | `DELETE PAYLOAD field FROM <collection> WHERE id = '<id>'` |
44+
45+
## Later Ideas
46+
47+
These are worth exploring, but they should not distract from the smaller parity gaps above.
48+
49+
| Area | Possible work |
50+
|---|---|
51+
| Retrieval quality | MMR, score boosting, named vector search, batch search |
52+
| Collection configuration | Distance selection, HNSW config, quantization, on-disk payload |
53+
| Developer experience | Connection profiles, clearer JSON contracts, better error messages |
54+
| Ecosystem | Syntax highlighting, examples, tutorials, migration guides |
55+
| Operations | Collection aliases, snapshots, backup/restore workflows |
56+
57+
## Contribution Process
58+
59+
Use an RFC when a change affects syntax, CLI behavior, or JSON output. Small documentation fixes, tests, and bug fixes do not need an RFC.
60+
61+
Good roadmap issues should include:
62+
63+
- the Qdrant API being exposed
64+
- the proposed QQL syntax
65+
- expected human-readable output
66+
- expected JSON output
67+
- Python tests required
68+
- Go parity notes, if relevant
69+
70+
## Documentation Goals
71+
72+
The documentation should stay practical:
73+
74+
- README: quick start and common usage
75+
- `docs/syntax/`: compact syntax reference
76+
- `docs/COMPATIBILITY.md`: checked feature matrix
77+
- `docs/CONTRIBUTING.md`: contributor workflow
78+
- `docs/RFCS/`: proposed and accepted syntax decisions
79+
- `docs/TUTORIALS/`: runnable examples as they are added
80+
- `docs/MIGRATING/`: focused migration notes
81+
82+
## Success Criteria
83+
84+
QQL is moving in the right direction when:
85+
86+
- users can inspect, insert, search, recommend, update, count, and export without dropping to raw SDK calls for common cases
87+
- syntax changes are discussed before implementation
88+
- docs describe what is implemented today, not only what is planned
89+
- Python and Go differences are visible and intentional
90+
- contributors can find small, well-scoped issues
91+
92+
This roadmap is intentionally modest. It should be revised as maintainers and contributors agree on scope.

‎SHARED.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Cross-Repository Documentation Notes
2+
3+
QQL has a Python implementation (`qql-cli`) and a companion Go implementation (`qql-go`). Some documentation is useful to keep conceptually aligned across both projects, but each repository remains responsible for its own implementation details.
4+
5+
This file is a coordination guide, not a hard synchronization system.
6+
7+
## Shared in Spirit
8+
9+
These documents should use the same terminology and avoid contradicting each other across implementations:
10+
11+
| File | Purpose |
12+
|---|---|
13+
| `ROADMAP.md` | Project direction and priority areas |
14+
| `.github/RFC_TEMPLATE.md` | Template for syntax and behavior proposals |
15+
| `.github/ISSUE_LABELS.md` | Suggested issue label taxonomy |
16+
| `docs/CONTRIBUTING.md` | Contributor workflow |
17+
| `docs/SYNTAX_GUIDELINES.md` | How to add or change QQL syntax |
18+
| `docs/COMPATIBILITY.md` | Feature matrix across Qdrant, Python, and Go |
19+
| `docs/RFCS/README.md` | RFC process overview |
20+
21+
## Repository-Specific
22+
23+
These files should normally stay different:
24+
25+
| Python `qql` | Go `qql-go` | Why |
26+
|---|---|---|
27+
| `README.md` | `README.md` | Different install, command, and release details |
28+
| `pyproject.toml` | `go.mod` | Different package managers |
29+
| `src/qql/` | Go source tree | Different implementations |
30+
| `tests/` | Go tests | Different test frameworks |
31+
| release notes | release notes | Different version history |
32+
33+
## Update Guidance
34+
35+
When a change affects the QQL language rather than one implementation:
36+
37+
1. Update the local documentation.
38+
2. Note whether the behavior is Python-only, Go-only, or shared.
39+
3. If the companion implementation is affected, open or link a tracking issue there.
40+
4. Avoid blocking one implementation's documentation on the other unless the feature requires true lockstep behavior.
41+
42+
## Long-Term Options
43+
44+
If cross-repo drift becomes painful, consider one of these later:
45+
46+
- a small `qql-spec` repository for syntax and compatibility docs
47+
- a CI check that compares selected docs between repos
48+
- release notes that explicitly call out Python and Go parity gaps
49+
50+
For now, keep the process lightweight and accurate.

‎docs/COMPATIBILITY.md‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# QQL / Qdrant Compatibility Matrix
2+
3+
> Tracks the current Python `qql-cli` surface and known companion status for `qql-go`.
4+
> Last checked: 2026-04-28.
5+
6+
This document should describe implemented behavior conservatively. If a feature is planned but not implemented, keep it marked as missing until tests exist.
7+
8+
## Legend
9+
10+
| Symbol | Meaning |
11+
|---|---|
12+
| Supported | Implemented and covered by normal usage/tests |
13+
| Partial | Implemented with known limits |
14+
| Missing | Not currently exposed by QQL |
15+
| Planned | Roadmap or RFC candidate |
16+
| Unknown | Needs verification in that implementation |
17+
18+
## Collection Management
19+
20+
| Feature | Qdrant API | Python `qql-cli` | Go `qql-go` | Notes |
21+
|---|---|---|---|---|
22+
| Create collection | `create_collection` | Supported | Supported | Dense collection |
23+
| Create hybrid collection | `create_collection` with sparse vectors | Supported | Supported | `CREATE COLLECTION ... HYBRID` |
24+
| Create with custom distance | Vector params | Missing | Missing | Currently cosine-only in Python |
25+
| Create with custom HNSW | `hnsw_config` | Missing | Missing | Roadmap candidate |
26+
| Create with quantization | `quantization_config` | Missing | Missing | Roadmap candidate |
27+
| Create with on-disk payload | `on_disk_payload` | Missing | Missing | Roadmap candidate |
28+
| Create with multivectors | `multivector_config` | Missing | Missing | Advanced roadmap candidate |
29+
| Drop collection | `delete_collection` | Supported | Supported | `DROP COLLECTION` |
30+
| List collections | `get_collections` | Supported | Supported | `SHOW COLLECTIONS` |
31+
| Collection info | `get_collection` | Missing | Missing | Proposed as `DESCRIBE COLLECTION` |
32+
| Collection aliases | Alias APIs | Missing | Missing | Later idea |
33+
| Collection snapshots | Snapshot APIs | Missing | Missing | Later idea |
34+
35+
## Points / Documents
36+
37+
| Feature | Qdrant API | Python `qql-cli` | Go `qql-go` | Notes |
38+
|---|---|---|---|---|
39+
| Insert point | `upsert` | Supported | Supported | Requires a `text` field for embedding |
40+
| Insert bulk | `upsert` | Supported | Supported | `INSERT BULK` |
41+
| Explicit point ID on insert | `upsert` | Supported | Supported | Integer or UUID string |
42+
| Get point by ID | `retrieve` | Missing | Missing | Near-term roadmap candidate |
43+
| Update payload | `set_payload` | Missing | Missing | Near-term roadmap candidate |
44+
| Delete point by ID | `delete` | Supported | Supported | `DELETE ... WHERE id = ...` |
45+
| Delete points by filter | `delete` with filter selector | Supported | Supported | Python parser/executor support non-ID filters |
46+
| Delete payload keys | `delete_payload` | Missing | Missing | Near-term roadmap candidate |
47+
| Count points | `count` | Missing | Missing | Near-term roadmap candidate |
48+
| Scroll points | `scroll` | Missing | Missing | Near-term roadmap candidate |
49+
50+
## Search
51+
52+
| Feature | Qdrant API | Python `qql-cli` | Go `qql-go` | Notes |
53+
|---|---|---|---|---|
54+
| Dense search | `query_points` | Supported | Supported | Default mode |
55+
| Hybrid search | `query_points` + RRF | Supported | Supported | `USING HYBRID` |
56+
| Sparse-only search | `query_points` sparse vector | Supported | Supported | `USING SPARSE` |
57+
| Exact search | `SearchParams.exact` | Supported | Supported | `EXACT` or `WITH { exact: true }` |
58+
| HNSW ef tuning | `SearchParams.hnsw_ef` | Supported | Supported | `WITH { hnsw_ef: N }` |
59+
| ACORN filtered search | `SearchParams.acorn` | Supported | Supported | Depends on Qdrant support |
60+
| Search with filters | `Filter` | Supported | Supported | `WHERE` clause |
61+
| Search pagination | `offset` | Missing | Missing | Near-term roadmap candidate |
62+
| Batch search | Batch/query APIs | Missing | Missing | Later idea |
63+
| MMR diversity | Query diversity controls | Missing | Missing | Later idea |
64+
| Score boosting | Formula/rescore APIs | Missing | Missing | Later idea |
65+
| Multivector search | Multivector query | Missing | Missing | Later idea |
66+
| Rerank | Cross-encoder / inference | Supported | Partial | Python uses local Fastembed cross-encoder; Go behavior should be checked against `qql-go` docs |
67+
| Relevance feedback | Feedback query | Missing | Missing | Later idea |
68+
69+
## Recommend
70+
71+
| Feature | Qdrant API | Python `qql-cli` | Go `qql-go` | Notes |
72+
|---|---|---|---|---|
73+
| Recommend by examples | Recommend query | Supported | Supported | `RECOMMEND FROM` |
74+
| Positive/negative IDs | Recommend input | Supported | Supported | |
75+
| Strategy selection | `RecommendStrategy` | Supported | Supported | `average_vector`, `best_score`, `sum_scores` |
76+
| Cross-collection lookup | `lookup_from` | Supported | Supported | |
77+
| Named vector usage | `using` | Supported | Supported | |
78+
| Offset | `offset` | Supported | Supported | |
79+
| Score threshold | `score_threshold` | Supported | Supported | |
80+
| Filtered recommend | `Filter` | Supported | Supported | `WHERE` clause |
81+
82+
## Payload Indexes
83+
84+
| Feature | Qdrant API | Python `qql-cli` | Go `qql-go` | Notes |
85+
|---|---|---|---|---|
86+
| Keyword index | `create_payload_index` | Supported | Supported | |
87+
| Integer index | `create_payload_index` | Supported | Supported | Python syntax uses `TYPE integer` |
88+
| Float index | `create_payload_index` | Supported | Supported | |
89+
| Bool index | `create_payload_index` | Supported | Supported | |
90+
| Text index | `create_payload_index` | Supported | Partial | Go support should be verified |
91+
| Geo index | `create_payload_index` | Supported | Missing | Python maps `TYPE geo` |
92+
| Datetime index | `create_payload_index` | Supported | Missing | Python maps `TYPE datetime` |
93+
94+
## Filtering
95+
96+
| Feature | Qdrant model | Python `qql-cli` | Go `qql-go` | Notes |
97+
|---|---|---|---|---|
98+
| Equality | `MatchValue` | Supported | Supported | `=` |
99+
| Inequality | `must_not` + `MatchValue` | Supported | Supported | `!=` |
100+
| Range | `Range` | Supported | Supported | `>`, `<`, `>=`, `<=` |
101+
| Between | `Range` | Supported | Supported | Inclusive |
102+
| In list | `MatchAny` | Supported | Supported | `IN (...)` |
103+
| Not in list | `MatchExcept` | Supported | Supported | `NOT IN (...)` |
104+
| Is null | `IsNull` | Supported | Supported | |
105+
| Is empty | `IsEmpty` | Supported | Supported | |
106+
| Full-text match | `MatchText` | Supported | Supported | `MATCH` |
107+
| Match any term | `MatchTextAny` | Supported | Supported | `MATCH ANY` |
108+
| Match phrase | `MatchPhrase` | Supported | Supported | `MATCH PHRASE` |
109+
| Logical operators | `must`, `should`, `must_not` | Supported | Supported | `AND`, `OR`, `NOT` |
110+
| Nested fields | Payload key paths | Supported | Supported | Dot notation |
111+
| Nested array access | Payload key paths | Partial | Partial | Keep examples conservative until integration-tested |
112+
113+
## Version Notes
114+
115+
| Implementation | Current version in this repo/docs | Notes |
116+
|---|---|---|
117+
| Python `qql-cli` | `1.4.0` | Source of truth for this repository |
118+
| Go `qql-go` | `0.1.x` | Companion implementation; verify exact behavior in the Go repo before release claims |
119+
120+
## Known Gaps
121+
122+
| Gap | Impact | Suggested next step |
123+
|---|---|---|
124+
| No `GET` statement | Hard to inspect one point from the CLI | Add RFC or issue |
125+
| No `SCROLL` statement | Hard to page/export large collections through QQL syntax | Add RFC or issue |
126+
| No `COUNT` statement | Hard to validate scripts and filters | Add RFC or issue |
127+
| No `DESCRIBE COLLECTION` | Users must drop to SDK/Qdrant UI for collection metadata | Add RFC or issue |
128+
| No payload update syntax | Metadata updates require SDK calls or full reinsert | Add RFC or issue |
129+
| Limited custom collection configuration | Advanced users need SDK for distance/HNSW/quantization | Define minimal syntax before implementing |
130+
131+
## Maintenance Rule
132+
133+
When changing QQL behavior:
134+
135+
1. Update this matrix in the same PR.
136+
2. Link or mention tests that prove the status.
137+
3. Mark companion implementation status as `Unknown` rather than guessing.
138+
4. Avoid future-tense claims unless there is an accepted RFC or linked issue.

0 commit comments

Comments
 (0)