Add RocksDB action backed by a RocksDB HTTP service - #1266
Merged
Paul Lizer (paullizer) merged 7 commits intoAug 18, 2026
Merged
Conversation
Adds a new `rocksdb` action type so agents can query an ordered RocksDB key-value store, with its own configuration card and Test Connection button in the action modal. RocksDB is an embedded library rather than a network database server, so the action supports two connection modes: - Embedded: opens a RocksDB directory on the application host through the `rocksdict` binding, using read-only or secondary access. - Remote: calls a RocksDB-backed HTTP/JSON service with optional bearer or API-key authentication. Reads are always available (get_value, get_values, key_exists, scan_prefix, scan_range, list_column_families, get_database_stats). Writes (put_value, delete_value, write_batch) are refused unless the action explicitly allows them, matching the read-only default of the SQL action. Security: - Embedded paths are gated by a new ROCKSDB_ALLOWED_ROOTS environment variable and validated with realpath plus a common-prefix check, so traversal, symlink escapes, and name-prefix siblings are rejected. Embedded mode stays disabled while the variable is unset. - The test-connection route requires an authenticated user and returns sanitized errors that never expose filesystem layout or driver text. - Remote base URLs are restricted to http and https, and service failures report the HTTP status code only. The `rocksdict` wheel is imported lazily, so an image without it still starts, still lists the action type, and still supports remote mode. Validation: 8/8 RocksDB functional tests including a real embedded RocksDB round trip, 12/12 route policy tests, 4/4 Cosmos regression tests, and a Playwright DOM pass over the action modal with no console errors. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
SimpleChat will not run RocksDB locally, so the action no longer opens a RocksDB database directory on the application host. The `rocksdb` action now talks exclusively to a RocksDB-backed HTTP/JSON service. Removed: - The `rocksdict` dependency and its lazy import helper. - The `ROCKSDB_ALLOWED_ROOTS` environment variable and the path allowlist resolution it guarded, along with the embedded connection test branch and its filesystem error sanitizer. - Embedded manifest fields (`connection_mode`, `db_path`, `access_mode`, `secondary_path`) plus their validation, modal controls, and summary rows. - Embedded scan, batch, and key/value codec helpers along with the cached RocksDB handle map. Because the connection mode selector is gone, the modal now presents the service configuration directly, which removes a step from the flow. The configured `key_encoding` and `value_encoding` are now sent with every service request so the service knows the wire representation of keys and values, and the documented contract records that. Read and write surfaces are unchanged: the same ten kernel functions are exposed and writes stay blocked until an action explicitly allows them. Net effect is -1658/+768 lines across 13 files. Validation: 10/10 RocksDB functional tests, 12/12 route policy tests, 4/4 Cosmos regression tests, and a Playwright DOM pass over the simplified action modal with no console errors, including assertions that the embedded controls are absent from the DOM. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Development added the Yamcs action, which touches the same action-type integration points as RocksDB, so the two features were interleaved by the automatic merge. Resolved to keep both side by side. Conflicts resolved: - config.py: bumped VERSION to 0.250.214, above Development's 0.250.213. - route_backend_plugins.py: kept both test_yamcs_connection and test_rocksdb_connection as separate routes. - plugin_modal_stepper.js: kept both test-connection button listeners, both method blocks, and merged isStructuredConfigType and the summary database-type conditions to cover isRocksDbType and isYamcsType. - release_notes.md: kept both sets of notes and renumbered the RocksDB entry to v0.250.214 so it sits above Development's v0.250.213. The RocksDB feature doc moved to docs/explanation/features/v0.250.214/ and its version references, the features index link, and the functional and UI test version headers were updated to match. Validation after the merge: 10/10 RocksDB functional tests, 14/14 Yamcs functional tests, 4/4 Cosmos regression tests, 12/12 route policy tests, and a Playwright DOM pass confirming both the RocksDB and Yamcs sections switch, configure, and post correctly with no console errors. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Development shipped the agent instruction context references feature and claimed VERSION 0.250.214, which collided with the RocksDB entry. Conflict resolved: - release_notes.md: split the shared v0.250.214 heading so Development's agent instruction feature keeps v0.250.214 and RocksDB moves to its own v0.250.215 section above it. Renumbered alongside it: config.py VERSION, the feature doc folder (docs/explanation/features/v0.250.215/), the features index link, the plugin module docstring reference, and the functional and UI test headers. Validation: 10/10 RocksDB functional tests, 14/14 Yamcs functional tests, 6/6 agent modal instruction step-order tests, 4/4 Cosmos regression tests, and 12/12 route policy tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CodeQL flagged 15 new alerts on this PR. Fixed the ones this change introduced. Always verify TLS (py/request-without-cert-validation, high): Removed the "Verify TLS Certificate" toggle and the verify_tls field instead of passing a possibly-false verify= into requests. Certificate validation is now always enforced. Operators fronting the service with a private certificate authority should install that CA in the application trust store or point REQUESTS_CA_BUNDLE at it, which is the documented path. This drops a config field, a UI control, and a request parameter. Keep exception text out of logs (py/clear-text-logging-sensitive-data, high): the plugin and the connection-test route no longer interpolate raw exceptions into log messages. They log the exception type plus a sanitized detail through sanitize_log_message, so a transport error carrying request details cannot reach the log sink verbatim. Stop echoing exception text to the browser (py/stack-trace-exposure, medium): the connection-test route returns authored messages for invalid base URLs, action lookup failures, and unresolved stored secrets rather than str(exc). Close test file handles (py/file-not-closed): the functional test reads the plugin module and requirements.txt through context managers. The remaining alerts on this PR are pre-existing on Development and are not introduced here: the functions_appinsights.py clear-text-logging and log-injection sinks, py/import-of-mutable-attribute in scripts/ and deployers/, and the same py/stack-trace-exposure pattern already open on the Yamcs and Cosmos routes. Validation: 10/10 RocksDB functional tests, 14/14 Yamcs functional tests, 12/12 route policy tests, and a Playwright DOM pass confirming the modal still configures and posts correctly with the TLS control removed and no console errors. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
CodeQL still traced a path from the RocksDB action into the shared logging sinks in functions_appinsights.py, so the action no longer puts anything credential-derived into log properties at all. - The initialization log dropped has_auth_key. It read the credential to compute a boolean, and auth_scheme already says whether the action authenticates. - The plugin and connection-test failure logs now record only the exception type. Both already pass exceptionTraceback=True, so Application Insights still captures the exception itself, and the message text added nothing that was not already available. Errors returned to the model still go through sanitize_log_message, which redacts secret assignments and Authorization values. Validation: 10/10 RocksDB functional tests and 12/12 route policy tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Development shipped the retrieved-sources and cited-references fix and claimed VERSION 0.250.215, which collided with the RocksDB entry. Conflict resolved: - release_notes.md: split the shared v0.250.215 heading so Development's citation fix keeps v0.250.215 and RocksDB moves to its own v0.250.216 section above it. Renumbered alongside it: config.py VERSION, the feature doc folder (docs/explanation/features/v0.250.216/), the features index link, the plugin module docstring reference, and the functional and UI test headers. Validation: 10/10 RocksDB functional tests, 16/16 cited-source tracking tests, 14/14 Yamcs functional tests, and 12/12 route policy tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
rocksdbaction type so agents can query an ordered RocksDB key-value store, with its own configuration card and Test Connection button in the action modal — similar in shape to the existing SQL and Cosmos actions.RocksDB is an embedded C++ library with no native network protocol. SimpleChat does not run RocksDB locally and never opens a database directory on the application host. Instead, the action calls a RocksDB-backed HTTP/JSON service that the operator runs alongside their data.
Agent functions
Reads are always available:
get_value·get_values·key_exists·scan_prefix·scan_range·list_column_families·get_database_statsWrites (
put_value,delete_value,write_batch) are refused unless the action explicitly allows them, matching the read-only default of the SQL action. While read-only they never issue a request.Configuration
http/httpsonlynone,bearer, orapi_key(configurable header name)defaultutf8,base64, andjsonfor valuesThe configured encodings are sent with every service request so the service knows the wire representation of keys and values. The service token lives in
auth.key, so existing Key Vault storage and masked-edit flows apply unchanged — noplugin.schema.jsonchange needed.Security
POST /api/plugins/test-rocksdb-connectionsits on thebpapblueprint with@swagger_route(security=get_auth_security())→@login_required→@user_required.http/https.value_truncatedflag plus the originalvalue_bytes, so one record cannot flood the model context.Service contract
The plugin speaks a documented SimpleChat contract:
/health,/get,/multi_get,/exists,/scan,/column_families,/stats,/put,/delete,/batch. Range semantics (inclusive start, exclusive end,reverse, clampedlimit) are specified in the feature doc so operators can implement a conforming service. The Test Connection button probesGET /healthand falls back toPOST /scanwith{"limit": 1}when/healthreturns 404.Validation
functional_tests/test_rocksdb_plugin.pyfunctional_tests/route_tests/functional_tests/test_cosmos_query_plugin.py(regression)functional_tests/test_yamcs_action_plugin.py(post-merge regression)functional_tests/test_agent_modal_instructions_step_order.py(post-merge regression)ui_tests/test_workspace_rocksdb_action_modal.pyI drove the real modal in a headless browser rather than only parsing the JS. That caught two genuine bugs, both fixed here:
"bad block"was misclassified as a lock error becauseblockcontainslock.After merging
DevelopmentI re-ran the DOM pass to confirm the RocksDB and Yamcs sections both still switch, configure, and post correctly.Files
New
application/single_app/semantic_kernel_plugins/rocksdb_plugin.pyapplication/single_app/static/json/schemas/rocksdb.definition.jsonfunctional_tests/test_rocksdb_plugin.pyui_tests/test_workspace_rocksdb_action_modal.pydocs/explanation/features/v0.250.215/ROCKSDB_ACTION.mdModified
route_backend_plugins.py— test-connection route and action type cardsemantic_kernel_plugins/plugin_health_checker.py— manifest validationtemplates/_plugin_modal.html— config section, summary card, CSS with dark themestatic/js/plugin_modal_stepper.js— full type wiringstatic/js/workspace/view-utils.js— action type iconconfig.py(VERSION→0.250.215), release notes, features index, logging tagsReview notes
Commit history. The first commit added the action with both an embedded mode (via the
rocksdictbinding, gated by aROCKSDB_ALLOWED_ROOTSpath allowlist) and the remote mode. The second commit removes embedded support entirely per the decision not to run RocksDB on the app host — dropping therocksdictdependency, the allowlist env var, the embedded manifest fields, and the connection-mode selector. Two later commits mergeDevelopment. Reviewing the squashed diff is by far the clearest view.Merge conflicts.
Developmentshipped the Yamcs action, which touches the same action-type integration points, so the automatic merge interleaved the two features. They were resolved to keep both side by side: separate routes, separate method blocks, and mergedisStructuredConfigType/ summary-type conditions coveringisRocksDbTypeandisYamcsType. The net diff againstDevelopmentis +2852/−6, and every one of those 6 deleted lines is either the version bump, a trailing-whitespace line, or a condition that gainsisRocksDbTypewhile keepingisYamcsType. NoDevelopmentcode was lost.Known limitations
Follow-up
No GitHub issue is currently linked — happy to file one if you would like this tracked.