Repository navigation
Move agent environment and Copilot session arguments into core - #931
Closed
Sun Haoran (haoranpb) wants to merge 9 commits into
Closed
Sun Haoran (haoranpb) wants to merge 9 commits into
Sun Haoran (haoranpb) wants to merge 9 commits into
Conversation
bcbench.operations.git_operations moves to bcbench_core.git with its history, together with the errors it raises (GitOperationError, PatchApplicationError, EmptyDiffError). Nothing caught them through BCBenchError, so they now derive from Exception. - apply_patch writes its temporary file with a fixed ".patch" suffix instead of reading app config; the now-unused patch_pattern field is removed. - Callers import from bcbench_core.git directly, and bcbench.operations no longer re-exports git functions. - The core README lists the git 2.49+ and authenticated gh prerequisites. - Tests move to core, with new coverage for apply_patch, clean_repo, commit --no-verify, and PatchApplicationError, keeping core at 100%. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The app.json, Word, and Markdown exclusions were BC-Bench policy baked into the core function. They become an `exclude` pathspec tuple with the same defaults, so callers keep today's behaviour and other applications can choose their own. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The data-query runner (wrap_query_as_api, execute_al_query and the query template) moves verbatim into bcbench.operations.dataquery_operations, so bc_operations.py holds only the build, test and artifact operations that move to bcbench-core next. escape_ps_string becomes public because both modules use it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bcbench.operations.bc_operations moves to bcbench_core.bc: the PowerShell script builders, escape_ps_string, build_and_publish_projects (Base Application gets its own timeout), run_tests (fail-to-pass and pass-to-pass must pass) and run_test_suite, with timeouts as parameters defaulting to 5, 30 and 3 minutes. Supporting pieces get their own core modules: - bcbench_core.artifacts: BcContainerHelper artifact cache helpers, with the cache path, artifact type and symbol folder as parameters (C:\bcartifacts.cache, "sandbox", ".alpackages" by default). - bcbench_core.container: ContainerConfig, moved from bcbench.types. - bcbench_core.dataset: the dataset's pydantic TestEntry, shared by the dataset models and the runners; future home of the dataset entries. - bcbench_core.exceptions: the build/test errors and output extractors from bcbench.exceptions, and the git errors from bcbench_core.git. PowerShell modules ship in bcbench_core/powershell/: - BCBenchUtils.psm1 (moved from scripts/) keeps the generic functions (Write-Log, Get-BCCredential, git clone/patch, Update-AppProjectVersion); the BC-Bench ones move to scripts/BenchmarkUtils.psm1. - AppUtils.psm1 (moved from scripts/) builds, publishes and tests apps. - Dataset.psm1 holds the TestEntry class from scripts/DatasetEntry.psm1; Invoke-DatasetTests keeps its [TestEntry[]] parameter. BC-Bench drops the config it no longer needs (ps_script_path, artifact cache, .alpackages name, build/test timeouts) and imports from core. Core signatures no longer use keyword-only markers, consistently. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bcbench.agent.shared.altool_paths moves to bcbench_core.altool_paths: the package-cache and assembly probing paths that `altool` (AL compiler, LSP and MCP servers) needs, from a container's BcContainerHelper compiler folder or from the BC artifact cache, plus system .NET runtime detection. - The BcContainerHelper compiler root and the artifact cache become parameters with their defaults (C:\ProgramData\BcContainerHelper\compiler, C:\bcartifacts.cache). - bcbench-core declares its packaging dependency. - The probing-path tests move to core, with new coverage for runtime detection, compiler folders and artifact-derived paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bcbench_core.altool_paths becomes bcbench_core.altool and gains the harness-neutral launch pieces from the LSP and MCP integrations: - resolve_symbol_paths: symbol and probing paths from the container's compiler folder, falling back to the BC artifact cache. - build_lsp_args: `launchlspserver` arguments. - connection_env: the BC_SERVER_* variables altool reads, from a ContainerConfig. lsp.py and mcp.py keep the BC-Bench wiring: dataset entries, harness config shapes, the MCP server template, plugins, and the symbol-download hint in the error message. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The bottom layer of the Copilot harness moves to bcbench_core.agent, keeping its file layout so the moves show as renames: - agent/copilot/cli.py: find, version and invoke the Copilot CLI. - agent/copilot/metrics.py: parse `--output-format=json` into metrics. - agent/version.py: generic CLI version detection (Claude uses it too). - agent/metrics.py: AgentMetrics, from bcbench.types; PRReviewMetrics and the result union stay in the app and build on it. - AgentError moves to bcbench_core.exceptions. invoke_copilot and parse_output drop their keyword-only markers, like the rest of core. Plugins, MCP, the gateway, prompts and the agent run loop stay in the app for the harness design. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- bcbench_core.agent.env.agent_subprocess_env (renamed from agent/shared/env.py) builds a launched agent's environment without the BC container credentials. It now takes the parent environment as a parameter; the Copilot and Claude agents pass os.environ. - bcbench_core.agent.copilot.cli.copilot_session_args builds the session arguments: debug logs, MCP config, plugin dirs, directory grants and a custom agent. The Copilot agent keeps the BC-Bench choices of which plugins and grants to pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sun Haoran (haoranpb)
force-pushed
the
core/stack-08-copilot-cli
branch
2 times, most recently
from
October 6, 2026 09:09
23aa2dc to
3a6aa7b
Compare
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack 9 of the
bcbench-coreextraction, stacked on #930. Moves the generic parts of launching a Copilot agent session into the library.bcbench_core.agent.env.agent_subprocess_env(renamed fromagent/shared/env.py): the agent's process environment without BC container credentials, now taking the parent environment as a parameter (callers passos.environ).copilot_session_argsinbcbench_core.agent.copilot.cli: debug logs, MCP config, plugin dirs, directory grants and custom agent. The Copilot agent keeps the BC-Bench choices of what to pass.Verified with tests (core at 100%) and a real Copilot CLI run using the session arguments and filtered environment.