| 1 | <!-- English translation of adr/0052-agent-contract-cli-and-snapshot-tests.md. Canonical Russian: ../../adr/0052-agent-contract-cli-and-snapshot-tests.md --> |
| 2 | |
| 3 | # ADR 0052: CLI for agent contract (parity with MCP) and snapshot tests |
| 4 | |
| 5 | **Status:** Accepted · Implemented |
| 6 | **Date:** 2026-04-17 |
| 7 | **Implementation:** [`Services/AgentContract/AgentContractRunner.cs`](../../Services/AgentContract/AgentContractRunner.cs), [`Services/AgentContract/AgentContractHeadlessRuntime.cs`](../../Services/AgentContract/AgentContractHeadlessRuntime.cs), input `CascadeIDE.exe --agent-contract [--workspace <dir>] <command>` in [`Program.cs`](../../Program.cs); tests [`CascadeIDE.Tests/AgentContractRunnerTests.cs`](../../CascadeIDE.Tests/AgentContractRunnerTests.cs), [`CascadeIDE.Tests/AgentContractRunnerHeadlessTests.cs`](../../CascadeIDE.Tests/AgentContractRunnerHeadlessTests.cs) (including snapshot slice `cockpit_surface`: [`CascadeIDE.Tests/AgentContractCockpitContractSlice.cs`](../../CascadeIDE.Tests/AgentContractCockpitContractSlice.cs), [`CascadeIDE.Tests/TestData/AgentContract/cockpit_surface_contract_slice.approved.json`](../../CascadeIDE.Tests/TestData/AgentContract/cockpit_surface_contract_slice.approved.json)). Parity with MCP: `ide_get_ui_modes_diagnostics`, `ide_get_supported_editor_languages`, `ide_get_ide_state` (full summary - CLI command `get_ide_state`), `ide_get_cockpit_surface` / CDS only - `get_cockpit_surface` (same JSON as the `cockpit_surface` field in `get_ide_state`), `ide_git_*` (same JSON fields as in `MainWindowViewModel` / `GitCommandBuilder` + `GitCommandRunner`). |
| 8 | ## Related ADRs |
| 9 | |
| 10 | | ADR | Role | |
| 11 | |-----|------| |
| 12 | | [0002](0002-debug-human-agent-parity.md) | parity “person ↔ agent” | |
| 13 | | [0008](0008-mcp-contracts-and-testable-infrastructure.md) | MCP contracts and testable infrastructure | |
| 14 | | [0043](0043-mcp-transport-recovery-human-agent-parity.md) | MCP transport and parity | |
| 15 | | [0047](0047-cockpit-instrument-descriptor-and-slot-composition.md) | Cabin tool (`Instrument`) - a handle to the slot composition, not `Control` | |
| 16 | |
| 17 | ### Outside ADR |
| 18 | |
| 19 | | Document | Role | |
| 20 | |----------|------| |
| 21 | | [MCP-PROTOCOL.md](../../MCP-PROTOCOL.md) | IDE/stdio MCP commands | |
| 22 | |
| 23 | ### Implementation snapshot |
| 24 | |
| 25 | | Element | Meaning | |
| 26 | |---------|----------| |
| 27 | | — | `get_ui_modes_diagnostics`, `get_supported_editor_languages`, `get_solution_info`, `get_cockpit_surface`, `get_ide_state`, read-only `git_*` with `--workspace` | |
| 28 | | — | CI: `.gitlab-ci.yml` - `dotnet test` + smoke `--agent-contract` | |
| 29 | | — | **golden slice** CDS: `CascadeIDE.Tests/TestData/AgentContract/cockpit_surface_contract_slice.approved.json` + `AgentContractCockpitContractSlice` | |
| 30 | |
| 31 | --- |
| 32 | ## Context |
| 33 | |
| 34 | Today, Cascade IDE gives the agent **rich JSON** (layout, workspace state, diagnostics, cockpit/tools projection, etc.) through **MCP tools** (`ide_*`, see contract in [MCP-PROTOCOL.md](../../MCP-PROTOCOL.md)). Some of the fields are **the same semantic layer** on which the UI is based (placement, visibility of slots, layout in the sense of CDS/host surface), and not “a parallel invention for the agent.” This is convenient interactively, but: |
| 35 | |
| 36 | - **Regressions** in the response form are difficult to catch without running the IDE and the MCP script. |
| 37 | - **CI** wants to have **deterministic** contract checks: “after changing the snapshot collector, the response did not break.” |
| 38 | - A separate implementation “as a CLI” risks **diverging** from the real MCP - the tests will be green, the agent in Cursor will not. |
| 39 | |
| 40 | ## Solution |
| 41 | |
| 42 | **Accept** a separate delivery - **headless CLI** (or a subcommand of an existing host), which: |
| 43 | |
| 44 | 1. **Calls the same** public/internal **data collectors and JSON serialization** as MCP handlers for the selected `ide_*` commands - **without a second copy of the logic** (general layer: services, `IdeMcpCommandExecutor` helpers, snapshot builders, etc., moved out so that they can be pulled from both the IDE process and the CLI). |
| 45 | 2. Takes **explicit context** as input: at least `--workspace` (workspace root), if necessary, the path to the solution, flags “which tool to emulate” and args parameters (according to the MCP scheme). |
| 46 | 3. Writes **to stdout** the same JSON (or the same **normalized** form) that would return the corresponding tool in MCP. |
| 47 | **CI:** in the root of the repository - [`.gitlab-ci.yml`](../../.gitlab-ci.yml): `dotnet build` / `dotnet test` and sequential smoke `dotnet run --project CascadeIDE -- --agent-contract ...` (expected **Windows** runner with .NET 10 SDK and tag `windows`; in a different environment - fix `default.tags`). Additionally, the repository accepts **`dotnet script`** (global `dotnet-script`, see `Financial/finplan/update-finplan-pdf.csx`, `agents-and-humans-book/update-agents-humans-pdf.csx`). To call `--agent-contract` from a pipeline with built `CascadeIDE.exe`: [`docs/samples/agent-contract-ci.csx`](../samples/agent-contract-ci.csx) - `ProcessStartInfo`, exit code, no surprises with `$LASTEXITCODE`. **PowerShell** alternative: [`docs/samples/agent-contract-ci.ps1`](../samples/agent-contract-ci.ps1) (`pwsh` 7+ or `Start-Process -PassThru.ExitCode` in Windows PowerShell 5.1). |
| 48 | |
| 49 | **Tests:** |
| 50 | |
| 51 | - **Integration / contract:** running CLI (or direct call to common API) on **fixture workspace** (minimal `.sln` / files are already in tests) → comparison with **expected JSON** or with **snapshot** with conscious update. |
| 52 | - **CDS (cockpit):** narrow **golden slice** - `schema_version` + `topology` + sorted `instruments` (`TestData/AgentContract/cockpit_surface_contract_slice.approved.json`); fields like `ui_mode` and `presentation_effective_line` are not included in the snapshot (they depend on the user's `settings.toml`). When changing the default placement or CDS scheme, update the approved file consciously. |
| 53 | - **Normalization for stability:** absolute paths → relative to fixture; cutting time/versions if necessary; or assert on a **subset of fields** (schema-shaped), if the full snapshot is too noisy. |
| 54 | - **“Rendering” at the data level:** for scenarios where MCP provides **cabin state / layout / tools** (what is consistent with what the UI draws), a snapshot of the same JSON in CI checks **representation regression** - not pixels or Skia/Avalonia-pipeline, but **what exactly should be in which slot and with what identifiers**. This is a meaningful "UI vs agent see the same thing" verification layer, without duplicating a separate "tests only" model. |
| 55 | |
| 56 | **Negative:** |
| 57 | |
| 58 | - **Pixels, fonts, visual regressions Skia/Avalonia** - still separate (screenshots, UI tests, manual review). CLI+MCP snapshots do not replace the render engine. |
| 59 | - Do not duplicate **business logic** “from scratch” - only a thin shell over the general code. |
| 60 | |
| 61 | ## Consequences |
| 62 | |
| 63 | | Pros | Cons/risks | |
| 64 | |--------|----------------| |
| 65 | | Contract regressions in CI without GUI | You need to remove the common layer once and make sure that MCP and CLI go to the same entry point | |
| 66 | | Documenting “what exactly” gives the tool (through tests) | Fixtures and normalization - support when changing OS/paths | |
| 67 | | Quick feedback when refactoring JSON collectors | Scope on tools to grow in stages - otherwise there will be a big explosion of work | |
| 68 | | Checking the consistency of **cockpit/UI semantics** with what the agent sees (single source in JSON) | Risk of mixing layers: keep in snapshots fields that are actually shared with the UI, and not the entire response if it is noisy | |
| 69 | |
| 70 | ## Implementation stages (recommendation) |
| 71 | |
| 72 | 1. Select **one** tool with stable JSON (for example, a layout snapshot or one read-only script). |
| 73 | 2. Create a **single** function “build payload” + “in JSON string”. |
| 74 | 3. Add a minimal **CLI** (`dotnet run --project ...` or a separate `net10.0` tool). |
| 75 | 4. Cover with **one** snapshot test; then expand the list of commands. |
| 76 | |
| 77 | ## Acceptance and open questions |
| 78 | |
| 79 | **There are no open questions** - the direction is fixed for implementation. |
| 80 | |
| 81 | Next: add commands to `AgentContractRunner.TryGetJson`, calling the same collectors as `IIdeMcpActions` / MCP; if there is a significant expansion, update the table here and in [MCP-PROTOCOL.md](../../MCP-PROTOCOL.md). |