docs/infrastructure-plugins.md

# Infrastructure plugins

The batch controller does not launch workers directly. It resolves the
`infrastructure.backend` in a batch configuration and delegates admission,
launch-command construction, shared-process execution, trace publishing, and
final cleanup to that backend.

The built-in `aws` plugin preserves the existing EC2 + SSM behavior. External
packages can provide workers on another cloud, an on-premises Windows pool, or
a local virtualization system without changing the benchmark harness or
evaluator.

AWS dependencies are optional in packaged installations:

```bash
pip install "autocad-bench[aws]"
```

The base controller, harness, plugin API, and resume planner import without
`boto3` or `botocore`; a third-party backend supplies only its own SDK.

## Configuration

`backend` defaults to `aws` for compatibility, but every AWS resource is
operator-supplied. There are no repository-owned resource defaults:

```json
{
  "infrastructure": {
    "backend": "aws",
    "broker_version": "windows-autocad-2019-v10",
    "image_id": "ami-...",
    "subnet_ids": ["subnet-..."],
    "security_group_id": "sg-...",
    "instance_profile_name": "AutoCADBenchWorkerProfile",
    "instance_type": "g4dn.xlarge",
    "aws_region": "us-east-1"
  },
  "evaluation": {
    "bucket": "my-autocad-bench-bucket"
  }
}
```

`aws_profile` is optional. When omitted, boto3 and the AWS CLI use their
standard credential chain. Automatic AWS evaluation is enabled by default and
requires an explicit bucket in the operator's account. Set
`"evaluation": {"enabled": false}` only when deliberately running without the
trusted automatic evaluator.

Third-party backends put backend-owned values in `settings`:

```json
{
  "infrastructure": {
    "backend": "my-windows-pool",
    "broker_version": "windows-autocad-2019-v10",
    "settings": {
      "pool": "autocad-2019",
      "controller_url_env": "WINDOWS_POOL_URL"
    }
  }
}
```

Do not put credentials in `settings`; reference environment variables or the
backend's native credential provider.

## Plugin contract

Implement `autocad_bench.infrastructure.InfrastructurePlugin`. The controller
calls its methods in this order:

1. `validate_spec` checks configuration without changing external state.
2. `preflight` performs read-only capacity and worker-image checks.
3. `build_rollout_command` wraps the infrastructure-neutral harness command in
   one command that owns the worker lifecycle.
4. `run_shared` runs that command when the controller is configured for
   in-process execution. Subprocess mode executes the returned command
   directly.
5. `create_trace_publisher` optionally mirrors live artifacts when trace
   streaming is configured.
6. `reap_batch` performs idempotent best-effort cleanup after all rollouts.

The complete protocol and request models live in
`src/autocad_bench/infrastructure/`. A backend must:

- isolate simultaneous rollouts from each other;
- make the configured desktop broker reachable by the child harness;
- execute the exact `RolloutLaunchRequest.child_command`;
- preserve the requested output directory and process exit code;
- never expose evaluator inputs or credentials to the model-visible desktop;
- make cleanup idempotent, including after a partially failed launch.

The plugin receives evaluation arguments separately because the lifecycle
wrapper, not the model harness, owns trusted DWG retrieval and evaluation.
Trace publishing is enabled with `AUTOCAD_BENCH_TRACE_DESTINATION` and an
optional `AUTOCAD_BENCH_TRACE_PREFIX`. The AWS plugin also accepts the legacy
`AUTOCAD_BENCH_TRACE_S3_BUCKET` and `AUTOCAD_BENCH_TRACE_S3_PREFIX` names.

## Registration

Publish the implementation as a Python entry point whose name matches
`InfrastructurePlugin.name`:

```toml
[project.entry-points."autocad_bench.infrastructure"]
my-windows-pool = "my_autocad_backend:WindowsPoolPlugin"
```

After installing that package in the same environment as AutoCAD Bench, use
`"backend": "my-windows-pool"` in the batch config. Entry points are loaded
only when selected. Applications embedding the controller can instead use
`register_infrastructure_plugin()` for an in-process registration.

The batch state and preflight report record `infrastructure_backend`, making
mixed-backend result directories auditable.

## Interrupted-run recovery

Recovery is an optional second protocol. Backends that support
`autocad-bench-resume` implement
`autocad_bench.infrastructure.InfrastructureRecoveryPlugin` in addition to the
standard lifecycle contract:

1. `validate_recovery_spec` validates reconnect-only settings.
2. `create_recovery_session` creates the recovery control-plane client.
3. `list_active_workers` maps rollout IDs to backend worker IDs for a batch.
4. `worker_id_from_log` supports legacy runs that predate durable batch worker
   indexing.
5. `worker_status` determines whether a discovered worker can be reconnected.
6. `build_recovery_command` wraps the infrastructure-neutral harness resume
   command.
7. `terminate_worker` idempotently releases a completed or exhausted worker.

New batch states persist the selected infrastructure spec. Recovery plans copy
that spec into every entry, use the backend-neutral `worker_id`, and can contain
entries from multiple backends. AWS plans retain `instance_id`,
`instance_state`, and `instance_termination_requested` as compatibility aliases
for existing controller tooling.

A backend that intentionally does not support interrupted-session recovery can
implement only `InfrastructurePlugin`. Normal batch cleanup still works, while
`autocad-bench-resume` fails explicitly instead of applying another backend's
recovery semantics.