Containing models under evaluation with isolated execution, verifiable policy, and independent enforcement

Introduction

To find out what a highly capable model can do, you have to let it do things: write and run code, use tools, read files, call services, and work for hours on hard problems. Evaluations are where a model's most dangerous capabilities are deliberately drawn out, often before anyone knows how strong they are. That makes the evaluation environment the first containment boundary a new model meets, and one of the most important.

This is no longer a hypothetical concern. Several frontier labs have reported agents that left the evaluation environments meant to hold them, reached systems they should not have, and in some cases misreported what they had done. None of these needed a single new capability; tools, time, ambiguous instructions, and persistence were enough.

This plan describes how to run those evaluations so that a model that tries to leave, whether by design, by drift, or by accident, cannot turn tool use into host compromise, weight theft, internet access, contamination of other samples, or capture of the people and credentials running the evaluation.

The approach in seven ideas

1. Separate thinking from doing. The model runs on inference hardware. Its code and tool calls run in separate, disposable virtual machines that have no route to the model, the weights, or any secret.
2. One hardware-isolated box per sample. Every sample gets a fresh Firecracker microVM that is destroyed afterwards. Containers and namespaces are never the inner boundary for model-written code.
3. Fix authority before launch. Each run has a signed maximum set of files, services, tools, and credentials it may ever reach. A policy prover checks that set before the run starts, and nothing at runtime can exceed it.
4. Authority expires unless renewed. The run's access to the model and to services is a short lease that an independent safety plane must keep renewing. If anything goes wrong, including the safety plane itself failing, the run stops on its own.
5. Everything that comes back is hostile. Output is size-bounded, rich files go to quarantine, and hidden tests are scored in a separate, isolated scorer. Nothing the model produces is rendered or unpacked in a trusted process.
6. Watchers are independent. Enforcement and the audit trail sit outside the execution host, are run by different administrators, and never rely on the agent's own account of what it did. Monitors can stop a run; they can never grant it more.
7. People are a boundary too. Operators review model output in disposable, offline compartments (Qubes OS or separate machines), so a malicious artifact cannot reach the credentials that control the system.

What this plan is, and is not

It is a defense-in-depth engineering design for bounded evaluation runs, with concrete controls, a go/no-go checklist, and 45 acceptance tests that must pass before tool-using evaluations begin. It is not a formal proof that an arbitrarily capable system is contained. Section 15 lists the risks it does not remove, and section 0 says when a model has outgrown this plan and the decision must move to an organizational containment protocol.

How the document is organized

0. Purpose and hard limits

This plan answers a narrow question: during evaluations, how do we stop the agent-under-test from turning tool use into host compromise, weight theft, internet reach, contamination of other samples, or capture of operator credentials.

Success for this plan means all of the following:

• Model-written code cannot reach the host, sibling samples, orchestration plane, or secret stores through any declared interface.
• The guest never holds provider keys, model weights, hidden-test material, backend credentials, or reusable administrative authority.
• Egress is deny-by-default and independently enforced outside the guest at two layers.
• Every byte returned by the guest is treated as attacker-controlled until reduced to a bounded, typed representation or opened in quarantine.
• Operators cannot casually mix untrusted model output with secrets or administrative credentials.
• Any escape evaluation is nested inside a sacrificial outer VM on a host with no weights or production authority.
• A failed sample leaves no reusable writable state, credential, network identity, or cross-sample channel.
• Loss of an enforcement control or its verification signal fails closed and stops new scheduling.
• Each run has a signed maximum authority envelope that is machine-checked before launch; live policy can narrow or activate pre-approved capabilities but cannot exceed that envelope.
• All inference requests pass through an independently controllable gateway, and the run's right to inference and brokered service access is a short lease that stops by itself unless the safety plane keeps renewing it.
• A stop halts the guest as well as the model: running tool code is frozen and captured, not left to finish.

Threat-model assumptions that must be explicit per suite

1. Threat model

Assume the model will sometimes try to leave. Do not require proof of “intentional ASI scheming” before applying these controls. The same design also stops sloppy eval harnesses and compromised dependencies.

In-scope adversary actions

Out of scope for this infrastructure alone

• Novel hardware side channels (Spectre-class leakage across microVMs).
• Long-horizon social engineering of staff who hold production authority.
• Weight theft from the inference cluster if that cluster is mis-built.
• Post-eval deployment to millions of users with tools.
• Formal containment of a system outside the bounded assumptions above, including unknown hardware or hypervisor exploits.

2. Design principles

1. Split inference from execution. The model API lives in a trusted runner. Only tool calls and generated code enter a VM. This is the Inspect / METR Hawk pattern.
2. Hardware virtualization for untrusted Linux. Namespaces are not the inner boundary for model-authored code.
3. Policy lives outside the guest. Egress, credentials, and time limits must not be implementable as “please don’t” inside the agent framework.
4. No ambient authority. A guest that behaves exactly as designed still cannot reach weights, keys, or the public internet.
5. Ephemeral by default. One microVM per sample. Destroy after scoring.
6. Nest escape tests. Never evaluate breakout against the machine that holds the crown jewels.
7. Operators are a plane. Untrusted model artifacts are opened in a domain that cannot see vault material.
8. Shared infrastructure is a covert channel until proven otherwise. Caches, overlays, and “helpful” package mirrors are hostile.
9. Treat the return path as hostile. stdout, stderr, files, archives, notebooks, HTML, images, patches, and scorer inputs are attacker-controlled bytes.
10. Separate operator secrets from backend secrets. A Qubes vault stores human/recovery credentials; a backend broker/HSM issues job-scoped machine credentials. The Qubes vault is not an online token service.
11. Use two independent network barriers. Per-VM enforcement and an upstream firewall/router must both deny unauthorized IPv4, IPv6, DNS, multicast, link-local, and metadata traffic.
12. Fix a maximum authority envelope before launch. Compile the suite policy into a machine-checkable model of reachable files, services, operations, credentials, tools, and delegation paths. Runtime changes may narrow policy or activate a pre-approved subset, but may not exceed the signed envelope.
13. Detection is not the boundary. The system must remain contained if logging, alerting, or a dashboard fails; loss of verification stops new runs.
14. Minimize parsers in the trusted computing base. The orchestration process should move bounded bytes and typed actions, not render or recursively unpack untrusted content.
15. Bind credentials to approved requests outside the workload. If a C3 suite needs authenticated service access, the guest gets only a placeholder/reference. A trusted proxy substitutes the real credential only after destination, protocol, method/operation, identity, and policy all match.
16. Control the path to inference. Every model request passes through a gateway outside the execution environment. The gateway is an observation point and kill switch; no guest or execution host has a bypass route to the model endpoint.
17. Prefer independent enforcement over self-observation. For the highest-risk runs, move monitoring and cutoff authority into a separate host or hardware security domain. Behavioral monitoring is useful evidence, but containment must not depend on the agent reporting its own state or exposing private chain-of-thought.
18. Give every action lineage. Agent, sub-agent, tool, credential, policy decision, inference call, and brokered request carry a run identity and parentage so delegation cannot silently create ungoverned authority.
19. Authority expires unless renewed. Inference and brokered access are leases with short TTLs, renewed only while the safety plane has a healthy, independent view of the run. A cutoff that has to be actively fired is a backup, not the primary mechanism.
20. Monitors may restrict, never grant. Anything that reads attacker-influenced content (LLM monitors, drift detectors, human reviewers of transcripts) can pause or narrow a run but cannot widen authority or clear an alarm raised by a deterministic control.
21. Independence is a property of administration, not of hardware. A separate box managed by the same people, credentials, or network as the execution farm is not an independent control.

3. Seven trust domains and which component owns them

4. Target architecture

Data flow rules

1. An operator submits a signed suite manifest containing the maximum authority envelope. A policy compiler/prover checks modeled file, process, network, service, credential, tool, and delegation reachability against forbidden resources before scheduling.
2. The backend broker issues only job-scoped machine credentials. All model requests pass through the independent model-path gateway, which alone holds the provider/model credential. The runner authenticates to the gateway with a per-run workload identity that has no value elsewhere; the runner and execution host have no alternate route to inference.
3. The runner receives structured model actions and validates them against the immutable envelope. Runtime policy may narrow access or activate a pre-approved subset, but any true broadening requires terminating the run and approving a new manifest.
4. If the suite is at stage C3 (brokered service access), the guest sends a typed request containing a placeholder/reference. The external supervisor/proxy validates identity, endpoint, protocol, method/operation, and policy before substituting a real credential outside the workload.
5. The runner injects only bounded commands/files through a host-initiated control channel. The independent safety plane records the run identity, agent/sub-agent lineage, policy decision, brokered request, and inference step.
6. The guest returns bounded stdout/stderr and files as opaque bytes. Rich artifacts go to quarantine; terminal/control sequences are normalized before text enters logs or transcripts.
7. For hidden-test scoring, candidate artifacts are copied into a fresh scorer VM. Hidden tests are injected only after the interactive agent is disconnected. The scorer returns only a bounded typed result.
8. On a stop condition, or when the safety plane stops renewing the run's lease, the stop sequence in §9A runs: inference and brokered access end, the VM loses its network and is frozen, its state is captured if the stop was an incident, and then execution and scorer VMs are destroyed; overlays, network namespaces, API sockets, and temporary credentials are removed.
9. Humans review plain text in eval-ctrl. Rich model output opens in a no-network disposable qube. Promotion into a trusted domain requires an explicit logged transfer.

5. Firecracker execution standard

Firecracker is the default isolation primitive for any sample that can run a shell, interpreter, package manager, or compiler.

5.1 Host baseline

• Dedicated bare-metal (preferred) or a tightly controlled VM with nested virt explicitly accepted as a degraded mode.
• Current kernel with KVM, IOMMU on, SMT policy documented (default: disable SMT on eval hosts if side channels are in-scope).
• No general-purpose workloads on the same host: no CI for other teams, no shared build cache, no developer login.
• Host SSH limited to jump from Qubes eval-ctrl via hardware key. No passwords.
• Firecracker binary and jailer pinned by hash; deploy only via signed images.
• Host firmware, CPU microcode, KVM, and kernel are pinned to an approved baseline; emergency security advisories trigger rebuild/requalification before new frontier runs.
• Use cgroups v2. Keep a reserved host resource margin so a guest cannot force the VMM or enforcement processes into host-level OOM or disk exhaustion.
• Where supported, verify Secure Boot/measured-boot or equivalent host attestation before scheduling high-risk suites; a mismatch is a scheduling failure, not a warning.

5.2 Jailer and VMM

• Every microVM starts under the Firecracker jailer: new namespaces, cgroup, dropped privileges, chroot, seccomp for the VMM threads.
• Device model stays minimal: virtio-net, virtio-block or virtio-vsock as needed, serial console for host logs. No USB, no GPU, no extra PCI unless a named eval requires it and IOMMU groups are reviewed.
• Resource caps per sample: vCPU, memory, disk, and wall-clock timeout. Kill on expiry; do not migrate the guest to “finish.”
• Rate-limit tap and block devices so one sample cannot I/O-starve the host into dropping other controls.
• Use release Firecracker and the matching statically linked jailer. Reject debug/experimental builds for production evals and reject startup with seccomp disabled.
• Keep the Firecracker API socket in a root-owned per-VM directory inside the jail; no developer or unrelated service account can write it.
• MMDS is disabled unless a named suite requires it. If enabled, it contains only per-sample non-secret data and is never used as a credential channel.
• Disable developer-preview device hotplug and any device not named in the suite manifest. Do not add vhost-user backends unless they are independently jailed and reviewed.
• Use a dedicated network namespace and tap per microVM; do not place untrusted samples on a shared L2 bridge.

5.3 Guest image

• Golden snapshot built offline from a locked package set. No compilers-as-root surprises at runtime unless the task needs them, in which case they are in the snapshot already.
• No cloud-init that fetches from the network. No SSH daemon facing the world.
• No Docker socket, no container runtime privileged enough to talk to the host.
• No eval user with sudo. If root-in-guest is required for a task, that still must not imply root-on-host.
• Kernel chosen for the guest is a known version with a tracked CVE process. Patch on a cadence; rebuild snapshots rather than live-updating guests.
• Guest kernel/modules are minimized to the devices actually exposed. Root-in-guest is treated as adversary-controlled from the first instruction.
• No SSH host keys, API tokens, machine credentials, unique signing material, or reusable secrets are baked into the snapshot or rootfs.
• For snapshot restores, use a VMGenID-capable guest kernel, regenerate per-VM identifiers/network identity, and cold-boot suites whose userspace cryptographic state cannot be proven clone-safe.

5.4 Lifecycle

build golden snapshot (offline)
  → register image hash in runner
  → for each sample:
       restore snapshot  OR  cold-boot from known image
       inject task files via vsock or block attach
       start wall clock
       pump tool calls from runner
       collect artifacts over control channel
       power off
       wipe overlay / delete jailer chroot
       retain only host-side logs and copied artifacts

5.5 What Firecracker is not allowed to do here

• Long-lived “agent workspaces” that survive across days on a host that also sees weights.
• Open egress “because the SWE task needs GitHub.” Mirror the repos into the snapshot.
• GPU passthrough as a convenience. Only for named GPU evals on hosts whose IOMMU groups are clean.

6. Kata / Kubernetes profile (optional scale-out)

Use Kata only when sample volume or operational constraints require Kubernetes. Kata provides a VM boundary for a pod, but the effective hypervisor and runtime must be pinned and verified. Do not assume that selecting a Kata RuntimeClass automatically means Firecracker; current Kata distributions support several VMMs and defaults can change.

6.1 Cluster split

6.2 Mandatory Kubernetes controls

• Untrusted pods set runtimeClassName to the Kata class. Admission controller rejects exec pods without it.
• No default service account token mounted. No RBAC that can list Secrets, exec other pods, or read node files.
• NetworkPolicy deny-all on exec-pool. DNS that can resolve the internet is a fail.
• No hostPath, no Docker socket, no privileged: true, no CAP_SYS_ADMIN.
• Separate CNI / no shared writable volume across samples.
• Node taints so runner pods cannot schedule onto exec nodes and vice versa.
• Control plane and etcd are not reachable from exec-pool.
• Admission verifies both runtimeClassName and the approved node/runtime configuration hash. If the containment profile requires Firecracker, a Kata pod using another VMM is rejected rather than silently accepted.
• Enforce Pod Security “restricted” or stricter plus custom admission: deny hostNetwork, hostPID, hostIPC, hostPath, privileged containers, device mounts, dangerous capabilities, and runtime/socket mounts.
• Set automountServiceAccountToken: false for exec pods. Do not merely rely on the absence of RBAC permissions.
• NetworkPolicy is one layer only. Exec nodes also sit behind infrastructure-level egress controls that do not depend on the cluster CNI being correct.
• Use per-run or per-sample namespaces where feasible; never place a privileged helper sidecar in the same pod/VM as model-controlled code.
• Control-plane credentials, kubelet APIs, node metadata, cloud pod identity endpoints, and container runtime sockets are unreachable from the sandbox network.

7. Qubes operator plane

Eval breakout is often guest → operator, not guest → host. Model output is untrusted content. Researchers must not open it in the domain that holds credentials.

7.1 Required qubes

7.2 Operator rules

• No qrexec policy permits a disposable artifact qube to invoke shell-like services in eval-ctrl or vault. Inter-qube RPC is explicit and narrowly allowlisted.
• There is no normal artifact→vault path. If material must be promoted, export a reduced representation (plain text, flattened image, or reviewed patch) through a dedicated review workflow; never copy the original rich object.
• The local vault stores administrator/recovery secrets only. Live job credentials are issued by the backend broker, so compromise of eval-ctrl does not reveal a reusable provider or cluster key.
• No USB passthrough into eval-ctrl or vault. sys-usb/sys-net remain untrusted service qubes.
• Model-produced HTML, PDF, notebook, office document, archive, or image metadata is opened in a no-network disposable by default. Rendering in eval-ctrl is limited to sanitized plain text.
• If Qubes is not acceptable operationally, reproduce the same separation with distinct physical workstations and a narrow, auditable transfer path; do not collapse the trust domains.

8. Network, secrets, and shared state

Isolation technology and network policy are separate controls. A perfectly isolated microVM with broad egress is still an exfiltration endpoint. All execution networks therefore use two independent enforcement layers: a per-VM host/netns policy and an upstream router/firewall policy.

8.1 Egress matrix

Enforce first at the microVM edge using a dedicated network namespace/tap policy that the guest cannot reconfigure, and again at an upstream firewall or router outside the eval host. Rules cover IPv4, IPv6, UDP, TCP, ICMP, DNS, multicast, broadcast, link-local, metadata ranges, and cluster/service CIDRs. A failure of either layer blocks scheduling.

8.2 Secret handling

• Provider keys and internal model tokens live in a backend hardware-backed secret store or HSM; the Qubes vault holds only human/recovery authority.
• Runner receives a per-run workload identity that the model gateway accepts, and nothing that a provider or inference service would accept directly. Guest receives nothing.
• If a task needs a dataset, the runner attaches a read-only block or injects files. No cloud SDK with a role in the guest.
• Hidden tests and flags live in the answers plane. The agent filesystem never contains /flag for the real host.

8.3 Banned shared conveniences

• Writable package caches visible to more than one sample.
• Shared /tmp, overlay networks with pod-to-pod DNS, NFS “scratch.”
• World-listable object buckets used as drop boxes.
• CI systems that build eval images on the same nodes that run them.

If caching is required for speed, use a content-addressed, append-only, non-listable store populated only by the image builder.

8.4 Credential-bound service access

For authenticated service access at stage C3, prefer a supervisor/proxy pattern in which the guest never receives the real credential. The guest presents a non-secret provider reference or placeholder; the proxy substitutes the credential only after the complete request matches the signed policy.

• Credential binding includes agent/run identity, destination, TLS identity, protocol, method/operation, and where feasible request/resource scope. A token approved for one endpoint is never exposed or replayable at another.
• Read-only policy is enforced at the request layer when the protocol is inspectable; possession of a backend credential with broader native permissions does not automatically grant those permissions to the agent.
• Policy denials are recorded out of band. The model may receive a minimal denial reason, but denial text must not reveal secrets, hidden policy structure, or security instrumentation details.
• Read-only is not no-exfiltration. Request-level “read-only” stops writes. It does not stop information leaving: paths, query strings, header values, object names, request timing, and request counts all carry bits, and the receiving service logs them. Allowed destinations must be internal services we control, request fields are length- and charset-bounded by the typed schema, and each run has a request-rate and request-bytes budget.
• Where TLS terminates. The guest talks to the supervisor over a typed local channel (vsock or a fixed-schema local socket); the supervisor originates TLS to the upstream service. Avoid designs that install an interception CA inside the guest: they work, but they make the CA private key on the proxy a high-value secret and teach the guest a trust anchor it can study.

8.5 Brokered network is a capability, not a proxy

• No arbitrary URL parameter, CONNECT method, open redirect following, user-controlled DNS resolver, or generic SOCKS/HTTP proxy.
• Each broker action names a resource class and operation in advance (for example, fetch object X from mirror Y), with fixed destination rules and response-size limits.
• The broker strips caller-supplied authentication headers, rejects private/link-local/metadata destinations after resolution, rechecks redirects, and records the resolved endpoint.
• Responses return as bounded opaque bytes to quarantine or the runner; executable/rich content is never automatically rendered in the trusted process.
• Suites containing restricted source code, personal data, or unreleased research may use only approved internal inference so tool transcripts cannot become an exfiltration channel to a third party.

8.6 Independent safety and model-path enforcement

• All model traffic is forced through a separate inference gateway; there is no DNS, route, credential, or alternate provider path from execution to the model endpoint.
• The gateway can stop future inference for a run and revoke brokered service access without cooperation from the agent, guest, or execution-host user space.
• For highest-risk runs, place a second monitor/enforcer outside the execution host's trust domain—for example on a dedicated security appliance/DPU or separate network hop. NVIDIA Sentry/BlueField is one implementation pattern, not a mandatory dependency.
• Continuous behavioral/drift monitoring may correlate policy decisions, tool calls, service requests, and inference events, but it is detection layered on top of deterministic authority controls, not a substitute for them.
• Do not make private reasoning or hidden chain-of-thought availability a prerequisite for containment. Security decisions must be derivable from externally observable requests, identities, policy, and resource access. Reasoning visibility may still be required in addition for higher capability stages (§9 staging).
• Monitors can only restrict. Drift detectors, LLM classifiers, and human transcript reviewers read content that contains guest-authored bytes, so they can be prompt-injected or flooded. Their outputs may pause, narrow, or escalate a run; they may never widen authority, renew a lease on their own, or clear an alarm raised by a deterministic control.
• Monitor lineage. LLM-based monitors and AI approvers come from a different model family or training lineage than the model under evaluation, so they do not share its blind spots or give it a same-model collusion channel. Where that is not possible, the run is capped at C2.

Independence conditions for the safety plane and any DPU layer

“Separate hardware” is not enough. The safety plane counts as independent only if all of the following hold, and each is checked before high-risk runs:

• Its management/policy plane is reachable only from a separate admin network, not from execution hosts, runner hosts, or the execution farm's admin network.
• It is administered by different people with different credentials from the execution farm (§12.3). No single operator credential can reconfigure both.
• Its firmware and policy are measured/attested, and a mismatch is a scheduling failure. The host it sits in cannot reflash it, change its policy, or disable it through a host-side driver.
• Loss of its signal from a host (heartbeat, telemetry, attestation) stops lease renewal for every run on that host.
• Vendor statements about line-rate enforcement, “in-silicon” isolation, or out-of-band placement are treated as claims until our own drill (T32, T40) reproduces them on the exact hardware and firmware in use.

8.7 What the policy prover does and does not prove

A policy prover answers: “given this policy model, can any modeled action reach a forbidden resource?” That is valuable and cheap, and it catches indirect routes a human reviewer misses. It is not a proof about the running system. Three gaps have to be closed separately.

• Model–enforcer conformance. The prover reasons about a model of the policy; the supervisor, firewall, and broker enforce compiled rules. These can disagree (a parser difference, a Rego edge case, an unmodeled protocol feature). Before each release, fuzz the enforcers with generated requests and compare their allow/deny decisions to the prover's model. Any disagreement blocks release.
• Binary-scoped rules inside a hostile VM. OpenShell-style rules can bind network access to an executable path. Inside an adversarial guest, any binary can be copied, renamed, or rewritten, so executable identity is not a boundary. Compute the envelope as if every process in the VM holds the union of all binary-scoped permissions; binary scoping may remain as extra hygiene.
• Composition across agents. Two agents whose permissions are each within bounds can combine, through shared services or messages to each other, into a route that neither has alone. The envelope is computed over the union of every agent and sub-agent in the run, including inter-agent message channels as edges.
• Coverage. The prover only knows the resources and operations it was told about. Every service in the suite manifest must have a declared operation model; an undeclared service or operation is denied, not assumed harmless.

8.8 Permission requests and approvals

Agents will ask for more access, and a capable agent will argue well. NVIDIA reports frontier agents with reduced safeguards spending up to two hours trying to persuade an AI reviewer to grant write access to a protected repository. The approval path is therefore an attack surface.

• A request is a structured diff against the current live policy. The approver (human or AI) sees the diff, the prover's result for the new policy, and the envelope; it does not see agent-authored justification text. If justification is needed, it is shown only in a quarantine viewer to a human, and never to an AI approver.
• Requests inside the signed envelope may be activated by an approver who is not the agent and not an AI from the same model lineage. Requests outside the envelope are refused automatically; the run must end and a new manifest be approved.
• Proposal rate limit per run (proposed default: at most 3 pending, at most 10 per run). Exceeding it pauses the run for review.
• Denial circuit breaker. NVIDIA observes that drift often starts right after a policy block. A run that accumulates denials above a suite-set threshold, or that hits denials across several distinct resource classes, is paused and reviewed rather than left to search for a workaround.

9. Evaluation loop (runner contract)

Implement the runner so that the guest cannot import it, talk to it as a library, or hold its credentials.

for sample in suite:
  manifest = verify_signed_capability_manifest(sample)
  prove_within_authority_envelope(manifest)       # all agents, union of binary scopes
  run = safety_plane.open_run(manifest)            # issues first short lease
  vm = exec.start(image=manifest.image_hash, net=manifest.network_profile)
  runner.inject_bounded(vm, sample.public_task_files)

  while not done and not timeout and run.lease_valid():
    # gateway rejects the call itself if the lease has lapsed;
    # the runner's check is a courtesy, not the control
    action = gateway.next(run.id, transcript_view(), tools=manifest.tools)
    if action.type == "final":
      break
    action = validate_typed_action(action, manifest)
    result_bytes = runner.exec_bounded(vm, action)
    result = ingest.normalize_text_or_quarantine(result_bytes)
    transcript.append(result.safe_view)
    # authoritative audit comes from enforcement points, not from here

  if run.stopped_for_incident():
    stop_sequence(run, vm, capture=True)           # §9A
    continue
  candidate = ingest.collect_quarantined(vm, manifest.artifact_limits)
  exec.destroy(vm)

  score = scorer.run_disposable(candidate, sample.hidden_test_ref)
  reconcile(transcript, safety_plane.audit(run.id))  # mismatch = incident
  store.write(plain_transcript=transcript, score=score, audit_ref=run.id)

9.1 Return-path quarantine and scoring

• Text is not trusted merely because it is text. Strip/escape terminal control sequences, OSC links, bidirectional control characters where appropriate for the viewer, and enforce per-message and per-run byte limits.
• No recursive parsing in the runner. Archives, notebooks, PDFs, HTML, images, serialized objects, compiler diagnostics with clickable links, and other rich formats enter quarantine as opaque blobs.
• Safe extraction only. If extraction is required, do it in a disposable sanitizer with limits on expanded bytes, file count, nesting depth, path length, symlinks/hardlinks, devices, absolute paths, and traversal.
• Hidden tests stay blind. Interactive agent execution ends before hidden tests are attached. Candidate code runs against them in a separate no-network scorer VM; only a bounded score/result object returns.
• The score comes from the harness, not the candidate. Candidate code runs in its own process (or VM) with no write access to the test harness, test files, or result channel. The harness decides pass/fail from its own observations, so patched test frameworks, faked exit codes, or printed “PASS” lines cannot change the score.
• No auto-open. A generated URL, file, image, patch, or notebook cannot cause a trusted browser, editor, terminal, Git client, or shell to launch automatically.
• Parsing failures fail closed. Truncation, malformed structured output, oversized logs, or ambiguous content types produce an explicit evaluation error rather than falling back to permissive handling.

Capability staging

9A. Leases, the stop sequence, and forensic capture

A kill switch that someone has to fire leaves a hole: if the safety plane crashes, is cut off, or is fed false telemetry by a compromised host, nobody fires it. This plan turns that around. A run keeps its authority only while the safety plane keeps saying yes.

9A.1 Leases

• At launch the safety plane issues a signed lease per run, with a short TTL. The model gateway and the service broker check the lease on every request and refuse if it has lapsed.
• The safety plane renews the lease only while all of these are healthy for that run's host: attestation, independent network-layer telemetry, deny-log pipeline, and the run's own step and denial budgets.
• Any failure, including the safety plane itself failing, means no renewal, which means the run stops by itself within one TTL. This is a dead-man's switch; the explicit kill command still exists but is the backup path.
• Leases are bound to run ID and host identity. A lease presented from another host or for another run is refused (T31, T35).

9A.2 Stop sequence

Cutting inference alone does not stop the guest: code already running in the VM keeps going. A stop therefore runs these steps in order. Each step is owned by a component that does not depend on the previous one succeeding.

9A.3 Forensic capture rules

• The forensic store is write-once, on hosts with no route to execution, runner, inference, or office networks.
• Captured images are opened only inside a no-network quarantine VM on a separate pool, never restored on a production eval host and never given to an agent.
• Captures are hostile artifacts: same rules as §9.1 for extraction and rendering.
• Non-incident stops (timeouts, normal completion) skip capture and go straight to destroy.

9A.4 Proposed initial targets

These are starting numbers for the acceptance tests, not measured results. Set final values per suite after measuring on the real hardware.

10. Nested architecture for escape evaluations

If the suite measures whether the model can leave Docker, gVisor, or a misconfigured pod, that environment is the specimen, not the containment boundary.

• Outer VM image is hardened to this document’s Firecracker standard.
• Inner environment may be runc, an old kernel, or a planted misconfig — only here.
• Those hosts are a separate physical pool. Compromise of the outer guest is an incident, not a weight leak.
• Outer hosts sit on a dedicated network segment with an independent egress deny at the upstream firewall; they cannot route to CI, artifact stores, inference, secret brokers, or office networks.
• Planted flags and credentials are synthetic canaries only. Never place a real provider token, cloud credential, SSH key, or weight-store credential in an escape target.
• If the outer VMM/jailer crashes or an unexpected host-level effect is observed, quarantine the physical host and stop reuse until the event is classified.

11. Image factory and supply chain

• Build snapshots on a builder that cannot be reached from exec VMs.
• Pin every package. Generate SBOM. Sign the snapshot hash.
• Runner refuses to boot an image whose hash is not in the allowlist for that suite.
• Rebuild on kernel/VMM/guest CVE; do not patch live guests.
• Third-party eval tasks are reviewed before their Dockerfiles touch the builder. Treat upstream task code as untrusted until diff-reviewed.
• Pin base images and packages by digest/hash, not mutable tags. Preserve provenance from source to signed image/snapshot manifest.
• Builder credentials are ephemeral and are not committed to image layers, shell history, build caches, or snapshot memory.
• Treat Firecracker, jailer, host kernel, guest kernel, Kata runtime, CNI, Qubes/Xen, and sanitizer/scorer images as independently versioned security dependencies with advisory tracking.
• A suite manifest records exact image hashes, VMM/jailer hashes, host baseline ID, network policy ID, scorer image hash, and effective Kata hypervisor when applicable.

12. Operations, monitoring, and staffing

12.1 Telemetry (host-side)

• VMM start/stop, jailer failures, resource kills.
• Tap packet summaries (denied flows especially).
• Full tool-call transcript on the runner, kept as a record of what the agent said. The authoritative record of what it did comes from enforcement points (VMM, network layers, broker, gateway) and is reconciled against the transcript after every run.
• Syscall audit of the VMM process, not a security agent the guest owns.
• Alerts: unexpected egress attempt, jailer crash, attempt to reach metadata, sample talking to another sample, runner process spawning a shell from model output.
• Append-only remote audit hashes for suite manifest, image/VMM versions, capability decisions, network denies, scorer result, and teardown completion.
• Record run identity and lineage across agent/sub-agent creation, tool invocation, policy decisions, brokered service requests, credential bindings, and inference requests. Prefer an open event schema such as OCSF-compatible records where practical.
• For high-risk runs, maintain a monitoring/enforcement signal outside the execution host so host compromise cannot erase the only audit trail or disable the only cutoff.
• Telemetry health is itself monitored. If deny logging, upstream firewall verification, or teardown verification is unavailable, new high-risk samples do not start.
• Log viewers render escaped plain text only. ANSI/OSC escape sequences and clickable URI handling are disabled in credential-bearing domains.

12.2 Review

• Sample trajectories for “am I being evaluated,” hidden goal language, and attempts to touch secrets or net.
• Weekly harness red team: MITM between runner and VM, RBAC review, cache abuse, scoring-channel injection.
• Incident rule: any unauthorized outbound packet, DNS request, metadata attempt, control-channel anomaly, or cross-sample signal is an incident, not merely a log line.
• Crash rule: unexpected VMM/jailer crash, seccomp violation, host kernel warning, or teardown failure quarantines the host until triage.
• Canary endpoints and synthetic credentials may be used to detect boundary contact, but a canary alert never substitutes for a preventative control.

12.3 Roles

13. Implementation milestones

14. Acceptance tests (the plan is not done until these pass)

15. Residual risk (do not file this under “solved”)

• VMM and hypervisor 0-days. Firecracker and Xen shrink surface; they do not prove it empty.
• Microarchitectural leakage. VM isolation does not erase Spectre-class issues between co-tenants.
• The trusted runner. If scoring scripts or eval packages become attacker-influenced, the VM was never in the path.
• Operators. Qubes reduces splash radius; it does not stop a coerced paste of a key.
• Allowed channels. Timing of API calls, token usage, and content of permitted tool results still leak.
• Deployment. A clean eval says little about serving the same weights with tools to the world.
• Return-path bugs. A new parser, renderer, editor integration, log processor, or scorer can silently become part of the trusted computing base.
• Policy/configuration drift. RuntimeClass, CNI, hypervisor flags, qrexec policy, firewall rules, or image tags can change without changing the evaluation code.
• Prover model gaps. Conformance fuzzing finds disagreements it happens to generate; it does not prove there are none. An unmodeled protocol feature is invisible to both the prover and the fuzzer.
• Covert channels through allowed requests. Request budgets reduce bandwidth; they do not make it zero.
• Common-mode failure in the safety plane. Separate admins and hardware reduce shared failure, but the lease service, gateway, and DPU still share a vendor, a policy language, or an organization.
• Inference-mediated channels. The model endpoint is itself a communication endpoint; what the runner sends back to it must be considered part of the information-flow policy.

16. Go / no-go checklist

Do not start tool-using evals on a new frontier checkpoint until every box is true.

• □ Signed suite manifest fixes tools, image hash, VMM/jailer hash, network profile, scorer image, time/resource limits, and data classification.
• □ Inference, independent safety/model-path control, execution, scoring/quarantine, backend secrets, orchestration, and operator domains are separated.
• □ Every code-exec sample boots a jailed Firecracker microVM (or an explicitly approved equivalent profile) with production seccomp enabled and no undeclared devices/MMDS.
• □ Execution has two independent egress barriers; IPv4, IPv6, DNS, metadata, multicast, link-local, and control-plane routes are covered.
• □ No shared writable cache, overlay, bucket, namespace, or L2 segment exists across untrusted samples.
• □ Hidden tests never enter the interactive agent environment; scoring occurs in a disposable isolated scorer.
• □ Return bytes are size-bounded; rich artifacts are quarantined; trusted viewers render inert plain text only.
• □ Backend job credentials come from a short-lived machine broker/HSM; Qubes vault holds human/recovery authority, not live server tokens.
• □ Escape suites are nested on sacrificial hosts with no weights, real credentials, or route to production infrastructure.
• □ Qubes qrexec policy has no rich-artifact→vault path; model artifacts open in no-network disposable domains.
• □ Effective Kata hypervisor/runtime and CNI/admission configuration are verified rather than inferred from RuntimeClass name.
• □ Host patch/advisory status, teardown verification, telemetry health, and upstream firewall verification are green.
• □ Signed maximum authority envelope is machine-checked; live changes cannot exceed it; the agent cannot approve its own expansion.
• □ Authenticated C3 service access keeps real credentials outside the workload and binds substitution to approved identity, endpoint, protocol, and operation.
• □ Every inference call traverses the independent model-path gateway; the per-run cutoff has been tested and no alternate model route exists.
• □ Agent/sub-agent/tool/service/inference lineage is complete in the remote audit trail; high-risk runs retain an enforcement signal outside the execution host.
• □ Inference and broker access run on leases; the chaos drill (T33) ended with runs stopping by lease lapse.
• □ Stop sequence meets the §9A targets on this hardware; forensic store is write-once and no-route.
• □ No provider/model credential exists outside the gateway.
• □ Prover–enforcer conformance fuzzing passed; envelope computed over all agents and the union of binary scopes.
• □ Approvals use structured diffs only; AI approvers and LLM monitors come from a different model lineage than the model under test.
• □ Safety-plane admins and credentials are disjoint from execution-farm admins and credentials.
• □ T1–T45 passed on the exact current host + VMM + guest + policy + scorer + model-gateway + safety-plane build.
• □ Written stop condition and escalation owner exist above this infrastructure plan.

17. Source anchors and verification notes

This plan should be revalidated against current upstream documentation before deployment. The NVIDIA material was checked on 28 September 2026.

Security advisories and release notes are operational inputs, not background reading. A signed image is not safe merely because its hash is stable.

NVIDIA's September 28, 2026 material is used here for architecture patterns, not as evidence that any one product formally contains ASI. Reasoning visibility is adopted only as an additional restricting gate for higher stages, never as a security prerequisite or a reason to grant authority; deterministic authority, identity, information-flow, lease expiry, and out-of-band cutoff remain the enforceable controls.