Lesson 12: Sandbox Deployment Options — From Cloud Providers to Self-Hosted Kubernetes
This lesson covers the four main options for deploying agent sandboxes: cloud-hosted providers, custom Kubernetes backends, self-hosted NemoClaw/OpenShell on Kubernetes, and hybrid approaches. We compare security features, credential handling, and when to use each option.
Primary sources: LangChain DeepAgents Sandboxes | OpenShell Kubernetes Setup | OpenShell Helm Chart
1. The Four Deployment Options
Four Deployment Options Overview
2. Option A: Cloud Sandbox Providers (LangChain Native)
Cloud-Hosted Sandboxes External
LangChain DeepAgents integrates with multiple cloud sandbox providers:
- E2B — Cloud sandbox with Python focus
- Modal — Serverless containers with GPU support
- Daytona — Development environments with git integration
- Vercel — Edge-optimized sandboxes
- LangSmith Sandbox — First-party managed sandboxes
- Runloop — Devbox-based execution
Architecture
Cloud Provider Architecture
Code Example
# Using LangSmith Sandbox with DeepAgents from deepagents import create_deep_agent from deepagents.backends import LangSmithSandbox from langsmith.sandbox import SandboxClient client = SandboxClient() ls_sandbox = client.create_sandbox() backend = LangSmithSandbox(sandbox=ls_sandbox) agent = create_deep_agent( model="anthropic:claude-sonnet-4-20250514", system_prompt="You are a coding assistant with sandbox access.", backend=backend, ) try: result = agent.invoke({ "messages": [{"role": "user", "content": "Create and run a Python script"}] }) finally: client.delete_sandbox(ls_sandbox.name)
What You Get
| Feature | Status | Notes |
|---|---|---|
| Filesystem isolation | Yes | Agent can't read your local files |
| Shell execution | Yes | Via execute() method |
| Process isolation | Yes | Container-level |
| File upload/download | Yes | Seed and retrieve artifacts |
| Binary-scoped network policy | No | Can't say "only git can reach github.com" |
| Credential injection at runtime | No | Must inject into sandbox (security risk) |
| Kernel-level enforcement | No | Container-level only |
| OPA policy language | No | Provider-specific configuration |
"Never put secrets inside a sandbox. API keys, tokens, database credentials... can be read and exfiltrated by a context-injected agent."
When to Use
- Quick prototyping and development
- When you don't want to manage infrastructure
- When basic filesystem/process isolation is sufficient
- When you can keep credentials outside via external tools
3. Option B: Custom Kubernetes Backend
DIY Kubernetes Sandbox Self-Hosted
Implement your own SandboxBackendProtocol to run agent commands in Kubernetes pods you control.
Architecture
Custom K8s Backend Architecture
Implementation
from deepagents.backends.sandbox import BaseSandbox, ExecuteResult import kubernetes class KubernetesSandbox(BaseSandbox): """Custom sandbox backend that runs commands in Kubernetes pods.""" def __init__(self, pod_name: str, namespace: str = "default"): self.pod_name = pod_name self.namespace = namespace self.k8s_client = kubernetes.client.CoreV1Api() def execute(self, command: str, timeout: int = 300) -> ExecuteResult: """Execute command in the K8s pod via kubectl exec.""" response = kubernetes.stream.stream( self.k8s_client.connect_get_namespaced_pod_exec, self.pod_name, self.namespace, command=["/bin/bash", "-c", command], stdout=True, stderr=True, stdin=False, tty=False, ) return ExecuteResult( output=response, exit_code=0, # Parse from response )
Pod Template with Security
# sandbox-pod.yaml apiVersion: v1 kind: Pod metadata: name: agent-sandbox-${SESSION_ID} namespace: sandboxes spec: securityContext: # Seccomp profile for syscall filtering seccompProfile: type: Localhost localhostProfile: agent-sandbox-seccomp.json containers: - name: sandbox image: python:3.12-slim securityContext: # Drop dangerous capabilities capabilities: drop: - SYS_ADMIN - SYS_PTRACE - NET_RAW - SYS_MODULE - MKNOD - SETUID - SETGID readOnlyRootFilesystem: true allowPrivilegeEscalation: false runAsNonRoot: true runAsUser: 1000 volumeMounts: - name: workspace mountPath: /sandbox - name: tmp mountPath: /tmp volumes: - name: workspace emptyDir: sizeLimit: 10Gi - name: tmp emptyDir: sizeLimit: 1Gi
NetworkPolicy for Egress Control
# network-policy.yaml apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: sandbox-egress namespace: sandboxes spec: podSelector: matchLabels: app: agent-sandbox policyTypes: - Egress egress: # Allow only specific destinations - to: - ipBlock: cidr: 10.0.0.0/8 # Internal services only ports: - protocol: TCP port: 5432 # PostgreSQL
What You Get
| Feature | Status | Notes |
|---|---|---|
| Filesystem isolation | Yes | Pod-level isolation |
| Shell execution | Yes | Via kubectl exec |
| Process isolation | Yes | Container + seccomp |
| Capability drops | Yes | Via securityContext |
| Network policy | Partial | Per-pod, not per-binary |
| Binary-scoped network | No | K8s NetworkPolicy is pod-level |
| Credential injection | DIY | Build with sidecar proxy |
| Landlock LSM | DIY | Requires kernel 5.13+ |
When to Use
- You already have Kubernetes infrastructure
- You need more control than cloud providers offer
- Basic K8s security primitives are sufficient
- You can build credential injection yourself
4. Option C: Self-Hosted OpenShell on Kubernetes
Full OpenShell Stack Self-Hosted
Deploy NVIDIA OpenShell's complete security stack on your own Kubernetes cluster using the official Helm chart.
oci://ghcr.io/nvidia/openshell/helm-chart and runs a K3s cluster inside a single Docker container. The gateway image is available for linux/amd64 and linux/arm64.
Architecture
Self-Hosted OpenShell on Kubernetes
ghcr.io/nvidia/openshell/gateway"] OPA["Policy Engine
OPA/Rego"] IR["Inference Router
inference.local handler"] PKI["PKI Init Job
mTLS certificates"] end subgraph SANDBOX_NS["Sandbox Pods - sandboxes namespace"] SB1["Sandbox Pod 1"] SB2["Sandbox Pod 2"] SBN["Sandbox Pod N"] subgraph INSIDE_POD["Inside Each Pod"] SUP["Supervisor"] AGENT["Agent Process"] PROXY["Policy Proxy Sidecar"] end end subgraph YOUR_APP["Your Application"] APP["Your Services"] CB["Context Builder"] PG["Policy Gateway"] end end APP --> CB CB --> GW GW --> SB1 GW --> SB2 GW --> SBN SUP --> AGENT AGENT --> PROXY PROXY --> OPA AGENT -->|"inference.local"| IR SB1 --> PG PG --> APP style CONTROL_PLANE fill:#dcfce7,stroke:#16a34a style GW fill:#4ade80 style OPA fill:#4ade80 style IR fill:#4ade80
OpenShell Security Layers
From the OpenShell documentation:
| Layer | Component | Function |
|---|---|---|
| Process | Supervisor | Drops privileges, applies identity rules, disables escalation paths, spawns agent as restricted child |
| Filesystem | Landlock LSM | Pre-start policy application; undeclared paths blocked (kernel-enforced) |
| Network | Policy Proxy | All egress through proxy; evaluates destination, port, binary identity, L7 rules |
| Credentials | Gateway | Controlled delivery via policy paths or proxy rules |
| Inference | Inference Router | Intercepts inference.local, hides provider credentials |
| Observability | Supervisor | Security/lifecycle logs, relay endpoints |
Helm Installation
# 1. Install Agent Sandbox CRDs (prerequisite) kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/manifest.yaml # 2. Create namespace kubectl create namespace openshell # 3. Install OpenShell from OCI registry helm upgrade --install openshell \ oci://ghcr.io/nvidia/openshell/helm-chart \ --version "0.1.0" \ --namespace openshell \ --set server.sandboxNamespace=sandboxes \ --set server.oidc.issuer="https://your-idp.example.com/realms/openshell" # 4. Wait for rollout kubectl -n openshell rollout status statefulset/openshell # 5. Port-forward for local access kubectl -n openshell port-forward svc/openshell 8080:8080
Key Helm Values
# values.yaml for production replicaCount: 2 # HA requires external PostgreSQL workload: kind: deployment # Use deployment with external DB server: logLevel: "info" sandboxNamespace: "sandboxes" sandboxImage: "ghcr.io/nvidia/openshell-community/sandboxes/base:latest" # External PostgreSQL for multi-replica externalDbSecret: "openshell-pg-credentials" # OIDC authentication oidc: issuer: "https://your-idp.example.com/realms/openshell" audience: "openshell-cli" adminRole: "openshell-admin" userRole: "openshell-user" # AppArmor profile for sandbox containers appArmorProfile: "runtime/default" # Supervisor topology supervisor: topology: "sidecar" # or "combined" sidecar: processBinaryAwareNetworkPolicy: true proxyUid: 1337 # Network policy for sandbox pods networkPolicy: enabled: true
Compute Drivers
OpenShell supports four compute drivers for provisioning sandboxes:
| Driver | When to Use | Security Features |
|---|---|---|
| Docker | Local development, single-machine gateways | Container isolation, mount restrictions, GPU via CDI |
| Podman | Rootless container requirements on Linux | Rootless isolation, same mount restrictions as Docker |
| MicroVM | VM-level isolation, enhanced security, GPU workloads | Full VM boundary, nftables per TAP interface, immutable root disk |
| Kubernetes | Shared clusters, remote compute, GPU scheduling | RBAC, AppArmor, ServiceAccount tokens, PVC isolation, Landlock |
Supervisor Topologies
Supervisor Topologies Comparison
for network, Landlock, etc."] C_AGENT --> C_CAP end subgraph SIDECAR["Sidecar Topology"] S_INIT["Init Container
nftables setup"] S_AGENT["Agent Container
runs as sandbox UID"] S_SIDECAR["Sidecar
binary-aware policy"] S_INIT --> S_AGENT S_AGENT --> S_SIDECAR end style COMBINED fill:#e8f5e9,stroke:#76b900 style SIDECAR fill:#e3f2fd,stroke:#1a5fb4
Combined: Agent container has Linux capabilities for network namespace, Landlock, privilege changes.
Sidecar: Agent runs with no added capabilities; root init container does nftables; sidecar handles binary-aware network policy.
What You Get
| Feature | Status | Notes |
|---|---|---|
| Filesystem isolation | Yes | Landlock LSM (kernel-enforced) |
| Binary-scoped network | Yes | CONNECT proxy + OPA policies |
| Credential injection | Yes | inference.local pattern |
| Capability drops | Yes | 15 capabilities dropped by supervisor |
| Syscall filtering | Yes | Seccomp-bpf via supervisor |
| OPA policy language | Yes | Rego policies for all layers |
| Blueprint versioning | Yes | SHA-256 verified, immutable |
| Multi-tenant | Partial | Namespace isolation; shared nodes need care |
When to Use
- You need binary-scoped network policies
- You need credential isolation via
inference.local - You need kernel-level enforcement (Landlock, seccomp)
- You're running on Linux 5.13+ and control the kernel
- You want OPA policies for fine-grained control
5. Option D: Hybrid — OpenShell + DeepAgents
Best of Both Worlds Hybrid
Use DeepAgents for orchestration with OpenShell as the sandbox backend. This is the official integration path.
Architecture
Hybrid Architecture: DeepAgents + OpenShell
AGENTS.md, Skills, Tools"] CB["Context Builder"] PG["Policy Gateway
Deterministic"] end subgraph OPENSHELL["OpenShell Runtime"] GW["OpenShell Gateway"] SUP["Supervisor"] PROXY["Policy Proxy"] IR["Inference Router"] end subgraph SANDBOX["Sandbox"] AGENT["Agent Process"] end end HARNESS --> CB CB --> GW GW --> SUP SUP --> AGENT AGENT --> PROXY AGENT -->|"inference.local"| IR AGENT -->|"structured output"| PG PG --> HARNESS style DEEPAGENTS fill:#fef3c7,stroke:#f59e0b style OPENSHELL fill:#dcfce7,stroke:#16a34a style GW fill:#4ade80
Code Example
# Using OpenShell as DeepAgents backend import openshell from deepagents import create_deep_agent from langchain_nvidia_openshell import OpenShellSandbox # Create OpenShell sandbox with openshell.Sandbox(delete_on_exit=True) as sandbox: backend = OpenShellSandbox(sandbox=sandbox) # Create DeepAgent with OpenShell backend agent = create_deep_agent( model="anthropic:claude-sonnet-4-20250514", system_prompt=load_skill("coding.md") + load_agents_md("client_a"), backend=backend, ) result = agent.invoke({ "messages": [{"role": "user", "content": "..."}] }) # Policy Gateway (outside sandbox) decision = policy_gateway(result, client_rules)
Why This Combination Works
From the LangChain blog:
"Deep agents connect to real data sources (Linear, Slack, Salesforce). You want the agent to learn new things on the fly, but you don't want to rely on prompt instructions alone to prevent misuse — something external to the agent has to enforce security, which is what OpenShell does."
| Component | From DeepAgents | From OpenShell |
|---|---|---|
| Orchestration | Yes | |
| AGENTS.md / Skills | Yes | |
| LangSmith tracing | Yes | |
| Human-in-the-loop | Yes | |
| Memory (SQLite) | Yes | |
| Kernel-level isolation | Yes | |
| Binary-scoped network | Yes | |
| Credential injection | Yes | |
| OPA policies | Yes |
6. Comprehensive Comparison
| Feature | Cloud Providers (E2B, Modal) |
Custom K8s Backend |
Self-Hosted OpenShell |
Hybrid (DeepAgents+OpenShell) |
|---|---|---|---|---|
| Infrastructure | External | Your K8s | Your K8s | Your K8s |
| Setup complexity | Low | Medium | High | High |
| Filesystem isolation | Container | Container | Landlock (kernel) | Landlock (kernel) |
| Network policy | All or nothing | Per-pod | Per-binary + L7 | Per-binary + L7 |
| Credential isolation | Must inject | DIY sidecar | inference.local | inference.local |
| Syscall filtering | No | Basic seccomp | Full seccomp-bpf | Full seccomp-bpf |
| Capability drops | Provider default | Configurable | 15 dropped | 15 dropped |
| Policy language | Provider-specific | K8s YAML | OPA/Rego | OPA/Rego |
| Harness tuning | DIY | DIY | DIY | Built-in |
| HITL support | Basic | DIY | Gateway level | Full |
| Observability | Provider logs | K8s logs | Supervisor logs | LangSmith + Supervisor |
| GPU support | Some providers | K8s GPU plugin | CDI / nvidia.com/gpu | CDI / nvidia.com/gpu |
| Linux kernel req | N/A | Any | 5.13+ (Landlock) | 5.13+ (Landlock) |
7. Decision Framework
Decision Flowchart: Choosing a Deployment Option
network policy?"} Q1 -->|"No"| Q2{"Need credential
isolation?"} Q1 -->|"Yes"| OPENSHELL["Option C or D:
Self-Hosted OpenShell"] Q2 -->|"No"| Q3{"Want to manage
infrastructure?"} Q2 -->|"Yes"| Q4{"Can build
sidecar proxy?"} Q3 -->|"No"| CLOUD["Option A:
Cloud Providers"] Q3 -->|"Yes"| CUSTOM["Option B:
Custom K8s Backend"] Q4 -->|"No"| OPENSHELL Q4 -->|"Yes"| CUSTOM OPENSHELL --> Q5{"Need DeepAgents
orchestration?"} Q5 -->|"Yes"| HYBRID["Option D:
Hybrid Approach"] Q5 -->|"No"| PURE["Option C:
Pure OpenShell"] style CLOUD fill:#dbeafe,stroke:#3b82f6 style CUSTOM fill:#dcfce7,stroke:#16a34a style PURE fill:#dcfce7,stroke:#16a34a style HYBRID fill:#fef3c7,stroke:#f59e0b
Quick Decision Table
| If you need... | Choose... |
|---|---|
| Quick setup, no infra management | Option A: Cloud Providers |
| K8s control, basic security | Option B: Custom K8s Backend |
| Binary-scoped network, credential isolation | Option C: Self-Hosted OpenShell |
| Full security + harness tuning + HITL | Option D: Hybrid |
| Financial/compliance workloads | Option C or D (kernel-level enforcement) |
8. Kubernetes-Specific Considerations
Node Requirements for OpenShell
# Label nodes that support Landlock (kernel 5.13+) kubectl label nodes node-1 openshell.nvidia.com/landlock=true # Check kernel version kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.nodeInfo.kernelVersion}{"\n"}{end}'
Multi-Tenancy
- Landlock provides kernel-level isolation, but multiple sandboxes on the same node share kernel resources
- For hard multi-tenancy, consider dedicated node pools per tenant
- Use Kubernetes namespaces + RBAC for tenant isolation at the control plane level
- Consider Kata Containers or gVisor for additional VM-level isolation
Production Checklist
- Verify nodes have Linux kernel 5.13+ (for Landlock)
- Install Agent Sandbox CRDs
- Configure external PostgreSQL for HA (not SQLite)
- Set up OIDC provider for authentication
- Configure TLS certificates (or use cert-manager)
- Define OPA policies for network/filesystem rules
- Test supervisor topology (combined vs sidecar)
- Monitor: gateway logs, supervisor logs, OPA decisions
Knowledge Check
Question 1: What is the key security limitation of cloud sandbox providers (E2B, Modal, etc.)?
Correct Answer: B - Cloud sandbox providers offer container-level isolation but cannot provide binary-scoped network policies ("only git can reach github.com") or credential injection at runtime (credentials must be injected into the sandbox, creating a security risk).
Question 2: What Linux kernel version is required for Landlock LSM support in OpenShell?
Correct Answer: C - Landlock LSM requires Linux kernel 5.13 or newer. Compatible systems include Ubuntu 20.04+, RHEL 8+, Rocky Linux 8+, Amazon Linux 2023+, and Fedora 32+.
Question 3: What is the purpose of the inference.local endpoint in OpenShell?
Correct Answer: B - The inference.local endpoint routes model traffic to configured backends while keeping provider credentials outside the sandbox. The supervisor intercepts requests, the inference router injects credentials and forwards to providers, and the agent never sees the raw API keys.
Question 4: What prerequisite must be installed before the OpenShell Helm chart?
Correct Answer: B - OpenShell uses the Agent Sandbox Kubernetes SIG project to provision sandbox pods. You must install the Agent Sandbox controller and its CRDs before installing the OpenShell Helm chart.
Question 5: When should you choose the Hybrid approach (DeepAgents + OpenShell)?
Correct Answer: C - The Hybrid approach combines DeepAgents' orchestration capabilities (AGENTS.md, Skills, HITL, LangSmith tracing) with OpenShell's kernel-level security (Landlock, binary-scoped network, credential isolation). This is ideal for production workloads requiring both flexibility and strong security.
Summary
Key Takeaways
- Option A (Cloud Providers): Easy setup, but limited to container-level isolation and no credential injection
- Option B (Custom K8s Backend): Full control, but DIY security with K8s primitives
- Option C (Self-Hosted OpenShell): Full security stack with Landlock, binary-scoped network, credential isolation via
inference.local - Option D (Hybrid): Best of both worlds — DeepAgents orchestration + OpenShell security
- OpenShell requires: Linux kernel 5.13+, Agent Sandbox CRDs, external PostgreSQL for HA
- Supervisor topologies: Combined (agent has capabilities) vs Sidecar (agent runs unprivileged)
- For financial/compliance workloads: Choose Option C or D for kernel-level enforcement