Lesson 4: Advanced Questions
Questions you should ask to deepen your understanding as an AI engineer
The 10 Questions for Deep Mastery
These questions go beyond the fundamentals. Each one opens a domain of investigation:
1. Performance Overhead: Sandbox vs Local
The Question
What is the latency impact of the sandbox-as-tool pattern versus local execution?
Why It Matters
Every tool call in dcode goes over the network to a remote sandbox. For a task making 100+ file operations, this adds up.
What to Investigate
- Round-trip latency for read_file, write_file, execute operations
- How does batch operation support mitigate this?
- When does local execution (Claude Code) outperform remote sandbox?
- How do different sandbox providers compare (LangSmith vs Modal vs E2B)?
Where to Look
2. Multi-Tenant Evolution
The Question
How will NemoClaw handle multi-tenant enterprise deployments once it exits alpha single-player mode?
Why It Matters
Current OpenShell is "single-player" - one developer, one environment, one gateway. Enterprise needs multiple users, shared resources, and isolation.
What to Investigate
- What changes to Gateway architecture are planned?
- How will tenant isolation work at kernel level?
- Will there be centralized policy management for organizations?
- How will credential isolation scale to hundreds of developers?
Where to Look
3. Blueprint Lifecycle Management
The Question
What is the upgrade path for blueprints when security patches are released?
Why It Matters
Blueprints are immutable once published. When a CVE affects a base image, how do you patch running agents?
What to Investigate
- How do version channels work (stable, preview, nightly)?
- Can you hot-swap blueprints without destroying state?
- What's the rollback strategy for broken blueprint updates?
- How do you test blueprint changes before production?
Where to Look
4. Custom Policy Development
The Question
How do you implement custom policy rules beyond the three preset tiers (Restricted/Balanced/Open)?
Why It Matters
Enterprise domains have unique requirements. A financial services agent might need to access Bloomberg but block social media. Presets won't cover this.
What to Investigate
- Policy file format and syntax
- Binary-scoped network rules (allow git to access github.com only)
- How to add custom filesystem paths to writable allowlist
- Testing custom policies before deployment
Where to Look
5. GPU Passthrough
The Question
How does the experimental sandbox GPU support work? What are the security implications?
Why It Matters
Local inference inside the sandbox (Ollama, vLLM) requires GPU access. But GPU passthrough creates new attack surfaces.
What to Investigate
- How is GPU memory isolated between sandbox and host?
- Can an agent leak data through GPU side channels?
- What's the performance overhead of GPU virtualization?
- How do you allocate GPU resources across multiple sandboxes?
Where to Look
OpenShell GitHub experimental features + NVIDIA GPU security documentation
6. Harness Portability
The Question
Does the harness tuning methodology developed for Nemotron 3 Ultra transfer to other model families?
Why It Matters
The "tuning the harness, not the model" approach was demonstrated on Nemotron. You need to know if this generalizes.
What to Investigate
- Which harness patterns are model-specific vs universal?
- Do Claude, GPT-4, Gemini respond differently to context engineering?
- How do you discover optimal harness configuration for a new model?
- Is there a harness tuning framework or just trial-and-error?
Where to Look
7. Failure Modes and Error Handling
The Question
What happens when an agent attempts to access a denied network endpoint or write to a restricted path?
Why It Matters
Understanding failure modes helps you design better prompts and handle edge cases gracefully.
What to Investigate
- What error message does the agent receive?
- How are policy violations logged?
- Can you configure alerts for repeated violations?
- Does the agent get a chance to retry with different approach?
Where to Look
NVIDIA OpenShell logs documentation + Policy Engine error handling
8. Local vs Cloud Inference Trade-offs
The Question
What are the trade-offs between local inference (Ollama/vLLM) versus cloud providers in NemoClaw?
Why It Matters
Local inference = no API costs, data stays on-prem. Cloud = better models, no GPU management. The math changes based on scale.
What to Investigate
- Break-even point: when does self-hosted become cheaper?
- Latency comparison: local GPU vs API call
- Model quality trade-offs at different parameter counts
- Operational complexity of managing GPU infrastructure
Where to Look
9. Evaluation Frameworks
The Question
What evaluation frameworks exist for comparing agent performance across dcode and Claude Code on identical tasks?
Why It Matters
Without standardized benchmarks, you can't make data-driven tool decisions. The NemoClaw benchmark showed 10x cost savings, but what about other metrics?
What to Investigate
- What tasks were in the benchmark suite?
- How do you create comparable test scenarios?
- What metrics beyond cost and accuracy matter (latency, reliability)?
- How do you evaluate agent safety and policy compliance?
Where to Look
LangSmith evaluations + HumanEval / SWE-Bench benchmarks
10. Long-Running Session State
The Question
How do the memory systems compare for very long-running tasks (days/weeks) in enterprise workflows?
Why It Matters
Enterprise tasks span multiple sessions, sometimes weeks. Memory must persist, be searchable, and not degrade.
What to Investigate
- How is memory stored (SQLite vs JSON vs external DB)?
- What's the memory retrieval strategy for relevant context?
- How do you handle memory conflicts across concurrent sessions?
- What's the memory size limit before performance degrades?
Where to Look
Learning Path Recommendation
Your Next Steps
Run your first sandboxed agent"] H2["Custom Blueprint
Create a domain-specific blueprint"] H3["Policy Development
Write custom network/FS rules"] H4["Benchmark
Compare dcode vs Claude Code
on your own tasks"] end L4 --> H1 H1 --> H2 H2 --> H3 H3 --> H4 style DONE fill:#e8f5e9,stroke:#76b900,stroke-width:2px style NEXT fill:#fff3e0,stroke:#f57c00,stroke-width:2px
Quick Reference Card
Print this and keep it at your desk:
NemoClaw Architecture Quick Reference
| Component | Is The | Does |
|---|---|---|
| NemoClaw | Orchestrator | Onboards, manages lifecycle, resolves blueprints |
| OpenShell | Sandbox | Enforces policies, routes inference, isolates credentials |
| OpenClaw | Agent | Runs inside sandbox, executes tasks |
| Blueprint | Config | Defines image, policy, inference profile |
| dcode | CLI | LangChain's terminal agent, any LLM |
Security Layers
Network → Filesystem → Process → Gateway → Inference
Mnemonic: "No Funny Paths Get In"
Key URLs
- NemoClaw Docs: docs.nvidia.com/nemoclaw
- DeepAgents: docs.langchain.com/oss/python/deepagents
- OpenShell: github.com/NVIDIA/OpenShell
Communities for Wisdom
Knowledge comes from docs. Wisdom comes from practitioners. Join these:
| Community | Focus | Link |
|---|---|---|
| LangChain Discord | DeepAgents, LangGraph, agent development | discord.gg/langchain |
| NVIDIA Developer Forums | NemoClaw, OpenShell, enterprise deployment | forums.developer.nvidia.com |
| r/LangChain | General LangChain discussion | reddit.com/r/LangChain |