fmt, clippy -D warnings and tests on hanzo-build-macos-arm64 with a
cargo check -p vm-linux leg on hanzo-build-linux-arm64; v* tags build
the codesigned CLI and the guest OS image and upload them as release
assets with gh.
Co-authored-by: Hanzo Dev <[email protected]>
Crates keep their history: vm (lib, was shuru-vm), hanzo-vm (CLI package
and binary, was shuru-cli), vm-guest, vm-proto, vm-proxy, vm-sdk,
vm-store, vm-darwin, vm-linux; all at 0.1.0, Apache-2.0, hanzoai/vm.
Config is vm.json, data lives in ~/.hanzo/vm (HANZO_VM_HOME), env vars
are HANZO_VM_*, the guest runs as /usr/bin/vm-guest, and assets download
from hanzoai/vm releases. The TypeScript SDK is @hanzo/vm. NOTICE
credits shuru as the origin. prepare-rootfs.sh builds the image natively
on arm64 Linux with mke2fs -d and re-execs in Docker only on macOS.
Includes rustfmt over the tree and fixes for clippy 1.98 lints.
Co-authored-by: Hanzo Dev <[email protected]>
The www/ Astro site, skills/, .claude/, GitHub workflows and changelogs
belong to the upstream project; CI moves to .hanzo/workflows.
Co-authored-by: Hanzo Dev <[email protected]>
Substitution was a blind byte replace over the whole guest to upstream
stream, which failed two ways. A placeholder straddling two reads went
upstream unsubstituted, reproduced on an 80KB upload where 3 of 2000 leaked.
And substituting in a body changed its length while leaving the framing that
declared it, so any request carrying a secret in its body hung or was closed.
Measured across framings: with a secret the same length as the placeholder
every shape returned 200, and with a differing length only the header case
did. Since placeholders are a fixed 30 bytes and no real credential is 30
bytes, body substitution has never worked for anyone, and no comparable
sandbox proxy offers it.
Substitution now applies to the request line and any header value, scoped by
position rather than by a list of header names, so an unknown vendor header
works with no configuration and a credential in a query parameter is covered.
Bodies stream through untouched, so no declared length is ever invalidated.
This is deterministic rather than timed. An unterminated head is
unambiguously an unfinished request, where an arbitrary byte offset is not,
so no flush heuristic is needed at any throughput.
Framing is read but never rewritten, only to find where the next head begins
on a reused connection. Anything RFC 9112 6.3 flags as a possible smuggling
attempt, both Content-Length and Transfer-Encoding, conflicting lengths, an
unparseable head, CONNECT or Upgrade, stops substitution and degrades to a
byte tunnel. An intermediary that never re-frames cannot desynchronise its
upstream by re-framing wrongly.
Also cheaper than before: only heads are scanned, not every byte of every
body.
A live relay additionally re-resolves its secrets every 5s, so a rotation
reaches a connection that is already open instead of waiting for it to
reconnect.
RFC 0002 is rewritten to match, and records the evidence, the rejected
options, and a separate defect found on the way: substitution is one
directional, so an upstream that reflects a request returns the real
credential to the guest.
Verified end to end on real VMs: header substitution with a length-changing
secret, an unknown custom header, a 40KB POST body forwarded untouched, five
requests substituted on one reused connection, a placeholder in a body
carried through verbatim with a 200 instead of a hang, and no placeholder
leaked in any case.
Secrets resolved from the host environment are frozen when the VM boots, so
a credential with a short lifetime (a GitHub App installation token lasts an
hour) strands any task that outlives it. A secret can now name a command
that mints its value instead:
"GITHUB_TOKEN": {
"command": ["./scripts/mint-installation-token.sh"],
"hosts": ["api.github.com", "github.com"]
}
The command prints {"version":1,"value":"...","expires_at":"..."} and is
re-run about a minute before the value it minted expires. Following
credential_process and git credential helpers, the provider reports its own
expiry rather than the config guessing a ttl; `ttl` covers providers that
cannot. argv is never passed to a shell, and the working directory is the
folder holding shuru.json.
- resolution moves to a SecretResolver behind the existing per-connection
call site, so a rotation is picked up by the next connection to the domain
and the placeholder in the guest is untouched
- single flight: the cache lock is held across the provider run, so a burst
of connections to one expired secret mints once
- command backed secrets fail closed, and the proxy warns with the provider
stderr; forwarding a placeholder upstream only yields a 401 that reads
like a scope problem. env backed secrets keep skipping substitution on a
missing variable, so existing configs behave as before
- values live in memory only, and neither stdout nor a parsed value reaches
a log, an error message, or a Debug impl
Breaking for direct consumers of shuru-proxy and shuru-sdk:
SecretConfig.from is now Option<String>. Use SecretConfig::from_env or
SecretConfig::from_command. Bumps shuru-proxy to 0.3.0, shuru-sdk to 0.4.0.
Implements docs/rfcs/0002-refreshable-secrets.md, closes#40.
Spec for backing a secret with a command instead of a host env var, so
short lived credentials (GitHub App installation tokens) survive a long
running sandbox session.
- provider contract follows credential_process: argv array, JSON on
stdout with an optional expires_at, no shell in the path of a secret
- resolution stays per TLS connection, which is where the substitution
list is already captured, so the diff is a resolver with a cache
rather than a lock free read in the relay loop
- command backed secrets fail closed; env backed secrets keep the
current skip behaviour so nothing that works today starts failing
Closes nothing yet; tracks superhq-ai/shuru#40.
Boot path:
- direct clonefile disk is the default; CAS NBD is opt-in via
SHURU_STORAGE=cas, and .idx checkpoints force NBD automatically (also
fixes SHURU_STORAGE=direct booting a zero placeholder disk from a .idx
checkpoint)
- vsock readiness poll at 10ms granularity, same 10s budget
- unused virtio balloon device removed from the VM configuration
- cmdline gains init=/usr/bin/shuru-init so no-initramfs boots land on the
right init
- SHURU_BOOT_TRACE=1 prints per-stage host boot timings
Correctness:
- checkpoint create now syncs the guest before stopping the VM; late guest
writes previously vanished from saved checkpoints and journaled images
replayed recovery on every boot from them
Guest:
- boot stage logs carry monotonic timestamps
- PID 1 stdio hardened against a missing console
Add a host-to-guest ingress primitive so external code can reach a TCP
port inside a running sandbox without binding a local listener.
- shuru-vm: factor the vsock forward handshake into open_forward_stream
and expose Sandbox::connect_forward, returning the connected stream
directly (works with or without --allow-net; no guest changes).
- shuru-sdk: add AsyncSandbox::dial_guest and a GuestStream wrapper.
This is the seam the neko tunnel builds on: dial_guest(port) yields a
byte stream to the guest service that can be bridged onto any transport.
Bump shuru-vm and shuru-sdk to 0.3.7.
check_size_against_backend enforced disk_size <= fb.size(), but the
virtual disk is intentionally larger than the flat file (sparse
expansion). Correct invariant is fb.size() <= disk_size — flat file
cannot exceed the virtual disk it backs.
This caused all checkpoints created with default disk-size (4GB) to
fail on next boot with 'index disk_size exceeds fallback file size',
making checkpoints broken by default.
Also updates the test to cover the real-world case: flat file smaller
than virtual disk must be valid, not an error.
Co-authored-by: Hanzo Dev <[email protected]>
ChunkIndex::load() now checks that num_chunks is consistent with
disk_size, and callers validate disk_size against the actual backing
store size before use. Prevents out-of-bounds reads when an index
file contains invalid length fields.
Co-authored-by: Hanzo Dev <[email protected]>
Implements the FUSE wire protocol directly over a virtio-fs request queue
and forwards each op to the host filesystem via libc/std::fs. No external
daemon (virtiofsd) or crate (fuse-backend-rs) required, matching the
hand-rolled virtio style of the other devices.
Ops covered: INIT, LOOKUP, FORGET (single + batch), GETATTR, SETATTR,
OPEN/READ/WRITE/RELEASE, OPENDIR/READDIR(PLUS)/RELEASEDIR, CREATE,
UNLINK/MKDIR/RMDIR/RENAME/RENAME2, STATFS, FLUSH, FSYNC/FSYNCDIR,
ACCESS, xattr stubs. Anything else returns ENOSYS.
Mounts plumb through VmCreateConfig.mounts and land as one virtio-mmio
virtio-fs device per mount. Read-only flag is honored at the op layer
(SETATTR/WRITE/CREATE/UNLINK/... return EROFS).
Verified on AWS a1.metal:
- --mount ./dir:/workspace -> guest reads/ls host files, guest
writes stay in the tmpfs overlay.
- --mount ./dir:/workspace:rw -> guest writes land on host with
correct contents.
Without an RTC device, the guest clock stayed at Unix epoch (1970-01-01),
which caused every TLS handshake inside the sandbox to fail with "certificate
not yet valid". macOS gets this for free from Virtualization.framework; Linux
needs us to expose the device ourselves.
Minimal read-only PL031: RTCDR/RTCLR return host wall clock seconds, the
counter is always enabled, no alarm IRQ. The guest kernel already ships with
CONFIG_RTC_DRV_PL031 and CONFIG_RTC_HCTOSYS, so the system clock is seeded at
boot from /dev/rtc0.
Verified on AWS a1.metal (us-east-1): guest date matches host, HTTPS returns
200 where it used to error with "cert not yet valid".
- shuru-linux backend wired into shuru-vm unconditionally on Linux targets
- install.sh detects Linux ARM64 and installs the matching tarball
- shuru upgrade picks the right tarball per host (darwin or linux aarch64)
- Release workflow builds both darwin-aarch64 and linux-aarch64 CLI tarballs
- Crates-publish workflow includes shuru-linux
- www: Linux section in install box, /linux setup guide, blog post, banner
- README and CHANGELOG updated for the 0.6.0 launch
The guest's tar extractor unconditionally stripped the first path component,
which silently dropped flat tarballs (e.g. OpenAI Codex's single-binary
release) on the floor while the download reported success. Callers had no
way to opt out.
Replace with an explicit `strip_components: u32` field on `DownloadRequest`,
mirroring `tar --strip-components=N`. Default 0. Callers spell out the value
that matches their archive.
- shuru-proto 0.3.2 -> 0.3.3: DownloadRequest gains strip_components.
- shuru-guest 0.3.4 -> 0.3.5: use req.strip_components instead of hardcoded 1.
- shuru-vm 0.3.3 -> 0.3.4: sandbox.download() takes strip_components.
- shuru-sdk 0.3.5 -> 0.3.6: AsyncSandbox::download() takes strip_components.
- shuru-cli 0.5.8 -> 0.5.9: release bump so a new shuru-os rootfs ships.
The `--allow-host` domain allowlist only blocked DNS resolution, not
direct TCP connections. A guest that knew a target IP could bypass the
allowlist entirely (e.g. `curl --resolve` with a hardcoded IP).
This commit closes three egress gaps:
1. DNS-pinning: when DNS resolves an allowed domain, record the
resulting IPs in a shared set. Reject TCP connections to IPs not
in that set. This blocks hardcoded-IP bypass.
2. SNI filtering: on TLS connections, verify the SNI hostname matches
an allowed domain before proxying. This prevents accessing
non-allowed services that share an IP with an allowed domain
(e.g. Cloudflare-fronted sites).
3. ICMP drop: filter ICMP frames at the device level before they
reach smoltcp, which otherwise auto-replies to echo requests for
any IP (due to any_ip=true).
All three checks are only active when `network.allow` is non-empty
(i.e. a domain allowlist is configured). Without an allowlist,
behavior is unchanged.
shuru-guest now watches the overlay mount directly instead of the upper
dir. Modern kernels (Linux >= 5.11) propagate inotify events from the
upper through the merged view, so we get proper CREATE/MODIFY/REMOVE
events on the caller's namespace paths — no whiteout char device
shenanigans, no recursive-watch race compensation, no path rewriting.
Also filters out Access events (open/read/close) since only content
changes matter.
shuru-sdk wraps the VM sandbox in Arc and fans ReadFile/ReadDir onto
worker threads. Each call already opens its own vsock connection on the
host side and the guest spawns a thread per connection, so the
previously single-threaded dispatch was serializing concurrent reads
for no reason. Bursts of file status probes now pipeline.
- Add AsyncSandbox::add_port_forward() for runtime port forwarding
- Use socket2 SO_REUSEADDR on port forward listeners to prevent
bind failures when forking from checkpoints
- Add OVERFLOW watch_kind for inotify queue overflow detection
- Guest sends OVERFLOW event on notify Flag::Rescan
Adds `env: HashMap<String, String>` to SandboxConfig, merged into the
base environment for all commands. Config env is applied first, then
proxy placeholders override, then caller extra_env overrides both.
zeekayandHanzo Dev