Stuffing a Real Browser into a VRChat World

vrc-browser-bridge stuffs a real browser into a VRChat world and uses it as a screen. VRChat’s Udon sandbox has no WebView, no CEF, can’t embed Chromium, and can’t even open a live socket — an embedded browser in the standard sense simply does not exist in this environment. The project’s approach is to work around the limits by keeping the browser outside: run a real Chromium locally, push its picture as a low-latency RTSP stream into the world as a screen, and forward the player’s in-world actions back to drive it. Chromium is always the sole authority on page state; what you see in the world is just its projection plus a layer of remote control.

Broken down, it’s two streams plus one loop. The control direction: mock or Udon events come in over WebSocket, and I catch them with Playwright/CDP and land them into Chromium. The media direction: Chromium’s picture goes through FFmpeg to MediaMTX, gets pushed as RTSP, and is played by AVPro inside the world. There’s also a log loop: VRChat writes content into output_log, a Companion tails it, scoops out the lines prefixed with VRCBRIDGE, and sends them back to the control server. Going through the log is because Udon has no proper outbound network channel — the log turns out to be the only stable place it can speak to the outside world.

The event protocol is factored out into @vrcbb/protocol. Every message is JSON, carries a fixed v: 1, a sessionId, and a monotonically increasing seq, with coordinates normalized to 0..1. Incoming events first pass validation, then get deduplicated and ordered by sessionId + playerId + source + seq. In a multiplayer environment, out-of-order and resent events are the norm; skip this layer and everything downstream turns into voodoo bugs. Control splits into open and locked: open accepts all valid events, while locked requires grabbing the control lock first (lock_request / lock_release with a 30-second TTL) — a public screen can’t have several people clicking at once. The operations cover pointer move/down/up, scroll wheel, text submit, Enter/Escape/Tab/Ctrl+L, URL submit, back/forward/reload, and quality at 360p/720p/1080p/auto switchable at runtime. The control server also exposes /health, /media-state, and /snapshot; snapshot just spits out a JPEG for easy debugging.

Two points worth noting. First, I built a mock input page early on, so the local loop can run closed without connecting to VRChat — saving me from strapping on a headset and going into the world to verify every line I change. Second, on Windows the PowerShell execution policy blocks the .ps1 shims, so I use npm.cmd / npx.cmd to bypass it; FFmpeg and MediaMTX aren’t bundled but treated as external binaries, and there’s a --dry-run that only prints the commands without executing them.

The repo is an npm workspaces monorepo, TS strict throughout, with control-server / companion / media-pipeline / mock-input each as its own app and the protocol as a separate package. It’s still WIP: the local loop works, the media stream and log path are being wired up incrementally, and the versions are all still 0.1.0.

Code is here: https://github.com/dwgx/vrc-browser-bridge