Modding Live2DPet: A Desktop Pet That Watches Your Screen

Live2DPet is an Electron desktop pet. It keeps a character parked on your desktop, periodically grabs screenshots and reads your active window to see what you’re doing, then has an LLM generate dialogue to chat with you. You can click it, drag it, scale it, and talk to it with voice input and voice replies. This is my branch, forked from x380kkm/Live2DPet and modded from there. I picked it because it hits several things I’ve been messing with for a long time: Live2D desktop pets, local voice, cramming an LLM onto the desktop, and getting some hands-on practice with VOICEVOX, the Japanese TTS stack.

Its biggest feature is chaining local capabilities into a single pipeline: Whisper for STT, VOICEVOX Core for Japanese synthesis, plus a set of locally persisted memory, so the whole chain runs without relying on the cloud. The character can be a Live2D model or an image folder, switched automatically at runtime.

On top of the original, I mainly touched a few spots: memory now lands in local files so it survives restarts, and context is retrieved by relevance; Live2D adapts to the window and auto-centers; local Whisper runs across multiple tiers from tiny to large with automatic continuous recognition; plus multi-language copy and a cleanup of a batch of duplicate code. To be honest about one thing: the v2.0 “smart text enhancement pipeline” is paused. The skeleton is still there, but the payoff wasn’t worth it, and forcing it through was worse than stopping first.

The tech stack is the Live2D Cubism SDK plus PixiJS plus pixi-live2d-display, with a transparent, borderless, always-on-top window and eyes that track the mouse. Voice synthesis goes through VOICEVOX Core, calling the local dll directly via koffi, a Node FFI, with the GPU on DirectML. This part is the most painful: get the VVM path or the ONNX Runtime directory wrong and it won’t start. Models support hot import: pick a directory containing a .model3.json and it scans automatically, no hand-written config needed.

One heads-up: by default it screenshots at intervals and sends them to the API you configured for analysis. The screenshots aren’t written to disk, but they do pass through a third-party endpoint, so if there’s sensitive info on screen, close the window first. The vision model also costs money, so don’t set the interval too tight. To run it, grab Live2DPet.exe from Releases and double-click, or from source run npm install then node launch.js — don’t use npx electron ., it’ll collide with ELECTRON_RUN_AS_NODE. The original project is MIT and the foundation is @x380kkm’s; this branch is me carrying it forward.