An Nvidia Engineer Built a Talking Haunted House Using Five AI Models on One PC

Key points

• Nvidia's Magpie TTS Zeroshot, a text-to-speech model that clones a voice from a short audio sample rather than weeks of recorded training data, powers the house's live pirate voice across seven distinct emotional moods.
• One workstation fitted with two Nvidia RTX PRO Blackwell graphics cards runs the full pipeline, covering speech recognition, a language model, voice synthesis and face animation, with no cloud connection required.
• Nemotron 3.5 Bolt E4B 4GB, an open small language model built for local conversational agents, sits at the centre; the builder tested more than a dozen models before choosing it over larger alternatives including Qwen3 30B-A3B.
• Cached responses cut the typical reply time from 1.70 seconds to 0.72 seconds, keeping the house's grumbling on pace with impatient eight-year-olds.

An Nvidia engineer spent Halloween behind a garage door that could argue back. Called MonsterHouse, the project lives on Hugging Face: a talking projection of a grumpy pirate ship reincarnated as suburban real estate, built from five Nvidia Nemotron AI models running entirely offline.

Visitors who skip the word "please" get brushed off. Those who use it get candy and a warm goodbye. The garage door stays won over once it's won over, because nobody wants to spend three minutes bargaining with a house.

How does a haunted house actually run AI?

Five models share the workload. Nemotron Speech ASR, a speech-recognition model that converts spoken words into text, listens at the door. Bolt E4B 4GB, a small language model compressed to run on a laptop-class chip, writes each reply. Magpie TTS Zeroshot, which needs only a short audio clip to imitate a speaking style, delivers the line. Audio2Face-3D animates the wooden mouth in Unreal Engine 5. A fifth model, Nemotron 3 Nano Omni, watches the driveway through a camera and describes costumes; YOLO and ByteTrack count and follow visitors.

All five run on two Nvidia RTX PRO Blackwell GPUs, the specialised chips that handle AI's heavy number-crunching, inside one machine. Nothing goes to the internet.

What makes the voice work?

Magpie TTS Zeroshot is the component worth watching. Conventional voice-cloning tools require hours of recorded speech. Magpie needs three to ten seconds per mood. The builder recorded himself doing happy pirate and angry pirate impressions, one short clip per emotional state, and the model handles the rest at runtime.

We first covered Magpie TTS Zeroshot on 10 August 2026, and MonsterHouse is the first public demonstration that folds it into a single consumer-facing, fully offline application alongside other Nemotron components.

The builder trained a LoRA adapter, a compact set of extra instructions layered onto an existing model, on more than 5,000 hand-reviewed pirate conversations. That adapter is what makes Bolt E4B sound like a ship rather than a customer-service bot. Training took one to two hours per model.

For anyone curious about Magpie TTS Zeroshot, Nvidia's build.nvidia.com listing offers free API access for developers, though rate limits apply.

The builder's own verdict: the Nemotron Voice Agent turned what looked like the hard part into the easy part. Giving the door a wooden mouth that moved convincingly took longer than teaching it what to say.

What to watch next

The real story here isn't a Halloween trick. It's that a solo engineer assembled a responsive, multi-model voice agent from open components in his spare time, entirely on local hardware. The rough edges, a 300-millisecond silence wait before the system knows you've stopped speaking, a wooden mouth harder to rig than the AI behind it, are the honest markers of where offline, multi-model pipelines actually stand today. The gap between "runs" and "runs smoothly" is still measured in small frustrations, but it's closing faster than most people expect.