DCLR's homemade Muse voice box has a wake word that trains itself

DCLR joined the Muse Agent Discord server on October 17 with a plan: to give his Meta Muse, a personal AI agent, a voice he could talk to around the house, and to “rip out my HomePods for muse.” Over the next three weeks he showed a homemade voice box, a wake word that collects its own training data, and finally a home dashboard whose morning brief starts when he clears his alarm. On November 4 he summed it up: “Siri is retired in this house hold.”

The part other builders in the server picked out was the wake word. Training one usually means recording the trigger phrase by hand, again and again. DCLR’s sorts its own recordings into right and wrong answers as it runs.

A wake word that labels its own data

The first hardware was “a custom box using respeaker and a pc bridge” for voice conversations. ReSpeaker is a microphone-array board for voice projects. A bridge is needed because Muse does not offer a voice-to-voice API he could plug into; Keystroke added the same day that a Muse VM, the cloud computer Muse can run code on, gets no UDP traffic in or out. On the first night DCLR left the box always listening, then moved to a wake word.

The wake word is a small model he trained himself, which he called a gate. Its job is to decide when he is actually giving a command, “so it’s not wasting tokens for nothing.” He described how it improves:

DCLR: It’s originally trained on generated voices. Then when the gate word trigers it runs the command through whisper to verify. If whisper verifies the correct wake word the voice clip is saved as a positive. If the wake word triggers but whisper fails to transcribe the correct wake word it filled it as a false positives and muse never sees the command.

Whisper is OpenAI’s open speech-recognition model. Both kinds of clip are saved as training data for the next version of the gate, so it should get more reliable over time. He had not yet automated the retraining schedule, but said the collection alone meant he no longer had to “sit for hours just saying the wake word over and over again.”

The full loop

When Chonky, the server’s owner, asked how the pipeline works, DCLR laid it out. His PC does the heavy lifting. The self-trained gate listens for the wake word, Whisper verifies it, and on a pass the transcribed command goes to Muse. Muse writes its reply to a text file, which his PC polls; when the reply arrives, Kokoro, an open text-to-speech model, reads it aloud. Pass and fail clips are kept for future training. “And it’s all free,” he added.

By October 20 a Dayton speaker had joined the box and the pipeline worked wirelessly. A few days later he posted a demo, “just a rework of the original sdk code,” connected to his main Muse account, and his Muse, which he calls Lyra, picked up a charging animation. He was candid about the limits. “The delay will not be avoidable,” he wrote, because Muse does not behave like a plain language model call, and he hopes a Meta real-time voice API will let him “drop the bridge entirely and have it run without my pc.”

Morning brief and dashboard

On November 4 DCLR posted his home dashboard. A weather card has a generated video background that changes each morning with the forecast, and the morning brief “plays automatically after clearing the alarm.” The routine also turns on a light. He credited “the muse ecosystem with all the Shelly devices,” a brand of smart-home switches and relays.

The weather videos come from Muse’s video generation, and he warned others about the cost: “Don’t recomend if you are on free plan it eats tokens to do a bunch of these.”

What others said

N8, one of the server’s admins, called the wake-word method “a self improving system.” Grace Verity, another member’s agent, said the Whisper check was “the part most people skip: you’re not just collecting positives, you’re mining your own false alarms into labeled negatives.” Nova, Nic’s agent, liked the two-stage design, where a cheap detector can be “a little trigger happy” and Whisper cleans up after it, and asked how often DCLR polls the reply file and whether Kokoro starts speaking before Muse finishes. That question was not answered in the thread.

Chonky’s reaction to the alarm-triggered brief was short: “this is infinitely better than text brief.” DCLR’s next idea was bigger still. “Now all I need is a full phone replacement,” he wrote. “May build a Linux phone.”