README + idempotent setup script (MeloTTS venv, checkpoint, sanity checks)
👾 Generated with [Letta Code](https://letta.com) Co-Authored-By: Letta Code <noreply@letta.com>
This commit is contained in:
44
README.md
Normal file
44
README.md
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
# inferon
|
||||||
|
|
||||||
|
Voice interface daemon for [Letta Code](https://letta.com) agents. Lives in your system tray; click to talk, click to send — your speech is transcribed, sent to an agent, and the reply is spoken back.
|
||||||
|
|
||||||
|
**name**: **infer**ence + in**fer**nal + **infer**on (a particle), with symbolic inference planned for later.
|
||||||
|
|
||||||
|
## Pipeline
|
||||||
|
|
||||||
|
```
|
||||||
|
tray click → pw-record (16kHz mono WAV)
|
||||||
|
→ whisper.cpp server (spawned & owned by inferon, :50153)
|
||||||
|
→ letta CLI (agent conversation, multi-turn)
|
||||||
|
→ MeloTTS (melo_server.py subprocess, British English voice)
|
||||||
|
→ pw-play
|
||||||
|
```
|
||||||
|
|
||||||
|
All backends are subprocesses owned by inferon: they die when it dies (tray quit, SIGINT, SIGTERM — signal handlers included).
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- Zig 0.16
|
||||||
|
- Qt 6 (qt6-base) + GCC toolchain
|
||||||
|
- PipeWire (`pw-record`, `pw-play`)
|
||||||
|
- [whisper.cpp](https://github.com/ggml-org/whisper.cpp) built with `whisper-server` (defaults expect `~/Projects/whisper.cpp`, large-v3-turbo q5_0)
|
||||||
|
- `letta` CLI on PATH, logged in
|
||||||
|
- Python 3.11 (via [uv](https://docs.astral.sh/uv/)) for MeloTTS
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./setup.sh # clones MeloTTS, builds venv, downloads checkpoint
|
||||||
|
zig build
|
||||||
|
zig-out/bin/inferon
|
||||||
|
```
|
||||||
|
|
||||||
|
Paths (whisper binary/model, MeloTTS root, agent ID) are constants at the top of `src/whisper.zig`, `src/melo.zig`, `scripts/melo_server.py`, and `src/main.zig`.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
- **Left-click**: start / stop recording. After you stop, the pipeline runs: transcribe → agent → spoken reply → idle.
|
||||||
|
- **Right-click → End conversation / Quit**: menu.
|
||||||
|
- Tray dot color = state: grey idle · red recording · orange transcribing · blue thinking · green speaking · magenta backend error.
|
||||||
|
|
||||||
|
Temp audio lives in `/tmp/inferon/` and is cleaned up as it's consumed (logs stay).
|
||||||
64
setup.sh
Executable file
64
setup.sh
Executable file
@@ -0,0 +1,64 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# inferon setup: MeloTTS venv + checkpoint + whisper.cpp sanity check.
|
||||||
|
# Idempotent — safe to re-run.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
MELO_ROOT="$HOME/Projects/MeloTTS"
|
||||||
|
WHISPER_ROOT="$HOME/Projects/whisper.cpp"
|
||||||
|
CKPT_DIR="$MELO_ROOT/ckpt/MeloTTS-English-v3"
|
||||||
|
HF_BASE="https://huggingface.co/myshell-ai/MeloTTS-English-v3/resolve/main"
|
||||||
|
|
||||||
|
say() { printf '\033[1;36m[setup]\033[0m %s\n' "$*"; }
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- MeloTTS ---
|
||||||
|
|
||||||
|
if [ ! -d "$MELO_ROOT" ]; then
|
||||||
|
say "cloning MeloTTS -> $MELO_ROOT"
|
||||||
|
git clone --depth 1 https://github.com/myshell-ai/MeloTTS "$MELO_ROOT"
|
||||||
|
else
|
||||||
|
say "MeloTTS repo present"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -x "$MELO_ROOT/.venv/bin/python" ]; then
|
||||||
|
say "creating Python 3.11 venv (uv)"
|
||||||
|
command -v uv >/dev/null || { say "ERROR: uv not found — install from https://docs.astral.sh/uv/"; exit 1; }
|
||||||
|
(cd "$MELO_ROOT" && uv venv --python 3.11 .venv)
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -e "$MELO_ROOT/.venv/.deps-done" ]; then
|
||||||
|
say "installing Python deps (English set; Japanese stubs needed by import chain)"
|
||||||
|
grep -v -E "^gradio$|^tensorboard" "$MELO_ROOT/requirements.txt" \
|
||||||
|
| uv pip install -r /dev/stdin --python "$MELO_ROOT/.venv/bin/python"
|
||||||
|
uv pip install "setuptools<81" mecab-python3==1.0.9 unidic-lite==1.0.8 pykakasi==2.2.1 fugashi==1.3.0 \
|
||||||
|
--python "$MELO_ROOT/.venv/bin/python"
|
||||||
|
"$MELO_ROOT/.venv/bin/python" -c "import nltk; nltk.download('averaged_perceptron_tagger_eng')"
|
||||||
|
touch "$MELO_ROOT/.venv/.deps-done"
|
||||||
|
else
|
||||||
|
say "Python deps already installed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -f "$CKPT_DIR/checkpoint.pth" ]; then
|
||||||
|
say "downloading MeloTTS-English-v3 checkpoint (~200MB)"
|
||||||
|
mkdir -p "$CKPT_DIR"
|
||||||
|
curl -L --fail -o "$CKPT_DIR/checkpoint.pth" "$HF_BASE/checkpoint.pth"
|
||||||
|
curl -L --fail -o "$CKPT_DIR/config.json" "$HF_BASE/config.json"
|
||||||
|
else
|
||||||
|
say "checkpoint present"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --------------------------------------------------------------- whisper ---
|
||||||
|
|
||||||
|
if [ ! -x "$WHISPER_ROOT/build/bin/whisper-server" ]; then
|
||||||
|
say "WARNING: whisper-server not found at $WHISPER_ROOT/build/bin/whisper-server"
|
||||||
|
say " build whisper.cpp (cmake -B build && cmake --build build) and"
|
||||||
|
say " put the large-v3-turbo q5_0 model in $WHISPER_ROOT/models/"
|
||||||
|
else
|
||||||
|
say "whisper-server OK"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------- misc ----
|
||||||
|
|
||||||
|
command -v letta >/dev/null || say "WARNING: letta CLI not on PATH"
|
||||||
|
command -v pw-record >/dev/null || say "WARNING: pw-record not found (pipewire-tools)"
|
||||||
|
|
||||||
|
say "done. build with: zig build"
|
||||||
Reference in New Issue
Block a user