feat(windows-dictate): Push-to-Talk-Tray-Client für Diktat am Cursor
Neuer Ordner `windows-dictate/`: kleiner Python-Tray-Client für Windows. Hotkey halten (Default F8) = Mikro aufnehmen, loslassen = WAV per Multipart an den bestehenden `POST /api/transcribe`-Endpoint vom Mac-Worker schicken, Antwort am Cursor via Zwischenablage + Strg+V einfügen. Keine Server-Änderung nötig. Enthält dictate.py (Recorder + Tray + Hotkey), config.example.toml, requirements.txt, build.bat (PyInstaller), README mit Setup und Streaming-Upgrade-Pfad.
This commit is contained in:
parent
5f0b41161e
commit
01526e8a91
8
windows-dictate/.gitignore
vendored
Normal file
8
windows-dictate/.gitignore
vendored
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
config.toml
|
||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
build/
|
||||||
|
dist/
|
||||||
|
*.spec
|
||||||
|
*.log
|
||||||
|
.venv/
|
||||||
77
windows-dictate/README.md
Normal file
77
windows-dictate/README.md
Normal file
@ -0,0 +1,77 @@
|
|||||||
|
# Voice-Agent — Windows-Diktat
|
||||||
|
|
||||||
|
Kleiner Push-to-Talk-Client für Windows. Hotkey halten = aufnehmen,
|
||||||
|
loslassen = an den Mac-Worker schicken, Text wird am Cursor eingefügt.
|
||||||
|
|
||||||
|
Läuft als Tray-Icon (blau = Idle, rot = Aufnahme, orange = sende, grau = Fehler).
|
||||||
|
Nutzt direkt den bestehenden `POST /api/transcribe`-Endpoint vom Mac-Worker —
|
||||||
|
keine Server-Änderung nötig.
|
||||||
|
|
||||||
|
## Voraussetzungen
|
||||||
|
- **Windows 10/11**
|
||||||
|
- **Python 3.11+** (`tomllib` und neuere `numpy`)
|
||||||
|
- Erreichbarkeit des Mac-Workers im internen Netz (Standard: `http://10.172.100.44:8080`)
|
||||||
|
|
||||||
|
## Schnellstart (entwickeln)
|
||||||
|
```cmd
|
||||||
|
cd windows-dictate
|
||||||
|
python -m venv .venv
|
||||||
|
.venv\Scripts\activate
|
||||||
|
pip install -r requirements.txt
|
||||||
|
copy config.example.toml config.toml
|
||||||
|
notepad config.toml
|
||||||
|
python dictate.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Im Tray erscheint ein blauer Punkt. Cursor in ein Eingabefeld setzen,
|
||||||
|
**F8 halten**, sprechen, **loslassen** — nach 1–3 s steht der Text drin.
|
||||||
|
|
||||||
|
## Standalone-EXE bauen
|
||||||
|
```cmd
|
||||||
|
pip install pyinstaller
|
||||||
|
build.bat
|
||||||
|
```
|
||||||
|
Ergebnis: `dist\VoiceAgentDictate.exe`. Daneben eine `config.toml` ablegen.
|
||||||
|
|
||||||
|
## Konfig (`config.toml`)
|
||||||
|
| Key | Default | Bedeutung |
|
||||||
|
|---|---|---|
|
||||||
|
| `mac_url` | `http://10.172.100.44:8080` | URL des Mac-Workers |
|
||||||
|
| `hotkey` | `f8` | Push-to-Talk-Taste (einzelne Taste, halten) |
|
||||||
|
| `language` | `de` | Whisper-Sprache |
|
||||||
|
| `timeout` | `120` | Sekunden für den Mac-Call |
|
||||||
|
| `suppress_hotkey` | `false` | Hotkey vor anderen Apps abfangen |
|
||||||
|
|
||||||
|
## Tray-Menü
|
||||||
|
Rechtsklick auf das Tray-Icon → **Beenden**.
|
||||||
|
|
||||||
|
## Hotkey-Hinweise
|
||||||
|
- `keyboard` unterstützt **einzelne Tasten** für press/release sauber. Kombis wie `Ctrl+Win+Space` brauchen eine andere Logik (Toggle statt Push-to-Talk).
|
||||||
|
- Falls F8 mit etwas kollidiert (z.B. Browser-Slideshow), in der Konfig auf `pause`, `scroll lock`, `f9`, `f12` o.ä. wechseln.
|
||||||
|
- `suppress_hotkey = true` schluckt die Taste vollständig — nur einschalten, wenn die Taste sonst nicht gebraucht wird.
|
||||||
|
|
||||||
|
## Was es **nicht** macht
|
||||||
|
- Kein Live-Streaming Wort-für-Wort. Du sprichst einen Block, dann erscheint der Block. Realistisch sind 3–8 s pro Diktat-Block.
|
||||||
|
- Keine Authentifizierung — der Mac-Worker hat selbst keine. Nur im internen Netz nutzen.
|
||||||
|
- Keine Audio-Vorverarbeitung. Mikro-Auswahl macht Windows selbst (Standard-Aufnahmegerät).
|
||||||
|
|
||||||
|
## Upgrade-Pfad: echtes Streaming
|
||||||
|
Wenn Du später Wort-für-Wort willst, müsste der Mac-Worker `lightning-whisper-mlx`
|
||||||
|
durch eine Streaming-Variante ersetzen — Kandidaten:
|
||||||
|
|
||||||
|
- **whisper-streaming** (https://github.com/ufal/whisper_streaming) — Wrapper um
|
||||||
|
`faster-whisper` mit LocalAgreement-Policy. Gibt partielle Transkripte alle 0.5–2 s.
|
||||||
|
- **whisper.cpp `--stream`** — sehr leichtgewichtig, aber CPU statt MLX.
|
||||||
|
- **wav2vec2 / Vosk** — echtes Streaming, schwächer als Whisper bei deutschem Audio.
|
||||||
|
|
||||||
|
Auf Client-Seite würde sich dann WebSocket statt POST anbieten, plus eine
|
||||||
|
„partial vs final"-Logik (partial löschen + neuen Text einfügen). Größerer Umbau —
|
||||||
|
aktuell bewusst nicht gemacht, weil der Push-to-Talk-Workflow für Diktat in
|
||||||
|
Office/Mail/Chat in der Praxis sehr gut funktioniert.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
- **Hotkey reagiert nicht**: `keyboard` braucht keine Admin-Rechte zum Lesen, aber `suppress_hotkey = true` schon. Erst mit `false` testen.
|
||||||
|
- **`OSError: PortAudio`** bei Start: Standard-Aufnahmegerät in Windows-Sound-Settings setzen.
|
||||||
|
- **Fehler-Icon (grau)** + Log `Mac-Fehler: ConnectionError`: Mac aus, falsche IP in `config.toml`, oder Firewall blockt 8080.
|
||||||
|
- **Text erscheint nicht** im Zielfenster: einige Apps (manche Terminals) blocken simuliertes Strg+V. Test in Notepad.
|
||||||
|
- **Sonderzeichen verstümmelt**: Ziel-App unterstützt Unicode-Paste nicht. Workaround: in `dictate.py` `keyboard.send("ctrl+v")` durch `keyboard.write(text)` ersetzen (langsamer, aber tastenweise).
|
||||||
16
windows-dictate/build.bat
Normal file
16
windows-dictate/build.bat
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
@echo off
|
||||||
|
REM Standalone .exe fuer Voice-Agent Diktat bauen.
|
||||||
|
REM Vorab einmal:
|
||||||
|
REM python -m venv .venv
|
||||||
|
REM .venv\Scripts\activate
|
||||||
|
REM pip install -r requirements.txt
|
||||||
|
REM pip install pyinstaller
|
||||||
|
|
||||||
|
pyinstaller --noconfirm --onefile ^
|
||||||
|
--name VoiceAgentDictate ^
|
||||||
|
--add-data "config.example.toml;." ^
|
||||||
|
dictate.py
|
||||||
|
|
||||||
|
echo.
|
||||||
|
echo Fertig. EXE liegt in dist\VoiceAgentDictate.exe
|
||||||
|
echo Daneben config.toml ablegen (aus config.example.toml kopiert).
|
||||||
19
windows-dictate/config.example.toml
Normal file
19
windows-dictate/config.example.toml
Normal file
@ -0,0 +1,19 @@
|
|||||||
|
# Voice-Agent Windows-Diktat — Konfig
|
||||||
|
# Diese Datei nach `config.toml` kopieren und anpassen.
|
||||||
|
|
||||||
|
# URL des Mac-Workers im internen Netz (ohne /api am Ende).
|
||||||
|
mac_url = "http://10.172.100.44:8080"
|
||||||
|
|
||||||
|
# Push-to-Talk-Hotkey. Einzelne Taste — halten = aufnehmen.
|
||||||
|
# Beispiele: "f8", "f9", "f12", "pause", "scroll lock"
|
||||||
|
hotkey = "f8"
|
||||||
|
|
||||||
|
# Whisper-Sprache (ISO-639-1).
|
||||||
|
language = "de"
|
||||||
|
|
||||||
|
# Timeout in Sekunden für den /api/transcribe-Call.
|
||||||
|
timeout = 120
|
||||||
|
|
||||||
|
# Hotkey vor anderen Programmen abfangen (true = F8 erreicht nichts anderes).
|
||||||
|
# Default false, damit z.B. der Browser-Slideshow-Hotkey weiter funktioniert.
|
||||||
|
suppress_hotkey = false
|
||||||
245
windows-dictate/dictate.py
Normal file
245
windows-dictate/dictate.py
Normal file
@ -0,0 +1,245 @@
|
|||||||
|
"""Voice-Agent Windows-Diktat — Push-to-Talk-Client.
|
||||||
|
|
||||||
|
Hotkey halten = aufnehmen, loslassen = an Mac-Worker schicken, Ergebnis
|
||||||
|
am Cursor einfuegen (Zwischenablage + Strg+V).
|
||||||
|
|
||||||
|
Konfig: config.toml neben dem Script (siehe config.example.toml).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
|
import logging
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import tomllib
|
||||||
|
import wave
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import keyboard
|
||||||
|
import numpy as np
|
||||||
|
import pyperclip
|
||||||
|
import requests
|
||||||
|
import sounddevice as sd
|
||||||
|
from PIL import Image, ImageDraw
|
||||||
|
from pystray import Icon, Menu, MenuItem
|
||||||
|
|
||||||
|
SAMPLE_RATE = 16000
|
||||||
|
CHANNELS = 1
|
||||||
|
DTYPE = "int16"
|
||||||
|
MIN_SECONDS = 0.2
|
||||||
|
MAX_SECONDS = 120
|
||||||
|
|
||||||
|
HERE = Path(__file__).resolve().parent
|
||||||
|
CONFIG_PATH = HERE / "config.toml"
|
||||||
|
EXAMPLE_PATH = HERE / "config.example.toml"
|
||||||
|
|
||||||
|
log = logging.getLogger("voice-agent.dictate")
|
||||||
|
|
||||||
|
|
||||||
|
def load_config() -> dict:
|
||||||
|
path = CONFIG_PATH if CONFIG_PATH.exists() else EXAMPLE_PATH
|
||||||
|
if not path.exists():
|
||||||
|
raise SystemExit(
|
||||||
|
f"Weder {CONFIG_PATH.name} noch {EXAMPLE_PATH.name} gefunden in {HERE}."
|
||||||
|
)
|
||||||
|
with path.open("rb") as f:
|
||||||
|
cfg = tomllib.load(f)
|
||||||
|
if path == EXAMPLE_PATH:
|
||||||
|
log.warning(
|
||||||
|
"Nutze config.example.toml als Fallback — bitte nach config.toml kopieren."
|
||||||
|
)
|
||||||
|
return cfg
|
||||||
|
|
||||||
|
|
||||||
|
class Recorder:
|
||||||
|
"""Sammelt 16-kHz-Mono-Audio waehrend die Hotkey-Taste gehalten wird."""
|
||||||
|
|
||||||
|
def __init__(self, sample_rate: int = SAMPLE_RATE):
|
||||||
|
self.sample_rate = sample_rate
|
||||||
|
self._frames: list[np.ndarray] = []
|
||||||
|
self._stream: sd.InputStream | None = None
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self.recording = False
|
||||||
|
|
||||||
|
def _callback(self, indata, _frames, _time_info, status) -> None:
|
||||||
|
if status:
|
||||||
|
log.debug("audio status: %s", status)
|
||||||
|
with self._lock:
|
||||||
|
self._frames.append(indata.copy())
|
||||||
|
if len(self._frames) * indata.shape[0] >= self.sample_rate * MAX_SECONDS:
|
||||||
|
log.warning("MAX_SECONDS erreicht — Aufnahme wird automatisch beendet")
|
||||||
|
self._stop_stream()
|
||||||
|
|
||||||
|
def _stop_stream(self) -> None:
|
||||||
|
if self._stream is not None:
|
||||||
|
try:
|
||||||
|
self._stream.stop()
|
||||||
|
self._stream.close()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass
|
||||||
|
self._stream = None
|
||||||
|
self.recording = False
|
||||||
|
|
||||||
|
def start(self) -> None:
|
||||||
|
if self.recording:
|
||||||
|
return
|
||||||
|
with self._lock:
|
||||||
|
self._frames = []
|
||||||
|
self._stream = sd.InputStream(
|
||||||
|
samplerate=self.sample_rate,
|
||||||
|
channels=CHANNELS,
|
||||||
|
dtype=DTYPE,
|
||||||
|
callback=self._callback,
|
||||||
|
)
|
||||||
|
self._stream.start()
|
||||||
|
self.recording = True
|
||||||
|
|
||||||
|
def stop(self) -> bytes | None:
|
||||||
|
if self._stream is None and not self._frames:
|
||||||
|
return None
|
||||||
|
self._stop_stream()
|
||||||
|
with self._lock:
|
||||||
|
frames = list(self._frames)
|
||||||
|
self._frames = []
|
||||||
|
if not frames:
|
||||||
|
return None
|
||||||
|
audio = np.concatenate(frames, axis=0)
|
||||||
|
if len(audio) < self.sample_rate * MIN_SECONDS:
|
||||||
|
return None
|
||||||
|
return _to_wav_bytes(audio, self.sample_rate)
|
||||||
|
|
||||||
|
|
||||||
|
def _to_wav_bytes(audio: np.ndarray, sample_rate: int) -> bytes:
|
||||||
|
buf = io.BytesIO()
|
||||||
|
with wave.open(buf, "wb") as w:
|
||||||
|
w.setnchannels(CHANNELS)
|
||||||
|
w.setsampwidth(2)
|
||||||
|
w.setframerate(sample_rate)
|
||||||
|
w.writeframes(audio.tobytes())
|
||||||
|
return buf.getvalue()
|
||||||
|
|
||||||
|
|
||||||
|
def paste_at_cursor(text: str) -> None:
|
||||||
|
"""Text in Zwischenablage legen + Strg+V senden. Alter Inhalt wird nach 500 ms wiederhergestellt."""
|
||||||
|
if not text:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
previous = pyperclip.paste()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
previous = None
|
||||||
|
pyperclip.copy(text)
|
||||||
|
time.sleep(0.05)
|
||||||
|
keyboard.send("ctrl+v")
|
||||||
|
|
||||||
|
def _restore() -> None:
|
||||||
|
time.sleep(0.5)
|
||||||
|
if previous is not None:
|
||||||
|
try:
|
||||||
|
pyperclip.copy(previous)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass
|
||||||
|
|
||||||
|
threading.Thread(target=_restore, daemon=True).start()
|
||||||
|
|
||||||
|
|
||||||
|
def transcribe(wav: bytes, mac_url: str, language: str, timeout: int) -> str:
|
||||||
|
files = {"audio": ("dictate.wav", wav, "audio/wav")}
|
||||||
|
data = {"language": language}
|
||||||
|
r = requests.post(
|
||||||
|
f"{mac_url.rstrip('/')}/api/transcribe",
|
||||||
|
files=files,
|
||||||
|
data=data,
|
||||||
|
timeout=timeout,
|
||||||
|
)
|
||||||
|
r.raise_for_status()
|
||||||
|
return (r.json().get("text") or "").strip()
|
||||||
|
|
||||||
|
|
||||||
|
_COLORS = {
|
||||||
|
"idle": "#3b82f6",
|
||||||
|
"rec": "#dc2626",
|
||||||
|
"send": "#f59e0b",
|
||||||
|
"err": "#737373",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _make_icon(color: str) -> Image.Image:
|
||||||
|
img = Image.new("RGBA", (64, 64), (0, 0, 0, 0))
|
||||||
|
d = ImageDraw.Draw(img)
|
||||||
|
d.ellipse((6, 6, 58, 58), fill=color)
|
||||||
|
return img
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
logging.basicConfig(
|
||||||
|
level=logging.INFO,
|
||||||
|
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
|
||||||
|
)
|
||||||
|
cfg = load_config()
|
||||||
|
mac_url = cfg["mac_url"]
|
||||||
|
hotkey = str(cfg.get("hotkey", "f8")).lower()
|
||||||
|
language = cfg.get("language", "de")
|
||||||
|
timeout = int(cfg.get("timeout", 120))
|
||||||
|
suppress = bool(cfg.get("suppress_hotkey", False))
|
||||||
|
|
||||||
|
recorder = Recorder()
|
||||||
|
icon = Icon("voice-agent-dictate", _make_icon(_COLORS["idle"]), "Voice-Agent Diktat")
|
||||||
|
state_lock = threading.Lock()
|
||||||
|
|
||||||
|
def set_state(state: str, tip: str) -> None:
|
||||||
|
icon.icon = _make_icon(_COLORS[state])
|
||||||
|
icon.title = f"Voice-Agent Diktat — {tip}"
|
||||||
|
|
||||||
|
def on_press(_evt) -> None:
|
||||||
|
with state_lock:
|
||||||
|
if recorder.recording:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
recorder.start()
|
||||||
|
except Exception as e: # noqa: BLE001
|
||||||
|
log.error("Audio-Start fehlgeschlagen: %s", e)
|
||||||
|
set_state("err", f"Audio-Fehler: {e}")
|
||||||
|
return
|
||||||
|
set_state("rec", "Aufnahme...")
|
||||||
|
log.info("recording start")
|
||||||
|
|
||||||
|
def on_release(_evt) -> None:
|
||||||
|
with state_lock:
|
||||||
|
if not recorder.recording and not recorder._frames:
|
||||||
|
return
|
||||||
|
wav = recorder.stop()
|
||||||
|
if not wav:
|
||||||
|
log.info("zu kurz / leer — verworfen")
|
||||||
|
set_state("idle", "Idle")
|
||||||
|
return
|
||||||
|
set_state("send", "Sende an Mac...")
|
||||||
|
log.info("recording stop (%d KB)", len(wav) // 1024)
|
||||||
|
try:
|
||||||
|
text = transcribe(wav, mac_url, language, timeout)
|
||||||
|
except requests.RequestException as e:
|
||||||
|
log.error("transcribe failed: %s", e)
|
||||||
|
set_state("err", f"Mac-Fehler: {e}")
|
||||||
|
return
|
||||||
|
if not text:
|
||||||
|
log.info("leeres Transkript")
|
||||||
|
set_state("idle", "Idle")
|
||||||
|
return
|
||||||
|
log.info("text: %s", text[:120])
|
||||||
|
paste_at_cursor(text)
|
||||||
|
set_state("idle", "Idle")
|
||||||
|
|
||||||
|
keyboard.on_press_key(hotkey, on_press, suppress=suppress)
|
||||||
|
keyboard.on_release_key(hotkey, on_release, suppress=suppress)
|
||||||
|
|
||||||
|
def quit_app(_icon=None, _item=None) -> None:
|
||||||
|
icon.stop()
|
||||||
|
sys.exit(0)
|
||||||
|
|
||||||
|
icon.menu = Menu(MenuItem("Beenden", quit_app))
|
||||||
|
log.info("Bereit. Hotkey halten: %s | Mac: %s | Sprache: %s", hotkey, mac_url, language)
|
||||||
|
icon.run()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
7
windows-dictate/requirements.txt
Normal file
7
windows-dictate/requirements.txt
Normal file
@ -0,0 +1,7 @@
|
|||||||
|
sounddevice>=0.4.6
|
||||||
|
numpy>=1.26
|
||||||
|
requests>=2.31
|
||||||
|
keyboard>=0.13.5
|
||||||
|
pyperclip>=1.8.2
|
||||||
|
pystray>=0.19.5
|
||||||
|
Pillow>=10.0
|
||||||
Loading…
x
Reference in New Issue
Block a user