feat(windows-dictate): Push-to-Talk-Tray-Client für Diktat am Cursor
Neuer Ordner `windows-dictate/`: kleiner Python-Tray-Client für Windows. Hotkey halten (Default F8) = Mikro aufnehmen, loslassen = WAV per Multipart an den bestehenden `POST /api/transcribe`-Endpoint vom Mac-Worker schicken, Antwort am Cursor via Zwischenablage + Strg+V einfügen. Keine Server-Änderung nötig. Enthält dictate.py (Recorder + Tray + Hotkey), config.example.toml, requirements.txt, build.bat (PyInstaller), README mit Setup und Streaming-Upgrade-Pfad.
This commit is contained in:
parent
5f0b41161e
commit
01526e8a91
8
windows-dictate/.gitignore
vendored
Normal file
8
windows-dictate/.gitignore
vendored
Normal file
@ -0,0 +1,8 @@
|
||||
config.toml
|
||||
__pycache__/
|
||||
*.pyc
|
||||
build/
|
||||
dist/
|
||||
*.spec
|
||||
*.log
|
||||
.venv/
|
||||
77
windows-dictate/README.md
Normal file
77
windows-dictate/README.md
Normal file
@ -0,0 +1,77 @@
|
||||
# Voice-Agent — Windows-Diktat
|
||||
|
||||
Kleiner Push-to-Talk-Client für Windows. Hotkey halten = aufnehmen,
|
||||
loslassen = an den Mac-Worker schicken, Text wird am Cursor eingefügt.
|
||||
|
||||
Läuft als Tray-Icon (blau = Idle, rot = Aufnahme, orange = sende, grau = Fehler).
|
||||
Nutzt direkt den bestehenden `POST /api/transcribe`-Endpoint vom Mac-Worker —
|
||||
keine Server-Änderung nötig.
|
||||
|
||||
## Voraussetzungen
|
||||
- **Windows 10/11**
|
||||
- **Python 3.11+** (`tomllib` und neuere `numpy`)
|
||||
- Erreichbarkeit des Mac-Workers im internen Netz (Standard: `http://10.172.100.44:8080`)
|
||||
|
||||
## Schnellstart (entwickeln)
|
||||
```cmd
|
||||
cd windows-dictate
|
||||
python -m venv .venv
|
||||
.venv\Scripts\activate
|
||||
pip install -r requirements.txt
|
||||
copy config.example.toml config.toml
|
||||
notepad config.toml
|
||||
python dictate.py
|
||||
```
|
||||
|
||||
Im Tray erscheint ein blauer Punkt. Cursor in ein Eingabefeld setzen,
|
||||
**F8 halten**, sprechen, **loslassen** — nach 1–3 s steht der Text drin.
|
||||
|
||||
## Standalone-EXE bauen
|
||||
```cmd
|
||||
pip install pyinstaller
|
||||
build.bat
|
||||
```
|
||||
Ergebnis: `dist\VoiceAgentDictate.exe`. Daneben eine `config.toml` ablegen.
|
||||
|
||||
## Konfig (`config.toml`)
|
||||
| Key | Default | Bedeutung |
|
||||
|---|---|---|
|
||||
| `mac_url` | `http://10.172.100.44:8080` | URL des Mac-Workers |
|
||||
| `hotkey` | `f8` | Push-to-Talk-Taste (einzelne Taste, halten) |
|
||||
| `language` | `de` | Whisper-Sprache |
|
||||
| `timeout` | `120` | Sekunden für den Mac-Call |
|
||||
| `suppress_hotkey` | `false` | Hotkey vor anderen Apps abfangen |
|
||||
|
||||
## Tray-Menü
|
||||
Rechtsklick auf das Tray-Icon → **Beenden**.
|
||||
|
||||
## Hotkey-Hinweise
|
||||
- `keyboard` unterstützt **einzelne Tasten** für press/release sauber. Kombis wie `Ctrl+Win+Space` brauchen eine andere Logik (Toggle statt Push-to-Talk).
|
||||
- Falls F8 mit etwas kollidiert (z.B. Browser-Slideshow), in der Konfig auf `pause`, `scroll lock`, `f9`, `f12` o.ä. wechseln.
|
||||
- `suppress_hotkey = true` schluckt die Taste vollständig — nur einschalten, wenn die Taste sonst nicht gebraucht wird.
|
||||
|
||||
## Was es **nicht** macht
|
||||
- Kein Live-Streaming Wort-für-Wort. Du sprichst einen Block, dann erscheint der Block. Realistisch sind 3–8 s pro Diktat-Block.
|
||||
- Keine Authentifizierung — der Mac-Worker hat selbst keine. Nur im internen Netz nutzen.
|
||||
- Keine Audio-Vorverarbeitung. Mikro-Auswahl macht Windows selbst (Standard-Aufnahmegerät).
|
||||
|
||||
## Upgrade-Pfad: echtes Streaming
|
||||
Wenn Du später Wort-für-Wort willst, müsste der Mac-Worker `lightning-whisper-mlx`
|
||||
durch eine Streaming-Variante ersetzen — Kandidaten:
|
||||
|
||||
- **whisper-streaming** (https://github.com/ufal/whisper_streaming) — Wrapper um
|
||||
`faster-whisper` mit LocalAgreement-Policy. Gibt partielle Transkripte alle 0.5–2 s.
|
||||
- **whisper.cpp `--stream`** — sehr leichtgewichtig, aber CPU statt MLX.
|
||||
- **wav2vec2 / Vosk** — echtes Streaming, schwächer als Whisper bei deutschem Audio.
|
||||
|
||||
Auf Client-Seite würde sich dann WebSocket statt POST anbieten, plus eine
|
||||
„partial vs final"-Logik (partial löschen + neuen Text einfügen). Größerer Umbau —
|
||||
aktuell bewusst nicht gemacht, weil der Push-to-Talk-Workflow für Diktat in
|
||||
Office/Mail/Chat in der Praxis sehr gut funktioniert.
|
||||
|
||||
## Troubleshooting
|
||||
- **Hotkey reagiert nicht**: `keyboard` braucht keine Admin-Rechte zum Lesen, aber `suppress_hotkey = true` schon. Erst mit `false` testen.
|
||||
- **`OSError: PortAudio`** bei Start: Standard-Aufnahmegerät in Windows-Sound-Settings setzen.
|
||||
- **Fehler-Icon (grau)** + Log `Mac-Fehler: ConnectionError`: Mac aus, falsche IP in `config.toml`, oder Firewall blockt 8080.
|
||||
- **Text erscheint nicht** im Zielfenster: einige Apps (manche Terminals) blocken simuliertes Strg+V. Test in Notepad.
|
||||
- **Sonderzeichen verstümmelt**: Ziel-App unterstützt Unicode-Paste nicht. Workaround: in `dictate.py` `keyboard.send("ctrl+v")` durch `keyboard.write(text)` ersetzen (langsamer, aber tastenweise).
|
||||
16
windows-dictate/build.bat
Normal file
16
windows-dictate/build.bat
Normal file
@ -0,0 +1,16 @@
|
||||
@echo off
|
||||
REM Standalone .exe fuer Voice-Agent Diktat bauen.
|
||||
REM Vorab einmal:
|
||||
REM python -m venv .venv
|
||||
REM .venv\Scripts\activate
|
||||
REM pip install -r requirements.txt
|
||||
REM pip install pyinstaller
|
||||
|
||||
pyinstaller --noconfirm --onefile ^
|
||||
--name VoiceAgentDictate ^
|
||||
--add-data "config.example.toml;." ^
|
||||
dictate.py
|
||||
|
||||
echo.
|
||||
echo Fertig. EXE liegt in dist\VoiceAgentDictate.exe
|
||||
echo Daneben config.toml ablegen (aus config.example.toml kopiert).
|
||||
19
windows-dictate/config.example.toml
Normal file
19
windows-dictate/config.example.toml
Normal file
@ -0,0 +1,19 @@
|
||||
# Voice-Agent Windows-Diktat — Konfig
|
||||
# Diese Datei nach `config.toml` kopieren und anpassen.
|
||||
|
||||
# URL des Mac-Workers im internen Netz (ohne /api am Ende).
|
||||
mac_url = "http://10.172.100.44:8080"
|
||||
|
||||
# Push-to-Talk-Hotkey. Einzelne Taste — halten = aufnehmen.
|
||||
# Beispiele: "f8", "f9", "f12", "pause", "scroll lock"
|
||||
hotkey = "f8"
|
||||
|
||||
# Whisper-Sprache (ISO-639-1).
|
||||
language = "de"
|
||||
|
||||
# Timeout in Sekunden für den /api/transcribe-Call.
|
||||
timeout = 120
|
||||
|
||||
# Hotkey vor anderen Programmen abfangen (true = F8 erreicht nichts anderes).
|
||||
# Default false, damit z.B. der Browser-Slideshow-Hotkey weiter funktioniert.
|
||||
suppress_hotkey = false
|
||||
245
windows-dictate/dictate.py
Normal file
245
windows-dictate/dictate.py
Normal file
@ -0,0 +1,245 @@
|
||||
"""Voice-Agent Windows-Diktat — Push-to-Talk-Client.
|
||||
|
||||
Hotkey halten = aufnehmen, loslassen = an Mac-Worker schicken, Ergebnis
|
||||
am Cursor einfuegen (Zwischenablage + Strg+V).
|
||||
|
||||
Konfig: config.toml neben dem Script (siehe config.example.toml).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import logging
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import tomllib
|
||||
import wave
|
||||
from pathlib import Path
|
||||
|
||||
import keyboard
|
||||
import numpy as np
|
||||
import pyperclip
|
||||
import requests
|
||||
import sounddevice as sd
|
||||
from PIL import Image, ImageDraw
|
||||
from pystray import Icon, Menu, MenuItem
|
||||
|
||||
SAMPLE_RATE = 16000
|
||||
CHANNELS = 1
|
||||
DTYPE = "int16"
|
||||
MIN_SECONDS = 0.2
|
||||
MAX_SECONDS = 120
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
CONFIG_PATH = HERE / "config.toml"
|
||||
EXAMPLE_PATH = HERE / "config.example.toml"
|
||||
|
||||
log = logging.getLogger("voice-agent.dictate")
|
||||
|
||||
|
||||
def load_config() -> dict:
|
||||
path = CONFIG_PATH if CONFIG_PATH.exists() else EXAMPLE_PATH
|
||||
if not path.exists():
|
||||
raise SystemExit(
|
||||
f"Weder {CONFIG_PATH.name} noch {EXAMPLE_PATH.name} gefunden in {HERE}."
|
||||
)
|
||||
with path.open("rb") as f:
|
||||
cfg = tomllib.load(f)
|
||||
if path == EXAMPLE_PATH:
|
||||
log.warning(
|
||||
"Nutze config.example.toml als Fallback — bitte nach config.toml kopieren."
|
||||
)
|
||||
return cfg
|
||||
|
||||
|
||||
class Recorder:
|
||||
"""Sammelt 16-kHz-Mono-Audio waehrend die Hotkey-Taste gehalten wird."""
|
||||
|
||||
def __init__(self, sample_rate: int = SAMPLE_RATE):
|
||||
self.sample_rate = sample_rate
|
||||
self._frames: list[np.ndarray] = []
|
||||
self._stream: sd.InputStream | None = None
|
||||
self._lock = threading.Lock()
|
||||
self.recording = False
|
||||
|
||||
def _callback(self, indata, _frames, _time_info, status) -> None:
|
||||
if status:
|
||||
log.debug("audio status: %s", status)
|
||||
with self._lock:
|
||||
self._frames.append(indata.copy())
|
||||
if len(self._frames) * indata.shape[0] >= self.sample_rate * MAX_SECONDS:
|
||||
log.warning("MAX_SECONDS erreicht — Aufnahme wird automatisch beendet")
|
||||
self._stop_stream()
|
||||
|
||||
def _stop_stream(self) -> None:
|
||||
if self._stream is not None:
|
||||
try:
|
||||
self._stream.stop()
|
||||
self._stream.close()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
self._stream = None
|
||||
self.recording = False
|
||||
|
||||
def start(self) -> None:
|
||||
if self.recording:
|
||||
return
|
||||
with self._lock:
|
||||
self._frames = []
|
||||
self._stream = sd.InputStream(
|
||||
samplerate=self.sample_rate,
|
||||
channels=CHANNELS,
|
||||
dtype=DTYPE,
|
||||
callback=self._callback,
|
||||
)
|
||||
self._stream.start()
|
||||
self.recording = True
|
||||
|
||||
def stop(self) -> bytes | None:
|
||||
if self._stream is None and not self._frames:
|
||||
return None
|
||||
self._stop_stream()
|
||||
with self._lock:
|
||||
frames = list(self._frames)
|
||||
self._frames = []
|
||||
if not frames:
|
||||
return None
|
||||
audio = np.concatenate(frames, axis=0)
|
||||
if len(audio) < self.sample_rate * MIN_SECONDS:
|
||||
return None
|
||||
return _to_wav_bytes(audio, self.sample_rate)
|
||||
|
||||
|
||||
def _to_wav_bytes(audio: np.ndarray, sample_rate: int) -> bytes:
|
||||
buf = io.BytesIO()
|
||||
with wave.open(buf, "wb") as w:
|
||||
w.setnchannels(CHANNELS)
|
||||
w.setsampwidth(2)
|
||||
w.setframerate(sample_rate)
|
||||
w.writeframes(audio.tobytes())
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def paste_at_cursor(text: str) -> None:
|
||||
"""Text in Zwischenablage legen + Strg+V senden. Alter Inhalt wird nach 500 ms wiederhergestellt."""
|
||||
if not text:
|
||||
return
|
||||
try:
|
||||
previous = pyperclip.paste()
|
||||
except Exception: # noqa: BLE001
|
||||
previous = None
|
||||
pyperclip.copy(text)
|
||||
time.sleep(0.05)
|
||||
keyboard.send("ctrl+v")
|
||||
|
||||
def _restore() -> None:
|
||||
time.sleep(0.5)
|
||||
if previous is not None:
|
||||
try:
|
||||
pyperclip.copy(previous)
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
threading.Thread(target=_restore, daemon=True).start()
|
||||
|
||||
|
||||
def transcribe(wav: bytes, mac_url: str, language: str, timeout: int) -> str:
|
||||
files = {"audio": ("dictate.wav", wav, "audio/wav")}
|
||||
data = {"language": language}
|
||||
r = requests.post(
|
||||
f"{mac_url.rstrip('/')}/api/transcribe",
|
||||
files=files,
|
||||
data=data,
|
||||
timeout=timeout,
|
||||
)
|
||||
r.raise_for_status()
|
||||
return (r.json().get("text") or "").strip()
|
||||
|
||||
|
||||
_COLORS = {
|
||||
"idle": "#3b82f6",
|
||||
"rec": "#dc2626",
|
||||
"send": "#f59e0b",
|
||||
"err": "#737373",
|
||||
}
|
||||
|
||||
|
||||
def _make_icon(color: str) -> Image.Image:
|
||||
img = Image.new("RGBA", (64, 64), (0, 0, 0, 0))
|
||||
d = ImageDraw.Draw(img)
|
||||
d.ellipse((6, 6, 58, 58), fill=color)
|
||||
return img
|
||||
|
||||
|
||||
def main() -> None:
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
|
||||
)
|
||||
cfg = load_config()
|
||||
mac_url = cfg["mac_url"]
|
||||
hotkey = str(cfg.get("hotkey", "f8")).lower()
|
||||
language = cfg.get("language", "de")
|
||||
timeout = int(cfg.get("timeout", 120))
|
||||
suppress = bool(cfg.get("suppress_hotkey", False))
|
||||
|
||||
recorder = Recorder()
|
||||
icon = Icon("voice-agent-dictate", _make_icon(_COLORS["idle"]), "Voice-Agent Diktat")
|
||||
state_lock = threading.Lock()
|
||||
|
||||
def set_state(state: str, tip: str) -> None:
|
||||
icon.icon = _make_icon(_COLORS[state])
|
||||
icon.title = f"Voice-Agent Diktat — {tip}"
|
||||
|
||||
def on_press(_evt) -> None:
|
||||
with state_lock:
|
||||
if recorder.recording:
|
||||
return
|
||||
try:
|
||||
recorder.start()
|
||||
except Exception as e: # noqa: BLE001
|
||||
log.error("Audio-Start fehlgeschlagen: %s", e)
|
||||
set_state("err", f"Audio-Fehler: {e}")
|
||||
return
|
||||
set_state("rec", "Aufnahme...")
|
||||
log.info("recording start")
|
||||
|
||||
def on_release(_evt) -> None:
|
||||
with state_lock:
|
||||
if not recorder.recording and not recorder._frames:
|
||||
return
|
||||
wav = recorder.stop()
|
||||
if not wav:
|
||||
log.info("zu kurz / leer — verworfen")
|
||||
set_state("idle", "Idle")
|
||||
return
|
||||
set_state("send", "Sende an Mac...")
|
||||
log.info("recording stop (%d KB)", len(wav) // 1024)
|
||||
try:
|
||||
text = transcribe(wav, mac_url, language, timeout)
|
||||
except requests.RequestException as e:
|
||||
log.error("transcribe failed: %s", e)
|
||||
set_state("err", f"Mac-Fehler: {e}")
|
||||
return
|
||||
if not text:
|
||||
log.info("leeres Transkript")
|
||||
set_state("idle", "Idle")
|
||||
return
|
||||
log.info("text: %s", text[:120])
|
||||
paste_at_cursor(text)
|
||||
set_state("idle", "Idle")
|
||||
|
||||
keyboard.on_press_key(hotkey, on_press, suppress=suppress)
|
||||
keyboard.on_release_key(hotkey, on_release, suppress=suppress)
|
||||
|
||||
def quit_app(_icon=None, _item=None) -> None:
|
||||
icon.stop()
|
||||
sys.exit(0)
|
||||
|
||||
icon.menu = Menu(MenuItem("Beenden", quit_app))
|
||||
log.info("Bereit. Hotkey halten: %s | Mac: %s | Sprache: %s", hotkey, mac_url, language)
|
||||
icon.run()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
7
windows-dictate/requirements.txt
Normal file
7
windows-dictate/requirements.txt
Normal file
@ -0,0 +1,7 @@
|
||||
sounddevice>=0.4.6
|
||||
numpy>=1.26
|
||||
requests>=2.31
|
||||
keyboard>=0.13.5
|
||||
pyperclip>=1.8.2
|
||||
pystray>=0.19.5
|
||||
Pillow>=10.0
|
||||
Loading…
x
Reference in New Issue
Block a user