christian/rmm2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

PostgreSQL + TimescaleDB Setup-Anleitung und Installation auf .13

Browse Source 
- PostgreSQL 17.8 auf Debian 13 Trixie installiert
- TimescaleDB 2.25.1 Extension aktiviert
- DB 'rmm', User 'rmm' angelegt
- docs/POSTGRES_SETUP.md: Komplette Anleitung
  - Installation (PG17 + TimescaleDB via Bookworm-Repo)
  - Schema (agents, config_backups, metrics Hypertable, system_data)
  - Retention Policy (90 Tage Raw, 365 Tage Hourly Aggregate)
  - Compression (nach 7 Tagen)
  - Continuous Aggregate (stuendliche Durchschnitte)
  - Nuetzliche SQL-Abfragen
  - Backend config.yaml Anpassung
This commit is contained in:
cynfo3000 2026-02-28 12:40:44 +01:00
parent 62eeaf79e2
commit 27d6492258
1 changed files with 284 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

284
docs/POSTGRES_SETUP.md Normal file
View File

@ -0,0 +1,284 @@
# PostgreSQL + TimescaleDB Setup
Anleitung fuer die Einrichtung von PostgreSQL mit TimescaleDB-Extension auf dem RMM-Backend-Server (192.168.85.13, Debian 13 Trixie).
## 1. PostgreSQL installieren
```bash
# Debian-Pakete
apt update
apt install -y postgresql postgresql-contrib postgresql-common
# Dienst starten und aktivieren
systemctl enable postgresql
systemctl start postgresql
# Version pruefen
psql --version
# Erwartet: psql (PostgreSQL) 17.x (Debian 13 Trixie liefert PG17)
```
## 2. TimescaleDB installieren
```bash
# TimescaleDB APT-Repository hinzufuegen
apt install -y curl gnupg
curl -s https://packagecloud.io/install/repositories/timescale/timescaledb/script.deb.sh | bash
# Bookworm-Repo verwenden (Trixie-Repo existiert noch nicht)
echo "deb https://packagecloud.io/timescale/timescaledb/debian/ bookworm main" \
> /etc/apt/sources.list.d/timescaledb.list
curl -sL https://packagecloud.io/timescale/timescaledb/gpgkey | gpg --dearmor > /etc/apt/trusted.gpg.d/timescaledb.gpg
apt update
# WICHTIG: Version muss zur installierten PG-Version passen
# Pruefen: dpkg -l | grep postgresql-17 → z.B. 17.8
# Dann passende TimescaleDB-Version installieren:
apt install -y timescaledb-2-loader-postgresql-17=2.25.1~debian12-1708 \
timescaledb-2-postgresql-17=2.25.1~debian12-1708
# shared_preload_libraries setzen
sed -i "s/#shared_preload_libraries = ''/shared_preload_libraries = 'timescaledb'/" \
/etc/postgresql/17/main/postgresql.conf
# PostgreSQL neustarten
systemctl restart postgresql
```
## 3. Datenbank und User anlegen
```bash
# Als postgres User
sudo -u postgres psql
```
```sql
-- Datenbank anlegen
CREATE DATABASE rmm;
-- User anlegen (Passwort anpassen!)
CREATE USER rmm WITH PASSWORD 'Cynf0-RMM-2026!';
-- Rechte vergeben
GRANT ALL PRIVILEGES ON DATABASE rmm TO rmm;
-- In die rmm-Datenbank wechseln
\c rmm
-- Schema-Rechte
GRANT ALL ON SCHEMA public TO rmm;
-- TimescaleDB Extension aktivieren
CREATE EXTENSION IF NOT EXISTS timescaledb;
-- Pruefen
\dx
-- Sollte timescaledb in der Liste zeigen
\q
```
## 4. pg_hba.conf anpassen (lokaler Zugriff)
```bash
# Zeile hinzufuegen (lokaler TCP-Zugriff fuer rmm User)
echo "host rmm rmm 127.0.0.1/32 scram-sha-256" >> /etc/postgresql/17/main/pg_hba.conf
# PostgreSQL neu laden
systemctl reload postgresql
```
## 5. Verbindung testen
```bash
psql -h 127.0.0.1 -U rmm -d rmm -c "SELECT version();"
# Passwort: Cynf0-RMM-2026!
psql -h 127.0.0.1 -U rmm -d rmm -c "SELECT extname, extversion FROM pg_extension WHERE extname = 'timescaledb';"
# Sollte timescaledb mit Version zeigen
```
## 6. Schema anlegen
Das Backend legt die Tabellen automatisch beim Start an (Auto-Migration). Die Struktur:
### Relationale Tabellen
```sql
-- Agent-Registry
CREATE TABLE IF NOT EXISTS agents (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
hostname TEXT NOT NULL,
ip TEXT NOT NULL,
opnsense_version TEXT NOT NULL DEFAULT '',
registered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
last_heartbeat TIMESTAMPTZ
);
-- Config-Backups (versioniert, dedupliziert nach Hash)
CREATE TABLE IF NOT EXISTS config_backups (
id SERIAL PRIMARY KEY,
agent_id TEXT NOT NULL REFERENCES agents(id) ON DELETE CASCADE,
hash TEXT NOT NULL,
size INTEGER NOT NULL,
config_xml TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_config_backups_agent ON config_backups(agent_id, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_config_backups_hash ON config_backups(agent_id, hash);
```
### TimescaleDB Hypertable (Metriken)
```sql
-- Metriken-Tabelle
CREATE TABLE IF NOT EXISTS metrics (
time TIMESTAMPTZ NOT NULL,
agent_id TEXT NOT NULL,
metric TEXT NOT NULL, -- z.B. 'cpu_usage', 'memory_used_percent', 'disk_used_percent'
value DOUBLE PRECISION NOT NULL,
tags JSONB -- z.B. {"interface":"igb0"}, {"disk":"/dev/ada0"}
);
-- In Hypertable umwandeln (automatische Partitionierung nach Zeit)
SELECT create_hypertable('metrics', 'time', if_not_exists => TRUE);
-- Index fuer schnelle Agent+Metrik-Abfragen
CREATE INDEX IF NOT EXISTS idx_metrics_agent_metric ON metrics(agent_id, metric, time DESC);
```
### Systemdaten-Snapshots (aktueller Zustand)
```sql
-- Aktueller Snapshot der vollstaendigen Systemdaten (JSON)
CREATE TABLE IF NOT EXISTS system_data (
agent_id TEXT PRIMARY KEY REFERENCES agents(id) ON DELETE CASCADE,
data_json JSONB NOT NULL,
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
## 7. Retention Policies einrichten
```sql
-- Metriken aelter als 90 Tage automatisch loeschen
SELECT add_retention_policy('metrics', INTERVAL '90 days');
-- Optional: Continuous Aggregate fuer stuendliche Durchschnitte
CREATE MATERIALIZED VIEW metrics_hourly
WITH (timescaledb.continuous) AS
SELECT
time_bucket('1 hour', time) AS bucket,
agent_id,
metric,
AVG(value) AS avg_value,
MIN(value) AS min_value,
MAX(value) AS max_value,
COUNT(*) AS sample_count
FROM metrics
GROUP BY bucket, agent_id, metric
WITH NO DATA;
-- Automatisch aktualisieren
SELECT add_continuous_aggregate_policy('metrics_hourly',
start_offset => INTERVAL '3 hours',
end_offset => INTERVAL '1 hour',
schedule_interval => INTERVAL '1 hour'
);
-- Stuendliche Aggregate 1 Jahr behalten
SELECT add_retention_policy('metrics_hourly', INTERVAL '365 days');
```
## 8. Compression aktivieren (optional, spart Speicher)
```sql
-- Compression fuer Metriken aelter als 7 Tage
ALTER TABLE metrics SET (
timescaledb.compress,
timescaledb.compress_segmentby = 'agent_id, metric',
timescaledb.compress_orderby = 'time DESC'
);
SELECT add_compression_policy('metrics', INTERVAL '7 days');
```
## 9. Backend-Konfiguration
In der `config.yaml` des Backends:
```yaml
# Alte SQLite-Config entfernen:
# db_path: "rmm.db"
# Neue PostgreSQL-Config:
database:
host: "127.0.0.1"
port: 5432
user: "rmm"
password: "Cynf0-RMM-2026!"
dbname: "rmm"
sslmode: "disable"
```
## 10. Metriken die der Agent sendet
Bei jedem Heartbeat (alle 60s) werden folgende Metriken geschrieben:
| Metrik | Beschreibung | Tags |
|--------|-------------|------|
| `cpu_usage` | CPU-Auslastung in Prozent | — |
| `memory_used_percent` | RAM-Nutzung in Prozent | — |
| `memory_used_bytes` | RAM genutzt in Bytes | — |
| `memory_total_bytes` | RAM gesamt in Bytes | — |
| `disk_used_percent` | Disk-Belegung pro Mountpoint | `{"mountpoint":"/", "dataset":"zroot/ROOT"}` |
| `uptime_seconds` | Uptime in Sekunden | — |
| `interface_rx_bytes` | Empfangene Bytes pro Interface | `{"interface":"igb0"}` |
| `interface_tx_bytes` | Gesendete Bytes pro Interface | `{"interface":"igb0"}` |
| `gateway_rtt_ms` | Gateway RTT in ms | `{"gateway":"WAN_GW"}` |
| `gateway_loss_percent` | Gateway Paketverlust | `{"gateway":"WAN_GW"}` |
## Nuetzliche Abfragen
```sql
-- CPU-Auslastung der letzten 24h (5-Minuten-Durchschnitt)
SELECT time_bucket('5 minutes', time) AS t,
agent_id,
AVG(value) AS cpu_avg
FROM metrics
WHERE metric = 'cpu_usage'
AND time > NOW() - INTERVAL '24 hours'
GROUP BY t, agent_id
ORDER BY t;
-- Aktuelle Werte aller Agents
SELECT DISTINCT ON (agent_id, metric)
agent_id, metric, value, time
FROM metrics
ORDER BY agent_id, metric, time DESC;
-- Disk-Warnung: > 80% belegt
SELECT agent_id, tags->>'mountpoint' AS mount,
value AS used_percent, time
FROM metrics
WHERE metric = 'disk_used_percent'
AND value > 80
AND time > NOW() - INTERVAL '1 hour'
ORDER BY value DESC;
```
## Zusammenfassung
| Komponente | Zweck |
|-----------|-------|
| PostgreSQL 17 | Relationale Daten (Agents, Backups) |
| TimescaleDB 2.x | Time-Series Metriken (CPU, RAM, Disk, ...) |
| `agents` | Agent-Registry |
| `config_backups` | Versionierte Firewall-Configs |
| `system_data` | Aktueller JSON-Snapshot |
| `metrics` | Hypertable mit 60s-Aufloesung |
| `metrics_hourly` | Continuous Aggregate (1h) |
| Retention | Raw: 90 Tage, Hourly: 365 Tage |
| Compression | Nach 7 Tagen automatisch |