PostgreSQL + TimescaleDB Setup-Anleitung und Installation auf .13
- PostgreSQL 17.8 auf Debian 13 Trixie installiert - TimescaleDB 2.25.1 Extension aktiviert - DB 'rmm', User 'rmm' angelegt - docs/POSTGRES_SETUP.md: Komplette Anleitung - Installation (PG17 + TimescaleDB via Bookworm-Repo) - Schema (agents, config_backups, metrics Hypertable, system_data) - Retention Policy (90 Tage Raw, 365 Tage Hourly Aggregate) - Compression (nach 7 Tagen) - Continuous Aggregate (stuendliche Durchschnitte) - Nuetzliche SQL-Abfragen - Backend config.yaml Anpassung
This commit is contained in:
cynfo3000
parent
62eeaf79e2
commit
27d6492258
284
docs/POSTGRES_SETUP.md
Normal file
284
docs/POSTGRES_SETUP.md
Normal file
@ -0,0 +1,284 @@
|
||||
# PostgreSQL + TimescaleDB Setup
|
||||
|
||||
Anleitung fuer die Einrichtung von PostgreSQL mit TimescaleDB-Extension auf dem RMM-Backend-Server (192.168.85.13, Debian 13 Trixie).
|
||||
|
||||
## 1. PostgreSQL installieren
|
||||
|
||||
```bash
|
||||
# Debian-Pakete
|
||||
apt update
|
||||
apt install -y postgresql postgresql-contrib postgresql-common
|
||||
|
||||
# Dienst starten und aktivieren
|
||||
systemctl enable postgresql
|
||||
systemctl start postgresql
|
||||
|
||||
# Version pruefen
|
||||
psql --version
|
||||
# Erwartet: psql (PostgreSQL) 17.x (Debian 13 Trixie liefert PG17)
|
||||
```
|
||||
|
||||
## 2. TimescaleDB installieren
|
||||
|
||||
```bash
|
||||
# TimescaleDB APT-Repository hinzufuegen
|
||||
apt install -y curl gnupg
|
||||
curl -s https://packagecloud.io/install/repositories/timescale/timescaledb/script.deb.sh | bash
|
||||
|
||||
# Bookworm-Repo verwenden (Trixie-Repo existiert noch nicht)
|
||||
echo "deb https://packagecloud.io/timescale/timescaledb/debian/ bookworm main" \
|
||||
> /etc/apt/sources.list.d/timescaledb.list
|
||||
curl -sL https://packagecloud.io/timescale/timescaledb/gpgkey | gpg --dearmor > /etc/apt/trusted.gpg.d/timescaledb.gpg
|
||||
apt update
|
||||
|
||||
# WICHTIG: Version muss zur installierten PG-Version passen
|
||||
# Pruefen: dpkg -l | grep postgresql-17 → z.B. 17.8
|
||||
# Dann passende TimescaleDB-Version installieren:
|
||||
apt install -y timescaledb-2-loader-postgresql-17=2.25.1~debian12-1708 \
|
||||
timescaledb-2-postgresql-17=2.25.1~debian12-1708
|
||||
|
||||
# shared_preload_libraries setzen
|
||||
sed -i "s/#shared_preload_libraries = ''/shared_preload_libraries = 'timescaledb'/" \
|
||||
/etc/postgresql/17/main/postgresql.conf
|
||||
|
||||
# PostgreSQL neustarten
|
||||
systemctl restart postgresql
|
||||
```
|
||||
|
||||
## 3. Datenbank und User anlegen
|
||||
|
||||
```bash
|
||||
# Als postgres User
|
||||
sudo -u postgres psql
|
||||
```
|
||||
|
||||
```sql
|
||||
-- Datenbank anlegen
|
||||
CREATE DATABASE rmm;
|
||||
|
||||
-- User anlegen (Passwort anpassen!)
|
||||
CREATE USER rmm WITH PASSWORD 'Cynf0-RMM-2026!';
|
||||
|
||||
-- Rechte vergeben
|
||||
GRANT ALL PRIVILEGES ON DATABASE rmm TO rmm;
|
||||
|
||||
-- In die rmm-Datenbank wechseln
|
||||
\c rmm
|
||||
|
||||
-- Schema-Rechte
|
||||
GRANT ALL ON SCHEMA public TO rmm;
|
||||
|
||||
-- TimescaleDB Extension aktivieren
|
||||
CREATE EXTENSION IF NOT EXISTS timescaledb;
|
||||
|
||||
-- Pruefen
|
||||
\dx
|
||||
-- Sollte timescaledb in der Liste zeigen
|
||||
|
||||
\q
|
||||
```
|
||||
|
||||
## 4. pg_hba.conf anpassen (lokaler Zugriff)
|
||||
|
||||
```bash
|
||||
# Zeile hinzufuegen (lokaler TCP-Zugriff fuer rmm User)
|
||||
echo "host rmm rmm 127.0.0.1/32 scram-sha-256" >> /etc/postgresql/17/main/pg_hba.conf
|
||||
|
||||
# PostgreSQL neu laden
|
||||
systemctl reload postgresql
|
||||
```
|
||||
|
||||
## 5. Verbindung testen
|
||||
|
||||
```bash
|
||||
psql -h 127.0.0.1 -U rmm -d rmm -c "SELECT version();"
|
||||
# Passwort: Cynf0-RMM-2026!
|
||||
|
||||
psql -h 127.0.0.1 -U rmm -d rmm -c "SELECT extname, extversion FROM pg_extension WHERE extname = 'timescaledb';"
|
||||
# Sollte timescaledb mit Version zeigen
|
||||
```
|
||||
|
||||
## 6. Schema anlegen
|
||||
|
||||
Das Backend legt die Tabellen automatisch beim Start an (Auto-Migration). Die Struktur:
|
||||
|
||||
### Relationale Tabellen
|
||||
|
||||
```sql
|
||||
-- Agent-Registry
|
||||
CREATE TABLE IF NOT EXISTS agents (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
hostname TEXT NOT NULL,
|
||||
ip TEXT NOT NULL,
|
||||
opnsense_version TEXT NOT NULL DEFAULT '',
|
||||
registered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
last_heartbeat TIMESTAMPTZ
|
||||
);
|
||||
|
||||
-- Config-Backups (versioniert, dedupliziert nach Hash)
|
||||
CREATE TABLE IF NOT EXISTS config_backups (
|
||||
id SERIAL PRIMARY KEY,
|
||||
agent_id TEXT NOT NULL REFERENCES agents(id) ON DELETE CASCADE,
|
||||
hash TEXT NOT NULL,
|
||||
size INTEGER NOT NULL,
|
||||
config_xml TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_config_backups_agent ON config_backups(agent_id, created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_config_backups_hash ON config_backups(agent_id, hash);
|
||||
```
|
||||
|
||||
### TimescaleDB Hypertable (Metriken)
|
||||
|
||||
```sql
|
||||
-- Metriken-Tabelle
|
||||
CREATE TABLE IF NOT EXISTS metrics (
|
||||
time TIMESTAMPTZ NOT NULL,
|
||||
agent_id TEXT NOT NULL,
|
||||
metric TEXT NOT NULL, -- z.B. 'cpu_usage', 'memory_used_percent', 'disk_used_percent'
|
||||
value DOUBLE PRECISION NOT NULL,
|
||||
tags JSONB -- z.B. {"interface":"igb0"}, {"disk":"/dev/ada0"}
|
||||
);
|
||||
|
||||
-- In Hypertable umwandeln (automatische Partitionierung nach Zeit)
|
||||
SELECT create_hypertable('metrics', 'time', if_not_exists => TRUE);
|
||||
|
||||
-- Index fuer schnelle Agent+Metrik-Abfragen
|
||||
CREATE INDEX IF NOT EXISTS idx_metrics_agent_metric ON metrics(agent_id, metric, time DESC);
|
||||
```
|
||||
|
||||
### Systemdaten-Snapshots (aktueller Zustand)
|
||||
|
||||
```sql
|
||||
-- Aktueller Snapshot der vollstaendigen Systemdaten (JSON)
|
||||
CREATE TABLE IF NOT EXISTS system_data (
|
||||
agent_id TEXT PRIMARY KEY REFERENCES agents(id) ON DELETE CASCADE,
|
||||
data_json JSONB NOT NULL,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
## 7. Retention Policies einrichten
|
||||
|
||||
```sql
|
||||
-- Metriken aelter als 90 Tage automatisch loeschen
|
||||
SELECT add_retention_policy('metrics', INTERVAL '90 days');
|
||||
|
||||
-- Optional: Continuous Aggregate fuer stuendliche Durchschnitte
|
||||
CREATE MATERIALIZED VIEW metrics_hourly
|
||||
WITH (timescaledb.continuous) AS
|
||||
SELECT
|
||||
time_bucket('1 hour', time) AS bucket,
|
||||
agent_id,
|
||||
metric,
|
||||
AVG(value) AS avg_value,
|
||||
MIN(value) AS min_value,
|
||||
MAX(value) AS max_value,
|
||||
COUNT(*) AS sample_count
|
||||
FROM metrics
|
||||
GROUP BY bucket, agent_id, metric
|
||||
WITH NO DATA;
|
||||
|
||||
-- Automatisch aktualisieren
|
||||
SELECT add_continuous_aggregate_policy('metrics_hourly',
|
||||
start_offset => INTERVAL '3 hours',
|
||||
end_offset => INTERVAL '1 hour',
|
||||
schedule_interval => INTERVAL '1 hour'
|
||||
);
|
||||
|
||||
-- Stuendliche Aggregate 1 Jahr behalten
|
||||
SELECT add_retention_policy('metrics_hourly', INTERVAL '365 days');
|
||||
```
|
||||
|
||||
## 8. Compression aktivieren (optional, spart Speicher)
|
||||
|
||||
```sql
|
||||
-- Compression fuer Metriken aelter als 7 Tage
|
||||
ALTER TABLE metrics SET (
|
||||
timescaledb.compress,
|
||||
timescaledb.compress_segmentby = 'agent_id, metric',
|
||||
timescaledb.compress_orderby = 'time DESC'
|
||||
);
|
||||
|
||||
SELECT add_compression_policy('metrics', INTERVAL '7 days');
|
||||
```
|
||||
|
||||
## 9. Backend-Konfiguration
|
||||
|
||||
In der `config.yaml` des Backends:
|
||||
|
||||
```yaml
|
||||
# Alte SQLite-Config entfernen:
|
||||
# db_path: "rmm.db"
|
||||
|
||||
# Neue PostgreSQL-Config:
|
||||
database:
|
||||
host: "127.0.0.1"
|
||||
port: 5432
|
||||
user: "rmm"
|
||||
password: "Cynf0-RMM-2026!"
|
||||
dbname: "rmm"
|
||||
sslmode: "disable"
|
||||
```
|
||||
|
||||
## 10. Metriken die der Agent sendet
|
||||
|
||||
Bei jedem Heartbeat (alle 60s) werden folgende Metriken geschrieben:
|
||||
|
||||
| Metrik | Beschreibung | Tags |
|
||||
|--------|-------------|------|
|
||||
| `cpu_usage` | CPU-Auslastung in Prozent | — |
|
||||
| `memory_used_percent` | RAM-Nutzung in Prozent | — |
|
||||
| `memory_used_bytes` | RAM genutzt in Bytes | — |
|
||||
| `memory_total_bytes` | RAM gesamt in Bytes | — |
|
||||
| `disk_used_percent` | Disk-Belegung pro Mountpoint | `{"mountpoint":"/", "dataset":"zroot/ROOT"}` |
|
||||
| `uptime_seconds` | Uptime in Sekunden | — |
|
||||
| `interface_rx_bytes` | Empfangene Bytes pro Interface | `{"interface":"igb0"}` |
|
||||
| `interface_tx_bytes` | Gesendete Bytes pro Interface | `{"interface":"igb0"}` |
|
||||
| `gateway_rtt_ms` | Gateway RTT in ms | `{"gateway":"WAN_GW"}` |
|
||||
| `gateway_loss_percent` | Gateway Paketverlust | `{"gateway":"WAN_GW"}` |
|
||||
|
||||
## Nuetzliche Abfragen
|
||||
|
||||
```sql
|
||||
-- CPU-Auslastung der letzten 24h (5-Minuten-Durchschnitt)
|
||||
SELECT time_bucket('5 minutes', time) AS t,
|
||||
agent_id,
|
||||
AVG(value) AS cpu_avg
|
||||
FROM metrics
|
||||
WHERE metric = 'cpu_usage'
|
||||
AND time > NOW() - INTERVAL '24 hours'
|
||||
GROUP BY t, agent_id
|
||||
ORDER BY t;
|
||||
|
||||
-- Aktuelle Werte aller Agents
|
||||
SELECT DISTINCT ON (agent_id, metric)
|
||||
agent_id, metric, value, time
|
||||
FROM metrics
|
||||
ORDER BY agent_id, metric, time DESC;
|
||||
|
||||
-- Disk-Warnung: > 80% belegt
|
||||
SELECT agent_id, tags->>'mountpoint' AS mount,
|
||||
value AS used_percent, time
|
||||
FROM metrics
|
||||
WHERE metric = 'disk_used_percent'
|
||||
AND value > 80
|
||||
AND time > NOW() - INTERVAL '1 hour'
|
||||
ORDER BY value DESC;
|
||||
```
|
||||
|
||||
## Zusammenfassung
|
||||
|
||||
| Komponente | Zweck |
|
||||
|-----------|-------|
|
||||
| PostgreSQL 17 | Relationale Daten (Agents, Backups) |
|
||||
| TimescaleDB 2.x | Time-Series Metriken (CPU, RAM, Disk, ...) |
|
||||
| `agents` | Agent-Registry |
|
||||
| `config_backups` | Versionierte Firewall-Configs |
|
||||
| `system_data` | Aktueller JSON-Snapshot |
|
||||
| `metrics` | Hypertable mit 60s-Aufloesung |
|
||||
| `metrics_hourly` | Continuous Aggregate (1h) |
|
||||
| Retention | Raw: 90 Tage, Hourly: 365 Tage |
|
||||
| Compression | Nach 7 Tagen automatisch |
|
||||
Loading…
x
Reference in New Issue
Block a user