README: Vollstaendige Installationsanleitung Backend+Agent, alle SQLite-Referenzen durch PostgreSQL+TimescaleDB ersetzt
This commit is contained in:
cynfo3000
parent
3f3d555658
commit
d8bc99873f
244
README.md
244
README.md
@ -13,7 +13,8 @@ Go-basierter Agent + Backend mit WebSocket-Kommunikation und Session-basierten T
|
|||||||
│ rmm-agent │ Heartbeat │ REST API │
|
│ rmm-agent │ Heartbeat │ REST API │
|
||||||
│ - Collectors │ WebSocket │ WebSocket Hub │
|
│ - Collectors │ WebSocket │ WebSocket Hub │
|
||||||
│ - WS Client │ Binary Msgs │ Tunnel Mgr │
|
│ - WS Client │ Binary Msgs │ Tunnel Mgr │
|
||||||
│ - Tunnel Mgr │ │ SQLite DB │
|
│ - Tunnel Mgr │ │ PostgreSQL + │
|
||||||
|
│ │ │ TimescaleDB │
|
||||||
└────────────────┘ └────────────────┘
|
└────────────────┘ └────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -87,9 +88,56 @@ Client ──► Backend:ProxyPort ──► WebSocket (Binary) ──► Agent
|
|||||||
- **REST API**: Agent-Registrierung, Heartbeat, Systemdaten, Tunnel-Management
|
- **REST API**: Agent-Registrierung, Heartbeat, Systemdaten, Tunnel-Management
|
||||||
- **WebSocket Hub**: Verwaltet Agent-Verbindungen, routet Commands und Binary-Messages
|
- **WebSocket Hub**: Verwaltet Agent-Verbindungen, routet Commands und Binary-Messages
|
||||||
- **Tunnel Manager**: Proxy-Listener, Session-Tracking, Ready-Handshake
|
- **Tunnel Manager**: Proxy-Listener, Session-Tracking, Ready-Handshake
|
||||||
- **SQLite DB**: Persistente Agent- und Systemdaten
|
- **PostgreSQL + TimescaleDB**: Relationale Daten + Time-Series Metriken
|
||||||
|
|
||||||
## Build
|
## Voraussetzungen
|
||||||
|
|
||||||
|
- **Build-System**: Go 1.24+ (Cross-Compilation, kein CGO)
|
||||||
|
- **Backend-Server**: Linux (Debian 12+), PostgreSQL 17+, TimescaleDB 2.x
|
||||||
|
- **Agent-Ziel**: OPNsense / FreeBSD (amd64)
|
||||||
|
|
||||||
|
## Installation (Backend)
|
||||||
|
|
||||||
|
### 1. PostgreSQL + TimescaleDB installieren
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Debian 12+ (Bookworm/Trixie)
|
||||||
|
apt install -y postgresql postgresql-client
|
||||||
|
|
||||||
|
# TimescaleDB Repository hinzufuegen
|
||||||
|
echo "deb https://packagecloud.io/timescale/timescaledb/debian/ bookworm main" \
|
||||||
|
> /etc/apt/sources.list.d/timescaledb.list
|
||||||
|
curl -L https://packagecloud.io/timescale/timescaledb/gpgkey | gpg --dearmor \
|
||||||
|
> /etc/apt/trusted.gpg.d/timescaledb.gpg
|
||||||
|
apt update
|
||||||
|
|
||||||
|
# TimescaleDB passend zur PG-Version installieren (z.B. PG 17)
|
||||||
|
apt install -y timescaledb-2-postgresql-17
|
||||||
|
|
||||||
|
# TimescaleDB in postgresql.conf aktivieren
|
||||||
|
echo "shared_preload_libraries = 'timescaledb'" >> /etc/postgresql/17/main/conf.d/timescaledb.conf
|
||||||
|
systemctl restart postgresql
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Datenbank und User anlegen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo -u postgres psql <<EOF
|
||||||
|
CREATE USER rmm WITH PASSWORD 'SICHERES_PASSWORT';
|
||||||
|
CREATE DATABASE rmm OWNER rmm;
|
||||||
|
\c rmm
|
||||||
|
CREATE EXTENSION IF NOT EXISTS timescaledb;
|
||||||
|
GRANT ALL ON SCHEMA public TO rmm;
|
||||||
|
EOF
|
||||||
|
```
|
||||||
|
|
||||||
|
Verbindung testen:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
psql -h 127.0.0.1 -U rmm -d rmm -c "SELECT default_version FROM pg_available_extensions WHERE name='timescaledb';"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Build
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
make all # Backend (linux/amd64) + Agent (freebsd/amd64)
|
make all # Backend (linux/amd64) + Agent (freebsd/amd64)
|
||||||
@ -97,57 +145,181 @@ make backend # Nur Backend
|
|||||||
make agent # Nur Agent
|
make agent # Nur Agent
|
||||||
```
|
```
|
||||||
|
|
||||||
Binaries in `build/`. Benoetigt Go 1.22+.
|
Binaries landen in `build/`. Kein CGO — reine Go-Compilation.
|
||||||
|
|
||||||
## Konfiguration
|
### 4. TLS-Zertifikate generieren
|
||||||
|
|
||||||
### Backend (`config.yaml`)
|
```bash
|
||||||
|
make certs
|
||||||
|
```
|
||||||
|
|
||||||
|
Oder manuell:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p certs
|
||||||
|
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 \
|
||||||
|
-keyout certs/server.key -out certs/server.crt \
|
||||||
|
-days 3650 -nodes \
|
||||||
|
-subj "/CN=rmm-backend/O=RMM" \
|
||||||
|
-addext "subjectAltName=IP:BACKEND_IP,IP:127.0.0.1,DNS:localhost"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Backend konfigurieren (`config.yaml`)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
listen_addr: ":8443"
|
listen_addr: ":8443"
|
||||||
tls_cert: "certs/server.crt" # Wird automatisch generiert
|
tls_cert: "certs/server.crt"
|
||||||
tls_key: "certs/server.key"
|
tls_key: "certs/server.key"
|
||||||
db_path: "rmm.db"
|
|
||||||
api_keys:
|
api_keys:
|
||||||
- "dein-api-key-hier"
|
- "SICHERER_API_KEY"
|
||||||
|
|
||||||
|
database:
|
||||||
|
host: "127.0.0.1"
|
||||||
|
port: 5432
|
||||||
|
user: "rmm"
|
||||||
|
password: "SICHERES_PASSWORT"
|
||||||
|
dbname: "rmm"
|
||||||
|
sslmode: "disable" # "require" wenn PG ueber Netz
|
||||||
```
|
```
|
||||||
|
|
||||||
### Agent (`config.yaml`)
|
Alternativ per Environment-Variablen: `RMM_LISTEN_ADDR`, `RMM_TLS_CERT`, `RMM_TLS_KEY`, `RMM_DB_HOST`, `RMM_DB_PASSWORD`.
|
||||||
|
|
||||||
|
### 6. Backend deployen und starten
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Verzeichnis anlegen
|
||||||
|
ssh root@BACKEND_IP "mkdir -p /opt/rmm/certs"
|
||||||
|
|
||||||
|
# Binary + Config + Certs kopieren
|
||||||
|
scp build/rmm-backend root@BACKEND_IP:/opt/rmm/rmm-backend
|
||||||
|
scp config.yaml root@BACKEND_IP:/opt/rmm/config.yaml
|
||||||
|
scp certs/server.* root@BACKEND_IP:/opt/rmm/certs/
|
||||||
|
|
||||||
|
# Systemd-Service einrichten
|
||||||
|
ssh root@BACKEND_IP 'cat > /etc/systemd/system/rmm-backend.service << EOF
|
||||||
|
[Unit]
|
||||||
|
Description=RMM Backend
|
||||||
|
After=network.target postgresql.service
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
WorkingDirectory=/opt/rmm
|
||||||
|
ExecStart=/opt/rmm/rmm-backend /opt/rmm/config.yaml
|
||||||
|
Restart=always
|
||||||
|
RestartSec=5
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
EOF'
|
||||||
|
|
||||||
|
ssh root@BACKEND_IP "systemctl daemon-reload && systemctl enable rmm-backend && systemctl start rmm-backend"
|
||||||
|
```
|
||||||
|
|
||||||
|
Oder per Makefile: `make deploy-backend`
|
||||||
|
|
||||||
|
### 7. Starten und verifizieren
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Status pruefen
|
||||||
|
systemctl status rmm-backend
|
||||||
|
|
||||||
|
# Log pruefen
|
||||||
|
journalctl -u rmm-backend -f
|
||||||
|
|
||||||
|
# API testen
|
||||||
|
curl -sk -H "X-API-Key: DEIN_API_KEY" https://BACKEND_IP:8443/api/v1/agents
|
||||||
|
```
|
||||||
|
|
||||||
|
Beim ersten Start:
|
||||||
|
- Tabellen `agents`, `system_data`, `config_backups`, `agent_events` werden automatisch angelegt
|
||||||
|
- Hypertable `metrics` wird erstellt (TimescaleDB)
|
||||||
|
- Retention Policy (90 Tage) und Compression Policy (nach 7 Tagen) werden automatisch gesetzt
|
||||||
|
|
||||||
|
### 8. Datenbank-Schema (automatisch)
|
||||||
|
|
||||||
|
Das Backend erstellt folgendes Schema bei erstem Start:
|
||||||
|
|
||||||
|
| Tabelle | Typ | Beschreibung |
|
||||||
|
|---------|-----|-------------|
|
||||||
|
| `agents` | Relational | Agent-Registry (ID, Name, Hostname, IP, Version, Heartbeat) |
|
||||||
|
| `system_data` | Relational | Letzte Systemdaten pro Agent (JSONB) |
|
||||||
|
| `config_backups` | Relational | OPNsense Config-Backups (dedupliziert per SHA256) |
|
||||||
|
| `agent_events` | Relational | Online/Offline/Connected Events |
|
||||||
|
| `metrics` | TimescaleDB Hypertable | Time-Series Metriken (CPU, RAM, Disk, Network, Gateway) |
|
||||||
|
|
||||||
|
TimescaleDB Policies:
|
||||||
|
- **Retention**: Rohdaten werden nach 90 Tagen automatisch geloescht
|
||||||
|
- **Compression**: Daten aelter als 7 Tage werden komprimiert (segmentby: agent_id+metric)
|
||||||
|
|
||||||
|
## Installation (Agent / OPNsense)
|
||||||
|
|
||||||
|
### 1. Agent bauen
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make agent # Erzeugt build/rmm-agent (freebsd/amd64)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Agent konfigurieren (`agent/config.yaml`)
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
backend_url: "https://backend-ip:8443"
|
backend_url: "https://BACKEND_IP:8443"
|
||||||
api_key: "dein-api-key-hier"
|
api_key: "SICHERER_API_KEY"
|
||||||
interval_seconds: 60
|
interval_seconds: 60
|
||||||
agent_name: "opnsense-fw01"
|
agent_name: "opnsense-fw01"
|
||||||
insecure: true # Self-signed Certs akzeptieren
|
insecure: true # Self-signed Certs akzeptieren (false bei CA-Certs)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Deployment
|
### 3. Agent deployen
|
||||||
|
|
||||||
### Backend
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Build + Copy
|
# Verzeichnis anlegen + Binary + Config kopieren
|
||||||
make backend
|
ssh root@OPNSENSE "mkdir -p /usr/local/rmm"
|
||||||
scp build/rmm-backend user@backend:/pfad/rmm-backend
|
scp build/rmm-agent root@OPNSENSE:/usr/local/rmm/rmm-agent
|
||||||
|
scp agent/config.yaml root@OPNSENSE:/usr/local/rmm/config.yaml
|
||||||
# Starten (als normaler User, kein root noetig)
|
|
||||||
cd /pfad && nohup ./rmm-backend config.yaml > rmm-backend.log 2>&1 &
|
|
||||||
```
|
|
||||||
|
|
||||||
### Agent (OPNsense/FreeBSD)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Build + Copy
|
|
||||||
make agent
|
|
||||||
scp build/rmm-agent root@opnsense:/usr/local/rmm/rmm-agent
|
|
||||||
|
|
||||||
# Starten via FreeBSD daemon
|
# Starten via FreeBSD daemon
|
||||||
daemon -f -p /var/run/rmm-agent.pid -o /usr/local/rmm/rmm-agent.log \
|
ssh root@OPNSENSE 'daemon -f -p /var/run/rmm-agent.pid -o /usr/local/rmm/rmm-agent.log \
|
||||||
/usr/local/rmm/rmm-agent --config /usr/local/rmm/config.yaml --insecure
|
/usr/local/rmm/rmm-agent --config /usr/local/rmm/config.yaml --insecure'
|
||||||
```
|
```
|
||||||
|
|
||||||
**Wichtig**: Agent-ID wird persistent in `/usr/local/rmm/agent_id` gespeichert.
|
Oder per Makefile: `make deploy-agent`
|
||||||
|
|
||||||
|
### 4. Autostart einrichten (rc.d)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh root@OPNSENSE 'cat > /usr/local/etc/rc.d/rmm_agent << RCEOF
|
||||||
|
#!/bin/sh
|
||||||
|
# PROVIDE: rmm_agent
|
||||||
|
# REQUIRE: NETWORKING
|
||||||
|
# KEYWORD: shutdown
|
||||||
|
|
||||||
|
. /etc/rc.subr
|
||||||
|
|
||||||
|
name="rmm_agent"
|
||||||
|
rcvar="rmm_agent_enable"
|
||||||
|
pidfile="/var/run/${name}.pid"
|
||||||
|
command="/usr/local/rmm/rmm-agent"
|
||||||
|
command_args="--config /usr/local/rmm/config.yaml --insecure &"
|
||||||
|
|
||||||
|
load_rc_config $name
|
||||||
|
run_rc_command "$1"
|
||||||
|
RCEOF
|
||||||
|
chmod +x /usr/local/etc/rc.d/rmm_agent'
|
||||||
|
|
||||||
|
ssh root@OPNSENSE 'sysrc rmm_agent_enable="YES" && service rmm_agent restart'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Verifizieren
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Agent-Log pruefen
|
||||||
|
ssh root@OPNSENSE "tail -20 /usr/local/rmm/rmm-agent.log"
|
||||||
|
|
||||||
|
# Auf dem Backend: Agent sollte erscheinen
|
||||||
|
curl -sk -H "X-API-Key: DEIN_API_KEY" https://BACKEND_IP:8443/api/v1/agents
|
||||||
|
```
|
||||||
|
|
||||||
|
**Wichtig**: Agent-ID wird persistent in `/usr/local/rmm/agent_id` gespeichert. Bei Neuinstallation bleibt die ID erhalten.
|
||||||
|
|
||||||
## API
|
## API
|
||||||
|
|
||||||
@ -239,7 +411,7 @@ Retention: 90 Tage (Raw), Compression nach 7 Tagen.
|
|||||||
| GET | `/api/v1/agents/{id}/backups/diff?a={id}&b={id}` | Zeilenweises Diff zwischen zwei Backups |
|
| GET | `/api/v1/agents/{id}/backups/diff?a={id}&b={id}` | Zeilenweises Diff zwischen zwei Backups |
|
||||||
|
|
||||||
Backups werden dedupliziert: gleicher SHA256-Hash = kein neuer Eintrag.
|
Backups werden dedupliziert: gleicher SHA256-Hash = kein neuer Eintrag.
|
||||||
Config-XML wird Base64-kodiert vom Agent zum Backend uebertragen und als Klartext in SQLite gespeichert.
|
Config-XML wird Base64-kodiert vom Agent zum Backend uebertragen und als Klartext in PostgreSQL gespeichert.
|
||||||
|
|
||||||
### Tunnel-Management
|
### Tunnel-Management
|
||||||
|
|
||||||
@ -574,8 +746,8 @@ rmm/
|
|||||||
|
|
||||||
## Technische Details
|
## Technische Details
|
||||||
|
|
||||||
- **PostgreSQL + TimescaleDB**: Agent-Registry, Config-Backups, Time-Series Metriken (pgx Driver)
|
- **PostgreSQL 17 + TimescaleDB 2.x**: Agent-Registry, Config-Backups, Time-Series Metriken
|
||||||
- **CGO-frei**: Kein CGO noetig (pure-Go PostgreSQL Driver)
|
- **Pure-Go**: pgx/v5 Driver, kein CGO noetig
|
||||||
- **Cross-Compilation**: Backend linux/amd64, Agent freebsd/amd64
|
- **Cross-Compilation**: Backend linux/amd64, Agent freebsd/amd64
|
||||||
- **TLS**: Self-signed Zertifikate (automatisch generiert)
|
- **TLS**: Self-signed Zertifikate (automatisch generiert)
|
||||||
- **WebSocket**: gorilla/websocket, Binary Messages fuer Tunnel-Daten
|
- **WebSocket**: gorilla/websocket, Binary Messages fuer Tunnel-Daten
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user