Multi-source setup

If traffic comes in through more than one reverse proxy or host, GeoMetrikks can run as one full instance plus lightweight agents instead of one instance tailing everything remotely. Each agent runs next to its own access logs (same host or Docker network as the proxy it tails) and does the ingestion locally: tail, parse, geolocate, write, publish. One full instance owns everything else: the UI, the API, database migrations, the scheduler, and CrowdSec. Every agent and the full instance share one TimescaleDB. The live map stays in sync whichever process ingested a request, because every writer publishes committed events over PostgreSQL LISTEN/NOTIFY and the full instance’s /ws/live feed relays all of them.

Agents are not the only shape. If the log files already reach one machine (a shared mount, rsyslog, log shipping), a single full instance can tail them all and keep the sources apart by giving LOGPARSER_HOST_NAME a JSON list matched positionally to LOGPARSER_LOG_PATHS:

LOGPARSER_LOG_PATHS=["/var/log/access/edge-01.log", "/var/log/access/edge-02.log"]
LOGPARSER_HOST_NAME=["edge-01", "edge-02"]

Everything downstream (the access-logs hostname filter, the map’s Source filter, per-site homes) treats those files like traffic from separate agents.

An agent needs only APP_MODE=agent, database credentials for the shared instance, GeoIP credentials, and its own log mount:

services:
  agent:
    image: ghcr.io/gilbn/geometrikks:0.14.2   # same tag as the full instance
    restart: unless-stopped
    stop_grace_period: 20s
    environment:
      APP_MODE: agent
      DB_HOST: timescale.example.internal
      DB_PORT: "5432"
      DB_USER: geouser
      DB_PASSWORD: ${DB_PASSWORD}
      DB_DATABASE: geometrikks
      MAXMINDDB_USER_ID: ${MAXMINDDB_USER_ID}
      MAXMINDDB_LICENSE_KEY: ${MAXMINDDB_LICENSE_KEY}
      LOGPARSER_LOG_PATHS: '["/var/log/access/access.log"]'
      LOGPARSER_HOST_NAME: edge-01
    volumes:
      - geoip_data:/app/data/geoip
      - /var/log/nginx:/var/log/access:ro
    healthcheck:
      # The image's own healthcheck probes /health, which answers 200 for
      # the whole schema wait. /health/ready is the one that reports 503
      # while the agent waits for the primary to migrate.
      test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8000/health/ready', timeout=5).read()\" || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 3
      # The schema gate runs inside ASGI startup, so the port does not
      # accept at all for up to 120s. Without this grace window the first
      # probes fail a perfectly healthy cold start.
      start_period: 150s

volumes:
  geoip_data:

APP_MODE=agent is a headless process: it tails, geolocates, writes and publishes like a full instance, but serves only /health and /health/ready. No UI, no API, no OpenAPI schema, no session auth, so it needs no APP_ADMIN_PASSWORD. It downloads and refreshes its own GeoLite2 database, so it needs MaxMind credentials and a geoip volume. It never runs migrations or creates TimescaleDB objects; its writes are events and its own site-homes row. At startup it waits for the shared database’s schema to reach the revision it was built against. If that wait times out, the agent stays up in degraded mode rather than exiting, with /health answering 200 and /health/ready answering 503. An orchestrator with a readiness probe restarts it into a fresh wait; without one, restart the agent container yourself once the full instance has finished migrating.

Compose does not act on health status, so on plain Docker the probe above shows a stuck agent as unhealthy in docker ps and gates any depends_on: condition: service_healthy you add. Turning that signal into a restart takes Swarm, Kubernetes, or an autoheal sidecar. restart: unless-stopped will not do it: it reacts to a container exiting, and a waiting agent stays up.

CDN peer advisories for an agent’s tailed sources reach the head’s Settings > Status page. The head scans the shared database’s last hour of access-log rows every 5 minutes, covering only sources with LOGPARSER_SEND_LOGS=true. Private-peer advisories still surface only on the agent’s own /health and logs, since those lines are never stored. See docs/proxy-setup.md for details.

The reverse case works too. To keep a full instance’s UI and API without it tailing local files (a machine that only hosts the app, with all traffic ingested by agents elsewhere), set:

LOGPARSER_ENABLED=false

It still serves the UI, API, migrations, scheduler and CrowdSec; it never tails a log file itself.

Site homes. Live map routes fly to the home location of the source that recorded them, one beacon per site. Each ingesting instance detects its own public-IP location and re-checks it every MAP_HOME_REFRESH_HOURS (default 24h). When detection is wrong for a source (CGNAT, a VPN egress, or logs shipped from another machine), pin that hostname on the full instance with MAP_HOME_LOCATIONS, for example MAP_HOME_LOCATIONS={"edge-01": [60.39, 5.32]}. Overrides win over detection, and removing one restores it. Settings > Status lists each source’s home and whether it came from detection or an override.

[!WARNING] Trust model. Agents authenticate with the database using ordinary database credentials, and the app does not care where data comes from. Anyone who can run an agent has full read/write access to the entire database: all traffic history from every source, any hostname, no tenant isolation, no per-agent identity, and no revocation short of rotating the shared password. Sharing one instance across parties works as long as everyone understands they share everything. Run agent connections over a VPN or tailnet, not the open internet.

Keep versions aligned. Run the same image tag on every agent and the full instance. An agent tolerates the shared database running slightly ahead of its own bundled schema (a full instance mid rolling-restart) by logging a warning and proceeding rather than refusing to start. That is an allowance for a brief mismatch, not a reason to run agents and the full instance on different versions.

CrowdSec. Point CROWDSEC_LAPI_URL and the bouncer/machine credentials (see CrowdSec integration) at the central LAPI on the full instance only; per-machine CrowdSec agents keep reporting to that same LAPI as usual. GeoMetrikks agents ignore CROWDSEC_* settings; ban visibility and management stay with the full instance.