NASA Log Parsing → Prometheus → Grafana

A Python application that parses the NASA July 1995 HTTP access log, exposes derived metrics in the Prometheus exposition format, and visualizes them through a provisioned Grafana dashboard. The full stack (app + Prometheus + Grafana) is orchestrated with Docker Compose.

The application is intentionally memory-conservative: it streams log entries through a generator and processes them in time-bounded "ticks", so the full ~205 MB log is never held in memory at once.

---

Table of Contents

• Architecture
• Features
• Project Layout
• Requirements
• Quick Start (Docker Compose)
• Local Development (without Docker)
• Run Modes
• Exposed Metrics
• HTTP Endpoints
• Prometheus Configuration
• Alerting Rules
• Grafana Dashboard
• Testing
• Configuration Reference
• Data Source

---

Architecture

┌──────────────────────┐     scrape /metrics      ┌──────────────────┐     query     ┌──────────────────┐
│  app (Python:3.14)   │ ───────────────────────► │ Prometheus       │ ────────────► │ Grafana          │
│  port 8000           │       every 15s          │ port 9090        │               │ port 3000        │
│  - log parser        │                          │ - alert_rules    │               │ - provisioned DS │
│  - metrics formatter │                          │                  │               │ - dashboard      │
│  - HTTP server       │                          │                  │               │                  │
└──────────────────────┘                          └──────────────────┘               └──────────────────┘
         ▲
         │ reads
         │
  data/NASA_access_log_Jul95

The app spins up two threads inside one process:

1. Background processor — pulls entries off the parser generator in tick() windows (default 15 s) and updates an in-memory MetricsCollector.
2. HTTP server — serves /metrics and /health. Each scrape renders the collector's current state plus a fresh CPU/memory sample from psutil.

A lock guards the counters so the background thread and HTTP handler can read/write safely.

---

Features

• Apache Common Log Format parser with a regex front door and strict timestamp parsing (%d/%b/%Y:%H:%M:%S %z). Malformed lines are counted and skipped.
• Streaming, time-bounded ingestion via MetricsCollector.tick(duration=15) — bounded memory regardless of log size.
• Prometheus exposition format for HTTP status buckets, top hosts, host system metrics (CPU/RAM via psutil), and processing progress.
• Three run modes: HTTP server, periodic file writer (textfile collector style), and stdout printer.
• Multi-stage Docker build (Python 3.14-slim) that downloads and unpacks the NASA log inside the image at build time.
• Docker Compose stack: app + Prometheus + Grafana with auto-provisioned data source and dashboard.
• Alert rules for app-down, high CPU, high memory, 5xx errors, and stalled processing.
• Container resource limits: 0.20 CPU / 128 MiB memory on the app service — a deliberate constraint that the streaming design is built to respect.
• Health endpoint (/health) wired into both the Dockerfile HEALTHCHECK and the Compose service healthcheck.
• Unit tests with pytest and coverage support.

---

Project Layout

.
├── Dockerfile                              # multi-stage build, downloads NASA log
├── requirements.txt                        # psutil, pytest, pytest-cov
├── src/
│   ├── __main__.py                         # CLI entrypoint: python -m src <logfile> [--mode ...]
│   ├── log_parser.py                       # CLF regex parser, generator, status buckets, top hosts
│   ├── metrics_formatter.py                # MetricsCollector + Prometheus exposition formatters
│   ├── metrics_server.py                   # http.server-based /metrics + /health HTTP server
│   └── system_metrics.py                   # psutil CPU/RAM probes
├── tests/                                  # pytest suite
│   ├── test_log_parser.py
│   ├── test_metrics_formatter.py
│   ├── test_metrics_server.py
│   └── test_system_metrics.py
├── docker/
│   ├── docker-compose.yml                  # app + Prometheus + Grafana
│   ├── .env.example                        # GRAFANA_PASSWORD template
│   └── .env                                # local secret (gitignored)
├── prometheus/
│   ├── prometheus.yml                      # scrape config (scrape_interval: 15s)
│   └── alert_rules.yml                     # AppDown, HighCPU, HighMemory, ServerErrors, ProcessingStalled
├── grafana/
│   ├── dashboards/nasa-metrics.json        # NASA metrics dashboard
│   └── provisioning/
│       ├── dashboards/dashboard.yml
│       └── datasources/prometheus.yml
└── data/
    └── NASA_access_log_Jul95               # placed here for local runs (gitignored)

---

Requirements

For Docker-based runs (recommended):

• Docker 20+
• Docker Compose v2

For local runs:

• Python 3.14 (the Dockerfile pins python:3.14-slim; earlier 3.x should also work, but 3.14 is the target)
• The NASA Jul 95 access log at data/NASA_access_log_Jul95 — see Data Source

Python dependencies (requirements.txt):

psutil==7.2.2
pytest==9.0.2
pytest-cov==7.1.0

---

Quick Start (Docker Compose)

1. Configure the Grafana admin password.

   cd docker
   cp .env.example .env
   # edit docker/.env and set GRAFANA_PASSWORD=...

2. Build and start the stack.

   docker compose up --build

   The Dockerfile pulls the NASA log via FTP during the build stage, so no manual download is required for the container path.

3. Open the UIs.

   Service	URL	Notes
   App metrics	http://localhost:8000/metrics	Raw Prometheus exposition output
   App health	http://localhost:8000/health	Returns OK
   Prometheus	http://localhost:9090	Targets → nasa-metrics should be UP
   Grafana	http://localhost:3000	Login: admin / $GRAFANA_PASSWORD

   In Grafana, the NASA Metrics folder will be auto-provisioned with the dashboard from grafana/dashboards/nasa-metrics.json.

4. Shut down.

   docker compose down

---

Local Development (without Docker)

1. Create a virtualenv and install dependencies.

   python -m venv .venv
   source .venv/bin/activate
   pip install -r requirements.txt

2. Place the NASA log.

   Put the uncompressed file at data/NASA_access_log_Jul95 (see Data Source).

3. Run a mode (see next section for details):

   # HTTP server on :8000
   python -m src data/NASA_access_log_Jul95 --mode http
   
   # Or stream to stdout every 15s
   python -m src data/NASA_access_log_Jul95 --mode print

---

Run Modes

The entrypoint is src/__main__.py.

python -m src <logfile> [--mode {http|print|file}]

• --mode http — start the HTTP metrics server on 0.0.0.0:8000. Background thread processes the log; /metrics and /health are served. This is what the Docker container runs.
• --mode print — print the full Prometheus exposition block to stdout every 15 seconds. Useful for quick eyeballing.
• --mode file — write the exposition block to /var/log/nasa_metrics.prom every 15 seconds (textfile-collector style; suitable for node_exporter --collector.textfile).
• (no flag) — defaults to print.

The HTTP path also goes through src/metrics_server.py, which can be invoked directly:

python -m src.metrics_server data/NASA_access_log_Jul95

---

Exposed Metrics

All metrics are emitted in the Prometheus text exposition format from src/metrics_formatter.py.

Metric	Type	Labels	Meaning
http_requests_total	gauge	code="Nxx"	Count of requests per HTTP status bucket (1xx–5xx)
http_requests_by_host	gauge	host="..."	Request count for the current top 10 hosts
node_cpu_usage_percent	gauge	—	Container/host CPU usage (psutil, non-blocking)
node_memory_usage_percent	gauge	—	Memory usage percentage
node_memory_total_bytes	gauge	—	Total physical memory
node_memory_used_bytes	gauge	—	Memory currently in use
node_memory_available_bytes	gauge	—	Memory available
log_entries_processed	gauge	—	Entries processed so far
log_entries_total	gauge	—	Valid entries detected in the first-pass count

Note: http_requests_total is declared as a gauge (not a counter) because it represents the running total accumulated from a finite log file rather than a monotonic counter of live traffic.

---

HTTP Endpoints

Exposed by src/metrics_server.py:

• GET /metrics → 200 text/plain; charset=utf-8 with the full Prometheus exposition payload.
• GET /health → 200 text/plain with body OK. Used by both the Dockerfile HEALTHCHECK and the Compose-level healthcheck.
• Any other path → 404 with a brief hint.

---

Prometheus Configuration

prometheus/prometheus.yml:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

rule_files:
  - /etc/prometheus/alert_rules.yml

scrape_configs:
  - job_name: 'nasa-metrics'
    scrape_interval: 15s
    static_configs:
      - targets: ['app:8000']

The app hostname resolves via Docker Compose's service network.

---

Alerting Rules

Defined in prometheus/alert_rules.yml:

Alert	Expression	For	Severity
AppDown	up{job="nasa-metrics"} == 0	1m	critical
HighCPU	node_cpu_usage_percent > 90	5m	warning
HighMemory	node_memory_usage_percent > 90	5m	warning
ServerErrors	http_requests_total{code="5xx"} > 0	0m	warning
ProcessingStalled	rate(log_entries_processed[2m]) == 0 and log_entries_processed < log_entries_total	2m	warning

These fire into Prometheus only; no Alertmanager is wired up by default — add one if you want notification delivery.

---

Grafana Dashboard

Grafana is auto-provisioned with:

• Datasource: Prometheus at http://prometheus:9090 (default, editable). See grafana/provisioning/datasources/prometheus.yml.
• Dashboard provider: file-based loader pointed at /var/lib/grafana/dashboards. See grafana/provisioning/dashboards/dashboard.yml.
• Dashboard JSON: grafana/dashboards/nasa-metrics.json — visualizes the metrics listed above, grouped under the NASA Metrics folder.

Admin password is taken from ${GRAFANA_PASSWORD} (defined in docker/.env).

---

Testing

Run the full suite:

pytest

With coverage:

pytest --cov=src --cov-report=term-missing

Test modules:

• tests/test_log_parser.py — parsing, bucketing, daily aggregation
• tests/test_metrics_formatter.py — MetricsCollector and exposition formatters
• tests/test_metrics_server.py — HTTP handler and end-to-end server behavior
• tests/test_system_metrics.py — psutil-backed probes

---

Configuration Reference

Knob	Where	Default
Log file path	CLI arg	required
Run mode	--mode CLI flag	print
Server bind host	run_server(host=...)	0.0.0.0
Server port	run_server(port=...)	8000
Tick / scrape interval	tick(duration=...), prometheus.yml	15s
File-mode output path	hardcoded in __main__.py	/var/log/nasa_metrics.prom
App CPU limit (Compose)	docker/docker-compose.yml	0.20 CPU
App memory limit (Compose)	docker/docker-compose.yml	128M
Grafana admin password	docker/.env → GRAFANA_PASSWORD	required

---

Data Source

The application is built around the NASA-HTTP trace from the Internet Traffic Archive:

• Original mirror: ftp://ita.ee.lbl.gov/traces/NASA_access_log_Jul95.gz
• Format: Apache Common Log Format
• Period: July 1–31, 1995
• Uncompressed size: ~205 MB, ~1.9M entries

For local runs, download and decompress into data/NASA_access_log_Jul95:

mkdir -p data
curl -o data/NASA_access_log_Jul95.gz ftp://ita.ee.lbl.gov/traces/NASA_access_log_Jul95.gz
gunzip data/NASA_access_log_Jul95.gz

The Dockerfile performs the same fetch inside its builder stage, so containerized runs work out of the box.