log-forwarder is a small, low-footprint alternative to Fluent Bit — written in Go, MIT-licensed, and built for teams that want a simple file → structured JSON → sink pipeline without a heavy agent.
Download the source, add your sink, and run. It tails log files, transforms and enriches records, and ships them to Kafka, files, HTTP, or your own integration via a custom binary — no fork required. Same operational ideas you expect (watermarks, rotation, dead letters, metrics), in a ~15 MB static binary.
Small agent. Your sink. Your rules.
Documentation: GitHub Wiki — install, configuration, sinks, watermarks, monitoring, deployment, and custom extensions.
| log-forwarder | Typical heavy agents (Fluent Bit, Fluentd, Vector) | |
|---|---|---|
| Footprint | Single static Go binary (~15 MB) | Larger images, more moving parts |
| Scope | File → JSON → one sink per process | Broad input/output plugins, pipelines |
| Ops model | Watermarks, dead letter, metrics built in | Varies; often more config surface |
| Extension | Custom binary (examples/custom/) |
Plugin SDK / Lua / WASM (product-dependent) |
Use log-forwarder when you want a small, MIT-licensed tail-and-ship agent without operating a full observability stack. See ROADMAP.md for direction and SUPPORT.md for help channels.
git clone https://github.com/sanjuthomas/log-forwarder.git
cd log-forwarder
go build -o bin/log-forwarder ./cmd/log-forwarder
./bin/log-forwarder -config configs/example.yamlSee Installation and First Run on the wiki for requirements, defaults, and verification steps.
| Topic | Wiki page |
|---|---|
| Install, build, run | Installation and First Run |
| Pipeline overview | How It Works |
| YAML config (overview) | Configuration Guide |
| YAML config (full reference) | Configuration-Reference |
| Config key catalog (what / when) | Config-Catalog |
| Kafka, file, HTTP sinks | Choosing a Sink |
| Spring Boot / multiline logs | Spring Boot Logs |
| Filter vendor DEBUG noise (ingest cost) | Filtering Vendor Noise |
| Fleet monitoring (log-forwarder-atc) | Log Forwarder ATC |
| Watermarks, rotation, restarts | Watermarks and Restarts |
| Built-in components | Built-in-Components |
| Custom parsers, sinks, filters | Custom-Extensions |
| Metrics, health, alerts | Monitoring |
| Tests and smoke scripts | Testing |
| Docker images | Docker |
| Production deployment | Deployment |
| Repo layout | Development |
| Example configs | Example Configs |
| Troubleshooting | Troubleshooting |
| Roadmap & support | ROADMAP.md, SUPPORT.md, CHANGELOG.md |
| Path | Contents |
|---|---|
configs/ |
Example YAML configs |
examples/config-catalog.yaml |
Master list of example configs and Prometheus metrics_catalog |
examples/kafka/ |
Kafka security example configs |
examples/custom/ |
Custom binary with registered extensions |
docs/integration-test-cases.txt |
E2E test catalog |
docs/production-battle-test.txt |
Staging / battle-test checklist |
docs/docker.md |
Container and sidecar notes |
wiki/ |
Wiki source (auto-synced to GitHub Wiki on merge to main; run ./scripts/sync-wiki.sh locally to preview) |
scripts/ |
Lint, coverage gate, copyright check, Docker and Kafka smoke tests |
CHANGELOG.md |
Notable changes by version |
ROADMAP.md |
Planned direction and out-of-scope items |
SUPPORT.md |
Where to ask questions vs file bugs |
AGENTS.md |
Conventions for AI coding agents and advanced contributors |
docker pull sanjuthomas/log-forwarder:latest
./scripts/docker-smoke.sh
./scripts/kafka-smoke.shSee Docker on the wiki for compose stacks and CI smoke tests.
Contributions are welcome — bug reports, documentation, tests, and code. See CONTRIBUTING.md for:
- How to open an issue (bug, feature, docs)
- How to submit a pull request (branch workflow, tests, CI checks including ≥80% coverage per
internal/*package) - Coding guidelines and wiki sync notes
This project follows the Contributor Covenant. Report security issues privately via SECURITY.md, not public issues.
This project is licensed under the MIT License.