OpenTelemetry Python Flask Example

This project is a hands-on demonstration of instrumenting a Python Flask application with OpenTelemetry for distributed tracing. It includes a complete, containerized observability pipeline using an OpenTelemetry Collector for processing and Jaeger for visualization.

This repository is designed to be a practical, production-grade starting point for anyone looking to get started with OpenTelemetry.

Features

• Python Flask Application: A simple web server with multiple endpoints to demonstrate different tracing scenarios (/, /slow, /error).
• Manual OpenTelemetry SDK Configuration: The app.py is configured manually, showing the core components of the OTel SDK for maximum control and clarity.
• Production-Grade Collector Configuration: The config.yaml is configured with best practices, including:
  • Tail-Based Sampling: Keeps 100% of important traces (errors, slow requests) while sampling normal traffic.
  • Resource Protection: A memory_limiter to prevent the collector from crashing under heavy load.
  • Batch Processing: For efficient data export.
• Dockerized Backend: The entire observability backend (Collector, Jaeger) is containerized for easy and consistent setup.
• Custom Collector Image: A Dockerfile is used to build a custom collector image with our configuration baked in, which is a robust pattern for deployment.

Architecture

The data flows from the instrumented application to the collector, which then processes and exports the data to Jaeger for visualization.

+------------------------+      +------------------------+      +--------------------+
|                        |      |                        |      |                    |
|  Flask App (OTel SDK)  |----->| OTel Collector (Docker)|----->|   Jaeger (Docker)  |
| (sends OTLP gRPC data) |      | (processes & samples)  |      |  (stores & visualizes) |
|                        |      |                        |      |                    |
+------------------------+      +------------------------+      +--------------------+

Getting Started

Follow these instructions to get the project running on your local machine.

Prerequisites

• Python 3.9+
• Git
• Docker Desktop (must be running)

Installation & Setup

1. Clone the repository (if you haven't already):

   git clone <your-repository-url>
   cd OTel

2. Create and activate a Python virtual environment:

   # Create the environment
   python -m venv venv
   
   # Activate it (on Windows Git Bash)
   source venv/Scripts/activate
   
   # Or on Windows Command Prompt
   .\venv\Scripts\activate.bat

3. Install the Python dependencies:

   pip install Flask opentelemetry-sdk opentelemetry-instrumentation-flask opentelemetry-exporter-otlp-proto-grpc

4. Build the custom OpenTelemetry Collector Docker image: This command packages your config.yaml into a new image named my-otel-collector.

   docker build -t my-otel-collector .

Usage

To run the full application and observability pipeline, follow these steps in order.

1. Create a dedicated Docker network: This allows the containers to communicate with each other easily.

   docker network create otel-net

2. Start the Jaeger Container: Run this in one terminal. This will be your tracing backend.

   docker run -d --name jaeger --network otel-net -p 16686:16686 -p 4317:4317 jaegertracing/all-in-one:latest

3. Start the OTel Collector Container: In the same terminal, run your custom collector. It will connect to Jaeger.

   docker run --rm -p 4317:4317 --network otel-net --name my-collector my-otel-collector

   Leave this terminal running to see the collector's logs.

4. Run the Flask Application: In a separate terminal, run the Python application.

   python app.py

Verify the Pipeline

You can now test the full pipeline:

1. Generate traffic by visiting these URLs in your browser:

   • http://localhost:5000/ (a normal, fast request)
   • http://localhost:5000/slow (a request that takes >1 second)
   • http://localhost:5000/error (a request that generates an error)

2. View the traces in the Jaeger UI:

   • Open http://localhost:16686 in your browser.
   • Select my-flask-app from the "Service" dropdown.
   • Click "Find Traces" to see the results of your sampling policies.

Configuration Files

• app.py: The main Flask application. Contains the manual OpenTelemetry SDK configuration, including the gRPC exporter and resource attributes (service.name, etc.).
• collector_config/config.yaml: The brain of the OTel Collector. Defines the entire data processing pipeline, including receivers, processors (sampling, batching), and exporters.
• Dockerfile: Instructions for building a custom OTel Collector image that includes our config.yaml.
• .gitignore: Specifies intentionally untracked files to ignore (e.g., __pycache__, venv).