gcloud-logs-anomaly-detection

Machine Learning tool to find anomalies in Google Cloud Logging events

Features

• Anomaly Detection: Uses Isolation Forest ML algorithm to detect anomalies in log data
• LLM Summarization: Uses Google's Gemini LLM to summarize log entries
• Test Data Generation: Generate sample log events for testing

Installation

pip install -r requirements.txt

For development:

pip install -e ".[dev]"

Configuration

Copy .env.example to .env and configure:

cp .env.example .env
# Edit .env with your settings

Environment Variables

Variable	Description	Default
GCP_PROJECT	GCP Project ID (required)	-
LOG_NAME	Log name to monitor	loremipsumevents
MODEL_NAME	LLM model for summarization	gemini-2.0-flash-lite
NUMEVENTS	Number of test events to generate	1000
MODELNAME	Ollama model for llmtest	smollm2:135m

Authentication

This tool uses Google Cloud Application Default Credentials (ADC). Set up authentication:

gcloud auth application-default login

Or set the GOOGLE_APPLICATION_CREDENTIALS environment variable to point to your service account key file.

Usage

Anomaly Detection

Detect anomalies in your logs:

python gcloud_logs_detect.py

Log Summarization

Summarize logs using LLM:

python gcloud_logs_llmsummary.py

Generate Test Events

Generate sample log events:

python gcloud_event_create.py

Test Local LLM

Test a local Ollama LLM:

python llmtest.py "What is Python?"

Docker

Build and run with Docker:

docker build -t gcloud-logs-anomaly-detection .
docker run -it --rm \
  -v ~/.config/gcloud:/root/.config/gcloud \
  -e GCP_PROJECT=your-project \
  gcloud-logs-anomaly-detection

Development

Running Tests

pytest

Linting

ruff check .
mypy .

License

MIT