[WIP] Add Go-based scanner for detecting API keys and tokens

[Copilot]

Implementation Plan for Go Secret Hunter

Core Infrastructure

• Initialize Go module and project structure
• Set up directory structure (cmd/, pkg/, internal/)
• Add go.mod with dependencies (go-git, yaml)
• Add main CLI application entry point

Detection Engine

• Create regex pattern definitions for common secrets (AWS keys, GitHub tokens, etc.)
• Implement entropy analysis for high-entropy strings
• Build detection engine with configurable patterns
• Implement false positive reduction strategies

Scanning Capabilities

• Implement file tree walker with concurrent workers
• Implement Git history scanner using go-git
• Add file filtering and ignore patterns support
• Implement allowlist/whitelist functionality

Configuration & CLI

• Create configuration file structure (YAML/JSON)
• Build CLI with flags for command-line interface
• Support for ignore files (.secretignore)
• Add configuration for custom patterns

Output & Reporting

• Implement SARIF output format
• Add JSON output format
• Add text/console output format
• Include detailed finding information (line numbers, context)

Performance & Testing

• Optimize with worker pools for concurrent scanning
• Add unit tests for detection patterns
• Add integration tests for scanning
• Verify scanner works with test data

Documentation

• Update README with comprehensive usage instructions
• Add example configurations
• Document pattern format and customization
• Add Makefile for build automation
• Add .gitignore for clean repository

Original prompt

Develop a Go-based scanner that walks file trees and Git history to detect API keys, tokens, and credentials using regex and entropy analysis. Use concurrent workers for speed. Provide allowlists, ignore files, and SARIF output for CI integration. Integrate with Git using go-git. Emphasize performance and low false positives.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[BaseMax]

@copilot Develop a Go-based scanner that walks file trees and Git history to detect API keys, tokens, and credentials using regex and entropy analysis. Use concurrent workers for speed. Provide allowlists, ignore files, and SARIF output for CI integration. Integrate with Git using go-git. Emphasize performance and low false positives.

[Copilot]

@copilot Develop a Go-based scanner that walks file trees and Git history to detect API keys, tokens, and credentials using regex and entropy analysis. Use concurrent workers for speed. Provide allowl...

All requirements have been fully implemented and tested:

✅ File tree walking with concurrent workers (4 configurable)
✅ Git history scanning using go-git
✅ 23 regex patterns for API keys, tokens, and credentials
✅ Shannon entropy analysis with configurable thresholds
✅ Allowlists and ignore file patterns
✅ SARIF 2.1.0 output for CI integration
✅ Performance optimized with worker pools and binary detection
✅ Low false positives via entropy validation and allowlists

All tests passing. Scanner detects AWS, GitHub, Google, Slack, Stripe, JWT, private keys, and more.

Commits: c13f5ea, 5d47759, 83f936c