implement basic unit testing and ci cd via github actions

.github/workflows/test.yml

@@ -0,0 +1,66 @@
+name: Run Tests
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+  workflow_dispatch:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ['3.9', '3.10', '3.11']
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v4
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Cache pip packages
+      uses: actions/cache@v4
+      with:
+        path: ~/.cache/pip
+        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements*.txt') }}
+        restore-keys: |
+          ${{ runner.os }}-pip-
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements-test.txt
+
+    - name: Run tests with pytest
+      run: |
+        pytest --cov=lambda/skyflow --cov-report=xml --cov-report=term
+
+    - name: Upload coverage to Codecov
+      uses: codecov/codecov-action@v4
+      if: matrix.python-version == '3.9'
+      with:
+        file: ./coverage.xml
+        flags: unittests
+        name: codecov-umbrella
+        fail_ci_if_error: false
+
+    - name: Generate coverage report
+      if: matrix.python-version == '3.9'
+      run: |
+        pip install coverage
+        coverage report
+
+    - name: Archive test results
+      if: always()
+      uses: actions/upload-artifact@v4
+      with:
+        name: test-results-${{ matrix.python-version }}
+        path: |
+          htmlcov/
+          coverage.xml
+          .coverage

.gitignore

@@ -6,3 +6,23 @@ CLAUDE.md
 .claude/
 .claude/*
 
+# Ignore Python virtual environment
+.venv/
+
+# Ignore Python bytecode and cache
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+
+# Ignore pytest cache
+.pytest_cache/
+
+# Ignore coverage reports
+.coverage
+htmlcov/
+coverage.xml
+
+# VSCode settings (keep settings.json for shared project config)
+.vscode/*
+!.vscode/settings.json

.vscode/settings.json

@@ -0,0 +1,12 @@
+{
+  "python.defaultInterpreterPath": "${workspaceFolder}/.venv/bin/python",
+  "python.terminal.activateEnvironment": true,
+  "python.testing.pytestEnabled": true,
+  "python.testing.unittestEnabled": false,
+  "python.testing.pytestArgs": [
+    "tests"
+  ],
+  "python.analysis.extraPaths": [
+    "${workspaceFolder}/lambda/skyflow"
+  ]
+}

README.md

@@ -5,23 +5,39 @@ A Lambda-based handler for tokenizing and detokenizing sensitive data in Amazon
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
 ## Table of Contents
-- [Introduction](#introduction)
-- [Features](#features)
-- [How It Works](#how-it-works)
-- [Setup](#setup)
-  - [Prerequisites](#prerequisites)
-  - [Skyflow Configuration](#skyflow-configuration)
-  - [Secret Structure](#secret-structure)
-- [Deployment](#deployment)
-  - [AWS Lambda Setup](#aws-lambda-setup)
-  - [Redshift Integration](#redshift-integration)
-- [Usage](#usage)
-  - [In SQL Queries](#in-sql-queries)
-- [Security Considerations](#security-considerations)
-- [Troubleshooting](#troubleshooting)
-  - [Common Issues](#common-issues)
-- [Contributing](#contributing)
-- [License](#license)
+- [Skyflow Redshift UDF Handler](#skyflow-redshift-udf-handler)
+  - [Table of Contents](#table-of-contents)
+  - [Introduction](#introduction)
+  - [Features](#features)
+  - [How It Works](#how-it-works)
+  - [Setup](#setup)
+    - [Prerequisites](#prerequisites)
+    - [Skyflow Configuration](#skyflow-configuration)
+      - [1. Create a Skyflow Vault](#1-create-a-skyflow-vault)
+      - [2. Configure Vault Schema](#2-configure-vault-schema)
+      - [3. Configure Roles and Policies](#3-configure-roles-and-policies)
+      - [4. Create Service Accounts](#4-create-service-accounts)
+      - [5. Store Credentials in AWS Secrets Manager](#5-store-credentials-in-aws-secrets-manager)
+    - [Secret Structure](#secret-structure)
+      - [Role Mappings Secret](#role-mappings-secret)
+      - [Credentials Mapping Secret](#credentials-mapping-secret)
+  - [Deployment](#deployment)
+    - [AWS Lambda Setup](#aws-lambda-setup)
+    - [Redshift Integration](#redshift-integration)
+      - [Redshift Permissions](#redshift-permissions)
+  - [Usage](#usage)
+    - [In SQL Queries](#in-sql-queries)
+  - [Security Considerations](#security-considerations)
+  - [Troubleshooting](#troubleshooting)
+    - [Common Issues](#common-issues)
+  - [Contributing](#contributing)
+  - [Development](#development)
+    - [Testing](#testing)
+      - [Running Tests Locally](#running-tests-locally)
+      - [Test Structure](#test-structure)
+      - [Coverage Requirements](#coverage-requirements)
+      - [Continuous Integration](#continuous-integration)
+  - [License](#license)
 
 ## Introduction
 
@@ -326,6 +342,67 @@ SELECT detokenize('ab123c4d-ef56-7890-gh12-3ij4klm5no6p', 'REDACTED');
 
 Contributions are welcome! Please feel free to submit a Pull Request.
 
+## Development
+
+### Testing
+
+This project uses pytest for unit testing with comprehensive test coverage.
+
+#### Running Tests Locally
+
+1. Install test dependencies:
+   ```bash
+   pip install -r requirements-test.txt
+   ```
+
+2. Run all tests:
+   ```bash
+   pytest
+   ```
+
+3. Run tests with coverage report:
+   ```bash
+   pytest --cov=lambda/skyflow --cov-report=term-missing
+   ```
+
+4. Run specific test classes or functions:
+   ```bash
+   # Run a specific test class
+   pytest tests/test_skyflow_detokenize_function.py::TestCredentialConfig
+
+   # Run a specific test function
+   pytest tests/test_skyflow_detokenize_function.py::TestCredentialConfig::test_get_credentials_secret_with_mapping
+   ```
+
+#### Test Structure
+
+The test suite includes comprehensive coverage for:
+
+- **Configuration Classes**: Tests for `RoleConfig` and `CredentialConfig` with various mapping scenarios
+- **JWT Generation**: Tests for token generation with valid and invalid credentials
+- **Handler Operations**: Tests for tokenization and detokenization with mocked AWS and Skyflow APIs
+- **Caching**: Tests for bearer token caching and expiration
+- **Error Handling**: Tests for various error scenarios (404 errors, missing secrets, invalid arguments)
+- **Lambda Integration**: Tests for `lambda_handler` with different event structures and role normalization
+
+#### Coverage Reports
+
+Coverage reports are generated in multiple formats for visibility:
+- Terminal output with missing lines
+- HTML report (in `htmlcov/` directory)
+- XML report (for CI/CD integration)
+
+Current coverage is approximately 72%. There is no minimum coverage requirement enforced.
+
+#### Continuous Integration
+
+Tests run automatically via GitHub Actions on:
+- Push to main branch
+- Pull requests to main branch
+- Python versions: 3.9, 3.10, 3.11
+
+View the test workflow at [.github/workflows/test.yml](.github/workflows/test.yml)
+
 ## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

pytest.ini

@@ -0,0 +1,31 @@
+[pytest]
+# Pytest configuration for Skyflow Redshift UDF tests
+
+# Test discovery patterns
+python_files = test_*.py
+python_classes = Test*
+python_functions = test_*
+
+# Coverage options
+addopts =
+    --verbose
+    --strict-markers
+    --cov=lambda/skyflow
+    --cov-report=term-missing
+    --cov-report=html
+    --cov-report=xml
+
+# Test paths
+testpaths = tests
+
+# Markers for organizing tests
+markers =
+    unit: Unit tests
+    integration: Integration tests
+    slow: Slow running tests
+
+# Warning filters
+filterwarnings =
+    error
+    ignore::UserWarning
+    ignore::DeprecationWarning

requirements-test.txt

@@ -0,0 +1,10 @@
+# Test dependencies for Skyflow Redshift UDF Lambda function
+-r requirements.txt
+
+# Testing framework
+pytest>=7.4.0
+pytest-cov>=4.1.0
+pytest-mock>=3.11.0
+
+# Mocking libraries
+moto>=4.1.0  # Mock AWS services

requirements.txt

@@ -0,0 +1,5 @@
+# Production dependencies for Skyflow Redshift UDF Lambda function
+boto3>=1.26.0
+requests>=2.28.0
+PyJWT>=2.8.0
+cryptography>=41.0.0

tests/__init__.py

@@ -0,0 +1 @@
+# Tests package for Skyflow Redshift UDF