[PostgREST][PortalDB] Add PostgREST data source for fetching Portal Applications data

README.md

@@ -20,6 +20,9 @@
 - [Portal App Store Refresh](#portal-app-store-refresh)
   - [How does Portal App Store Refresh Work?](#how-does-portal-app-store-refresh-work)
   - [Configuration](#configuration)
+- [PostgREST Data Source](#postgrest-data-source)
+  - [PostgREST Configuration](#postgrest-configuration)
+  - [JWT Authentication](#jwt-authentication)
 - [Envoy Gateway Integration](#envoy-gateway-integration)
 - [Prometheus Metrics](#prometheus-metrics)
   - [Key Metrics](#key-metrics)
@@ -120,8 +123,8 @@ PEAS adds the following headers to authorized requests before forwarding them to
 
 | Header                  | Contents                                       | Included For All Requests | Example Value |
 | ----------------------- | ---------------------------------------------- | ------------------------- | ------------- |
-| `Portal-Application-ID` | The portal app ID of the authorized portal app | ✅                        | "a12b3c4d"    |
-| `Portal-Account-ID`     | The account ID associated with the portal app  | ✅                        | "3f4g2js2"    |
+| `Portal-Application-ID` | The portal app ID of the authorized portal app | ✅                         | "a12b3c4d"    |
+| `Portal-Account-ID`     | The account ID associated with the portal app  | ✅                         | "3f4g2js2"    |
 
 ## Rate Limiting Implementation
 
@@ -164,6 +167,44 @@ The refresh interval is configurable via the `REFRESH_INTERVAL` environment vari
 - **Format**: Duration string (e.g., `30s`, `1m`, `2m30s`)
 - **Purpose**: Balance between data freshness and database load
 
+## PostgREST Data Source
+
+PEAS supports [PostgREST](https://docs.postgrest.org/en/v13/) as an alternative data source to direct PostgreSQL connections. PostgREST provides a RESTful API layer over PostgreSQL databases, enabling PEAS to fetch portal application and account data via HTTP rather than direct database connections.
+
+For more information about PostgREST configuration and usage, see the [official PostgREST documentation](https://docs.postgrest.org/en/v13/).
+
+<div align="center">
+  <a href="https://docs.postgrest.org/en/v13/">
+    <img src="https://github.com/PostgREST/postgrest/blob/main/static/logo.png?raw=true" alt="PostgREST logo" width="200"/>
+  </a>
+</div>
+
+### PostgREST Configuration
+
+To use PostgREST as the data source, configure the following environment variables:
+
+```bash
+# Set data source type to PostgREST
+DATA_SOURCE_TYPE="postgrest"
+
+# PostgREST API endpoint
+POSTGREST_BASE_URL="https://db.rpc.com/api"
+
+# JWT authentication (required for PostgREST access)
+POSTGREST_JWT_SECRET="supersecretjwtsecretforlocaldevelopment123456789"
+POSTGREST_JWT_EMAIL="service@rpc.com"
+POSTGREST_JWT_ROLE="admin"
+
+# Request timeout (optional)
+POSTGREST_TIMEOUT="30s"  # Optional, defaults to 30s
+```
+
+### JWT Authentication
+
+PEAS generates fresh JWT tokens for each PostgREST API request using the configured secret, email, and role. 
+
+This ensures secure access to the PostgREST API with proper authentication and authorization based on your [PostgREST JWT configuration](https://docs.postgrest.org/en/v13/references/auth.html).
+
 ## Envoy Gateway Integration
 
 PEAS exposes a gRPC service that adheres to the spec expected by Envoy Proxy's `ext_authz` HTTP Filter.
@@ -285,17 +326,27 @@ This tool uses gRPC reflection to communicate with PEAS, testing the same author
 
 PEAS is configured via environment variables.
 
-| Variable                          | Required | Type     | Description                                                  | Example                                              | Default Value |
-| --------------------------------- | -------- | -------- | ------------------------------------------------------------ | ---------------------------------------------------- | ------------- |
-| POSTGRES_CONNECTION_STRING        | ✅       | string   | PostgreSQL connection string for the PortalApp database      | postgresql://username:password@localhost:5432/dbname | -             |
-| GCP_PROJECT_ID                    | ✅       | string   | GCP project ID for the data warehouse used by rate limiting  | your-project-id                                      | -             |
-| PORT                              | ❌       | int      | Port to run the external auth server on                      | 10001                                                | 10001         |
-| METRICS_PORT                      | ❌       | int      | Port to run the Prometheus metrics server on                 | 9090                                                 | 9090          |
-| PPROF_PORT                        | ❌       | int      | Port to run the pprof server on                              | 6060                                                 | 6060          |
-| LOGGER_LEVEL                      | ❌       | string   | Log level for the external auth server                       | info, debug, warn, error                             | info          |
-| IMAGE_TAG                         | ❌       | string   | Image tag/version for the application                        | v1.0.0                                               | development   |
-| PORTAL_APP_STORE_REFRESH_INTERVAL | ❌       | duration | Refresh interval for portal app data from the database       | 30s, 1m, 2m30s                                       | 30s           |
-| RATE_LIMIT_STORE_REFRESH_INTERVAL | ❌       | duration | Refresh interval for rate limit data from the data warehouse | 30s, 1m, 2m30s                                       | 5m            |
+| Variable                          | Required | Type     | Description                                                                        | Example                                              | Default Value |
+| --------------------------------- | -------- | -------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------- | ------------- |
+| **Data Source Configuration**     |          |          |                                                                                    |                                                      |               |
+| DATA_SOURCE_TYPE                  | ❌        | string   | Data source type: "postgres" or "postgrest"                                        | postgres, postgrest                                  | postgres      |
+| POSTGRES_CONNECTION_STRING        | ✅*       | string   | PostgreSQL connection string (required when DATA_SOURCE_TYPE=postgres)             | postgresql://username:password@localhost:5432/dbname | -             |
+| POSTGREST_BASE_URL                | ✅*       | string   | PostgREST API base URL (required when DATA_SOURCE_TYPE=postgrest)                  | https://db.rpc.com/api                               | -             |
+| POSTGREST_JWT_SECRET              | ✅*       | string   | JWT secret for PostgREST authentication (required when DATA_SOURCE_TYPE=postgrest) | supersecretjwtsecretforlocaldevelopment123456789     | -             |
+| POSTGREST_JWT_EMAIL               | ✅*       | string   | JWT email for PostgREST authentication (required when DATA_SOURCE_TYPE=postgrest)  | service@rpc.com                                      | -             |
+| POSTGREST_JWT_ROLE                | ❌        | string   | JWT role for PostgREST authentication                                              | admin                                                | -             |
+| POSTGREST_TIMEOUT                 | ❌        | duration | PostgREST request timeout                                                          | 30s, 1m, 2m30s                                       | 30s           |
+| **System Configuration**          |          |          |                                                                                    |                                                      |               |
+| GCP_PROJECT_ID                    | ✅        | string   | GCP project ID for the data warehouse used by rate limiting                        | your-project-id                                      | -             |
+| PORT                              | ❌        | int      | Port to run the external auth server on                                            | 10001                                                | 10001         |
+| METRICS_PORT                      | ❌        | int      | Port to run the Prometheus metrics server on                                       | 9090                                                 | 9090          |
+| PPROF_PORT                        | ❌        | int      | Port to run the pprof server on                                                    | 6060                                                 | 6060          |
+| LOGGER_LEVEL                      | ❌        | string   | Log level for the external auth server                                             | info, debug, warn, error                             | info          |
+| IMAGE_TAG                         | ❌        | string   | Image tag/version for the application                                              | v1.0.0                                               | development   |
+| PORTAL_APP_STORE_REFRESH_INTERVAL | ❌        | duration | Refresh interval for portal app data from the database                             | 30s, 1m, 2m30s                                       | 30s           |
+| RATE_LIMIT_STORE_REFRESH_INTERVAL | ❌        | duration | Refresh interval for rate limit data from the data warehouse                       | 30s, 1m, 2m30s                                       | 5m            |
+
+**\* Required when the corresponding DATA_SOURCE_TYPE is selected**
 
 ## Developing Metrics Dashboard Locally
 
@@ -329,9 +380,9 @@ This section describes how to run and test the PEAS metrics dashboard locally us
    - **PEAS Health**: `http://localhost:9090/healthz`
    - **PEAS pprof**: `http://localhost:6060/debug/pprof/`
    - **Prometheus**: `http://localhost:9091`
-   - **Grafana**: `http://localhost:3000` (admin/admin)
+   - **Grafana**: `https://db.rpc.com/api` (admin/admin)
 4. **View the dashboard**:
-   - Go to Grafana at `http://localhost:3000`
+   - Go to Grafana at `https://db.rpc.com/api`
    - Login with admin/admin
    - The PEAS dashboard should be automatically loaded
 

env.example

@@ -2,13 +2,37 @@
 # REQUIRED ENVIRONMENT VARIABLES
 # ================================
 
-# [REQUIRED]: PostgreSQL connection string for the PortalApp database used by the auth server.
+# [OPTIONAL]: Data source type - "postgres" or "postgrest"
+#   - Default: "postgres" if not set
+# DATA_SOURCE_TYPE="postgres"
+
+# [REQUIRED when DATA_SOURCE_TYPE=postgres]: PostgreSQL connection string for the PortalApp database used by the auth server.
 #   - Example: "postgresql://username:password@localhost:5432/dbname"
 POSTGRES_CONNECTION_STRING=
 
+# [REQUIRED when DATA_SOURCE_TYPE=postgrest]: PostgREST base URL
+#   - Example: "https://db.rpc.com/api"
+# POSTGREST_BASE_URL="https://db.rpc.com/api"
+
+# [REQUIRED when DATA_SOURCE_TYPE=postgrest]: PostgREST JWT secret for authentication
+#   - Example: "supersecretjwtsecretforlocaldevelopment123456789"
+# POSTGREST_JWT_SECRET="supersecretjwtsecretforlocaldevelopment123456789"
+
+# [REQUIRED when DATA_SOURCE_TYPE=postgrest]: PostgREST JWT role
+# POSTGREST_JWT_ROLE="admin"
+
+# [REQUIRED when DATA_SOURCE_TYPE=postgrest]: PostgREST JWT email for authentication
+#   - Example: "service@rpc.com"
+# POSTGREST_JWT_EMAIL="service@rpc.com"
+
+# [OPTIONAL]: PostgREST request timeout
+#   - Default: 30s if not set
+#   - Examples: "30s", "1m", "2m30s"
+# POSTGREST_TIMEOUT="30s"
+
 # [REQUIRED]: GCP project ID for the data warehouse used by the rate limit store.
 #   - Example: "your-project-id"
-GCP_PROJECT_ID=
+GCP_PROJECT_ID="your-project-id"
 
 # ================================
 # OPTIONAL ENVIRONMENT VARIABLES

env.go

@@ -13,14 +13,41 @@ import (
 )
 
 const (
+	// [OPTIONAL]: Data source type - "postgres" or "postgrest"
+	//   - Default: "postgres" if not set
+	dataSourceTypeEnv     = "DATA_SOURCE_TYPE"
+	defaultDataSourceType = "postgres"
+
 	// [REQUIRED]: PostgreSQL connection string for the PortalApp database used by the auth server.
 	//   - Example: "postgresql://username:password@localhost:5432/dbname"
 	postgresConnectionStringEnv = "POSTGRES_CONNECTION_STRING"
 
+	// [REQUIRED when DATA_SOURCE_TYPE=postgrest]: PostgREST base URL
+	//   - Example: "https://db.rpc.com/api"
+	postgrestBaseURLEnv = "POSTGREST_BASE_URL"
+
+	// [REQUIRED when DATA_SOURCE_TYPE=postgrest]: JWT secret for PostgREST authentication
+	//   - Example: "supersecretjwtsecretforlocaldevelopment123456789"
+	postgrestJWTSecretEnv = "POSTGREST_JWT_SECRET"
+
+	// [OPTIONAL]: JWT role for PostgREST authentication
+	//   - Examples: "admin"
+	postgrestJWTRoleEnv = "POSTGREST_JWT_ROLE"
+
+	// [REQUIRED when DATA_SOURCE_TYPE=postgrest]: JWT email for PostgREST authentication
+	//   - Example: "service@rpc.com"
+	postgrestJWTEmailEnv = "POSTGREST_JWT_EMAIL"
+
 	// [REQUIRED]: GCP project ID for the data warehouse used by the rate limit store.
 	//   - Example: "your-project-id"
 	gcpProjectIDEnv = "GCP_PROJECT_ID"
 
+	// [OPTIONAL]: PostgREST request timeout
+	//   - Default: 30s if not set
+	//   - Examples: "30s", "1m", "2m30s"
+	postgrestTimeoutEnv     = "POSTGREST_TIMEOUT"
+	defaultPostgrestTimeout = 30 * time.Second
+
 	// [OPTIONAL]: Port to run the external auth server on.
 	//   - Default: 10001 if not set
 	portEnv     = "PORT"
@@ -61,14 +88,31 @@ const (
 
 var postgresConnectionStringRegex = regexp.MustCompile(`^postgres(?:ql)?:\/\/[^:]+:[^@]+@[^:]+:\d+\/[^?]+(?:\?.+)?$`)
 
+// DataSourceType represents the type of data source to use
+type DataSourceType string
+
+const (
+	DataSourceTypePostgres  DataSourceType = "postgres"
+	DataSourceTypePostgREST DataSourceType = "postgrest"
+)
+
 // envVars holds configuration values.
 //   - All fields are private.
 //   - Use gatherEnvVars to load, validate, and hydrate defaults from environment variables.
 type envVars struct {
-	// Database and external service configuration
+	// Database configuration
+	dataSourceType DataSourceType
+
 	postgresConnectionString string
-	gcpProjectID             string
 
+	postgrestBaseURL   string
+	postgrestJWTSecret string
+	postgrestJWTRole   string
+	postgrestJWTEmail  string
+	postgrestTimeout   time.Duration
+
+	// Data warehouse configuration
+	gcpProjectID string
 	// Server port configuration
 	port        int
 	metricsPort int
@@ -87,10 +131,18 @@ type envVars struct {
 //   - Loads configuration from environment variables
 //   - Validates and hydrates defaults for missing/invalid values
 func gatherEnvVars() (envVars, error) {
-	// Initialize with Postgres connection string from environment
+	// Initialize with environment variables
 	e := envVars{
+		dataSourceType: DataSourceType(os.Getenv(dataSourceTypeEnv)),
+
 		postgresConnectionString: os.Getenv(postgresConnectionStringEnv),
-		gcpProjectID:             os.Getenv(gcpProjectIDEnv),
+
+		postgrestBaseURL:   os.Getenv(postgrestBaseURLEnv),
+		postgrestJWTSecret: os.Getenv(postgrestJWTSecretEnv),
+		postgrestJWTRole:   os.Getenv(postgrestJWTRoleEnv),
+		postgrestJWTEmail:  os.Getenv(postgrestJWTEmailEnv),
+
+		gcpProjectID: os.Getenv(gcpProjectIDEnv),
 	}
 
 	// Parse port environment variable (if provided)
@@ -155,6 +207,16 @@ func gatherEnvVars() (envVars, error) {
 		e.rateLimitStoreRefreshInterval = duration
 	}
 
+	// Parse PostgREST timeout from environment (if provided)
+	postgrestTimeoutStr := os.Getenv(postgrestTimeoutEnv)
+	if postgrestTimeoutStr != "" {
+		duration, err := time.ParseDuration(postgrestTimeoutStr)
+		if err != nil {
+			return envVars{}, fmt.Errorf("invalid PostgREST timeout format: %v", err)
+		}
+		e.postgrestTimeout = duration
+	}
+
 	// Apply defaults for any unset configuration
 	e.hydrateDefaults()
 
@@ -167,30 +229,50 @@ func gatherEnvVars() (envVars, error) {
 
 // validate checks that all required environment variables are set and valid
 func (e *envVars) validate() error {
-	// Postgres connection string must be set
-	if e.postgresConnectionString == "" {
-		return fmt.Errorf("%s is not set", postgresConnectionStringEnv)
-	}
-
 	// GCP project ID must be set
 	if e.gcpProjectID == "" {
 		return fmt.Errorf("%s is not set", gcpProjectIDEnv)
 	}
 
-	// Connection string must match expected format
-	matched, err := regexp.MatchString(postgresConnectionStringRegex.String(), e.postgresConnectionString)
-	if err != nil {
-		return fmt.Errorf("failed to validate postgresConnectionString: %v", err)
-	}
-	if !matched {
-		return fmt.Errorf("postgresConnectionString does not match the required pattern")
+	// Validate based on data source type
+	switch e.dataSourceType {
+	case DataSourceTypePostgres:
+		if e.postgresConnectionString == "" {
+			return fmt.Errorf("%s is required when DATA_SOURCE_TYPE=postgres", postgresConnectionStringEnv)
+		}
+		// Connection string must match expected format
+		matched, err := regexp.MatchString(postgresConnectionStringRegex.String(), e.postgresConnectionString)
+		if err != nil {
+			return fmt.Errorf("failed to validate postgresConnectionString: %v", err)
+		}
+		if !matched {
+			return fmt.Errorf("postgresConnectionString does not match the required pattern")
+		}
+	case DataSourceTypePostgREST:
+		if e.postgrestBaseURL == "" {
+			return fmt.Errorf("%s is required when DATA_SOURCE_TYPE=postgrest", postgrestBaseURLEnv)
+		}
+		if e.postgrestJWTSecret == "" {
+			return fmt.Errorf("%s is required when DATA_SOURCE_TYPE=postgrest", postgrestJWTSecretEnv)
+		}
+		if e.postgrestJWTEmail == "" {
+			return fmt.Errorf("%s is required when DATA_SOURCE_TYPE=postgrest", postgrestJWTEmailEnv)
+		}
+	default:
+		return fmt.Errorf("unsupported data source type: %s", e.dataSourceType)
 	}
 
 	return nil
 }
 
 // hydrateDefaults sets defaults for missing/invalid values
 func (e *envVars) hydrateDefaults() {
+	if e.dataSourceType == "" {
+		e.dataSourceType = DataSourceType(defaultDataSourceType)
+	}
+	if e.postgrestTimeout == 0 {
+		e.postgrestTimeout = defaultPostgrestTimeout
+	}
 	if e.port == 0 {
 		e.port = defaultPort
 	}