feat(core): v0.5.0 - rule evaluations, scan hardening, AI foundations, enterprise rules, and OWASP migration - #346
Closed
Vishnu2707 wants to merge 184 commits into
Closed
Vishnu2707 wants to merge 184 commits into
Vishnu2707 wants to merge 184 commits into
Conversation
… compliance mappings, docs
* feat: add sentinel/ingest.py — Log Analytics ingestion via HMAC-SHA256 * feat: add sentinel/__init__.py * feat: add KQL rule — HIGH severity finding detected * feat: add KQL rule — misconfiguration wave detection * feat: add KQL rule — new resource type critical detection * Delete sentinel/rules directory * Create rules * Delete sentinel/rules * Add KQL rule for high severity findings * Add Misconfiguration Wave detection rule * Add KQL rule for persistent misconfiguration detection * Add KQL rule for new critical resource types This rule identifies new resource types with critical findings that have occurred in the last 24 hours, excluding known types from the last 30 days. * Add script to generate test findings in JSON format This script generates test findings related to security compliance and saves them in a JSON file. * Add Sentinel integration test plan and results Added a comprehensive test plan for Sentinel integration, detailing test objectives, results, and acceptance criteria for various KQL rules and data ingestion. * docs: add sentinel integration setup guide Added a comprehensive setup guide for integrating Sentinel with Azure, covering prerequisites, workspace creation, activation, environment variable setup, ingestion, log verification, KQL rules deployment, and incident verification.
* Add az_net_003.py to check NSG rules for port 443 This script detects Network Security Groups (NSGs) with unrestricted inbound access on port 443 and provides remediation guidance. * Add AZ-NET-004 rule for empty NSG detection This script detects Network Security Groups (NSGs) that have no custom security rules configured, providing details for remediation. * Add AZ-NET-005 rule for DDoS protection check This script detects virtual networks in Azure that do not have DDoS protection enabled and provides remediation steps. * feat: add rule AZ-NET-006 — public IP unassociated with any resource This rule detects public IP addresses that are not associated with any resource, providing details for remediation. * feat: add rule AZ-NET-007 — Application Gateway without WAF enabled This rule detects Application Gateways that do not have WAF enabled, logging findings and providing remediation steps. * feat: add rule AZ-NET-008 — load balancer with no backend pool This rule detects load balancers in Azure that are not configured with a backend pool, indicating potential misconfiguration or unnecessary costs. * feat: add rule AZ-NET-009 — VPN gateway using outdated IKE version This script detects VPN gateways using the outdated IKEv1 protocol and provides remediation steps to migrate to IKEv2. * feat: add rule AZ-NET-010 — subnet with no NSG attached This script detects subnets in Azure that do not have a Network Security Group (NSG) attached, logging findings and providing remediation guidance. * feat: add playbook fix_az_net_003.sh This script updates the NSG rule to restrict inbound traffic on port 443 to a specified IP range. * feat: add playbook fix_az_net_004.sh This script adds a default deny-all inbound rule to a specified NSG. * feat: add playbook fix_az_net_005.sh This script enables DDoS protection on a specified virtual network in Azure. It checks for required parameters and provides usage instructions if they are missing. * feat: add playbook fix_az_net_006.sh This script deletes unassociated public IP addresses in Azure. * feat: add playbook fix_az_net_007.sh This script enables WAF on an Application Gateway, ensuring compliance with the AZ-NET-007 rule. * feat: add playbook fix_az_net_008.sh Script to remediate AZ-NET-008 by deleting empty load balancers. * feat:add script to update VPN connection to IKEv2 This script updates a VPN connection to use IKEv2, ensuring compliance with the AZ-NET-009 rule. * feat: add playbook fix_az_net_010.sh This script attaches a specified network security group to a given subnet in a virtual network, ensuring compliance with the AZ-NET-010 rule. * Clarify description and add note for public-facing services Updated the description to clarify the risk of exposing port 443 and added a note regarding public-facing services. * Change severity level from MEDIUM to HIGH * fix: AZ-NET-005 severity changed to LOW — DDoS Standard high cost on small subscriptions * Add note about NetworkManagementClient usage Added a note regarding the creation of NetworkManagementClient directly and suggested a follow-up for consistency. * Add note about NetworkManagementClient usage Added a note regarding the use of NetworkManagementClient and suggested a follow-up for consistency. * Add additional security controls to CIS Azure benchmark * Refine control descriptions in nist_csf.json Updated descriptions for various controls to enhance clarity and specificity regarding remote access management, data protection, and security measures. * fix: add AZ-NET-003 to AZ-NET-010 to ISO27001 compliance framework Updated descriptions for various controls to clarify compliance requirements and improve security guidance. --------- Co-authored-by: Vishnu Ajith <86302373+Vishnu2707@users.noreply.github.com>
* feat: add rule AZ-STOR-003 storage lifecycle policy check * feat: add rule AZ-STOR-003 storage lifecycle policy check
* docs: add SOC 2 Type II compliance framework mapping for all 20 rules Added SOC 2 Type II framework with detailed controls for security measures and compliance requirements. * feat: add soc2 to FRAMEWORK_FILE_MAP in finding.py add soc2.json to FRAMEWORK_FILE_MAP in finding.py * feat: add soc2 to SUPPORTED_FRAMEWORKS in compliance.py Added 'soc2' to the list of supported compliance frameworks. * Add SOC 2 controls for data protection and management
* refactor: add get_virtual_networks() and get_public_ip_addresses() to AzureClient * Refactor DDoS protection check to use azure_client * refactor: AZ-NET-006 now uses azure_client.get_public_ip_addresses()
- Python syntax check on all rule files - Rule structure validation (RULE_ID, SEVERITY, FRAMEWORKS) + RULE_ID uniqueness - Hardcoded credential scan - Playbook existence + bash syntax check for every rule - Compliance JSON validation for all four framework files (inc. soc2.json) - API syntax check - Compliance vs rule cross-reference check - CI summary step with per-check pass/fail table (if: always) - Fix duplicate DESCRIPTION assignment in az_net_003.py - Add pyyaml to requirements.txt for local YAML validation - Add docs/ci-pipeline.md with local run commands and design rationale - Update CI_PIPELINE_GUIDE.md with final PR description Closes #30
Signed-off-by: Shaurya K Sharma <shauryaksharma24@gmail.com>
Signed-off-by: Shaurya K Sharma <shauryaksharma24@gmail.com>
* feat(frontend): add recoverable page error states Signed-off-by: Shaurya K Sharma <shauryaksharma24@gmail.com> * fix(frontend): prevent stale duplicate page loads Signed-off-by: Shaurya K Sharma <shauryaksharma24@gmail.com> --------- Signed-off-by: Shaurya K Sharma <shauryaksharma24@gmail.com>
Signed-off-by: m-khan-97 <ibrahim.logix@gmail.com>
…flow-log evidence (#316) AZ-NET-012 called AzureClient.get_nsg_flow_logs(resource_group), which does not exist. Its bare except turned that AttributeError into a finding for every NSG in production, and the test suite mocked the invented method instead of catching the bug. The remediation script also only ever created NSG flow logs. Microsoft stopped new NSG flow log creation on 2025-06-30 and retires the feature entirely on 2027-09-30, so recommending new NSG flow log creation is itself broken guidance. - AzureClient.get_flow_logs() lists Network Watcher flow logs per-region (there is no flat subscription-wide endpoint), returning a dict keyed by normalized region. A region maps to its list of FlowLog resources, None when that region's watcher/flow-log listing failed, or is simply absent when no watcher exists there - callers must treat all three as indeterminate, never as "no flow log configured". A failure on one region's watcher does not discard results already collected from another region. - az_net_012.scan() now evaluates VNet-scoped flow logs at the correct Azure scope. An existing (never newly created) legacy NSG flow log on one of the VNet's own subnet NSGs is accepted as coverage rather than driving a migration finding, since NSG flow logs remain functional until 2027 and creating a new one is blocked. A region with no reachable Network Watcher evidence is skipped, not flagged - a missing watcher is AZ-NET-011's finding, not this rule's. - playbooks/cli/fix_az_net_012.sh now creates a VNet flow log instead of an NSG flow log, and is idempotent (checks for an already-enabled flow log first), preview-first (prints the exact command before requiring APPLY confirmation), and target-verified (re-reads state after creation, prints the rollback command on verification failure). - Framework mapping descriptions (all four compliance JSONs), docs/rules-reference.md, and website/content.js updated to describe the VNet flow-log evidence semantics instead of NSG-only. Tests: replaced the two tests that validated the rule against a mocked method the real client never implemented with nine tests covering no VNets, VNet flow log compliant/noncompliant, legacy NSG flow log compliant/noncompliant, an unwatched region, a failed region, and partial collection (one region fails without suppressing a real finding in another). Added three unit tests for AzureClient.get_flow_logs() directly covering the multi-region merge, total failure, and per-region partial failure. Signed-off-by: parthrohit22 <parthrohit60@gmail.com>
…6-45833) (#317) * fix(deps): move chromadb to optional requirements-ai.txt (CVE-2026-45830, CVE-2026-45833) Both CVEs affect chromadb >= 0.4.24 with no patch available. Remove the package from requirements.txt so the core install and Docker image are no longer exposed. The ai/ RAG pipeline already handles ImportError gracefully, so the feature continues to work when requirements-ai.txt is installed separately. Add a regression test asserting chromadb never re-enters the core requirements file. Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(deps): pin six explicitly and fix E501 in RAG test azure-mgmt-rdbms==10.1.0 requires six, which was previously satisfied transitively by chromadb. Pin it directly now that chromadb is no longer in core requirements. Also wrap the pytest.importorskip call to stay within the 120-character line limit. Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(deps): wire AI deps opt-in to Dockerfile, restore six for rdbms - Add INSTALL_AI_DEPS build arg (default false); chromadb excluded by default due to CVE-2026-45830/45833 with no upstream fix. AI routes return 503 without it; build with --build-arg INSTALL_AI_DEPS=true to enable the RAG pipeline. - Restore six>=1.16.0: azure-mgmt-rdbms 10.1.0 imports six internally but does not declare it as a metadata dependency, so pip does not install it transitively. Reverting the strict version pin to a floor constraint satisfies the reviewer concern while keeping CI green. - Strip UTF-8 BOM from ai/README.md (Windows editor artifact). Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(ai): replace chromadb with pure-Python BM25 retrieval Removes chromadb entirely (CVE-2026-45830 tenant-isolation bypass, CVE-2026-45833 RCE) and replaces the vector store with a JSON BM25 index built from the same loader/chunker pipeline. - ai/embed.py: tokenise chunks, compute IDF, write bm25_index.json atomically via a .tmp swap so a failed build never corrupts the index - ai/retriever.py: load index and score with BM25 (k1=1.5, b=0.75); VectorStoreNotBuilt exception preserved for API route compatibility - requirements-ai.txt: deleted; no optional install step needed - Dockerfile: removed INSTALL_AI_DEPS build-arg complexity; single pip install -r requirements.txt is sufficient - ai/README.md: documents the BM25 approach and build command - tests/test_rag_dependencies.py: replaces skipped chromadb test with 8 BM25 unit tests covering tokeniser, scoring, and absence checks The retrieve() interface is unchanged: callers receive the same list of {text, source, source_meta} dicts. Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(ai): validate Dockerfile and BM25 retrieval Remove the UTF-8 BOM that made the Dockerfile's first instruction invalid. Add regression coverage for the Dockerfile byte boundary and for building and querying the dependency-free BM25 index with representative OpenShield content. Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(ai): initialize and validate BM25 index Build the dependency-free index into every production image and smoke-test retrieval from that fresh image in CI. Validate the persisted schema before scoring so malformed indexes fail through VectorStoreNotBuilt. Ensure chunking always advances when newlines fall inside the overlap window. Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> --------- Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…#307) * feat: add rule AZ-CMP-007 management ports open without JIT VM access Adds a Compute scan rule that flags VMs whose management ports (SSH 22 / RDP 3389) are open to the internet through their NIC's NSG but are not covered by a Microsoft Defender for Cloud Just-In-Time (JIT) VM access policy, leaving those ports standing open instead of only during an approved, time-boxed request. Detection reuses the repo's proven building blocks: az_net_001's "allow inbound port from any source" test and az_cmp_001's VM -> NIC -> NSG resolution, cross-referenced against JIT policies from a new AzureClient.get_jit_network_access_policies() getter (Defender for Cloud SecurityCenter SDK; azure-mgmt-security is already a dependency). VMs with no management ports open are NOT_APPLICABLE, and when Defender for Cloud cannot be queried (None) coverage is indeterminate and the VM is not flagged, so the rule never false-positives on unknown coverage. Includes the remediation playbook (playbooks/cli/fix_az_cmp_007.sh, with guarded args per the fix_az_net_016.sh convention), mock support (set_jit_policies / get_jit_network_access_policies), six unit tests (open-no-JIT, JIT-covered, NOT_APPLICABLE, trusted-source, partial-coverage, indeterminate), and mappings across the four compliance frameworks (NIST CSF PR.AC-3, ISO 27001 A.13.1.1, SOC 2 CC6.6; CIS uses the repo's N/A convention as Azure Foundations 2.0.0 has no numbered JIT control). Closes #270 Signed-off-by: shariqueahmad108-ship-it <shariqueahmad108@gmail.com> * fix(AZ-CMP-007): port ranges, subnet NSGs, and JIT-preserving remediation Addresses @ritiksah141's review on #307: - Port ranges: _rule_allows_port_from_any now parses '*', single ports, and inclusive ranges (e.g. 20-30 exposing SSH) in both destination_port_range and destination_port_ranges, instead of only exact/'*' matches. - Subnet-level NSGs: exposure is evaluated across every NSG that governs the VM -- the NIC's NSG and the NSG on the NIC IP configuration's subnet (resolved via each NSG's own .subnets back-reference) -- so a VM with no NIC-level NSG but an exposing subnet NSG is no longer a false NOT_APPLICABLE. - Remediation: fix_az_cmp_007.sh now GETs the existing 'default' JIT policy and merges this VM in via jit_policy_merge.py (preserving every other VM's entry) before the create-or-update PUT, instead of overwriting the shared policy with a single VM. Adds regression tests: 22 inside a range, 3389 inside a ranges-list entry, a subnet-only NSG exposure case, and jit_policy_merge tests proving an existing multi-VM policy is preserved (and that re-remediation updates in place). Signed-off-by: shariqueahmad108-ship-it <shariqueahmad108@gmail.com> --------- Signed-off-by: shariqueahmad108-ship-it <shariqueahmad108@gmail.com>
* fix(core): enforce severity contract v1 Signed-off-by: m-khan-97 <ibrahim.logix@gmail.com> * fix(core): make scan persistence atomic Signed-off-by: m-khan-97 <ibrahim.logix@gmail.com> * fix(sentinel): require explicit finding severity Signed-off-by: m-khan-97 <ibrahim.logix@gmail.com> --------- Signed-off-by: m-khan-97 <ibrahim.logix@gmail.com>
* docs: update learn page statistics and add CI/CD workflow
Signed-off-by: Parth Rohit <parthrohit60@gmail.com>
* fix: push to branch explicitly, regenerate stats from current dev
update-learn-page.yml: actions/checkout leaves the runner in detached
HEAD, so the final `git push` had no upstream and would fail with
"You are not currently on a branch". Push HEAD to the triggering ref
by name instead.
Rebased onto dev and re-ran update_learn_page.py against the current
rule set rather than hand-editing numbers that would be stale again
immediately. The README feature-table regex still had the old
rule-category wording from before this branch was cut, so it failed
loud as designed instead of silently no-op'ing; updated it to match
the current sentence.
Removed the "known cleanup item" note about KeyVault vs Key Vault
naming now that the category chart renders the declared category
string directly, and corrected the Known Gaps bullet that still said
README.md counts were updated by hand, now that render_readme() covers
both files.
Signed-off-by: Parth Rohit <parthrohit60@gmail.com>
* fix: avoid inline workflow-context interpolation in push step
github-advanced-security flagged interpolating ${{ github.ref_name }}
directly into the Commit and push run: script as a potential
script-injection pattern. Pass it through an env var instead and
reference it as a shell variable, which keeps the same behavior
(push HEAD to the triggering ref) without templating untrusted
workflow context straight into a shell command.
Signed-off-by: Parth Rohit <parthrohit60@gmail.com>
* fix(docs): warn about excluded severity counts
Signed-off-by: PARTH J ROHIT <parthrohit60@gmail.com>
* style(docs): format learn page updater
Signed-off-by: PARTH J ROHIT <parthrohit60@gmail.com>
* fix(docs): count playbooks by rule match, not raw file glob
count_playbooks() globbed every *.sh in playbooks/cli/, which also picked
up review_enterprise_resilience.sh -- a shared helper several fix_*.sh
wrappers exec into, not a playbook any single rule owns. That inflated
the count past rule_count (95 rules / 96 playbooks) and broke the "every
rule ships with a matching playbook" claim in the README.
find_matching_playbooks() now mirrors the fix_<rule filename stem>.sh
convention ci.yml's playbook_check step already enforces, derives the
count from actual rule-to-playbook matches, and fails loudly if a rule
has no matching playbook or the two counts disagree. Unmatched files
(shared helpers) are reported as a non-fatal warning instead of silently
inflating the total.
Also: collect_rule_stats() now skips files with no parseable RULE_ID,
matching count_rules()'s own filter, so the severity-box totals can no
longer disagree with the headline rule count; and render_category_rows()
HTML-escapes the category name before embedding it.
Regenerated docs/learn/index.html and README.md: 95 rules / 95 playbooks.
Signed-off-by: PARTH J ROHIT <parthrohit60@gmail.com>
---------
Signed-off-by: Parth Rohit <parthrohit60@gmail.com>
Signed-off-by: PARTH J ROHIT <parthrohit60@gmail.com>
* Add enterprise network perimeter controls Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * Fix DNS evidence and WAF logging checks Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * Fix firewall threat intel and WAF rate checks Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * Preserve unknown perimeter evidence states Signed-off-by: ritiksah141 <ritiksah141@gmail.com> --------- Signed-off-by: ritiksah141 <ritiksah141@gmail.com>
…320) Partially addresses #294 - the immediate, in-repo containment slice. Not a fix for the full issue: durable multi-tenant authorization (Entra/OIDC Authorization Code + PKCE, JWKS validation, per-tenant schema/RLS) is a separate, larger body of work that needs an actual identity-provider app registration against this deployment's real domains, which is outside what a PR to this repository can provision or decide alone. That part stays open and un-claimed here. What this closes, confirmed live before this commit (verify_jwt() in api/app.py only checked the signature and expiry-if-present; g.user was never read by any route; the baseline schema has no tenant/org column at all): - A JWT with no exp claim decoded successfully forever - the documented VITE_JWT_TOKEN demo credential had no expiry so it works without rotation, per its own prior docstring. jwt.decode() now requires exp; a token missing it is rejected the same as an invalid signature. - role was decoded into g.user but never checked by any route, so a viewer-role token could call every write endpoint (scan trigger, every AI endpoint) exactly like an admin token. verify_jwt() now requires a recognized role (viewer/operator/admin) on every accepted token, and rejects any non-GET/HEAD request from a viewer token with 403 - enforced once, in the middleware, so it covers every current and future write route uniformly rather than needing a per-route check. Confirmed this still holds under OPENSHIELD_PUBLIC_DEMO=true - demo mode only ever widens read access, never write. - POST /api/scans/trigger accepted an arbitrary caller-supplied subscription_id with no authorization check - any operator/admin token could trigger a scan against any subscription. Added an optional OPENSHIELD_AUTHORIZED_SUBSCRIPTIONS allowlist (issue #294's own suggested interim for a single-tenant deployment); left unset, behavior is unchanged from today, but the API now logs a loud startup warning when it is, matching the existing ALLOWED_ORIGINS/JWT_SECRET convention in the same function. - scripts/generate_demo_jwt.py minted a permanent, no-expiry token by design. It now requires an expiry (DEMO_JWT_TTL_HOURS, default 24h) and its docstring/output no longer claim the token works without rotation. - docs/api-reference.md and .env.example document the new requirements and the allowlist variable. Deliberately left untouched: frontend/src/App.jsx's local-dev dev-local-token bootstrap. It is not a valid signed JWT (verify_jwt() would reject it outright outside demo mode) and only ever worked because OPENSHIELD_PUBLIC_DEMO bypasses the JWT check for GET - removing it without a replacement login flow would break local dev for every contributor while fixing nothing real, since it was never a working credential to begin with. The actual leaked-credential risk is VITE_JWT_TOKEN, which is what this commit bounds with expiry and role. Verified: full backend suite (796 passed, 5 skipped - pre-existing/ environment-only, unrelated - confirmed identical skip count against unmodified dev), including all pre-existing tests/test_auth.py and tests/test_async_scan_persistence.py cases (every existing token fixture in this repo already carried role+exp, so nothing broke). New coverage in tests/test_auth.py (missing-exp, missing-role, unrecognized-role, viewer-read-allowed, viewer-write-rejected including under demo mode, operator/admin-write-allowed) and the new tests/test_subscription_authorization.py (unset allowlist preserves today's behavior, allowlisted/non-allowlisted subscriptions, case-insensitive comparison, blank-entry handling, startup-warning presence/absence). ruff check and format --check clean. Signed-off-by: Parth J Rohit <parthrohit60@gmail.com> Signed-off-by: parthrohit22 <parthrohit60@gmail.com>
…les (AZ-IDN-016-025) (#279) * feat: complete enterprise data protection rules Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * style: format storage rule tests Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix: satisfy rule validation and refresh image packages Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * feat: implement enterprise privileged access and identity security rules (AZ-IDN-016-025) Add 10 new rules covering privileged identity security for Microsoft Entra ID: - AZ-IDN-016: Privileged user missing phishing-resistant MFA (CRITICAL) - AZ-IDN-017: Global Administrator permanently assigned outside PIM (HIGH) - AZ-IDN-018: Privileged role assigned outside PIM (HIGH) - AZ-IDN-019: Stale privileged account retains active access (HIGH) - AZ-IDN-020: No emergency access accounts detected (HIGH) - AZ-IDN-021: Legacy authentication protocols not blocked (HIGH) - AZ-IDN-022: No MFA requirement for Azure Management (HIGH) - AZ-IDN-023: Identity Protection risk policies disabled (MEDIUM) - AZ-IDN-024: Service principals excluded from MFA enforcement (MEDIUM) - AZ-IDN-025: Privileged role-assignable group has no owner (MEDIUM) Add 5 new Graph API collectors to azure_client.py: - get_privileged_role_members, get_privileged_users_mfa_methods, get_pim_role_assignments, get_identity_protection_policies, get_privileged_groups Add MockAzureClient support for all new collectors (fully offline tests). Add 58 tests in test_rules_identity_priv.py covering compliant, violating, empty inventory, API failure (None), and edge cases. Add 10 remediation playbooks (fix_az_idn_016.sh through fix_az_idn_025.sh). Add compliance mappings for CIS Azure 2.0.0, NIST CSF, ISO 27001, SOC 2. Add docs/rules-reference.md entries for all 10 rules. Closes #258 Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(scanner): correct privileged identity evaluation Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(scanner): address review blockers in privileged access and identity rules PIM detection (IDN-017, IDN-018): - Collector now queries both roleAssignmentSchedules (Active) and roleEligibilitySchedules (Eligible) so eligible assignments are actually detected; previously only Active schedules were fetched - principalId/principalType tracked for all principal types (users, groups, service principals), not only users Identity Protection (IDN-023): - Replace unreliable direct policy endpoints with CA-policy inspection: checks conditions.userRiskLevels / signInRiskLevels on enabled policies CA policy user scope (IDN-021, IDN-022): - _covers_all_users() helper added; policies that target a subset of users no longer suppress a tenant-wide finding Workload identity exclusion (IDN-024): - Fix excludeServicePrincipals check: the field holds SP IDs, not All; rewrite to use includeServicePrincipals presence and All-exclude logic Collection failures (Cosmos DB, Redis cache): - get_cosmos_accounts / get_managed_caches return None on failure instead of []; callers log a warning and skip rather than treating missing inventory as compliant Enum normalization (AZ-STOR-009, AZ-DB-007): - Replace raw str() calls with enum_str() for SDK enum fields - Fix immutability retention property: period_since_creation_in_days -> immutability_period_since_creation_in_days Test fixtures updated to match new API contracts (principalId, deep-merge of CA policy conditions, correct immutability property name). CI: add chromadb CVE-2026-45830 and CVE-2026-45833 to pip-audit ignore list (no patched version; removal tracked in PR #317). Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(tests): update vector-store skip guard to check bm25_index.json The skip condition checked for the ai/vectorstore/ directory, which can exist from a previous build without the BM25 index file. After PR #317 replaced chromadb with a JSON BM25 index, the test ran and raised VectorStoreNotBuilt instead of skipping. Changed skipif to check for the actual index file so the class is skipped correctly when the index has not been built. Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(lint): shorten E501 line in test_ai_hallucination_guard.py Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> --------- Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
) * fix(api): rate-limit /ready and /metrics, expose db pool telemetry Closes the remaining scope of #296. The connection-checkout leak itself was already fixed by #306 (g.db + the teardown handler, with a real PostgreSQL-backed integration test proving connections balance across repeated readiness probes). What was still open: /ready and /metrics are unauthenticated by design (probe/scrape endpoints must never require a token), which also made them the one place an unauthenticated caller could trigger repeated pooled-connection work with no rate limiting at all, and there was no visibility into how close the pool was to exhaustion before it happened. - Add api.observability.probe_rate_limit: an in-memory, per-process, per-source-IP rate limiter for probe/scrape endpoints. Deliberately not the existing Postgres-backed api.rate_limit.rate_limit, which would add a database round trip (and a second, separately-tracked pooled connection under its own g.db) to the exact endpoint whose job is to protect the database from overload. The connection pool it guards is itself process-local under Gunicorn's multi-worker model, so a per-process budget is the matching granularity, not a weaker substitute for a shared one. Wired onto /ready (budget of 5 per 10s per source IP, half the default DB_POOL_MAX_CONN) and /metrics (20 per 10s, generous for normal Prometheus scrape intervals). The check runs before the view body, so a rejected request never reaches the database work it would otherwise trigger. - Add api.models.finding.get_pool_stats(): a point-in-time snapshot of the shared pool's in-use/idle/max-connection counts and utilization percentage. Reports only counts - never the DSN, host, or credentials - so it's safe on a public surface. Returns zeroed stats before any connection has been made instead of raising. - Wire get_pool_stats() into three new Prometheus gauges (openshield_db_pool_connections_in_use/idle/max), refreshed lazily on every /metrics scrape via a provider callback registered from api/app.py - api/observability.py stays free of project imports (its own documented constraint, since the worker reuses it too) by never importing api.models.finding directly. - Document in docs/deployment/render.md that Render's Blueprint format has no path-based access control, so the in-app rate limiter is a defense-in-depth backstop, not a substitute for restricting network reachability to /ready and /metrics at whatever reverse proxy/CDN/ WAF fronts a real deployment - that configuration is operational, outside what render.yaml can express. New tests in tests/test_readiness_hardening.py cover: the rate limiter's budget/window/per-IP-isolation/testing-bypass/key-pruning behavior in isolation, /ready actually rejecting a source once its budget is spent without touching the database for the rejected request, get_pool_stats()'s zero/nonzero/never-leaks-the-dsn behavior, and /metrics rendering the three new gauges (and surviving a broken stats provider without failing the whole scrape). Verified: full backend suite (796 passed, 3 skipped - pre-existing, unrelated), including tests/test_observability.py's real PostgreSQL-backed readiness-leak test run against a local Postgres instance to confirm the new decorator doesn't disturb #306's fix; ruff check and format --check clean. Signed-off-by: Parth J Rohit <parthrohit60@gmail.com> Signed-off-by: parthrohit22 <parthrohit60@gmail.com> * fix(api): restore dropped constant, bound probe-limiter memory Two fixes on top of the dev merge that landed on this branch: 1. The merge into dev (which now carries #294/#320's own changes to api/app.py) dropped this branch's _READY_MAX_REQUESTS_PER_WINDOW constant definition while keeping its usage on the /ready route, leaving api/app.py with an undefined name that only surfaced at create_app() call time (ruff's F821 caught it as CI's first failure; Backend Tests failed for the same underlying reason). Restored the constant and its comment. 2. m-khan-97's review: probe_rate_limit's per-key cleanup only ever prunes the exact (address, path) key the current request touches. A caller that continuously rotates its source address - or a spoofed forwarded address wherever the trusted-proxy boundary is misconfigured - creates a new one-shot dictionary entry per address that's never revisited and therefore never pruned, making the limiter's own tracking dict an unbounded memory sink. Fixed with the two things asked for: - A periodic global sweep (every _PROBE_SWEEP_INTERVAL calls, not every call - a full-dict scan per request would defeat the point of a cheap in-memory limiter) that prunes every key whose hits have all expired, not just the current request's key. - A hard cap (_PROBE_MAX_TRACKED_KEYS) on distinct tracked keys, with deterministic least-recently-touched eviction via an OrderedDict instead of the previous plain dict - every hit (including one that just survives a sweep) moves its key to the end, so eviction always drops the coldest entry first. Also added the suggested (non-blocking) Retry-After header on 429. New tests in tests/test_readiness_hardening.py cover exactly the scenario m-khan-97 described: many distinct one-shot addresses, advance past the window, trigger cleanup through different addresses, and prove the stale keys are gone and the map stays bounded. Plus direct hard-cap/LRU-eviction tests and the Retry-After header. Verified: full backend suite (862 passed, 5 skipped - pre-existing/ environment-only), including all 17 tests in tests/test_readiness_hardening.py and the pre-existing tests/test_auth.py / tests/test_observability.py suites unaffected. ruff check and format --check clean. Signed-off-by: Parth J Rohit <parthrohit60@gmail.com> Signed-off-by: parthrohit22 <parthrohit60@gmail.com> * fix(api): address TFT444's 3 probe-limiter findings 1. move_to_end(key) ran on every request including rejected ones, so an attacker hammering an already-exhausted key kept it permanently at the back of the eviction order while quiet, legitimate keys drifted toward the front and got evicted instead - defeating the memory cap's actual purpose for exactly the caller it exists to bound. Now only a request that counts against the budget (allowed) refreshes a key's position; a rejected request leaves it wherever it already was. 2. _probe_hits is one dict shared by every probe_rate_limit-decorated endpoint, but the periodic sweep applied whichever endpoint's request happened to trigger it - its own window_seconds - to every tracked key regardless of which endpoint's window actually applies to it. Dormant today since /ready and /metrics both default to the same 10s window, but the first endpoint added with a different one would have caused premature resets or lingering stale entries across every other endpoint's keys. Added _ProbeEntry to carry window_seconds alongside each key's own hits deque, so the sweep (and the per-call prune) always uses the window that key was actually registered under. 3. Documented the ProxyFix(x_for=1) trust boundary explicitly in api/app.py: what it assumes (Render's edge is the only thing able to append to X-Forwarded-For before this process sees it) and what breaks if that's violated (a directly-reachable origin lets a caller set their own forwarded IP per request, which is equivalent to no per-IP rate limiting at all for every control that depends on request.remote_addr). New regression tests for both behavioral fixes: one proves a caller hammering an exhausted key doesn't stay artificially warm while a quiet legitimate key gets evicted in its place; the other runs two endpoints with different window_seconds sharing the tracked-key dict and proves a sweep triggered by one doesn't misapply its window to the other's entries. Verified: full backend suite (864 passed, 5 skipped - pre-existing/ environment-only), all 19 tests in tests/test_readiness_hardening.py including the 2 new ones, ruff check and format --check clean. Signed-off-by: Parth J Rohit <parthrohit60@gmail.com> Signed-off-by: parthrohit22 <parthrohit60@gmail.com> --------- Signed-off-by: Parth J Rohit <parthrohit60@gmail.com> Signed-off-by: parthrohit22 <parthrohit60@gmail.com>
Bumps the npm_and_yarn group with 1 update in the /frontend directory: [postcss-selector-parser](https://github.com/postcss/postcss-selector-parser). Updates `postcss-selector-parser` from 6.1.2 to 6.1.4 - [Release notes](https://github.com/postcss/postcss-selector-parser/releases) - [Changelog](https://github.com/postcss/postcss-selector-parser/blob/main/CHANGELOG.md) - [Commits](postcss/postcss-selector-parser@v6.1.2...6.1.4) --- updated-dependencies: - dependency-name: postcss-selector-parser dependency-version: 6.1.4 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the npm_and_yarn group with 1 update in the /frontend directory: [browserslist](https://github.com/browserslist/browserslist). Updates `browserslist` from 4.28.2 to 4.28.9 - [Release notes](https://github.com/browserslist/browserslist/releases) - [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md) - [Commits](browserslist/browserslist@4.28.2...4.28.9) --- updated-dependencies: - dependency-name: browserslist dependency-version: 4.28.9 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat(frontend): add API request timeout support Signed-off-by: Shaurya K Sharma <shauryaksharma24@gmail.com> * fix(frontend): preserve transient API fallback semantics Signed-off-by: Shaurya K Sharma <shauryaksharma24@gmail.com> --------- Signed-off-by: Shaurya K Sharma <shauryaksharma24@gmail.com>
* website: migrate to Astro + Decap CMS with GitHub Pages pipeline Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * website: commit src/lib, bump astro to patched 7.x, drop obsolete CI job Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * website: build hero legend and rule info with DOM APIs, no innerHTML Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * fix(website): resolve manual test findings across hero, docs, feed and articles Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * feat(website): complete site and automate Pages deployment Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * fix(website): keep CMS optional and resolve CodeQL Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * ci: include website validation in required summary Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * style(ci): keep website gate comment concise Signed-off-by: ritiksah141 <ritiksah141@gmail.com> * fix(ci): preserve container workflow contract Signed-off-by: ritiksah141 <ritiksah141@gmail.com> --------- Signed-off-by: ritiksah141 <ritiksah141@gmail.com>
* feat: complete enterprise data protection rules Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix: satisfy rule validation and refresh image packages Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(scanner): address review blockers in enterprise data-protection rules - Move AZ-STOR-009 opt-in check from BlobContainer (no ARM tags) to the parent storage account, which exposes tags via the SDK; all containers under a tagged account are now evaluated for immutability. - Replace incorrect NIST mapping A.12.4.1 (ISO 27001) on AZ-DB-007 with PR.PT-1 across az_db_007.py, nist_csf.json, and rules-reference. - Add executable az CLI commands to fix_az_cache_001, fix_az_cosmos_001, fix_az_cosmos_002, fix_az_db_005, fix_az_db_006, and fix_az_db_007 playbooks; each validates the target and requires APPLY confirmation before modifying any Azure resource. - Update storage-protection-controls.md to document the account-level tagging scope for AZ-STOR-009. Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(scanner): check immutability tag on container, not account (AZ-STOR-009) The policy_required guard was placed at the account level, but the oshield:immutability-required tag is set per container. Moving the check inside the container loop allows containers with the tag to be evaluated regardless of whether the parent account carries it. Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(scanner): check immutability tag on container or parent account (AZ-STOR-009) The policy_required guard was placed at the account level only, but the oshield:immutability-required tag may be set per-container or per-account. Now uses OR logic: a container is evaluated if the account carries the requirement tag (protecting all containers) OR if the container itself carries it (per-container opt-in). Both cases were previously broken: the account-level check did not reach container-tagged resources, and no per-container check existed at all. Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(scanner): address storage rule correctness gaps in AZ-STOR-006/007/008 - AZ-STOR-006: treat allow_shared_key_access=None as insecure (Azure documents unset as equivalent to True); only False is compliant - AZ-STOR-007: treat minimum_tls_version=None as TLS 1.0 (Azure default); use enum_str() instead of str() to handle SDK enum objects correctly - AZ-STOR-008 playbook: fix Key Vault URI parsing; the previous bash expansion passed the wrong segments to --encryption-key-vault and --encryption-key-name; now splits vault URI, key name, and optional key version correctly - ci.yml: remove CVE-2026-45830 and CVE-2026-45833 pip-audit exclusions (chromadb CVEs unrelated to this PR; resolved by PR #317) Adds regression tests for None-as-default behavior and SDK enum handling in AZ-STOR-006 and AZ-STOR-007 (22 storage tests, all passing). Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> * fix(scanner): address all review feedback and CI failures for PR #278 Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com> --------- Signed-off-by: Tanvir Farhad <tamimtarafder12@gmail.com>
…336) * build: lock Python runtime and development dependencies with hashes Signed-off-by: Muhammad Ibrahim <135441675+m-khan-97@users.noreply.github.com> * fix(ci): isolate lock tooling outside the source checkout Signed-off-by: Muhammad Ibrahim <135441675+m-khan-97@users.noreply.github.com> --------- Signed-off-by: Muhammad Ibrahim <135441675+m-khan-97@users.noreply.github.com>
Signed-off-by: ritiksah141 <ritiksah141@gmail.com>
…attestations (#339) Signed-off-by: Muhammad Ibrahim <135441675+m-khan-97@users.noreply.github.com>
Signed-off-by: ritiksah141 <ritiksah141@gmail.com>
Vishnu2707
requested review from
SHAURYAKSHARMA24,
TFT444,
parthrohit22 and
vogonPrayas
as code owners
September 16, 2026 12:32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Merges 149 commits from dev into main covering the full v0.5.0 release —
rule evaluations per rule per resource, scan durability and idempotency
hardening, enterprise rule packs, AI layer foundations, frontend
accessibility and i18n, signed releases, supply chain security, OWASP
migration, and dependency security fixes.
Type of change
Key changes
Testing
Related issues
Closes #263, #303
Depends on #321, #325 already merged to dev
Checklist