Skip to content

Improvement/ add pandera schema to rapid dataframe schema - #197

Open
Jarek-Rolski wants to merge 11 commits into
no10ds:mainfrom
co-cddo:pandera-schema
Open

Jarek-Rolski wants to merge 11 commits into
no10ds:mainfrom
co-cddo:pandera-schema

Conversation

@Jarek-Rolski

@Jarek-Rolski Jarek-Rolski commented May 5, 2026

Copy link
Copy Markdown
Contributor

This PR adds the panderaDataFrameSchema field to the rAPId Pydantic schema.

Changes included

  • Added an optional panderaDataFrameSchema field that requires the Pandera DataFrameSchema type.

  • Added pandera_schema_validation function to validate pandera schema and parse_pandera_schema_errors function to parse pandera error message and return it as list of strings.

  • Added support for registering custom Pandera checks through the PANDERA_FILE environment variable.

    • The variable stores a JSON object where:

      • keys = file paths
      • values = file contents
    • The API imports these files during startup to register custom checks.

  • Updated Terraform configuration to:

    • add a variable for generating the PANDERA_FILE environment variable
    • inject the variable into the ECS service configuration

Compatibility

This change is backward compatible.

@Jarek-Rolski
Jarek-Rolski marked this pull request as ready for review May 7, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant