Integration Airflow#38

Closed

abir-oumghar wants to merge 37 commits into

mainfrom

integration-airflow

Commits on Feb 1, 2026

feat: add spark-operator package and release
mlahouar
committed

Commits on Feb 9, 2026

feat(airflow): add airflow package and release configuration

Abir Oumghar
authored and
Abir Oumghar
committed
feat(rbac): add spark-airflow RBAC configuration

Abir Oumghar
authored and
Abir Oumghar
committed
feat(ui): integrate airflow into okdp ui context

Abir Oumghar
authored and
Abir Oumghar
committed
feat(examples): add airflow dag examples with spark integration

Abir Oumghar
authored and
Abir Oumghar
committed
feat(airflow): add Airflow package and release configuration

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): update package, release and context for proper integration

Abir Oumghar
authored and
Abir Oumghar
committed

Commits on Feb 12, 2026

feat(airflow): integrate dags, spark submission, and airflow docs

Abir Oumghar
authored and
Abir Oumghar
committed
fix(ci): avoid empty pending pod describe in kubocd package workflow

Abir Oumghar
authored and
Abir Oumghar
committed
fix(ci): wait for release ready condition and extend timeout

Abir Oumghar
authored and
Abir Oumghar
committed
fix(ci): poll release phase readiness instead of kubectl wait

Abir Oumghar
authored and
Abir Oumghar
committed
fix(ci): stabilize release readiness loop and diagnostics

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): rewrite package for Helm chart 1.17.0 compatibility

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): switch to LocalExecutor for CI Kind cluster compatibility

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): drastically reduce resource requests for CI Kind cluster

Abir Oumghar
authored and
Abir Oumghar
committed

Commits on Feb 16, 2026

fix(airflow): pin PostgreSQL image to stable tag to fix ImagePullBackOff in CI

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): use shared CNPG PostgreSQL instead of bundled bitnami subchart

Abir Oumghar
authored and
Abir Oumghar
committed
fix(cnpg-postgresql): add tools dependency to ensure secret replication before schema creation

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): align chart version and ingress hosts

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): expose default instance and add missing s3 secret

Abir Oumghar
authored and
Abir Oumghar
committed
chore(airflow): bump package tags to force rollout

Abir Oumghar
authored and
Abir Oumghar
committed
fix airflow ingress and add validated DAG test workflow

Abir Oumghar
authored and
Abir Oumghar
committed

Commits on Feb 17, 2026

fix(ci): create ci-registry-secret in all release namespaces

Abir Oumghar
authored and
Abir Oumghar
committed
fix(ci): unblock airflow migrations and improve failure diagnostics

Abir Oumghar
authored and
Abir Oumghar
committed
fix(ci): prevent airflow and keycloak OOM in sandbox packages

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): increase runtime resources and bump release to p06

Abir Oumghar
authored and
Abir Oumghar
committed
docs(readme): refine platform services wording

Abir Oumghar
authored and
Abir Oumghar
committed
docs(readme): remove BI wording for superset descriptions

Abir Oumghar
authored and
Abir Oumghar
committed
docs(readme): remove keycloak from existing services list

Abir Oumghar
authored and
Abir Oumghar
committed

Commits on Feb 19, 2026

feat(airflow): add git/s3 DAG sources and align example DAG tests

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): escape s3 sync env vars for kubocd packaging

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): preserve values template line breaks for kubocd render

Abir Oumghar
authored and
Abir Oumghar
committed

Commits on Feb 22, 2026

feat(airflow): add daily hello + spark ETL DAGs and harden deploy/tests

Abir Oumghar
authored and
Abir Oumghar
committed

Commits on Feb 23, 2026

fix(airflow): read DB credentials from context secret in runtime env

Abir Oumghar
authored and
Abir Oumghar
committed
fix(airflow): load DB user/password via chart secret env entries

Abir Oumghar
authored and
Abir Oumghar
committed

Commits on Feb 25, 2026

spark rbac fix

Abir Oumghar
authored and
Abir Oumghar
committed

Commits on Mar 9, 2026

add nyc taxi spark pipeline example

Abir Oumghar
authored and
Abir Oumghar
committed