Consolidate EnvGene instance pipeline jobs into a single job

[popoveugene]

Context

The EnvGene instance pipeline today runs each functional area as a separate GitLab job. Each job pays the cost of runner startup, image pull, and artifact handoff. Consolidating these areas into one job removes that overhead and improves end-to-end runtime.

Requirements

First Priority

1. Consolidate all current EnvGene instance pipeline jobs into a single GitLab CI job. Excluded from consolidation: cmdb_import and the new sync job.

2. Group logs in the unified job by current job name, for example using GitLab collapsible log sections (section_start / section_end).

3. After PoC, only the unified job remains; the per-job pipeline is retired.

4. Current validations must run within the unified job.

5. Multiple env handling support is removed. Concurrency arbitration across parallel pipelines on different environments is out of scope. A locking or arbitration system is assumed to exist independently.

6. Keep each current function atomic. Its inputs, outputs, invocation criteria, CI variables, and behavior do not change. This is a structural consolidation, not a functional rewrite.

7. Output is unchanged. Given identical inputs, the unified job must produce the same effective set and the same environment instance as the current pipeline.

8. Caching policy is unchanged. SBOM, registry resolution, and app-defs continue to cache between runs as today.

9. Failure semantics match today's. A failure inside the unified job aborts it; retry of individual internal functions is not required.

10. Sensitive data handling is unchanged. Sensitive variables that today are filtered from propagated job variables continue to be handled the same way — no extra exposure in logs or shared scope.

Second Priority

1. The unified job emits per-phase duration metrics so the runtime NFR can be verified per phase, not only end-to-end.

Open questions

1. Image strategy. The proposed direction is to introduce a new combined image and remove old ones

Deliverables

• A dev design proposal addressing the Open questions and the design directives in Requirements.
• The implementation of the unified job per the approved design.

Acceptance

• Given identical inputs, the unified job produces the same effective set and the same environment instance as the existing per-job pipeline.