feat: add support for parallel dag processor in helm

[kush-wisdom]

• closes: [Helm Chart] Support multiple DagProcessors #26494
• related: Helmchart: Provision Standalone Dag Processor #23711

This PR adds a new sub-section under dag-processor of values.yaml of the airflow helm chart to allow running multiple-dag processor as per the sub-directory #26494

I also have added tests via the help of Opus in here to make sure the changes are well-off.

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: [Opus 4.5] following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[kush-wisdom]

cc: @potiuk @shahar1 This has been a blocker for us to scale airflow; and we want to prefer using upstream instead of a fork.

[jscheffl]

We are in the middle of releasing Helm chart 1.19. Therefore please contribute testing and we will re-visit the PR afterwards. No hurry.

[jscheffl]

Sorry, coming a bit late back to this PR as review. Soon will cut release 1.20. Do you want to resolve conflicts and rework/respond to comments just added?

[jscheffl]

For me it does not make much sense to define multiple Dag processors and if no args given then falling back to the base. Then you would just have multiple replicas. If I decide to have multiple deployments then I'd assume there is a reason for different args == falback does not make sense in my view.

[jscheffl]

For me it does not make much sense if the user must specify the full CLI here. It should rather present "additional args" and not the base shell command. Or would there be a reason to all another shell with another CLI for the use case?