Skip to content

Refactor Slice 2: Move batch/scheduler logic into CIME/core/batch/ #4997

Description

@jasonb5

Part of #4950. Slice 2 of 5. Estimated 4 weeks. Depends on #4996.

Goal

Move batch logic from scattered locations (XML/env_batch.py,
case/case_submit.py, utils.py) into a coherent CIME/core/batch/ package.
Existing entrypoints become thin wrappers.

Why

EnvBatch (XML/env_batch.py, 1590 lines, 43 methods) is the second-largest
class in CIME. It handles batch config parsing, job submission, queue
management, and directive generation all in one class. Submission logic is
also smeared into case_submit.py and utils.py.

Tasks

Decompose EnvBatch

  • CIME/core/batch/config.py — batch config / queue parsing.
  • CIME/core/batch/submit.py — job submission logic (consumes from
    EnvBatch and case_submit.py).
  • CIME/core/batch/directives.py — directive generation.
  • EnvBatch becomes a thinner wrapper delegating to the above.

Scheduler protocol

  • Define Scheduler protocol in CIME/core/batch/scheduler.py.
    PBS, Slurm, LSF are genuine polymorphic backends — DI is warranted here.
  • Concrete implementations: PBSScheduler, SlurmScheduler, LSFScheduler.

Move from utils.pycore/batch/

  • batch_jobid, get_batch_script_for_job, get_project,
    get_charge_account, add_mail_type_args, resolve_mail_type_args,
    transform_vars.

Move from utils.pycore/shell.py

  • run_cmd, run_cmd_no_fail (general utility, pulled here because batch
    is the first heavy consumer).

Consolidation

  • transform_vars (utils.py:2118) takes case and calls
    case.get_value() repeatedly — consolidate as a Case method, with
    core template logic in CIME/core/batch/.
  • batch_jobid (utils.py:2360) calls case.get_job_id() — move into
    the batch subsystem.

Imports & re-exports

  • Update internal CIME imports to point at core/batch/.
  • Leave re-exports in utils.py, case/case_submit.py for external
    consumers.

Definition of done

  • All batch tests pass.
  • External models (E3SM, CESM, NorESM) submit jobs unchanged.
  • Batch logic lives in core/batch/, not in XML/, case/, utils.py.
  • EnvBatch is a thin wrapper.
  • Scheduler protocol with PBS/Slurm/LSF implementations.
  • 80%+ coverage on core/batch/.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions