Skip to content

Call ecflow_client to set event for post jobs#53

Merged
aerorahul merged 3 commits into
NOAA-EMC:production/mlglobal.v1from
RussellManser-NCO:post_jobs
Sep 26, 2025
Merged

Call ecflow_client to set event for post jobs#53
aerorahul merged 3 commits into
NOAA-EMC:production/mlglobal.v1from
RussellManser-NCO:post_jobs

Conversation

@RussellManser-NCO

Copy link
Copy Markdown

This PR calls set_event.sh to trigger post jobs for aigfs (see NOAA-EMC/aigfs#4). Post jobs in in NOAA-EMC/aigfs#4 create grib2 .idx files and copy fiels to COM. This PR replaces #48.

Please note that this solution has already been implemented in para.

Comment thread oper/utils/grib2writer.py
@RussellManser-NCO

Copy link
Copy Markdown
Author

Here is an example of implementing an ecflow API solution

# import statements
# ...

import ecflow

# ...

        # Release post job to create index files and copy files to COM
        if os.getenv("envir") is not None:
            client = ecflow.Client(
                host=os.getenv("ECF_HOST"),
                port=os.getenv("ECF_PORT"),
            )
            ECF_NAME = os.getenv("ECF_NAME")
            client.force_event(f"{ECF_NAME}:release_f${lead:03d}", "set")

To the best of my knowledge, this would work if ecflow on WCOSS were built with python 3.12. Thank you, Rahul, for pointing out that the venv and ecflow would use the same python interpreter.

@StevenEarle-NCO

Copy link
Copy Markdown

I'm recommending we go live with the version in parallel, which is the subshell implementation @RussellManser-NCO describes in this PR. I understand your concerns, @aerorahul, and ultimately the solution here is to use the python API. We will begin pursuing that, so we can switch out the subshell for consistent python 3.12 ecflow API calls. We should be able to build just the API (not the client or server binaries) and put it inside the venv for aigfs/gefs.

Comment thread oper/utils/grib2writer.py Outdated
Comment thread oper/utils/grib2writer.py Outdated
Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com>

@aerorahul aerorahul left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.
thanks for working through the comments, and evaluating alternatives.
This will work for v1, but we should create a NCO bugzilla for a later update.

@LinlinCui-NOAA LinlinCui-NOAA left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks.

@StevenEarle-NCO

StevenEarle-NCO commented Sep 26, 2025

Copy link
Copy Markdown

@aerorahul if it's okay with you... I'd like to only use github issues for the AI applications, not NCO bugzilla. It's time for us to start the transition and this app has been leading the way into the future from the start. Can we create / agree on a template to use for ops issues and then we start using that?

@aerorahul
aerorahul merged commit f112457 into NOAA-EMC:production/mlglobal.v1 Sep 26, 2025
@aerorahul

aerorahul commented Sep 26, 2025

Copy link
Copy Markdown
Contributor

@aerorahul if it's okay with you... I'd like to only use github issues for the AI applications, not NCO bugzilla. It's time for us to start the transition and this app has been leading the way into the future from the start. Can we create / agree on a template to use for ops issues and then we start using that?

YES!
Absolutely. I can create an issue template that will apply a tag nco-bug to report and track an ops issue. We use this approach in the global-workflow to manage/mirror NCO Bugzilla's.

Here is an example from the global-workflow on the template, if you and others are interested.
Issues => Click on New Issue => Select NCO Bug Report => Fill out the details.
When filled, it will create a Github issue that looks like this

Suggestions welcome!

@aerorahul

Copy link
Copy Markdown
Contributor

@StevenEarle-NCO See #57

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants