Introduce a plain foreman flavor by ekohl · Pull Request #511 · theforeman/foremanctl

ekohl · 2026-05-15T14:27:24Z

Why are you introducing these changes? (Problem description, related links)

Today Foreman can be deployed without Katello on both RPM and Debian installations and this allows users to mirror that.

What are the changes introduced in this pull request?

The flavor itself is rather trivial, but there are some changes made to only deploy certain roles depending on the features enabled. That is likely incomplete. In particular, Apache is not yet modified to only deploy the required services. In the future when a plain foreman-proxy role is introduced then that will need adjustment.

There are also heavy testing changes, heavily based on #509 which in turn is based on #508. I'm wondering how much of those commits already belong there.

Right now the test suite fails because the foreman feature is only set in the flavor and not in the parameters.

It's also important to look at how this will play into #431.

How to test this pull request

Steps to reproduce:

Run foremanctl deploy --flavor foreman
Observe only Foreman it deployed without Pulp and Candlepin
Test suite passes

Checklist

Tests added/updated (if applicable)
Documentation updated (if applicable)

arvind4501 · 2026-06-02T09:11:13Z

Our CI is currently not designed to run flavor specific tests https://github.com/theforeman/foremanctl/blob/master/.github/workflows/test.yml#L54, it adds all features but surely that won't make sense, as we want to only add relevent features for a flavor(ex: add bmc to plain foreman does not make any sense),

Thinking out loud:-
Should we make test workflow per flavor, I am still trying to get all data points which can help in multi-flavor testing

ekohl · 2026-06-02T10:34:57Z

Should we make test workflow per flavor, I am still trying to get all data points which can help in multi-flavor testing

If you're talking about a GitHub Actions workflow: I think that does make sense. We probably can't test all combinations, but finding a good set is exactly what I think we should do.

arvind4501 · 2026-06-02T11:13:30Z

If you're talking about a GitHub Actions workflow

yes,and i think without flavor specific workflows we might end up with a single tests workflow which adds all available features(as of today) and will keep CI red

ekohl · 2026-07-14T20:14:17Z

I'd still like to see this. Some PRs that are opened because of this, but can be merged already:

After that I'll rebase.

ekohl · 2026-07-17T14:09:12Z

The exception is that import_paths and export_paths are unbound in this code:

foremanctl/tests/pulp_test.py

Lines 11 to 22 in 4d2d756

    
           def load_pulp_paths_from_parameters(): 
        
               test_dir = os.path.dirname(os.path.abspath(__file__)) 
        
               foremanctl_dir = os.path.dirname(test_dir) 
        
               params_file = os.path.join(foremanctl_dir, '.var', 'lib', 'foremanctl', 'parameters.yaml') 
        
               if os.path.exists(params_file): 
        
                   with open(params_file, 'r') as f: 
        
                       params = yaml.safe_load(f) 
        
                       import_paths = params.get('pulp_import_paths', []) 
        
                       export_paths = params.get('pulp_export_paths', []) 
        
               return import_paths, export_paths

That means the parameters.yaml file doesn't exist.

ekohl · 2026-07-31T21:23:01Z

That means the parameters.yaml file doesn't exist.

Ah, I moved the file around and it tries to derive the root from the file location.

ekohl · 2026-07-31T21:28:54Z

 import pytest
 import yaml

+from tests.conftest import PARAMETERS_FILE


I wasn't sure about this, but I see tests/feature/foreman-proxy/base_test.py also imports FOREMAN_PROXY_PORT.

Then there's also this:

foremanctl/tests/migration_test.py

Lines 7 to 9 in 9ffe360

@pytest.fixture(scope="module")

def obsah_state_path():

return os.environ.get("OBSAH_STATE", "/var/lib/foremanctl")

Should it be a global fixture?

I think using

foremanctl/tests/conftest.py

Line 20 in 9ffe360

PARAMETERS_FILE = os.path.join(OBSAH_STATE, 'parameters.yaml')

from confest make sense to me, thoughts?

Are you suggesting to also update the migration test to use the constant?

yes, looking at its used just to read params from the file , i think having a single defination for obsah state dir makes much more sense to me

So question: should be have a global fixture or a constant? I'm leaning to avoid importing constants and instead rely on fixtures, but curious what you think.

I think those both ways are valid, but personally for me i would choose constant as the state path is going to remain static throughout the test execution and does not need the overhead of using fixtures, but i might be wrong, thats my thesis

I went with reusing the existing obsah_params fixture in #719 instead.

ekohl · 2026-08-01T13:37:27Z

I think this is largely ready, but we need to rethink CI a bit here. Adding IOP to plain Foreman implies adding Katello to it. While that's nice and all, it means we don't really test just Foreman. The additional proxy tests also all just fail.

arvind4501 · 2026-08-03T09:15:49Z

    assert "Index of /pulp/content/" in cmd.stdout


+@pytest.mark.feature('pulp')


I am sure that ./foremanctl features will not show pulp as feature directly, its a internal feature, which can be found as has_feature filter but not as foremanctl features output, so these tests will be skipped always

So we're finally hitting the point we discussed at the start when we talked about feature-based tsets: the test suite needs to know about internal features. Good catch.

Yes, thats what forced me to use flavor directories for such cases https://github.com/theforeman/foremanctl/blob/master/tests/flavor/foreman-proxy-content/pulp_test.py ,
but yeah we can have a forge command to list internal features too

At least the test suite is correctly flagging that:

ERROR tests/httpd_test.py::test_http_pulp_api_status - pytest.PytestConfigWarning: Invalid feature(s) {'pulp'} on <Function test_http_pulp_api_status> ERROR tests/httpd_test.py::test_https_pulp_api_status - pytest.PytestConfigWarning: Invalid feature(s) {'pulp'} on <Function test_https_pulp_api_status> ERROR tests/httpd_test.py::test_http_pulp_content - pytest.PytestConfigWarning: Invalid feature(s) {'pulp'} on <Function test_http_pulp_content> ERROR tests/httpd_test.py::test_https_pulp_content - pytest.PytestConfigWarning: Invalid feature(s) {'pulp'} on <Function test_https_pulp_content> ERROR tests/httpd_test.py::test_https_pulp_auth - pytest.PytestConfigWarning: Invalid feature(s) {'pulp'} on <Function test_https_pulp_auth> ERROR tests/httpd_test.py::test_https_pypi_endpoint - pytest.PytestConfigWarning: Invalid feature(s) {'pulp'} on <Function test_https_pypi_endpoint>

ekohl · 2026-08-03T09:51:59Z

+        - "enabled_features | has_feature('foreman')"
+        - "enabled_features | has_feature('katello')"


I missed there's now a built in candlepin feature that should be used.

Suggested change

- "enabled_features | has_feature('foreman')"

- "enabled_features | has_feature('katello')"

- "enabled_features | has_feature('candlepin')"

ekohl · 2026-08-03T16:33:11Z

I've decided to split up some parts. #719 is already merged and I just opened #722. Once that's merged, this and #527 should be much smaller. Then we can focus on how to actually test those flavors.

ekohl · 2026-08-25T09:12:10Z

Right now all the prerequisites I could think of are merged. This now needs testing; both manual and CI.

This moves Candlepin and Pulp behind the feature flags. Candlepin is only deployed on Foreman with Katello servers while Pulp is deployed on Katello enabled servers.

pr-processor Bot added the Not yet reviewed label May 15, 2026

arvind4501 mentioned this pull request May 17, 2026

Expose user enabled features as pytest markers #508

Merged

2 tasks

ekohl mentioned this pull request May 20, 2026

Introduce feature-based test directory structure #509

Merged

2 tasks

ekohl force-pushed the plain-foreman-flavor branch from 48a7446 to 6aad15c Compare May 28, 2026 10:12

ekohl mentioned this pull request May 28, 2026

[INCOMPLETE] Introduce plain foreman-proxy flavor #527

Draft

2 tasks

arvind4501 reviewed Jun 2, 2026

View reviewed changes

Comment thread tests/postgresql_test.py Outdated

pr-processor Bot removed the Not yet reviewed label Jun 2, 2026

ekohl mentioned this pull request Jun 5, 2026

Fixes #525 - Add checks to health subcommand #521

Merged

2 tasks

ekohl force-pushed the plain-foreman-flavor branch 3 times, most recently from a0a4d57 to 2c3a344 Compare July 9, 2026 19:14

This was referenced Jul 14, 2026

Mark slow tests to allow for quicker iteration #655

Merged

Only test for azure-rm and google features when enabled #656

Merged

Mark tests with the appropriate feature #657

Merged

ekohl force-pushed the plain-foreman-flavor branch from 2c3a344 to ddb679e Compare July 16, 2026 17:11

ekohl force-pushed the plain-foreman-flavor branch from ddb679e to 905eb13 Compare July 31, 2026 21:27

ekohl commented Jul 31, 2026

View reviewed changes

ekohl force-pushed the plain-foreman-flavor branch 2 times, most recently from b669faa to c2a38c5 Compare August 1, 2026 12:09

arvind4501 reviewed Aug 3, 2026

View reviewed changes

Comment thread src/playbooks/deploy-proxy/deploy-proxy.yaml Outdated

ekohl force-pushed the plain-foreman-flavor branch from c2a38c5 to 0bc4e69 Compare August 3, 2026 09:06

arvind4501 reviewed Aug 3, 2026

View reviewed changes

ekohl commented Aug 3, 2026

View reviewed changes

This was referenced Aug 3, 2026

Reuse obsah_params fixture to read parameters #719

Merged

Guard all deployment roles behind features where possible #722

Merged

ekohl force-pushed the plain-foreman-flavor branch from 0bc4e69 to 06b8f53 Compare August 22, 2026 20:17

ekohl mentioned this pull request Aug 25, 2026

Include ext DB migration for containerized theforeman/foreman-documentation#5113

Open

2 tasks

Introduce a plain Foreman flavor

b596aef

This moves Candlepin and Pulp behind the feature flags. Candlepin is only deployed on Foreman with Katello servers while Pulp is deployed on Katello enabled servers.

ekohl force-pushed the plain-foreman-flavor branch from 06b8f53 to b596aef Compare August 27, 2026 09:40

	@pytest.fixture(scope="module")
	def obsah_state_path():
	return os.environ.get("OBSAH_STATE", "/var/lib/foremanctl")

		assert "Index of /pulp/content/" in cmd.stdout


		@pytest.mark.feature('pulp')

		- "enabled_features \| has_feature('foreman')"
		- "enabled_features \| has_feature('katello')"

	- "enabled_features \| has_feature('foreman')"
	- "enabled_features \| has_feature('katello')"
	- "enabled_features \| has_feature('candlepin')"

Uh oh!

Conversation

ekohl commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are you introducing these changes? (Problem description, related links)

What are the changes introduced in this pull request?

How to test this pull request

Checklist

Uh oh!

Uh oh!

arvind4501 commented Jun 2, 2026

Uh oh!

ekohl commented Jun 2, 2026

Uh oh!

arvind4501 commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ekohl commented Jul 14, 2026

Uh oh!

ekohl commented Jul 17, 2026

Uh oh!

ekohl commented Jul 31, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ekohl commented Aug 1, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ekohl Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ekohl commented Aug 3, 2026

Uh oh!

ekohl commented Aug 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ekohl commented May 15, 2026 •

edited

Loading

arvind4501 commented Jun 2, 2026 •

edited

Loading

ekohl Aug 3, 2026 •

edited

Loading