feat: implement extra_settings_files by kurokobo · Pull Request #125 · ansible/galaxy-operator

kurokobo · 2024-05-24T13:33:58Z

SUMMARY

This PR implements extra_settings_files, which is almost the same as my PR for AWX Operator ansible/awx-operator#1836

Required module (split_settings) to load additional *.py files is already installed in galaxy images: https://github.com/ansible/galaxy_ng/blob/c52e9b4e5548ca38946d7cc1bd8624459def1ec1/requirements/requirements.common.txt#L127-L128

Design:

Users can add extra settings by mounting *.py files from ConfigMaps or Secrets
This feature mounts specified *.py files under /etc/pulp/conf.d in following pods which already have settngs.py
- galaxy-api
- galaxy-content
- galaxy-worker
Users can specify multiple ConfigMaps and Secrets
Users can include multiple files in a single ConfigMap or Secret

Restrictions:

Completely the same as described in ansible/awx-operator#1836

ADDITIONAL INFORMATION

Tested with and without extra_settings_files:

Without extra_settings_files

---
apiVersion: galaxy.ansible.com/v1beta1
kind: Galaxy
metadata:
  namespace: galaxy
  name: galaxy
spec:
  ingress_type: nodeport
  nodeport_port: "30180"
  storage_type: File
  file_storage_access_mode: ReadWriteOnce
  file_storage_size: 8Gi
  no_log: false

  image: quay.io/ansible/galaxy-ng
  image_version: 9d2f8ce1
  image_web: quay.io/ansible/galaxy-ui
  image_web_version: 6116e760

Ensure there is no /etc/pulp/conf.d

$ kubectl -n galaxy exec -it deployment/galaxy-api -- ls -l /etc/pulp
total 4
drwxrwxr-x. 2 galaxy root    6 May  1 13:01 certs
drwxrwxr-x. 1 galaxy root  118 May 24 12:53 keys
-rw-r--r--. 1 root   root 1531 May 24 12:52 settings.py

Even if there is no actual directory and files, include() does not cause error since optional() is used to load files.

$ kubectl -n galaxy exec -it deployment/galaxy-api -- cat /etc/pulp/settings.py
import os
import traceback

from split_settings.tools import optional, include

GALAXY_API_PATH_PREFIX = "api/galaxy"
...
CSRF_TRUSTED_ORIGINS = ["http://192.168.0.221:30180", "https://192.168.0.221:30180"]

# Attempt to load settings from /etc/pulp/conf.d/*.py.
settings_dir = os.environ.get('PULP_SETTINGS_DIR', '/etc/pulp/conf.d/')
settings_files = os.path.join(settings_dir, '*.py')
try:
    include(optional(settings_files), scope=locals())
except ImportError:
    traceback.print_exc()
    sys.exit(1)

With extra_settings_files

---
apiVersion: galaxy.ansible.com/v1beta1
kind: Galaxy
metadata:
  namespace: galaxy
  name: galaxy
spec:
  ingress_type: nodeport
  nodeport_port: "30180"
  storage_type: File
  file_storage_access_mode: ReadWriteOnce
  file_storage_size: 8Gi
  no_log: false

  image: quay.io/ansible/galaxy-ng
  image_version: 9d2f8ce1
  image_web: quay.io/ansible/galaxy-ui
  image_web_version: 6116e760

  extra_settings_files:
    configmaps:
      - name: demo-configmap-01
        key: democonfigmap01_01.py
    secrets:
      - name: demo-secret-01
        key: demosecret01_01.py
      - name: demo-secret-01
        key: demosecret01_02.py
      - name: demo-secret-02
        key: demosecret02_01.py

---
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: galaxy
  name: demo-configmap-01
data:
  democonfigmap01_01.py: |
    GALAXY_FEATURE_FLAGS = {
      "display_repositories": True,
      # True by default so set False for testing
      "execution_environments": False,
      "legacy_roles": False,
      "ai_deny_index": False,
      "dab_resource_registry": False,
    }
---
apiVersion: v1
kind: Secret
metadata:
  namespace: galaxy
  name: demo-secret-01
stringData:
  demosecret01_01.py: |
    # False by default
    GALAXY_ENABLE_UNAUTHENTICATED_COLLECTION_ACCESS = True
  demosecret01_02.py: |
    # False by default
    GALAXY_ENABLE_UNAUTHENTICATED_COLLECTION_DOWNLOAD = True
---
apiVersion: v1
kind: Secret
metadata:
  namespace: galaxy
  name: demo-secret-02
stringData:
  demosecret02_01.py: |
    GALAXY_COLLECTION_SIGNING_SERVICE = None
    GALAXY_CONTAINER_SIGNING_SERVICE = None
    TOKEN_AUTH_DISABLED = True

Result:

$ kubectl -n galaxy exec -it deployment/galaxy-api -- ls -l /etc/pulp/conf.d
total 16
-rw-r--r--. 1 root root 224 May 24 12:59 democonfigmap01_01.py
-rw-r--r--. 1 root root  74 May 24 12:59 demosecret01_01.py
-rw-r--r--. 1 root root  76 May 24 12:59 demosecret01_02.py
-rw-r--r--. 1 root root 108 May 24 13:06 demosecret02_01.py

$ kubectl -n galaxy exec -it deployment/galaxy-content -- ls -l /etc/pulp/conf.d
total 16
-rw-r--r--. 1 root 1000 224 May 24 12:59 democonfigmap01_01.py
-rw-r--r--. 1 root 1000  74 May 24 12:59 demosecret01_01.py
-rw-r--r--. 1 root 1000  76 May 24 12:59 demosecret01_02.py
-rw-r--r--. 1 root 1000 108 May 24 13:05 demosecret02_01.py

$ kubectl -n galaxy exec -it deployment/galaxy-worker -- ls -l /etc/pulp/conf.d
total 16
-rw-r--r--. 1 root root 224 May 24 12:59 democonfigmap01_01.py
-rw-r--r--. 1 root root  74 May 24 12:59 demosecret01_01.py
-rw-r--r--. 1 root root  76 May 24 12:59 demosecret01_02.py
-rw-r--r--. 1 root root 108 May 24 13:06 demosecret02_01.py

$ kubectl -n galaxy exec -it deployment/galaxy-api -- grep -r "" /etc/pulp/conf.d 
/etc/pulp/conf.d/demosecret02_01.py:GALAXY_COLLECTION_SIGNING_SERVICE = None
/etc/pulp/conf.d/demosecret02_01.py:GALAXY_CONTAINER_SIGNING_SERVICE = None
/etc/pulp/conf.d/demosecret02_01.py:TOKEN_AUTH_DISABLED = True
/etc/pulp/conf.d/demosecret01_02.py:# False by default
/etc/pulp/conf.d/demosecret01_02.py:GALAXY_ENABLE_UNAUTHENTICATED_COLLECTION_DOWNLOAD = True
/etc/pulp/conf.d/demosecret01_01.py:# False by default
/etc/pulp/conf.d/demosecret01_01.py:GALAXY_ENABLE_UNAUTHENTICATED_COLLECTION_ACCESS = True
/etc/pulp/conf.d/democonfigmap01_01.py:GALAXY_FEATURE_FLAGS = {
/etc/pulp/conf.d/democonfigmap01_01.py:  "display_repositories": True,
/etc/pulp/conf.d/democonfigmap01_01.py:  # True by default so set False for testing
/etc/pulp/conf.d/democonfigmap01_01.py:  "execution_environments": False,
/etc/pulp/conf.d/democonfigmap01_01.py:  "legacy_roles": False,
/etc/pulp/conf.d/democonfigmap01_01.py:  "ai_deny_index": False,
/etc/pulp/conf.d/democonfigmap01_01.py:  "dab_resource_registry": False,
/etc/pulp/conf.d/democonfigmap01_01.py:}

$ kubectl -n galaxy exec -it deployment/galaxy-api -- cat /etc/pulp/settings.py
import os
import traceback

from split_settings.tools import optional, include

GALAXY_API_PATH_PREFIX = "api/galaxy"
CACHE_ENABLED = True
DB_ENCRYPTION_KEY = "/etc/pulp/keys/database_fields.symmetric.key"
GALAXY_COLLECTION_SIGNING_SERVICE = "ansible-default"
GALAXY_CONTAINER_SIGNING_SERVICE = "container-default"
ANSIBLE_CERTS_DIR = "/etc/pulp/keys/"
DATABASES = {"default": {"HOST": "galaxy-postgres-15", "ENGINE": "django.db.backends.postgresql_psycopg2", "NAME": "galaxy", "USER": "galaxy", "PASSWORD": "ZhMCgM3XJfyTLyjhMaf7hrkGjmi2NHWw", "PORT": "5432", "CONN_MAX_AGE": 0, "OPTIONS": {"sslmode": "prefer"}}}
STATIC_ROOT = "/app/galaxy_ng/app/static/"
REDIS_HOST = "galaxy-redis-svc"
REDIS_PORT = 6379
REDIS_PASSWORD = ""
CONTENT_ORIGIN = "http://192.168.0.221:30180"
ANSIBLE_API_HOSTNAME = "http://192.168.0.221:30180"
X_PULP_CONTENT_HOST = "galaxy-content-svc"
X_PULP_CONTENT_PORT = "24816"
TOKEN_SERVER = "http://192.168.0.221:30180/token/"
TOKEN_AUTH_DISABLED = False
TOKEN_SIGNATURE_ALGORITHM = "ES256"
PUBLIC_KEY_PATH = "/etc/pulp/keys/container_auth_public_key.pem"
PRIVATE_KEY_PATH = "/etc/pulp/keys/container_auth_private_key.pem"
CSRF_TRUSTED_ORIGINS = ["http://192.168.0.221:30180", "https://192.168.0.221:30180"]

# Attempt to load settings from /etc/pulp/conf.d/*.py.
settings_dir = os.environ.get('PULP_SETTINGS_DIR', '/etc/pulp/conf.d/')
settings_files = os.path.join(settings_dir, '*.py')
try:
    include(optional(settings_files), scope=locals())
except ImportError:
    traceback.print_exc()
    sys.exit(1)

I can confirm that additional settings in *.py take effects, e.g. browsing existing collections without logging in to the Galaxy by GALAXY_ENABLE_UNAUTHENTICATED_COLLECTION_ACCESS = True.

kurokobo · 2024-05-24T13:34:10Z

TODO:

Documentation

Docs previews:

kurokobo · 2024-05-24T13:37:14Z

@rooftopcellist @Denney-tech
F.Y.I.

kurokobo · 2024-05-24T14:58:31Z

@rooftopcellist @Denney-tech
Marked as ready for review. I always welcome any feedback you may have, thanks!

Denney-tech

Looks good to me, I don't know if I can "approve" but I will try.

Edit: Apparently "Approving" in a review is not the same as approving a workflow lol. TIL

kurokobo · 2024-05-27T12:52:03Z

Updated, ready for review again😃

rooftopcellist · 2024-06-06T01:08:45Z

o/ @kurokobo @Denney-tech Thank you for the PR and for testing it. This is a great addition!

Sorry for the delay here, it has been a busy couple weeks.. I have this on my list to review. CI has been triggered.

Denney-tech · 2024-06-14T20:00:58Z

LGTM

sonarqubecloud · 2024-07-25T16:18:42Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

Denney-tech · 2024-10-02T20:17:03Z

Any chance we can see this get merged soon? Not sure if it's too stale to merge as-is, but I am disappointed it's been sitting here so long.

feat: implement extra_settings_files

857463e

kurokobo added 2 commits May 24, 2024 23:52

docs: update docs for extra_settings_files

e3b2df6

docs: publish docs for db configuration

5a0d68b

kurokobo marked this pull request as ready for review May 24, 2024 14:56

Denney-tech approved these changes May 24, 2024

View reviewed changes

kurokobo commented May 26, 2024

View reviewed changes

Comment thread docs/user-guide/advanced-configuration/extra-settings.md Outdated

docs: remove incorrect restrictions

6bd9d66

Navigation Menu

Uh oh!

feat: implement extra_settings_files - #125

feat: implement extra_settings_files#125
kurokobo wants to merge 4 commits into
ansible:mainfrom
kurokobo:extra_settings_files

kurokobo commented May 24, 2024

Uh oh!

kurokobo commented May 24, 2024 •

edited

Loading

Uh oh!

kurokobo commented May 24, 2024

Uh oh!

kurokobo commented May 24, 2024

Uh oh!

Denney-tech left a comment •

edited

Loading

Uh oh!

Uh oh!

kurokobo commented May 27, 2024

Uh oh!

rooftopcellist commented Jun 6, 2024

Uh oh!

Denney-tech commented Jun 14, 2024

Uh oh!

sonarqubecloud Bot commented Jul 25, 2024

Uh oh!

Denney-tech commented Oct 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

kurokobo commented May 24, 2024

SUMMARY

ADDITIONAL INFORMATION

Uh oh!

kurokobo commented May 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kurokobo commented May 24, 2024

Uh oh!

kurokobo commented May 24, 2024

Uh oh!

Denney-tech left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kurokobo commented May 27, 2024

Uh oh!

rooftopcellist commented Jun 6, 2024

Uh oh!

Denney-tech commented Jun 14, 2024

Uh oh!

sonarqubecloud Bot commented Jul 25, 2024

Quality Gate passed

Uh oh!

Denney-tech commented Oct 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kurokobo commented May 24, 2024 •

edited

Loading

Denney-tech left a comment •

edited

Loading