You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
playbooks/sync-splunkbase.yml (added in #125) is a ~250-line pure-Ansible
playbook that authenticates to Splunkbase via an XML session-token flow,
queries existing MinIO tags per object, downloads drifted archives, uploads
them via mc cp, and tags via mc tag set. It works, it's idempotent, it
fails loud on real errors — but the implementation is long, fragile in a few
places, and relies on three separate command: mc … shell-outs per entry.
Goal
Replace the hand-rolled sync with something more native. Candidates:
amazon.aws.s3_object (or community.aws.s3_object) with a custom endpoint_url pointing at MinIO. Supports tags: in a single module
call, replacing both mc cp and mc tag set. Adds a boto3 dependency
on the controller but removes the MC_HOST_homelab env-var trick, the mc tag list --json parsing, and the NoSuchKey stdout-filter
workaround. Likely the biggest structural simplification available.
Splunk's own docker-splunk app installer (SPLUNK_APPS_URL env
var on the container). The official Splunk Docker image can accept a
comma-separated list of Splunkbase URLs at startup and install them
itself. If this works for all our apps, the entire sync layer could
be deleted — Splunk pulls directly from Splunkbase on container start,
no MinIO mirror needed for version rotation. Would require rethinking
the role to pass SPLUNK_APPS_URL through docker-compose, and
credentials would need to live in the container environment.
Splunkbase JSON login endpoint instead of the XML api/account:login flow. If /api/v1/accounts/login returns JSON, the
regex XML parse (regex_search('<id>([^<]+)</id>')) goes away.
A dedicated community collection if one exists — check
ansible-galaxy for anything like splunk.splunkbase or community.splunkbase. Unlikely, but worth a search.
Acceptance criteria
Resulting playbook is materially shorter AND cleaner (target: under 100
lines, or deleted entirely if option 2 works)
Same functionality: pinned-version sync from Splunkbase to the install
target, idempotent re-runs, fails loud on auth/network errors
No new shell scripts. Pure-Ansible modules where possible
Version metadata still visible per-app (either MinIO tags OR Splunkbase
URL in the container env)
End-to-end verification against live cluster before merge
Context
playbooks/sync-splunkbase.yml(added in #125) is a ~250-line pure-Ansibleplaybook that authenticates to Splunkbase via an XML session-token flow,
queries existing MinIO tags per object, downloads drifted archives, uploads
them via
mc cp, and tags viamc tag set. It works, it's idempotent, itfails loud on real errors — but the implementation is long, fragile in a few
places, and relies on three separate
command: mc …shell-outs per entry.Goal
Replace the hand-rolled sync with something more native. Candidates:
amazon.aws.s3_object(orcommunity.aws.s3_object) with a customendpoint_urlpointing at MinIO. Supportstags:in a single modulecall, replacing both
mc cpandmc tag set. Adds aboto3dependencyon the controller but removes the
MC_HOST_homelabenv-var trick, themc tag list --jsonparsing, and theNoSuchKeystdout-filterworkaround. Likely the biggest structural simplification available.
Splunk's own
docker-splunkapp installer (SPLUNK_APPS_URLenvvar on the container). The official Splunk Docker image can accept a
comma-separated list of Splunkbase URLs at startup and install them
itself. If this works for all our apps, the entire sync layer could
be deleted — Splunk pulls directly from Splunkbase on container start,
no MinIO mirror needed for version rotation. Would require rethinking
the role to pass
SPLUNK_APPS_URLthrough docker-compose, andcredentials would need to live in the container environment.
Splunkbase JSON login endpoint instead of the XML
api/account:loginflow. If/api/v1/accounts/loginreturns JSON, theregex XML parse (
regex_search('<id>([^<]+)</id>')) goes away.A dedicated community collection if one exists — check
ansible-galaxy for anything like
splunk.splunkbaseorcommunity.splunkbase. Unlikely, but worth a search.Acceptance criteria
lines, or deleted entirely if option 2 works)
target, idempotent re-runs, fails loud on auth/network errors
URL in the container env)
Non-goals
home-LAN reach)
References
playbooks/sync-splunkbase.yml— current implementationroles/splunk_docker/vars/addons.yml— registry consumed by the syncsplunk/docker-splunk—SPLUNK_APPS_URLenvironment variable docsamazon.aws.s3_object— Ansible module supporting custom S3 endpoints🤖 Generated with Claude Code