chore: remove proprietary crypto code, dead code, and ~79MB of orphaned fixtures - #1888
Merged
Merged
Conversation
Three files under datasources/connection/encrypt/ were not Saiku code:
CryptoUtil.java "Copyright (C) 1994-2006 ... Unauthorized
SecureDataSourceFactory.java Duplication Prohibited / Program Property
of Embarcadero Technologies, Inc."
Des.java no header at all - a 1,159-line hand-rolled
DES/3DES implementation, same provenance
The repo is Apache-2.0 with no NOTICE file, and that header explicitly
forbids duplication. This shipped in the Docker image and the launcher
fat-JAR.
Des.java turns out to be a plain two-key 3DES (EDE, K1-K2-K1) in ECB, so
javax.crypto's DESede reproduces it exactly - verified byte-for-byte
against the original implementation before the swap, including the
sample ciphertext that lived in CryptoUtil.main. CryptoUtil's legacy
encrypt/decrypt now call the JDK; the wire format is unchanged, so every
pre-v2 .sds file still loads and still upgrades to v2 on next save.
Pinned by new golden-vector tests. The pre-existing legacy test only
round-tripped legacyEncrypt through decrypt, which would have stayed
green even if both sides changed algorithm together; the new tests assert
hard-coded historical ciphertext, so a future change to the legacy format
fails loudly instead of silently locking users out of their passwords.
Also removed while here:
- SecureDataSourceFactory - zero references, and package-private so
JNDI could never have reached it
- TripleDesPasswordEncoder - zero references
- CryptoUtil.isPasswordValid/encodePassword - unused, and a third copy
of the block-padding logic
- CryptoUtil.main - debug harness that printed a decrypted password
Added a NOTICE file covering the Pentaho EPL header on the Mondrian
SqlStatement override, which had no attribution anywhere.
Net: -1,459 lines of third-party code, legacy decryption provably intact.
Everything here has zero references. Verified by simple name AND fully
qualified name across .java/.xml/.properties/.json/.ts/.svelte, so Spring
XML wiring and reflection lookups are covered, then re-checked against
the tree immediately before deletion.
Java (24 classes, ~1,100 lines):
- olap/query2/filter/Thin{Generic,N,Name}Filter - vestigial. ThinFilter
is a flat class with a FilterFlavour enum; the subclass design was
abandoned and there is no @JsonSubTypes keeping them reachable.
- SaikuDefaultXmlaServlet - web.xml wires SaikuXmlaServlet, which
extends Olap4jXmlaServlet, not this.
- mondrian/olap4j/Checker (a one-method class from 2015) and
LevelInterface (never implemented).
- auth leftovers: CasAuthenticationProvider, OrbisLogoutHandler,
RoleBasedAuthorisation, FileBackedPasswordProvider,
RoleDatasourceProcessor.
- REST DTOs never constructed: ConnectionType, SelectionListRestObject,
UserList.
- SimpleConnectionManager, SaikuAuditHelper, EvalReportWriter,
ObjectHolder, OlapUtil, SaikuObjectComparator, TqUtil,
Parameterizable, SimpleMember.
UI:
- 5 orphaned modals (DataSourcesModal, DeleteRepositoryObject,
OpenDialogModal, OverwriteModal, ParentMemberSelectorModal) - no
static or dynamic imports.
- cube-designer/bundle.ts - not exported from the package barrel and
not imported anywhere.
- api/repository.ts listSavedQueries and api/admin.ts getVersion.
flatten() and SavedQueryFile survive - both are used elsewhere.
Deliberately NOT removed, despite looking unreferenced:
- mondrian/rolap/SqlStatement.java shadows Mondrian's own class by
classpath precedence and IS live.
- mondrian/olap4j/package-info.java documents a live package.
- NoReHashPasswordEncoder is reachable only from its own test, but is
public API a deployment could wire via Spring XML - left for a human
call.
226 of 923 Java files carried no licence header at all - almost entirely first-party org.saiku.* code, and concentrated in the newer subsystems (service/mail, repository, web/email, schema/generate, olap/query2), so this is drift rather than legacy. Nothing enforced it: license-maven-plugin and apache-rat-plugin are declared in pluginManagement but bound to no phase, and Spotless had no <licenseHeader>. Stamped the standard block on the files that had none. Files that already carried a notice were left byte-for-byte untouched. Enforcement is a small presence-check script rather than Spotless's <licenseHeader>. That was the obvious choice and it is the wrong one: Spotless REPLACES whatever sits above the `package` line, so it would have rewritten every "Copyright OSBI Ltd" header, Paul Stoellberger's attribution on CubeDesignerResource, and the Pentaho EPL header on the Mondrian SqlStatement override into "Copyright 2026 Spicule Ltd" - stripping other people's copyright notices. Spotless has no escape hatch for this: its LicenseHeader exposes `delimiter` and `skipLinesMatching` but no `contentPattern`, so it cannot express "add if missing, preserve otherwise". Verified absent on BOTH 3.8.0 and 3.9.0 (the dependabot bump this branch rebased onto), so please don't reintroduce <licenseHeader> on a future version bump without checking that parameter actually landed. scripts/check-licence-headers.sh checks for the PRESENCE of a copyright/licence line, which every legitimate header satisfies. Wired into CI ahead of `mvn verify` and into the pre-commit hook alongside spotless:check. Takes --fix to stamp anything missing.
util/ was 99MB, of which only two files are live build inputs (the
antrun block in saiku-core/saiku-service/pom.xml uses FoodMart.xml and
foodmart_hsql.script.zip). Removed:
foodmart_hsql.script 70MB - the build unzips the .zip into
target/test/ and copies from THERE, so
the uncompressed copy was never read
foodmart.zip 9.4MB - zero references
foodmart_h2.sql.zip 8.2MB - byte-identical duplicate (md5
5924b314059024324ce5b3f9c5bf2bb3) of
saiku-launcher/src/main/resources/seed/,
which is the copy the launcher loads
earthquakes.sql.zip 2.0MB \ the earthquakes demo bundle, one
Earthquakes.xml | coherent unused set
sample_reports/ (6) /
FoodMart4.xml - legacy Pentaho-era sample; the runtime
schema is the launcher seed
foodmart_annotated.xml, saiku_initscript (SysV init, superseded by the
launcher and Docker), foodmart (legacy datasource descriptor the build
never copies)
util/configuration.xml is KEPT: it is the only copy of the Jackrabbit
repo config in the tree, and saiku-beans.xml wires configurationpath at
${saiku.home}/repository/configuration.xml, so it is the template a
JCR-backed deployment needs.
Stale directories, no references anywhere:
saiku-legal/ one .doc, untouched since 2011
snap/ snapcraft prototype, untouched since 2017
exporter/ one shell script, untouched since 2015
Root-level PNGs (5 tracked, ~1MB) referenced by nothing. .gitignore now
ignores /*.png generally - root-level PNGs are always scratch, and the
pattern deliberately does not match docs/images/, which stays tracked.
Also:
- saiku-ui: dompurify was imported by production TextTile.svelte but
absent from package.json, working only via transitive hoisting - it
would break on a lockfile change. Declared at ^3.4.0 to match the
existing security override (npm rejects mismatched ranges).
- saiku-ui: added the license field.
- CLAUDE.md: dropped the note steering readers away from
util/FoodMart4.xml, which no longer exists.
Note: history still carries the blobs, so this shrinks fresh clones
going forward rather than .git itself.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Audit of dead code and missing/incorrect licence headers across the repo. Four independent commits — the first is the one that actually matters.
1. Proprietary code in an Apache-2.0 tree (
fix:)Three files under
datasources/connection/encrypt/were not Saiku code:CryptoUtil.javaCopyright (C) 1994-2006 … Unauthorized Duplication Prohibited / Program Property of Embarcadero Technologies, Inc.SecureDataSourceFactory.javaDes.javaThe repo is Apache-2.0, had no
NOTICE, and that header explicitly forbids duplication. It shipped in the Docker image and the launcher fat-JAR.Des.javacould not just be deleted —CryptoUtil.decrypt()still reads pre-v2.sdspasswords, so removing it would lock existing installs out of their own datasources.It turned out to be plain two-key 3DES (EDE, K1-K2-K1, ECB), so
javax.crypto'sDESedereproduces it exactly. Verified byte-for-byte against the original implementation before the swap, including the sample ciphertext that was sitting inCryptoUtil.main. Wire format unchanged; legacy files still load and still upgrade to v2 on next save.Pinned by new golden-vector tests. The pre-existing legacy test only round-tripped
legacyEncrypt→decrypt, which would have stayed green even if both sides changed algorithm together; the new tests assert hard-coded historical ciphertext.Also added a
NOTICEcovering the Pentaho EPL header on the MondrianSqlStatementoverride, which had no attribution anywhere.Net −1,459 lines of third-party code.
2. Dead code (
refactor:)24 Java classes, 5 Svelte modals, 3 exports (~1,500 lines). All verified to have zero references by simple name and FQN across
.java/.xml/.properties/.json/.ts/.svelte, so Spring XML wiring and reflection lookups are covered.Deliberately not removed despite looking unreferenced:
mondrian/rolap/SqlStatement.java(live via classpath shadowing),mondrian/olap4j/package-info.java, andNoReHashPasswordEncoder(test-only but public API a deployment could wire via Spring XML — left for a human call).3. Licence headers (
chore:)226 of 923 Java files had no header. Now 923/923.
<licenseHeader>, and that's deliberate. Spotless replaces everything above thepackageline — it would have rewritten every OSBI Ltd notice, Paul Stoellberger's attribution, and the Pentaho EPL header into "Copyright 2026 Spicule Ltd", stripping other people's copyright. Spotless has nocontentPattern(verified absent on both 3.8.0 and 3.9.0), so it cannot express "add if missing, preserve otherwise".scripts/check-licence-headers.shdoes a presence check instead, wired into CI and the pre-commit hook.4. Repo hygiene (
chore:)util/99MB → 9.4MB. The big ones:foodmart_hsql.script(70MB — the build reads the.zip, not this),foodmart.zip(9.4MB, unreferenced),foodmart_h2.sql.zip(8.2MB, byte-identical duplicate of the launcher seed). Plussaiku-legal/(2011),snap/(2017),exporter/(2015), and 5 root PNGs.util/configuration.xmlkept — it's the only copy of the Jackrabbit templatesaiku-beans.xmlpoints at.Also:
dompurifywas imported by productionTextTile.sveltebut missing frompackage.json, working only via transitive hoisting.Test plan
mvn -B -ntp -DskipITs=false verify— BUILD SUCCESS, all 11 modules (JDK 22)DatasourcePasswordEncryptionTest— 11/11, golden vectors green against the original implementation and after the swapCryptoUtilPropertyTest(saiku-proptest) — 3/3npm run check— 0 errorsnpm test— 152 files, 2175 tests passed; lint 0 errorsprettier --checkclean against the bumped prettier-plugin-svelte 4check-licence-headers.sh— 923/923, and verified it genuinely fails when a header is removed.sdsfrom a live installRebased onto
developmentafter the dependabot batch landed.