major refactoring inc.

[RicYaben]

I won't list all the changes, and I won't fix the documentation.
This is a major overhaul of the app on top of the GSoC 2025 branch (the latest changes).
I am sending the changes to v3.0 since there is no current branch tracking versioning right now.

Will finish it whenever I can put some time into it. It should not take more than a couple of weeks.

ISSUES

Everything is combined into a very unusual monorepo. Dicomhawk contains (many) artifacts directly copied from its own dependencies, the requirements file is not complete (there is no way to install the app), Dockerfiles do weird loops to load each resource (who knows why), the app loads directly in memory API keys (ugh), and a million issues. Namely:

• logging misses half of the incoming requests (see the difference between a pcap file and the logs stored)
• Some requests are outright aborted without any reason or vibe-coded errors (nowhere to be found in the DICOM standard)
• The app claims to be an ORTHANC server, yet, not even the AE title is set, and even when set, it tells is sourcing from pynetdicom (ORTHANC uses a different UID and version names)
• File uploads go directly into a directory with a timestamp, never compressed or quarantined (we expect malware)
• There are more than 50 environment variable configuration options. None of them is documented. Some of them are very sensitive (e.g., API tokens). 30 of them do not even have anything to do with the honeypot; they are external services or apps that could easily be plugins, components, or stand-alone services (i.e., TCIA file rotation, Redis event consumer, OSM service, Blackhole (?), and the Threat intelligence module).
• For some reason, the TCIA manager is responsible for injecting the honey-tokens into the DICOM datasets. Not sure why this is the case.

This PR will be a major refactor of the honeypot.

SCOPE

• dicomhawk, the server.

TODO

• Add connection middlewares (sinkhole)
• Add event consumers (redis)
• Add Monitor (logger for the app, digest events)
• Add plugins (TCIA file rotation, OSM institutions)
• Add safe loading for API keys
• Add commands to pre-fetch files instead of loading them into the app
• Split the repo, not sure why everything is smashed together
• Added support for middlewares to inject (honeytokens) data into outgoing datasets
• Added command support. Dicomhawk can now be loaded as a regular CLI app
• Added space for quarantined and safe DICOM files. Quarantined are those uploaded by clients
• Support config Implementation UID, version, and AE titles.
• Set limit to max assocs. Can be changed as part of the config. [Simple Logger Stops working after too many association requests #86 ]
• Implemented local storage for traces uploaded with C-STORE functions
• Overhaul of settings
  • Load as params
  • Load as env

[anshikasharmaa1517]

Hey @RicYaben, I have already started working on the installation part as well as the documentation. Will make a pr soon.

[RicYaben]

@anshikasharmaa1517 can you halt for now? I am refactoring the whole thing

[RicYaben]

this work continues in #v3.0-dev, will merge with v3.0 when that one is ready

[kaffie-1517]

Hey @RicYaben ,

I was going through the DICOMHawk codebase and setting it up locally to better understand the current workflow and the architecture.

From what I can see, the v2.0 (default branch) setup has a few friction points during initialization. After repetitively trying to bring the stack up, the dicomhawk-api-1 container consistently crashes post-startup.

At the same time, v3.0 is easy to setup using the following commands:

1. python3 customizer/customizer.py
2. docker-compose build customizer
3. docker-compose --profile main up -d
4. docker-compose --profile main --profile monitoring up -d

But appears to be v3.0, moving toward a significant refactor. Fully rewrites the backend into a cleaner modular structure under a proper dicomhawk/ python package with a commands/ CLI structure, replaces docker-compose with a Makefile as the primary build/run interface, drops logstash.conf, install.sh; adds .dockerignore, pyproject.toml, and Changelog.md; removes standalone customizer component, etc., all of which are visible in this draft PR #91, feel like a strong step forward. Because of this, I wanted to check whether new contributors are expected to start directly with v3.0 rather than v2.0.

I also noticed that the README doesn’t fully capture the current setup flow. I’ve successfully set up the project locally using the v3.0 branch, and it’s working smoothly. I’d be happy to submit an updated README with clear, step-by-step instructions, both for the current state of the repository and in preparation for v3.0.

Additionally, I’m interested in contributing to fill the gaps around hardening the Docker setup using best practices.

Please let me know if this would be useful, or if you’d prefer the documentation and improvements to target v3.0 going forward. I’d be happy to work in whichever direction maintainers want me to be inclined towards.

[RicYaben]

Hey @RicYaben ,

I was going through the DICOMHawk codebase and setting it up locally to better understand the current workflow and the architecture.
...

Please read above.

this work continues in #v3.0-dev, will merge with v3.0 when that one is ready
#91 (comment)