EMDS - Java UDP Receiver ( mono-repo )

Short description

EMDS (Equities Market Data Stack) is a compact mono-repo that implements a reliable, testable UDP message receiver in Java and a Python traffic simulator to exercise end-to-end behavior, It focuses on correctness (parsing, sequencing, error handling) and reproducible E2E tests as a foundation for later performance work

Quickstart

Prerequisites: Java 17+, Python 3.x, use the Gradle wrapper

Run feature tests (all):

# Linux / macOS
./gradlew :java-receiver:runCucumber

# Windows
.\gradlew.bat :java-receiver:runCucumber

Run a tagged scenario:

# Linux / macOS
./gradlew :java-receiver:runCucumber -PcucumberTags="@parse"

# Windows
.\gradlew.bat :java-receiver:runCucumber -PcucumberTags="@parse"

Run the simulator manually ( ensure receiver is running ):

python3 python/udp_simulator.py --host 127.0.0.1 --port 9000 --rate 200 --duration 10

What's included

• java-receiver/
  • Java UDP receiver with readiness signaling ( awaitReady ) and metrics
  • Integration tests with JUnit Platform + Cucumber and a runCucumber Gradle task
  • Step definitions that support docStrings and data-driven scenarios
• python/
  • udp_simulator.py - simple UDP traffic generator ( rate, count, duration, start seq, malformed).

Simulator executable resolution (robustness) Different developer machines and CI hosts may have different Python executables or loactions ( eg. python, python3, py.exe, or an absolute path on Windows ). The test step that launches the Python simulator uses a resolution order to make runs reliable across OSes and environments :

Resolution order ( how SimulatorSteps picks Python)

1. JVM system property: -Dsimulator.python=/abs/path/to/python
2. Environment variable: SIMULATOR_PYTHON
3. Evironment variable: PYTHON
4. OS-specific fallbacks:
   • Windows: tried py.exe, python.exe, and some common install paths (e.g., C:\Python39\python.exe).
   • Linux/macOS: tries python3 then python on PATH.

Why this matters

• On Windows python or python3 may not be on PATH, py.exe or an absolute interpreter path might be required.
• Some CI agents or custom dev machines install python in non-standard locations.
• Explicit overrides avoid flaky runs and make debugging faster

How to override (examples)

• Preferred: pass an absolute interpreter via JVM property when running Gradle:

#Linux / maxOS - explicit python3
./gradlew :java-receiver:runCucumber -Dsimulator.python=/usr/bin/python3

#Windows - explicit path
./gradlew '-Dsimulator.python="C:\Python39\python.exe"' :java-receiver:runCucumber 

• Or set environment variables before running Gradle:

# Linux / macOS
export SIMULATOR_PYTHON=/usr/bin/python3
./gradlew :java-receiver:runCucumber

# Windows
$env:SIMULATOR_PYTHON = "C:\Python39\python.exe"
.\gradlew.bat :java-receiver:runCucumber

Troubleshooting tips

• If tests fail because the simulator process cannot be started, the step will throw a clear message telling you to set -Dsimulator.python or SIMULATOR_PYTHON
• If you used VM option in your IDE and it still read null, prefer setting SIMULATOR_PYTHON or pass -Dsimulator.python directly on the Gradle command line. Some IDE run configurations require adding the CM option to the Gradle invocation or to the runner configuration.

CI recommendations

• Ensure Python 3 is available on CI runners ( Github-hosted ubuntu images include Python, Windows images may require action/setup-python)
• Use deterministic simulator runs in CI:
  • Always pass --duration or --count so the simulator terminates.
  • Example (Gradle step runs Cucumber which starts simulator with duration): ./gradlew :java-receiver:runCucumber -PcucumberTags="@simulator_dynamic"
• Persist simulator stdout/stderr for debugging:
  • Modify SimulatorSteps to redirect simulator output to a file under java-receiver/build/tmp/simulator/ and include that path in CI artifact uploads so you can inspect exact simulator logs on failure.

Edge cases and robustness covered

• Avoid send-before-bind races by using UdpReceiver.awaitReady() and getBoundPort, tests always waitfor the reciever tobind before sending
• Payloads that contain | are handled via docStrings ( triple-quoted blocks) in feature files to avoid Gherkin table parsing conflicts.
• Simulator resolution uses multiple fallbacks and validates candidates (by testing --version for PATH commands or verifying file existence for absolute paths).
• Test use polling helpers with timeouts (or BooleanSupplier) to avoid flaky timing assertions, simulator-driven tests use tolerant expected counts (e.g. ~90% of rate*duration) to allow scheduling jitter

Adding simulator logs to artifacts (quick recipe)

1. In SimulatorSteps, write simulator stdout/stderr to: java-receiver/build/tmp/simulator/<scenario>-simulator.log
2. In your CI workflow, include that path in the upload-artifacts globs:

path: |
  **/build/test-results/**/*.xml
  **/build/reports/**/*.html
  **/build/reports/**/*.json
  java-receiver/build/reports/cucumber.json
  java-receiver/build/tmp/simulator/**/*.log

3. On failures, download the artifact ZIP from the run page to inspect simulator behavior and timing

Contributing

• Run unit tests : ./gradlew test
• Run integration features: ./gradlew :java-receiver:runCucumber
• To run simulator-driven tests in CI, ensure Python 3 is available and pass an absolute script path via -Dsimulator.script or set SIMULATOR_SCRIPT if needed.

Key files

• java-receiver/src/main/java/.../UDPReceiver.java
• java-receiver/src/integrationTest/java/.../RunCucumberTestSuite.java
• java-receiver/src/integrationTest/java/.../UDPEndToEndTest.java
• java-receiver/src/integrationTest/java/.../SimulatorSteps.java
• python/udp_simulator.py
• java-receiver/src/integrationTest/resources/features/*.feature

Contact

Open an issue or PR in the repo for questions or to suggest enhancements