Pathfinder Tester

A black-box conformance tester for the command-line Pathfinder project described in TASK.md and EVAL.md.

The tester builds a Pathfinder implementation, runs it against the supplied maps and invalid-input cases, and validates the resulting train schedule rather than comparing it with one fixed output. This allows different valid routing algorithms to be tested with the same suite.

Requirements

• Go installed and available as go

• A Pathfinder project that can be built with go build .

• The Pathfinder project must support:

  pathfinder MAP_FILE START_STATION END_STATION NUMBER_OF_TRAINS
  

Usage

From this repository, run:

go run . -project ../path/to/pathfinder

The default map directory is this repository's testdata folder. When invoking the tester from another working directory, provide its location explicitly:

go run ./path/to/pathfinder-tester \
  -project ./path/to/pathfinder \
  -maps ./path/to/pathfinder-tester/testdata

Available flags:

Flag	Default	Description
-project	.	Directory containing the Pathfinder Go project
-maps	testdata	Directory containing the evaluation map files
-timeout	5s	Maximum duration of each Pathfinder invocation

For a slower implementation or machine, increase the per-run timeout:

go run . -project ../pathfinder -timeout 15s

Testing several variants

Run the tester once for each project directory:

for variant in ../pathfinder-v1 ../pathfinder-v2 ../pathfinder-v3; do
  echo "Testing $variant"
  go run . -project "$variant" || true
done

The || true lets the loop continue after a failing variant. Remove it when a failure should stop the script immediately.

Results

Each check is printed as PASS or FAIL, followed by a final summary:

PASS  London_one_train
PASS  London_two_trains
FAIL  Bond: used 7 turns; maximum is 6

31 passed, 1 failed

The tester exits with:

• 0 when every check passes
• 1 when a check fails or the tester cannot build/configure the target

What is tested

The suite covers the mandatory evaluation scenarios, including:

• London routing with 1, 2, 3, 4, and 100 trains
• Use of multiple routes where required
• Turn limits for the Bond, Jungle, Beginning, Two, Beethoven, and Small maps
• A tricky cyclic network with 100 trains
• A deliberately nonoptimal reference that must be improved from five turns to four by using two routes
• A 10,000-station network
• Too few and too many command-line arguments
• Missing stations, identical start/end stations, and disconnected routes
• Invalid train counts
• Duplicate stations, coordinates, and connections
• Invalid coordinates and station names
• Connections to unknown stations
• Missing map sections
• Maps containing more than 10,000 stations
• Per-process timeout protection

For successful runs, the schedule validator checks that:

• every token uses the exact T<number>-station format;
• every train moves at most once per turn;
• every move follows an existing track;
• a track is used at most once per turn, in either direction;
• intermediate stations contain at most one train after each turn;
• every train reaches the requested end station;
• the schedule stays within the case's turn limit; and
• the required number of distinct routes is used.

Moves printed on the same line are treated as simultaneous, matching the reference schedules in EVAL.md. The start and end stations may contain any number of trains.

For invalid input, the current tester expects Pathfinder to exit unsuccessfully, write text containing Error to standard error, and leave standard output empty.

Test maps

The committed fixtures live in testdata:

• london.map and the six specified efficiency networks
• tricky.map for cycles and multiple long routes
• optimization.map for the route-efficiency bonus
• large.map for the 10,000-station boundary
• disconnected.map for the no-path case
• supplementary parsing maps

Malformed maps are generated in a temporary directory for each test run and removed automatically afterward. The compiled Pathfinder binary is also created in a temporary directory and cleaned up when testing finishes.

Developing the tester

Format and check the tester with:

gofmt -w *.go
go test ./...
go vet ./...