////------------------------ FOR STUDENTS -----------------------////
WHAT THIS PROJECT DOES
This framework runs your Assembler and Simulator on test cases, compares the output with the correct (golden) output, and gives marks.
BIG IDEA
- Assembler tests and Simulator tests are separate.
- Your files must be named exactly:
- SimpleAssembler/Assembler.py
- SimpleSimulator/Simulator.py
---------------------------------------------------------------
1) FOLDER MAP (WHERE EVERYTHING LIVES)
---------------------------------------------------------------
ASSEMBLER TESTS
Location: automatedTesting/tests/assembly/
- simpleBin/ -> input assembly programs (easy)
- hardBin/ -> input assembly programs (hard)
- errorGen/ -> invalid assembly programs (error tests; no comparison/grading by default)
- bin_s/ -> golden (correct) machine-code outputs for simpleBin/
- bin_h/ -> golden (correct) machine-code outputs for hardBin/
- user_bin_s/ -> machine code generated by YOUR Assembler for simpleBin/
- user_bin_h/ -> machine code generated by YOUR Assembler for hardBin/
SIMULATOR TESTS
Locations:
- automatedTesting/tests/bin/
- automatedTesting/tests/traces/
- automatedTesting/tests/user_traces/
- tests/bin/simple/ -> input machine-code programs (easy)
- tests/bin/hard/ -> input machine-code programs (hard)
- tests/traces/simple/ -> golden (correct) simulator traces
- tests/traces/hard/ -> golden (correct) simulator traces
- tests/user_traces/simple/ -> traces generated by YOUR Simulator
- tests/user_traces/hard/ -> traces generated by YOUR Simulator
---------------------------------------------------------------
2) VERY IMPORTANT RULE: FILENAMES MUST MATCH
---------------------------------------------------------------
The framework matches files by name.
Examples:
- If input is:
automatedTesting/tests/assembly/simpleBin/simple_1.txt
then the golden output must be:
automatedTesting/tests/assembly/bin_s/simple_1.txt
- If simulator input is:
automatedTesting/tests/bin/simple/simple_1.txt
then the golden trace must be:
automatedTesting/tests/traces/simple/simple_1.txt
If names do not match, the grader will fail or skip correctly.
---------------------------------------------------------------
3) HOW TO WRITE YOUR ASSEMBLER
---------------------------------------------------------------
Your assembler MUST:
- Be placed at: SimpleAssembler/Assembler.py
- Be runnable like this:
python3 Assembler.py <input_assembly_path> <output_machine_code_path> [output_readable_path]
Notes:
- The 3rd "readable" output path is OPTIONAL.
- The framework will pass 3 paths. If you accept the 3rd path and create the file, that is best.
- All files should use the .txt extension.
---------------------------------------------------------------
4) HOW TO WRITE YOUR SIMULATOR
---------------------------------------------------------------
Your simulator MUST:
- Be placed at: SimpleSimulator/Simulator.py
- Be runnable like this:
python3 Simulator.py <input_machine_code_path> <output_trace_path> [output_readable_path]
Notes:
- The 3rd "readable" output path is OPTIONAL.
- The framework will pass 3 paths. If you accept the 3rd path and create the file, that is best.
- All files should use the .txt extension.
---------------------------------------------------------------
5) HOW TO RUN THE GRADER (STEP BY STEP)
---------------------------------------------------------------
Always run commands from inside the automatedTesting/ folder.
Linux:
cd automatedTesting
Windows:
(open terminal inside automatedTesting)
A) Run ONLY assembler tests
Linux:
python3 src/main.py --no-sim --linux
Windows:
python3 src\main.py --no-sim --windows
B) Run ONLY simulator tests
Linux:
python3 src/main.py --no-asm --linux
Windows:
python3 src\main.py --no-asm --windows
C) Run BOTH assembler and simulator tests
Linux:
python3 src/main.py --linux
Windows:
python3 src\main.py --windows
---------------------------------------------------------------
6) OPTIONAL CLEANUP: CLEAR OLD STUDENT OUTPUTS
---------------------------------------------------------------
If you are grading a NEW student, clear old generated files first:
Linux:
python3 src/main.py --linux --clear-residue
Windows:
python3 src\main.py --windows --clear-residue
What this does:
- Deletes previously generated files in:
- automatedTesting/tests/assembly/user_bin_s/
- automatedTesting/tests/assembly/user_bin_h/
- automatedTesting/tests/user_traces/simple/
- automatedTesting/tests/user_traces/hard/
- Then exits (no grading).
////------------------------ FOR STUDENTS -----------------------////
////------------------------ FOR TAs -----------------------////
TA WORKFLOW (CREATE YOUR OWN TESTS)
1) Create assembly programs
- Put them in:
automatedTesting/tests/assembly/simpleBin/
(and/or hardBin/)
- Programs should end with a halt behavior as per course spec.
2) Grade the assembler
From inside automatedTesting/ run:
python3 src/main.py --no-sim --linux
(or --windows)
3) Collect the generated machine code
- You will find it in:
automatedTesting/tests/assembly/user_bin_s/
(or user_bin_h/)
4) Prepare simulator inputs
- Copy the generated machine code into:
automatedTesting/tests/bin/simple/
(or tests/bin/hard/)
5) Grade the simulator
From inside automatedTesting/ run:
python3 src/main.py --no-asm --linux
(or --windows)
6) Check generated traces
- You will find them in:
automatedTesting/tests/user_traces/simple/
(or tests/user_traces/hard/)
7) Promote traces to golden references
After verifying correctness, copy them into:
- automatedTesting/tests/traces/simple/
- automatedTesting/tests/traces/hard/
These become the golden outputs for future grading.
////------------------------ FOR TAs -----------------------////