Test Suite by Jeevesh28 · Pull Request #234 · deweylab/RSEM

Jeevesh28 · 2026-03-17T16:33:00Z

Add:

Test suite with SARS-CoV-2 data and gold standard

Goal	Command
Single test	`make test-prepare-reference`
	`make test-calculate-expression`
	`make test-calculate-expression-ci`
	`make test-simulate-reads`
All tests, stop on first failure	`make test`
All tests, run all and report at end	`make test-all`

- Replace boost::random with std::random - Replace boost::math::normal/cdf with std::erf-based implementation - Update Makefile: -std=c++17, remove BOOST - Fix C++11/17 compatibility in buildReadIndex, SingleHit, PairedEndHit

- Remove boost dependencies in bed2vector and bamread files - Replace hash_map and hash_set

- Add Makefile test targets: test-prepare-reference, test-calculate-expression, test-simulate-reads - Add test-all for CI (continue on failure) - Add generate-gold target to regenerate gold standard - Use SARS-CoV-2 reference (sarscov2.fasta, sarscov2.gtf) and reads - Gold includes: reference (.grp, .ti), expression (genes.results, isoforms.results, stat/), simulated (fq, sim.isoforms.results, sim.genes.results) - TEST_THREADS=4, TEST_SEED=0, N=1000 for reproducibility - Add tests/output/ to .gitignore

- Boost-compatible RNG for reproducible results

…e-gold target

- Copy all rsem-prepare-reference outputs to gold (grp, ti, chrlist, idx.fa, n2g.idx.fa, seq, transcripts.fa, bt2 index files) - Update test-prepare-reference to diff all reference files

…t-calculate-expression target

…l tests - Set TEST_THREADS=1 so Gibbs/calcCI produce reproducible output - Replace CI sanity checks with exact diff for genes.results, isoforms.results - generate-gold now invokes generate-gold-ci (all gold in one run) - Add expression_ci gold (genes, isoforms, stat)

cndewey · 2026-04-03T13:11:36Z

+	@echo "==> Testing rsem-simulate-reads"
+	rm -rf $(TEST_OUTPUT)/simulated
+	mkdir -p $(TEST_OUTPUT)/simulated
+	@theta0=$$(awk 'NR==3 {print $$1}' $(TEST_OUTPUT)/expression/$(SAMPLE_NAME).stat/$(SAMPLE_NAME).theta) && \


have this depend on $(GOLD), not $(TEST_OUTPUT)

cndewey · 2026-04-03T13:11:58Z

+	diff $(TEST_OUTPUT)/expression_ci/$(SAMPLE_NAME_CI).stat/$(SAMPLE_NAME_CI).model $(GOLD)/expression_ci/$(SAMPLE_NAME_CI).stat/$(SAMPLE_NAME_CI).model
+	diff $(TEST_OUTPUT)/expression_ci/$(SAMPLE_NAME_CI).stat/$(SAMPLE_NAME_CI).theta $(GOLD)/expression_ci/$(SAMPLE_NAME_CI).stat/$(SAMPLE_NAME_CI).theta
+
+test-simulate-reads: test-calculate-expression


remove dependency on test-calculate-expression. Should just use $(GOLD)

cndewey · 2026-04-03T13:12:18Z

+	./rsem-simulate-reads \
+		$(GOLD)/reference/$(REF_NAME) \
+		$(TEST_OUTPUT)/expression/$(SAMPLE_NAME).stat/$(SAMPLE_NAME).model \
+		$(TEST_OUTPUT)/expression/$(SAMPLE_NAME).isoforms.results \


use $(GOLD)

cndewey · 2026-04-03T13:12:39Z

+	@theta0=$$(awk 'NR==3 {print $$1}' $(TEST_OUTPUT)/expression/$(SAMPLE_NAME).stat/$(SAMPLE_NAME).theta) && \
+	./rsem-simulate-reads \
+		$(GOLD)/reference/$(REF_NAME) \
+		$(TEST_OUTPUT)/expression/$(SAMPLE_NAME).stat/$(SAMPLE_NAME).model \


use $(GOLD)

cndewey · 2026-04-03T13:13:45Z

+	diff $(TEST_OUTPUT)/expression/$(SAMPLE_NAME).stat/$(SAMPLE_NAME).model $(GOLD)/expression/$(SAMPLE_NAME).stat/$(SAMPLE_NAME).model
+	diff $(TEST_OUTPUT)/expression/$(SAMPLE_NAME).stat/$(SAMPLE_NAME).theta $(GOLD)/expression/$(SAMPLE_NAME).stat/$(SAMPLE_NAME).theta
+
+test-calculate-expression-ci: test-prepare-reference


remove dependency on test-prepare-reference

cndewey · 2026-04-03T13:13:57Z

+		diff $(TEST_OUTPUT)/reference/$$bf $$f; \
+	done
+
+test-calculate-expression: test-prepare-reference


remove dependency

cndewey · 2026-04-03T13:14:52Z

should these and other related changes be in the remove_boost branch?

cndewey · 2026-04-03T13:16:18Z

+
+# ---- Individual tests --------------------------------------------
+
+test-prepare-reference:


it would be helpful to have a message printed at the end of each test indicating success

cndewey · 2026-04-03T13:21:28Z

@Jeevesh28 - excellent work getting these test cases set up. I have left some specific comments within the files. Some other comments:

I think we should make the test read file much smaller (maybe just 1000 reads) and avoid having to use git lfs
I am seeing slight numerical differences on an older intel mac. So we will need to move to checks that allow for some very small numerical diffs. Here is the diff for my_sample.theta:

diff tests/output/expression/my_sample.stat/my_sample.theta tests/gold/expression/my_sample.stat/my_sample.theta
1a2
> 0.459553999999847 0.0333932276352768 0.00709735704896766 0.0428769103538539 0.000226531894875526 0.0151884751124963 0.007935117636998 0.0826895888836575 0.223538257936758 0.0224108021009595 0.10508973139631
3d3
< 0.459553999999847 0.0333932276352768 0.00709735704896765 0.0428769103538539 0.000226531894875526 0.0151884751124963 0.007935117636998 0.0826895888836575 0.223538257936758 0.0224108021009595 0.10508973139631
make: *** [test-calculate-expression] Error 1

- Print OK after each test, drop redundant prerequisites - Replace reads.fastq with subset (10k reads) - Regenerate tests/gold expression

Jeevesh28 added 10 commits February 27, 2026 23:18

refactor: modernize to C++17 and remove boost dependency

199abca

- Replace boost::random with std::random - Replace boost::math::normal/cdf with std::erf-based implementation - Update Makefile: -std=c++17, remove BOOST - Fix C++11/17 compatibility in buildReadIndex, SingleHit, PairedEndHit

refactor: modernize to C++17 and remove boost dependency in pRSEM

1313d8f

- Remove boost dependencies in bed2vector and bamread files - Replace hash_map and hash_set

refactor: modernize to C++17 and remove boost dependency

da53474

- Boost-compatible RNG for reproducible results

fix: update Makefile to create missing simulated directory in generat…

08bcb7f

…e-gold target

test: regenerate gold standard

5c59574

test: add all reference files to gold and update Makefile

f847101

- Copy all rsem-prepare-reference outputs to gold (grp, ti, chrlist, idx.fa, n2g.idx.fa, seq, transcripts.fa, bt2 index files) - Update test-prepare-reference to diff all reference files

fix: update Makefile to use GOLD directory for reference files in tes…

55342ed

…t-calculate-expression target

fix: simplify gold standard generation in Makefile

a9fa817

Jeevesh28 requested a review from cndewey March 17, 2026 16:33

Jeevesh28 assigned Jeevesh28 and cndewey Mar 17, 2026

Jeevesh28 changed the title ~~Test suite~~ Test Suite Mar 17, 2026

cndewey reviewed Apr 3, 2026

View reviewed changes

Jeevesh28 added 2 commits April 7, 2026 00:26

Improve test Makefile target

a3640b6

- Print OK after each test, drop redundant prerequisites - Replace reads.fastq with subset (10k reads) - Regenerate tests/gold expression

Remove Git LFS for reads.fastq

7c80fad

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test Suite#234

Test Suite#234
Jeevesh28 wants to merge 12 commits intomasterfrom
test-suite

Jeevesh28 commented Mar 17, 2026 •

edited

Loading

Uh oh!

cndewey Apr 3, 2026

Uh oh!

cndewey Apr 3, 2026

Uh oh!

cndewey Apr 3, 2026

Uh oh!

cndewey Apr 3, 2026

Uh oh!

cndewey Apr 3, 2026

Uh oh!

cndewey Apr 3, 2026

Uh oh!

cndewey Apr 3, 2026

Uh oh!

cndewey Apr 3, 2026

Uh oh!

cndewey commented Apr 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		# ---- Individual tests --------------------------------------------

		test-prepare-reference:

Conversation

Jeevesh28 commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cndewey commented Apr 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Jeevesh28 commented Mar 17, 2026 •

edited

Loading