Repository navigation
Make field matching meaningful and add an eligibility evaluation harness - #145
Open
ma7moudalysalem wants to merge 1 commit into
Open
ma7moudalysalem wants to merge 1 commit into
ma7moudalysalem wants to merge 1 commit into
Conversation
…ation harness The recommender and the eligibility checker both compared a student's field of study against a listing by exact string equality, across two vocabularies that never met: listings are written from a fifteen-item canonical list, while the seeder wrote fine-grained specialisms and the student profile takes free text. A student interested in "Software Engineering" could never match a listing open to "Engineering". Scoring - The match score is now the share of the attainable weight a listing earns across the criteria it actually constrains, rather than a fixed point sum. A criterion the listing leaves open drops out of both sides of the ratio, so an award open to every discipline is no longer punished for that openness. - Field comparison is graded: an exact match counts in full, a shared significant word counts as a half. - The explanation now names the criteria that matched instead of restating the score band, and says plainly when nothing matched. Eligibility - The field criterion could never report "not met" — its final branch returned "partially met", so a law student was told a computer science award was partially met. Unrelated fields now report "not met" and "partially met" is reserved for a genuinely adjacent field. Data - The seeder writes FieldsOfStudyJson on generated listings and states student preferences in the same canonical vocabulary. Measured on a seeded database, listings carrying a field went from 0 of 336 to 733 of 763, and the share of (student, listing) pairs sharing a field from 0.0% to 12.2%. - The curated external dataset carries fieldsOfStudy; five entries restrict discipline and nineteen are explicitly open to any field. Evaluation - server/tools/ScholarPath.Eval runs the delivered checker over a seeded database with a fixed seed and reports per-criterion outcomes. On 3,600 pairs, fields with nothing in common reported as "partially met" fell from 93.5% to zero. Also enables the SignalR Redis backplane when Redis is configured, and adds WebKit and Edge to the Playwright projects, so both claims hold in fact. Tests: 951 passing (18 new).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
The recommender and the eligibility checker both compared a student's field of study against a listing by exact string equality, across two vocabularies that never met. Listings are written from a fifteen-item canonical list; the seeder wrote fine-grained specialisms and the student profile takes free text. A student interested in Software Engineering could never match a listing open to Engineering.
Scoring
The match score is now the share of the attainable weight a listing earns across the criteria it actually constrains, instead of a fixed point sum. A criterion the listing leaves open drops out of both the numerator and the denominator, so an award open to every discipline is no longer punished for that openness — previously Gates Cambridge and Fulbright scored zero on the 40-point field term for being open to everyone.
Field comparison is graded: an exact match counts in full, a shared significant word counts as a half. The explanation now names the criteria that matched rather than restating the score band, and says plainly when nothing matched.
Eligibility
The field criterion could never report "not met" — its final branch returned
partial, so a law student was told a computer science award was partially met. Unrelated fields now report not met, and "partially met" is reserved for a genuinely adjacent field.Data
The seeder writes
FieldsOfStudyJsonon generated listings and states student preferences in the same canonical vocabulary.The curated external dataset carries
fieldsOfStudy: five entries restrict discipline, nineteen are explicitly open to any field.Evaluation harness
server/tools/ScholarPath.Evalruns the delivered checker over a seeded database with a fixed seed and reports per-criterion outcomes, so a figure can be reproduced:On 3,600 pairs:
Two claims made true in code
The SignalR Redis backplane is now wired when Redis is configured (
AddStackExchangeRediswas never called), and WebKit and Edge are added to the Playwright projects.Verification
dotnet build— 0 warnings, 0 errorsdotnet test— 951 passing, 18 new (10 scoring, 8 field matching). The scoring path had no test coverage at all before this.Note for the reviewer
Program.csin the working tree also carries an unrelated, unfinished "Live System Activity" feature whose files are untracked. That work is deliberately not included here — only the SignalR hunk is.