Improve search matching diagnostics and ranking - #88
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 33c2ae664d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| current = await asyncio.to_thread(_public_http_url, url) | ||
| headers = {"User-Agent": "BertJobAnalyzer/20 (+https://github.com/emnl51/bert)"} | ||
| async with httpx.AsyncClient(timeout=20, follow_redirects=False, headers=headers) as client: | ||
| for _ in range(4): | ||
| response = await client.get(current) |
There was a problem hiding this comment.
When an attacker-controlled hostname changes its DNS answer between _public_http_url() and client.get(), validation can see a global address while HTTPX's second resolution connects to a private or link-local service. This leaves both the diagnostic endpoint and automatic enrichment vulnerable to DNS-rebinding SSRF; resolve once and pin the connection to the validated address, or validate the actual connected peer.
Useful? React with 👍 / 👎.
| eligible = bool( | ||
| not blocked | ||
| and role.relevant | ||
| and not hard_employment | ||
| and employment_ok | ||
| and job.overall_score >= min_score | ||
| and job.language_score >= min_language | ||
| and language_allowed | ||
| ) |
There was a problem hiding this comment.
When a Search Job has an assigned candidate and the vacancy has sufficient description evidence, run_search_job() can exclude it because analyze_job() falls below min_cv_match, but diagnose_job() never loads the assignment or evaluates that gate. Such vacancies are reported as eligible and saved benchmarks produce incorrect precision/recall, so the diagnostic must replay the candidate gate or explicitly report it as unevaluated.
Useful? React with 👍 / 👎.
| company = _company_signature(getattr(job, "company", "")) | ||
| candidates = company_buckets.get(company, []) if company else range(len(unique)) | ||
| matched_index = next((index for index in candidates if _same_vacancy(unique[index], job)), None) |
There was a problem hiding this comment.
When two providers return the same canonical URL with different non-empty company strings, this bucket lookup only compares the new job against entries having the same normalized company, so _same_vacancy() never gets a chance to apply its URL match. Common variants such as Acme GmbH versus Acme Deutschland GmbH therefore remain duplicate rows and lose the promised combined source links; exact URL candidates need a separate index checked before company bucketing.
Useful? React with 👍 / 👎.
| """SELECT scope,term,weight FROM positive_rules | ||
| WHERE enabled=1 AND profile_id=? | ||
| AND (evidence_count>=? OR strongest_event IN ('interview','offer'))""", | ||
| (profile_id, MIN_POSITIVE_EVIDENCE), |
There was a problem hiding this comment.
When one vacancy is first marked suitable and later reaches applied, record_positive_event() increments the same rules once for each event type, so evidence_count becomes 2 and this predicate activates the boost without a second example. That defeats the new two-example safeguard for ordinary evidence; readiness should count distinct job keys, while retaining the explicit interview/offer exception.
Useful? React with 👍 / 👎.
| job.discovered_queries = list(dict.fromkeys([*job.discovered_queries, term])) | ||
| if job.url in seen: | ||
| continue |
There was a problem hiding this comment.
When the same StepStone vacancy is returned for multiple search terms, each parsed object receives only the current term and all occurrences after the first are immediately discarded, making the query funnel credit only whichever term ran first. The same ordering-dependent pattern exists in Adzuna and JobSpy, so fetched, recommended, and new_matches are inaccurate for overlapping queries; merge the term into the retained job before skipping the duplicate.
Useful? React with 👍 / 👎.
1 participant
Summary
Redesigns the next layer of job discovery and matching around measurable retrieval quality, evidence-backed eligibility and conservative learning.
Matching and recall
Ranking and learning
Sources and data quality
UI and operations
Validation
263 passed74.10%(required: 50%)ruff check app testsruff format --check app testspython -m compileall -q app testsnode --check app/*.jspip-audit -r requirements.txt --ignore-vuln PYSEC-2026-1604: clean (one documented upstream exception ignored)The local runner does not provide Docker; the existing GitHub Actions container job will perform the image build and health smoke test.