diff --git a/CHANGELOG.md b/CHANGELOG.md index 103ffc8..30a5b93 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,16 +4,27 @@ ### Added +- Adds a structural role-relevance gate so location, language, schedule, and skill points cannot admit an unrelated occupation. +- Adds Strong, Match, Stretch, and Excluded result tiers plus a Match Tier filter in Job Review. +- Adds per-Search Job preferred or strict working-time handling. +- Adds paint/coating, industrialization, supplier-quality, manufacturing-engineering, and production-planning aliases in English and German. - Adds a persistent Light, Dark, and System theme selector to the main workspace and sign-in screen. - Adds a focused job-detail dialog with the complete description, fit evidence, work schedule, source, date, and original-listing link. - Adds an idempotent release workflow that turns published GitHub Release notes into an automated changelog pull request. ### Fixed +- Reclassifies legacy profile scores during migration and hides rows that were supported only by soft signals. +- Keeps title-relevant vacancies with incomplete source descriptions instead of failing them on an unevaluable CV Match. +- Prevents a slow or blocked JobSpy board from starving the other configured boards. +- Prevents one slow experimental provider from delaying all stable API and feed sources. - Prevents the sidebar shell from stopping before the rebuilt navigation is mounted. ### Changed +- Plans broad, unqualified role queries before schedule-specific variants and distributes capped provider budgets across requested role families. +- Runs independent sources concurrently while preserving deterministic source order. +- Keeps strong full-time or working-time-unclear roles as labeled stretch results by default; strict exclusion remains selectable per Search Job. - Aligns the workspace with Fredy’s compact visual system: a 220/60 px sidebar, neutral surfaces, restrained shadows, tighter radii, and denser navigation. - Moves Settings and Administration subpages into contextual content tabs while keeping their sidebar entries singular. diff --git a/README.md b/README.md index 91f4f46..ee5cb35 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,8 @@ It supports separate user workspaces, scheduled searches, profile-specific rules - Search Profiles and independently scheduled Search Jobs - Job Fit, Language Fit, Overall Fit, and evidence-based CV Match - German, English, mixed, and unknown job-ad language detection -- Strict employment-format filtering for full-time, part-time, Werkstudent, and Minijob searches +- Per-search preferred or strict working-time handling for full-time, part-time, Werkstudent, and Minijob searches +- Bilingual role-first query planning, structural role relevance, and Strong / Match / Stretch result tiers - Candidate Profiles with encrypted CV text - Optional local Ollama context for CV analysis - Application workspace with Kanban/list views, follow-up dates, contacts, activity history, and source conversion @@ -40,6 +41,37 @@ Experimental integrations: Experimental sources can be affected by rate limits, anti-bot controls, or upstream page changes. Bert does not bypass CAPTCHAs or other access controls. +## Search matching + +Bert treats provider search results as discovery candidates, not as confirmed matches. Each vacancy passes through +separate stages so a high language, location, or skill score cannot rescue an unrelated occupation: + +1. **Query planning** removes location and working-time constraints from base role queries, distributes the first + provider queries across requested role families, and adds English/German variants before narrower schedule variants. + Custom Search Job queries remain isolated from the profile's other provider queries. +2. **Role relevance** requires a requested phrase or role family in the title. A generic professional title such as + `Engineer II` is accepted only when its description supplies both role-family and industrial-domain evidence. + Conflicting software/data occupations are rejected unless supported by multiple manufacturing signals. +3. **Independent dimensions** rank Job Fit, Language Fit, working-time fit, learned preferences, and optional CV Match. +4. **Result tiers** expose `Strong`, `Match`, and `Stretch` vacancies in Job Review. Rows without structural role + evidence—or rows that fail an explicitly strict eligibility rule—are marked excluded and are not shown, even when + their old Overall Fit would have crossed the threshold. + +The CV Match threshold becomes a hard gate only when the provider supplied a sufficiently complete description. +Title-relevant vacancies with a short card/snippet remain visible as `Stretch` with a deferred-CV explanation instead +of being rejected for evidence the source did not provide. + +Search Jobs default to **Prefer profile hours; keep stretch roles**. This keeps a strong technical vacancy visible when +it is full-time or its hours are missing, while labeling the constraint for review; it can still be notified when the +other configured thresholds pass. Select **Strictly exclude other/unknown hours** when the working arrangement is a +hard requirement; those vacancies are excluded from that Search Job's review queue and notifications. Student-only +vacancies are always excluded unless the profile explicitly targets enrolled-student work. + +During the database migration, existing profile scores are re-evaluated by the role gate. Run each Search Job once +after updating to fetch fresh source data and populate the new match tiers. For a profile with four target role +families, six provider queries cover every base role plus two translated variants; use eight when provider capacity +permits to cover both English and German for all four families. + ## Quick start Requirements: diff --git a/app/employment_filter.py b/app/employment_filter.py index e3666af..123179a 100644 --- a/app/employment_filter.py +++ b/app/employment_filter.py @@ -1,7 +1,7 @@ import re from .models import Job -from .search_intent import ROLE_FAMILIES, role_families_for_terms +from .search_intent import ROLE_FAMILIES, ROLE_QUERY_TERMS, role_families_for_terms from .text_match import contains_affirmed_phrase @@ -42,7 +42,13 @@ "full_time", ) -STUDENT_SIGNALS = ("werkstudent", "working student", "student assistant", "studentische aushilfe") +STUDENT_SIGNALS = ( + "werkstudent", + "working student", + "student assistant", + "studentische aushilfe", + "studentische hilfskraft", +) HOURS_PATTERN = re.compile( r"(?\d{1,2})(?:\s*(?:-|–|bis|to)\s*(?P\d{1,2}))?" r"\s*(?:stunden|std\.?|hours?|h|wochenstunden)" @@ -95,24 +101,41 @@ def profile_targets_full_time(profile: dict) -> bool: return any(term in FULL_TIME_SIGNALS for term in format_terms) -def search_terms_for_profile(profile: dict) -> list[str]: - """Return profile-specific provider queries without mixing other enabled profiles. +QUERY_ARRANGEMENT_PATTERN = re.compile( + r"(?i)\b(?:werkstudent\w*|working student|student assistant|studentische(?:r|n)? \w+|" + r"teilzeit|part[ -]?time|full[ -]?time|vollzeit|minijob|mini-job|geringf(?:ü|ue)gig\w*)\b" +) - The built-in Werkstudent/Part-time profile gets diversified broad queries first so - low JobSpy max_search_terms values still cover student, part-time and minijob work. - User-configured queries remain included afterwards. + +def _base_provider_query(value: str, profile: dict) -> str: + """Remove scheduling/location constraints that unnecessarily narrow discovery.""" + query = QUERY_ARRANGEMENT_PATTERN.sub(" ", _norm(value)) + locations = [profile.get("target_location", ""), *(profile.get("location_terms") or [])] + for location in sorted({_norm(x) for x in locations if _norm(x)}, key=len, reverse=True): + query = re.sub(rf"(? list[str]: + """Plan balanced provider queries for one profile. + + Unqualified role queries are intentionally placed before schedule-specific variants. + Providers such as JobSpy and StepStone cap the number of phrases they execute; the + previous ordering spent that budget on several part-time spellings of the first role + and never searched later role families. """ keywords = profile.get("keywords") or {} - configured = [str(term).strip() for term in (keywords.get("search") or {}) if str(term).strip()] - # Mixed profiles intentionally accept both arrangements. Their configured - # provider phrases must not be rewritten into part-time-only searches. - if not profile_targets_part_time(profile) or profile_targets_full_time(profile): - return list(dict.fromkeys(configured or list((keywords.get("title") or {}).keys()))) - + raw_configured = configured_terms if configured_terms is not None else list((keywords.get("search") or {}).keys()) + configured = [str(term).strip() for term in raw_configured if str(term).strip()] title_terms = list((keywords.get("title") or {}).keys()) - requested_families = role_families_for_terms([*title_terms, *configured]) + intent_terms = configured if configured_terms is not None else [*title_terms, *configured] + requested_families = role_families_for_terms(intent_terms) student_targeted = _profile_targets_students(profile) - if student_targeted and requested_families in ([], ["logistics"], ["procurement", "logistics"]): + if ( + student_targeted + and not profile_targets_full_time(profile) + and requested_families in ([], ["logistics"], ["procurement", "logistics"]) + ): priority = [ "werkstudent supply chain", "part time supply chain", @@ -123,31 +146,39 @@ def search_terms_for_profile(profile: dict) -> list[str]: ] return list(dict.fromkeys([*priority, *configured])) - generated = list(configured) - query_batches = [] - preferred_german_roles = { - "quality": "qualitätskontrolle", - "production": "produktionsassistenz", - "planning": "arbeitsvorbereitung", - "process": "prozessoptimierung", - "technical office": "technische sachbearbeitung", - "procurement": "einkauf", - "logistics": "logistik", - } + base_configured = list( + dict.fromkeys(term for value in configured if (term := _base_provider_query(value, profile))) + ) + generated: list[str] = [] + covered_families: set[str] = set() + + # Keep the first configured phrase for each family and every unknown/custom role. + # Repeated synonyms from one family move behind this coverage pass instead of + # consuming the provider's complete query budget. + for query in base_configured: + families = role_families_for_terms([query]) + if not families or any(family not in covered_families for family in families): + generated.append(query) + covered_families.update(families) for family in requested_families: - aliases = ROLE_FAMILIES[family] - english = next((alias for alias in aliases if alias.isascii() and " " in alias), aliases[0]) - german = preferred_german_roles.get(family) or next((alias for alias in aliases if not alias.isascii()), "") - queries = [f"{german or english} teilzeit", f"{english} part time"] - if german: - queries.append(f"{german} minijob") - if student_targeted: - queries.append(f"werkstudent {german or english}") - query_batches.append(queries) - for index in range(max((len(batch) for batch in query_batches), default=0)): - generated.extend(batch[index] for batch in query_batches if index < len(batch)) + if family not in covered_families: + generated.append(ROLE_QUERY_TERMS.get(family, ROLE_FAMILIES[family][:2])[0]) + covered_families.add(family) + + # Add the other language, then any remaining user phrases, before narrower + # part-time variants. This preserves user intent without starving later roles. + for family in requested_families: + for query in ROLE_QUERY_TERMS.get(family, ROLE_FAMILIES[family][:2]): + if query not in generated: + generated.append(query) + generated.extend(base_configured) + generated.extend(configured) + if profile_targets_part_time(profile) and not profile_targets_full_time(profile): + for family in requested_families: + english, german = ROLE_QUERY_TERMS.get(family, ROLE_FAMILIES[family][:2]) + generated.extend((f"{german} teilzeit", f"{english} part time", f"{german} minijob")) if not generated: - generated.extend(f"{term} teilzeit" for term in title_terms) + generated.extend(title_terms) return list(dict.fromkeys(generated)) @@ -162,42 +193,46 @@ def _weekly_hours(text: str) -> int | None: return min(values) if values else None -def assess_employment_fit(job: Job, profile: dict) -> tuple[bool, str, list[str]]: - """Hard employment-format gate for profiles that explicitly target part-time work. +def assess_employment_fit(job: Job, profile: dict, strict: bool = True) -> tuple[bool, str, list[str]]: + """Classify employment format, optionally enforcing it as a hard gate. A positive part-time/student signal wins over generic full-time boilerplate. For a - strict part-time profile, explicit full-time jobs and jobs with no confirmable target - format are not recommended. Other profiles keep the previous permissive behaviour. + strict part-time search, explicit full-time jobs and jobs with no confirmable target + format are rejected. In preference mode they remain visible as stretch results. """ - if not profile_targets_part_time(profile) or profile_targets_full_time(profile): - return True, "not_restricted", [] - + targets_part_time = profile_targets_part_time(profile) + targets_full_time = profile_targets_full_time(profile) title = _norm(job.title) body = _norm(f"{job.title} {job.description}") configured = _profile_format_terms(profile) + student_only = any(contains_affirmed_phrase(body, term) for term in STUDENT_SIGNALS) - positive_terms = tuple(dict.fromkeys((*configured, *PART_TIME_SIGNALS))) - positive_title = any(contains_affirmed_phrase(title, term) for term in positive_terms) - positive_body = positive_title or any(contains_affirmed_phrase(body, term) for term in positive_terms) + # Student vacancies are a genuine eligibility constraint, independent of the + # full-time/part-time preference selected on the profile. + if student_only and not _profile_targets_students(profile): + return False, "student_only", ["employment mismatch: enrolled student required"] + if targets_part_time == targets_full_time: + return True, "not_restricted", [] + + part_time_terms = ( + tuple(dict.fromkeys((*configured, *PART_TIME_SIGNALS))) if targets_part_time else PART_TIME_SIGNALS + ) + part_time_title = any(contains_affirmed_phrase(title, term) for term in part_time_terms) + part_time = part_time_title or any(contains_affirmed_phrase(body, term) for term in part_time_terms) full_time = any(contains_affirmed_phrase(body, term) for term in FULL_TIME_SIGNALS) weekly_hours = _weekly_hours(body) workload_match = WORKLOAD_PATTERN.search(body) workload = int(workload_match.group(1) or workload_match.group(2)) if workload_match else None - student_only = any(contains_affirmed_phrase(title, term) for term in STUDENT_SIGNALS) - - if student_only and not _profile_targets_students(profile): - return False, "student_only", ["employment mismatch: enrolled student required"] - if weekly_hours is not None and weekly_hours <= 32: - positive_body = True + part_time = True if workload is not None and workload <= 80: - positive_body = True - if weekly_hours is not None and weekly_hours >= 35 and not positive_body: + part_time = True + if weekly_hours is not None and weekly_hours >= 35 and not part_time: full_time = True - if workload is not None and workload >= 90 and not positive_body: + if workload is not None and workload >= 90 and not part_time: full_time = True - if positive_body: + if targets_part_time and part_time: reasons = ["employment: part-time/student confirmed"] if weekly_hours is not None: reasons.append(f"schedule: {weekly_hours} hours/week") @@ -208,6 +243,16 @@ def assess_employment_fit(job: Job, profile: dict) -> tuple[bool, str, list[str] ): reasons.append("schedule: afternoon/flexible") return True, "part_time", reasons + if targets_part_time and full_time: + reasons = ["employment mismatch: full-time"] + return (not strict), "full_time", reasons + if targets_part_time: + reasons = ["employment mismatch: part-time/minijob not confirmed"] + return (not strict), "unclear", reasons + if full_time: - return False, "full_time", ["employment mismatch: full-time"] - return False, "unclear", ["employment mismatch: part-time/minijob not confirmed"] + return True, "full_time", ["employment: full-time confirmed"] + if part_time: + reasons = ["employment mismatch: part-time/student"] + return (not strict), "part_time", reasons + return (not strict), "unclear", ["employment mismatch: full-time not confirmed"] diff --git a/app/job_metadata.py b/app/job_metadata.py index cdda60b..76cf76e 100644 --- a/app/job_metadata.py +++ b/app/job_metadata.py @@ -32,6 +32,7 @@ "production": "Production", "planning": "Planning", "process": "Process engineering", + "coating": "Paint / coating", "technical office": "Technical office", "procurement": "Procurement", "logistics": "Logistics", @@ -98,6 +99,8 @@ def classify_job_metadata(job: dict, today: date | None = None) -> dict: text = f"{title} {description}" families = matched_role_families(text, list(ROLE_FAMILIES)) categories = [_CATEGORY_LABELS[family] for family in families] + if re.search(r"(?i)\b(production|produktion|fertigung|manufacturing)\b", text) and "Production" not in categories: + categories.append("Production") if _STUDENT_RE.search(text): employment_type, employment_label = "working_student", "Working student" diff --git a/app/jobspy_provider.py b/app/jobspy_provider.py index ec7643b..5c21aeb 100644 --- a/app/jobspy_provider.py +++ b/app/jobspy_provider.py @@ -69,16 +69,16 @@ async def fetch_jobspy(source: dict, search_terms: list[str], target_location: s if not terms: raise RuntimeError("JobSpy requires at least one profile-specific search phrase") - jobs = [] - seen = set() - failures = [] loop = asyncio.get_running_loop() started = loop.time() - for term in terms: - for site in sites: + + async def scrape_site(site: str): + site_jobs = [] + site_failures = [] + for term in terms: remaining = total_timeout_seconds - (loop.time() - started) if remaining <= 0: - failures.append(f"provider total timeout after {total_timeout_seconds}s") + site_failures.append(f"{site}: provider total timeout after {total_timeout_seconds}s") log.warning("JobSpy total provider timeout reached after %ss", total_timeout_seconds) break call_timeout = max(1, min(timeout_seconds, int(remaining))) @@ -87,11 +87,11 @@ async def fetch_jobspy(source: dict, search_terms: list[str], target_location: s asyncio.to_thread(_scrape_one, term, site, source, target_location), timeout=call_timeout ) except TimeoutError: - failures.append(f"{site}: timeout after {call_timeout}s") + site_failures.append(f"{site}: timeout after {call_timeout}s") log.warning("JobSpy %s timed out after %ss for query %r", site, call_timeout, term) continue except Exception as exc: - failures.append(f"{site}: {type(exc).__name__}") + site_failures.append(f"{site}: {type(exc).__name__}") log.warning("JobSpy %s failed for query %r: %s", site, term, type(exc).__name__) continue if frame is None or getattr(frame, "empty", False): @@ -122,11 +122,7 @@ async def fetch_jobspy(source: dict, search_terms: list[str], target_location: s description = "\n".join([*metadata, description]) created = _date_text(row.get("date_posted")) external_id = _text(row.get("id")) or _stable_id(row_site, title, company, url) - dedupe = f"{row_site}:{external_id}" - if dedupe in seen: - continue - seen.add(dedupe) - jobs.append( + site_jobs.append( Job( source=f"{source['name']} / {row_site}", external_id=external_id, @@ -139,8 +135,22 @@ async def fetch_jobspy(source: dict, search_terms: list[str], target_location: s remote=bool(row.get("is_remote", False)), ) ) - if loop.time() - started >= total_timeout_seconds: - break + return site_jobs, site_failures + + # One sequential worker per board avoids request bursts while ensuring a slow or + # blocked board cannot consume the complete provider budget before the others run. + results = await asyncio.gather(*(scrape_site(site) for site in sites)) + jobs = [] + failures = [] + seen = set() + for site_jobs, site_failures in results: + failures.extend(site_failures) + for job in site_jobs: + dedupe = job.key + if dedupe in seen: + continue + seen.add(dedupe) + jobs.append(job) if failures and not jobs: raise RuntimeError("JobSpy returned no jobs; " + ", ".join(dict.fromkeys(failures))) return jobs diff --git a/app/main.py b/app/main.py index b72b441..a205ab2 100644 --- a/app/main.py +++ b/app/main.py @@ -493,6 +493,7 @@ def api_jobs( decision: str = Query("active"), language: str = Query("preferred"), content_language: str = Query("profile"), + tier: str = Query("all"), profile_id: int | None = Query(None), actor: dict = Depends(require_workspace), ): @@ -502,6 +503,8 @@ def api_jobs( raise HTTPException(400, "Invalid language filter") if content_language not in ("all", "profile", "de", "en", "mixed", "unknown"): raise HTTPException(400, "Invalid content language filter") + if tier not in ("all", "strong", "match", "stretch"): + raise HTTPException(400, "Invalid match tier filter") profile = get_profile(profile_id, user_id=actor["user_id"]) if not profile: raise HTTPException(404, "Profile not found") @@ -514,6 +517,7 @@ def api_jobs( None if decision == "all" else decision, language, content_language, + tier, user_id=actor["user_id"], ), } diff --git a/app/models.py b/app/models.py index 22db4df..32e48a6 100644 --- a/app/models.py +++ b/app/models.py @@ -17,6 +17,8 @@ class Job: reasons: list[str] = field(default_factory=list) language_score: int = 55 overall_score: int = 0 + role_relevant: bool = True + match_tier: str = "match" language_label: str = "unclear" language_reasons: list[str] = field(default_factory=list) diff --git a/app/notifier.py b/app/notifier.py index 47cad8f..0718296 100644 --- a/app/notifier.py +++ b/app/notifier.py @@ -23,6 +23,7 @@ def build_text_digest(jobs: list[Job], title: str = "JobTrack") -> str: f"{i}. {job.title}", f" {job.company} | {job.location}", f" Overall {job.overall_score}/100 | Job {job.score}/100 | Language {job.language_score}/100", + f" Match tier: {str(getattr(job, 'match_tier', 'match')).title()}", f" Language: {LABELS.get(job.language_label, job.language_label)}", ] ) diff --git a/app/profile_store.py b/app/profile_store.py index 2ecb588..bb87138 100644 --- a/app/profile_store.py +++ b/app/profile_store.py @@ -39,6 +39,8 @@ job_score INTEGER NOT NULL DEFAULT 0, language_score INTEGER NOT NULL DEFAULT 55, overall_score INTEGER NOT NULL DEFAULT 0, + role_relevant INTEGER NOT NULL DEFAULT 0, + match_tier TEXT NOT NULL DEFAULT 'excluded', language_label TEXT NOT NULL DEFAULT 'unclear', reasons_json TEXT NOT NULL DEFAULT '[]', language_reasons_json TEXT NOT NULL DEFAULT '[]', @@ -188,6 +190,38 @@ def _now() -> str: return datetime.now(timezone.utc).isoformat() +def _backfill_role_relevance(con) -> None: + """Re-evaluate legacy profile scores once when the structural role gate is added.""" + from .models import Job + from .ranker import assess_role_relevance + + rows = con.execute( + """SELECT s.job_key,s.profile_id,j.source,j.title,j.company,j.location,j.url,j.description, + j.created_at,j.remote,p.keywords_json + FROM job_profile_scores s + JOIN jobs j ON j.job_key=s.job_key + JOIN search_profiles p ON p.id=s.profile_id""" + ).fetchall() + for row in rows: + keywords = json.loads(row["keywords_json"] or "{}") + job = Job( + source=row["source"], + external_id=row["job_key"], + title=row["title"], + company=row["company"], + location=row["location"], + url=row["url"], + description=row["description"], + created_at=row["created_at"], + remote=bool(row["remote"]), + ) + relevant = row["source"] == "Manual" or assess_role_relevance(job, keywords).relevant + con.execute( + "UPDATE job_profile_scores SET role_relevant=? WHERE job_key=? AND profile_id=?", + (int(relevant), row["job_key"], row["profile_id"]), + ) + + def _migrate_profile_ownership(con) -> None: columns = {row[1] for row in con.execute("PRAGMA table_info(search_profiles)").fetchall()} if not columns or "user_id" in columns: @@ -245,6 +279,18 @@ def ensure_profile_schema(user_id: int | None = None) -> None: con.execute( 'ALTER TABLE search_profiles ADD COLUMN content_languages_json TEXT NOT NULL DEFAULT \'["de","en","mixed"]\'' ) + score_columns = {row[1] for row in con.execute("PRAGMA table_info(job_profile_scores)").fetchall()} + if "role_relevant" not in score_columns: + con.execute("ALTER TABLE job_profile_scores ADD COLUMN role_relevant INTEGER NOT NULL DEFAULT 0") + _backfill_role_relevance(con) + if "match_tier" not in score_columns: + con.execute("ALTER TABLE job_profile_scores ADD COLUMN match_tier TEXT NOT NULL DEFAULT 'excluded'") + con.execute( + """UPDATE job_profile_scores SET match_tier=CASE + WHEN role_relevant=0 THEN 'excluded' + WHEN overall_score>=75 THEN 'strong' + ELSE 'match' END""" + ) for row in con.execute("SELECT id,keywords_json FROM search_profiles").fetchall(): keywords = json.loads(row["keywords_json"] or "{}") changed = False @@ -426,17 +472,30 @@ def delete_profile(profile_id: int, user_id: int | None = None) -> None: con.execute("DELETE FROM search_profiles WHERE id=? AND user_id IS ?", (profile_id, user_id)) -def upsert_profile_score(job, profile_id: int) -> None: +def upsert_profile_score( + job, + profile_id: int, + role_relevant: bool | None = None, + match_tier: str | None = None, +) -> None: + if role_relevant is None: + role_relevant = bool(getattr(job, "role_relevant", True)) + if match_tier is None: + match_tier = str(getattr(job, "match_tier", "match" if role_relevant else "excluded")) + if match_tier not in ("strong", "match", "stretch", "excluded"): + match_tier = "match" if role_relevant else "excluded" with connection() as con: con.execute( - """INSERT INTO job_profile_scores(job_key,profile_id,job_score,language_score,overall_score,language_label,reasons_json,language_reasons_json,updated_at) - VALUES(?,?,?,?,?,?,?,?,?) ON CONFLICT(job_key,profile_id) DO UPDATE SET job_score=excluded.job_score,language_score=excluded.language_score,overall_score=excluded.overall_score,language_label=excluded.language_label,reasons_json=excluded.reasons_json,language_reasons_json=excluded.language_reasons_json,updated_at=excluded.updated_at""", + """INSERT INTO job_profile_scores(job_key,profile_id,job_score,language_score,overall_score,role_relevant,match_tier,language_label,reasons_json,language_reasons_json,updated_at) + VALUES(?,?,?,?,?,?,?,?,?,?,?) ON CONFLICT(job_key,profile_id) DO UPDATE SET job_score=excluded.job_score,language_score=excluded.language_score,overall_score=excluded.overall_score,role_relevant=excluded.role_relevant,match_tier=excluded.match_tier,language_label=excluded.language_label,reasons_json=excluded.reasons_json,language_reasons_json=excluded.language_reasons_json,updated_at=excluded.updated_at""", ( job.key, profile_id, job.score, job.language_score, job.overall_score, + int(role_relevant), + match_tier, job.language_label, json.dumps(job.reasons, ensure_ascii=False), json.dumps(job.language_reasons, ensure_ascii=False), @@ -459,6 +518,7 @@ def list_jobs_for_profile( decision: str = "active", language: str = "preferred", content_language: str = "profile", + tier: str = "all", user_id: int | None = None, ) -> list[dict[str, Any]]: ensure_profile_schema(user_id) @@ -466,7 +526,7 @@ def list_jobs_for_profile( if not profile: return [] owner_key = "admin" if user_id is None else f"user:{int(user_id)}" - where = ["s.profile_id=?", "s.overall_score>=?"] + where = ["s.profile_id=?", "s.role_relevant=1", "s.match_tier!='excluded'", "s.overall_score>=?"] params: list[Any] = [profile_id, min_score] if decision == "active": where.append("COALESCE(js.decision,'unreviewed')!='skip'") @@ -486,13 +546,16 @@ def list_jobs_for_profile( elif content_language in ("de", "en", "mixed", "unknown"): where.append("j.content_language=?") params.append(content_language) + if tier in ("strong", "match", "stretch"): + where.append("s.match_tier=?") + params.append(tier) params = [owner_key, owner_key, *params, limit] with connection() as con: rows = con.execute( f"""SELECT j.job_key,j.source,j.title,j.company,j.location,j.url,j.description,j.created_at,j.first_seen,j.remote, COALESCE(js.decision,'unreviewed') AS decision,js.decision_at, j.content_language,j.content_language_confidence,j.content_language_source, - s.job_score AS score,s.language_score,s.overall_score,s.language_label,s.reasons_json,s.language_reasons_json, + s.job_score AS score,s.language_score,s.overall_score,s.role_relevant,s.match_tier,s.language_label,s.reasons_json,s.language_reasons_json, a.status AS application_status,a.applied_at FROM job_profile_scores s JOIN jobs j ON j.job_key=s.job_key LEFT JOIN user_job_state js ON js.owner_key=? AND js.job_key=j.job_key @@ -509,6 +572,7 @@ def list_jobs_for_profile( item.update(classify_job_metadata(item)) item.pop("description", None) item["remote"] = bool(item["remote"]) + item["role_relevant"] = bool(item["role_relevant"]) out.append(item) return out @@ -525,13 +589,13 @@ def get_job_for_profile(job_key: str, profile_id: int, user_id: int | None = Non j.first_seen,j.remote,j.content_language,j.content_language_confidence, j.content_language_source,COALESCE(js.decision,'unreviewed') AS decision, js.decision_at,s.job_score AS score,s.language_score,s.overall_score, - s.language_label,s.reasons_json,s.language_reasons_json, + s.role_relevant,s.match_tier,s.language_label,s.reasons_json,s.language_reasons_json, a.status AS application_status,a.applied_at FROM job_profile_scores s JOIN jobs j ON j.job_key=s.job_key LEFT JOIN user_job_state js ON js.owner_key=? AND js.job_key=j.job_key LEFT JOIN applications a ON a.owner_key=? AND a.job_key=j.job_key - WHERE s.profile_id=? AND j.job_key=?""", + WHERE s.profile_id=? AND s.role_relevant=1 AND s.match_tier!='excluded' AND j.job_key=?""", (owner_key, owner_key, profile_id, job_key), ).fetchone() if not row: @@ -541,4 +605,5 @@ def get_job_for_profile(job_key: str, profile_id: int, user_id: int | None = Non item["language_reasons"] = json.loads(item.pop("language_reasons_json") or "[]") item.update(classify_job_metadata(item)) item["remote"] = bool(item["remote"]) + item["role_relevant"] = bool(item["role_relevant"]) return item diff --git a/app/providers.py b/app/providers.py index c6dadce..00f3817 100644 --- a/app/providers.py +++ b/app/providers.py @@ -364,17 +364,24 @@ async def _fetch_with_resilience(provider, source: dict, search_terms: list[str] async def fetch_all_jobs( sources: list[dict], search_terms: list[str], target_location: str ) -> tuple[list[Job], list[str]]: - jobs = [] - errors = [] - for source in sources: + async def fetch_source(source: dict) -> tuple[list[Job], str | None]: provider = PROVIDERS.get(source["source_type"]) if not provider: - errors.append(f"{source['name']}: unsupported source type {source['source_type']}") - continue + return [], f"{source['name']}: unsupported source type {source['source_type']}" try: - jobs.extend(await _fetch_with_resilience(provider, source, search_terms, target_location)) + return await _fetch_with_resilience(provider, source, search_terms, target_location), None except Exception as exc: - errors.append(f"{source['name']}: {_safe_provider_error(source, exc)}") + return [], f"{source['name']}: {_safe_provider_error(source, exc)}" + + jobs = [] + errors = [] + # Sources are independent. Running them together prevents a slow experimental + # scraper from delaying stable API/feed results until the request or scheduler + # budget has already expired. asyncio.gather preserves configured source order. + for source_jobs, error in await asyncio.gather(*(fetch_source(source) for source in sources)): + jobs.extend(source_jobs) + if error: + errors.append(error) if errors and not jobs: raise RuntimeError("; ".join(errors)) return jobs, errors diff --git a/app/ranker.py b/app/ranker.py index 3ddbb3d..ba77a94 100644 --- a/app/ranker.py +++ b/app/ranker.py @@ -1,7 +1,14 @@ import re +from dataclasses import dataclass from .models import Job -from .search_intent import matched_role_families, role_families_for_terms +from .search_intent import ( + CONFLICTING_TITLE_SIGNALS, + INDUSTRIAL_DOMAIN_SIGNALS, + PROFESSIONAL_TITLE_SIGNALS, + matched_role_families, + role_families_for_terms, +) from .text_match import contains_affirmed_phrase, contains_phrase, normalize_text @@ -24,39 +31,175 @@ def blocklist_matches(job: Job, keywords: dict[str, dict[str, int]]) -> list[str return matches -def score_job(job: Job, keywords: dict[str, dict[str, int]], location_terms: list[str]) -> tuple[int, list[str]]: +@dataclass(frozen=True) +class RoleAssessment: + relevant: bool + confidence: str + requested_families: tuple[str, ...] + matched_families: tuple[str, ...] + reasons: tuple[str, ...] + + +def classify_match_tier( + *, + role_relevant: bool, + eligible: bool, + overall_score: int, + employment_constraint: bool, + language_label: str, + evidence_constraint: bool = False, +) -> str: + """Map independent matching decisions to a stable review label.""" + if not role_relevant: + return "excluded" + has_constraint = ( + employment_constraint + or evidence_constraint + or language_label + in ( + "stretch", + "german_heavy", + ) + ) + if not eligible or has_constraint: + return "stretch" + return "strong" if overall_score >= 75 else "match" + + +def _matching_terms(text: str, terms) -> list[str]: + matches: list[str] = [] + for value in terms: + term = _normalise(str(value)) + if term and contains_affirmed_phrase(text, term) and term not in matches: + matches.append(term) + return matches + + +def assess_role_relevance( + job: Job, + keywords: dict, + intent_terms=(), + restrict_to_intent: bool = False, +) -> RoleAssessment: + """Require occupational evidence before softer signals can influence ranking. + + Location, language, work arrangement, and skills are useful ranking dimensions, + but none of them proves that a vacancy belongs to the requested occupation. Direct + title evidence is preferred; a generic professional title can fall back to strong + role-family evidence in the description. + """ + title = _normalise(job.title) + body = _normalise(f"{job.title} {job.description}") + intent_terms = tuple(intent_terms) + title_rules = list((keywords.get("title") or {}).keys()) + search_rules = list((keywords.get("search") or {}).keys()) + role_scope = list(intent_terms) if restrict_to_intent else [*title_rules, *search_rules, *intent_terms] + title_evidence = list(intent_terms) if restrict_to_intent else [*title_rules, *search_rules, *intent_terms] + requested = role_families_for_terms(role_scope) + title_hits = _matching_terms(title, title_evidence) + title_families = matched_role_families(title, requested) + body_families = matched_role_families(body, requested) + professional_title = bool(_matching_terms(title, PROFESSIONAL_TITLE_SIGNALS)) + conflict_hits = _matching_terms(title, CONFLICTING_TITLE_SIGNALS) + industrial_hits = _matching_terms(body, INDUSTRIAL_DOMAIN_SIGNALS) + + reasons: list[str] = [] + if title_hits: + reasons.append(f"role title evidence: {', '.join(title_hits[:3])}") + if title_families: + reasons.append(f"role family evidence: {', '.join(title_families)}") + + # A title such as "Software Quality Engineer" contains a valid family phrase but + # belongs to a different occupational domain. Two independent industrial signals + # are enough to keep legitimate digital/manufacturing crossover roles. + conflict_explicitly_requested = any( + contains_affirmed_phrase(str(term), conflict) for term in role_scope for conflict in conflict_hits + ) + if conflict_hits and not conflict_explicitly_requested and len(industrial_hits) < 2: + reasons.append(f"conflicting occupation: {', '.join(conflict_hits)}") + return RoleAssessment(False, "conflict", tuple(requested), tuple(title_families), tuple(reasons)) + + if title_hits or title_families: + return RoleAssessment(True, "direct", tuple(requested), tuple(title_families), tuple(reasons)) + + description_only = [family for family in body_families if family not in title_families] + industrial_scope = bool({"quality", "production", "planning", "process", "coating"}.intersection(requested)) + if professional_title and description_only and (industrial_hits or not industrial_scope): + reasons.append(f"role description evidence: {', '.join(description_only)}") + return RoleAssessment(True, "supported", tuple(requested), tuple(description_only), tuple(reasons)) + + # Profiles created before role-title rules existed may intentionally be skill-only. + # Preserve their behaviour, but do not use this fallback when an occupation was + # explicitly configured and simply failed to match. + if not title_evidence and not requested: + reasons.append("role evidence: legacy skill-only profile") + return RoleAssessment(True, "legacy", (), (), tuple(reasons)) + + reasons.append("role evidence missing from title") + return RoleAssessment(False, "none", tuple(requested), (), tuple(reasons)) + + +def score_job( + job: Job, + keywords: dict[str, dict[str, int]], + location_terms: list[str], + intent_terms=(), + restrict_to_intent: bool = False, +) -> tuple[int, list[str]]: """Return a 0-100 role/skill fit score, independent from language fit.""" title = _normalise(job.title) body = _normalise(f"{job.title} {job.description}") location = _normalise(job.location) + intent_terms = tuple(intent_terms) score = 0 reasons: list[str] = [] + title_points = 0 for term, weight in keywords.get("title", {}).items(): if contains_affirmed_phrase(title, term): - score += weight + title_points += int(weight) reasons.append(f"title: {term}") - requested_families = role_families_for_terms((keywords.get("title") or {}).keys()) - for family in matched_role_families(title, requested_families): - if not any(reason.startswith("title:") for reason in reasons): - score += 20 + score += min(48, max(0, title_points)) + role_scope = ( + list(intent_terms) + if restrict_to_intent + else [*(keywords.get("title") or {}).keys(), *(keywords.get("search") or {}).keys(), *intent_terms] + ) + requested_families = role_families_for_terms(role_scope) + family_matches = matched_role_families(title, requested_families) + if family_matches: + score += min(36, 28 + (len(family_matches) - 1) * 4) + for family in family_matches: reasons.append(f"role family: {family}") + query_evidence = ( + list(intent_terms) if restrict_to_intent else [*(keywords.get("search") or {}).keys(), *intent_terms] + ) + query_title_hits = _matching_terms(title, query_evidence) + if not title_points and not family_matches and query_title_hits: + score += 30 + reasons.append(f"query role: {query_title_hits[0]}") + format_points = 0 for term, weight in keywords.get("format", {}).items(): if contains_affirmed_phrase(body, term): - score += weight + format_points += int(weight) reasons.append(f"format: {term}") + score += min(34, max(0, format_points)) + skill_points = 0 for term, weight in keywords.get("skill", {}).items(): if contains_affirmed_phrase(body, term): - score += weight + skill_points += int(weight) if len(reasons) < 8: reasons.append(f"skill: {term}") + score += min(28, max(0, skill_points)) + allowlist_points = 0 for value, weight in (keywords.get("allowlist") or {}).items(): term = _normalise(str(value)) boost = max(0, int(weight)) if term and contains_affirmed_phrase(body, term) and boost: - score += boost + allowlist_points += boost if len(reasons) < 8: reasons.append(f"allowlist: {term}") + score += min(30, allowlist_points) for term, penalty in keywords.get("negative", {}).items(): if contains_affirmed_phrase(title, term): score += penalty diff --git a/app/review-ui.js b/app/review-ui.js index 3779258..6f8b9ec 100644 --- a/app/review-ui.js +++ b/app/review-ui.js @@ -25,13 +25,14 @@
+
`;main.appendChild(section);button.addEventListener('click',()=>{document.querySelectorAll('.section').forEach(x=>x.classList.remove('active'));document.querySelectorAll('.nav button[data-tab]').forEach(x=>x.classList.remove('active'));section.classList.add('active');button.classList.add('active');if($('pageTitle'))$('pageTitle').textContent='Job Review';window.loadReviewJobs()}); const overview=$('overview');const oldTitle=overview?[...overview.querySelectorAll('.section-title')].find(x=>x.textContent.includes('Latest matches')):null;const oldTable=oldTitle?.nextElementSibling;if(oldTitle)oldTitle.remove();if(oldTable?.classList.contains('table-wrap'))oldTable.remove(); - ['reviewDecision','reviewLanguage','reviewContentLanguage','reviewMin'].forEach(id=>$(id).addEventListener('change',window.loadReviewJobs));$('reviewReload').addEventListener('click',window.loadReviewJobs);$('reviewProfile').addEventListener('change',()=>{window.activeProfileId=+$('reviewProfile').value;try{localStorage.setItem('jobtrack-profile',String(window.activeProfileId))}catch(e){};window.loadReviewJobs();if($('learning')?.classList.contains('active'))window.loadLearning()});$('jobDetailClose').addEventListener('click',window.closeJobDetail);$('jobDetailBackdrop').addEventListener('click',event=>{if(event.target===$('jobDetailBackdrop'))window.closeJobDetail()}); + ['reviewDecision','reviewTier','reviewLanguage','reviewContentLanguage','reviewMin'].forEach(id=>$(id).addEventListener('change',window.loadReviewJobs));$('reviewReload').addEventListener('click',window.loadReviewJobs);$('reviewProfile').addEventListener('change',()=>{window.activeProfileId=+$('reviewProfile').value;try{localStorage.setItem('jobtrack-profile',String(window.activeProfileId))}catch(e){};window.loadReviewJobs();if($('learning')?.classList.contains('active'))window.loadLearning()});$('jobDetailClose').addEventListener('click',window.closeJobDetail);$('jobDetailBackdrop').addEventListener('click',event=>{if(event.target===$('jobDetailBackdrop'))window.closeJobDetail()}); } function installLearning(){const nav=document.querySelector('.nav'),main=document.querySelector('.main');if(!nav||!main||$('learning'))return;const b=document.createElement('button');b.dataset.tab='learning';b.textContent='Learning';nav.appendChild(b);const s=document.createElement('section');s.id='learning';s.className='section';s.innerHTML=` @@ -42,16 +43,16 @@ window.loadReviewProfiles=async function(){const d=await api('/api/profiles');const sel=$('reviewProfile');if(!sel)return;let saved=null;try{saved=+localStorage.getItem('jobtrack-profile')}catch(e){};const active=d.profiles.find(p=>p.id===saved&&p.enabled)||d.profiles.find(p=>p.is_default)||d.profiles.find(p=>p.enabled)||d.profiles[0];window.activeProfileId=active?.id||null;sel.innerHTML=d.profiles.length?d.profiles.filter(p=>p.enabled).map(p=>``).join(''):'';if(active)$('reviewMin').value=active.min_score??35;return d.profiles}; - window.loadReviewJobs=async function(){if(!window.activeProfileId)await window.loadReviewProfiles();const grid=$('jobReviewGrid');if(!grid)return;if(!window.activeProfileId){grid.innerHTML='
Create a search profile to start reviewing jobs.
';return}const q=new URLSearchParams({limit:'150',min_score:$('reviewMin')?.value||'35',decision:$('reviewDecision')?.value||'active',language:$('reviewLanguage')?.value||'preferred',content_language:$('reviewContentLanguage')?.value||'profile',profile_id:String(window.activeProfileId)});const d=await api('/api/jobs?'+q);grid.innerHTML=d.jobs.length?d.jobs.map((j,i)=>`
+ window.loadReviewJobs=async function(){if(!window.activeProfileId)await window.loadReviewProfiles();const grid=$('jobReviewGrid');if(!grid)return;if(!window.activeProfileId){grid.innerHTML='
Create a search profile to start reviewing jobs.
';return}const q=new URLSearchParams({limit:'150',min_score:$('reviewMin')?.value||'35',decision:$('reviewDecision')?.value||'active',tier:$('reviewTier')?.value||'all',language:$('reviewLanguage')?.value||'preferred',content_language:$('reviewContentLanguage')?.value||'profile',profile_id:String(window.activeProfileId)});const d=await api('/api/jobs?'+q);grid.innerHTML=d.jobs.length?d.jobs.map((j,i)=>`
${esc(j.title)}
${esc(j.company||'Company not stated')} · ${esc(j.location||'Location not stated')}
${j.overall_score}
- ${j.overall_score>=75?'
Strong profile match
':''}
${esc(j.employment_label||'Work time unknown')}${esc(j.freshness_label||'Date unknown')}
+ ${j.match_tier==='strong'?'
Strong role match
':j.match_tier==='stretch'?'
Relevant role · constraint to review
':''}
${esc(j.match_tier||'match')}${esc(j.employment_label||'Work time unknown')}${esc(j.freshness_label||'Date unknown')}
Click for scores, description and listing link
`).join(''):'
No jobs match this profile and filters.
'}; let detailTrigger=null; - window.openJobDetail=async function(jobKey,trigger){detailTrigger=trigger||document.activeElement;const backdrop=$('jobDetailBackdrop'),body=$('jobDetailBody');backdrop.hidden=false;document.body.classList.add('jt-dialog-open');$('jobDetailTitle').textContent='Loading job…';$('jobDetailSubtitle').textContent='';body.innerHTML='
Loading details…
';$('jobDetailClose').focus();try{const d=await api(`/api/jobs/${encodeURIComponent(jobKey)}/detail?profile_id=${encodeURIComponent(window.activeProfileId)}`),j=d.job,description=j.description||j.description_preview;$('jobDetailTitle').textContent=j.title||'Job details';$('jobDetailSubtitle').textContent=`${j.company||'Company not stated'} · ${j.location||'Location not stated'}`;body.innerHTML=`
${j.overall_score}Overall fit
${j.score}Job fit
${j.language_score}Language fit
${esc(j.employment_label||'Unknown')}Working time
${esc(j.schedule_label||'Unknown')}Schedule
${esc(j.primary_category||'Other')}Category
${esc(j.published_date||'Unknown')}Published
${esc(j.postal_code||'—')}Postal code
${j.data_quality||0}%Data completeness
${esc(j.source||'Unknown')}Source

Match evidence

${(j.reasons||[]).map(x=>`${esc(x)}`).join('')||'No scoring evidence available.'}
${(j.language_reasons||[]).length?`
${j.language_reasons.map(esc).join(' · ')}
`:''}

Job description

${esc(description||'No description was provided by the source.')}
Open original listing ↗
`}catch(error){body.innerHTML=`
${esc(error.message)}
`}}; + window.openJobDetail=async function(jobKey,trigger){detailTrigger=trigger||document.activeElement;const backdrop=$('jobDetailBackdrop'),body=$('jobDetailBody');backdrop.hidden=false;document.body.classList.add('jt-dialog-open');$('jobDetailTitle').textContent='Loading job…';$('jobDetailSubtitle').textContent='';body.innerHTML='
Loading details…
';$('jobDetailClose').focus();try{const d=await api(`/api/jobs/${encodeURIComponent(jobKey)}/detail?profile_id=${encodeURIComponent(window.activeProfileId)}`),j=d.job,description=j.description||j.description_preview;$('jobDetailTitle').textContent=j.title||'Job details';$('jobDetailSubtitle').textContent=`${j.company||'Company not stated'} · ${j.location||'Location not stated'}`;body.innerHTML=`
${esc(j.match_tier||'match')}Match tier
${j.overall_score}Overall fit
${j.score}Job fit
${j.language_score}Language fit
${esc(j.employment_label||'Unknown')}Working time
${esc(j.schedule_label||'Unknown')}Schedule
${esc(j.primary_category||'Other')}Category
${esc(j.published_date||'Unknown')}Published
${esc(j.postal_code||'—')}Postal code
${j.data_quality||0}%Data completeness
${esc(j.source||'Unknown')}Source

Match evidence

${(j.reasons||[]).map(x=>`${esc(x)}`).join('')||'No scoring evidence available.'}
${(j.language_reasons||[]).length?`
${j.language_reasons.map(esc).join(' · ')}
`:''}

Job description

${esc(description||'No description was provided by the source.')}
Open original listing ↗
`}catch(error){body.innerHTML=`
${esc(error.message)}
`}}; window.closeJobDetail=function(){const backdrop=$('jobDetailBackdrop');if(!backdrop||backdrop.hidden)return;backdrop.hidden=true;document.body.classList.remove('jt-dialog-open');detailTrigger?.focus?.();detailTrigger=null}; document.addEventListener('keydown',event=>{if(event.key==='Escape'&&!$('jobDetailBackdrop')?.hidden)window.closeJobDetail()}); diff --git a/app/search-job-ui.js b/app/search-job-ui.js index e77ea0a..72734d1 100644 --- a/app/search-job-ui.js +++ b/app/search-job-ui.js @@ -43,8 +43,8 @@

General

Name the job and choose the profile that supplies its base values.
Optional CV Match intelligence for eligible results.
-

Search

Inherit profile defaults or replace them only for this job.
Inherited values update automatically when the selected profile changes. Custom values stay isolated to this job.
One phrase per line. These are provider queries, not scoring rules.
-

Filters and scoring

Profile rules are inherited unless you explicitly replace a section.
Applied only when a candidate profile is assigned.

Allowlist

A match can only add points. It never forces inclusion.

Blacklist

Any match excludes the vacancy from this job.
Blacklist terms are hard filters. Matching vacancies are not scored, stored for this profile, or notified.
+

Search

Inherit profile defaults or replace them only for this job.
Inherited values update automatically when the selected profile changes. Custom values stay isolated to this job.
One phrase per line. Bert runs unqualified bilingual role queries before schedule-specific variants so capped providers cover every role.
+

Filters and scoring

Profile rules are inherited unless you explicitly replace a section.
Preference mode keeps strong role matches visible when hours differ or are missing.
Applied only when a candidate profile is assigned.

Allowlist

A match can only add points. It never forces inclusion.

Blacklist

Any match excludes the vacancy from this job.
Blacklist terms are hard filters. Matching vacancies are not scored, stored for this profile, or notified.

Sources

Choose none to use every enabled automatic source.

Notifications

Blank override fields use the global notification settings.

Schedule

Manual jobs can still be started with Run now.
@@ -197,6 +197,7 @@ $('sjMin').value = job?.min_score_override ?? selectedProfile()?.min_score ?? 35; $('sjLangMinInherit').checked = job?.min_language_score_override === null || job?.min_language_score_override === undefined; $('sjLangMin').value = job?.min_language_score_override ?? selectedProfile()?.min_language_score ?? 40; + $('sjEmploymentMode').value = job?.employment_mode || 'prefer'; $('sjCvMin').value = job?.min_cv_match ?? 58; $('sjMax').value = job?.max_results ?? 20; $('sjAllowInherit').checked = job?.allowlist_terms === null || job?.allowlist_terms === undefined; @@ -269,6 +270,7 @@ interval_hours: Number($('sjInterval').value) || 12, min_score_override: $('sjMinInherit').checked ? null : Number($('sjMin').value), min_language_score_override: $('sjLangMinInherit').checked ? null : Number($('sjLangMin').value), + employment_mode: $('sjEmploymentMode').value, min_cv_match: Number($('sjCvMin').value) || 0, max_results: Number($('sjMax').value) || 20, notify_telegram: $('sjTelegram').checked, @@ -292,7 +294,7 @@ allowlist_boost: job.allowlist_boost ?? 15, source_ids: job.source_ids || [], frequency: job.frequency, day_of_week: job.day_of_week, hour: job.hour, minute: job.minute, interval_hours: job.interval_hours, min_score_override: job.min_score_override, - min_language_score_override: job.min_language_score_override, max_results: job.max_results, + min_language_score_override: job.min_language_score_override, employment_mode: job.employment_mode || 'prefer', max_results: job.max_results, min_cv_match: job.min_cv_match ?? 58, notify_telegram: job.notify_telegram, notify_email: job.notify_email, notification: job.notification || {}, secrets: job.secrets || {}, ...changes, diff --git a/app/search_intent.py b/app/search_intent.py index 36fd2c3..343d20c 100644 --- a/app/search_intent.py +++ b/app/search_intent.py @@ -6,14 +6,35 @@ ROLE_FAMILIES: dict[str, tuple[str, ...]] = { "quality": ( "quality engineer", + "supplier quality engineer", + "customer quality engineer", + "plant quality engineer", + "production quality engineer", + "quality assurance engineer", + "quality planning engineer", + "apqp engineer", + "quality manager", + "quality specialist", "quality assurance", "quality control", "quality inspection", "quality inspector", - "quality assistant", "quality technician", "qualitätsingenieur", "qualitaetsingenieur", + "supplier quality", + "qualitätsmanager", + "qualitaetsmanager", + "qualitätsspezialist", + "qualitaetsspezialist", + "qualitätsplanung", + "qualitaetsplanung", + "qualitätsplaner", + "qualitaetsplaner", + "lieferantenqualitätsingenieur", + "lieferantenqualitaetsingenieur", + "kundenqualitätsingenieur", + "kundenqualitaetsingenieur", "qualitätssicherung", "qualitaetssicherung", "qualitätskontrolle", @@ -25,45 +46,94 @@ "wareneingangsprüfung", ), "production": ( - "production", - "production assistant", - "production support", + "production engineer", + "manufacturing engineer", + "industrial engineer", + "production manager", "production coordinator", "production technician", - "manufacturing assistant", - "produktionsassistenz", - "produktionsmitarbeiter", - "produktionshelfer", - "mitarbeiter produktion", - "fertigungsmitarbeiter", - "fertigungsassistenz", - "produktion", - "fertigung", + "manufacturing specialist", + "produktionsingenieur", + "produktionsmanager", + "produktionskoordinator", + "produktionstechniker", + "fertigungsingenieur", + "fertigungsleiter", + "fertigungskoordinator", + "manufacturing engineering", ), "planning": ( "production planner", "production planning", "production scheduling", - "planning assistant", + "manufacturing planner", + "production scheduler", "material planning", "produktionsplanung", "produktionsplaner", "fertigungsplanung", + "fertigungsplaner", + "fertigungssteuerung", + "fertigungssteuerer", "arbeitsvorbereitung", - "disposition", + "arbeitsvorbereiter", ), "process": ( "process engineer", "process engineering", + "manufacturing process engineer", + "production process engineer", + "process development engineer", + "process improvement engineer", + "process specialist", + "industrialization engineer", + "industrialisation engineer", "process optimization", "continuous improvement", + "operational excellence", + "lean engineer", "prozessingenieur", + "prozessentwickler", + "prozessentwicklungsingenieur", + "prozessoptimierungsingenieur", + "prozesstechniker", "prozessoptimierung", "prozessplanung", "prozesstechnik", + "verfahrenstechniker", + "industrialisierungsingenieur", + "fertigungsprozessingenieur", + ), + "coating": ( + "coating engineer", + "paint engineer", + "painting engineer", + "paint shop engineer", + "paint process engineer", + "paint quality engineer", + "coating process engineer", + "surface treatment engineer", + "surface technology engineer", + "coating specialist", + "lackieringenieur", + "lackierprozessingenieur", + "lackiertechniker", + "lackierspezialist", + "lackierplaner", + "lackierprozess", + "beschichtungsingenieur", + "beschichtungstechniker", + "beschichtungstechnik", + "oberflächentechnik", + "oberflaechentechnik", + "oberflächenbehandlung", + "oberflaechenbehandlung", + "oberflächeningenieur", + "oberflaecheningenieur", ), "technical office": ( "technical office", + "technical specialist", "technical assistant", "technical documentation", "office assistant", @@ -78,6 +148,7 @@ ), "procurement": ( "procurement", + "procurement specialist", "purchasing", "buyer", "purchasing assistant", @@ -96,6 +167,87 @@ } +# Provider queries deliberately use a small, high-signal vocabulary. The complete +# aliases above are for matching; sending all of them to a job board would consume +# query budgets without improving coverage. +ROLE_QUERY_TERMS: dict[str, tuple[str, str]] = { + "quality": ("quality engineer", "qualitätsingenieur"), + "production": ("manufacturing engineer", "produktionsingenieur"), + "planning": ("production planner", "produktionsplaner"), + "process": ("process engineer", "prozessingenieur"), + "coating": ("coating engineer", "lackieringenieur"), + "technical office": ("technical specialist", "technische sachbearbeitung"), + "procurement": ("procurement specialist", "einkauf spezialist"), + "logistics": ("supply chain specialist", "supply chain spezialist"), +} + + +PROFESSIONAL_TITLE_SIGNALS = ( + "engineer", + "ingenieur", + "specialist", + "spezialist", + "manager", + "technician", + "techniker", + "planner", + "planer", + "coordinator", + "koordinator", + "technologist", + "technologe", + "arbeitsvorbereiter", +) + + +CONFLICTING_TITLE_SIGNALS = ( + "software", + "android", + "ios developer", + "frontend", + "backend", + "full stack", + "full-stack", + "devops", + "cloud", + "data engineer", + "machine learning", + "cyber security", + "cybersecurity", + "test automation", + "qa automation", +) + + +INDUSTRIAL_DOMAIN_SIGNALS = ( + "automotive", + "manufacturing", + "production process", + "production line", + "factory", + "plant", + "shop floor", + "supplier quality", + "iatf 16949", + "iso 9001", + "fmea", + "pfmea", + "spc", + "8d", + "root cause", + "lean manufacturing", + "six sigma", + "yield", + "cycle time", + "coating", + "paint process", + "lackierprozess", + "beschichtung", + "fertigung", + "produktion", +) + + def role_families_for_terms(terms) -> list[str]: """Return only families explicitly requested by this profile.""" requested = [normalize_text(str(term)) for term in terms if normalize_text(str(term))] diff --git a/app/search_job_service.py b/app/search_job_service.py index bf4b874..54f43d2 100644 --- a/app/search_job_service.py +++ b/app/search_job_service.py @@ -10,7 +10,15 @@ from .feedback_store import apply_learned_penalty from .profile_store import get_profile, upsert_profile_score from .providers import fetch_all_jobs -from .ranker import assess_language_fit, blocklist_matches, calculate_overall_score, profile_english_level, score_job +from .ranker import ( + assess_language_fit, + assess_role_relevance, + blocklist_matches, + calculate_overall_score, + classify_match_tier, + profile_english_level, + score_job, +) from .runtime import runtime_config from .search_job_store import ( acquire_search_job_lock, @@ -53,9 +61,9 @@ def _selected_sources(search_job: dict) -> list[dict]: def search_terms_for_job(search_job: dict, profile: dict) -> list[str]: - """Prefer isolated per-job queries and fall back to the scoring profile.""" + """Expand isolated job queries without losing the scoring profile's role coverage.""" configured = [str(term).strip() for term in (search_job.get("search_terms") or []) if str(term).strip()] - return list(dict.fromkeys(configured)) or search_terms_for_profile(profile) + return search_terms_for_profile(profile, configured if configured else None) def keyword_rules_for_job(search_job: dict, profile: dict) -> dict[str, dict[str, int]]: @@ -114,6 +122,12 @@ def passes_candidate_threshold(analysis: dict | None, minimum: int) -> bool: return bool(analysis) and int(analysis.get("cv_match", 0)) >= max(0, min(100, int(minimum))) +def has_sufficient_candidate_evidence(job) -> bool: + """Return whether a provider supplied enough vacancy text for a hard CV gate.""" + description = normalize_text(getattr(job, "description", "")) + return len(description) >= 240 and len(description.split()) >= 35 + + async def run_search_job(search_job_id: int) -> dict: search_job = get_search_job_any(search_job_id, mask_secrets=False) if not search_job: @@ -125,7 +139,7 @@ async def run_search_job(search_job_id: int) -> dict: base_cfg = runtime_config(search_job.get("user_id")) provider_errors = [] channels = [] - filtered = {"blocklist": 0, "employment": 0, "fit": 0, "language": 0, "cv_match": 0} + filtered = {"blocklist": 0, "role": 0, "employment": 0, "fit": 0, "language": 0, "cv_match": 0} try: profile = get_profile(int(search_job["profile_id"]), user_id=search_job.get("user_id")) if not profile: @@ -168,16 +182,31 @@ async def run_search_job(search_job_id: int) -> dict: ) keyword_rules = keyword_rules_for_job(search_job, profile) min_cv_match = int(search_job.get("min_cv_match", 58)) + strict_employment = search_job.get("employment_mode", "prefer") == "strict" + custom_role_intent = bool(search_job.get("search_terms")) for source_job in unique_jobs: job = deepcopy(source_job) if blocklist_matches(job, keyword_rules): filtered["blocklist"] += 1 continue - job.score, job.reasons = score_job(job, keyword_rules, location_terms) - employment_ok, _employment_label, employment_reasons = assess_employment_fit(job, profile) - if not employment_ok: - filtered["employment"] += 1 - continue + job.score, job.reasons = score_job( + job, + keyword_rules, + location_terms, + search_terms, + restrict_to_intent=custom_role_intent, + ) + role = assess_role_relevance( + job, + keyword_rules, + search_terms, + restrict_to_intent=custom_role_intent, + ) + job.role_relevant = role.relevant + job.reasons.extend(reason for reason in role.reasons if reason not in job.reasons) + employment_ok, _employment_label, employment_reasons = assess_employment_fit( + job, profile, strict=strict_employment + ) job.reasons.extend(employment_reasons) job.language_score, job.language_label, job.language_reasons = assess_language_fit(job, language_profile) job.score, neg = apply_learned_penalty(job, job.score, profile_id=profile["id"]) @@ -187,9 +216,23 @@ async def run_search_job(search_job_id: int) -> dict: job.overall_score = calculate_overall_score(job.score, job.language_score, profile["language_weight"]) upsert_job(job) upsert_language_fit(job) - upsert_profile_score(job, profile["id"]) - eligible = job.language_score >= min_lang and job.overall_score >= min_score - if job.overall_score < min_score: + if not role.relevant: + filtered["role"] += 1 + job.match_tier = "excluded" + upsert_profile_score(job, profile["id"], role_relevant=False, match_tier="excluded") + continue + + hard_employment_exclusion = not employment_ok and (strict_employment or _employment_label == "student_only") + if hard_employment_exclusion: + filtered["employment"] += 1 + job.match_tier = "excluded" + upsert_profile_score(job, profile["id"], role_relevant=True, match_tier="excluded") + continue + + eligible = employment_ok and job.language_score >= min_lang and job.overall_score >= min_score + if not employment_ok: + filtered["employment"] += 1 + elif job.overall_score < min_score: filtered["fit"] += 1 elif job.language_score < min_lang: filtered["language"] += 1 @@ -201,29 +244,49 @@ async def run_search_job(search_job_id: int) -> dict: if eligible: filtered["language"] += 1 eligible = False + cv_deferred = False if eligible and candidate: - try: - # Ollama enrichment is synchronous; keep it off the scheduler/event loop. - # The worker boundary remains `await asyncio.to_thread(analyze_job`. - job.intelligence = await asyncio.to_thread( - analyze_job, - job.key, - candidate["id"], - search_job_id, - False, - search_job.get("user_id"), - ) - except Exception as exc: - job.reasons.append(f"intelligence-error: {exc}") - if not passes_candidate_threshold(getattr(job, "intelligence", None), min_cv_match): - filtered["cv_match"] += 1 - eligible = False + if not has_sufficient_candidate_evidence(job): + cv_deferred = True + job.reasons.append("cv match deferred: source description incomplete") + else: + try: + # Ollama enrichment is synchronous; keep it off the scheduler/event loop. + # The worker boundary remains `await asyncio.to_thread(analyze_job`. + job.intelligence = await asyncio.to_thread( + analyze_job, + job.key, + candidate["id"], + search_job_id, + False, + search_job.get("user_id"), + ) + except Exception as exc: + cv_deferred = True + job.reasons.append(f"intelligence-error: {exc}") + job.reasons.append("cv match deferred: analysis unavailable") + if not cv_deferred and not getattr(job, "intelligence", None): + cv_deferred = True + job.reasons.append("cv match deferred: analysis unavailable") + elif not cv_deferred and not passes_candidate_threshold(job.intelligence, min_cv_match): + filtered["cv_match"] += 1 + eligible = False + job.match_tier = classify_match_tier( + role_relevant=True, + eligible=eligible, + overall_score=job.overall_score, + employment_constraint=any(reason.startswith("employment mismatch:") for reason in employment_reasons), + language_label=job.language_label, + evidence_constraint=cv_deferred, + ) + upsert_profile_score(job, profile["id"], role_relevant=True, match_tier=job.match_tier) if eligible: fresh_for_this_search = mark_search_job_seen(search_job_id, job.key) if fresh_for_this_search: matches.append(job) matches.sort( key=lambda j: ( + {"strong": 2, "match": 1, "stretch": 0}.get(getattr(j, "match_tier", "match"), 0), getattr(j, "intelligence", {}).get("cv_match", -1), j.overall_score, j.language_score, diff --git a/app/search_job_store.py b/app/search_job_store.py index 2182b0f..0130a83 100644 --- a/app/search_job_store.py +++ b/app/search_job_store.py @@ -26,6 +26,7 @@ interval_hours INTEGER NOT NULL DEFAULT 12, min_score_override INTEGER, min_language_score_override INTEGER, + employment_mode TEXT NOT NULL DEFAULT 'prefer', min_cv_match INTEGER NOT NULL DEFAULT 58, max_results INTEGER NOT NULL DEFAULT 20, notify_telegram INTEGER NOT NULL DEFAULT 0, @@ -168,6 +169,8 @@ def ensure_search_job_schema(user_id: int | None = None) -> None: con.execute("ALTER TABLE search_jobs ADD COLUMN allowlist_boost INTEGER NOT NULL DEFAULT 15") if "min_cv_match" not in columns: con.execute("ALTER TABLE search_jobs ADD COLUMN min_cv_match INTEGER NOT NULL DEFAULT 58") + if "employment_mode" not in columns: + con.execute("ALTER TABLE search_jobs ADD COLUMN employment_mode TEXT NOT NULL DEFAULT 'prefer'") run_columns = {row[1] for row in con.execute("PRAGMA table_info(search_job_runs)").fetchall()} if "filter_counts_json" not in run_columns: con.execute("ALTER TABLE search_job_runs ADD COLUMN filter_counts_json TEXT NOT NULL DEFAULT '{}'") @@ -307,6 +310,11 @@ def save_search_job(data: dict[str, Any], job_id: int | None = None, user_id: in "interval_hours": int(data.get("interval_hours", 12)), "min_score_override": data.get("min_score_override"), "min_language_score_override": data.get("min_language_score_override"), + "employment_mode": ( + str(data.get("employment_mode", "prefer")) + if str(data.get("employment_mode", "prefer")) in ("prefer", "strict") + else "prefer" + ), "min_cv_match": max(0, min(100, int(data.get("min_cv_match", 58)))), "max_results": int(data.get("max_results", 20)), "notify_telegram": int(bool(data.get("notify_telegram", False))), @@ -317,13 +325,13 @@ def save_search_job(data: dict[str, Any], job_id: int | None = None, user_id: in } if job_id: con.execute( - """UPDATE search_jobs SET name=:name,enabled=:enabled,profile_id=:profile_id,inherit_location=:inherit_location,target_location=:target_location,location_terms_json=:location_terms_json,search_terms_json=:search_terms_json,allowlist_terms_json=:allowlist_terms_json,blocklist_terms_json=:blocklist_terms_json,allowlist_boost=:allowlist_boost,source_ids_json=:source_ids_json,frequency=:frequency,day_of_week=:day_of_week,hour=:hour,minute=:minute,interval_hours=:interval_hours,min_score_override=:min_score_override,min_language_score_override=:min_language_score_override,min_cv_match=:min_cv_match,max_results=:max_results,notify_telegram=:notify_telegram,notify_email=:notify_email,notification_json=:notification_json,secrets_json=:secrets_json,updated_at=:updated_at WHERE id=:id AND user_id IS :user_id""", + """UPDATE search_jobs SET name=:name,enabled=:enabled,profile_id=:profile_id,inherit_location=:inherit_location,target_location=:target_location,location_terms_json=:location_terms_json,search_terms_json=:search_terms_json,allowlist_terms_json=:allowlist_terms_json,blocklist_terms_json=:blocklist_terms_json,allowlist_boost=:allowlist_boost,source_ids_json=:source_ids_json,frequency=:frequency,day_of_week=:day_of_week,hour=:hour,minute=:minute,interval_hours=:interval_hours,min_score_override=:min_score_override,min_language_score_override=:min_language_score_override,employment_mode=:employment_mode,min_cv_match=:min_cv_match,max_results=:max_results,notify_telegram=:notify_telegram,notify_email=:notify_email,notification_json=:notification_json,secrets_json=:secrets_json,updated_at=:updated_at WHERE id=:id AND user_id IS :user_id""", {**vals, "id": job_id}, ) return job_id cur = con.execute( - """INSERT INTO search_jobs(user_id,name,enabled,profile_id,inherit_location,target_location,location_terms_json,search_terms_json,allowlist_terms_json,blocklist_terms_json,allowlist_boost,source_ids_json,frequency,day_of_week,hour,minute,interval_hours,min_score_override,min_language_score_override,min_cv_match,max_results,notify_telegram,notify_email,notification_json,secrets_json,created_at,updated_at) - VALUES(:user_id,:name,:enabled,:profile_id,:inherit_location,:target_location,:location_terms_json,:search_terms_json,:allowlist_terms_json,:blocklist_terms_json,:allowlist_boost,:source_ids_json,:frequency,:day_of_week,:hour,:minute,:interval_hours,:min_score_override,:min_language_score_override,:min_cv_match,:max_results,:notify_telegram,:notify_email,:notification_json,:secrets_json,:created_at,:updated_at)""", + """INSERT INTO search_jobs(user_id,name,enabled,profile_id,inherit_location,target_location,location_terms_json,search_terms_json,allowlist_terms_json,blocklist_terms_json,allowlist_boost,source_ids_json,frequency,day_of_week,hour,minute,interval_hours,min_score_override,min_language_score_override,employment_mode,min_cv_match,max_results,notify_telegram,notify_email,notification_json,secrets_json,created_at,updated_at) + VALUES(:user_id,:name,:enabled,:profile_id,:inherit_location,:target_location,:location_terms_json,:search_terms_json,:allowlist_terms_json,:blocklist_terms_json,:allowlist_boost,:source_ids_json,:frequency,:day_of_week,:hour,:minute,:interval_hours,:min_score_override,:min_language_score_override,:employment_mode,:min_cv_match,:max_results,:notify_telegram,:notify_email,:notification_json,:secrets_json,:created_at,:updated_at)""", {**vals, "created_at": now}, ) return int(cur.lastrowid) diff --git a/app/service.py b/app/service.py index 6aa24c9..c765350 100644 --- a/app/service.py +++ b/app/service.py @@ -9,7 +9,15 @@ from .language_store import upsert_language_fit from .profile_store import ensure_profile_schema, list_profiles, upsert_profile_score from .providers import fetch_all_jobs -from .ranker import assess_language_fit, blocklist_matches, calculate_overall_score, profile_english_level, score_job +from .ranker import ( + assess_language_fit, + assess_role_relevance, + blocklist_matches, + calculate_overall_score, + classify_match_tier, + profile_english_level, + score_job, +) from .runtime import runtime_config from .source_analytics import ensure_source_analytics_schema, save_source_run_stats @@ -87,13 +95,7 @@ async def run_search() -> dict: profile_match_counts = {p["id"]: 0 for p in profiles} for raw_job in fetched: - # Do not persist obvious full-time or employment-unclear contamination in a - # strict part-time default run. This keeps both the review queue and the raw - # Stored jobs metric clean after a reset/re-index. - default_employment_ok, _default_employment_label, _default_employment_reasons = assess_employment_fit( - raw_job, default_profile - ) - if not default_employment_ok or blocklist_matches(raw_job, default_profile.get("keywords") or {}): + if blocklist_matches(raw_job, default_profile.get("keywords") or {}): continue is_new = None @@ -105,7 +107,10 @@ async def run_search() -> dict: continue location_terms = profile.get("location_terms") or cfg["location_terms"] job.score, job.reasons = score_job(job, keywords, location_terms) - employment_ok, _employment_label, employment_reasons = assess_employment_fit(job, profile) + role = assess_role_relevance(job, keywords) + job.role_relevant = role.relevant + job.reasons.extend(reason for reason in role.reasons if reason not in job.reasons) + employment_ok, _employment_label, employment_reasons = assess_employment_fit(job, profile, strict=False) job.reasons.extend(employment_reasons) language_profile = { @@ -126,34 +131,49 @@ async def run_search() -> dict: if positive_reasons: job.reasons.extend(positive_reasons) - if not employment_ok: - job.score = 0 - job.overall_score = 0 - else: - job.overall_score = calculate_overall_score( - job.score, job.language_score, profile.get("language_weight", 35) - ) + job.overall_score = calculate_overall_score( + job.score, job.language_score, profile.get("language_weight", 35) + ) if profile["id"] == default_profile["id"]: is_new = upsert_job(job) upsert_language_fit(job) default_scored = job - if job.score >= int(profile.get("min_score", 35)): + if role.relevant and job.score >= int(profile.get("min_score", 35)): source_stats[job.source]["job_fit"] += 1 if job.language_score >= int(profile.get("min_language_score", 40)): source_stats[job.source]["language_fit"] += 1 elif is_new is None: is_new = upsert_job(job) - upsert_profile_score(job, profile["id"]) - eligible_language = job.language_score >= int(profile.get("min_language_score", 40)) if profile.get("hide_german_heavy", True) and job.language_label == "german_heavy": eligible_language = False if not profile.get("show_b2_stretch", True) and job.language_label == "stretch": eligible_language = False eligible = ( - employment_ok and job.overall_score >= int(profile.get("min_score", 35)) and eligible_language + role.relevant + and employment_ok + and job.overall_score >= int(profile.get("min_score", 35)) + and eligible_language + ) + if not employment_ok and _employment_label == "student_only": + job.match_tier = "excluded" + else: + job.match_tier = classify_match_tier( + role_relevant=role.relevant, + eligible=eligible, + overall_score=job.overall_score, + employment_constraint=any( + reason.startswith("employment mismatch:") for reason in employment_reasons + ), + language_label=job.language_label, + ) + upsert_profile_score( + job, + profile["id"], + role_relevant=role.relevant, + match_tier=job.match_tier, ) if eligible: profile_match_counts[profile["id"]] += 1 diff --git a/app/stepstone-ui.js b/app/stepstone-ui.js index 3934bc1..3aedc59 100644 --- a/app/stepstone-ui.js +++ b/app/stepstone-ui.js @@ -12,7 +12,7 @@ const cfg = existing?.config || {}; const name = prompt('Display name', existing?.name || 'StepStone Germany'); if(name === null) return; - const maxTerms = numberPrompt('Maximum search terms per run (1-10)', cfg.max_search_terms || 3, 1, 10); + const maxTerms = numberPrompt('Maximum search terms per run (1-10)', cfg.max_search_terms || 6, 1, 10); if(maxTerms === null) return; const pages = numberPrompt('Pages per search term (1-3)', cfg.pages_per_term || 1, 1, 3); if(pages === null) return; diff --git a/app/stepstone_provider.py b/app/stepstone_provider.py index d416ceb..b9f097a 100644 --- a/app/stepstone_provider.py +++ b/app/stepstone_provider.py @@ -154,7 +154,7 @@ def parse_stepstone_search_html(html: str, source_name: str = "StepStone Germany async def fetch_stepstone(source: dict, search_terms: list[str], target_location: str) -> list[Job]: config = source.get("config") or {} - max_terms = max(1, min(int(config.get("max_search_terms", 3)), 10)) + max_terms = max(1, min(int(config.get("max_search_terms", 6)), 10)) pages_per_term = max(1, min(int(config.get("pages_per_term", 1)), 3)) results_per_term = max(1, min(int(config.get("results_per_term", 25)), 75)) timeout_seconds = max(10, min(int(config.get("timeout_seconds", 30)), 90)) diff --git a/app/ui-shell.js b/app/ui-shell.js index 9c08779..abebaab 100644 --- a/app/ui-shell.js +++ b/app/ui-shell.js @@ -46,15 +46,19 @@ 'Administrator':'Yönetici','Your workspace':'Çalışma alanınız','Your job search workspace':'İş arama çalışma alanınız', 'Keep profiles, opportunities and applications moving in one place.':'Profilleri, fırsatları ve başvuruları tek yerden yönetin.', 'Review jobs':'İlanları incele','Job Review Queue':'İlan inceleme listesi','Search profile':'Arama profili','Decision':'Karar', + 'Match tier':'Eşleşme seviyesi','All relevant roles':'Tüm ilgili roller','Strong':'Güçlü','Match':'Eşleşme','Constraint to review':'Kısıtı incele', 'Language requirement':'Dil gereksinimi','Ad language':'İlan dili','Min fit':'En düşük uyum','Refresh jobs':'İlanları yenile', 'Active':'Aktif','Unreviewed':'İncelenmedi','Suitable':'Uygun','Maybe':'Belki','Not suitable':'Uygun değil','All':'Tümü', 'Recommended':'Önerilen','English-first':'Öncelikle İngilizce','German-growth':'Almanca geliştirmeye uygun', 'B2 stretch':'B2 gelişim fırsatı','Unclear':'Belirsiz','Profile preference':'Profil tercihi','German (DE)':'Almanca (DE)', 'English (EN)':'İngilizce (EN)','Mixed (DE/EN)':'Karışık (DE/EN)','Unknown':'Bilinmiyor','Overall fit':'Genel uyum', 'Job fit':'İş uyumu','Language fit':'Dil uyumu','Strong profile match':'Profilinizle güçlü eşleşme', + 'Strong role match':'Güçlü rol eşleşmesi','Relevant role · constraint to review':'İlgili rol · kısıtı inceleyin', + 'strong':'güçlü','match':'eşleşme','stretch':'kısıtlı eşleşme', 'Provider search phrases':'Sağlayıcı arama ifadeleri','Target positions and roles':'Hedef pozisyonlar ve roller', 'Working arrangements':'Çalışma biçimleri','Advanced scoring keywords (JSON)':'Gelişmiş puanlama anahtar kelimeleri (JSON)', 'One phrase per line. These are provider queries, not scoring rules.':'Her satıra bir ifade. Bunlar sağlayıcı sorgularıdır, puanlama kuralı değildir.', + 'One phrase per line. Bert runs unqualified bilingual role queries before schedule-specific variants so capped providers cover every role.':'Her satıra bir ifade. Bert, sınırlı sorgu kabul eden kaynakların her rolü tarayabilmesi için önce çalışma saati eklenmemiş iki dilli rol sorgularını çalıştırır.', 'One role per line. Used for role matching and bilingual expansion.':'Her satıra bir rol. Rol eşleştirme ve iki dilli genişletme için kullanılır.', 'One format per line. Part-time hours are also recognized automatically.':'Her satıra bir çalışma biçimi. Yarı zamanlı saatler otomatik tanınır.', 'Guided profile builder':'Rehberli profil oluşturucu','Choose your real target roles, working arrangements and language ability; the guide creates ready-to-use job-board searches.':'Gerçek hedef rollerinizi, çalışma biçiminizi ve dil seviyenizi seçin; kılavuz ilan siteleri için hazır arama ifadeleri oluşturur.', @@ -77,6 +81,8 @@ 'New profile':'Yeni profil','Edit search profile':'Arama profilini düzenle','New search profile':'Yeni arama profili', 'Name':'Ad','Slug':'Kısa ad','Primary location':'Ana konum','Location terms':'Konum terimleri', 'Minimum Overall Fit':'En düşük genel uyum','Minimum Language Fit':'En düşük dil uyumu','Language weight %':'Dil ağırlığı %', + 'Working-time handling':'Çalışma saati yaklaşımı','Prefer profile hours; keep stretch roles':'Profil saatlerini tercih et; kısıtlı rolleri göster','Strictly exclude other/unknown hours':'Diğer veya belirsiz saatleri kesin olarak dışla', + 'Preference mode keeps strong role matches visible when hours differ or are missing.':'Tercih modu, çalışma saatleri farklı veya belirsiz olsa da güçlü rol eşleşmelerini görünür tutar.', 'Current German':'Mevcut Almanca','Maximum preferred German':'Tercih edilen en yüksek Almanca','Preferred ad languages':'Tercih edilen ilan dilleri', 'German':'Almanca','English':'İngilizce','Mixed':'Karışık','Enabled':'Etkin','Disabled':'Devre dışı','Default profile':'Varsayılan profil', 'Show B2 stretch':'B2 fırsatlarını göster','Hide German-heavy':'İleri Almanca isteyenleri gizle','Prefer German-growth':'Almanca gelişimini tercih et', diff --git a/app/v10_main.py b/app/v10_main.py index 25ec780..63c62f5 100644 --- a/app/v10_main.py +++ b/app/v10_main.py @@ -1,5 +1,5 @@ from contextlib import asynccontextmanager -from typing import Any +from typing import Any, Literal from fastapi import Depends, HTTPException from fastapi.responses import Response from pydantic import BaseModel, Field @@ -61,6 +61,7 @@ class SearchJobPayload(BaseModel): interval_hours: int = Field(12, ge=1, le=168) min_score_override: int | None = Field(default=None, ge=0, le=100) min_language_score_override: int | None = Field(default=None, ge=0, le=100) + employment_mode: Literal["prefer", "strict"] = "prefer" min_cv_match: int = Field(default=58, ge=0, le=100) max_results: int = Field(20, ge=1, le=100) notify_telegram: bool = False diff --git a/app/v16_main.py b/app/v16_main.py index 8c0a72c..9d22476 100644 --- a/app/v16_main.py +++ b/app/v16_main.py @@ -334,7 +334,7 @@ def stepstone_ui(_: str = Depends(require_admin)): class StepStonePayload(BaseModel): name: str = Field(default="StepStone Germany", min_length=1, max_length=100) enabled: bool = False - max_search_terms: int = Field(default=3, ge=1, le=10) + max_search_terms: int = Field(default=6, ge=1, le=10) pages_per_term: int = Field(default=1, ge=1, le=3) results_per_term: int = Field(default=25, ge=1, le=75) timeout_seconds: int = Field(default=30, ge=10, le=90) diff --git a/tests/test_employment_filter.py b/tests/test_employment_filter.py index 465b872..ac61f8a 100644 --- a/tests/test_employment_filter.py +++ b/tests/test_employment_filter.py @@ -76,11 +76,33 @@ def test_mixed_full_and_part_time_profile_accepts_both_and_keeps_queries(): "format": {"Vollzeit": 16, "Teilzeit": 16}, }, } - assert search_terms_for_profile(mixed) == ["Qualitätsprüfer Vollzeit", "Qualitätsprüfer Teilzeit"] + terms = search_terms_for_profile(mixed) + assert terms[0] == "qualitätsprüfer" + assert "Qualitätsprüfer Vollzeit" in terms + assert "Qualitätsprüfer Teilzeit" in terms assert assess_employment_fit(job("Qualitätsprüfer Vollzeit"), mixed)[0] is True assert assess_employment_fit(job("Qualitätsprüfer Teilzeit"), mixed)[0] is True +def test_mixed_hours_profile_does_not_admit_student_only_jobs_without_enrollment(): + mixed = { + "name": "Quality engineering / Full-time and part-time", + "slug": "quality-both", + "keywords": {"format": {"Vollzeit": 16, "Teilzeit": 16}}, + } + result = assess_employment_fit(job("Werkstudent Quality Engineering"), mixed, strict=False) + assert result[0] is False + assert result[1] == "student_only" + + body_only = assess_employment_fit( + job("Quality Engineering Assistant", "Employment type: working student; enrollment is required."), + mixed, + strict=False, + ) + assert body_only[0] is False + assert body_only[1] == "student_only" + + def test_part_time_signal_wins_over_full_time_boilerplate(): ok, label, _ = assess_employment_fit( job("Working Student Operations", "This is a working student role. Our company also has full-time employees."), @@ -114,13 +136,75 @@ def test_part_time_search_terms_are_diversified_before_configured_terms(): def test_engineering_profile_gets_its_own_bilingual_role_queries(): terms = search_terms_for_profile(ENGINEERING_PROFILE) - assert terms[0] == "qualitätskontrolle teilzeit" + assert terms[0] == "qualitätskontrolle" + assert "quality engineer" in terms[:6] assert any("quality" in term and "part time" in term for term in terms) - assert any("arbeitsvorbereitung" in term or "produktionsplanung" in term for term in terms) - assert any("sachbearbeitung" in term for term in terms[:6]) + assert any( + "arbeitsvorbereitung" in term or "produktionsplanung" in term or "produktionsplaner" in term for term in terms + ) + assert any("technical" in term or "sachbearbeitung" in term for term in terms[:7]) assert not any("werkstudent" in term or "supply chain" in term for term in terms) +def test_preference_mode_keeps_full_time_role_as_a_visible_stretch(): + vacancy = job("Process Engineer", "Employment type: full-time, 40 hours per week.") + strict = assess_employment_fit(vacancy, ENGINEERING_PROFILE, strict=True) + preferred = assess_employment_fit(vacancy, ENGINEERING_PROFILE, strict=False) + assert strict[0] is False + assert preferred[0] is True + assert preferred[1] == "full_time" + assert "employment mismatch: full-time" in preferred[2] + + +def test_full_time_profile_can_prefer_or_strictly_require_its_working_arrangement(): + profile = { + "name": "Industrial engineering / Full-time", + "slug": "industrial-full-time", + "keywords": {"format": {"full-time": 16, "vollzeit": 16}}, + } + full_time = job("Process Engineer", "Employment type: full-time, 40 hours per week.") + part_time = job("Process Engineer Teilzeit", "20 Stunden pro Woche.") + assert assess_employment_fit(full_time, profile, strict=True)[:2] == (True, "full_time") + assert assess_employment_fit(part_time, profile, strict=True)[0] is False + preferred = assess_employment_fit(part_time, profile, strict=False) + assert preferred[0] is True + assert preferred[1] == "part_time" + assert "employment mismatch: part-time/student" in preferred[2] + + +def test_first_provider_queries_cover_each_industrial_role_before_schedule_variants(): + profile = { + "name": "Industrial engineering / Part-time", + "slug": "industrial-part-time", + "target_location": "Berlin", + "location_terms": ["berlin"], + "keywords": { + "search": { + "part time quality engineer berlin": 0, + "supplier quality engineer teilzeit berlin": 0, + "quality assurance engineer part time berlin": 0, + "teilzeit process engineer berlin": 0, + "lackieringenieur teilzeit berlin": 0, + "production planner part time berlin": 0, + }, + "title": { + "process engineer": 35, + "quality engineer": 35, + "lackieringenieur": 32, + "production planner": 32, + }, + "format": {"teilzeit": 12, "part time": 12}, + }, + } + terms = search_terms_for_profile(profile) + first_six = terms[:6] + assert "process engineer" in first_six + assert "quality engineer" in first_six + assert "lackieringenieur" in first_six + assert "production planner" in first_six + assert not any("part time" in term or "teilzeit" in term for term in first_six) + + def test_part_time_is_confirmed_from_weekly_hours_and_afternoon_schedule(): ok, label, reasons = assess_employment_fit( job("Qualitätsprüfer", "Arbeitszeit: 15-20 Stunden pro Woche, nachmittags."), ENGINEERING_PROFILE diff --git a/tests/test_jobspy_provider.py b/tests/test_jobspy_provider.py index 44fc8c4..561e71e 100644 --- a/tests/test_jobspy_provider.py +++ b/tests/test_jobspy_provider.py @@ -1,4 +1,5 @@ import asyncio +import threading from app import providers from app import jobspy_provider @@ -75,3 +76,25 @@ def fake_scrape(term, site, source, location): jobs = asyncio.run(jobspy_provider.fetch_jobspy(source, ["supply chain"], "Berlin")) assert len(jobs) == 2 assert any(job.source.endswith("/ indeed") for job in jobs) + + +def test_jobspy_starts_independent_board_workers_concurrently(monkeypatch): + linkedin_started = threading.Event() + indeed_started = threading.Event() + + def fake_scrape(term, site, source, location): + if site == "linkedin": + linkedin_started.set() + assert indeed_started.wait(1), "Indeed was starved behind the LinkedIn worker" + if site == "indeed": + indeed_started.set() + return FakeFrame() + + monkeypatch.setattr(jobspy_provider, "_scrape_one", fake_scrape) + source = { + "name": "JobSpy Multi-board", + "config": {"sites": ["linkedin", "indeed"], "max_search_terms": 1}, + } + jobs = asyncio.run(jobspy_provider.fetch_jobspy(source, ["process engineer"], "Berlin")) + assert linkedin_started.is_set() and indeed_started.is_set() + assert len(jobs) == 2 diff --git a/tests/test_matching_quality.py b/tests/test_matching_quality.py index 8dba1da..5c7affa 100644 --- a/tests/test_matching_quality.py +++ b/tests/test_matching_quality.py @@ -1,7 +1,7 @@ from app import intelligence as intel from app.models import Job from app.ranker import blocklist_matches, score_job -from app.search_job_service import deduplicate_jobs, passes_candidate_threshold +from app.search_job_service import deduplicate_jobs, has_sufficient_candidate_evidence, passes_candidate_threshold from app.text_match import contains_phrase, normalize_text @@ -106,3 +106,20 @@ def test_candidate_threshold_is_inclusive_and_clamped(): assert passes_candidate_threshold({"cv_match": 58}, 58) assert not passes_candidate_threshold({"cv_match": 57}, 58) assert not passes_candidate_threshold(None, 0) + + +def test_hard_cv_gate_requires_a_complete_source_description(): + short = vacancy("Quality Engineer", "Part-time role. English required.") + complete = vacancy( + "Quality Engineer", + " ".join( + [ + "Own supplier and production quality using SPC, PFMEA, control plans, root cause analysis,", + "8D corrective actions, audits, KPI reporting, cross-functional improvement, and customer", + "complaint management across an automotive manufacturing operation in Berlin with English", + "as the working language and flexible part-time scheduling for the engineering team.", + ] + ), + ) + assert has_sufficient_candidate_evidence(short) is False + assert has_sufficient_candidate_evidence(complete) is True diff --git a/tests/test_notifier_matching.py b/tests/test_notifier_matching.py new file mode 100644 index 0000000..1b3414c --- /dev/null +++ b/tests/test_notifier_matching.py @@ -0,0 +1,19 @@ +from app.models import Job +from app.notifier import build_text_digest + + +def test_digest_exposes_match_tier_for_constraint_review(): + job = Job( + source="test", + external_id="stretch", + title="Process Engineer", + company="Example", + location="Berlin", + url="https://example.com/stretch", + score=82, + language_score=92, + overall_score=86, + language_label="english_first", + ) + job.match_tier = "stretch" + assert "Match tier: Stretch" in build_text_digest([job]) diff --git a/tests/test_profiles.py b/tests/test_profiles.py index 9c10cee..51ed08b 100644 --- a/tests/test_profiles.py +++ b/tests/test_profiles.py @@ -95,6 +95,88 @@ def test_complete_job_detail_is_limited_to_scored_profile(tmp_path, monkeypatch) assert get_job_for_profile(job.key, profiles[1]["id"]) is None +def test_role_irrelevant_scores_are_not_shown_in_review_queue(tmp_path, monkeypatch): + setup_db(tmp_path, monkeypatch) + profile = list_profiles()[0] + relevant = add_job() + upsert_profile_score(relevant, profile["id"], role_relevant=True, match_tier="strong") + irrelevant = Job( + source="test", + external_id="software-1", + title="Software Quality Engineer", + company="Example GmbH", + location="Berlin", + url="https://example.com/software-1", + description="Cloud test automation and backend quality.", + ) + irrelevant.score = 82 + irrelevant.language_score = 92 + irrelevant.overall_score = 86 + db.upsert_job(irrelevant) + upsert_profile_score(irrelevant, profile["id"], role_relevant=False, match_tier="excluded") + hard_constraint = Job( + source="test", + external_id="strict-full-time", + title="Supply Chain Manager", + company="Example GmbH", + location="Berlin", + url="https://example.com/strict-full-time", + ) + hard_constraint.score = 82 + hard_constraint.language_score = 92 + hard_constraint.overall_score = 86 + db.upsert_job(hard_constraint) + upsert_profile_score(hard_constraint, profile["id"], role_relevant=True, match_tier="excluded") + + rows = list_jobs_for_profile(profile["id"], decision="all", language="all") + assert [row["job_key"] for row in rows] == [relevant.key] + assert rows[0]["match_tier"] == "strong" + assert not list_jobs_for_profile(profile["id"], decision="all", language="all", tier="stretch") + assert get_job_for_profile(hard_constraint.key, profile["id"]) is None + + +def test_profile_score_migration_hides_soft_signal_only_legacy_rows(tmp_path, monkeypatch): + setup_db(tmp_path, monkeypatch) + profile = list_profiles()[0] + relevant = add_job() + irrelevant = Job( + source="test", + external_id="legacy-noise", + title="Android Engineer", + company="Example GmbH", + location="Berlin", + url="https://example.com/legacy-noise", + ) + db.upsert_job(irrelevant) + with db.connection() as con: + con.execute("DROP TABLE job_profile_scores") + con.execute( + """CREATE TABLE job_profile_scores ( + job_key TEXT NOT NULL,profile_id INTEGER NOT NULL,job_score INTEGER NOT NULL DEFAULT 0, + language_score INTEGER NOT NULL DEFAULT 55,overall_score INTEGER NOT NULL DEFAULT 0, + language_label TEXT NOT NULL DEFAULT 'unclear',reasons_json TEXT NOT NULL DEFAULT '[]', + language_reasons_json TEXT NOT NULL DEFAULT '[]',updated_at TEXT NOT NULL, + PRIMARY KEY(job_key,profile_id))""" + ) + values = (profile["id"], 80, 92, 84, "english_first", "[]", "2026-09-02T00:00:00+00:00") + con.execute( + """INSERT INTO job_profile_scores + (job_key,profile_id,job_score,language_score,overall_score,language_label,reasons_json, + language_reasons_json,updated_at) VALUES(?,?,?,?,?,?,?,?,?)""", + (relevant.key, *values[:5], '["title: supply chain"]', *values[5:]), + ) + con.execute( + """INSERT INTO job_profile_scores + (job_key,profile_id,job_score,language_score,overall_score,language_label,reasons_json, + language_reasons_json,updated_at) VALUES(?,?,?,?,?,?,?,?,?)""", + (irrelevant.key, *values[:5], '["skill: sap", "target area"]', *values[5:]), + ) + + rows = list_jobs_for_profile(profile["id"], decision="all", language="all") + assert [row["job_key"] for row in rows] == [relevant.key] + assert rows[0]["match_tier"] == "strong" + + def test_learning_is_isolated_by_profile(tmp_path, monkeypatch): setup_db(tmp_path, monkeypatch) job = add_job() diff --git a/tests/test_provider_concurrency.py b/tests/test_provider_concurrency.py new file mode 100644 index 0000000..1d5453e --- /dev/null +++ b/tests/test_provider_concurrency.py @@ -0,0 +1,41 @@ +import asyncio + +from app.models import Job +from app import providers + + +def test_sources_run_concurrently_but_results_keep_configuration_order(monkeypatch): + slow_started = asyncio.Event() + fast_started = asyncio.Event() + + def result(source): + return Job( + source=source["name"], + external_id=source["name"], + title="Process Engineer", + company="Example", + location="Berlin", + url=f"https://example.com/{source['name']}", + ) + + async def slow(source, _terms, _location): + slow_started.set() + await asyncio.wait_for(fast_started.wait(), timeout=1) + return [result(source)] + + async def fast(source, _terms, _location): + fast_started.set() + return [result(source)] + + monkeypatch.setitem(providers.PROVIDERS, "test_slow", slow) + monkeypatch.setitem(providers.PROVIDERS, "test_fast", fast) + sources = [ + {"id": 901, "name": "Slow source", "source_type": "test_slow", "config": {}}, + {"id": 902, "name": "Fast source", "source_type": "test_fast", "config": {}}, + ] + + jobs, errors = asyncio.run(providers.fetch_all_jobs(sources, ["process engineer"], "Berlin")) + + assert not errors + assert slow_started.is_set() and fast_started.is_set() + assert [job.source for job in jobs] == ["Slow source", "Fast source"] diff --git a/tests/test_role_relevance_v2.py b/tests/test_role_relevance_v2.py new file mode 100644 index 0000000..88644fd --- /dev/null +++ b/tests/test_role_relevance_v2.py @@ -0,0 +1,179 @@ +from app.models import Job +from app.ranker import assess_role_relevance, score_job + + +KEYWORDS = { + "search": { + "process engineer": 0, + "quality engineer": 0, + "production planner": 0, + "coating engineer": 0, + }, + "title": { + "process engineer": 35, + "quality engineer": 35, + "production planner": 32, + "coating engineer": 32, + "lackieringenieur": 32, + }, + "format": {"teilzeit": 12, "part time": 12}, + "skill": { + "fmea": 8, + "spc": 8, + "root cause": 7, + "lean": 6, + "six sigma": 6, + "process optimization": 7, + }, + "allowlist": {"automotive": 12, "manufacturing": 10}, + "negative": {}, +} + + +def vacancy(external_id: str, title: str, description: str) -> Job: + return Job( + source="benchmark", + external_id=external_id, + title=title, + company="Example", + location="Berlin", + url=f"https://example.com/{external_id}", + description=description, + ) + + +def test_manual_hach_quality_engineer_is_a_direct_high_fit_match(): + job = vacancy( + "hach", + "Quality Engineer (m/w/d)", + "Industrial quality for logistics, corrective actions, root cause, 8D, Ishikawa and cross-functional KPIs.", + ) + assessment = assess_role_relevance(job, KEYWORDS) + score, reasons = score_job(job, KEYWORDS, ["berlin"]) + assert assessment.relevant is True + assert assessment.confidence == "direct" + assert score >= 70 + assert "role family: quality" in reasons + + +def test_manual_asml_cleaning_and_etching_role_is_not_mistaken_for_cleaning_work(): + job = vacancy( + "asml", + "Process Engineer – Cleaning & Etching / Prozessingenieur:in Reinigungs- und Ätztechnologien", + "Own manufacturing cleaning and etching processes using Lean, Six Sigma, SPC, FMEA, yield and cycle time.", + ) + assessment = assess_role_relevance(job, KEYWORDS) + score, _ = score_job(job, KEYWORDS, ["berlin"]) + assert assessment.relevant is True + assert "process" in assessment.matched_families + assert score >= 75 + + +def test_software_quality_title_is_rejected_without_industrial_domain_evidence(): + job = vacancy( + "software", + "Software Quality Engineer", + "Build cloud test automation for backend services using Python and Kubernetes.", + ) + assessment = assess_role_relevance(job, KEYWORDS) + assert assessment.relevant is False + assert assessment.confidence == "conflict" + + +def test_generic_production_and_hr_jobs_cannot_be_rescued_by_soft_signals(): + production_worker = vacancy( + "worker", + "Production Worker", + "Full-time manufacturing work with quality checks at a Berlin plant.", + ) + hr_partner = vacancy( + "hr", + "HR Business Partner", + "Support a manufacturing plant, continuous improvement, Lean and root cause workshops.", + ) + assert assess_role_relevance(production_worker, KEYWORDS).relevant is False + assert assess_role_relevance(hr_partner, KEYWORDS).relevant is False + + +def test_generic_engineer_title_can_use_strong_description_role_evidence(): + job = vacancy( + "engineer-ii", + "Engineer II", + "Responsible for process engineering, process optimization, PFMEA and SPC in automotive manufacturing.", + ) + assessment = assess_role_relevance(job, KEYWORDS) + assert assessment.relevant is True + assert assessment.confidence == "supported" + assert "process" in assessment.matched_families + + +def test_unmapped_search_role_still_requires_direct_title_evidence_and_scores_it(): + keywords = { + "search": {"mechanical engineer": 0}, + "title": {}, + "format": {}, + "skill": {}, + "allowlist": {}, + "negative": {}, + } + mechanical = vacancy("mechanical", "Senior Mechanical Engineer", "Design industrial equipment.") + unrelated = vacancy("unrelated", "HR Manager", "Support an engineering organization.") + assert assess_role_relevance(mechanical, keywords).relevant is True + assert assess_role_relevance(unrelated, keywords).relevant is False + assert score_job(mechanical, keywords, ["berlin"])[0] >= 40 + + +def test_conflicting_occupation_is_allowed_when_the_profile_explicitly_requests_it(): + keywords = { + "search": {"software engineer": 0}, + "title": {"software engineer": 35}, + "format": {}, + "skill": {}, + "allowlist": {}, + "negative": {}, + } + assessment = assess_role_relevance( + vacancy("explicit-software", "Senior Software Engineer", "Build distributed systems."), + keywords, + ) + assert assessment.relevant is True + assert assessment.confidence == "direct" + + +def test_industrial_title_variants_expand_recall_without_generic_worker_terms(): + titles = ( + "Lieferantenqualitätsingenieur (m/w/d)", + "Process Development Engineer", + "Paint Shop Engineer", + "Fertigungsplaner", + ) + assert all( + assess_role_relevance(vacancy(f"variant-{index}", title, "Automotive manufacturing."), KEYWORDS).relevant + for index, title in enumerate(titles) + ) + + +def test_live_jobspy_noise_sample_is_rejected_by_role_gate(): + # Titles returned by the live Indeed adapter for the queries "quality engineer" + # and "process engineer" on 2026-09-02. Provider-side search is intentionally + # treated as candidate discovery, never as proof of role relevance. + live_titles = ( + "Senior Android Engineer (m/f/d)", + "Job Posting Title Construction Surveillance Technician", + "Junior Software Engineer (all genders)", + "Principal Software Engineer", + "Applied Mathematician", + "Head of Mass Spectrometry Core Facility (m/f/d)", + "DevOps Engineer - CI/CD & Platform Engineering", + "Senior Full-Stack Engineer - Team Agent", + "Project Engineer (Utility BESS)", + "Product Manager (DevTools & AI Reliability, remote)", + "Senior Electronics Engineer EW (All Genders)", + "Deployment Engineer, Google Cloud Public Sector", + "Technical Project Manager, Carrier Integrations", + ) + assessments = [ + assess_role_relevance(vacancy(f"live-{index}", title, "English-speaking role."), KEYWORDS) + for index, title in enumerate(live_titles) + ] + assert not [assessment for assessment in assessments if assessment.relevant] diff --git a/tests/test_search_improvement_ui.py b/tests/test_search_improvement_ui.py index 07be99d..a80247b 100644 --- a/tests/test_search_improvement_ui.py +++ b/tests/test_search_improvement_ui.py @@ -16,7 +16,8 @@ def test_review_cards_explain_role_schedule_language_and_strong_matches(): assert "scheduleDetail" in text assert "job-insight language" in text assert "job-language-reasons" in text - assert "Strong profile match" in text + assert "Strong role match" in text + assert "constraint to review" in text def test_interface_offers_persistent_english_and_turkish_language_selection(): diff --git a/tests/test_search_jobs.py b/tests/test_search_jobs.py index 4b4a892..dee4692 100644 --- a/tests/test_search_jobs.py +++ b/tests/test_search_jobs.py @@ -93,7 +93,9 @@ def test_search_terms_are_isolated_and_normalized_per_search_job(tmp_path, monke ) saved = next(job for job in list_search_jobs() if job["id"] == job_id) assert saved["search_terms"] == ["teilzeit process engineer", "part time quality engineer"] - assert search_terms_for_job(saved, p) == saved["search_terms"] + planned = search_terms_for_job(saved, p) + assert planned[:2] == ["process engineer", "quality engineer"] + assert all(term in planned for term in saved["search_terms"]) assert not any("werkstudent" in term for term in search_terms_for_job(saved, p)) @@ -136,6 +138,16 @@ def test_cv_match_threshold_is_saved_per_search_job(tmp_path, monkeypatch): assert saved["min_cv_match"] == 72 +def test_working_time_can_be_preferred_or_strict_per_search_job(tmp_path, monkeypatch): + setup_db(tmp_path, monkeypatch) + profile = list_profiles()[0] + preferred_id = save_search_job({"name": "Broad discovery", "profile_id": profile["id"]}) + strict_id = save_search_job({"name": "Part-time only", "profile_id": profile["id"], "employment_mode": "strict"}) + jobs = {job["id"]: job for job in list_search_jobs()} + assert jobs[preferred_id]["employment_mode"] == "prefer" + assert jobs[strict_id]["employment_mode"] == "strict" + + def test_run_history_preserves_filter_reason_counts(tmp_path, monkeypatch): setup_db(tmp_path, monkeypatch) profile = list_profiles()[0] diff --git a/tests/test_search_matching_pipeline_v2.py b/tests/test_search_matching_pipeline_v2.py new file mode 100644 index 0000000..9a2a401 --- /dev/null +++ b/tests/test_search_matching_pipeline_v2.py @@ -0,0 +1,137 @@ +import asyncio + +import pytest + +from app.models import Job +from app import search_job_service as service + + +PROFILE = { + "id": 7, + "name": "Industrial engineering / Part-time", + "slug": "industrial-part-time", + "target_location": "Berlin", + "location_terms": ["berlin"], + "min_score": 35, + "min_language_score": 30, + "language_weight": 35, + "current_german_level": "b1", + "max_german_requirement": "b1", + "prefer_german_growth": True, + "hide_german_heavy": False, + "show_b2_stretch": True, + "keywords": { + "search": {"process engineer part time": 0, "quality engineer part time": 0}, + "title": {"process engineer": 35, "quality engineer": 35}, + "format": {"part time": 12, "teilzeit": 12}, + "skill": {"spc": 8, "fmea": 8, "sap": 6}, + "allowlist": {"manufacturing": 10}, + "blocklist": {}, + "negative": {}, + }, +} + + +@pytest.mark.parametrize( + ("employment_mode", "expected_matches", "process_tier", "employment_filtered"), + (("prefer", 2, "stretch", 0), ("strict", 1, "excluded", 1)), +) +def test_pipeline_separates_role_relevance_from_working_time_constraints( + monkeypatch, employment_mode, expected_matches, process_tier, employment_filtered +): + search_job = { + "id": 11, + "user_id": None, + "name": "Engineering discovery", + "profile_id": PROFILE["id"], + "inherit_location": True, + "target_location": "Berlin", + "location_terms": [], + "search_terms": [], + "allowlist_terms": None, + "blocklist_terms": None, + "source_ids": [], + "employment_mode": employment_mode, + "min_score_override": None, + "min_language_score_override": None, + "min_cv_match": 58, + "max_results": 20, + "notify_email": False, + "notify_telegram": False, + } + jobs = [ + Job( + source="test", + external_id="software", + title="Software Quality Engineer", + company="Cloud Co", + location="Berlin", + url="https://example.com/software", + description="English-speaking cloud test automation with SAP integrations.", + ), + Job( + source="test", + external_id="process", + title="Process Engineer – Cleaning & Etching", + company="ASML", + location="Berlin", + url="https://example.com/process", + description=( + "Full-time manufacturing process ownership with SPC, FMEA, yield improvement. " + "The international team works in English." + ), + ), + Job( + source="test", + external_id="quality-short", + title="Quality Engineer Part Time", + company="Hach", + location="Berlin", + url="https://example.com/quality-short", + description="Part-time role in an English-speaking team.", + ), + ] + writes = [] + + async def fetch_all(_sources, _terms, _location): + return jobs, [] + + monkeypatch.setattr(service, "get_search_job_any", lambda *_args, **_kwargs: search_job) + monkeypatch.setattr(service, "acquire_search_job_lock", lambda *_args: True) + monkeypatch.setattr(service, "create_search_job_run", lambda *_args: 19) + monkeypatch.setattr(service, "finish_search_job_run", lambda *_args, **_kwargs: None) + monkeypatch.setattr(service, "release_search_job_lock", lambda *_args: None) + monkeypatch.setattr(service, "runtime_config", lambda *_args: {}) + monkeypatch.setattr(service, "get_profile", lambda *_args, **_kwargs: PROFILE) + monkeypatch.setattr(service, "sync_application_events", lambda *_args, **_kwargs: None) + monkeypatch.setattr( + service, + "candidate_for_search_job", + lambda *_args, **_kwargs: {"id": 3, "name": "Test Candidate"}, + ) + monkeypatch.setattr(service, "_selected_sources", lambda *_args: []) + monkeypatch.setattr(service, "fetch_all_jobs", fetch_all) + monkeypatch.setattr(service, "apply_learned_penalty", lambda _job, score, **_kwargs: (score, [])) + monkeypatch.setattr(service, "apply_positive_boost", lambda _job, score, **_kwargs: (score, [])) + monkeypatch.setattr(service, "upsert_job", lambda *_args: True) + monkeypatch.setattr(service, "upsert_language_fit", lambda *_args: None) + monkeypatch.setattr( + service, + "upsert_profile_score", + lambda job, _profile_id, **kwargs: writes.append((job.external_id, job.match_tier, kwargs)), + ) + monkeypatch.setattr(service, "mark_search_job_seen", lambda *_args: True) + monkeypatch.setattr( + service, + "analyze_job", + lambda *_args, **_kwargs: (_ for _ in ()).throw(AssertionError("sparse descriptions must be deferred")), + ) + + result = asyncio.run(service.run_search_job(search_job["id"])) + + assert result["matches"] == expected_matches + assert result["filtered"]["role"] == 1 + assert ("software", "excluded", {"role_relevant": False, "match_tier": "excluded"}) in writes + assert result["filtered"]["employment"] == employment_filtered + assert ("process", process_tier, {"role_relevant": True, "match_tier": process_tier}) in writes + assert ("quality-short", "stretch", {"role_relevant": True, "match_tier": "stretch"}) in writes