Feature/recommendation system improvements - #1585
Open
OlegPhenomenon wants to merge 42 commits into
Open
OlegPhenomenon wants to merge 42 commits into
OlegPhenomenon wants to merge 42 commits into
Conversation
Initial implementation of personalized auction sorting: - recommendation_profiles, recommendation_events, user_auction_scores tables - baseline rule-based Scorer (wishlist hit, tag/custom-interest match, bid/wishlist affinity, structural bonuses, AI prior) - AuctionDomainClassifier via OpenAI structured outputs - ClassifyAuctionDomainsJob + RefreshUserAuctionScoresJob - Auction::UserSortable extended to 4/5-tier priority with LEFT JOIN on user_auction_scores - DatasetSnapshot service for offline model training pipeline - ScoreImporter for external score uploads - RecommendationProfile fields embedded in sign-up form - Prompt modal on /auctions for users without filled profile - Event tracking from controllers (impressions, clicks, bids, wishlist) - recommendation_tracker Stimulus controller for client-side clicks - custom_interest_tags Stimulus controller (tag input UX) - OpenaiStructuredOutputSupport helper (model fallback, temperature) - Tests for scorer, classifier, dataset snapshot, score importer, recommendation profile model and controller - en/et locales for recommendation profile UI This commit captures the pre-v2 state on branch feature/recommendation-system-improvements. Following commits introduce the v2 plan documented in docs/architecture/recommendation-system.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two living documents describing the planned v2 of the recommendation system on this branch: - docs/architecture/recommendation-system.md Goal, data flow, table responsibilities, scorer signals, classification tiers, cron schedule, cost estimate, phase list. - docs/technical/domain-classification-pipeline.md Component layout, Tier 0 heuristic algorithm, Tier 2 LLM batch flow, embedding pipeline, triggers, backfill, migration from v1, tests. Key v2 decisions captured: - domain_classifications becomes single source of truth, decoupled from auctions (covers wishlist domains and historical bids too) - LLM is cron-only (daily k8s CronJob), never per-request - pgvector embeddings for similarity-based affinity - Time decay on behavioural signals (half-life 60d) - Heuristic Ruby classifier handles 60-70% offline, LLM enriches the rest - AWS RDS Postgres 17 supports pgvector natively; no Dockerfile/IaC changes required, only one-time CREATE EXTENSION Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 of recommendation system v2 (see docs/architecture/recommendation-system.md). - EventTracker.track_impressions now uses RecommendationEvent.insert_all for a single SQL INSERT regardless of page size. /auctions index used to fire N inserts per render (one per visible card). - RefreshSingleUserAuctionScoresJob gains enqueue_debounced (30s wait) and skip-if-fresh guard, so a burst of user actions collapses into a single recompute pass rather than N identical jobs. - All controllers switched to enqueue_debounced. - Recommendation::Scorer default scope narrows to auctions ending within SCORING_HORIZON (30 days). top_auctions_for, refresh_for and the constructor share the same default. - Auction::UserSortable#interest_match_sql now whitelists categories against InterestCatalog before SQL quoting and caps custom-interest LIKE clauses at MAX_CUSTOM_INTERESTS_IN_SQL (10) with a 2-char min to avoid pathological patterns. - Log level for tracking failures bumped from info to warn. - New tests for batched impressions (single INSERT assertion) and debounce skip/refresh behaviour. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The .claude/ directory contains per-developer local state from the Claude Code agentic harness (ralph-loop checkpoint, local settings). It accidentally landed in the previous commit; this commit untracks it and excludes the whole directory going forward. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2 of recommendation system v2 (see docs/architecture/recommendation-system.md). Schema: - New table domain_classifications, one row per domain_name. Stores rich features: primary_category, tags[], description, description_locale, keywords[], audience, languages[], suggested_use_cases[], structural cache (has_digits, has_hyphens, token_count, dictionary_word, brandability_score), provenance (classification_source, classification_model, confidence, classified_at, raw_llm_response). - Indexes: unique on domain_name and uuid; GIN on tags and keywords; btree on primary_category, audience, classified_at, classification_source. Model: - DomainClassification with validations, normalized domain_name, source/refresh scopes (needs_llm_enrichment, low_confidence, unclassified, classified) and predicates. Tier 0 services: - DomainStructuralAnalyzer extracts deterministic structural features and tokens. - DomainDictionary holds et+en root -> InterestCatalog category mappings and a greedy longest-prefix tokenizer. - DomainHeuristicClassifier composes the two and emits a hash ready to upsert into domain_classifications, including confidence and brandability_score heuristics. Tests: - Analyzer covers digits, hyphens, tokenization, numeric-only. - Heuristic classifier covers Estonian dictionary words, compound English domains, numeric domains, unknown low-confidence cases, metadata, and brandability deltas. - DomainClassification model: validation, normalization, uniqueness, needs_llm_enrichment scope across heuristic / low-confidence / stale branches. Note: embedding column and its scope land in Phase 5 (pgvector). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 3a of recommendation system v2. - Recommendation::DomainClassifier is the synchronous entry point for classifying a single domain. It runs structural + heuristic Tier 0 and upserts into domain_classifications. The LLM path is never triggered from runtime — it lives behind the cron-only batch job (Phase 3b). - Idempotent: skips work when an existing row is fresh (within FRESH_WINDOW = 1.hour) unless force: true. - Preserves LLM-enriched fields when re-running on a row whose source is openai, so heuristic passes never clobber better data. - Recommendation::ClassifyDomainHeuristicallyJob wraps the orchestrator for use by event triggers (Phase 4). - Tests cover unseen domain, freshness guard, force recompute, LLM-field preservation, blank input, and case-insensitive input. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 3b of recommendation system v2. - Recommendation::LlmDomainClassifier: replaces the existing AuctionDomainClassifier with a richer JSON schema covering description, description_locale, keywords, audience, languages, suggested_use_cases, brandability_score, confidence. Tags and primary_category are enum-constrained to InterestCatalog. Raw response is captured so future schema migrations can re-parse without re-billing OpenAI. Accepts plain domain names rather than Auction records, decoupling classification from the auctions table. - Recommendation::ClassifyUnclassifiedDomainsJob: cron-only entry point. Pulls DomainClassification rows that need LLM enrichment (heuristic source, low confidence, or stale openai rows), batches them at LlmDomainClassifier::BATCH_LIMIT, and upserts. Guarded by Feature.open_ai_integration_enabled?. Logs processed count. Provides needs_to_run? for the admin Job UI. - lib/tasks/recommendation.rake: defines classify_unclassified, embed_unembedded, and backfill task targets for k8s CronJobs. Placeholder branches gracefully skip until later phases land. - app/models/job.rb: registers the new job in ALLOWED_JOB_NAMES so admins can also trigger it manually. - Tests cover feature-flag gating, scope selection (heuristic, low-conf, stale, fresh-llm), and needs_to_run? predicate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 4 of recommendation system v2. Triggers fire ClassifyDomainHeuristicallyJob (Tier 0 only — heuristic, free, no external calls). LLM enrichment still happens exclusively through the nightly cron job. - Auction after_create: enqueue_domain_classification by domain_name. - WishlistItem create: trigger on @wishlist_item.domain_name so domains that aren't auctions still get tags. - OffersController create/update: trigger on @auction.domain_name so legacy auctions created before this feature still get classified when someone bids. - EnglishOffersController create/update: same pattern. BackfillDomainClassificationsJob: - Collects unique domain names from Auction, WishlistItem, DomainOfferHistory, Result via safe_pluck (skips missing tables/ columns). - Skips already-classified domains. - Runs heuristic for each via DomainClassifier (idempotent). - Logs progress. - Invoked by `rake recommendation:backfill` (k8s one-shot). Tests cover heuristic-job idempotency, blank input, and the auction-create -> classification enqueue path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 5 of recommendation system v2. Dependencies / schema: - Gemfile: add neighbor (~> 0.5) for pgvector ActiveRecord integration. - Migration enable_pgvector_and_add_embeddings: enables 'vector' extension (idempotent), adds embedding vector(1536), embedding_model, embedded_at columns, and an HNSW index for cosine distance. - AWS RDS Postgres 17.4 supports pgvector natively; no Dockerfile or Terraform changes required. CREATE EXTENSION may need a one-time manual run as rds_superuser if the app DB role lacks the grant. Model: - DomainClassification has_neighbors :embedding (guarded so model loads before the migration has run). - New needs_embedding scope, also guarded. Services / jobs: - Recommendation::DomainEmbedder wraps OpenAI text-embedding-3-small, batches up to BATCH_LIMIT=100 rows per call. Input is "domain_name. description. keywords...". Accepts both records and hashes. - Recommendation::EmbedUnembeddedDomainsJob is the cron-only entry point. Picks rows with description but no embedding, batches them, upserts vectors via update_columns. Guarded by feature flag and presence of the embedding column. Wiring: - Job model registers EmbedUnembeddedDomainsJob in ALLOWED_JOB_NAMES. - rake recommendation:embed_unembedded already wired in Phase 3b. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 6 of recommendation system v2. Recommendation::Scorer rewritten to consume the v2 signal stack: - Tags + keywords + audience pulled from domain_classifications, joined by domain_name. Falls back to auctions.classification_tags for legacy or yet-to-be-classified domains. - Behavioural affinity (bids, wishlist, views, domain_offer_history) is time-decayed with HALF_LIFE_DAYS=60 via exp(-days/60). Old signals naturally vanish without explicit cutoff dates. - New audience match bonus (+10) when domain_classifications.audience matches a profile preference. - View affinity (weight 4, cap 12) sourced from RecommendationEvent#auction_detail_view (Phase 7 hooks it up). - DomainOfferHistory affinity (weight 3, cap 12) — Estonian-auction historical bids beyond the offers table. - Result signal: lost auctions on a tag boost the tag (+25 decayed); won auctions slightly damp it (-5 decayed). Defensive guards around variant Result schemas. - Embedding multiplier: 1 + max(0, cosine(auction.embedding, user_centroid)) where user_centroid is the time-decayed weighted average of embeddings from bids + wishlist + views. Guarded so it returns 1.0 when pgvector column missing or no user signals. - Classifications preloaded for the entire @scope up front to avoid N+1. - FEATURES_VERSION='rich_v1' and BASELINE_MODEL_NAME='baseline_rules_v2' mark scored rows so a future model upgrade can detect stale rows. Existing scorer_test.rb relies on auctions.classification_tags fallback path and continues to work. New scorer_rich_features_test.rb verifies keyword overlap boost, time-decay attenuation of ancient bids, and features_version metadata. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 7 of recommendation system v2. Server-side capture: - OffersController#new and EnglishOffersController#new now call EventTracker with event_type='auction_detail_view'. Source field records which controller fired so debugging is easy. - Both ignore unauthenticated requests via the existing authenticate_user! pipeline. Client-side capture: - New Stimulus controller recommendation_dwell_controller attaches to auction cards and fires an auction_detail_view event ONLY after the card stays at >= 50% visibility for dwellMsValue (default 1500ms). Uses navigator.sendBeacon when available so events survive page navigation. - Registered in app/javascript/controllers/index.js. - Cards opt in by attaching the controller and providing recommendation-dwell-auction-uuid-value. No-op when the value is absent or IntersectionObserver is unsupported. Scorer-side: view_feature_aggregate (weight 4, cap 12) already wired in Phase 6 reads from RecommendationEvent#auction_detail_view, so these new signals flow directly into the score. Integration test verifies offers#new persists the detail-view event with the expected user, auction, and source attribution. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 8 of recommendation system v2.
- AuctionsController#index preloads DomainClassification rows for the
paginated auctions into @domain_classifications (single batched
SELECT keyed by lowercased domain_name). Defensively returns {}
if the table is not yet migrated.
- index.html.erb passes the matching DomainClassification to each
row partial as a local.
- _auction.html.erb renders, when present:
- description as a muted paragraph beneath the domain name
- first four keywords as small badges
Layout degrades silently when nothing is classified.
- Each authenticated row gains recommendation-dwell Stimulus values
so a 1.5s in-viewport hover fires auction_detail_view via
navigator.sendBeacon. Anonymous visitors don't trigger tracking.
Layout uses inline styles to avoid touching the global CSS pipeline
in this phase; a follow-up can move them to dartsass partials.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 9 of recommendation system v2. UX polish: - UsersController#create: if a brand-new user signed up without filling the recommendation profile, call dismiss_prompt! on the empty profile. This stops the modal from popping up on the next /auctions render right after they explicitly skipped the form. They will see it again after PROMPT_REMINDER_INTERVAL (14 days). Documentation: - docs/architecture/recommendation-system.md: mark every phase as done, link to the operator runbook. - docs/guides/recommendation-operations.md: new operator-facing runbook with environment prerequisites, k8s cron schedule, first-time rollout checklist, monitoring queries, tunable constants, rollback steps, and a troubleshooting section covering pgvector permission errors and language mismatches. - CHANGELOG: high-level entry under today's date. No code changes outside the sign-up dismiss and docs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-review pass after iteration 1. Scorer improvements: - compute_result_signal previously did one SELECT per Result row to fetch tags. preload_result_classifications now batches a single SELECT keyed by lowercased domain_name. - dominant_user_audience replaces the unreachable profile.audience_preference branch. Audience preference is inferred from the user's own bid+wishlist history (majority audience across classified domains). Cached per scorer instance. - bid_wishlist_classification_cache memoises the join so audience inference doesn't trigger extra round-trips when several auctions are scored in the same pass. Deprecation markers: - Recommendation::AuctionDomainClassifier and Recommendation::ClassifyAuctionDomainsJob get clear DEPRECATED comments. Both stay live so the legacy auctions.classification_* fallback path keeps working until those columns are dropped in a later release. New tests: - LlmDomainClassifierTest uses WebMock stubs to verify structured- output parsing, enum filtering of bogus tags, batch-limit truncation, empty input handling, and incomplete-response error. - DomainEmbedderTest covers vector length, AR-row input, empty input, and OpenAI error response. Test compatibility verified by walking through the legacy scorer_test expectations: - wishlist > category > custom > neutral ordering preserved via the auctions.classification_tags fallback path in aggregate_features. - bid-tag affinity still flows through the same fallback for fixtures that have no domain_classifications row. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Final polish pass. Style extraction: - _auction.html.erb dropped all inline style attributes. - New _recommendation.scss component module defines .c-auction__domain-description, .c-auction__domain-keywords, and .c-auction__domain-keyword. Imported from _components.scss. Tests: - BackfillDomainClassificationsJobTest covers full happy-path classification of auction + wishlist domains, skip-existing behaviour, and resilience to per-domain failures. - EmbedUnembeddedDomainsJobTest covers no-op without the pgvector column, no-op when OpenAI is disabled, and scope filtering of rows without descriptions. Architectural decision record: - docs/architecture/adr-001-recommendation-v2.md captures the seven design decisions of v2: per-domain classification table, three-tier pipeline, RDS-native pgvector with no Dockerfile changes, infra- side cron, time decay over SQL date cutoffs, multiplicative embedding multiplier, and heuristic-only-at-runtime policy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 5 migration crashed on local dev because docker-images/ docker-compose.dev.v2.yml uses plain postgres:13.4 which has no pgvector. Production (RDS 17) was always fine. Two fixes: Migration: - ensure_pgvector_available! probes pg_available_extensions before trying CREATE EXTENSION and raises a clear PgvectorUnavailable error with remediation steps (point at the pgvector image and the SKIP env var) instead of the cryptic "vector.control not found". - Down migration uses column_exists? guards so a partial-rollback doesn't blow up. - SKIP_PGVECTOR_MIGRATION=true env var lets operators skip the whole migration when they cannot upgrade the image right away; the recommendation system degrades to tag/keyword scoring with no similarity multiplier and can be re-migrated later. Operations: - Updated docs/guides/recommendation-operations.md prerequisites table and troubleshooting section with the dev-image guidance and the SKIP env-var instructions. Note: the shared compose change to pgvector/pgvector:pg13 lives in registry/docker-images/docker-compose.dev.v2.yml (separate repo) and is staged as part of this hotfix flow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 5 of the recommendation v2 plan (pgvector + embedding similarity) is reversed before merge. Rationale: provisioning pgvector touches shared infrastructure across dev, staging, test and prod for marginal ranking value. See docs/architecture/adr-001-recommendation-v2.md decision D3 for the full reasoning. Removed: - Gemfile / Gemfile.lock: neighbor gem dropped - app/services/recommendation/domain_embedder.rb - app/jobs/recommendation/embed_unembedded_domains_job.rb - test/services/recommendation/domain_embedder_test.rb - test/jobs/recommendation/embed_unembedded_domains_job_test.rb - Embedding multiplier, user centroid, cosine similarity helpers, embedding_for, EMBEDDING_DIMENSIONS, needs_embedding scope, and has_neighbors association from Scorer / DomainClassification. - ALLOWED_JOB_NAMES entry for EmbedUnembeddedDomainsJob. - `recommendation:embed_unembedded` rake task. Migration 20260527090100_enable_pgvector_and_add_embeddings now does the inverse: if a local environment previously applied the original version and has the embedding column, the up branch drops it cleanly. Production and fresh local environments hit a no-op. What stays: - domain_classifications table including description, keywords, audience, suggested_use_cases — all the rich fields the scorer actually uses. - LLM enrichment via nightly cron, heuristic Tier 0 at runtime. - Tag overlap, keyword overlap, behavioural affinity with time decay, audience inference, result signals, structural bonuses, view tracking. Docs updated: architecture/recommendation-system.md, technical/domain-classification-pipeline.md, guides/recommendation-operations.md, architecture/adr-001-recommendation-v2.md, CHANGELOG. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Description was originally added to back UI card copy and to feed the embedding text input. With embeddings dropped (and now coming back as keywords-only input — see follow-up commit) the description no longer earns its OpenAI tokens. Schema: - New migration 20260527090200 removes description and description_locale columns from domain_classifications. Services: - LlmDomainClassifier JSON schema and system prompt drop the description / description_locale fields. Required-fields list trimmed accordingly. - DomainHeuristicClassifier no longer emits description / locale in its attribute hash. - DomainClassifier orchestrator's preserve_llm_fields! no longer references description fields. UI: - _auction.html.erb stops rendering the description paragraph. Keywords-as-badges remain. - _recommendation.scss simplified to keyword-badge styling only. Tests: - LlmDomainClassifierTest assertions and fixtures dropped the two description fields. - DomainClassifierTest preserve-LLM-fields case switched to assert keywords are preserved instead of description. Docs: - recommendation-system.md, adr-001, and operations runbook updated to reflect the schema change and the keyword-only UX. structure.sql includes the original create-table migration run from local dev — the new remove-column migration will rewrite it cleanly on next db:migrate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings back vector similarity matching without pgvector, without any
shared-infrastructure changes. Trade-off documented in ADR-001 D3.
Schema:
- Migration 20260527090300 adds three columns to domain_classifications:
embedding double precision[]
embedding_model string
embedded_at datetime (indexed)
- No extensions, no special types. Works on plain Postgres 13/14/15/16/17.
Services:
- Recommendation::DomainEmbedder wraps OpenAI text-embedding-3-small
(1536 dim). Input is "<domain_name>. <keywords>" — description was
dropped earlier, keywords carry the semantic context. Batches up to
BATCH_LIMIT=100 rows per OpenAI call. Accepts both ActiveRecord rows
and plain hashes.
- Recommendation::EmbedUnembeddedDomainsJob is the cron-only entry
point (rake recommendation:embed_unembedded). Picks classified rows
without an embedding, batches them, persists via update_columns.
Guarded by Feature.open_ai_integration_enabled? and column presence.
Scorer:
- score_for now finishes with `score *= embedding_multiplier(auction)`.
- Multiplier formula:
1 + max(0, cosine_similarity(user_centroid, auction.embedding))
Range 1.0..2.0. No-op (1.0) when:
- embedding column not migrated
- user has no behavioural history yet
- auction not embedded yet
- User centroid = time-decayed weighted average of embeddings from
bids + wishlist + recent views.
- cosine_similarity computed in plain Ruby — ~50ms across 200 vectors
at 1536 dims. No HNSW, no vector index needed at this scale.
Tests:
- DomainEmbedderTest: vector shape, AR-row input, empty input,
OpenAI error handling.
- EmbedUnembeddedDomainsJobTest: column-missing safety, feature-flag
gating, scope selection.
- ScorerEmbeddingTest: aligned-vector boost, no-op without history,
with explicit skips when the embedding column is not yet migrated.
Wiring:
- Job model registers EmbedUnembeddedDomainsJob in ALLOWED_JOB_NAMES.
- rake recommendation:embed_unembedded restored.
Docs:
- architecture/recommendation-system.md, technical/domain-classification-pipeline.md,
guides/recommendation-operations.md and ADR-001 updated to reflect
the embedding path and the rationale for Postgres-native storage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- DEMO_DOMAINS grouped by InterestCatalog category (~140 domains across local_service, health, shop_brand, saas, b2b_service, finance, legal, education, travel, automotive, real_estate, media_content, brandable, numeric, other). Mix of Estonian and English roots so the heuristic classifier hits cleanly on dictionary entries and falls through to LLM on novel patterns. - demo:create_blind_auctions now seeds the full set (was 57, now 140). - New demo:create_varied_auctions creates a small fan-out of auctions with different ends_at horizons (hour/day/week/month) so the SCORING_HORIZON=30.days clipping in Scorer can be exercised. - New demo:seed_user_signals attaches a recommendation profile, wishlist items and historical Offers to the first participant user, then enqueues a score refresh — gives a one-command setup for verifying the personalised /auctions sort.
`starts_at = Time.zone.now + 1.second` was racing the starts_at_cannot_be_in_the_past validation: by the time Rails 8.1 finished booting and the validation ran, the timestamp was already in the past. Tighter Time.current check in the validator (or just slower boot) tipped it over. Bumped the buffer to 5.minutes via a shared AUCTION_START_BUFFER constant used by both demo:create_blind_auctions and demo:create_varied_auctions. ends_at is now derived from starts_at so the relative horizons stay correct.
Contributor
🚀 Deploy Complete!
(Environment ready for testing) |
|
Contributor
🚀 Deploy Complete!
(Environment ready for testing) |
1 similar comment
Contributor
🚀 Deploy Complete!
(Environment ready for testing) |
Contributor
🚀 Deploy Complete!
(Environment ready for testing) |
1 similar comment
Contributor
🚀 Deploy Complete!
(Environment ready for testing) |
Add a per-domain Postgres advisory lock to ClassifyDomainJob so concurrent enqueues (auction after_create + first bid/offer/wishlist) can no longer each fire a paid OpenAI call for the same domain; a worker that cannot grab the lock skips (the nightly ClassifyUnclassifiedDomainsJob is the backstop). Reset embedding/embedding_model/embedded_at on every (re)classification so EmbedUnembeddedDomainsJob recomputes the vector after keywords change, closing the gap where a re-classified domain kept its stale embedding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…gnals Respect the wishlist_size Setting instead of blindly create!-ing, so a participant whose wishlist is already full no longer aborts the task. Switch wishlist and offer creation to non-bang save with error logging, and report what was actually added rather than what was requested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Orchestration: - PipelineRunner: classify -> embed -> ai_score -> per-user scores, run synchronously with batch draining and a placeholder-prompt guard - rake recommendation:init (prod, incremental) and :init_demo (staging: active mock auctions + signals, then pipeline) - RebuildRecommendationsJob in /admin/jobs (force re-tag after the interest catalog changes) + localized EN/ET notice on the categories page Fixes: - refresh_single_user job re-enqueues instead of silently dropping a debounced recompute - backfill upsert resets stale embeddings (mirrors classify job) - InterestCategory#destroy purges orphaned codes from profiles and domain classifications - correct stale comments in classify_domain_job / llm_domain_classifier Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Delete unused DomainClassification scopes (unclassified, by_source,
low_confidence) and #heuristic?, RecommendationProfile#interest_categories_labels,
Auction#classified? — no call sites anywhere
- Scorer: memoize the four signal collectors (bid/wishlist/view/offer-history);
they were re-queried 2-4x per refresh, now once
- Add missing recommendation_profiles.summary.{custom_interests,length} keys
(EN/ET) that summary_lines rendered as "translation missing"
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Performance (behavior-preserving): - Add functional index on LOWER(domain_classifications.domain_name) so the /auctions user-sort join uses an index instead of seq-scanning - Add wishlist_items(user_id) index for the recommendation hot paths Refactors: - Extract embedding vector math (cosine similarity, weighted centroid) from the 579-line Scorer into a pure Recommendation::Embedding module + unit tests - Move prompt-lifecycle side effects (rescore enqueue + event tracking) from RecommendationProfilesController/UsersController into RecommendationProfile #complete!/#skip!/#dismiss!, deduping the two controllers - Drop the redundant user_auction_scores uniqueness validation (DB unique index + upsert_all already enforce it; no form surfaces it) - Remove dead error-swallowing rescues in Scorer result signals (guards already cover the model-absent case) - Extract the removable-interest badge markup into .c-badge--interest / .c-badge__remove so the ERB and Stimulus renderings can't drift; share the CSRF-token lookup between the two recommendation-event controllers Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the v2 hybrid (string-matched interests + ~12 hand-tuned weights + embedding multiplier) with a single vector space where everything about the user — bids, wishlist, views, selected categories, custom interests — is an embedding "magnet". A candidate is scored by the mean of its two strongest weighted-cosine pulls; no magnet/embedding means no row, so it falls to the ai_score/RANDOM tail. Phase A (data + enrichment): - interest_categories: description, keywords[], embedding[], embedded_at - recommendation_profiles: custom_interest_vectors (jsonb), embedded_at - InterestCategoryEnricher (LLM keywords EN+ET → embedding), TextEmbedder - EnrichInterestCategoriesJob, EmbedCustomInterestsJob + model auto-triggers - PipelineRunner gains enrich_categories + force-mode invalidation Phase B (shadow): MagnetScorer computes v3 in memory (no persistence); rake recommendation:compare_versions prints v2 vs v3 side by side. Phase C (switchover): Scorer writes v3 as primary (magnet base x100 + length/digits/hyphen + result signal); drops WISHLIST_HIT, TAG/KEYWORD/ AUDIENCE weights, all affinities, embedding multiplier, centroid, ai_prior, SIMILAR_DOMAIN_BONUS. UserSortable collapses 5 tiers to 3 (offer / wishlist / score-or-tail); interest_match_sql and its domain_classifications join removed. Tests: 159 runs, 0 failures in Docker. Removed obsolete v2 scorer tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The after_save_commit trigger only embeds a profile's custom interests on save, so users who set free-text interests before v3 (or before a force reset) would never get custom magnets on `recommendation:init`. Add an idempotent embed_custom_interests stage (only profiles with unembedded custom interests) plus a force-mode invalidation, so init fully processes existing users. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two operator-reported fixes: 1. Custom (free-text "other") interests could not be removed. The profile form had no empty sentinel for custom_interests[], so removing every tag dropped the param entirely, strong-params discarded it, and the setter that clears old values never ran. Add the sentinel; make `other` a purely derived marker of "has custom interests" so a dangling checkbox can't linger as an empty category. 2. Admin auction list & detail now surface the LLM-added classification — category, tags, keywords, audience, languages, use cases, brandability, confidence, source/model, and embedding status. Preloaded by domain name (no N+1); joined the same way the scorer does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Behind the auction_tags_display_enabled flag in customization.yml (default off). When enabled, the public auction list swaps the "auction type" column for the domain's LLM-derived tags (rendered as badges); when off, the auction-type icon is shown exactly as before. Tags come from DomainClassification, joined by domain name. The list preloads them into a hash (no N+1) only when the flag is on; the single-row turbo-stream path falls back to one lookup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Missing key already resolves to nil (feature off) via OrderedOptions, so the absence is safe; add it to the sample for discoverability. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The extra column overflowed the already-wide admin auctions table. Keep the full LLM breakdown on the auction detail page (vertical layout, no width pressure); only the list column is removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The category-side LLM enrichment (description, keywords, embedding) produced by InterestCategoryEnricher was stored but never surfaced. Add an "AI status" badge to the categories list (enriched/pending + keyword count) and a read-only enrichment panel on the edit page showing the description, keyword badges, embedding status, model and timestamp — with a clear "not enriched yet" note and how to generate it when the vector is missing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add an operator guide: the OpenAI feature flag prerequisite, what runs automatically (event-driven jobs), what runs on cron, what to run manually (recommendation:init after deploy, RebuildRecommendationsJob after catalog changes), where to see results in admin, and the experimental tags flag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Address the ranking gaps surfaced in review: the per-domain embedding was built from a deliberately sparse "<domain>. <keywords>" text, so custom interests and wishlist look-alikes barely moved the ranking (cosine over coarse vectors, no string matching). Embedding enrichment (rolls out through the existing pipeline, no new tasks): - DomainEmbedder#build_input now builds a rich structured text (description + category + tags + use cases + audience + keywords), skipping blank parts. Bump DomainEmbedder::INPUT_VERSION so needs_embedding re-embeds legacy rows on the next `recommendation:init` (explicit NULL branch — SQL where.not excludes NULLs). RebuildRecommendationsJob backfills descriptions. - LlmDomainClassifier emits a new `description` (schema + prompt + persisted column via migration); shown on the admin auction classification panel. - Custom-interest backfill in PipelineRunner now re-embeds a profile when its stored vectors no longer match its interests, so a silently-failed embed (OpenAI off / worker down at save time) self-heals on the next init. Event retention: - PruneRecommendationEventsJob deletes recommendation_events >6mo and auction_impression >1mo in batches; on /admin/jobs and `prune_events` rake. Slim the operational surface (7 rake tasks -> 4): - Drop the classify_unclassified/embed_unembedded rake wrappers (init already drains those batched jobs; the jobs stay, reachable via init and /admin/jobs) and the dev-only compare_versions inspector. - Cron is now a single command: `recommendation:init` (+ `prune_events`). - Rewrite the README recommendation section into one flow-map and fix the job comments that pointed at the removed tasks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The "Other" checkbox was a client-side-only toggle that revealed the free-text keyword input. Being the last item in the category grid it was easy to miss, and unchecking it did not remove already-added keywords — users could overlook the feature entirely (we have had support emails asking for functionality that already exists). Server-side, `other` is a derived marker: RecommendationProfile derives it from having any free-text keyword and excludes it from scoring, so the checkbox never affected stored data. Removing it aligns the UI with the model: - _fields.html.erb: exclude `other` from the checkbox grid; the keyword field is now always visible. - custom_interest_tags_controller.js: drop the otherToggle/field targets and the toggle/syncVisibility logic; keep add/remove. - Relabel to "Add your own keywords" / "Lisa oma märksõnad". No model/controller/migration change: the sentinel empty param (needed to clear the last keyword) stays, and the server still strips a stray `other` posted without custom interests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Personalises the public auction list. Every auction is ranked per user by how
well the domain matches their behaviour and stated interests, built on a single
embedding space ("magnets"): everything about a user — bids, wishlist, views,
selected interest categories, free-text interests — becomes a vector, and
domains are ranked by their pull towards those vectors. Multilingual for free
(Estonian ≈ English in the embedding space).
Admins can also see the AI-generated data: LLM classification on the auction
detail page, and enrichment (description / keywords / embedding) on interest
categories.
How to use the recommendation system
Prerequisite: feature flag
The whole system is gated by the OpenAI integration in
config/customization.yml:When disabled, nothing runs and the list falls back to the global
ai_score/default order. A running job worker (
bundle exec rails jobs:work) is requiredfor every
*_laterjob.Runs automatically (event-driven — no manual action)
ClassifyDomainJob(Auction after_create)RefreshSingleUserAuctionScoresJob(30 s debounce, viaEventTracker)RefreshSingleUserAuctionScoresJob(debounced)EmbedCustomInterestsJobEnrichInterestCategoriesJobIn normal operation this is all that happens — the system keeps itself current.
Runs on a schedule (cron)
Batched catch-up for bulk-created or stale (6-month refresh) domains. Schedule
these like the other cron jobs (outside the app):
Run manually
After every deploy (and after first enabling the feature) run the full
pipeline once. It is incremental (
force: false) — enriches interestcategories, backfills embeddings for existing users' custom interests,
classifies + embeds any domains still missing it, refreshes the global
ai_score, and recomputes every participant's personal scores. Work alreadydone is not redone.
bundle exec rake recommendation:initOther manual entry points:
rake recommendation:initrake recommendation:init_demorake recommendation:backfillrake recommendation:compare_versionsLIMIT=,USER_ID=)Recommendation::RebuildRecommendationsJobon /admin/jobsforce: true). Run after the interest-category catalog changes (add/rename/delete)Where to see results (admin)
the edit page shows the generated description, keywords and embedding details.
keywords, audience, languages, brandability, confidence, embedding status).
Experimental: tags on the public list
auction_tags_display_enabledinconfig/customization.yml(defaultfalse)swaps the "auction type" column on the public auction list for the domain's
LLM-derived tags. A missing key is treated as
false.Deployment note
The pipeline includes a backfill step for existing users' custom interests, so
running
rake recommendation:initonce after deploy is enough — existinginterests and already-processed domains are handled without re-processing.