diff --git a/.gitignore b/.gitignore index cfa756a62..579f3e396 100644 --- a/.gitignore +++ b/.gitignore @@ -69,4 +69,10 @@ CLAUDE.md /app/assets/builds/* !/app/assets/builds/.keep .cursorindexingignore -.specstory \ No newline at end of file +.specstory +# Local Claude Code workspace artifacts +.claude/ + +# AI / working docs — kept locally, not committed to the shared repo +/docs/ +/style-guide/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 2f0029297..72595f18d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,14 @@ +27.05.2026 +* Recommendation system v2: per-user auction sorting backed by + domain_classifications, time-decayed bid/wishlist/view affinity, + and nightly LLM enrichment as a k8s CronJob + (rake recommendation:classify_unclassified). No shared-infrastructure + changes — no RDS extensions, no Docker image bumps for the shared + dev Postgres. See docs/architecture/recommendation-system.md and + docs/architecture/adr-001-recommendation-v2.md. +* Batch impressions on /auctions index via insert_all (one query + regardless of page size), debounced score recompute on user actions. + 30.12.2025 * Allow comma in billing profile street field https://github.com/internetee/auction_center/issues/1502 * Fixed false error message bug on billing profile update https://github.com/internetee/auction_center/issues/1503 diff --git a/README.md b/README.md index fed0889a9..64b3add2c 100644 --- a/README.md +++ b/README.md @@ -59,6 +59,84 @@ To send out emails and perform other asynchronous tasks, we use a background pro Part of running the application according to EIS business rules includes creating new auctions at the beginning of the day. Jobs are scheduled outside of the application, as the exact times are no concern of the application. +## Recommendation system + +The public auction list is personalised: each user sees auctions ranked by how +well every domain matches their behaviour and stated interests. Ranking is +built on a single embedding space ("magnets") — everything about a user (bids, +wishlist, views, selected categories, free-text interests) becomes a vector, +and domains are ranked by their pull towards those vectors. + +### Prerequisite: feature flag + +The whole system is gated by the OpenAI integration. In +`config/customization.yml`: + +```yaml +openai: + enabled: true + access_token: 'sk-...' +``` + +When disabled, no classification, embedding or scoring runs and the list falls +back to the global `ai_score` / default order. A running job worker +(`bundle exec rails jobs:work`) is required for every `*_later` job below. + +### What runs automatically (event-driven, no manual action) + +| Trigger | Job | Effect | +|---|---|---| +| A new auction is created | `ClassifyDomainJob` (`Auction after_create`) | Classifies + embeds that domain | +| A user bids / wishlists / views a domain | `RefreshSingleUserAuctionScoresJob` (30 s debounce, via `EventTracker`) | Re-scores that one user | +| A user saves their interest profile | `RefreshSingleUserAuctionScoresJob` (debounced) | Re-scores that user | +| A user adds/edits free-text ("other") interests | `EmbedCustomInterestsJob` | Embeds the new interests, then re-scores | +| An admin adds/renames an interest category | `EnrichInterestCategoriesJob` | Generates description + keywords + embedding for that category | + +In normal operation this is all that happens — the system keeps itself current +as auctions, bids and profiles change. + +### What runs on a schedule (cron) + +Only two scheduled commands (outside the app, like the other cron jobs): + +```bash +bundle exec rake recommendation:init # the whole pipeline, incremental +bundle exec rake recommendation:prune_events # retention: events >6mo (impressions >1mo) +``` + +`init` is the single pipeline entry point. It is incremental (`force: false`): +it enriches interest categories, backfills embeddings for existing users' custom +interests, then **drains the batched classify + embed passes itself** — picking +up any domain that is unclassified, stale (>6mo), or whose vector was built under +an older embedding-input format (`DomainEmbedder::INPUT_VERSION`, so a format +bump rolls out on the next run with no extra command) — refreshes the global +`ai_score`, and recomputes every participant's personal scores. Already-done work +is **not** redone. Run it on every deploy; the same run also serves as the cron +catch-up, so there is no separate classify/embed cron task. + +### What you run manually (rarely) + +| Command / action | When to use | +|---|---| +| `rake recommendation:backfill` | **Once**, when first enabling the feature — classifies the full historical domain set (ended auctions, wishlist, offer histories, results), a wider universe than `init` (active only), so past-bid domains get embeddings and can form magnets | +| `rake recommendation:init_demo` | Staging/test only — seeds mock active auctions + signals, then runs the pipeline | +| `Recommendation::RebuildRecommendationsJob` on **/admin/jobs** | Full re-tag of **every** domain (`force: true`): re-classifies via the LLM (fills the `description` field) and re-embeds. Run after the interest-category catalog changes, or once to backfill descriptions after the embedding-enrichment change | + +### Where to see the results (admin) + +- **/admin/interest_categories** — each category shows its AI status + (enriched / pending); the edit page shows the generated description, + keywords and embedding details. +- **/admin/auctions/:id** — the auction detail page shows the domain's full LLM + classification (category, tags, keywords, audience, languages, brandability, + confidence, embedding status). + +### Experimental: tags on the public list + +`auction_tags_display_enabled` in `config/customization.yml` (default `false`) +swaps the "auction type" column on the public auction list for the domain's +LLM-derived tags. A missing key is treated as `false`. + ## Audits Due to various regulatory requirements, all database tables are audited according to the following procedure: diff --git a/app/assets/stylesheets/components/_account.scss b/app/assets/stylesheets/components/_account.scss index 09b15cc3b..5125e041d 100644 --- a/app/assets/stylesheets/components/_account.scss +++ b/app/assets/stylesheets/components/_account.scss @@ -157,6 +157,8 @@ .c-acount__buttons { gap: 5px; + // Separate the Billing / Delete row from the "Edit interests" button above it. + margin-top: 24px; @include mq(mobile) { gap: 10px; diff --git a/app/assets/stylesheets/components/_badge.scss b/app/assets/stylesheets/components/_badge.scss index e3d2b1eee..74683f661 100644 --- a/app/assets/stylesheets/components/_badge.scss +++ b/app/assets/stylesheets/components/_badge.scss @@ -60,6 +60,21 @@ } +// Removable interest tag: spacing + a bare "x" button. Shared by the +// server-rendered tags (recommendation_profiles/_fields) and the ones the +// custom-interest-tags Stimulus controller injects, so the two can't drift. +.c-badge--interest { + margin: rem(4px) rem(8px) rem(4px) 0; +} + +.c-badge__remove { + border: none; + background: transparent; + cursor: pointer; + padding: 0; + line-height: 1; +} + .c-badge--gray { background-color: $grey-100-color; color: $grey-400-color; diff --git a/app/components/common/header/component.rb b/app/components/common/header/component.rb index 3fd0ad5e8..61dc3b0a2 100644 --- a/app/components/common/header/component.rb +++ b/app/components/common/header/component.rb @@ -57,6 +57,7 @@ def admin_menu_list_items { name: t(:invoices_name), path: admin_invoices_path }, { name: t(:jobs_name), path: admin_jobs_path }, { name: t(:settings_name), path: admin_settings_path }, + { name: t(:interest_categories_name), path: admin_interest_categories_path }, { name: t(:paid_deposits_name), path: admin_paid_deposits_path }, { name: t(:statistics_name), path: admin_statistics_path }] end diff --git a/app/components/pages/auction/auction_action_button/component.html.erb b/app/components/pages/auction/auction_action_button/component.html.erb index 0a2951ea8..333d40679 100644 --- a/app/components/pages/auction/auction_action_button/component.html.erb +++ b/app/components/pages/auction/auction_action_button/component.html.erb @@ -6,7 +6,17 @@ <% else %> <%# if need to create new offer for english auction %> <%= component 'common/links/link_button', link_title: deposit_value[:link_title], href: new_auction_english_offer_path(auction_uuid: auction.uuid), - color: deposit_value[:color], options: { data: { turbo_frame: 'modal' } } %> + color: deposit_value[:color], + options: { + data: { + turbo_frame: 'modal', + controller: 'recommendation-tracker', + action: 'click->recommendation-tracker#track', + recommendation_tracker_auction_uuid_value: auction.uuid, + recommendation_tracker_source_value: 'auction_action_new_english_offer', + recommendation_tracker_event_type_value: 'auction_click' + } + } %> <% end %> <% else %> @@ -17,7 +27,17 @@ <% else %> <%# if need to create new offer for blind auction %> <%= component 'common/links/link_button', link_title: t('auctions.bid'), href: new_auction_offer_path(auction_uuid: auction.uuid), - color: 'green', options: { data: { turbo_frame: 'modal' } } %> + color: 'green', + options: { + data: { + turbo_frame: 'modal', + controller: 'recommendation-tracker', + action: 'click->recommendation-tracker#track', + recommendation_tracker_auction_uuid_value: auction.uuid, + recommendation_tracker_source_value: 'auction_action_new_blind_offer', + recommendation_tracker_event_type_value: 'auction_click' + } + } %> <% end %> <% end %> diff --git a/app/components/pages/auction/auction_action_button/edit_and_remove_blind_offer/component.html.erb b/app/components/pages/auction/auction_action_button/edit_and_remove_blind_offer/component.html.erb index 97d99a731..0a7bb378a 100644 --- a/app/components/pages/auction/auction_action_button/edit_and_remove_blind_offer/component.html.erb +++ b/app/components/pages/auction/auction_action_button/edit_and_remove_blind_offer/component.html.erb @@ -1,6 +1,16 @@
<%= component 'common/links/link_button', link_title: nil, href: edit_offer_path(@auction.users_offer_uuid), - color: 'ghost', options: { data: { turbo_frame: 'modal' } } do %> + color: 'ghost', + options: { + data: { + turbo_frame: 'modal', + controller: 'recommendation-tracker', + action: 'click->recommendation-tracker#track', + recommendation_tracker_auction_uuid_value: @auction.uuid, + recommendation_tracker_source_value: 'auction_action_edit_blind_offer', + recommendation_tracker_event_type_value: 'auction_click' + } + } do %> <% end %> <%= component 'common/buttons/button_to', title_caption: nil, href: offer_path(@auction.offer_from_user(@user).uuid), color: 'ghost', diff --git a/app/components/pages/auction/auction_action_button/edit_english_offer/component.html.erb b/app/components/pages/auction/auction_action_button/edit_english_offer/component.html.erb index f35ad3610..92fe35a1f 100644 --- a/app/components/pages/auction/auction_action_button/edit_english_offer/component.html.erb +++ b/app/components/pages/auction/auction_action_button/edit_english_offer/component.html.erb @@ -1,6 +1,16 @@
<%= component 'common/links/link_button', link_title: nil, href: edit_english_offer_path(@auction.users_offer_uuid), - color: 'ghost', options: { data: { turbo_frame: 'modal' } } do %> + color: 'ghost', + options: { + data: { + turbo_frame: 'modal', + controller: 'recommendation-tracker', + action: 'click->recommendation-tracker#track', + recommendation_tracker_auction_uuid_value: @auction.uuid, + recommendation_tracker_source_value: 'auction_action_edit_english_offer', + recommendation_tracker_event_type_value: 'auction_click' + } + } do %> <% end %>
diff --git a/app/controllers/admin/auctions_controller.rb b/app/controllers/admin/auctions_controller.rb index 2a8631c43..0d9b9fe2b 100644 --- a/app/controllers/admin/auctions_controller.rb +++ b/app/controllers/admin/auctions_controller.rb @@ -37,6 +37,7 @@ def index # GET /admin/auctions/1 def show + @classification = DomainClassification.find_by(domain_name: @auction.domain_name.to_s.downcase) @offers = @auction.offers.order(cents: :desc) users = User.search_deposit_participants(params) @pagy, @users = pagy(users, items: params[:per_page] ||= 20, link_extra: 'data-turbo-action="advance"') diff --git a/app/controllers/admin/interest_categories_controller.rb b/app/controllers/admin/interest_categories_controller.rb new file mode 100644 index 000000000..5b7d87d2d --- /dev/null +++ b/app/controllers/admin/interest_categories_controller.rb @@ -0,0 +1,64 @@ +module Admin + class InterestCategoriesController < BaseController + before_action :authorize_user + before_action :set_interest_category, only: %i[edit update destroy] + + # GET /admin/interest_categories + def index + @interest_categories = InterestCategory.ordered + end + + # GET /admin/interest_categories/new + def new + @interest_category = InterestCategory.new(active: true, position: next_position) + end + + # POST /admin/interest_categories + def create + @interest_category = InterestCategory.new(interest_category_params) + + if @interest_category.save + redirect_to admin_interest_categories_path, notice: t(:created) + else + render :new, status: :unprocessable_entity + end + end + + # GET /admin/interest_categories/1/edit + def edit; end + + # PUT /admin/interest_categories/1 + def update + if @interest_category.update(interest_category_params) + redirect_to admin_interest_categories_path, notice: t(:updated) + else + render :edit, status: :unprocessable_entity + end + end + + # DELETE /admin/interest_categories/1 + def destroy + @interest_category.destroy + redirect_to admin_interest_categories_path, notice: t(:deleted) + end + + private + + def set_interest_category + @interest_category = InterestCategory.find(params[:id]) + end + + def interest_category_params + params.require(:interest_category) + .permit(:code, :name_en, :name_et, :position, :active) + end + + def next_position + (InterestCategory.maximum(:position) || 0) + 1 + end + + def authorize_user + authorize! :manage, InterestCategory + end + end +end diff --git a/app/controllers/auctions_controller.rb b/app/controllers/auctions_controller.rb index 519da0e98..54039d53b 100644 --- a/app/controllers/auctions_controller.rb +++ b/app/controllers/auctions_controller.rb @@ -14,6 +14,10 @@ def index limit: per_page_count, link_extra: 'data-turbo-action="advance"' ) + @show_recommendation_prompt = current_user&.recommendation_profile_promptable? + @auction_classifications = preloaded_classifications(@auctions) + + track_recommendation_impressions respond_to do |format| format.html @@ -36,6 +40,17 @@ def cors_preflight_check def fetch_auctions_list = Auction.active.search(params, current_user) + # Preload LLM tags for the list only when the experimental tags column is on, + # keyed by downcased domain name so the partial avoids an N+1. + def preloaded_classifications(auctions) + return nil unless AuctionCenter::Application.config.customization[:auction_tags_display_enabled] + + domain_names = auctions.map { |auction| auction.domain_name.to_s.downcase }.uniq + DomainClassification + .where(domain_name: domain_names) + .index_by { |classification| classification.domain_name.to_s.downcase } + end + def per_page_count count = params[:show_all] == 'true' ? @auctions_list.count : per_page count = nil if count.zero? @@ -60,4 +75,17 @@ def set_access_control_headers def authorize_user authorize! :read, Auction end + + def track_recommendation_impressions + return unless current_user + return unless request.format.html? + + Recommendation::EventTracker.track_impressions( + user: current_user, + auctions: @auctions, + source: 'auctions#index', + request: + ) + end + end diff --git a/app/controllers/english_offers_controller.rb b/app/controllers/english_offers_controller.rb index fb14eaf44..c8ff7af53 100644 --- a/app/controllers/english_offers_controller.rb +++ b/app/controllers/english_offers_controller.rb @@ -20,6 +20,7 @@ def new BillingProfile.create_default_for_user(current_user.id) @offer = Offer.new(auction_id: @auction.id, user_id: current_user.id) + track_detail_view end # POST /auctions/aa450f1a-45e2-4f22-b2c3-f5f46b5f906b/offers @@ -36,6 +37,16 @@ def create send_outbided_notification(auction: @auction, offer: @offer, flash:) update_auction_values(@auction, t('english_offers.create.created')) Rails.logger.info("User #{current_user.id} created offer #{@offer.id} for auction #{@auction.id}") + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(current_user.id) + Recommendation::ClassifyDomainJob.perform_later(@auction.domain_name) + Recommendation::EventTracker.call( + user: current_user, + auction: @auction, + event_type: 'bid_create', + source: 'english_offers#create', + properties: { offer_id: @offer.id, cents: @offer.cents }, + request: + ) else errors = if @offer.errors.full_messages_for(:cents).present? @offer.errors.full_messages_for(:cents).join @@ -70,6 +81,16 @@ def update send_outbided_notification(auction: @auction, offer: @offer, flash:) update_auction_values(@auction, t('english_offers.edit.bid_updated')) Rails.logger.info("User #{current_user.id} updated offer #{@offer.id} for auction #{@auction.id}") + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(current_user.id) + Recommendation::ClassifyDomainJob.perform_later(@auction.domain_name) + Recommendation::EventTracker.call( + user: current_user, + auction: @auction, + event_type: 'bid_update', + source: 'english_offers#update', + properties: { offer_id: @offer.id, cents: @offer.cents }, + request: + ) else errors = if @offer.errors.full_messages_for(:cents).present? @offer.errors.full_messages_for(:cents).join @@ -103,4 +124,16 @@ def update_params update_params = params.require(:offer).permit(:price, :billing_profile_id) merge_updated_by(update_params) end + + def track_detail_view + return unless current_user && @auction + + Recommendation::EventTracker.call( + user: current_user, + auction: @auction, + event_type: 'auction_detail_view', + source: 'english_offers#new', + request: + ) + end end diff --git a/app/controllers/offers_controller.rb b/app/controllers/offers_controller.rb index ab15b2589..8d66d5d6d 100644 --- a/app/controllers/offers_controller.rb +++ b/app/controllers/offers_controller.rb @@ -13,6 +13,7 @@ class OffersController < ApplicationController def new BillingProfile.create_default_for_user(current_user.id) @offer = Offer.new(auction_id: @auction.id, user_id: current_user.id) + track_detail_view end # POST /auctions/aa450f1a-45e2-4f22-b2c3-f5f46b5f906b/offers @@ -30,6 +31,16 @@ def create end elsif create_predicate Rails.logger.info("User #{current_user.id} created offer #{@offer.id} for auction #{@auction.id}") + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(current_user.id) + Recommendation::ClassifyDomainJob.perform_later(@auction.domain_name) + Recommendation::EventTracker.call( + user: current_user, + auction: @auction, + event_type: 'bid_create', + source: 'offers#create', + properties: { offer_id: @offer.id, cents: @offer.cents }, + request: + ) format.html { redirect_to root_path, notice: t('.created') } format.json { render :show, status: :created, location: @offer } else @@ -67,6 +78,16 @@ def update if update_predicate Rails.logger.info("User #{current_user.id} updated offer #{@offer.id} for auction #{@offer.auction.id}") + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(current_user.id) + Recommendation::ClassifyDomainJob.perform_later(@offer.auction.domain_name) + Recommendation::EventTracker.call( + user: current_user, + auction: @offer.auction, + event_type: 'bid_update', + source: 'offers#update', + properties: { offer_id: @offer.id, cents: @offer.cents }, + request: + ) format.html { redirect_to root_path, notice: t(:updated), status: :see_other } format.json { render :show, status: :ok, location: @offer } else @@ -119,4 +140,16 @@ def update_params update_params = params.require(:offer).permit(:price, :billing_profile_id) merge_updated_by(update_params) end + + def track_detail_view + return unless current_user && @auction + + Recommendation::EventTracker.call( + user: current_user, + auction: @auction, + event_type: 'auction_detail_view', + source: 'offers#new', + request: + ) + end end diff --git a/app/controllers/recommendation_events_controller.rb b/app/controllers/recommendation_events_controller.rb new file mode 100644 index 000000000..84dbe00ef --- /dev/null +++ b/app/controllers/recommendation_events_controller.rb @@ -0,0 +1,29 @@ +class RecommendationEventsController < ApplicationController + before_action :authenticate_user! + + def create + Recommendation::EventTracker.call( + user: current_user, + auction: auction, + event_type: recommendation_event_params[:event_type], + source: recommendation_event_params[:source], + properties: recommendation_event_params[:properties], + request: + ) + + head :created + end + + private + + def auction + return if recommendation_event_params[:auction_uuid].blank? + + Auction.find_by(uuid: recommendation_event_params[:auction_uuid]) + end + + def recommendation_event_params + params.require(:recommendation_event) + .permit(:event_type, :source, :auction_uuid, properties: {}) + end +end diff --git a/app/controllers/recommendation_profiles_controller.rb b/app/controllers/recommendation_profiles_controller.rb new file mode 100644 index 000000000..0a1038254 --- /dev/null +++ b/app/controllers/recommendation_profiles_controller.rb @@ -0,0 +1,67 @@ +class RecommendationProfilesController < ApplicationController + before_action :authenticate_user! + before_action :ensure_selection_enabled, only: %i[edit update] + before_action :set_recommendation_profile + + def edit; end + + def update + @recommendation_profile.assign_attributes(recommendation_profile_params) + + unless @recommendation_profile.filled? + @recommendation_profile.skip!(source: 'recommendation_profiles#update_blank', request:) + redirect_to after_update_path, notice: t('.skipped') + return + end + + if @recommendation_profile.save + @recommendation_profile.complete!(source: 'recommendation_profiles#update', request:) + redirect_to after_update_path, notice: t('.updated') + else + render :edit, status: :unprocessable_entity + end + end + + def dismiss + @recommendation_profile.dismiss!(source: 'recommendation_profiles#dismiss', request:) + redirect_to dismiss_redirect_path, notice: t('.dismissed') + end + + private + + def ensure_selection_enabled + return if RecommendationProfile.selection_enabled? + + redirect_to user_path(current_user.uuid) + end + + def set_recommendation_profile + @recommendation_profile = current_user.recommendation_profile || current_user.build_recommendation_profile + end + + def recommendation_profile_params + params.require(:recommendation_profile) + .permit( + :preferred_length_min, + :preferred_length_max, + :allow_numbers, + :allow_hyphens, + interest_categories: [], + custom_interests: [] + ) + end + + def after_update_path + requested_path = params[:return_to].to_s + return requested_path if requested_path.starts_with?('/') + + user_path(current_user.uuid) + end + + def dismiss_redirect_path + requested_path = params[:return_to].to_s + return requested_path if requested_path.starts_with?('/') + + root_path + end +end diff --git a/app/controllers/users_controller.rb b/app/controllers/users_controller.rb index 22971f333..635bc5261 100644 --- a/app/controllers/users_controller.rb +++ b/app/controllers/users_controller.rb @@ -16,6 +16,7 @@ def index; end def new redirect_to user_path(current_user.uuid), notice: t('.already_signed_in') if current_user @user = User.new + @user.build_recommendation_profile end # GET /profile/edit @@ -42,6 +43,15 @@ def create respond_to do |format| if @user.save + if @user.recommendation_profile&.filled? + @user.recommendation_profile.complete!(source: 'users#create', request:) + elsif @user.recommendation_profile.present? + # User signed up without filling the profile. Suppress the modal + # on the next /auctions render so they're not nagged immediately — + # they'll see it again after PROMPT_REMINDER_INTERVAL. + @user.recommendation_profile.dismiss_prompt! + end + flash[:notice] = t(:created) format.html do @@ -54,6 +64,7 @@ def create render :show, status: :created, location: @user end else + @user.build_recommendation_profile if @user.recommendation_profile.nil? flash.now[:alert] = @user.errors.full_messages.join(', ') format.html { render :new, status: :unprocessable_entity } @@ -116,9 +127,19 @@ def destroy def create_params params.require(:user) - .permit(:email, :password, :password_confirmation, :country_code, - :given_names, :surname, :mobile_phone, :accepts_terms_and_conditions, - :locale, :daily_summary, :identity_code) + .permit( + :email, :password, :password_confirmation, :country_code, + :given_names, :surname, :mobile_phone, :accepts_terms_and_conditions, + :locale, :daily_summary, :identity_code, + recommendation_profile_attributes: [ + :preferred_length_min, + :preferred_length_max, + :allow_numbers, + :allow_hyphens, + { interest_categories: [] }, + { custom_interests: [] } + ] + ) end def params_for_update diff --git a/app/controllers/wishlist_items_controller.rb b/app/controllers/wishlist_items_controller.rb index b4c68e055..42b6e6963 100644 --- a/app/controllers/wishlist_items_controller.rb +++ b/app/controllers/wishlist_items_controller.rb @@ -24,6 +24,17 @@ def create respond_to do |format| if create_predicate + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(current_user.id) + Recommendation::ClassifyDomainJob.perform_later(@wishlist_item.domain_name) + Recommendation::EventTracker.call( + user: current_user, + auction: Auction.find_by(domain_name: @wishlist_item.domain_name), + event_type: 'wishlist_add', + source: 'wishlist_items#create', + properties: { domain_name: @wishlist_item.domain_name }, + request: + ) + format.html { redirect_to wishlist_items_path, notice: t(:created) } format.json { render json: @wishlist_item, status: :created } else @@ -38,6 +49,16 @@ def destroy respond_to do |format| if @wishlist_item.destroy + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(current_user.id) + Recommendation::EventTracker.call( + user: current_user, + auction: Auction.find_by(domain_name: @wishlist_item.domain_name), + event_type: 'wishlist_remove', + source: 'wishlist_items#destroy', + properties: { domain_name: @wishlist_item.domain_name }, + request: + ) + format.turbo_stream do render turbo_stream: [ turbo_stream.replace('flash', partial: 'common/flash', locals: { flash: }), @@ -56,6 +77,7 @@ def destroy def update if @wishlist_item.update(strong_params) + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(current_user.id) flash[:notice] = t(:updated) render turbo_stream: [ turbo_stream.replace('flash', partial: 'common/flash', locals: { flash: }), diff --git a/app/helpers/application_helper.rb b/app/helpers/application_helper.rb index 74c2f20ad..2ed24182c 100644 --- a/app/helpers/application_helper.rb +++ b/app/helpers/application_helper.rb @@ -91,4 +91,20 @@ def cached_footer def ended_auctions_link_available? AuctionCenter::Application.config.customization[:ended_auctions_link_available] end + + # Experimental flag (config/customization.yml): show LLM tags instead of the + # auction-type column on the public auction list. + def auction_tags_display_enabled? + AuctionCenter::Application.config.customization[:auction_tags_display_enabled] + end + + # Tags to display for an auction, sourced from its LLM DomainClassification + # (joined by domain name, not FK). Pass `preloaded` — a hash keyed by + # downcased domain name — from the list to avoid an N+1; the single-row + # turbo-stream path falls back to one lookup. + def auction_display_tags(auction, preloaded = nil) + key = auction.domain_name.to_s.downcase + classification = preloaded ? preloaded[key] : DomainClassification.find_by(domain_name: key) + Array(classification&.tags).map(&:to_s).reject(&:blank?) + end end diff --git a/app/javascript/controllers/form/custom_interest_tags_controller.js b/app/javascript/controllers/form/custom_interest_tags_controller.js new file mode 100644 index 000000000..019c55853 --- /dev/null +++ b/app/javascript/controllers/form/custom_interest_tags_controller.js @@ -0,0 +1,62 @@ +import { Controller } from "@hotwired/stimulus" + +export default class extends Controller { + static targets = ["input", "list"] + + add(event) { + if (event.key !== "Enter") return + + event.preventDefault() + + const value = this.normalizedValue(this.inputTarget.value) + if (!value) return + if (this.existingValues().includes(value)) { + this.inputTarget.value = "" + return + } + + this.listTarget.insertAdjacentHTML("beforeend", this.tagHtml(value)) + this.inputTarget.value = "" + } + + remove(event) { + event.preventDefault() + const tag = event.currentTarget.closest("[data-custom-interest-value]") + if (!tag) return + + tag.remove() + } + + existingValues() { + return Array.from(this.listTarget.querySelectorAll("[data-custom-interest-value]")) + .map((node) => node.dataset.customInterestValue) + } + + normalizedValue(value) { + return value.toString().trim().toLowerCase() + } + + tagHtml(value) { + const escaped = this.escapeHtml(value) + return ` + + ${escaped} + + + + ` + } + + hiddenInputName() { + return this.inputTarget.dataset.hiddenInputName + } + + escapeHtml(value) { + return value + .replaceAll("&", "&") + .replaceAll("<", "<") + .replaceAll(">", ">") + .replaceAll('"', """) + .replaceAll("'", "'") + } +} diff --git a/app/javascript/controllers/index.js b/app/javascript/controllers/index.js index 493f0bd1c..fe3dafb8e 100644 --- a/app/javascript/controllers/index.js +++ b/app/javascript/controllers/index.js @@ -34,6 +34,9 @@ application.register("form--autosave", Form__AutosaveController); import Form__CheckAllController from "./form/check_all_controller"; application.register("form--check-all", Form__CheckAllController); +import Form__CustomInterestTagsController from "./form/custom_interest_tags_controller"; +application.register("form--custom-interest-tags", Form__CustomInterestTagsController); + import Table__OrdeableController from "./table/ordeable_controller"; application.register("table--ordeable", Table__OrdeableController); @@ -87,3 +90,9 @@ application.register("slashed-zero", SlashedZeroController); import OfferPriceValidatorController from "./offer_price_validator_controller"; application.register("offer-price-validator", OfferPriceValidatorController); + +import RecommendationTrackerController from "./recommendation_tracker_controller"; +application.register("recommendation-tracker", RecommendationTrackerController); + +import RecommendationDwellController from "./recommendation_dwell_controller"; +application.register("recommendation-dwell", RecommendationDwellController); diff --git a/app/javascript/controllers/recommendation_dwell_controller.js b/app/javascript/controllers/recommendation_dwell_controller.js new file mode 100644 index 000000000..df2e236de --- /dev/null +++ b/app/javascript/controllers/recommendation_dwell_controller.js @@ -0,0 +1,90 @@ +import { Controller } from "@hotwired/stimulus" +import { csrfToken } from "../helpers/csrf" + +// Fires a `recommendation_event` when the attached element has been +// visible in the viewport for at least `dwellMs` and the page is active. +// Idempotent per session via Set tracking. +// +// Usage on an auction card: +// data-controller="recommendation-dwell" +// data-recommendation-dwell-auction-uuid-value="..." +// data-recommendation-dwell-source-value="auctions#index" +// data-recommendation-dwell-event-type-value="auction_detail_view" +// data-recommendation-dwell-dwell-ms-value="1500" +export default class extends Controller { + static values = { + auctionUuid: String, + source: { type: String, default: "card" }, + eventType: { type: String, default: "auction_detail_view" }, + dwellMs: { type: Number, default: 1500 } + } + + connect() { + if (!this.hasAuctionUuidValue) return + if (typeof IntersectionObserver === "undefined") return + + this.fired = false + this.observer = new IntersectionObserver( + (entries) => this.handle(entries), + { threshold: 0.5 } + ) + this.observer.observe(this.element) + } + + disconnect() { + if (this.observer) this.observer.disconnect() + if (this.timerId) clearTimeout(this.timerId) + } + + handle(entries) { + for (const entry of entries) { + if (entry.isIntersecting) this.startTimer() + else this.cancelTimer() + } + } + + startTimer() { + if (this.fired || this.timerId) return + this.timerId = setTimeout(() => this.fire(), this.dwellMsValue) + } + + cancelTimer() { + if (this.timerId) { + clearTimeout(this.timerId) + this.timerId = null + } + } + + fire() { + this.timerId = null + if (this.fired || document.visibilityState !== "visible") return + this.fired = true + + const payload = { + recommendation_event: { + auction_uuid: this.auctionUuidValue, + source: this.sourceValue, + event_type: this.eventTypeValue + } + } + + const data = JSON.stringify(payload) + if (navigator.sendBeacon) { + const blob = new Blob([data], { type: "application/json" }) + navigator.sendBeacon("/recommendation_events", blob) + return + } + + fetch("/recommendation_events", { + method: "POST", + headers: { + "Content-Type": "application/json", + "Accept": "application/json", + "X-CSRF-Token": csrfToken() + }, + credentials: "same-origin", + keepalive: true, + body: data + }).catch(() => null) + } +} diff --git a/app/javascript/controllers/recommendation_tracker_controller.js b/app/javascript/controllers/recommendation_tracker_controller.js new file mode 100644 index 000000000..50fc73ca6 --- /dev/null +++ b/app/javascript/controllers/recommendation_tracker_controller.js @@ -0,0 +1,32 @@ +import { Controller } from "@hotwired/stimulus" +import { csrfToken } from "../helpers/csrf" + +export default class extends Controller { + static values = { + auctionUuid: String, + source: String, + eventType: String + } + + track() { + if (!this.hasAuctionUuidValue) return + + fetch("/recommendation_events", { + method: "POST", + headers: { + "Content-Type": "application/json", + "Accept": "application/json", + "X-CSRF-Token": csrfToken() + }, + credentials: "same-origin", + keepalive: true, + body: JSON.stringify({ + recommendation_event: { + auction_uuid: this.auctionUuidValue, + source: this.sourceValue || "ui", + event_type: this.eventTypeValue || "auction_click" + } + }) + }).catch(() => null) + } +} diff --git a/app/javascript/helpers/csrf.js b/app/javascript/helpers/csrf.js new file mode 100644 index 000000000..cc1cd1b3f --- /dev/null +++ b/app/javascript/helpers/csrf.js @@ -0,0 +1,5 @@ +// Reads the Rails CSRF token from the page tag. Shared by the +// recommendation event controllers so the lookup lives in one place. +export function csrfToken() { + return document.querySelector('meta[name="csrf-token"]')?.getAttribute("content") +} diff --git a/app/jobs/active_auctions_ai_sorting_job.rb b/app/jobs/active_auctions_ai_sorting_job.rb index 4bce7e072..4689357ce 100644 --- a/app/jobs/active_auctions_ai_sorting_job.rb +++ b/app/jobs/active_auctions_ai_sorting_job.rb @@ -50,15 +50,15 @@ def fetch_ai_response(auctions_list, temperature) end def chat_parameters(auctions_list, temp) + model_name = openai_model { - model: openai_model, + model: model_name, response_format: { type: 'json_schema', json_schema: schema }, - messages: messages(auctions_list), - temperature: temp - } + messages: messages(auctions_list) + }.merge(OpenaiStructuredOutputSupport.temperature_options(model_name, temp)) end # rubocop:disable Metrics/MethodLength @@ -144,6 +144,6 @@ def system_message end def openai_model - Setting.find_by(code: 'openai_model').retrieve + OpenaiStructuredOutputSupport.model(Setting.find_by(code: 'openai_model').retrieve) end end diff --git a/app/jobs/daily_broadcast_auctions_job.rb b/app/jobs/daily_broadcast_auctions_job.rb index 39bfe074b..b3c6b7784 100644 --- a/app/jobs/daily_broadcast_auctions_job.rb +++ b/app/jobs/daily_broadcast_auctions_job.rb @@ -5,14 +5,17 @@ def perform unsubscribe = Rails.application.message_verifier(:unsubscribe).generate(user.id) NotificationMailer.daily_auctions_broadcast_email( recipient: user.email, - auctions: active_auctions, + auctions: active_auctions_for(user), unsubscribe: unsubscribe ).deliver_later end end end - def active_auctions + def active_auctions_for(user) + scored_auctions = Recommendation::Scorer.top_auctions_for(user:, limit: 20).to_a + return scored_auctions if scored_auctions.any? + Auction.active.to_a end diff --git a/app/jobs/recommendation/backfill_domain_classifications_job.rb b/app/jobs/recommendation/backfill_domain_classifications_job.rb new file mode 100644 index 000000000..fde10092c --- /dev/null +++ b/app/jobs/recommendation/backfill_domain_classifications_job.rb @@ -0,0 +1,79 @@ +module Recommendation + # One-shot job that collects every domain name we know about from + # auctions, wishlist_items, domain_offer_histories, and result records, + # then classifies the ones without a row via the LLM (batched). + # + # The LLM maps each domain onto an InterestCatalog category code. There + # is no heuristic fallback — if OpenAI is disabled this job is a no-op. + # + # Invoked manually or via `rake recommendation:backfill`. One-time cost + # is ~$1 over a few thousand historical domains. + class BackfillDomainClassificationsJob < ApplicationJob + BATCH_SIZE = Recommendation::LlmDomainClassifier::BATCH_LIMIT + + def perform + return unless Feature.open_ai_integration_enabled? + + domains = collect_domains + Rails.logger.info("BackfillDomainClassificationsJob found #{domains.size} unique domains") + + already_known = DomainClassification.where(domain_name: domains).pluck(:domain_name).to_set + pending = domains.reject { |d| already_known.include?(d) } + + Rails.logger.info("BackfillDomainClassificationsJob classifying #{pending.size} new domains") + + classified = 0 + pending.each_slice(BATCH_SIZE) do |slice| + classified += upsert(Recommendation::LlmDomainClassifier.call(domain_names: slice)) + rescue StandardError => e + Rails.logger.warn("Backfill batch failed (#{slice.first}..): #{e.message}") + end + + Rails.logger.info("BackfillDomainClassificationsJob created #{classified} classifications") + classified + end + + private + + def upsert(attributes_list) + return 0 if attributes_list.blank? + + timestamps = { created_at: Time.current, updated_at: Time.current } + # Invalidate any stale embedding so EmbedUnembeddedDomainsJob recomputes it + # from the fresh keywords — mirrors ClassifyDomainJob / ClassifyUnclassifiedDomainsJob. + reset = DomainClassification.embedding_reset_attributes + rows = attributes_list.map { |attrs| attrs.merge(timestamps, reset) } + DomainClassification.upsert_all(rows, unique_by: :domain_name) + rows.size + end + + def collect_domains + sources = [ + Auction.distinct.pluck(:domain_name), + WishlistItem.distinct.pluck(:domain_name), + safe_pluck('DomainOfferHistory', :domain_name), + safe_pluck('Result', :domain_name) + ] + + sources.flatten + .map { |d| d.to_s.strip.downcase } + .reject(&:blank?) + .uniq + end + + # Resolve the model by name so an absent constant (e.g. DomainOfferHistory + # is not defined in this codebase) degrades to [] instead of raising + # NameError when the argument is evaluated. + def safe_pluck(model_name, column) + return [] unless Object.const_defined?(model_name) + + model = Object.const_get(model_name) + return [] unless model.respond_to?(:column_names) && model.column_names.include?(column.to_s) + + model.distinct.pluck(column) + rescue StandardError => e + Rails.logger.warn("Backfill source #{model_name} failed: #{e.message}") + [] + end + end +end diff --git a/app/jobs/recommendation/classify_domain_job.rb b/app/jobs/recommendation/classify_domain_job.rb new file mode 100644 index 000000000..329f94483 --- /dev/null +++ b/app/jobs/recommendation/classify_domain_job.rb @@ -0,0 +1,75 @@ +module Recommendation + # Classifies a single domain via the LLM when a new auction is created + # (enqueued from Auction after_create). Runs in the background, bounded to + # at most one LLM call per not-yet-classified domain, so cost stays + # predictable. + # + # The LLM maps any domain — Estonian, English, Finnish, transliterated + # Russian, invented brand — onto the fixed InterestCatalog category codes. + # No heuristic dictionary: the model figures out the category. + # + # The nightly ClassifyUnclassifiedDomainsJob is the safety net: it picks + # up anything this job missed (e.g. OpenAI was disabled when the domain + # first appeared, or the job failed) and re-classifies stale rows. + class ClassifyDomainJob < ApplicationJob + retry_on StandardError, wait: 10.seconds, attempts: 3 + + def perform(domain_name) + name = domain_name.to_s.strip.downcase + return if name.blank? + return unless Feature.open_ai_integration_enabled? + return if already_classified?(name) + + # Serialize concurrent jobs for the SAME domain. Two auctions for the same + # domain (or a retry racing the original) can enqueue almost + # simultaneously, and without this both pass already_classified? and each + # makes a paid LLM call. A worker that can't grab the lock skips the domain + # — it's being handled, and the nightly ClassifyUnclassifiedDomainsJob is + # the backstop. + with_domain_lock(name) do |acquired| + next unless acquired + next if already_classified?(name) # re-check now that we hold the lock + + attributes_list = Recommendation::LlmDomainClassifier.call(domain_names: [name]) + upsert(attributes_list) + end + end + + private + + # Skip if we already have a non-stale LLM classification. A bare row + # with no LLM data (or a stale one) is re-classified. + def already_classified?(name) + existing = DomainClassification.find_by(domain_name: name) + existing&.from_llm? && !existing.stale? + end + + # Postgres session-level advisory lock, keyed by the domain. pg_try_* is + # non-blocking: it yields false immediately if another worker holds it, so + # we never tie up a worker waiting. Always unlocked in the ensure. + def with_domain_lock(name) + key = advisory_key(name) + conn = DomainClassification.connection + acquired = conn.select_value("SELECT pg_try_advisory_lock(#{key})") + yield acquired + ensure + conn.execute("SELECT pg_advisory_unlock(#{key})") if acquired + end + + # Stable 60-bit positive integer (fits a signed bigint) derived from the + # domain, namespaced so it can't collide with advisory locks elsewhere. + def advisory_key(name) + Digest::SHA256.hexdigest("classify_domain:#{name}")[0, 15].to_i(16) + end + + def upsert(attributes_list) + return 0 if attributes_list.blank? + + timestamps = { created_at: Time.current, updated_at: Time.current } + reset = DomainClassification.embedding_reset_attributes + rows = attributes_list.map { |attrs| attrs.merge(timestamps, reset) } + DomainClassification.upsert_all(rows, unique_by: :domain_name) + rows.size + end + end +end diff --git a/app/jobs/recommendation/classify_unclassified_domains_job.rb b/app/jobs/recommendation/classify_unclassified_domains_job.rb new file mode 100644 index 000000000..08ac6c30b --- /dev/null +++ b/app/jobs/recommendation/classify_unclassified_domains_job.rb @@ -0,0 +1,75 @@ +module Recommendation + # Batched LLM classifier pass. Drained by PipelineRunner (rake + # recommendation:init) and exposed on /admin/jobs; not a standalone cron task. + # + # Safety-net + maintenance pass for the LLM classifier: + # - discovers active-auction / wishlist domains that have NO classification + # row yet (e.g. ClassifyDomainJob never ran because OpenAI was disabled + # when the domain appeared, or the job failed), and + # - re-classifies existing rows that are low-confidence or stale (>6mo). + # + # Batches everything through the LLM. NEVER called from request paths. + class ClassifyUnclassifiedDomainsJob < ApplicationJob + MAX_DOMAINS_PER_RUN = 200 + BATCH_SIZE = Recommendation::LlmDomainClassifier::BATCH_LIMIT + + retry_on StandardError, wait: 30.seconds, attempts: 2 + + def perform + return unless Feature.open_ai_integration_enabled? + + domains = pending_domains + return if domains.empty? + + processed = 0 + domains.each_slice(BATCH_SIZE) do |batch| + attributes_list = Recommendation::LlmDomainClassifier.call(domain_names: batch) + processed += upsert(attributes_list) + end + + Rails.logger.info("ClassifyUnclassifiedDomainsJob processed #{processed} domains") + processed + end + + # Existing rows that need re-classification (low confidence / stale LLM). + def self.scope + DomainClassification.needs_llm_enrichment.order(Arel.sql('COALESCE(classified_at, to_timestamp(0)) ASC')) + end + + # Active-auction / wishlist domains with no classification row at all. + def self.missing_domains + known = DomainClassification.pluck(:domain_name).to_set + candidates = (Auction.active.distinct.pluck(:domain_name) + + WishlistItem.distinct.pluck(:domain_name)) + .map { |d| d.to_s.strip.downcase } + .reject(&:blank?) + .uniq + candidates.reject { |d| known.include?(d) } + end + + def self.needs_to_run? + return false unless Feature.open_ai_integration_enabled? + + scope.exists? || missing_domains.any? + end + + private + + def pending_domains + (self.class.missing_domains + self.class.scope.pluck(:domain_name)) + .uniq + .first(MAX_DOMAINS_PER_RUN) + end + + def upsert(attributes_list) + return 0 if attributes_list.blank? + + timestamps = { created_at: Time.current, updated_at: Time.current } + reset = DomainClassification.embedding_reset_attributes + rows = attributes_list.map { |attrs| attrs.merge(timestamps, reset) } + + DomainClassification.upsert_all(rows, unique_by: :domain_name) + rows.size + end + end +end diff --git a/app/jobs/recommendation/embed_custom_interests_job.rb b/app/jobs/recommendation/embed_custom_interests_job.rb new file mode 100644 index 000000000..e39c35706 --- /dev/null +++ b/app/jobs/recommendation/embed_custom_interests_job.rb @@ -0,0 +1,49 @@ +module Recommendation + # v3: embeds a profile's free-text custom interests into their own vectors so + # each acts as a separate magnet in the unified scorer (never averaged, never + # LIKE-matched). Enqueued from RecommendationProfile when the custom interests + # change. Reuses already-stored vectors for unchanged texts so editing one + # interest never re-embeds the rest. + # + # On completion it enqueues the debounced rescore so the fresh magnets take + # effect on the next scoring pass. + class EmbedCustomInterestsJob < ApplicationJob + retry_on StandardError, wait: 30.seconds, attempts: 2 + + def perform(profile_id) + return unless Feature.open_ai_integration_enabled? + return unless RecommendationProfile.column_names.include?('custom_interest_vectors') + + profile = RecommendationProfile.find_by(id: profile_id) + return unless profile + + texts = profile.custom_interests + if texts.empty? + profile.update_columns(custom_interest_vectors: [], custom_interests_embedded_at: Time.current) + return 0 + end + + vectors = build_vectors(profile, texts) + profile.update_columns( + custom_interest_vectors: vectors, + custom_interests_embedded_at: Time.current + ) + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(profile.user_id) + vectors.size + end + + private + + # Keep existing vectors for unchanged texts; embed only the newcomers. + def build_vectors(profile, texts) + cached = Array(profile.custom_interest_vectors).index_by { |entry| entry['text'] } + missing = texts.reject { |text| cached[text] } + + fresh = missing.zip(Recommendation::TextEmbedder.embed(missing)).to_h + + texts.map do |text| + cached[text] || { 'text' => text, 'embedding' => fresh[text] } + end + end + end +end diff --git a/app/jobs/recommendation/embed_unembedded_domains_job.rb b/app/jobs/recommendation/embed_unembedded_domains_job.rb new file mode 100644 index 000000000..510331c35 --- /dev/null +++ b/app/jobs/recommendation/embed_unembedded_domains_job.rb @@ -0,0 +1,60 @@ +module Recommendation + # Batched embedder pass. Drained by PipelineRunner (rake recommendation:init) + # and exposed on /admin/jobs; not a standalone cron task. + # Picks classified rows that don't yet have an embedding, batches + # them through DomainEmbedder, and persists the vectors as plain + # Postgres double precision[] arrays. + class EmbedUnembeddedDomainsJob < ApplicationJob + MAX_DOMAINS_PER_RUN = 500 + BATCH_SIZE = Recommendation::DomainEmbedder::BATCH_LIMIT + + retry_on StandardError, wait: 30.seconds, attempts: 2 + + def perform + return unless Feature.open_ai_integration_enabled? + return unless DomainClassification.column_names.include?('embedding') + + rows = self.class.scope.limit(MAX_DOMAINS_PER_RUN).to_a + return if rows.empty? + + processed = 0 + rows.each_slice(BATCH_SIZE) do |batch| + results = Recommendation::DomainEmbedder.call(rows: batch) + processed += persist(results) + end + + Rails.logger.info("EmbedUnembeddedDomainsJob embedded #{processed} domains") + processed + end + + def self.scope + DomainClassification.needs_embedding.order(:classified_at) + end + + def self.needs_to_run? + return false unless DomainClassification.column_names.include?('embedding') + + Feature.open_ai_integration_enabled? && scope.exists? + end + + private + + def persist(results) + return 0 if results.empty? + + by_name = results.index_by { |r| r[:domain_name] } + DomainClassification.where(domain_name: by_name.keys).find_each do |record| + payload = by_name[record.domain_name] + next unless payload + + record.update_columns( + embedding: payload[:embedding], + embedding_model: payload[:embedding_model], + embedded_at: payload[:embedded_at], + embedding_input_version: payload[:embedding_input_version] + ) + end + results.size + end + end +end diff --git a/app/jobs/recommendation/enrich_interest_categories_job.rb b/app/jobs/recommendation/enrich_interest_categories_job.rb new file mode 100644 index 000000000..71ac8dd87 --- /dev/null +++ b/app/jobs/recommendation/enrich_interest_categories_job.rb @@ -0,0 +1,67 @@ +module Recommendation + # Enriches interest categories (LLM description + keywords) and embeds them + # into the same 1536-dim space as domains, so a selected category can act as + # a magnet in the v3 unified scorer. + # + # Two entry points: + # - perform_later(category_id) — reactive, from InterestCategory after_save + # when an admin adds/edits a category. + # - perform_now — batch, from PipelineRunner: enriches every + # category still missing an embedding. + class EnrichInterestCategoriesJob < ApplicationJob + BATCH_SIZE = Recommendation::InterestCategoryEnricher::BATCH_LIMIT + + retry_on StandardError, wait: 30.seconds, attempts: 2 + + def perform(category_id = nil) + return unless Feature.open_ai_integration_enabled? + return unless InterestCategory.column_names.include?('embedding') + + categories = category_id ? Array(InterestCategory.find_by(id: category_id)) : self.class.scope.to_a + categories.reject! { |c| c.code.to_s.blank? } + return if categories.empty? + + processed = 0 + categories.each_slice(BATCH_SIZE) do |batch| + results = Recommendation::InterestCategoryEnricher.call(categories: batch) + processed += persist(results) + end + + Rails.logger.info("EnrichInterestCategoriesJob enriched #{processed} categories") + processed + end + + # Active categories that have never been embedded (fresh) or whose keywords + # are still empty — i.e. everything the enrichment hasn't produced yet. + def self.scope + InterestCategory.active.where(embedded_at: nil) + end + + def self.needs_to_run? + return false unless InterestCategory.column_names.include?('embedding') + + Feature.open_ai_integration_enabled? && scope.exists? + end + + private + + def persist(results) + return 0 if results.empty? + + by_code = results.index_by { |r| r[:code] } + InterestCategory.where(code: by_code.keys).find_each do |record| + payload = by_code[record.code] + next unless payload + + record.update_columns( + description: payload[:description], + keywords: payload[:keywords], + embedding: payload[:embedding], + embedding_model: payload[:embedding_model], + embedded_at: payload[:embedded_at] + ) + end + results.size + end + end +end diff --git a/app/jobs/recommendation/prune_recommendation_events_job.rb b/app/jobs/recommendation/prune_recommendation_events_job.rb new file mode 100644 index 000000000..d3a279c05 --- /dev/null +++ b/app/jobs/recommendation/prune_recommendation_events_job.rb @@ -0,0 +1,53 @@ +module Recommendation + # Retention/cleanup for recommendation_events. The table is append-only and + # otherwise grows forever. Scoring reads only 'auction_detail_view' events, and + # those decay (60-day half-life) to a negligible weight well before the + # retention window, so pruning old rows does not measurably change rankings. + # + # Two policies: + # - everything older than DEFAULT_RETENTION is dropped; + # - 'auction_impression' (never read by the scorer, and by far the highest + # volume) is dropped much sooner. + # + # Deletes in batches so the first run on a large table never holds a long lock. + # Exposed on /admin/jobs and as `rake recommendation:prune_events` (cron). + class PruneRecommendationEventsJob < ApplicationJob + DEFAULT_RETENTION = 6.months + IMPRESSION_RETENTION = 1.month + BATCH_SIZE = 10_000 + + def perform(batch_size: BATCH_SIZE) + return 0 unless RecommendationEvent.table_exists? + + deleted = 0 + deleted += prune(RecommendationEvent.where(occurred_at: ...DEFAULT_RETENTION.ago), batch_size) + deleted += prune( + RecommendationEvent.where(event_type: 'auction_impression') + .where(occurred_at: ...IMPRESSION_RETENTION.ago), + batch_size + ) + Rails.logger.info("PruneRecommendationEventsJob deleted #{deleted} event(s)") + deleted + end + + def self.needs_to_run? + return false unless RecommendationEvent.table_exists? + + RecommendationEvent.where(occurred_at: ...DEFAULT_RETENTION.ago).exists? || + RecommendationEvent.where(event_type: 'auction_impression') + .where(occurred_at: ...IMPRESSION_RETENTION.ago).exists? + end + + private + + def prune(scope, batch_size) + total = 0 + loop do + count = scope.limit(batch_size).delete_all + total += count + break if count < batch_size + end + total + end + end +end diff --git a/app/jobs/recommendation/rebuild_recommendations_job.rb b/app/jobs/recommendation/rebuild_recommendations_job.rb new file mode 100644 index 000000000..8f990fdbb --- /dev/null +++ b/app/jobs/recommendation/rebuild_recommendations_job.rb @@ -0,0 +1,18 @@ +module Recommendation + # Admin-triggered full rebuild of the recommendation pipeline, exposed in + # /admin/jobs. Run it after changing the interest catalog (adding, renaming, + # or deleting categories) so every domain is re-classified under the new + # vocabulary, re-embedded, re-scored globally, and re-scored per user. + # + # Force mode re-hits the LLM for every domain, so this is intentionally a + # manual button, not a callback on InterestCategory. + class RebuildRecommendationsJob < ApplicationJob + def perform + Recommendation::PipelineRunner.run(force: true) + end + + def self.needs_to_run? + Feature.open_ai_integration_enabled? + end + end +end diff --git a/app/jobs/recommendation/refresh_single_user_auction_scores_job.rb b/app/jobs/recommendation/refresh_single_user_auction_scores_job.rb new file mode 100644 index 000000000..35fed2b1c --- /dev/null +++ b/app/jobs/recommendation/refresh_single_user_auction_scores_job.rb @@ -0,0 +1,32 @@ +module Recommendation + class RefreshSingleUserAuctionScoresJob < ApplicationJob + DEBOUNCE_WINDOW = 30.seconds + + retry_on StandardError, wait: 5.seconds, attempts: 3 + + def self.enqueue_debounced(user_id) + set(wait: DEBOUNCE_WINDOW).perform_later(user_id) + end + + def perform(user_id) + user = User.find_by(id: user_id) + return unless user + + # A refresh just ran (<30s ago). Don't drop this request — re-enqueue it + # for the next window so the change that triggered us isn't silently lost. + if recently_refreshed?(user) + self.class.set(wait: DEBOUNCE_WINDOW).perform_later(user_id) + return + end + + Recommendation::Scorer.refresh_for(user:) + end + + private + + def recently_refreshed?(user) + last_calculated = UserAuctionScore.where(user_id: user.id).maximum(:calculated_at) + last_calculated.present? && last_calculated > DEBOUNCE_WINDOW.ago + end + end +end diff --git a/app/models/ability.rb b/app/models/ability.rb index f1a44f630..8f45a8275 100644 --- a/app/models/ability.rb +++ b/app/models/ability.rb @@ -83,6 +83,7 @@ def administrator can %i[read create], Job can :manage, User can %i[read update], Setting + can :manage, InterestCategory can :read, Offer can :read, Result can :read, PaymentOrder diff --git a/app/models/auction.rb b/app/models/auction.rb index a920aca12..17db2af69 100644 --- a/app/models/auction.rb +++ b/app/models/auction.rb @@ -8,6 +8,7 @@ class Auction < ApplicationRecord # rubocop:disable Metrics ENGLISH = '1'.freeze after_create :find_auction_turns + after_create :enqueue_domain_classification validates :domain_name, presence: true attr_accessor :skip_broadcast, :skip_validation @@ -21,6 +22,8 @@ class Auction < ApplicationRecord # rubocop:disable Metrics has_many :offers, dependent: :delete_all has_many :domain_participate_auctions, dependent: :delete_all has_many :domain_offer_histories + has_many :recommendation_events, dependent: :nullify + has_many :user_auction_scores, dependent: :delete_all has_one :result, required: false, dependent: :destroy enum :platform, %i[blind english] @@ -205,6 +208,12 @@ def find_auction_turns update(turns_count: calculate_turns_count) end + def enqueue_domain_classification + return if domain_name.blank? + + Recommendation::ClassifyDomainJob.perform_later(domain_name) + end + def calculate_turns_count auctions = Auction.unscoped.where(domain_name:).where('starts_at <= ?', starts_at) result_statuses = auctions.order(:ends_at).map { |auction| auction.result&.status } diff --git a/app/models/concerns/auction/searchable.rb b/app/models/concerns/auction/searchable.rb index e9214678f..8af2501c8 100644 --- a/app/models/concerns/auction/searchable.rb +++ b/app/models/concerns/auction/searchable.rb @@ -92,7 +92,9 @@ def search(params = {}, current_user = nil) sort_admin_column = params[:sort_by].presence_in(FILTERING_COLUMNS) || 'id' sort_direction = params[:sort_direction].presence_in(DIRECTION) || 'desc' is_from_admin = params[:admin] == 'true' - should_apply_user_sorting = params[:sort_by].blank? && params[:sort_direction].blank? && !is_from_admin && current_user + no_explicit_sort = params[:sort_by].blank? && params[:sort_direction].blank? && !is_from_admin + should_apply_user_sorting = no_explicit_sort && current_user + should_apply_anonymous_default = no_explicit_sort && current_user.nil? query = with_highest_offers @@ -107,6 +109,8 @@ def search(params = {}, current_user = nil) if should_apply_user_sorting query.sorted_for_user(current_user) + elsif should_apply_anonymous_default + query.order(Arel.sql('auctions.ends_at ASC NULLS LAST')) elsif params[:sort_by] == 'users_price' query.with_max_offer_cents_for_english_auction(current_user) .order("offers_subquery.max_offer_cents #{sort_direction} NULLS LAST") diff --git a/app/models/concerns/auction/user_sortable.rb b/app/models/concerns/auction/user_sortable.rb index ec0cf94fc..1e5632c95 100644 --- a/app/models/concerns/auction/user_sortable.rb +++ b/app/models/concerns/auction/user_sortable.rb @@ -1,6 +1,16 @@ module Auction::UserSortable extend ActiveSupport::Concern + # v3: ordering collapses to three buckets. Personalisation now lives entirely + # in user_auction_scores.score (magnet pull + structural nudges — see + # Recommendation::Scorer), so the old string-matched "interest" tier and its + # domain_classifications join are gone. An auction with no score row falls to + # the tail, ranked by global ai_score then RANDOM. + # + # bucket 0: the user's own offer + # bucket 1: a wishlisted domain (only when a wishlist exists) + # bucket 2: has a personal score + # bucket 3: everything else (ai_score / RANDOM tail) class_methods do def sorted_for_user(user) = user ? with_user_priority_sorting(user) : self @@ -8,36 +18,62 @@ def sorted_for_user(user) = user ? with_user_priority_sorting(user) : self def with_user_priority_sorting(user) wishlist_domains = user.wishlist_items.pluck(:domain_name) - + query = with_recommendation_scores(user.id) + order_sql = if wishlist_domains.any? - build_three_tier_priority_sql(wishlist_domains) + build_wishlist_priority_sql(wishlist_domains) else - build_two_tier_priority_sql + build_priority_sql end - - order(Arel.sql(order_sql)) + + query.order(Arel.sql(order_sql)) + end + + def with_recommendation_scores(user_id) + scores_join = ActiveRecord::Base.sanitize_sql_array([ + <<~SQL.squish, + LEFT JOIN user_auction_scores + ON user_auction_scores.auction_id = auctions.id + AND user_auction_scores.user_id = ? + SQL + user_id + ]) + + joins(scores_join) end - def build_three_tier_priority_sql(wishlist_domains) + def build_wishlist_priority_sql(wishlist_domains) sanitized_domains = wishlist_domains.map { |d| ActiveRecord::Base.connection.quote(d) }.join(',') <<~SQL.squish - CASE + CASE WHEN auctions.users_offer_id IS NOT NULL THEN 0 WHEN auctions.domain_name IN (#{sanitized_domains}) THEN 1 - ELSE 2 + WHEN user_auction_scores.score IS NOT NULL THEN 2 + ELSE 3 END, - CASE WHEN auctions.ai_score > 0 THEN auctions.ai_score ELSE RANDOM() END DESC + #{secondary_key_sql} SQL end - def build_two_tier_priority_sql + def build_priority_sql <<~SQL.squish - CASE + CASE WHEN auctions.users_offer_id IS NOT NULL THEN 0 - ELSE 1 + WHEN user_auction_scores.score IS NOT NULL THEN 1 + ELSE 2 END, - CASE WHEN auctions.ai_score > 0 THEN auctions.ai_score ELSE RANDOM() END DESC + #{secondary_key_sql} + SQL + end + + def secondary_key_sql + <<~SQL.squish + CASE + WHEN user_auction_scores.score IS NOT NULL THEN user_auction_scores.score + WHEN auctions.ai_score > 0 THEN auctions.ai_score + ELSE RANDOM() + END DESC SQL end end -end \ No newline at end of file +end diff --git a/app/models/domain_classification.rb b/app/models/domain_classification.rb new file mode 100644 index 000000000..da9449970 --- /dev/null +++ b/app/models/domain_classification.rb @@ -0,0 +1,71 @@ +class DomainClassification < ApplicationRecord + HEURISTIC_SOURCE = 'heuristic'.freeze + OPENAI_SOURCE = 'openai'.freeze + MANUAL_SOURCE = 'manual'.freeze + IMPORTED_SOURCE = 'imported'.freeze + + SOURCES = [HEURISTIC_SOURCE, OPENAI_SOURCE, MANUAL_SOURCE, IMPORTED_SOURCE].freeze + + LOW_CONFIDENCE_THRESHOLD = 0.6 + LLM_REFRESH_INTERVAL = 6.months + + validates :domain_name, presence: true, uniqueness: { case_sensitive: false } + validates :classification_source, inclusion: { in: SOURCES }, allow_nil: true + validates :confidence, + numericality: { greater_than_or_equal_to: 0, less_than_or_equal_to: 1 }, + allow_nil: true + validates :brandability_score, + numericality: { greater_than_or_equal_to: 0, less_than_or_equal_to: 1 }, + allow_nil: true + + before_validation :normalize_domain_name + + scope :needs_llm_enrichment, lambda { + where(classification_source: [HEURISTIC_SOURCE, nil]) + .or(where(confidence: ...LOW_CONFIDENCE_THRESHOLD)) + .or(where('classification_source = ? AND classified_at < ?', OPENAI_SOURCE, LLM_REFRESH_INTERVAL.ago)) + } + scope :classified, -> { where.not(classified_at: nil) } + # A classified row needs (re-)embedding when it has no vector yet OR its vector + # was built under an older DomainEmbedder input format. The explicit NULL branch + # is required: `where.not(embedding_input_version: current)` excludes NULLs in + # SQL (NULL != x is unknown), and every pre-versioning row has NULL here. + scope :needs_embedding, lambda { + next none unless column_names.include?('embedding') + + base = classified + next base.where(embedding: nil) unless column_names.include?('embedding_input_version') + + current = Recommendation::DomainEmbedder::INPUT_VERSION + base.where(embedding: nil) + .or(base.where(embedding_input_version: nil)) + .or(base.where.not(embedding_input_version: current)) + } + + # Columns to null out whenever a row is (re)classified. The embedding is built + # from the classification fields (see Recommendation::DomainEmbedder), so a + # fresh classification makes the stored vector stale; nulling these lets + # EmbedUnembeddedDomainsJob recompute it on its next run. + def self.embedding_reset_attributes + return {} unless column_names.include?('embedding') + + attrs = { embedding: nil, embedding_model: nil, embedded_at: nil } + attrs[:embedding_input_version] = nil if column_names.include?('embedding_input_version') + attrs + end + + def from_llm? = classification_source == OPENAI_SOURCE + + def stale? + return true if classification_source != OPENAI_SOURCE + return true if classified_at.nil? + + classified_at < LLM_REFRESH_INTERVAL.ago + end + + private + + def normalize_domain_name + self.domain_name = domain_name.to_s.strip.downcase if domain_name.present? + end +end diff --git a/app/models/interest_category.rb b/app/models/interest_category.rb new file mode 100644 index 000000000..25a17ac0d --- /dev/null +++ b/app/models/interest_category.rb @@ -0,0 +1,98 @@ +class InterestCategory < ApplicationRecord + # Initial vocabulary, also mirrored in Recommendation::InterestCatalog::FALLBACK_CATEGORIES. + # Seeded via .seed_defaults! from both db/seeds.rb (fresh setup) and a data + # migration (existing environments). Idempotent — never clobbers admin edits. + DEFAULTS = [ + { code: 'brandable', name_en: 'Brandable names', name_et: 'Bränditavad nimed' }, + { code: 'shop_brand', name_en: 'Shop or store names', name_et: 'Poe- või kaubamärgi nimed' }, + { code: 'saas', name_en: 'SaaS and software', name_et: 'SaaS ja tarkvara' }, + { code: 'b2b_service', name_en: 'B2B services', name_et: 'B2B teenused' }, + { code: 'local_service', name_en: 'Local or service businesses', name_et: 'Kohalikud ja teenusettevõtted' }, + { code: 'media_content', name_en: 'Media and content', name_et: 'Meedia ja sisu' }, + { code: 'finance', name_en: 'Finance and fintech', name_et: 'Finants ja fintech' }, + { code: 'legal', name_en: 'Legal and professional services', name_et: 'Õigus- ja professionaalsed teenused' }, + { code: 'health', name_en: 'Health and wellness', name_et: 'Tervis ja heaolu' }, + { code: 'education', name_en: 'Education and courses', name_et: 'Haridus ja kursused' }, + { code: 'travel', name_en: 'Travel and tourism', name_et: 'Reisimine ja turism' }, + { code: 'automotive', name_en: 'Automotive', name_et: 'Autondus' }, + { code: 'real_estate', name_en: 'Real estate', name_et: 'Kinnisvara' }, + { code: 'numeric', name_en: 'Numeric domains', name_et: 'Numbrilised domeenid' }, + { code: 'other', name_en: 'Other', name_et: 'Muu' } + ].freeze + + def self.seed_defaults! + DEFAULTS.each_with_index do |attrs, index| + find_or_create_by(code: attrs[:code]) do |category| + category.name_en = attrs[:name_en] + category.name_et = attrs[:name_et] + category.position = index + 1 + category.active = true + end + end + end + + validates :code, presence: true, uniqueness: { case_sensitive: false } + validates :name_en, presence: true + validates :name_et, presence: true + validates :position, numericality: { only_integer: true } + + before_validation :normalize_code + after_save_commit :enqueue_enrichment, if: :should_enrich? + after_destroy_commit :purge_code_references + + scope :active, -> { where(active: true) } + scope :ordered, -> { order(:position, :code) } + + # v3: a category carries its own embedding (built from name + LLM keywords), + # so a selected category acts as a magnet in the unified scorer. + def embedded? = respond_to?(:embedding) && embedding.present? + + # Locale-aware display name. Falls back to the English name. + def name + I18n.locale.to_s == 'et' ? name_et.presence || name_en : name_en + end + + private + + def normalize_code + self.code = code.to_s.strip.downcase.presence + end + + # Re-enrich only when the semantic content changed (name) or a vector is still + # missing — never on unrelated saves (position, active toggle). Enrichment + # itself uses update_columns (skips callbacks), so this never loops. + def should_enrich? + return false unless self.class.column_names.include?('embedding') + return false unless Feature.open_ai_integration_enabled? + return false unless active? + + saved_change_to_name_en? || saved_change_to_name_et? || embedding.blank? + end + + def enqueue_enrichment + Recommendation::EnrichInterestCategoriesJob.perform_later(id) + end + + # There are no FKs — user profiles and domain classifications reference a + # category only by its string code. When a category is deleted, strip that + # orphaned code so it stops lingering as garbage (in profiles it would + # otherwise silently degrade into a custom substring interest; on domains it + # would remain an unmatchable tag). Cheap set-based updates, no LLM calls. + def purge_code_references + return if code.blank? + + RecommendationProfile + .where('? = ANY (interest_keywords)', code) + .update_all(['interest_keywords = array_remove(interest_keywords, ?)', code]) + + if DomainClassification.column_names.include?('tags') + DomainClassification + .where('? = ANY (tags)', code) + .update_all(['tags = array_remove(tags, ?)', code]) + end + + DomainClassification + .where(primary_category: code) + .update_all(primary_category: nil) + end +end diff --git a/app/models/job.rb b/app/models/job.rb index e34d7818e..700747d1c 100644 --- a/app/models/job.rb +++ b/app/models/job.rb @@ -3,7 +3,11 @@ class Job AuctionCreationJob DomainRegistrationCheckJob ResultStatusUpdateJob DomainRegistrationReminderJob UnpaidInvoiceReminderJob DailySummaryJob DailyBroadcastAuctionsJob DailyViewRefreshJob - SharedFooterFetcherJob DirectoInvoiceForwardJob ActiveAuctionsAiSortingJob].freeze + SharedFooterFetcherJob DirectoInvoiceForwardJob ActiveAuctionsAiSortingJob + Recommendation::ClassifyUnclassifiedDomainsJob + Recommendation::EmbedUnembeddedDomainsJob + Recommendation::PruneRecommendationEventsJob + Recommendation::RebuildRecommendationsJob].freeze include ActiveModel::Model diff --git a/app/models/recommendation_event.rb b/app/models/recommendation_event.rb new file mode 100644 index 000000000..4bb9763e2 --- /dev/null +++ b/app/models/recommendation_event.rb @@ -0,0 +1,19 @@ +class RecommendationEvent < ApplicationRecord + EVENT_TYPES = %w[ + auction_impression + auction_click + auction_detail_view + wishlist_add + wishlist_remove + bid_create + bid_update + recommendation_profile_completed + recommendation_prompt_dismissed + ].freeze + + belongs_to :user, optional: true + belongs_to :auction, optional: true + + validates :event_type, presence: true, inclusion: { in: EVENT_TYPES } + validates :occurred_at, presence: true +end diff --git a/app/models/recommendation_profile.rb b/app/models/recommendation_profile.rb new file mode 100644 index 000000000..58c08ec5e --- /dev/null +++ b/app/models/recommendation_profile.rb @@ -0,0 +1,224 @@ +class RecommendationProfile < ApplicationRecord + PROMPT_REMINDER_INTERVAL = 14.days + CUSTOM_INTEREST_PREFIX = 'custom:'.freeze + OTHER_CATEGORY = 'other'.freeze + + belongs_to :user + + before_validation :normalize_interest_categories + after_save_commit :enqueue_custom_interest_embedding, if: :custom_interests_changed? + + validates :preferred_length_min, + numericality: { only_integer: true, greater_than: 0, less_than_or_equal_to: 63 }, + allow_nil: true + validates :preferred_length_max, + numericality: { only_integer: true, greater_than: 0, less_than_or_equal_to: 63 }, + allow_nil: true + validate :length_range_is_valid + validate :interest_categories_are_supported + + SELECTION_ENABLED_SETTING = 'recommendation_interests_enabled'.freeze + + # Admin toggle (Setting). When false, the interest-selection UI is hidden + # from users (no prompt, no form). Already-saved interests still affect + # ranking — this only gates the UI. Defaults to false when the setting is + # missing, so a fresh / un-seeded deploy never shows an empty picker. + def self.selection_enabled? + Setting.find_by(code: SELECTION_ENABLED_SETTING)&.retrieve == true + rescue StandardError + false + end + + def completed? = completed_at.present? + + def promptable? + return false if completed? + return true if prompt_dismissed_at.blank? + + prompt_dismissed_at <= PROMPT_REMINDER_INTERVAL.ago + end + + def filled? + rankable_interest_categories.any? || + custom_interests.any? || + preferred_length_min.present? || + preferred_length_max.present? || + !allow_numbers.nil? || + !allow_hyphens.nil? + end + + def mark_completed! + update!(completed_at: Time.current, prompt_dismissed_at: nil) + end + + def dismiss_prompt! + update!( + prompt_dismissed_at: Time.current, + last_prompted_at: Time.current, + prompt_shown_count: prompt_shown_count + 1 + ) + end + + # ---------- Prompt lifecycle ---------------------------------------- + # + # Bundle each persistence change with its domain side effects (rescore the + # user's auctions, record the interaction) so every entry point — the + # profile form, sign-up, the dismiss button — triggers them the same way. + # `source` names the originating action for analytics. + + def complete!(source:, request: nil) + mark_completed! + enqueue_rescore + track_event('recommendation_profile_completed', source, request:) + end + + def skip!(source:, request: nil) + dismiss_prompt! + enqueue_rescore + track_event('recommendation_prompt_dismissed', source, request:) + end + + def dismiss!(source:, request: nil) + dismiss_prompt! + track_event('recommendation_prompt_dismissed', source, request:) + end + + def interest_categories + interest_keywords.select { |value| known_category?(value) } + end + + def interest_categories=(values) + self.interest_keywords = combine_interest_values(categories: values, custom_values: custom_interests) + end + + def rankable_interest_categories + interest_categories - [OTHER_CATEGORY] + end + + def custom_interests + custom_interests_from(interest_keywords) + end + + def custom_interests=(values) + self.interest_keywords = combine_interest_values(categories: interest_categories, custom_values: values) + end + + def summary_lines + [].tap do |lines| + if rankable_interest_categories.any? + labels = rankable_interest_categories.map { |category| Recommendation::InterestCatalog.label_for(category) } + lines << "#{I18n.t('recommendation_profiles.summary.categories')}: #{labels.join(', ')}" + end + + if custom_interests.any? + lines << "#{I18n.t('recommendation_profiles.summary.custom_interests')}: #{custom_interests.join(', ')}" + end + + if preferred_length_min.present? || preferred_length_max.present? + min = preferred_length_min || '?' + max = preferred_length_max || '?' + lines << "#{I18n.t('recommendation_profiles.summary.length')}: #{min}-#{max}" + end + end + end + + private + + def enqueue_rescore + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(user_id) + end + + def custom_interests_from(keywords) + Array(keywords).filter_map do |value| + next unless value.to_s.start_with?(CUSTOM_INTEREST_PREFIX) + + value.to_s.delete_prefix(CUSTOM_INTEREST_PREFIX) + end + end + + # v3: only re-embed when the free-text custom set actually changed — a + # category-only edit reuses the cached vectors and needn't touch OpenAI. + def custom_interests_changed? + return false unless self.class.column_names.include?('custom_interest_vectors') + return false unless Feature.open_ai_integration_enabled? + return false unless saved_change_to_interest_keywords? + + previous = custom_interests_from(interest_keywords_before_last_save) + previous.sort != custom_interests.sort + end + + def enqueue_custom_interest_embedding + Recommendation::EmbedCustomInterestsJob.perform_later(id) + end + + def track_event(event_type, source, request:) + Recommendation::EventTracker.call(user:, event_type:, source:, request:) + end + + def normalize_interest_categories + known_categories = normalize_list(interest_keywords).select { |item| known_category?(item) } + normalized_custom_interests = normalize_custom_interests( + interest_keywords.reject { |item| known_category?(item) } + ) + + self.interest_keywords = combine_categories_with_custom(known_categories, normalized_custom_interests) + end + + def combine_interest_values(categories:, custom_values:) + normalized_categories = normalize_list(categories).select { |item| known_category?(item) } + normalized_custom_interests = normalize_custom_interests(custom_values) + + combine_categories_with_custom(normalized_categories, normalized_custom_interests) + end + + # `other` is purely a derived marker of "has custom interests": present iff + # there is at least one free-text interest. Deriving it here (rather than only + # appending it) means a dangling `other` checkbox left after the last custom + # tag was removed can never linger as an empty category. + def combine_categories_with_custom(categories, custom_interests) + categories = categories.uniq - [OTHER_CATEGORY] + categories << OTHER_CATEGORY if custom_interests.any? + (categories + custom_interests).uniq + end + + def normalize_list(value) + Array(value) + .flat_map do |item| + item.is_a?(String) ? item.to_s.split(',') : item + end + .map { |item| item.to_s.strip.downcase } + .reject(&:blank?) + .uniq + end + + def normalize_custom_interests(values) + normalize_list(values).filter_map do |value| + next if known_category?(value) + + normalized_value = value.delete_prefix(CUSTOM_INTEREST_PREFIX).strip + next if normalized_value.blank? + + "#{CUSTOM_INTEREST_PREFIX}#{normalized_value}" + end + end + + def length_range_is_valid + return unless preferred_length_min.present? && preferred_length_max.present? + return if preferred_length_min <= preferred_length_max + + errors.add(:preferred_length_min, :invalid) + end + + def interest_categories_are_supported + invalid_categories = interest_keywords.reject do |value| + known_category?(value) || value.to_s.start_with?(CUSTOM_INTEREST_PREFIX) + end + return if invalid_categories.empty? + + errors.add(:interest_keywords, :invalid) + end + + def known_category?(value) + Recommendation::InterestCatalog.categories.include?(value.to_s) + end +end diff --git a/app/models/user.rb b/app/models/user.rb index 03a9b0ed0..d50d17ddf 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -47,8 +47,13 @@ class User < ApplicationRecord has_many :autobiders, dependent: :destroy has_many :domain_participate_auctions has_many :notifications, as: :recipient, dependent: :destroy + has_many :recommendation_events, dependent: :destroy + has_many :user_auction_scores, dependent: :destroy has_one :webpush_subscription + has_one :recommendation_profile, dependent: :destroy + + accepts_nested_attributes_for :recommendation_profile, reject_if: :recommendation_profile_attributes_blank? scope :subscribed_to_daily_summary, -> { where(daily_summary: true) } scope :with_confirmed_phone, -> { where.not(mobile_phone_confirmed_at: nil) } @@ -239,4 +244,18 @@ def allow_to_send_sms_again? def active_for_authentication? signed_in_with_identity_document? || super end + + def recommendation_profile_promptable? + return false unless RecommendationProfile.selection_enabled? + + recommendation_profile&.promptable? != false + end + + private + + def recommendation_profile_attributes_blank?(attributes) + attributes.except('_destroy').values.all? do |value| + value.is_a?(Array) ? value.reject(&:blank?).empty? : value.blank? + end + end end diff --git a/app/models/user_auction_score.rb b/app/models/user_auction_score.rb new file mode 100644 index 000000000..73928db51 --- /dev/null +++ b/app/models/user_auction_score.rb @@ -0,0 +1,10 @@ +class UserAuctionScore < ApplicationRecord + belongs_to :user + belongs_to :auction + + validates :score, presence: true, numericality: true + validates :calculated_at, presence: true + # Uniqueness of (user_id, auction_id) is enforced by a DB unique index and the + # upsert_all(unique_by:) write path in Recommendation::Scorer; no form surfaces + # this, so a redundant AR validation would only add a lost-to-the-DB race. +end diff --git a/app/services/openai_structured_output_support.rb b/app/services/openai_structured_output_support.rb new file mode 100644 index 000000000..9dffbaf63 --- /dev/null +++ b/app/services/openai_structured_output_support.rb @@ -0,0 +1,34 @@ +module OpenaiStructuredOutputSupport + DEFAULT_MODEL = 'gpt-5'.freeze + SUPPORTED_MODEL_PREFIXES = ['gpt-4o', 'gpt-4.1', 'gpt-5'].freeze + CUSTOM_TEMPERATURE_UNSUPPORTED_PREFIXES = ['gpt-5'].freeze + + class << self + def model(configured_model) + normalized_model = configured_model.to_s.strip + return DEFAULT_MODEL if normalized_model.blank? + return normalized_model if supported?(normalized_model) + + Rails.logger.warn( + "OpenAI model #{normalized_model.inspect} does not support json_schema structured outputs. " \ + "Falling back to #{DEFAULT_MODEL.inspect}." + ) + DEFAULT_MODEL + end + + def supported?(model_name) + SUPPORTED_MODEL_PREFIXES.any? { |prefix| model_name.start_with?(prefix) } + end + + def temperature_options(model_name, temperature) + return {} if temperature.nil? + return {} if custom_temperature_unsupported?(model_name) + + { temperature: temperature } + end + + def custom_temperature_unsupported?(model_name) + CUSTOM_TEMPERATURE_UNSUPPORTED_PREFIXES.any? { |prefix| model_name.start_with?(prefix) } + end + end +end diff --git a/app/services/recommendation/domain_embedder.rb b/app/services/recommendation/domain_embedder.rb new file mode 100644 index 000000000..9e6f272a4 --- /dev/null +++ b/app/services/recommendation/domain_embedder.rb @@ -0,0 +1,107 @@ +module Recommendation + # Generates OpenAI text-embedding-3-small vectors for a batch of + # DomainClassification rows. NOT called from runtime paths — only + # from EmbedUnembeddedDomainsJob (cron). + # + # Input text per domain is a rich, structured summary: + # ". . Category: . + # Tags: . Use cases: . Audience: . + # Keywords: " + # (empty parts are dropped). The earlier format was only + # ". " — a deliberately sparse signal (ADR-001) that + # under-served interest/wishlist matching; INPUT_VERSION tracks the format so + # rows built under an older version can be re-embedded without re-classifying. + # + # Returns array of { domain_name:, embedding: [..1536..], embedding_model:, + # embedded_at:, embedding_input_version: } ready for update_columns on the + # matching DomainClassification. + class DomainEmbedder + MODEL = 'text-embedding-3-small'.freeze + DIMENSIONS = 1536 + BATCH_LIMIT = 100 + # Bump whenever build_input's format changes so needs_embedding re-embeds + # existing rows. v1 = ". "; v2 = rich structured text. + INPUT_VERSION = 2 + + class << self + def call(...) + new(...).call + end + end + + def initialize(rows:) + @rows = Array(rows).first(BATCH_LIMIT) + end + + def call + return [] if @rows.empty? + + inputs = @rows.map { |row| build_input(row) } + vectors = fetch_embeddings(inputs) + + @rows.each_with_index.map do |row, index| + { + domain_name: domain_name_for(row), + embedding: vectors[index], + embedding_model: MODEL, + embedded_at: Time.current, + embedding_input_version: INPUT_VERSION + } + end + rescue StandardError, OpenAI::Error => e + Rails.logger.warn("DomainEmbedder failed: #{e.message}") + raise + end + + private + + def fetch_embeddings(inputs) + client = OpenAI::Client.new + response = client.embeddings(parameters: { model: MODEL, input: inputs }) + + error = response.dig('error', 'message') + raise StandardError, error if error + + data = response.fetch('data') + data.sort_by { |item| item['index'] }.map { |item| item.fetch('embedding') } + end + + def build_input(row) + tags = Array(field(row, :tags)).map { |t| t.to_s.tr('_', ' ') }.reject(&:blank?) + use_cases = Array(field(row, :suggested_use_cases)).map(&:to_s).reject(&:blank?) + keywords = Array(keywords_for(row)).map(&:to_s).reject(&:blank?) + + parts = [ + domain_name_for(row), + field(row, :description).to_s.strip.presence, + labelled('Category', field(row, :primary_category)), + labelled('Tags', tags.join(', ').presence), + labelled('Use cases', use_cases.join(', ').presence), + labelled('Audience', field(row, :audience)), + labelled('Keywords', keywords.join(', ').presence) + ].compact + parts.join('. ') + end + + # "Label: value" or nil when value is blank, so absent fields never leave + # dangling "Category: ." fragments in the embedding input. + def labelled(label, value) + value = value.to_s.strip + return nil if value.blank? + + "#{label}: #{value}" + end + + def domain_name_for(row) + field(row, :domain_name) + end + + def keywords_for(row) + Array(field(row, :keywords)) + end + + def field(row, name) + row.respond_to?(name) ? row.public_send(name) : row[name] + end + end +end diff --git a/app/services/recommendation/embedding.rb b/app/services/recommendation/embedding.rb new file mode 100644 index 000000000..4810aef79 --- /dev/null +++ b/app/services/recommendation/embedding.rb @@ -0,0 +1,53 @@ +module Recommendation + # Pure vector math for the embedding-similarity multiplier. No DB, no models: + # OpenAI embeddings arrive as plain Float arrays (Postgres double precision[], + # no pgvector), so this stays trivially unit-testable in isolation. + module Embedding + module_function + + # Weighted average of equal-length vectors. `weighted_vectors` is a list of + # [vector, weight] pairs; a vector whose size differs from the first kept + # one is skipped (defensive against an embedding-model dimension change). + # Returns nil when nothing usable was supplied. + def weighted_centroid(weighted_vectors) + sum = nil + total_weight = 0.0 + + weighted_vectors.each do |vector, weight| + vec = Array(vector) + next if vec.empty? + + sum ||= Array.new(vec.size, 0.0) + next if vec.size != sum.size + + vec.each_with_index { |v, i| sum[i] += v.to_f * weight } + total_weight += weight + end + + return nil if sum.nil? || total_weight.zero? + + sum.map { |v| v / total_weight } + end + + # Cosine similarity of two equal-length vectors, or nil when it is undefined + # (mismatched sizes, a zero-magnitude vector). + def cosine_similarity(vec_a, vec_b) + return nil if vec_a.nil? || vec_b.nil? || vec_a.size != vec_b.size + + dot = 0.0 + norm_a = 0.0 + norm_b = 0.0 + vec_a.each_with_index do |a, i| + b = vec_b[i].to_f + dot += a * b + norm_a += a * a + norm_b += b * b + end + + denom = Math.sqrt(norm_a) * Math.sqrt(norm_b) + return nil if denom.zero? + + dot / denom + end + end +end diff --git a/app/services/recommendation/event_tracker.rb b/app/services/recommendation/event_tracker.rb new file mode 100644 index 000000000..664cc05a1 --- /dev/null +++ b/app/services/recommendation/event_tracker.rb @@ -0,0 +1,62 @@ +module Recommendation + class EventTracker + class << self + def call(...) + new(...).call + end + + def track_impressions(user:, auctions:, source:, request: nil) + now = Time.current + session_id = request&.session&.id&.to_s + request_id = request&.request_id + + records = Array(auctions).filter_map do |auction| + next unless auction&.id + + { + user_id: user&.id, + auction_id: auction.id, + event_type: 'auction_impression', + source: source, + session_id: session_id, + request_id: request_id, + occurred_at: now, + properties: {}, + created_at: now, + updated_at: now + } + end + + return if records.empty? + + RecommendationEvent.insert_all(records) + rescue StandardError => e + Rails.logger.warn("Recommendation impressions batch insert failed: #{e.message}") + end + end + + def initialize(user:, event_type:, auction: nil, source: nil, properties: {}, request: nil) + @user = user + @event_type = event_type + @auction = auction + @source = source + @properties = properties || {} + @request = request + end + + def call + RecommendationEvent.create( + user: @user, + auction: @auction, + event_type: @event_type, + source: @source, + session_id: @request&.session&.id&.to_s, + request_id: @request&.request_id, + occurred_at: Time.current, + properties: @properties + ) + rescue StandardError => e + Rails.logger.warn("Recommendation event tracking failed: #{e.message}") + end + end +end diff --git a/app/services/recommendation/interest_catalog.rb b/app/services/recommendation/interest_catalog.rb new file mode 100644 index 000000000..814c4879f --- /dev/null +++ b/app/services/recommendation/interest_catalog.rb @@ -0,0 +1,60 @@ +module Recommendation + # Single source of truth for the interest-category vocabulary. + # + # Categories are admin-managed (InterestCategory table). This module is the + # read API used by the user form, the LLM classifier (category enum), the + # scorer, and the sort. Because every consumer reads `categories` from here, + # adding a category in admin automatically widens the LLM vocabulary and the + # scoring match — no code change needed. + # + # FALLBACK_CATEGORIES is used only when the table is empty or unavailable + # (fresh DB before seeding, or mid-migration) so classification and scoring + # never break. The table holds ~15 rows queried over an (active, position) + # index, so reads are cheap and uncached — avoids cross-process staleness. + module InterestCatalog + FALLBACK_CATEGORIES = %w[ + brandable shop_brand saas b2b_service local_service media_content + finance legal health education travel automotive real_estate numeric other + ].freeze + + class << self + def categories + db_codes.presence || FALLBACK_CATEGORIES + end + + # Locale-aware label for a category code. Falls back to the I18n + # translation, then to the raw code, so it never returns blank. + def label_for(code) + key = code.to_s + labels_by_code[key] || + I18n.t("recommendation_profiles.categories.#{key}", default: key) + end + + private + + def db_codes + return [] unless table_available? + + InterestCategory.active.ordered.pluck(:code) + rescue StandardError + [] + end + + def labels_by_code + return {} unless table_available? + + InterestCategory.active.ordered.each_with_object({}) do |category, acc| + acc[category.code] = category.name + end + rescue StandardError + {} + end + + def table_available? + InterestCategory.table_exists? + rescue StandardError + false + end + end + end +end diff --git a/app/services/recommendation/interest_category_enricher.rb b/app/services/recommendation/interest_category_enricher.rb new file mode 100644 index 000000000..62783dbc3 --- /dev/null +++ b/app/services/recommendation/interest_category_enricher.rb @@ -0,0 +1,172 @@ +module Recommendation + # v3 unified embedding space — the category-side mirror of LlmDomainClassifier + # + DomainEmbedder. Turns an interest category ("Finance and fintech" / + # "Finants ja fintech") into the same kind of rich, embeddable object a domain + # is, so a selected category can act as a magnet in the same 1536-dim space. + # + # One pass, two OpenAI calls (chat for description+keywords, embeddings for the + # vector) because the whole catalog is ~15 rows — no need for the two-job + # split domains use. NOT a runtime path: only EnrichInterestCategoryJob and + # PipelineRunner call this. + # + # Returns array of attribute hashes keyed by category code, ready to update + # onto the matching InterestCategory: + # { code:, description:, keywords:, embedding:, embedding_model:, embedded_at: } + class InterestCategoryEnricher + DEFAULT_TEMPERATURE = 0.2 + BATCH_LIMIT = 50 + EMBED_MODEL = Recommendation::DomainEmbedder::MODEL + + class << self + def call(...) + new(...).call + end + end + + def initialize(categories:, temperature: DEFAULT_TEMPERATURE) + @categories = Array(categories).reject { |c| c.code.to_s.blank? }.first(BATCH_LIMIT) + @temperature = temperature + end + + def call + return [] if @categories.empty? + + enrichment = fetch_enrichment # code => { description:, keywords: } + embed_inputs = @categories.map { |category| build_embed_input(category, enrichment[category.code]) } + vectors = fetch_embeddings(embed_inputs) + + @categories.each_with_index.map do |category, index| + data = enrichment[category.code] || {} + { + code: category.code, + description: data[:description], + keywords: data[:keywords] || [], + embedding: vectors[index], + embedding_model: EMBED_MODEL, + embedded_at: Time.current + } + end + rescue StandardError, OpenAI::Error => e + Rails.logger.warn("InterestCategoryEnricher failed: #{e.message}") + raise + end + + private + + # ---------- Enrichment (chat, structured output) -------------------- + + def fetch_enrichment + content = fetch_ai_response + parsed = JSON.parse(content) + parsed.fetch('categories', []).each_with_object({}) do |entry, acc| + code = entry['code'].to_s.strip.downcase + next if code.blank? + + acc[code] = { + description: entry['description'].to_s.strip.presence, + keywords: Array(entry['keywords']).map { |k| k.to_s.strip.downcase }.reject(&:blank?).uniq + } + end + end + + def fetch_ai_response + client = OpenAI::Client.new + response = client.chat(parameters: chat_parameters) + + finish_reason = response.dig('choices', 0, 'finish_reason') + raise StandardError, 'Incomplete response' if finish_reason == 'length' + + refusal = response.dig('choices', 0, 'message', 'refusal') + raise StandardError, refusal if refusal + + content = response.dig('choices', 0, 'message', 'content') + raise StandardError, response.dig('error', 'message') || 'No response content' if content.nil? + + content + end + + def chat_parameters + model_name = openai_model + { + model: model_name, + response_format: { type: 'json_schema', json_schema: schema }, + messages: messages + }.merge(OpenaiStructuredOutputSupport.temperature_options(model_name, @temperature)) + end + + def schema + { + name: 'interest_categories', + schema: { + type: 'object', + properties: { + categories: { + type: 'array', + items: { + type: 'object', + properties: { + code: { type: 'string' }, + description: { type: 'string' }, + keywords: { type: 'array', items: { type: 'string' } } + }, + required: %w[code description keywords], + additionalProperties: false + } + } + }, + required: ['categories'], + additionalProperties: false + }, + strict: true + } + end + + def messages + payload = @categories.map { |c| { code: c.code, name_en: c.name_en, name_et: c.name_et } } + [ + { role: 'system', content: system_message }, + { role: 'user', content: { categories: payload }.to_json } + ] + end + + def system_message + override = Setting.find_by(code: 'openai_interest_category_prompt')&.retrieve + return override if override.present? + + <<~PROMPT.squish + You enrich interest categories for a .ee domain recommendation system. + For each category return one row keyed by its exact provided `code`. + + Rules: + - description: one or two sentences describing the kind of businesses, + products, or domains that belong to this category. + - keywords: 10 to 20 lowercase semantic tokens that domains in this + category would use. Include BOTH English and Estonian terms + (the system serves an Estonian registry), e.g. for real estate: + 'real estate','property','rent','mortgage','kinnisvara','üür','laen'. + No stopwords, no duplicates. + - Keep it concrete and domain-oriented; these keywords are embedded and + matched against real domain names by semantic similarity. + PROMPT + end + + def openai_model + OpenaiStructuredOutputSupport.model(Setting.find_by(code: 'openai_model')&.retrieve) + end + + # ---------- Embedding (embeddings endpoint) ------------------------- + + def build_embed_input(category, data) + keywords = Array(data && data[:keywords]) + [ + category.name_en, + category.name_et, + keywords.join(', ').presence + ].compact.join('. ') + end + + def fetch_embeddings(inputs) + Recommendation::TextEmbedder.embed(inputs) + end + end +end diff --git a/app/services/recommendation/llm_domain_classifier.rb b/app/services/recommendation/llm_domain_classifier.rb new file mode 100644 index 000000000..eaa8130ca --- /dev/null +++ b/app/services/recommendation/llm_domain_classifier.rb @@ -0,0 +1,197 @@ +module Recommendation + # Tier 2 classifier: OpenAI structured outputs. + # + # Accepts an array of domain names (strings), returns array of attribute + # hashes ready to upsert into domain_classifications. Called from + # ClassifyDomainJob (per-domain, on auction create), the nightly + # ClassifyUnclassifiedDomainsJob, and BackfillDomainClassificationsJob. + # + # Cost: ~$0.30/month at steady state (one batch of ~5-20 domains per + # day). Backfill: ~$1 one-time over thousands of historical domains. + class LlmDomainClassifier + DEFAULT_TEMPERATURE = 0.2 + BATCH_LIMIT = 50 + + AUDIENCE_VALUES = %w[b2b b2c mixed unclear].freeze + + class << self + def call(...) + new(...).call + end + end + + def initialize(domain_names:, temperature: DEFAULT_TEMPERATURE) + @domain_names = Array(domain_names).map { |d| d.to_s.strip.downcase }.uniq.reject(&:blank?).first(BATCH_LIMIT) + @temperature = temperature + end + + def call + return [] if @domain_names.empty? + + response_content = fetch_ai_response + parsed = JSON.parse(response_content) + raw_response = parsed + parsed.fetch('classifications', []).filter_map do |entry| + build_attributes(entry, raw_response) + end + rescue StandardError, OpenAI::Error => e + Rails.logger.warn("LlmDomainClassifier failed: #{e.message}") + raise + end + + private + + def fetch_ai_response + client = OpenAI::Client.new + response = client.chat(parameters: chat_parameters) + + finish_reason = response.dig('choices', 0, 'finish_reason') + raise StandardError, 'Incomplete response' if finish_reason == 'length' + + refusal = response.dig('choices', 0, 'message', 'refusal') + raise StandardError, refusal if refusal + + content = response.dig('choices', 0, 'message', 'content') + raise StandardError, response.dig('error', 'message') || 'No response content' if content.nil? + + content + end + + def chat_parameters + model_name = openai_model + { + model: model_name, + response_format: { type: 'json_schema', json_schema: schema }, + messages: messages + }.merge(OpenaiStructuredOutputSupport.temperature_options(model_name, @temperature)) + end + + def schema + { + name: 'domain_classifications', + schema: { + type: 'object', + properties: { + classifications: { + type: 'array', + items: classification_item_schema + } + }, + required: ['classifications'], + additionalProperties: false + }, + strict: true + } + end + + def classification_item_schema + { + type: 'object', + properties: { + domain_name: { type: 'string' }, + description: { type: 'string' }, + primary_category: { type: 'string', enum: Recommendation::InterestCatalog.categories }, + tags: { type: 'array', items: { type: 'string', enum: Recommendation::InterestCatalog.categories } }, + keywords: { type: 'array', items: { type: 'string' } }, + audience: { type: 'string', enum: AUDIENCE_VALUES }, + languages: { type: 'array', items: { type: 'string' } }, + suggested_use_cases: { type: 'array', items: { type: 'string' } }, + brandability_score: { type: 'number' }, + confidence: { type: 'number' } + }, + required: %w[ + domain_name description primary_category tags keywords audience languages + suggested_use_cases brandability_score confidence + ], + additionalProperties: false + } + end + + def messages + [ + { role: 'system', content: system_message }, + { role: 'user', content: { domains: @domain_names }.to_json } + ] + end + + def system_message + override = Setting.find_by(code: 'openai_domain_classification_prompt')&.retrieve + return override if override.present? + + <<~PROMPT.squish + You enrich .ee auction domain names with structured marketing metadata + for a recommendation system. For each provided domain return one row. + + Rules: + - description: 1-2 plain English sentences stating what the domain is + and what someone would use it for. Concrete, no marketing fluff. This + text is embedded for semantic matching, so name the topic explicitly + (e.g. 'An online store selling pet food and supplies.'). + - primary_category and tags MUST come from this fixed vocabulary: + #{Recommendation::InterestCatalog.categories.join(', ')}. + - tags: 1 to 4 entries; primary_category must be one of them. + - keywords: 2-6 lowercase semantic tokens extracted or inferred + from the domain. No stopwords. + - audience: 'b2b' for business buyers, 'b2c' for consumers, + 'mixed' if both, 'unclear' if not inferable. + - languages: subset of ['et','en'] depending on which language(s) + the domain reads as. + - suggested_use_cases: 1-4 short lowercase nouns describing what + someone could build (e.g. 'shop','blog','agency','marketplace', + 'directory','service'). + - brandability_score: float 0..1. 1.0 = memorable, short, no + numbers/hyphens, evocative. 0.0 = literal/awkward/numeric. + - confidence: float 0..1 measuring your overall certainty. + Prefer 'unclear' / lower confidence over invented data. + PROMPT + end + + def openai_model + OpenaiStructuredOutputSupport.model(Setting.find_by(code: 'openai_model')&.retrieve) + end + + def build_attributes(entry, raw_response) + domain_name = entry['domain_name'].to_s.strip.downcase + return nil if domain_name.blank? + + tags = sanitize_categories(entry['tags']) + primary = sanitize_category(entry['primary_category']) + primary = tags.first if primary.blank? && tags.any? + + { + domain_name: domain_name, + description: entry['description'].to_s.strip.presence, + primary_category: primary, + tags: tags, + keywords: Array(entry['keywords']).map { |k| k.to_s.strip.downcase }.reject(&:blank?).uniq, + audience: AUDIENCE_VALUES.include?(entry['audience']) ? entry['audience'] : nil, + languages: Array(entry['languages']).map { |l| l.to_s.strip.downcase }.reject(&:blank?).uniq, + suggested_use_cases: Array(entry['suggested_use_cases']).map { |u| u.to_s.strip.downcase }.reject(&:blank?).uniq, + brandability_score: clamp_unit(entry['brandability_score']), + confidence: clamp_unit(entry['confidence']), + classification_source: DomainClassification::OPENAI_SOURCE, + classification_model: openai_model, + classified_at: Time.current, + raw_llm_response: { entry: entry, batch_response_excerpt: raw_response['classifications']&.size } + } + end + + def sanitize_categories(values) + Array(values) + .map { |v| v.to_s.strip.downcase } + .select { |v| Recommendation::InterestCatalog.categories.include?(v) } + .uniq + end + + def sanitize_category(value) + cleaned = value.to_s.strip.downcase + Recommendation::InterestCatalog.categories.include?(cleaned) ? cleaned : nil + end + + def clamp_unit(value) + return nil if value.nil? + + value.to_f.clamp(0.0, 1.0).round(3) + end + end +end diff --git a/app/services/recommendation/magnet_scorer.rb b/app/services/recommendation/magnet_scorer.rb new file mode 100644 index 000000000..14f74e3fb --- /dev/null +++ b/app/services/recommendation/magnet_scorer.rb @@ -0,0 +1,221 @@ +module Recommendation + # Recommendation::MagnetScorer (v3, live engine) + # ---------------------------------------------- + # Computes the "unified embedding" score described in + # docs/planning/recommendation-v3-unified-embedding-plan.md. This is the LIVE + # ranking engine: Recommendation::Scorer delegates its `magnet_base` to this + # class, scales the result ×100, adds structural nudges, and persists it into + # user_auction_scores — which Auction::UserSortable joins to order /auctions. + # + # The user is a *set of magnets*, each a (vector, weight) pair: + # - every bid domain → weight 3.0, time-decayed + # - every wishlist domain → weight 3.0, time-decayed + # - every selected category (its embedding) → weight 2.0, no decay + # - every custom interest (its cached embedding) → weight 2.0, no decay + # - every viewed domain → weight 1.0, time-decayed + # + # A candidate auction's score is the mean of its two strongest pulls, where + # pull(magnet) = magnet.weight * cosine(magnet.vector, auction.embedding). + # top-2 (not raw max) keeps a single lucky magnet from deciding everything. + # + # Score is nil (→ auction falls to the ai_score/random tail) when the auction + # has no embedding or the user has no usable magnet. + # + # NOTE: signal queries here mirror those the Scorer would otherwise need; the + # two classes are kept separate so the pure embedding/behavioural computation + # (this class) stays independent of the structural scoring layer (Scorer). + class MagnetScorer + HALF_LIFE_DAYS = 60.0 + + BID_WEIGHT = 3.0 + WISHLIST_WEIGHT = 3.0 + CATEGORY_WEIGHT = 2.0 + CUSTOM_WEIGHT = 2.0 + VIEW_WEIGHT = 1.0 + + # Cap behavioural magnets by weight×freshness so a long history can't bloat + # the per-candidate cosine loop. Categories/custom are never capped. + MAX_BEHAVIOURAL_MAGNETS = 30 + # Average the N strongest pulls (tuning knob — see plan §3.4). + TOP_PULLS = 2 + + FEATURES_VERSION = 'unified_v3'.freeze + + Anchor = Struct.new(:vector, :weight) + + def self.default_scope + Auction.active + end + + def initialize(user:, scope: self.class.default_scope, calculated_at: Time.current) + @user = user + @scope = scope + @calculated_at = calculated_at + end + + # { auction_id => Float|nil } + def scores + auctions.each_with_object({}) { |auction, acc| acc[auction.id] = score_for(auction) } + end + + # [[auction, score], …] highest first, nil scores dropped. + def ranked(limit: nil) + scored = auctions.filter_map do |auction| + score = score_for(auction) + [auction, score] unless score.nil? + end + scored.sort_by! { |_, score| -score } + limit ? scored.first(limit) : scored + end + + private + + def score_for(auction) + return nil unless @user + + vector = auction_embeddings[auction.id] + return nil if vector.nil? + return nil if magnets.empty? + + pulls = magnets.filter_map do |anchor| + similarity = Recommendation::Embedding.cosine_similarity(anchor.vector, vector) + next if similarity.nil? + + anchor.weight * similarity + end + return nil if pulls.empty? + + strongest = pulls.max(TOP_PULLS) + (strongest.sum / strongest.size).round(6) + end + + # ---------- Magnets -------------------------------------------------- + + def magnets + @magnets ||= behavioural_magnets + category_magnets + custom_magnets + end + + def behavioural_magnets + weighted_signals = + bid_signals.map { |signal| [signal, BID_WEIGHT] } + + wishlist_signals.map { |signal| [signal, WISHLIST_WEIGHT] } + + view_signals.map { |signal| [signal, VIEW_WEIGHT] } + return [] if weighted_signals.empty? + + embeddings = domain_embeddings(weighted_signals.map { |signal, _| signal[:domain_name] }.uniq) + + anchors = weighted_signals.filter_map do |signal, base_weight| + vector = embeddings[signal[:domain_name]] + next if vector.nil? + + Anchor.new(vector, base_weight * decay_weight(signal[:age_days])) + end + + anchors.sort_by { |anchor| -anchor.weight }.first(MAX_BEHAVIOURAL_MAGNETS) + end + + def category_magnets + return [] unless InterestCategory.column_names.include?('embedding') + + codes = Array(profile&.rankable_interest_categories) + return [] if codes.empty? + + InterestCategory + .active + .where(code: codes) + .where.not(embedding: nil) + .pluck(:embedding) + .filter_map { |vector| build_anchor(vector, CATEGORY_WEIGHT) } + end + + def custom_magnets + return [] unless RecommendationProfile.column_names.include?('custom_interest_vectors') + + Array(profile&.custom_interest_vectors).filter_map do |entry| + vector = entry['embedding'] || entry[:embedding] + build_anchor(vector, CUSTOM_WEIGHT) + end + end + + def build_anchor(vector, weight) + vector = Array(vector) + return nil if vector.empty? + + Anchor.new(vector, weight) + end + + # ---------- Embeddings preload --------------------------------------- + + def auction_embeddings + @auction_embeddings ||= begin + by_domain = domain_embeddings(auctions.map { |auction| auction.domain_name.to_s.downcase }.uniq) + auctions.each_with_object({}) do |auction, acc| + acc[auction.id] = by_domain[auction.domain_name.to_s.downcase] + end + end + end + + def domain_embeddings(domain_names) + return {} if domain_names.empty? + return {} unless DomainClassification.column_names.include?('embedding') + + DomainClassification + .where(domain_name: domain_names) + .where.not(embedding: nil) + .index_by { |dc| dc.domain_name.to_s.downcase } + .transform_values { |dc| Array(dc.embedding).presence } + .compact + end + + # ---------- Signal collection (mirrors Scorer) ----------------------- + + def bid_signals + @bid_signals ||= + Offer + .joins(:auction) + .where(user_id: @user.id) + .pluck(Arel.sql('LOWER(auctions.domain_name)'), Arel.sql('offers.updated_at')) + .map { |domain, time| { domain_name: domain, age_days: age_in_days(time) } } + end + + def wishlist_signals + @wishlist_signals ||= + @user.wishlist_items + .pluck(Arel.sql('LOWER(wishlist_items.domain_name)'), Arel.sql('wishlist_items.updated_at')) + .map { |domain, time| { domain_name: domain, age_days: age_in_days(time) } } + end + + def view_signals + return [] unless RecommendationEvent.table_exists? + + @view_signals ||= + RecommendationEvent + .joins(:auction) + .where(user_id: @user.id, event_type: 'auction_detail_view') + .pluck(Arel.sql('LOWER(auctions.domain_name)'), Arel.sql('recommendation_events.occurred_at')) + .map { |domain, time| { domain_name: domain, age_days: age_in_days(time) } } + end + + # ---------- Helpers -------------------------------------------------- + + def auctions + @auctions ||= @scope.to_a + end + + def profile + @profile ||= @user&.recommendation_profile + end + + def decay_weight(age_days) + return 1.0 if age_days.nil? || age_days <= 0 + + Math.exp(-age_days.to_f / HALF_LIFE_DAYS) + end + + def age_in_days(timestamp) + return 0.0 if timestamp.nil? + + ((@calculated_at - timestamp).to_f / 1.day).clamp(0.0, Float::INFINITY) + end + end +end diff --git a/app/services/recommendation/pipeline_runner.rb b/app/services/recommendation/pipeline_runner.rb new file mode 100644 index 000000000..afc4910e8 --- /dev/null +++ b/app/services/recommendation/pipeline_runner.rb @@ -0,0 +1,176 @@ +module Recommendation + # One-command orchestrator for the whole recommendation pipeline: + # classify -> embed -> global ai_score -> per-user scores. + # + # Used by: + # - rake recommendation:init (prod, force: false — incremental) + # - rake recommendation:init_demo (staging — mock data, then this) + # - RebuildRecommendationsJob (admin /admin/jobs, force: true — re-tag + # every domain after the interest catalog + # changed) + # + # Every stage runs synchronously and drains the per-run batch caps, so when + # run returns the pipeline is actually complete — not merely enqueued. + class PipelineRunner + AI_SCORE_PROMPT_SETTING = 'openai_domains_evaluation_prompt'.freeze + # db/seeds.rb seeds this setting with the literal placeholder 'prompt'. + # Running ai_score against it produces meaningless scores, so we skip. + AI_SCORE_PLACEHOLDER_PROMPT = 'prompt'.freeze + # Safety cap so a stubborn domain that keeps re-entering scope can never + # loop forever (progress detection already stops normal backlogs sooner). + MAX_BATCHES = 100 + + def self.run(force: false) + new(force:).run + end + + def initialize(force:) + @force = force + @summary = {} + end + + def run + unless Feature.open_ai_integration_enabled? + log('OpenAI integration disabled — nothing to do') + return @summary + end + + if @force + invalidate_classifications + invalidate_category_embeddings + invalidate_custom_interest_embeddings + end + enrich_categories + embed_custom_interests + drain(Recommendation::ClassifyUnclassifiedDomainsJob, :classify) { classify_pending_count } + drain(Recommendation::EmbedUnembeddedDomainsJob, :embed) { embed_pending_count } + refresh_ai_scores + refresh_user_scores + + log("pipeline complete: #{@summary.inspect}") + @summary + end + + private + + # force mode: push every LLM classification past the staleness threshold so + # ClassifyUnclassifiedDomainsJob re-classifies it under the current interest + # catalog. The re-classify upsert resets the embedding automatically, so the + # embed stage repopulates vectors from the fresh keywords. + def invalidate_classifications + count = DomainClassification + .where(classification_source: DomainClassification::OPENAI_SOURCE) + .update_all(classified_at: 7.months.ago) + @summary[:invalidated] = count + log("force: invalidated #{count} classification(s) for re-run") + end + + # Run the batched job until its pending backlog stops shrinking. Stopping on + # "no progress" (rather than on needs_to_run?) means stubborn rows — e.g. a + # domain the LLM keeps scoring below the confidence threshold — cost at most + # one extra batch instead of looping. + def drain(job_class, key) + batches = 0 + previous = nil + while (current = yield).positive? && (previous.nil? || current < previous) && batches < MAX_BATCHES + job_class.perform_now + previous = current + batches += 1 + end + @summary[key] = { batches:, remaining: yield } + log("#{job_class.name}: #{batches} batch(es), #{@summary[key][:remaining]} remaining") + end + + # v3: turn interest categories into embeddable magnets (LLM keywords + vector) + # before scoring. Independent of domains — safe to run first. Cheap (~15 rows). + def enrich_categories + return unless InterestCategory.column_names.include?('embedding') + + count = Recommendation::EnrichInterestCategoriesJob.perform_now + @summary[:categories_enriched] = count + log("categories enriched: #{count.inspect}") + end + + # force mode (catalog changed): drop category vectors so enrich_categories + # rebuilds them under the new/edited vocabulary. + def invalidate_category_embeddings + return unless InterestCategory.column_names.include?('embedding') + + count = InterestCategory.update_all(embedded_at: nil, embedding: nil) + @summary[:categories_invalidated] = count + log("force: invalidated #{count} category embedding(s)") + end + + # force mode: drop per-user custom-interest vectors so embed_custom_interests + # rebuilds them (e.g. after an embedding-model change). + def invalidate_custom_interest_embeddings + return unless RecommendationProfile.column_names.include?('custom_interest_vectors') + + count = RecommendationProfile.update_all(custom_interest_vectors: [], custom_interests_embedded_at: nil) + @summary[:custom_interests_invalidated] = count + log("force: invalidated custom-interest vectors on #{count} profile(s)") + end + + # v3: backfill custom-interest vectors for profiles whose free-text interests + # were set before this feature existed (or before a force reset), or whose + # embed job silently no-op'd (OpenAI off / worker down). The after_save_commit + # trigger only fires on a profile *save*, so those would otherwise never get + # custom magnets until they re-saved. Idempotent: re-embeds a profile only + # when its stored vectors don't match its current custom interests. + def embed_custom_interests + return unless RecommendationProfile.column_names.include?('custom_interest_vectors') + + count = 0 + RecommendationProfile.where.not(interest_keywords: []).find_each do |profile| + interests = profile.custom_interests + next if interests.empty? + + vector_texts = Array(profile.custom_interest_vectors).filter_map { |v| v['text'] || v[:text] } + next if vector_texts.sort == interests.sort + + Recommendation::EmbedCustomInterestsJob.perform_now(profile.id) + count += 1 + end + @summary[:custom_interests_embedded] = count + log("custom interests embedded for #{count} profile(s)") + end + + def classify_pending_count + Recommendation::ClassifyUnclassifiedDomainsJob.scope.count + + Recommendation::ClassifyUnclassifiedDomainsJob.missing_domains.size + end + + def embed_pending_count + return 0 unless DomainClassification.column_names.include?('embedding') + + Recommendation::EmbedUnembeddedDomainsJob.scope.count + end + + def refresh_ai_scores + prompt = Setting.find_by(code: AI_SCORE_PROMPT_SETTING)&.retrieve.to_s.strip + if prompt.blank? || prompt == AI_SCORE_PLACEHOLDER_PROMPT + @summary[:ai_score] = :skipped + log("skipping ai_score: #{AI_SCORE_PROMPT_SETTING} is not configured (#{prompt.inspect})") + return + end + + ActiveAuctionsAiSortingJob.perform_now + @summary[:ai_score] = :done + log('ai_score refreshed') + end + + def refresh_user_scores + count = 0 + User.where('? = ANY (roles)', User::PARTICIPANT_ROLE).find_each do |user| + Recommendation::Scorer.refresh_for(user:) + count += 1 + end + @summary[:users_scored] = count + log("personal scores refreshed for #{count} participant(s)") + end + + def log(message) + Rails.logger.info("[Recommendation::PipelineRunner] #{message}") + end + end +end diff --git a/app/services/recommendation/scorer.rb b/app/services/recommendation/scorer.rb new file mode 100644 index 000000000..fd0d6334b --- /dev/null +++ b/app/services/recommendation/scorer.rb @@ -0,0 +1,260 @@ +module Recommendation + # Recommendation::Scorer (v3, unified embedding space) + # ---------------------------------------------------- + # Computes a per-user score for each active auction and upserts it into + # user_auction_scores. Auction::UserSortable LEFT JOINs it to personalise the + # /auctions index. + # + # v3 model (see docs/planning/recommendation-v3-unified-embedding-plan.md): + # the ranking signal is *magnet pull* — everything about the user (bids, + # wishlist, views, selected categories, custom interests) is embedded into one + # vector space and the score is the mean of a candidate's two strongest pulls. + # Recommendation::MagnetScorer owns that computation; this class scales it and + # adds the structural nudges that similarity can't express (name shape + past + # auction outcomes). + # + # score = magnet_base * MAGNET_SCALE + # + length/digits/hyphen preference bonuses + # + result signal (lost similar auction ↑ / won ↓) + # + # No magnet or no candidate embedding → no row is written (and any stale row + # is deleted), so the LEFT JOIN yields NULL and the domain falls to the + # ai_score / RANDOM tail — exactly the "score IS NULL" branch in UserSortable. + class Scorer + HALF_LIFE_DAYS = MagnetScorer::HALF_LIFE_DAYS + + # Magnet pull is ~[-3, 3] (weight ≤3 × cosine ≤1, top-2 averaged). Scaling + # it up lets similarity dominate the ordering while the structural bonuses + # below (tens) act as secondary nudges. Tuning knob — see plan §3.4. + MAGNET_SCALE = 100.0 + + LENGTH_MATCH_BONUS = 10 + RESULT_LOST_BONUS = 25 + RESULT_WON_PENALTY = -5 + + SCORER_NAME = 'unified_magnets_v3'.freeze + FEATURES_VERSION = 'unified_v3'.freeze + + class << self + def default_scope + Auction.active + end + + def top_auctions_for(user:, scope: default_scope, limit: nil) + query = scope + .joins(:user_auction_scores) + .where(user_auction_scores: { user_id: user.id }) + .order('user_auction_scores.score DESC, auctions.ends_at ASC') + + limit ? query.limit(limit) : query + end + + def refresh_for(user:, scope: default_scope, calculated_at: Time.current) + new(user:, scope:, calculated_at:).refresh! + end + end + + def initialize(user:, scope: self.class.default_scope, calculated_at: Time.current) + @user = user + @scope = scope + @calculated_at = calculated_at + end + + def refresh! + return 0 unless @user + + auctions = @scope.to_a + return 0 if auctions.empty? + + preload_classifications(auctions) + records = auctions.filter_map { |auction| build_score_record(auction) } + + # Drop rows for candidates that no longer earn a score (magnet vanished, + # embedding removed) so they correctly fall back to the tail. + stale_ids = auctions.map(&:id) - records.map { |record| record[:auction_id] } + UserAuctionScore.where(user_id: @user.id, auction_id: stale_ids).delete_all if stale_ids.any? + + UserAuctionScore.upsert_all(records, unique_by: %i[user_id auction_id]) if records.any? + records.size + end + + private + + # ---------- Per-auction scoring -------------------------------------- + + def build_score_record(auction) + value = score_for(auction) + return nil if value.nil? + + { + user_id: @user.id, + auction_id: auction.id, + score: value, + scorer_name: SCORER_NAME, + features_version: FEATURES_VERSION, + calculated_at: @calculated_at, + created_at: Time.current, + updated_at: Time.current + } + end + + def score_for(auction) + base = magnet_base[auction.id] + return nil if base.nil? + + domain_name = normalized_domain_name(auction.domain_name) + + score = base * MAGNET_SCALE + score += LENGTH_MATCH_BONUS if within_preferred_length?(domain_name) + score += digits_score(domain_name) + score += hyphen_score(domain_name) + score += result_signal(tags_for(auction)) + score.round(6) + end + + # ---------- Magnet base (delegated) ---------------------------------- + # + # { auction_id => Float|nil }. MagnetScorer owns all embedding/behavioural + # signal collection, so this class keeps only the structural layer. + + def magnet_base + @magnet_base ||= + MagnetScorer.new(user: @user, scope: @scope, calculated_at: @calculated_at).scores + end + + # ---------- Classification preload (for result signal) --------------- + + def preload_classifications(auctions) + domain_names = auctions.map { |a| a.domain_name.to_s.downcase }.uniq + @classifications_by_domain = + DomainClassification + .where(domain_name: domain_names) + .index_by { |dc| dc.domain_name.to_s.downcase } + end + + def classification_for(auction) + return nil unless defined?(@classifications_by_domain) + + @classifications_by_domain[auction.domain_name.to_s.downcase] + end + + def tags_for(auction) + dc = classification_for(auction) + (dc&.tags || Array(auction.classification_tags)).map(&:to_s).uniq + end + + # ---------- Result signal -------------------------------------------- + # + # If the user previously LOST an auction on a similar-tag domain, they're + # still in the market — bump similar tags. If they WON, mild down-weight. + + def result_signal(tags) + return 0 if tags.empty? || result_signal_by_tag.empty? + + tags.sum { |tag| result_signal_by_tag[tag.to_s].to_f } + end + + def result_signal_by_tag + @result_signal_by_tag ||= compute_result_signal + end + + def compute_result_signal + return {} unless defined?(Result) && Result.table_exists? + + results = lookup_user_results + return {} if results.blank? + + classifications = preload_result_classifications(results) + signals = Hash.new(0.0) + + results.each do |result| + domain = result.respond_to?(:domain_name) ? result.domain_name.to_s.downcase : nil + next if domain.blank? + + dc = classifications[domain] + next if dc.nil? + + won = result.respond_to?(:winner_user_id) && result.winner_user_id == @user.id + decay = decay_weight(age_in_days(result.updated_at)) + bonus = won ? RESULT_WON_PENALTY : RESULT_LOST_BONUS + + Array(dc.tags).each { |tag| signals[tag.to_s] += bonus * decay } + end + + signals + end + + def lookup_user_results + return [] unless Result.column_names.include?('winner_user_id') + + Result.where(winner_user_id: @user.id).to_a + end + + def preload_result_classifications(results) + domain_names = results.filter_map do |r| + r.domain_name.to_s.downcase if r.respond_to?(:domain_name) && r.domain_name.present? + end.uniq + return {} if domain_names.empty? + + DomainClassification.where(domain_name: domain_names).index_by(&:domain_name) + end + + # ---------- Structural (name shape) ---------------------------------- + + def within_preferred_length?(domain_name) + return false unless profile + + length = domain_name.length + return false if profile.preferred_length_min.present? && length < profile.preferred_length_min + return false if profile.preferred_length_max.present? && length > profile.preferred_length_max + + profile.preferred_length_min.present? || profile.preferred_length_max.present? + end + + def digits_score(domain_name) + return 0 unless domain_name.match?(/\d/) + + if profile&.allow_numbers == false + -20 + elsif profile&.allow_numbers == true + 8 + else + 0 + end + end + + def hyphen_score(domain_name) + return 0 unless domain_name.include?('-') + + if profile&.allow_hyphens == false + -12 + elsif profile&.allow_hyphens == true + 5 + else + 0 + end + end + + # ---------- Helpers -------------------------------------------------- + + def profile + @profile ||= @user.recommendation_profile + end + + def decay_weight(age_days) + return 1.0 if age_days.nil? || age_days <= 0 + + Math.exp(-age_days.to_f / HALF_LIFE_DAYS) + end + + def age_in_days(timestamp) + return 0.0 if timestamp.nil? + + ((@calculated_at - timestamp).to_f / 1.day).clamp(0.0, Float::INFINITY) + end + + def normalized_domain_name(value) + value.to_s.downcase.sub(/\.ee\z/, '') + end + end +end diff --git a/app/services/recommendation/text_embedder.rb b/app/services/recommendation/text_embedder.rb new file mode 100644 index 000000000..ee47294cd --- /dev/null +++ b/app/services/recommendation/text_embedder.rb @@ -0,0 +1,27 @@ +module Recommendation + # Thin wrapper over the OpenAI embeddings endpoint for arbitrary text. + # Shared by the v3 category enricher and custom-interest embedding so the + # request/response handling lives in one place. DomainEmbedder predates this + # and keeps its own copy to avoid touching the proven domain path. + # + # Returns an array of 1536-float vectors aligned to the input order. + module TextEmbedder + MODEL = Recommendation::DomainEmbedder::MODEL + + module_function + + def embed(texts) + inputs = Array(texts).map(&:to_s) + return [] if inputs.empty? + + client = OpenAI::Client.new + response = client.embeddings(parameters: { model: MODEL, input: inputs }) + + error = response.dig('error', 'message') + raise StandardError, error if error + + data = response.fetch('data') + data.sort_by { |item| item['index'] }.map { |item| item.fetch('embedding') } + end + end +end diff --git a/app/views/admin/auctions/_classification_details.html.erb b/app/views/admin/auctions/_classification_details.html.erb new file mode 100644 index 000000000..f1bd78076 --- /dev/null +++ b/app/views/admin/auctions/_classification_details.html.erb @@ -0,0 +1,78 @@ +<%# Full LLM-classification breakdown for the auction detail page. + `classification` is a DomainClassification or nil. %> + + + <%= t('admin.auctions.classification.heading') %> + + + +<% if classification.nil? %> + + + <%= t('admin.auctions.classification.none') %> + + +<% else %> + + <%= t('admin.auctions.classification.category') %> + <%= classification.primary_category.presence || '—' %> + + <% if classification.respond_to?(:description) %> + + <%= t('admin.auctions.classification.description') %> + <%= classification.description.presence || '—' %> + + <% end %> + <% { + tags: classification.tags, + keywords: classification.keywords, + audience: classification.audience, + languages: classification.languages, + use_cases: classification.suggested_use_cases + }.each do |key, values| %> + + <%= t("admin.auctions.classification.#{key}") %> + + <% if Array(values).any? %> +
+ <% Array(values).each do |value| %> + <%= value %> + <% end %> +
+ <% else %> + — + <% end %> + + + <% end %> + + <%= t('admin.auctions.classification.brandability') %> + <%= classification.brandability_score.present? ? "#{(classification.brandability_score.to_f * 100).round}%" : '—' %> + + + <%= t('admin.auctions.classification.confidence') %> + <%= classification.confidence.present? ? "#{(classification.confidence.to_f * 100).round}%" : '—' %> + + + <%= t('admin.auctions.classification.source') %> + <%= classification.classification_source.presence || '—' %> + + + <%= t('admin.auctions.classification.model') %> + <%= classification.classification_model.presence || '—' %> + + + <%= t('admin.auctions.classification.classified_at') %> + <%= classification.classified_at.present? ? I18n.l(classification.classified_at) : '—' %> + + + <%= t('admin.auctions.classification.embedding') %> + + <% if classification.embedded_at.present? %> + <%= t('admin.auctions.classification.embedded_yes') %> + <% else %> + <%= t('admin.auctions.classification.embedded_no') %> + <% end %> + + +<% end %> diff --git a/app/views/admin/auctions/show.html.erb b/app/views/admin/auctions/show.html.erb index 8cabe2a2a..e12c5a762 100644 --- a/app/views/admin/auctions/show.html.erb +++ b/app/views/admin/auctions/show.html.erb @@ -28,6 +28,7 @@ <%= @auction.current_price_from_user(current_user) %> <% end %> + <%= render 'admin/auctions/classification_details', classification: @classification %> <% end %> <% end %> diff --git a/app/views/admin/interest_categories/_enrichment.html.erb b/app/views/admin/interest_categories/_enrichment.html.erb new file mode 100644 index 000000000..0cf479ae8 --- /dev/null +++ b/app/views/admin/interest_categories/_enrichment.html.erb @@ -0,0 +1,45 @@ +<%# Read-only view of the LLM enrichment results for an interest category + (description, keywords, embedding). Generated by + Recommendation::InterestCategoryEnricher, not editable by hand. %> +
+

<%= t('interest_categories.enrichment_heading') %>

+ + <% if interest_category.embedded? %> + + + + + + + + + + + + + + + + + + + + + + + +
<%= t('interest_categories.description') %><%= interest_category.description.presence || '—' %>
<%= t('interest_categories.keywords') %> + <% if Array(interest_category.keywords).any? %> +
+ <% Array(interest_category.keywords).each do |keyword| %> + <%= keyword %> + <% end %> +
+ <% else %> + — + <% end %> +
<%= t('interest_categories.embedding') %><%= t('interest_categories.enriched') %>
<%= t('interest_categories.model') %><%= interest_category.embedding_model.presence || '—' %>
<%= t('interest_categories.embedded_at') %><%= interest_category.embedded_at.present? ? I18n.l(interest_category.embedded_at) : '—' %>
+ <% else %> + + <% end %> +
diff --git a/app/views/admin/interest_categories/_form.html.erb b/app/views/admin/interest_categories/_form.html.erb new file mode 100644 index 000000000..4518c2c3b --- /dev/null +++ b/app/views/admin/interest_categories/_form.html.erb @@ -0,0 +1,41 @@ +<%= form_with model: interest_category, url: url do |f| %> + <% if interest_category.errors.any? %> +
+
<%= t(:errors_name) %>
+
    + <% interest_category.errors.full_messages.each do |message| %> +
  • <%= message %>
    • + <% end %> +
+
+ <% end %> + +
+ <%= f.label :code, t('interest_categories.code'), style: 'width: 150px;' %> + <%= f.text_field :code, class: "form-control" %> +
+
+ <%= f.label :name_en, t('interest_categories.name_en'), style: 'width: 150px;' %> + <%= f.text_field :name_en, class: "form-control" %> +
+
+ <%= f.label :name_et, t('interest_categories.name_et'), style: 'width: 150px;' %> + <%= f.text_field :name_et, class: "form-control" %> +
+
+ <%= f.label :position, t('interest_categories.position'), style: 'width: 150px;' %> + <%= f.number_field :position, class: "form-control" %> +
+
+ <%= f.label :active, t('interest_categories.active'), style: 'width: 150px;' %> + <%= f.check_box :active %> +
+ +
+ <%= f.button t(:submit), class: "c-btn c-btn--blue", data: { turbo: false } %> +
+ +
+ <%= link_to t(:back), admin_interest_categories_path, class: "c-btn c-btn--green" %> +
+<% end %> diff --git a/app/views/admin/interest_categories/edit.html.erb b/app/views/admin/interest_categories/edit.html.erb new file mode 100644 index 000000000..03ec14e97 --- /dev/null +++ b/app/views/admin/interest_categories/edit.html.erb @@ -0,0 +1,6 @@ +<% content_for :title, t('.title') %> + +
+ <%= render 'form', interest_category: @interest_category, url: admin_interest_category_path(@interest_category) %> + <%= render 'enrichment', interest_category: @interest_category %> +
diff --git a/app/views/admin/interest_categories/index.html.erb b/app/views/admin/interest_categories/index.html.erb new file mode 100644 index 000000000..7416abdb6 --- /dev/null +++ b/app/views/admin/interest_categories/index.html.erb @@ -0,0 +1,50 @@ +<% content_for :title, t('.title') %> + +
+
+ <%= t('.rebuild_notice') %> +
+ +
+ <%= link_to t('.new'), new_admin_interest_category_path, class: "c-btn c-btn--blue" %> +
+ +
+ <% header_collection = [{ column: nil, caption: t('interest_categories.position'), options: {} }, + { column: nil, caption: t('interest_categories.code'), options: {} }, + { column: nil, caption: t('interest_categories.name_en'), options: {} }, + { column: nil, caption: t('interest_categories.name_et'), options: {} }, + { column: nil, caption: t('interest_categories.active'), options: {} }, + { column: nil, caption: t('interest_categories.enrichment_status'), options: {} }, + { column: nil, caption: '', options: {} }] %> + <%= component 'common/table', header_collection:, options: { class: 'js-table-dt dataTable no-footer' } do %> + <%= tag.tbody class: 'contents' do %> + <% @interest_categories.each do |category| %> + + <%= category.position %> + <%= category.code %> + <%= category.name_en %> + <%= category.name_et %> + <%= category.active? ? t('interest_categories.yes') : t('interest_categories.no') %> + + <% if category.embedded? %> + <%= t('interest_categories.enriched') %> + <% if category.keywords.present? %> + <%= category.keywords.size %> + <% end %> + <% else %> + <%= t('interest_categories.pending') %> + <% end %> + + + <%= link_to t(:edit), edit_admin_interest_category_path(category), class: "c-btn c-btn--green" %> + <%= link_to t(:delete), admin_interest_category_path(category), + data: { turbo_method: :delete, turbo_confirm: t(:are_you_sure) }, + class: "c-btn c-btn--red" %> + + + <% end %> + <% end %> + <% end %> +
+
diff --git a/app/views/admin/interest_categories/new.html.erb b/app/views/admin/interest_categories/new.html.erb new file mode 100644 index 000000000..3918f402f --- /dev/null +++ b/app/views/admin/interest_categories/new.html.erb @@ -0,0 +1,5 @@ +<% content_for :title, t('.title') %> + +
+ <%= render 'form', interest_category: @interest_category, url: admin_interest_categories_path %> +
diff --git a/app/views/auctions/_auction.html.erb b/app/views/auctions/_auction.html.erb index 573459eab..cc18e8969 100644 --- a/app/views/auctions/_auction.html.erb +++ b/app/views/auctions/_auction.html.erb @@ -1,11 +1,35 @@ - - + + data-recommendation-dwell-auction-uuid-value="<%= auction.uuid %>" + data-recommendation-dwell-source-value="auctions#index" + data-recommendation-dwell-event-type-value="auction_detail_view" + data-recommendation-dwell-dwell-ms-value="1500" + <% end %>> + <% english_auction_presenter = EnglishBidsPresenter.new(auction) %> -

<%= auction.domain_name %>

- - <%= component 'common/auction_type_icon', auction: auction %> + +

<%= auction.domain_name %>

+ + + <% if auction_tags_display_enabled? %> + + <% tags = auction_display_tags(auction, local_assigns.fetch(:classifications, nil)) %> + <% if tags.any? %> +
+ <% tags.each do |tag| %> + <%= tag %> + <% end %> +
+ <% else %> + — + <% end %> + + <% else %> + <%= component 'common/auction_type_icon', auction: auction %> + <% end %> <%= auction.ends_at&.strftime('%d/%m/%Y %H:%M') %> @@ -16,7 +40,7 @@ <%= auction.users_price.zero? ? '' : "#{auction.users_price} €" %> <% end %> - + <% if auction.english? %> <%= english_auction_presenter.bidder_name(auction.currently_winning_offer, user) %> diff --git a/app/views/auctions/index.html.erb b/app/views/auctions/index.html.erb index f9d60598d..1fef2d39a 100644 --- a/app/views/auctions/index.html.erb +++ b/app/views/auctions/index.html.erb @@ -7,6 +7,7 @@ <% end %> <%= component 'pages/auction/cards' %> +<%= render 'recommendation_profiles/prompt_modal' if @show_recommendation_prompt %>
+ <% type_or_tags_header = auction_tags_display_enabled? ? + { column: nil, caption: t('auctions.tags'), options: {} } : + { column: 'platform', caption: t('auctions.auction_type'), options: { class: "sorting" } } %> <% header_collection = [{ column: 'domain_name', caption: t('auctions.domain_name'), options: { class: "sorting", style: "width: 25% !important;" } }, - { column: 'platform', caption: t('auctions.auction_type'), options: { class: "sorting" } }, + type_or_tags_header, { column: 'ends_at', caption: t('auctions.ends_at'), options: { class: "sorting" } }, { column: 'users_price', caption: t('auctions.current_price'), options: { class: "sorting" } }, # { column: 'username', caption: t('auctions.offer_owner'), options: { class: "sorting" } }, # temporary turn of because at the moment we not provde english auction @@ -53,7 +57,12 @@ <%= tag.tbody id: "bids", class: 'contents' do %> <% @auctions.uniq.each do |auction| %> - <%= render partial: 'auction', locals: { auction: auction, user: current_user, updated: false } %> + <%= render partial: 'auction', locals: { + auction: auction, + user: current_user, + updated: false, + classifications: @auction_classifications + } %> <% end %> <% end %> <% end %> diff --git a/app/views/recommendation_profiles/_fields.html.erb b/app/views/recommendation_profiles/_fields.html.erb new file mode 100644 index 000000000..d28a3735a --- /dev/null +++ b/app/views/recommendation_profiles/_fields.html.erb @@ -0,0 +1,54 @@ +
+ + + + + +
diff --git a/app/views/recommendation_profiles/_prompt_modal.html.erb b/app/views/recommendation_profiles/_prompt_modal.html.erb new file mode 100644 index 000000000..291890cf2 --- /dev/null +++ b/app/views/recommendation_profiles/_prompt_modal.html.erb @@ -0,0 +1,38 @@ +
+
+
+
+
+
+
+ <%= t('recommendation_profiles.prompt.title') %> +
+
+ <%= t('recommendation_profiles.prompt.subject') %> +
+
+ <%= t('recommendation_profiles.prompt.description') %> +
+
+ +
+
+
+ <%= component 'common/links/link_button', + link_title: t('recommendation_profiles.prompt.fill_now'), + href: edit_recommendation_profile_path(return_to: request.fullpath), + color: 'green' %> + + <%= button_to t('recommendation_profiles.prompt.remind_later'), + dismiss_recommendation_profile_path, + method: :patch, + class: 'c-btn c-btn--ghost', + params: { return_to: request.fullpath } %> +
+
+
+
+
+
+
+
diff --git a/app/views/recommendation_profiles/edit.html.erb b/app/views/recommendation_profiles/edit.html.erb new file mode 100644 index 000000000..d954355d0 --- /dev/null +++ b/app/views/recommendation_profiles/edit.html.erb @@ -0,0 +1,28 @@ +<% content_for :title, t('.title') %> + +
+ +
diff --git a/app/views/users/_sign_up_form.html.erb b/app/views/users/_sign_up_form.html.erb index 30cfc2bc9..2f97e7730 100644 --- a/app/views/users/_sign_up_form.html.erb +++ b/app/views/users/_sign_up_form.html.erb @@ -76,5 +76,12 @@ <%= component 'common/form/checkboxes/checkbox_with_label', label_title: t('.daily_summary'), form: f, attribute: :daily_summary %>
+ + <% if RecommendationProfile.selection_enabled? %> + <%= f.fields_for :recommendation_profile do |recommendation_form| %> + <%= render 'recommendation_profiles/fields', form: recommendation_form %> + <% end %> + <% end %> +
diff --git a/app/views/users/_user_info.html.erb b/app/views/users/_user_info.html.erb index 4cab097a3..7665f4c77 100644 --- a/app/views/users/_user_info.html.erb +++ b/app/views/users/_user_info.html.erb @@ -76,6 +76,27 @@
<%= t('users.terms_and_conditions_link') %> + + <% if RecommendationProfile.selection_enabled? %> +
+

<%= t('recommendation_profiles.profile.summary_title') %>

+ + <% if @user.recommendation_profile&.summary_lines&.any? %> + <% @user.recommendation_profile.summary_lines.each do |line| %> +

<%= line %>

+ <% end %> + <% else %> +

<%= t('recommendation_profiles.profile.empty_state') %>

+ <% end %> + + <%= component 'common/links/link_button', + link_title: t('recommendation_profiles.profile.edit_action'), + href: edit_recommendation_profile_path(return_to: user_path(@user.uuid)), + color: 'ghost', + options: { target: '_top' } %> +
+ <% end %> +
<%= component 'common/links/link_button', link_title: t(:billing), href: billing_profiles_path, color: 'ghost', options: { target: '_top' } %> <%= component 'common/buttons/delete_button_with_text', path: user_path(@user.uuid), text: t('users.show.delete') %> diff --git a/config/customization.yml.sample b/config/customization.yml.sample index 89192daff..d2b6ad428 100644 --- a/config/customization.yml.sample +++ b/config/customization.yml.sample @@ -21,6 +21,9 @@ default: &default mobile_sms_sent_time_limit_in_minutes: 1 auction_filter_available: false ended_auctions_link_available: false + # Experimental: on the public auction list, replace the "auction type" column + # with the domain's LLM-derived tags. Off (or absent) => auction-type icon. + auction_tags_display_enabled: false mailer: # Host to which links from emails should redirect to diff --git a/config/locales/admin/auctions_classification.en.yml b/config/locales/admin/auctions_classification.en.yml new file mode 100644 index 000000000..324cfcc1b --- /dev/null +++ b/config/locales/admin/auctions_classification.en.yml @@ -0,0 +1,21 @@ +en: + admin: + auctions: + classification: + heading: "AI classification (LLM)" + none: "Not classified" + category: "Category" + description: "Description" + tags: "Tags" + keywords: "Keywords" + audience: "Audience" + languages: "Languages" + use_cases: "Use cases" + brandability: "Brandability" + confidence: "Confidence" + source: "Source" + model: "Model" + classified_at: "Classified at" + embedding: "Embedding" + embedded_yes: "Vector ready" + embedded_no: "No vector" diff --git a/config/locales/admin/auctions_classification.et.yml b/config/locales/admin/auctions_classification.et.yml new file mode 100644 index 000000000..83c965881 --- /dev/null +++ b/config/locales/admin/auctions_classification.et.yml @@ -0,0 +1,21 @@ +et: + admin: + auctions: + classification: + heading: "AI klassifikatsioon (LLM)" + none: "Klassifitseerimata" + category: "Kategooria" + description: "Kirjeldus" + tags: "Sildid" + keywords: "Märksõnad" + audience: "Sihtrühm" + languages: "Keeled" + use_cases: "Kasutusjuhud" + brandability: "Brändituvus" + confidence: "Kindlus" + source: "Allikas" + model: "Mudel" + classified_at: "Klassifitseeritud" + embedding: "Vektor" + embedded_yes: "Vektor olemas" + embedded_no: "Vektorit pole" diff --git a/config/locales/auctions.en.yml b/config/locales/auctions.en.yml index cf74009c7..d07c6fd1e 100644 --- a/config/locales/auctions.en.yml +++ b/config/locales/auctions.en.yml @@ -8,6 +8,7 @@ en: winning_bid: 'Winning bid' highest_price: "Highest price" auction_type: "Auction type" + tags: "Tags" english: "English" blind: "Blind" offers: "Offers" diff --git a/config/locales/auctions.et.yml b/config/locales/auctions.et.yml index 30f5e3863..4990ee344 100644 --- a/config/locales/auctions.et.yml +++ b/config/locales/auctions.et.yml @@ -15,6 +15,7 @@ et: winning_bid: "Võidupakkumus" highest_price: "Kõrgeim hind" auction_type: "Oksjoni tüüp" + tags: "Sildid" english: "Inglise" blind: "Pime" offers: "Pakkumused" diff --git a/config/locales/en.yml b/config/locales/en.yml index 5adebda42..8344ea560 100644 --- a/config/locales/en.yml +++ b/config/locales/en.yml @@ -48,6 +48,7 @@ en: auctions_name: "Auctions" billing_profiles_name: "Billing Profiles" settings_name: "Settings" + interest_categories_name: "Interest categories" users_name: "Users" results_name: "Results" finished_auctions: "Ended auctions" diff --git a/config/locales/et.yml b/config/locales/et.yml index 222211a92..2e1741559 100644 --- a/config/locales/et.yml +++ b/config/locales/et.yml @@ -47,6 +47,7 @@ et: auctions_name: "Oksjonid" billing_profiles_name: "Arve aadressid" settings_name: "Seadistus" + interest_categories_name: "Huvi kategooriad" users_name: "Kasutajad" results_name: "Tulemused" finished_auctions: "Lõppenud oksjonite arved" diff --git a/config/locales/interest_categories.en.yml b/config/locales/interest_categories.en.yml new file mode 100644 index 000000000..afc862e7e --- /dev/null +++ b/config/locales/interest_categories.en.yml @@ -0,0 +1,29 @@ +en: + interest_categories: + code: "Code" + name_en: "Name (English)" + name_et: "Name (Estonian)" + position: "Position" + active: "Active" + yes: "Yes" + no: "No" + enrichment_status: "AI status" + enriched: "Enriched" + pending: "Pending" + enrichment_heading: "AI enrichment (LLM)" + description: "Description" + keywords: "Keywords" + embedding: "Embedding" + embedded_at: "Enriched at" + model: "Model" + not_enriched: "Not enriched yet. Run the recommendation pipeline (rake recommendation:init) or the EnrichInterestCategoriesJob to generate the description, keywords and embedding." + admin: + interest_categories: + index: + title: "Interest categories" + new: "New category" + rebuild_notice: "After adding, renaming, or deleting categories, run the Recommendation::RebuildRecommendationsJob job on the Jobs page to re-classify domains and refresh recommendations under the new categories." + new: + title: "New interest category" + edit: + title: "Edit interest category" diff --git a/config/locales/interest_categories.et.yml b/config/locales/interest_categories.et.yml new file mode 100644 index 000000000..170c7529f --- /dev/null +++ b/config/locales/interest_categories.et.yml @@ -0,0 +1,29 @@ +et: + interest_categories: + code: "Kood" + name_en: "Nimi (inglise)" + name_et: "Nimi (eesti)" + position: "Järjekord" + active: "Aktiivne" + yes: "Jah" + no: "Ei" + enrichment_status: "AI staatus" + enriched: "Rikastatud" + pending: "Ootel" + enrichment_heading: "AI rikastamine (LLM)" + description: "Kirjeldus" + keywords: "Märksõnad" + embedding: "Vektor" + embedded_at: "Rikastatud" + model: "Mudel" + not_enriched: "Veel rikastamata. Käivita soovituste konveier (rake recommendation:init) või EnrichInterestCategoriesJob, et luua kirjeldus, märksõnad ja vektor." + admin: + interest_categories: + index: + title: "Huvi kategooriad" + new: "Uus kategooria" + rebuild_notice: "Pärast kategooriate lisamist, ümbernimetamist või kustutamist käivita Tööde lehel töö Recommendation::RebuildRecommendationsJob, et domeenid uute kategooriate järgi ümber klassifitseerida ja soovitused värskendada." + new: + title: "Uus huvi kategooria" + edit: + title: "Muuda huvi kategooriat" diff --git a/config/locales/recommendation_profiles.en.yml b/config/locales/recommendation_profiles.en.yml new file mode 100644 index 000000000..738a0c18f --- /dev/null +++ b/config/locales/recommendation_profiles.en.yml @@ -0,0 +1,49 @@ +en: + recommendation_profiles: + edit: + title: "Auction interests" + why_it_helps_title: "Why this helps" + why_it_helps_body: "Choose what kinds of domain names interest you. We combine these interests with your wishlist, bids, and domain tags to rank more relevant auctions higher for you." + submit: "Save interests" + update: + updated: "Your auction interests were updated." + skipped: "You can add your auction interests later." + dismiss: + dismissed: "We will remind you later." + form: + title: "Auction interests" + subtitle: "Optional. Choose the kinds of domain names that interest you. We will use these categories together with your bids and wishlist." + interest_categories: "Interest categories" + other_interests: "Add your own keywords" + other_interests_placeholder: "Type an interest and press Enter" + other_interests_hint: "Add any interests that are not in the list above." + prompt: + title: "Personalization" + subject: "Tell us what domains interest you" + description: "Add a few preferences and we will raise more relevant auctions in your list. This step is optional." + fill_now: "Set interests" + remind_later: "Remind me later" + profile: + summary_title: "Auction interests" + empty_state: "No interests saved yet." + edit_action: "Edit interests" + summary: + categories: "Interests" + custom_interests: "Custom interests" + length: "Preferred length" + categories: + brandable: "Brandable names" + shop_brand: "Shop or store names" + saas: "SaaS and software" + b2b_service: "B2B services" + local_service: "Local or service businesses" + media_content: "Media and content" + finance: "Finance and fintech" + legal: "Legal and professional services" + health: "Health and wellness" + education: "Education and courses" + travel: "Travel and tourism" + automotive: "Automotive" + real_estate: "Real estate" + numeric: "Numeric domains" + other: "Other" diff --git a/config/locales/recommendation_profiles.et.yml b/config/locales/recommendation_profiles.et.yml new file mode 100644 index 000000000..23bb764ac --- /dev/null +++ b/config/locales/recommendation_profiles.et.yml @@ -0,0 +1,49 @@ +et: + recommendation_profiles: + edit: + title: "Oksjoni huvid" + why_it_helps_title: "Miks see kasulik on" + why_it_helps_body: "Vali, millist tüüpi domeeninimed sind huvitavad. Kombineerime need huvid sinu wishlisti, pakkumiste ja domeeni siltidega, et tõsta sobivamad oksjonid nimekirjas kõrgemale." + submit: "Salvesta huvid" + update: + updated: "Sinu oksjoni huvid on uuendatud." + skipped: "Saad oma oksjoni huvid hiljem lisada." + dismiss: + dismissed: "Tuletame seda hiljem meelde." + form: + title: "Oksjoni huvid" + subtitle: "Valikuline. Vali, millised domeeninimede tüübid sind huvitavad. Kasutame neid kategooriaid koos pakkumiste ja wishlistiga." + interest_categories: "Huvi kategooriad" + other_interests: "Lisa oma märksõnad" + other_interests_placeholder: "Sisesta huvi ja vajuta Enter" + other_interests_hint: "Lisa huvid, mida ülal olevas nimekirjas ei ole." + prompt: + title: "Personaliseerimine" + subject: "Anna teada, millised domeenid sind huvitavad" + description: "Lisa mõned eelistused ja tõstame sulle sobivamad oksjonid nimekirjas kõrgemale. See samm on valikuline." + fill_now: "Määra huvid" + remind_later: "Tuleta hiljem meelde" + profile: + summary_title: "Oksjoni huvid" + empty_state: "Huvid pole veel salvestatud." + edit_action: "Muuda huve" + summary: + categories: "Huvid" + custom_interests: "Kohandatud huvid" + length: "Eelistatud pikkus" + categories: + brandable: "Bränditavad nimed" + shop_brand: "Poe- või kaubamärgi nimed" + saas: "SaaS ja tarkvara" + b2b_service: "B2B teenused" + local_service: "Kohalikud ja teenusettevõtted" + media_content: "Meedia ja sisu" + finance: "Finants ja fintech" + legal: "Õigus- ja professionaalsed teenused" + health: "Tervis ja heaolu" + education: "Haridus ja kursused" + travel: "Reisimine ja turism" + automotive: "Autondus" + real_estate: "Kinnisvara" + numeric: "Numbrilised domeenid" + other: "Muu" diff --git a/config/routes.rb b/config/routes.rb index b27a451cc..46fb72e2e 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -69,6 +69,7 @@ resources :results, only: %i[index create show], concerns: %i[auditable] resources :settings, except: %i[create destroy], concerns: [:auditable] + resources :interest_categories resources :users, concerns: %i[auditable] end @@ -145,6 +146,10 @@ resources :wishlist_items, param: :uuid, only: %i[index edit create destroy update] resources :autobider, param: :uuid, only: [:create, :update, :edit, :new] + resource :recommendation_profile, only: %i[edit update] + patch 'recommendation_profile/dismiss', to: 'recommendation_profiles#dismiss', + as: :dismiss_recommendation_profile + resources :recommendation_events, only: :create mount OkComputer::Engine, at: '/healthcheck', as: :healthcheck mount LetterOpenerWeb::Engine, at: "/letter_opener" if Rails.env.development? diff --git a/db/migrate/20260525095500_create_recommendation_tables.rb b/db/migrate/20260525095500_create_recommendation_tables.rb new file mode 100644 index 000000000..fbca74bdf --- /dev/null +++ b/db/migrate/20260525095500_create_recommendation_tables.rb @@ -0,0 +1,67 @@ +class CreateRecommendationTables < ActiveRecord::Migration[7.0] + def change + create_table :recommendation_profiles do |t| + t.references :user, null: false, foreign_key: true, index: { unique: true } + t.uuid :uuid, default: 'gen_random_uuid()' + t.string :preferred_tlds, array: true, default: [], null: false + t.string :interest_keywords, array: true, default: [], null: false + t.string :preferred_platforms, array: true, default: [], null: false + t.integer :preferred_length_min + t.integer :preferred_length_max + t.integer :budget_min_cents + t.integer :budget_max_cents + t.boolean :allow_numbers + t.boolean :allow_hyphens + t.datetime :completed_at + t.datetime :prompt_dismissed_at + t.datetime :last_prompted_at + t.integer :prompt_shown_count, default: 0, null: false + + t.timestamps + end + + add_index :recommendation_profiles, :uuid, unique: true + add_index :recommendation_profiles, :preferred_tlds, using: :gin + add_index :recommendation_profiles, :interest_keywords, using: :gin + add_index :recommendation_profiles, :preferred_platforms, using: :gin + + create_table :recommendation_events do |t| + t.references :user, foreign_key: true + t.references :auction, foreign_key: true + t.uuid :uuid, default: 'gen_random_uuid()' + t.string :event_type, null: false + t.string :source + t.string :session_id + t.string :request_id + t.datetime :occurred_at, null: false + t.jsonb :properties, default: {}, null: false + + t.timestamps + end + + add_index :recommendation_events, :uuid, unique: true + add_index :recommendation_events, :event_type + add_index :recommendation_events, :occurred_at + add_index :recommendation_events, %i[user_id event_type occurred_at], name: 'idx_rec_events_user_type_time' + add_index :recommendation_events, :properties, using: :gin + + # `scorer_name` (not `model_name`) — Rails 8.1 reserves `model_name` + # as a class method on ActiveRecord, and using it as a column raises + # ActiveRecord::DangerousAttributeError when records are loaded. + create_table :user_auction_scores do |t| + t.references :user, null: false, foreign_key: true + t.references :auction, null: false, foreign_key: true + t.uuid :uuid, default: 'gen_random_uuid()' + t.decimal :score, precision: 10, scale: 6, null: false + t.string :scorer_name + t.string :features_version + t.datetime :calculated_at, null: false + + t.timestamps + end + + add_index :user_auction_scores, :uuid, unique: true + add_index :user_auction_scores, %i[user_id auction_id], unique: true + add_index :user_auction_scores, %i[user_id score] + end +end diff --git a/db/migrate/20260525115000_add_classification_fields_to_auctions.rb b/db/migrate/20260525115000_add_classification_fields_to_auctions.rb new file mode 100644 index 000000000..e06463113 --- /dev/null +++ b/db/migrate/20260525115000_add_classification_fields_to_auctions.rb @@ -0,0 +1,12 @@ +class AddClassificationFieldsToAuctions < ActiveRecord::Migration[7.0] + def change + add_column :auctions, :classification_tags, :string, array: true, default: [], null: false + add_column :auctions, :primary_category, :string + add_column :auctions, :classification_source, :string + add_column :auctions, :classification_model, :string + add_column :auctions, :classified_at, :datetime + + add_index :auctions, :classification_tags, using: :gin + add_index :auctions, :primary_category + end +end diff --git a/db/migrate/20260527090000_create_domain_classifications.rb b/db/migrate/20260527090000_create_domain_classifications.rb new file mode 100644 index 000000000..a047ccdc0 --- /dev/null +++ b/db/migrate/20260527090000_create_domain_classifications.rb @@ -0,0 +1,47 @@ +class CreateDomainClassifications < ActiveRecord::Migration[7.0] + # Per-domain semantic classification. Single source of truth for what a + # domain *means*, decoupled from any specific auction. See ADR-001. + # + # Embeddings are stored as a plain Postgres array of doubles — no + # pgvector — so this migration runs on the shared dev image and on + # production RDS without extension setup. Cosine similarity is + # computed in Ruby at scoring time. + def change + create_table :domain_classifications do |t| + t.string :domain_name, null: false + t.uuid :uuid, default: 'gen_random_uuid()', null: false + + t.string :primary_category + t.string :tags, array: true, default: [], null: false + t.string :keywords, array: true, default: [], null: false + + t.string :audience + t.string :languages, array: true, default: [], null: false + t.string :suggested_use_cases, array: true, default: [], null: false + + t.decimal :brandability_score, precision: 4, scale: 3 + + t.string :classification_source # 'heuristic' | 'openai' | 'manual' | 'imported' + t.string :classification_model + t.decimal :confidence, precision: 4, scale: 3 + t.datetime :classified_at + t.jsonb :raw_llm_response, default: {}, null: false + + t.column :embedding, :"double precision[]" + t.string :embedding_model + t.datetime :embedded_at + + t.timestamps + end + + add_index :domain_classifications, :domain_name, unique: true + add_index :domain_classifications, :uuid, unique: true + add_index :domain_classifications, :tags, using: :gin + add_index :domain_classifications, :keywords, using: :gin + add_index :domain_classifications, :primary_category + add_index :domain_classifications, :audience + add_index :domain_classifications, :classified_at + add_index :domain_classifications, :classification_source + add_index :domain_classifications, :embedded_at + end +end diff --git a/db/migrate/20260601100000_create_interest_categories.rb b/db/migrate/20260601100000_create_interest_categories.rb new file mode 100644 index 000000000..9c25e36ab --- /dev/null +++ b/db/migrate/20260601100000_create_interest_categories.rb @@ -0,0 +1,18 @@ +class CreateInterestCategories < ActiveRecord::Migration[7.0] + def change + create_table :interest_categories do |t| + t.uuid :uuid, default: 'gen_random_uuid()', null: false + t.string :code, null: false + t.string :name_en, null: false + t.string :name_et, null: false + t.integer :position, default: 0, null: false + t.boolean :active, default: true, null: false + + t.timestamps + end + + add_index :interest_categories, :code, unique: true + add_index :interest_categories, :uuid, unique: true + add_index :interest_categories, %i[active position] + end +end diff --git a/db/migrate/20260601100100_seed_interest_categories_and_setting.rb b/db/migrate/20260601100100_seed_interest_categories_and_setting.rb new file mode 100644 index 000000000..95c26f7ca --- /dev/null +++ b/db/migrate/20260601100100_seed_interest_categories_and_setting.rb @@ -0,0 +1,23 @@ +class SeedInterestCategoriesAndSetting < ActiveRecord::Migration[7.0] + # Data migration so existing environments get the categories + toggle on + # deploy (db:migrate). Fresh setups get the same via db/seeds.rb. Both are + # idempotent. Schema-load (structure.sql) skips this — that path runs seeds. + def up + unless Setting.exists?(code: 'recommendation_interests_enabled') + Setting.create( + code: 'recommendation_interests_enabled', + value: 'false', + value_format: 'boolean', + description: "When 'true', users see the auction-interests selection " \ + "(prompt + form). Keep 'false' until interest categories are populated in admin." + ) + end + + InterestCategory.seed_defaults! + end + + def down + Setting.where(code: 'recommendation_interests_enabled').delete_all + InterestCategory.where(code: InterestCategory::DEFAULTS.map { |c| c[:code] }).delete_all + end +end diff --git a/db/migrate/20260708120000_add_lower_domain_name_index_to_domain_classifications.rb b/db/migrate/20260708120000_add_lower_domain_name_index_to_domain_classifications.rb new file mode 100644 index 000000000..fc82b0416 --- /dev/null +++ b/db/migrate/20260708120000_add_lower_domain_name_index_to_domain_classifications.rb @@ -0,0 +1,10 @@ +class AddLowerDomainNameIndexToDomainClassifications < ActiveRecord::Migration[8.1] + # The /auctions user-sorting join matches on LOWER(domain_classifications.domain_name) + # (see Auction::UserSortable). The plain btree on domain_name can't serve a + # LOWER() predicate, so every logged-in /auctions load seq-scanned this table. + # A functional index on LOWER(domain_name) restores index usage on that join. + def change + add_index :domain_classifications, 'LOWER(domain_name)', + name: 'index_domain_classifications_on_lower_domain_name' + end +end diff --git a/db/migrate/20260708120100_add_user_id_index_to_wishlist_items.rb b/db/migrate/20260708120100_add_user_id_index_to_wishlist_items.rb new file mode 100644 index 000000000..254c1e9ef --- /dev/null +++ b/db/migrate/20260708120100_add_user_id_index_to_wishlist_items.rb @@ -0,0 +1,8 @@ +class AddUserIdIndexToWishlistItems < ActiveRecord::Migration[8.1] + # wishlist_items(user_id) was unindexed. The recommendation hot paths query a + # user's wishlist on every /auctions load (Auction::UserSortable) and during + # per-user scoring (Recommendation::Scorer), so index the lookup column. + def change + add_index :wishlist_items, :user_id + end +end diff --git a/db/migrate/20260709100000_add_embedding_fields_to_interest_categories.rb b/db/migrate/20260709100000_add_embedding_fields_to_interest_categories.rb new file mode 100644 index 000000000..ba89ad51c --- /dev/null +++ b/db/migrate/20260709100000_add_embedding_fields_to_interest_categories.rb @@ -0,0 +1,16 @@ +class AddEmbeddingFieldsToInterestCategories < ActiveRecord::Migration[7.0] + # v3 unified embedding space: an interest category becomes "like a domain". + # We enrich it with an LLM description + keywords, then embed that text into + # the same 1536-dim space as domains so a selected category can act as a + # magnet the same way a bid/wishlist domain does. + # See docs/planning/recommendation-v3-unified-embedding-plan.md. + def change + add_column :interest_categories, :description, :text + add_column :interest_categories, :keywords, :string, array: true, default: [], null: false + add_column :interest_categories, :embedding, :float, array: true + add_column :interest_categories, :embedding_model, :string + add_column :interest_categories, :embedded_at, :datetime + + add_index :interest_categories, :embedded_at + end +end diff --git a/db/migrate/20260709100100_add_custom_interest_vectors_to_recommendation_profiles.rb b/db/migrate/20260709100100_add_custom_interest_vectors_to_recommendation_profiles.rb new file mode 100644 index 000000000..75a50b72a --- /dev/null +++ b/db/migrate/20260709100100_add_custom_interest_vectors_to_recommendation_profiles.rb @@ -0,0 +1,12 @@ +class AddCustomInterestVectorsToRecommendationProfiles < ActiveRecord::Migration[7.0] + # v3 unified embedding space: each free-text custom interest is embedded into + # its own vector (a separate "magnet"), not string-matched via LIKE. Stored + # as jsonb [{ "text" => ..., "embedding" => [..1536..] }, ...] because a + # profile can hold several heterogeneous custom interests and averaging them + # into one vector would blur them. + # See docs/planning/recommendation-v3-unified-embedding-plan.md. + def change + add_column :recommendation_profiles, :custom_interest_vectors, :jsonb, default: [], null: false + add_column :recommendation_profiles, :custom_interests_embedded_at, :datetime + end +end diff --git a/db/migrate/20260710100000_add_embedding_enrichment_to_domain_classifications.rb b/db/migrate/20260710100000_add_embedding_enrichment_to_domain_classifications.rb new file mode 100644 index 000000000..972e517bc --- /dev/null +++ b/db/migrate/20260710100000_add_embedding_enrichment_to_domain_classifications.rb @@ -0,0 +1,15 @@ +class AddEmbeddingEnrichmentToDomainClassifications < ActiveRecord::Migration[7.0] + # v3 embedding enrichment (see docs/technical/recommendation-embedding-enrichment.md): + # + # - description: a 1-2 sentence LLM summary of what the domain is for. Feeds the + # embedding input so semantically-similar domains cluster (previously the + # input was only ". ", a deliberately sparse signal — + # ADR-001 — that under-served interest/wishlist matching). + # - embedding_input_version: which DomainEmbedder input format produced the + # stored vector. Lets DomainClassification.needs_embedding re-embed rows built + # under an older format without a full LLM re-classification. + def change + add_column :domain_classifications, :description, :text + add_column :domain_classifications, :embedding_input_version, :integer + end +end diff --git a/db/seeds.rb b/db/seeds.rb index 775286d8d..522ae53b6 100644 --- a/db/seeds.rb +++ b/db/seeds.rb @@ -490,7 +490,7 @@ openai_model_description = <<~TEXT.squish OpenAI API model TEXT -openai_model_value = 'gpt-3.5-turbo' +openai_model_value = 'gpt-5' openai_model_setting = Setting.new(code: :openai_model, value: openai_model_value, description: openai_model_description, @@ -540,3 +540,18 @@ description: contact_organization_email_description, value_format: 'string') contact_organization_email_setting.save + +# Recommendation interests: admin toggle that controls whether the +# interest-selection UI is shown to users. Default OFF so a fresh deploy +# never shows an empty picker; turn it on once categories are filled. +recommendation_interests_enabled_description = <<~TEXT.squish + When 'true', users see the auction-interests selection (prompt + form). + Keep 'false' until interest categories are populated in admin. +TEXT +Setting.new(code: :recommendation_interests_enabled, + value: 'false', + description: recommendation_interests_enabled_description, + value_format: 'boolean').save + +# Recommendation interest categories (idempotent — see InterestCategory.seed_defaults!). +InterestCategory.seed_defaults! diff --git a/db/structure.sql b/db/structure.sql index 50070abff..aca63b83d 100644 --- a/db/structure.sql +++ b/db/structure.sql @@ -889,6 +889,11 @@ CREATE TABLE public.auctions ( enable_deposit boolean DEFAULT false NOT NULL, requirement_deposit_in_cents integer, ai_score integer DEFAULT 0, + classification_tags character varying[] DEFAULT '{}'::character varying[] NOT NULL, + primary_category character varying, + classification_source character varying, + classification_model character varying, + classified_at timestamp(6) without time zone, CONSTRAINT starts_at_earlier_than_ends_at CHECK ((starts_at < ends_at)) ); @@ -1177,6 +1182,59 @@ CREATE SEQUENCE public.directo_customers_id_seq ALTER SEQUENCE public.directo_customers_id_seq OWNED BY public.directo_customers.id; +-- +-- Name: domain_classifications; Type: TABLE; Schema: public; Owner: - +-- + +CREATE TABLE public.domain_classifications ( + id bigint NOT NULL, + domain_name character varying NOT NULL, + uuid uuid DEFAULT gen_random_uuid() NOT NULL, + primary_category character varying, + tags character varying[] DEFAULT '{}'::character varying[] NOT NULL, + keywords character varying[] DEFAULT '{}'::character varying[] NOT NULL, + audience character varying, + languages character varying[] DEFAULT '{}'::character varying[] NOT NULL, + suggested_use_cases character varying[] DEFAULT '{}'::character varying[] NOT NULL, + has_digits boolean DEFAULT false NOT NULL, + has_hyphens boolean DEFAULT false NOT NULL, + token_count integer, + dictionary_word boolean DEFAULT false NOT NULL, + brandability_score numeric(4,3), + classification_source character varying, + classification_model character varying, + confidence numeric(4,3), + classified_at timestamp(6) without time zone, + raw_llm_response jsonb DEFAULT '{}'::jsonb NOT NULL, + created_at timestamp(6) without time zone NOT NULL, + updated_at timestamp(6) without time zone NOT NULL, + embedding double precision[], + embedding_model character varying, + embedded_at timestamp(6) without time zone, + description text, + embedding_input_version integer +); + + +-- +-- Name: domain_classifications_id_seq; Type: SEQUENCE; Schema: public; Owner: - +-- + +CREATE SEQUENCE public.domain_classifications_id_seq + START WITH 1 + INCREMENT BY 1 + NO MINVALUE + NO MAXVALUE + CACHE 1; + + +-- +-- Name: domain_classifications_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: - +-- + +ALTER SEQUENCE public.domain_classifications_id_seq OWNED BY public.domain_classifications.id; + + -- -- Name: domain_participate_auctions; Type: TABLE; Schema: public; Owner: - -- @@ -1212,6 +1270,47 @@ CREATE SEQUENCE public.domain_participate_auctions_id_seq ALTER SEQUENCE public.domain_participate_auctions_id_seq OWNED BY public.domain_participate_auctions.id; +-- +-- Name: interest_categories; Type: TABLE; Schema: public; Owner: - +-- + +CREATE TABLE public.interest_categories ( + id bigint NOT NULL, + uuid uuid DEFAULT gen_random_uuid() NOT NULL, + code character varying NOT NULL, + name_en character varying NOT NULL, + name_et character varying NOT NULL, + "position" integer DEFAULT 0 NOT NULL, + active boolean DEFAULT true NOT NULL, + created_at timestamp(6) without time zone NOT NULL, + updated_at timestamp(6) without time zone NOT NULL, + description text, + keywords character varying[] DEFAULT '{}'::character varying[] NOT NULL, + embedding double precision[], + embedding_model character varying, + embedded_at timestamp(6) without time zone +); + + +-- +-- Name: interest_categories_id_seq; Type: SEQUENCE; Schema: public; Owner: - +-- + +CREATE SEQUENCE public.interest_categories_id_seq + START WITH 1 + INCREMENT BY 1 + NO MINVALUE + NO MAXVALUE + CACHE 1; + + +-- +-- Name: interest_categories_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: - +-- + +ALTER SEQUENCE public.interest_categories_id_seq OWNED BY public.interest_categories.id; + + -- -- Name: invoice_items; Type: TABLE; Schema: public; Owner: - -- @@ -1470,6 +1569,92 @@ CREATE SEQUENCE public.payment_orders_id_seq ALTER SEQUENCE public.payment_orders_id_seq OWNED BY public.payment_orders.id; +-- +-- Name: recommendation_events; Type: TABLE; Schema: public; Owner: - +-- + +CREATE TABLE public.recommendation_events ( + id bigint NOT NULL, + user_id bigint, + auction_id bigint, + uuid uuid DEFAULT gen_random_uuid(), + event_type character varying NOT NULL, + source character varying, + session_id character varying, + request_id character varying, + occurred_at timestamp(6) without time zone NOT NULL, + properties jsonb DEFAULT '{}'::jsonb NOT NULL, + created_at timestamp(6) without time zone NOT NULL, + updated_at timestamp(6) without time zone NOT NULL +); + + +-- +-- Name: recommendation_events_id_seq; Type: SEQUENCE; Schema: public; Owner: - +-- + +CREATE SEQUENCE public.recommendation_events_id_seq + START WITH 1 + INCREMENT BY 1 + NO MINVALUE + NO MAXVALUE + CACHE 1; + + +-- +-- Name: recommendation_events_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: - +-- + +ALTER SEQUENCE public.recommendation_events_id_seq OWNED BY public.recommendation_events.id; + + +-- +-- Name: recommendation_profiles; Type: TABLE; Schema: public; Owner: - +-- + +CREATE TABLE public.recommendation_profiles ( + id bigint NOT NULL, + user_id bigint NOT NULL, + uuid uuid DEFAULT gen_random_uuid(), + preferred_tlds character varying[] DEFAULT '{}'::character varying[] NOT NULL, + interest_keywords character varying[] DEFAULT '{}'::character varying[] NOT NULL, + preferred_platforms character varying[] DEFAULT '{}'::character varying[] NOT NULL, + preferred_length_min integer, + preferred_length_max integer, + budget_min_cents integer, + budget_max_cents integer, + allow_numbers boolean, + allow_hyphens boolean, + completed_at timestamp(6) without time zone, + prompt_dismissed_at timestamp(6) without time zone, + last_prompted_at timestamp(6) without time zone, + prompt_shown_count integer DEFAULT 0 NOT NULL, + created_at timestamp(6) without time zone NOT NULL, + updated_at timestamp(6) without time zone NOT NULL, + custom_interest_vectors jsonb DEFAULT '[]'::jsonb NOT NULL, + custom_interests_embedded_at timestamp(6) without time zone +); + + +-- +-- Name: recommendation_profiles_id_seq; Type: SEQUENCE; Schema: public; Owner: - +-- + +CREATE SEQUENCE public.recommendation_profiles_id_seq + START WITH 1 + INCREMENT BY 1 + NO MINVALUE + NO MAXVALUE + CACHE 1; + + +-- +-- Name: recommendation_profiles_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: - +-- + +ALTER SEQUENCE public.recommendation_profiles_id_seq OWNED BY public.recommendation_profiles.id; + + -- -- Name: remote_view_partials; Type: TABLE; Schema: public; Owner: - -- @@ -1599,6 +1784,43 @@ CREATE MATERIALIZED VIEW public.statistics_report_invoices AS WITH NO DATA; +-- +-- Name: user_auction_scores; Type: TABLE; Schema: public; Owner: - +-- + +CREATE TABLE public.user_auction_scores ( + id bigint NOT NULL, + user_id bigint NOT NULL, + auction_id bigint NOT NULL, + uuid uuid DEFAULT gen_random_uuid(), + score numeric(10,6) NOT NULL, + scorer_name character varying, + features_version character varying, + calculated_at timestamp(6) without time zone NOT NULL, + created_at timestamp(6) without time zone NOT NULL, + updated_at timestamp(6) without time zone NOT NULL +); + + +-- +-- Name: user_auction_scores_id_seq; Type: SEQUENCE; Schema: public; Owner: - +-- + +CREATE SEQUENCE public.user_auction_scores_id_seq + START WITH 1 + INCREMENT BY 1 + NO MINVALUE + NO MAXVALUE + CACHE 1; + + +-- +-- Name: user_auction_scores_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: - +-- + +ALTER SEQUENCE public.user_auction_scores_id_seq OWNED BY public.user_auction_scores.id; + + -- -- Name: users; Type: TABLE; Schema: public; Owner: - -- @@ -1866,6 +2088,13 @@ ALTER TABLE ONLY public.delayed_jobs ALTER COLUMN id SET DEFAULT nextval('public ALTER TABLE ONLY public.directo_customers ALTER COLUMN id SET DEFAULT nextval('public.directo_customers_id_seq'::regclass); +-- +-- Name: domain_classifications id; Type: DEFAULT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.domain_classifications ALTER COLUMN id SET DEFAULT nextval('public.domain_classifications_id_seq'::regclass); + + -- -- Name: domain_participate_auctions id; Type: DEFAULT; Schema: public; Owner: - -- @@ -1873,6 +2102,13 @@ ALTER TABLE ONLY public.directo_customers ALTER COLUMN id SET DEFAULT nextval('p ALTER TABLE ONLY public.domain_participate_auctions ALTER COLUMN id SET DEFAULT nextval('public.domain_participate_auctions_id_seq'::regclass); +-- +-- Name: interest_categories id; Type: DEFAULT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.interest_categories ALTER COLUMN id SET DEFAULT nextval('public.interest_categories_id_seq'::regclass); + + -- -- Name: invoice_items id; Type: DEFAULT; Schema: public; Owner: - -- @@ -1922,6 +2158,20 @@ ALTER TABLE ONLY public.offers ALTER COLUMN id SET DEFAULT nextval('public.offer ALTER TABLE ONLY public.payment_orders ALTER COLUMN id SET DEFAULT nextval('public.payment_orders_id_seq'::regclass); +-- +-- Name: recommendation_events id; Type: DEFAULT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.recommendation_events ALTER COLUMN id SET DEFAULT nextval('public.recommendation_events_id_seq'::regclass); + + +-- +-- Name: recommendation_profiles id; Type: DEFAULT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.recommendation_profiles ALTER COLUMN id SET DEFAULT nextval('public.recommendation_profiles_id_seq'::regclass); + + -- -- Name: remote_view_partials id; Type: DEFAULT; Schema: public; Owner: - -- @@ -1943,6 +2193,13 @@ ALTER TABLE ONLY public.results ALTER COLUMN id SET DEFAULT nextval('public.resu ALTER TABLE ONLY public.settings ALTER COLUMN id SET DEFAULT nextval('public.settings_id_seq'::regclass); +-- +-- Name: user_auction_scores id; Type: DEFAULT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.user_auction_scores ALTER COLUMN id SET DEFAULT nextval('public.user_auction_scores_id_seq'::regclass); + + -- -- Name: users id; Type: DEFAULT; Schema: public; Owner: - -- @@ -2196,6 +2453,14 @@ ALTER TABLE ONLY public.directo_customers ADD CONSTRAINT directo_customers_pkey PRIMARY KEY (id); +-- +-- Name: domain_classifications domain_classifications_pkey; Type: CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.domain_classifications + ADD CONSTRAINT domain_classifications_pkey PRIMARY KEY (id); + + -- -- Name: domain_participate_auctions domain_participate_auctions_pkey; Type: CONSTRAINT; Schema: public; Owner: - -- @@ -2204,6 +2469,14 @@ ALTER TABLE ONLY public.domain_participate_auctions ADD CONSTRAINT domain_participate_auctions_pkey PRIMARY KEY (id); +-- +-- Name: interest_categories interest_categories_pkey; Type: CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.interest_categories + ADD CONSTRAINT interest_categories_pkey PRIMARY KEY (id); + + -- -- Name: invoice_items invoice_items_pkey; Type: CONSTRAINT; Schema: public; Owner: - -- @@ -2252,6 +2525,22 @@ ALTER TABLE ONLY public.payment_orders ADD CONSTRAINT payment_orders_pkey PRIMARY KEY (id); +-- +-- Name: recommendation_events recommendation_events_pkey; Type: CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.recommendation_events + ADD CONSTRAINT recommendation_events_pkey PRIMARY KEY (id); + + +-- +-- Name: recommendation_profiles recommendation_profiles_pkey; Type: CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.recommendation_profiles + ADD CONSTRAINT recommendation_profiles_pkey PRIMARY KEY (id); + + -- -- Name: remote_view_partials remote_view_partials_pkey; Type: CONSTRAINT; Schema: public; Owner: - -- @@ -2284,6 +2573,14 @@ ALTER TABLE ONLY public.auctions ADD CONSTRAINT unique_domain_name_per_auction_duration EXCLUDE USING gist (domain_name WITH =, tsrange(starts_at, ends_at, '[]'::text) WITH &&); +-- +-- Name: user_auction_scores user_auction_scores_pkey; Type: CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.user_auction_scores + ADD CONSTRAINT user_auction_scores_pkey PRIMARY KEY (id); + + -- -- Name: users users_pkey; Type: CONSTRAINT; Schema: public; Owner: - -- @@ -2483,6 +2780,20 @@ CREATE INDEX wishlist_items_recorded_at_idx ON audit.wishlist_items USING btree CREATE INDEX delayed_jobs_priority ON public.delayed_jobs USING btree (priority, run_at); +-- +-- Name: idx_rec_events_user_type_time; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX idx_rec_events_user_type_time ON public.recommendation_events USING btree (user_id, event_type, occurred_at); + + +-- +-- Name: index_auctions_on_classification_tags; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_auctions_on_classification_tags ON public.auctions USING gin (classification_tags); + + -- -- Name: index_auctions_on_domain_name; Type: INDEX; Schema: public; Owner: - -- @@ -2490,6 +2801,13 @@ CREATE INDEX delayed_jobs_priority ON public.delayed_jobs USING btree (priority, CREATE INDEX index_auctions_on_domain_name ON public.auctions USING btree (domain_name); +-- +-- Name: index_auctions_on_primary_category; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_auctions_on_primary_category ON public.auctions USING btree (primary_category); + + -- -- Name: index_auctions_on_remote_id; Type: INDEX; Schema: public; Owner: - -- @@ -2581,6 +2899,76 @@ CREATE UNIQUE INDEX index_directo_customers_on_customer_code ON public.directo_c CREATE UNIQUE INDEX index_directo_customers_on_vat_number ON public.directo_customers USING btree (vat_number); +-- +-- Name: index_domain_classifications_on_audience; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_domain_classifications_on_audience ON public.domain_classifications USING btree (audience); + + +-- +-- Name: index_domain_classifications_on_classification_source; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_domain_classifications_on_classification_source ON public.domain_classifications USING btree (classification_source); + + +-- +-- Name: index_domain_classifications_on_classified_at; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_domain_classifications_on_classified_at ON public.domain_classifications USING btree (classified_at); + + +-- +-- Name: index_domain_classifications_on_domain_name; Type: INDEX; Schema: public; Owner: - +-- + +CREATE UNIQUE INDEX index_domain_classifications_on_domain_name ON public.domain_classifications USING btree (domain_name); + + +-- +-- Name: index_domain_classifications_on_embedded_at; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_domain_classifications_on_embedded_at ON public.domain_classifications USING btree (embedded_at); + + +-- +-- Name: index_domain_classifications_on_keywords; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_domain_classifications_on_keywords ON public.domain_classifications USING gin (keywords); + + +-- +-- Name: index_domain_classifications_on_lower_domain_name; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_domain_classifications_on_lower_domain_name ON public.domain_classifications USING btree (lower((domain_name)::text)); + + +-- +-- Name: index_domain_classifications_on_primary_category; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_domain_classifications_on_primary_category ON public.domain_classifications USING btree (primary_category); + + +-- +-- Name: index_domain_classifications_on_tags; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_domain_classifications_on_tags ON public.domain_classifications USING gin (tags); + + +-- +-- Name: index_domain_classifications_on_uuid; Type: INDEX; Schema: public; Owner: - +-- + +CREATE UNIQUE INDEX index_domain_classifications_on_uuid ON public.domain_classifications USING btree (uuid); + + -- -- Name: index_domain_participate_auctions_on_auction_id; Type: INDEX; Schema: public; Owner: - -- @@ -2595,6 +2983,34 @@ CREATE INDEX index_domain_participate_auctions_on_auction_id ON public.domain_pa CREATE INDEX index_domain_participate_auctions_on_user_id ON public.domain_participate_auctions USING btree (user_id); +-- +-- Name: index_interest_categories_on_active_and_position; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_interest_categories_on_active_and_position ON public.interest_categories USING btree (active, "position"); + + +-- +-- Name: index_interest_categories_on_code; Type: INDEX; Schema: public; Owner: - +-- + +CREATE UNIQUE INDEX index_interest_categories_on_code ON public.interest_categories USING btree (code); + + +-- +-- Name: index_interest_categories_on_embedded_at; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_interest_categories_on_embedded_at ON public.interest_categories USING btree (embedded_at); + + +-- +-- Name: index_interest_categories_on_uuid; Type: INDEX; Schema: public; Owner: - +-- + +CREATE UNIQUE INDEX index_interest_categories_on_uuid ON public.interest_categories USING btree (uuid); + + -- -- Name: index_invoice_items_on_invoice_id; Type: INDEX; Schema: public; Owner: - -- @@ -2714,6 +3130,83 @@ CREATE INDEX index_payment_orders_on_user_id ON public.payment_orders USING btre CREATE UNIQUE INDEX index_payment_orders_on_uuid ON public.payment_orders USING btree (uuid); +-- +-- Name: index_recommendation_events_on_auction_id; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_recommendation_events_on_auction_id ON public.recommendation_events USING btree (auction_id); + + +-- +-- Name: index_recommendation_events_on_event_type; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_recommendation_events_on_event_type ON public.recommendation_events USING btree (event_type); + + +-- +-- Name: index_recommendation_events_on_occurred_at; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_recommendation_events_on_occurred_at ON public.recommendation_events USING btree (occurred_at); + + +-- +-- Name: index_recommendation_events_on_properties; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_recommendation_events_on_properties ON public.recommendation_events USING gin (properties); + + +-- +-- Name: index_recommendation_events_on_user_id; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_recommendation_events_on_user_id ON public.recommendation_events USING btree (user_id); + + +-- +-- Name: index_recommendation_events_on_uuid; Type: INDEX; Schema: public; Owner: - +-- + +CREATE UNIQUE INDEX index_recommendation_events_on_uuid ON public.recommendation_events USING btree (uuid); + + +-- +-- Name: index_recommendation_profiles_on_interest_keywords; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_recommendation_profiles_on_interest_keywords ON public.recommendation_profiles USING gin (interest_keywords); + + +-- +-- Name: index_recommendation_profiles_on_preferred_platforms; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_recommendation_profiles_on_preferred_platforms ON public.recommendation_profiles USING gin (preferred_platforms); + + +-- +-- Name: index_recommendation_profiles_on_preferred_tlds; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_recommendation_profiles_on_preferred_tlds ON public.recommendation_profiles USING gin (preferred_tlds); + + +-- +-- Name: index_recommendation_profiles_on_user_id; Type: INDEX; Schema: public; Owner: - +-- + +CREATE UNIQUE INDEX index_recommendation_profiles_on_user_id ON public.recommendation_profiles USING btree (user_id); + + +-- +-- Name: index_recommendation_profiles_on_uuid; Type: INDEX; Schema: public; Owner: - +-- + +CREATE UNIQUE INDEX index_recommendation_profiles_on_uuid ON public.recommendation_profiles USING btree (uuid); + + -- -- Name: index_results_on_auction_id; Type: INDEX; Schema: public; Owner: - -- @@ -2777,6 +3270,41 @@ CREATE UNIQUE INDEX index_statistics_report_invoices_on_id ON public.statistics_ CREATE UNIQUE INDEX index_statistics_report_results_on_id ON public.statistics_report_results USING btree (id); +-- +-- Name: index_user_auction_scores_on_auction_id; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_user_auction_scores_on_auction_id ON public.user_auction_scores USING btree (auction_id); + + +-- +-- Name: index_user_auction_scores_on_user_id; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_user_auction_scores_on_user_id ON public.user_auction_scores USING btree (user_id); + + +-- +-- Name: index_user_auction_scores_on_user_id_and_auction_id; Type: INDEX; Schema: public; Owner: - +-- + +CREATE UNIQUE INDEX index_user_auction_scores_on_user_id_and_auction_id ON public.user_auction_scores USING btree (user_id, auction_id); + + +-- +-- Name: index_user_auction_scores_on_user_id_and_score; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_user_auction_scores_on_user_id_and_score ON public.user_auction_scores USING btree (user_id, score); + + +-- +-- Name: index_user_auction_scores_on_uuid; Type: INDEX; Schema: public; Owner: - +-- + +CREATE UNIQUE INDEX index_user_auction_scores_on_uuid ON public.user_auction_scores USING btree (uuid); + + -- -- Name: index_users_on_confirmation_token; Type: INDEX; Schema: public; Owner: - -- @@ -2833,6 +3361,13 @@ CREATE INDEX index_webpush_subscriptions_on_user_id ON public.webpush_subscripti CREATE INDEX index_wishlist_items_on_domain_name ON public.wishlist_items USING btree (domain_name); +-- +-- Name: index_wishlist_items_on_user_id; Type: INDEX; Schema: public; Owner: - +-- + +CREATE INDEX index_wishlist_items_on_user_id ON public.wishlist_items USING btree (user_id); + + -- -- Name: users_by_identity_code_and_country; Type: INDEX; Schema: public; Owner: - -- @@ -2924,6 +3459,14 @@ CREATE TRIGGER process_user_audit AFTER INSERT OR DELETE OR UPDATE ON public.use CREATE TRIGGER process_wishlist_item_audit AFTER INSERT OR DELETE OR UPDATE ON public.wishlist_items FOR EACH ROW EXECUTE FUNCTION public.process_wishlist_item_audit(); +-- +-- Name: recommendation_events fk_rails_04596f3101; Type: FK CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.recommendation_events + ADD CONSTRAINT fk_rails_04596f3101 FOREIGN KEY (user_id) REFERENCES public.users(id); + + -- -- Name: bans fk_rails_070022cd76; Type: FK CONSTRAINT; Schema: public; Owner: - -- @@ -2932,6 +3475,14 @@ ALTER TABLE ONLY public.bans ADD CONSTRAINT fk_rails_070022cd76 FOREIGN KEY (user_id) REFERENCES public.users(id); +-- +-- Name: recommendation_events fk_rails_1663165237; Type: FK CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.recommendation_events + ADD CONSTRAINT fk_rails_1663165237 FOREIGN KEY (auction_id) REFERENCES public.auctions(id); + + -- -- Name: invoice_items fk_rails_25bf3d2c5e; Type: FK CONSTRAINT; Schema: public; Owner: - -- @@ -3004,6 +3555,14 @@ ALTER TABLE ONLY public.results ADD CONSTRAINT fk_rails_9f5d06cf95 FOREIGN KEY (auction_id) REFERENCES public.auctions(id); +-- +-- Name: recommendation_profiles fk_rails_a650b1795d; Type: FK CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.recommendation_profiles + ADD CONSTRAINT fk_rails_a650b1795d FOREIGN KEY (user_id) REFERENCES public.users(id); + + -- -- Name: offers fk_rails_bb5f3f4ecb; Type: FK CONSTRAINT; Schema: public; Owner: - -- @@ -3028,6 +3587,22 @@ ALTER TABLE ONLY public.offers ADD CONSTRAINT fk_rails_e6095d6211 FOREIGN KEY (user_id) REFERENCES public.users(id); +-- +-- Name: user_auction_scores fk_rails_ea35c8f56c; Type: FK CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.user_auction_scores + ADD CONSTRAINT fk_rails_ea35c8f56c FOREIGN KEY (auction_id) REFERENCES public.auctions(id); + + +-- +-- Name: user_auction_scores fk_rails_f45bfd4b1b; Type: FK CONSTRAINT; Schema: public; Owner: - +-- + +ALTER TABLE ONLY public.user_auction_scores + ADD CONSTRAINT fk_rails_f45bfd4b1b FOREIGN KEY (user_id) REFERENCES public.users(id); + + -- -- Name: payment_orders fk_rails_f9dc5857c3; Type: FK CONSTRAINT; Schema: public; Owner: - -- @@ -3051,6 +3626,22 @@ ALTER TABLE ONLY public.invoices SET search_path TO "$user", public; INSERT INTO "schema_migrations" (version) VALUES +('20260710100000'), +('20260709100100'), +('20260709100000'), +('20260708120100'), +('20260708120000'), +('20260601100100'), +('20260601100000'), +('20260528100000'), +('20260527090300'), +('20260527090200'), +('20260527090100'), +('20260527090000'), +('20260525115000'), +('20260525095700'), +('20260525095600'), +('20260525095500'), ('20251105120000'), ('20251105100000'), ('20251104081848'), diff --git a/lib/tasks/demo_auctions.rake b/lib/tasks/demo_auctions.rake new file mode 100644 index 000000000..a398c12c5 --- /dev/null +++ b/lib/tasks/demo_auctions.rake @@ -0,0 +1,256 @@ +namespace :demo do + # Domain seed list grouped by intended category. After `rake demo:create_blind_auctions` + # the Auction.after_create callback enqueues ClassifyDomainJob for every domain, + # which calls the LLM (requires Feature.open_ai_integration_enabled?) to map the + # domain onto an InterestCatalog category code. The nightly cron re-classifies + # anything missed. Use the variety here to verify that different InterestCatalog + # categories surface correctly in /auctions sort. + DEMO_DOMAINS = { + # Estonian common nouns — dictionary should hit on first pass. + local_service: %w[ + kohvik.ee kohv.ee ehitus.ee remont.ee parandus.ee + ilusalong.ee kosmeetika.ee spa.ee + restoran.ee pubi.ee baar.ee + kingsepp.ee juuksur.ee maaler.ee elekter.ee + ], + health: %w[ + apteek.ee arst.ee tervis.ee hambaarst.ee + doctor.ee clinic.ee medic.ee pharma.ee + wellnesshub.ee fitness.ee yoga.ee + psyhholoog.ee labor.ee + ], + shop_brand: %w[ + pood.ee aiapood.ee kalapood.ee raamatupood.ee mobiilipood.ee + veebipood.ee tooriistad.ee lilled.ee mood.ee + shopline.ee dealzone.ee foodmarket.ee + megastore.ee bigshop.ee smartmart.ee outletshop.ee + ], + saas: %w[ + saasplatvorm.ee tarkvara.ee pilv.ee + cloudstack.ee pixelcraft.ee gamesuite.ee workzone.ee startupdesk.ee + apptools.ee devkit.ee codeflow.ee dataforge.ee + crmstack.ee apilayer.ee testlab.ee + ], + b2b_service: %w[ + turundus.ee nouv.ee konsultatsioon.ee + growthhub.ee marketflow.ee mediateam.ee accountflow.ee + adagency.ee leadgen.ee salesforce-tools.ee + brandstrategy.ee b2bpro.ee enterprisepro.ee + ], + finance: %w[ + laen.ee pank.ee raha.ee investeering.ee + fintechlab.ee fastloan.ee cashflow.ee + crypto-trade.ee invest24.ee bankhub.ee + kredit.ee finance-pro.ee + ], + legal: %w[ + jurist.ee oigusabi.ee notar.ee + legalhub.ee lawfirm.ee legaltech.ee + contractpro.ee compliance-pro.ee + ], + education: %w[ + haridus.ee kool.ee koolitus.ee opetaja.ee + academy.ee learnhub.ee coursestack.ee skillsup.ee + classmate.ee studyflow.ee + ], + travel: %w[ + reisid.ee matk.ee majutus.ee turism.ee + traveldesk.ee tripflow.ee hotelhub.ee + flightbooker.ee adventurelab.ee resort-deal.ee + ], + automotive: %w[ + autod.ee auto.ee rent.ee + carshop.ee motorflow.ee driveforge.ee + autopro.ee auto-rent.ee tireshop.ee + ], + real_estate: %w[ + kinnisvara.ee maja.ee korter.ee + propertylab.ee homehunt.ee realtypro.ee + apartment-finder.ee rentdesk.ee + ], + media_content: %w[ + meedia.ee uudised.ee ajakiri.ee + newsdesk.ee blogforge.ee podcastlab.ee + videohub.ee press-room.ee + ], + brandable: %w[ + brandforge.ee zylo.ee zenix.ee + koral.ee veylo.ee xolo.ee + qarra.ee jovi.ee plooma.ee + ], + numeric: %w[ + 24.ee 365.ee 100.ee 12345.ee + numeric24.ee 7eleven.ee top10.ee + shop42.ee deal99.ee + ], + other: %w[ + a-b-c.ee my-shop-online.ee long-domain-name-test.ee + short.ee mid-tier.ee + ] + }.freeze + + # `+ 1.second` was tight enough that Rails 8.1 boot could land us + # in the past by the time validation ran. 5 minutes is generous + # enough to survive a slow boot or a paused debugger. + AUCTION_START_BUFFER = 5.minutes + + desc 'Create temporary blind .ee auctions for recommendation testing (~140 domains)' + task create_blind_auctions: :environment do + starts_at = Time.zone.now + AUCTION_START_BUFFER + ends_at = starts_at + 1.month + + domains = DEMO_DOMAINS.values.flatten.uniq + + created = 0 + skipped = 0 + + domains.each do |domain_name| + if Auction.where(domain_name: domain_name).where('ends_at > ?', Time.zone.now).exists? + skipped += 1 + next + end + + Auction.create!( + domain_name: domain_name, + starts_at: starts_at, + ends_at: ends_at + ) + + created += 1 + end + + puts "Created #{created} blind auctions, skipped #{skipped} existing active auctions." + puts "Total demo domains: #{domains.size} across #{DEMO_DOMAINS.size} categories." + end + + desc 'Create varied auctions ending at different times (next hour, day, week, month)' + task create_varied_auctions: :environment do + starts_at = Time.zone.now + AUCTION_START_BUFFER + horizons = { + next_hour: { ends_at_offset: 1.hour, sample_size: 5 }, + next_day: { ends_at_offset: 1.day, sample_size: 10 }, + next_week: { ends_at_offset: 1.week, sample_size: 20 }, + next_month: { ends_at_offset: 1.month, sample_size: 40 } + } + + all_domains = DEMO_DOMAINS.values.flatten.uniq.shuffle + + created = 0 + offset = 0 + + horizons.each do |_label, opts| + batch = all_domains[offset, opts[:sample_size]] || [] + offset += opts[:sample_size] + + batch.each do |domain_name| + next if Auction.where(domain_name: domain_name).where('ends_at > ?', Time.zone.now).exists? + + Auction.create!( + domain_name: domain_name, + starts_at: starts_at, + ends_at: starts_at + opts[:ends_at_offset] + ) + created += 1 + end + end + + puts "Created #{created} varied-horizon auctions." + end + + desc 'Create demo auctions that are ACTIVE immediately (for staging pipeline bootstrap)' + task create_active_auctions: :environment do + # Unlike create_blind_auctions (which starts 5 min in the future), these + # start in the past so Auction.active includes them right away — otherwise + # the recommendation pipeline (classify / ai_score / scorer all scope to + # Auction.active) would run against an empty set. skip_validation bypasses + # starts_at_cannot_be_in_the_past; safe because this is demo data only. + starts_at = Time.zone.now - 1.minute + ends_at = starts_at + 1.month + + domains = DEMO_DOMAINS.values.flatten.uniq + + created = 0 + skipped = 0 + + domains.each do |domain_name| + if Auction.where(domain_name: domain_name).where('ends_at > ?', Time.zone.now).exists? + skipped += 1 + next + end + + Auction.create!( + domain_name: domain_name, + starts_at: starts_at, + ends_at: ends_at, + skip_validation: true + ) + + created += 1 + end + + puts "Created #{created} active demo auctions, skipped #{skipped} existing active auctions." + end + + desc 'Seed a participant user with wishlist + bid signals so recommendations have data to chew on' + task seed_user_signals: :environment do + user = User.where('? = ANY (roles)', User::PARTICIPANT_ROLE).first + if user.nil? + puts 'No participant user found. Create one via the UI first, then re-run.' + next + end + + profile = user.recommendation_profile || user.build_recommendation_profile + profile.interest_keywords = %w[saas b2b_service custom:cloud custom:agency] + profile.preferred_length_min = 5 + profile.preferred_length_max = 18 + profile.allow_numbers = false + profile.allow_hyphens = false + profile.save! + profile.mark_completed! + + wishlist_limit = Setting.find_by(code: 'wishlist_size').retrieve + wishlist_picks = %w[cloudstack.ee fintechlab.ee marketflow.ee growthhub.ee] + wishlist_added = [] + wishlist_picks.each do |domain| + next if user.wishlist_items.exists?(domain_name: domain) + + if user.wishlist_items.count >= wishlist_limit + puts " wishlist full (limit #{wishlist_limit}), skipping: #{(wishlist_picks - wishlist_added).inspect}" + break + end + + item = WishlistItem.new(user: user, domain_name: domain, cents: 5000) + if item.save + wishlist_added << domain + else + puts " skipped wishlist #{domain}: #{item.errors.full_messages.join(', ')}" + end + end + + bid_picks = %w[ + apteek.ee kohvik.ee jurist.ee reisid.ee + ] + bids_added = [] + bid_picks.each do |domain| + auction = Auction.where(domain_name: domain).where('ends_at > ?', Time.zone.now).first + next if auction.nil? + next if Offer.exists?(user_id: user.id, auction_id: auction.id) + + offer = Offer.new(user: user, auction: auction, cents: 1500, billing_profile_id: 0) + if offer.save + bids_added << domain + else + puts " skipped bid #{domain}: #{offer.errors.full_messages.join(', ')}" + end + end + + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(user.id) + + puts "Seeded signals on user ##{user.id} (#{user.email}):" + puts " profile interests: #{profile.interest_categories.inspect} + custom #{profile.custom_interests.inspect}" + puts " wishlist added: #{wishlist_added.inspect}" + puts " bids added: #{bids_added.inspect}" + puts + puts 'Score refresh enqueued. After the delayed_job runs (~30s), visit /auctions while signed in as this user.' + end +end diff --git a/lib/tasks/recommendation.rake b/lib/tasks/recommendation.rake new file mode 100644 index 000000000..f135c020e --- /dev/null +++ b/lib/tasks/recommendation.rake @@ -0,0 +1,46 @@ +namespace :recommendation do + # The single prod entry point. Incremental (force: false): enriches interest + # categories, backfills users' custom-interest vectors, then drains the batched + # classify + embed jobs (picking up any domain that is unclassified, stale, or + # built under an older embedding-input format), refreshes ai_score, and + # recomputes every participant's personal scores. Already-done work is skipped. + # + # Run on every deploy. This is also the cron catch-up: it drains the same + # classify/embed batches a standalone cron would, so no separate cron task is + # needed for them. + desc 'PROD/CRON: run the whole recommendation pipeline (classify -> embed -> ai_score -> per-user scores)' + task init: :environment do + summary = Recommendation::PipelineRunner.run(force: false) + puts "Recommendation pipeline done: #{summary.inspect}" + end + + # Retention for the append-only recommendation_events table (unrelated to the + # pipeline above; schedule daily). Deletes events >6 months old and + # auction_impression events >1 month old, in batches. + desc 'CRON: prune old recommendation_events (6mo; impressions 1mo)' + task prune_events: :environment do + deleted = Recommendation::PruneRecommendationEventsJob.perform_now + puts "Pruned #{deleted} recommendation event(s)." + end + + # One-shot historical seed: classify every domain we have ever seen (active + + # ended auctions, wishlist, offer histories, results) so past-bid domains get + # embeddings and can form magnets. init only classifies active auctions + + # wishlist, so run this once when first enabling the feature. + desc 'One-shot: classify all historical domains via LLM (wider universe than init)' + task backfill: :environment do + if defined?(Recommendation::BackfillDomainClassificationsJob) + Recommendation::BackfillDomainClassificationsJob.perform_now + else + puts 'BackfillDomainClassificationsJob is not yet available. Skipping.' + end + end + + desc 'TEST/STAGING: create active mock auctions + signals, then run the pipeline' + task init_demo: :environment do + Rake::Task['demo:create_active_auctions'].invoke + Rake::Task['demo:seed_user_signals'].invoke + summary = Recommendation::PipelineRunner.run(force: false) + puts "Recommendation pipeline (demo) done: #{summary.inspect}" + end +end diff --git a/test/controllers/auctions_controller_test.rb b/test/controllers/auctions_controller_test.rb index c7bf0485b..63d0e525c 100644 --- a/test/controllers/auctions_controller_test.rb +++ b/test/controllers/auctions_controller_test.rb @@ -93,6 +93,43 @@ def test_explicit_sorting_overrides_user_auction_priority assert_equal sorted_domains, auction_domains end + def test_recommendation_scores_are_used_before_global_ai_score_for_logged_in_users + sign_in @participant + + UserAuctionScore.create!( + user: @participant, + auction: @english_auction, + score: 0.95, + calculated_at: Time.current + ) + UserAuctionScore.create!( + user: @participant, + auction: @auction_without_offers, + score: 0.10, + calculated_at: Time.current + ) + + get auctions_path + + assert_response :success + + auction_domains = css_select('tbody#bids tr.contents td:first-child').map { |e| e.text.strip } + + with_offers_index = auction_domains.index(@auction_with_offers.domain_name) + english_index = auction_domains.index(@english_auction.domain_name) + without_offers_index = auction_domains.index(@auction_without_offers.domain_name) + + assert with_offers_index < english_index, "User's own auction should still come first" + assert english_index < without_offers_index, "Higher recommendation score should outrank lower score" + end + + # v3: interest prioritisation no longer has its own string-matched sort tier. + # A selected category / custom interest becomes a magnet whose pull lands in + # user_auction_scores.score (which requires embeddings, exercised in + # Recommendation::MagnetScorer / Scorer unit tests and the demo comparison). + # At the controller level, ordering is driven by that score — see + # test_recommendation_scores_are_used_before_global_ai_score_for_logged_in_users. + def test_sorting_with_pagination_keeps_user_auctions_prioritized sign_in @participant @@ -126,9 +163,47 @@ def test_json_api_returns_auctions_with_user_priority def test_admin_user_does_not_get_priority_sorting @admin = users(:administrator) sign_in @admin - + get auctions_path, params: { admin: 'true' } - + + assert_response :success + end + + def test_auction_type_column_shown_by_default + get auctions_path + assert_response :success + assert_includes response.body, I18n.t('auctions.auction_type') + assert_select 'td.auction-tags', false + end + + def test_tags_column_replaces_auction_type_when_flag_enabled + DomainClassification.create!( + domain_name: @auction_without_offers.domain_name, + tags: %w[crypto defi], + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.9, + classified_at: Time.current + ) + + with_tags_display(true) do + get auctions_path + end + + assert_response :success + assert_includes response.body, I18n.t('auctions.tags') + assert_select 'td.auction-tags' + assert_select 'td.auction-tags span.c-badge', text: 'crypto' + end + + private + + def with_tags_display(enabled) + config = AuctionCenter::Application.config.customization + original = config[:auction_tags_display_enabled] + config[:auction_tags_display_enabled] = enabled + yield + ensure + config[:auction_tags_display_enabled] = original end end \ No newline at end of file diff --git a/test/controllers/recommendation_detail_view_test.rb b/test/controllers/recommendation_detail_view_test.rb new file mode 100644 index 000000000..3e14b4bc8 --- /dev/null +++ b/test/controllers/recommendation_detail_view_test.rb @@ -0,0 +1,24 @@ +require 'test_helper' + +class RecommendationDetailViewTest < ActionDispatch::IntegrationTest + include Devise::Test::IntegrationHelpers + + def setup + super + @user = users(:participant) + @auction = auctions(:valid_without_offers) + sign_in @user + RecommendationEvent.where(event_type: 'auction_detail_view').delete_all + end + + def test_offers_new_records_detail_view + assert_difference -> { RecommendationEvent.where(event_type: 'auction_detail_view').count }, 1 do + get new_auction_offer_path(auction_uuid: @auction.uuid) + end + + event = RecommendationEvent.where(event_type: 'auction_detail_view').last + assert_equal @user.id, event.user_id + assert_equal @auction.id, event.auction_id + assert_equal 'offers#new', event.source + end +end diff --git a/test/controllers/recommendation_profiles_controller_test.rb b/test/controllers/recommendation_profiles_controller_test.rb new file mode 100644 index 000000000..834a28921 --- /dev/null +++ b/test/controllers/recommendation_profiles_controller_test.rb @@ -0,0 +1,94 @@ +require 'test_helper' + +class RecommendationProfilesControllerTest < ActionDispatch::IntegrationTest + include Devise::Test::IntegrationHelpers + include ActiveJob::TestHelper + + def setup + super + @user = users(:participant) + sign_in @user + clear_enqueued_jobs + end + + def test_user_can_update_recommendation_profile + assert_nil @user.recommendation_profile + + assert_difference -> { RecommendationProfile.count } do + assert_enqueued_with(job: Recommendation::RefreshSingleUserAuctionScoresJob, args: [@user.id]) do + put recommendation_profile_path, params: { + recommendation_profile: { + interest_categories: %w[legal other], + custom_interests: ['marketplace'] + } + } + end + end + + @user.reload + + assert_redirected_to user_path(@user.uuid) + assert_equal(%w[legal other], @user.recommendation_profile.interest_categories.sort) + assert_equal(['marketplace'], @user.recommendation_profile.custom_interests) + assert @user.recommendation_profile.completed? + end + + def test_edit_form_renders_custom_interest_sentinel + @user.create_recommendation_profile!(interest_keywords: %w[other custom:crypto]) + + get edit_recommendation_profile_path + + assert_response :success + # Without this sentinel, removing every custom-interest tag drops the param + # entirely and the old free-text interests can never be cleared. + assert_select "input[type=hidden][name='recommendation_profile[custom_interests][]'][value='']" + end + + def test_clearing_all_custom_interests_removes_them + @user.create_recommendation_profile!(interest_keywords: %w[legal other custom:crypto]) + + # `other` is derived server-side, not a user checkbox, so the form no longer + # submits it — but a stray `other` (old cached page, API) with no custom + # interests must still be stripped, not linger as an empty category. + put recommendation_profile_path, params: { + recommendation_profile: { + interest_categories: %w[legal other], + custom_interests: [''] + } + } + + @user.reload + assert_empty @user.recommendation_profile.custom_interests + # `other` must not linger as an empty category once its custom interests are gone. + assert_equal %w[legal], @user.recommendation_profile.interest_categories.sort + end + + def test_user_can_dismiss_recommendation_prompt + patch dismiss_recommendation_profile_path + + @user.reload + + assert_redirected_to root_path + assert @user.recommendation_profile.prompt_dismissed_at.present? + end + + def test_edit_redirects_when_selection_disabled + settings(:recommendation_interests_enabled).update!(value: 'false') + + get edit_recommendation_profile_path + + assert_redirected_to user_path(@user.uuid) + end + + def test_update_blocked_when_selection_disabled + settings(:recommendation_interests_enabled).update!(value: 'false') + + assert_no_difference -> { RecommendationProfile.count } do + put recommendation_profile_path, params: { + recommendation_profile: { interest_categories: %w[legal] } + } + end + + assert_redirected_to user_path(@user.uuid) + end +end diff --git a/test/fixtures/settings.yml b/test/fixtures/settings.yml index 2de99e062..c4dfd476d 100644 --- a/test/fixtures/settings.yml +++ b/test/fixtures/settings.yml @@ -274,7 +274,7 @@ openai_model: code: 'openai_model' description: | OpenAI API model - value: 'gpt-3.5-turbo' + value: 'gpt-5' value_format: string openai_domains_evaluation_prompt: @@ -325,3 +325,11 @@ contact_organization_email: value: 'info@internet.ee' value_format: string +recommendation_interests_enabled: + code: 'recommendation_interests_enabled' + description: | + When 'true', users see the auction-interests selection (prompt + form). + Keep 'false' until interest categories are populated in admin. + value: 'true' + value_format: boolean + diff --git a/test/integration/admin/auctions_controller_test.rb b/test/integration/admin/auctions_controller_test.rb index 79c42ffa0..bec16ad4c 100644 --- a/test/integration/admin/auctions_controller_test.rb +++ b/test/integration/admin/auctions_controller_test.rb @@ -30,6 +30,29 @@ def test_show_renders_ok assert_response :ok end + def test_show_renders_llm_classification + auction = auctions(:valid_without_offers) + DomainClassification.create!( + domain_name: auction.domain_name, + primary_category: 'saas', + tags: %w[cloud analytics], + keywords: %w[dashboard metrics], + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.91, + classified_at: Time.current + ) + + sign_in @administrator + User.stub(:search_deposit_participants, User.none) do + get admin_auction_path(auction.id) + end + + assert_response :ok + assert_includes response.body, 'saas' + assert_includes response.body, 'cloud' + assert_includes response.body, I18n.t('admin.auctions.classification.heading') + end + def test_destroy_deletes_auction_when_not_started auction = auctions(:english_nil_starts) diff --git a/test/integration/admin/interest_categories_controller_test.rb b/test/integration/admin/interest_categories_controller_test.rb new file mode 100644 index 000000000..adc5eee6e --- /dev/null +++ b/test/integration/admin/interest_categories_controller_test.rb @@ -0,0 +1,117 @@ +require 'test_helper' + +class AdminInterestCategoriesControllerTest < ActionDispatch::IntegrationTest + include Devise::Test::IntegrationHelpers + + def setup + @admin = users(:administrator) + @participant = users(:participant) + @category = InterestCategory.create!(code: 'saas', name_en: 'SaaS', name_et: 'SaaS', position: 1) + end + + def test_index_renders_for_admin + sign_in @admin + get admin_interest_categories_path + assert_response :ok + assert_includes response.body, 'saas' + end + + def test_new_renders_for_admin + sign_in @admin + get new_admin_interest_category_path + assert_response :ok + end + + def test_create_adds_category + sign_in @admin + + assert_difference -> { InterestCategory.count }, 1 do + post admin_interest_categories_path, params: { + interest_category: { code: 'Finance', name_en: 'Finance', name_et: 'Finants', position: 2, active: true } + } + end + + assert_redirected_to admin_interest_categories_path + assert InterestCategory.exists?(code: 'finance'), 'code should be normalized to lowercase' + end + + def test_update_edits_category + sign_in @admin + + patch admin_interest_category_path(@category), params: { + interest_category: { name_en: 'Software', name_et: 'Tarkvara' } + } + + assert_redirected_to admin_interest_categories_path + assert_equal 'Software', @category.reload.name_en + end + + def test_destroy_removes_category + sign_in @admin + + assert_difference -> { InterestCategory.count }, -1 do + delete admin_interest_category_path(@category) + end + end + + def test_create_rejects_blank_names + sign_in @admin + + assert_no_difference -> { InterestCategory.count } do + post admin_interest_categories_path, params: { + interest_category: { code: 'broken', name_en: '', name_et: '' } + } + end + + assert_response :unprocessable_entity + end + + def test_index_shows_enrichment_status + sign_in @admin + @category.update_columns( + description: 'SaaS and software tools', + keywords: %w[crm billing analytics], + embedding: Array.new(3, 0.1), + embedding_model: 'text-embedding-3-small', + embedded_at: Time.current + ) + + get admin_interest_categories_path + + assert_response :ok + assert_includes response.body, I18n.t('interest_categories.enriched') + end + + def test_edit_shows_llm_enrichment_results + sign_in @admin + @category.update_columns( + description: 'SaaS and software tools', + keywords: %w[crm billing analytics], + embedding: Array.new(3, 0.1), + embedding_model: 'text-embedding-3-small', + embedded_at: Time.current + ) + + get edit_admin_interest_category_path(@category) + + assert_response :ok + assert_includes response.body, I18n.t('interest_categories.enrichment_heading') + assert_includes response.body, 'SaaS and software tools' + assert_includes response.body, 'crm' + end + + def test_edit_shows_not_enriched_note_when_missing + sign_in @admin + + get edit_admin_interest_category_path(@category) + + assert_response :ok + assert_includes response.body, I18n.t('interest_categories.not_enriched') + end + + def test_not_accessible_for_non_admin + sign_in @participant + get admin_interest_categories_path + assert_response :not_found + end +end diff --git a/test/integration/english_offers_test.rb b/test/integration/english_offers_test.rb index b2010ccd5..5fbee1e69 100644 --- a/test/integration/english_offers_test.rb +++ b/test/integration/english_offers_test.rb @@ -18,6 +18,7 @@ def setup .to_return(status: 200, body: "{\"reference_number\":\"#{rand(111..999)}\"}", headers: {}) travel_to Time.parse('2010-07-05 11:30 +0000').in_time_zone + clear_enqueued_jobs end def test_user_can_create_a_bid @@ -34,9 +35,11 @@ def test_user_can_create_a_bid } } - post auction_english_offers_path(auction_uuid: @auction.uuid), - params: params, - headers: { "HTTP_REFERER" => root_path } + assert_enqueued_with(job: Recommendation::RefreshSingleUserAuctionScoresJob, args: [@user.id]) do + post auction_english_offers_path(auction_uuid: @auction.uuid), + params: params, + headers: { "HTTP_REFERER" => root_path } + end assert @auction.offers.present? assert_equal @auction.offers.first.cents, 500 @@ -87,9 +90,11 @@ def test_user_can_update_existed_bid } } - patch english_offer_path(uuid: @auction.offers.first.uuid), - params: params, - headers: { "HTTP_REFERER" => root_path } + assert_enqueued_with(job: Recommendation::RefreshSingleUserAuctionScoresJob, args: [@user.id]) do + patch english_offer_path(uuid: @auction.offers.first.uuid), + params: params, + headers: { "HTTP_REFERER" => root_path } + end @auction.reload diff --git a/test/integration/offers_test.rb b/test/integration/offers_test.rb index bc9192cca..848ed501d 100644 --- a/test/integration/offers_test.rb +++ b/test/integration/offers_test.rb @@ -4,6 +4,7 @@ class OffersAuctionFlowTest < ActionDispatch::IntegrationTest OFFER_COUNT = 'Offer.count' include Devise::Test::IntegrationHelpers + include ActiveJob::TestHelper def setup @user = users(:participant) @@ -15,6 +16,7 @@ def setup Recaptcha.configuration.skip_verify_env.push('test') travel_to Time.parse('2010-07-05 11:30 +0000').in_time_zone + clear_enqueued_jobs end def test_user_can_create_a_bid @@ -27,9 +29,11 @@ def test_user_can_create_a_bid } } - post auction_offers_path(auction_uuid: @auction.uuid), - params: params, - headers: {} + assert_enqueued_with(job: Recommendation::RefreshSingleUserAuctionScoresJob, args: [@user.id]) do + post auction_offers_path(auction_uuid: @auction.uuid), + params: params, + headers: {} + end assert @auction.offers.present? assert_equal @auction.offers.first.cents, 600 @@ -155,7 +159,9 @@ def test_user_can_update_his_bid } } - patch offer_path(uuid: offer.uuid), params: params + assert_enqueued_with(job: Recommendation::RefreshSingleUserAuctionScoresJob, args: [@user.id]) do + patch offer_path(uuid: offer.uuid), params: params + end offer.reload assert_equal offer.cents, 1500 diff --git a/test/integration/wishlist_items_test.rb b/test/integration/wishlist_items_test.rb index 4fc503bce..e9bb88b3a 100644 --- a/test/integration/wishlist_items_test.rb +++ b/test/integration/wishlist_items_test.rb @@ -2,6 +2,7 @@ class WishlistItemsIntegrationTest < ActionDispatch::IntegrationTest include Devise::Test::IntegrationHelpers + include ActiveJob::TestHelper def setup @user = users(:participant) @@ -12,6 +13,7 @@ def setup sign_in @user travel_to Time.parse('2010-07-05 10:30 +0000').in_time_zone + clear_enqueued_jobs end def test_should_be_returned_ok_code @@ -32,7 +34,9 @@ def test_user_can_create_wishlist_item } assert_difference -> { WishlistItem.count } do - post wishlist_items_path, params: params, headers: {} + assert_enqueued_with(job: Recommendation::RefreshSingleUserAuctionScoresJob, args: [@user.id]) do + post wishlist_items_path, params: params, headers: {} + end end end diff --git a/test/jobs/recommendation/backfill_domain_classifications_job_test.rb b/test/jobs/recommendation/backfill_domain_classifications_job_test.rb new file mode 100644 index 000000000..96c9f8d41 --- /dev/null +++ b/test/jobs/recommendation/backfill_domain_classifications_job_test.rb @@ -0,0 +1,135 @@ +require 'test_helper' + +module Recommendation + class BackfillDomainClassificationsJobTest < ActiveJob::TestCase + def setup + super + DomainClassification.delete_all + end + + def test_classifies_auction_and_wishlist_domains + auction = Auction.create!( + domain_name: 'backfill-auction.ee', + starts_at: 1.hour.ago, + ends_at: 1.day.from_now, + skip_validation: true + ) + WishlistItem.create!(user: users(:participant), domain_name: 'backfill-wishlist.ee', cents: 1_000) + + with_feature_flag(true) do + stub_llm do + Recommendation::BackfillDomainClassificationsJob.new.perform + end + end + + assert DomainClassification.exists?(domain_name: 'backfill-auction.ee') + assert DomainClassification.exists?(domain_name: 'backfill-wishlist.ee') + auction.destroy + end + + def test_skips_already_classified_domains + Auction.create!( + domain_name: 'skip-me.ee', + starts_at: 1.hour.ago, + ends_at: 1.day.from_now, + skip_validation: true + ) + DomainClassification.create!( + domain_name: 'skip-me.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.9, + classified_at: 1.hour.ago + ) + + with_feature_flag(true) do + stub_llm do + assert_no_difference -> { DomainClassification.where(domain_name: 'skip-me.ee').count } do + Recommendation::BackfillDomainClassificationsJob.new.perform + end + end + end + end + + def test_upsert_resets_stale_embedding_on_conflict + skip 'embedding column not present' unless DomainClassification.column_names.include?('embedding') + + DomainClassification.create!( + domain_name: 'reset-me.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.9, + classified_at: 1.hour.ago, + embedding: [0.1, 0.2, 0.3], + embedding_model: 'text-embedding-3-small', + embedded_at: 1.hour.ago + ) + + job = Recommendation::BackfillDomainClassificationsJob.new + job.send(:upsert, [{ + domain_name: 'reset-me.ee', + primary_category: 'other', + tags: ['other'], + keywords: [], + confidence: 0.9, + classification_source: DomainClassification::OPENAI_SOURCE, + classification_model: 'gpt-5', + classified_at: Time.current + }]) + + row = DomainClassification.find_by(domain_name: 'reset-me.ee') + assert_nil row.embedding + assert_nil row.embedding_model + assert_nil row.embedded_at + end + + def test_no_op_when_openai_disabled + Auction.create!( + domain_name: 'no-llm.ee', + starts_at: 1.hour.ago, + ends_at: 1.day.from_now, + skip_validation: true + ) + + with_feature_flag(false) do + assert_no_difference -> { DomainClassification.count } do + Recommendation::BackfillDomainClassificationsJob.new.perform + end + end + end + + private + + def stub_llm + original = Recommendation::LlmDomainClassifier.method(:call) + Recommendation::LlmDomainClassifier.define_singleton_method(:call) do |domain_names:, **| + Array(domain_names).map do |d| + { + domain_name: d.to_s.strip.downcase, + primary_category: 'other', + tags: ['other'], + keywords: [], + audience: nil, + languages: [], + suggested_use_cases: [], + brandability_score: nil, + confidence: 0.9, + classification_source: DomainClassification::OPENAI_SOURCE, + classification_model: 'gpt-5', + classified_at: Time.current, + raw_llm_response: {} + } + end + end + yield + ensure + Recommendation::LlmDomainClassifier.define_singleton_method(:call, original) + end + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end + end +end diff --git a/test/jobs/recommendation/classify_domain_job_test.rb b/test/jobs/recommendation/classify_domain_job_test.rb new file mode 100644 index 000000000..21b78d628 --- /dev/null +++ b/test/jobs/recommendation/classify_domain_job_test.rb @@ -0,0 +1,117 @@ +require 'test_helper' + +module Recommendation + class ClassifyDomainJobTest < ActiveJob::TestCase + def setup + super + DomainClassification.where(domain_name: 'apteek.ee').delete_all + Setting.find_by(code: 'openai_model')&.update!(value: 'gpt-5') + end + + def test_classifies_unknown_domain_via_llm + stub_openai('apteek.ee', primary: 'health') + + with_feature_flag(true) do + assert_difference -> { DomainClassification.count }, 1 do + Recommendation::ClassifyDomainJob.new.perform('apteek.ee') + end + end + + row = DomainClassification.find_by(domain_name: 'apteek.ee') + assert_equal 'health', row.primary_category + assert_equal DomainClassification::OPENAI_SOURCE, row.classification_source + end + + def test_no_op_when_openai_disabled + with_feature_flag(false) do + assert_no_difference -> { DomainClassification.count } do + Recommendation::ClassifyDomainJob.new.perform('apteek.ee') + end + end + assert_not_requested :post, 'https://api.openai.com/v1/chat/completions' + end + + def test_skips_fresh_llm_classification + DomainClassification.create!(domain_name: 'apteek.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.9, + classified_at: 1.hour.ago) + + with_feature_flag(true) do + assert_no_difference -> { DomainClassification.count } do + Recommendation::ClassifyDomainJob.new.perform('apteek.ee') + end + end + assert_not_requested :post, 'https://api.openai.com/v1/chat/completions' + end + + def test_skips_llm_when_domain_lock_is_held_by_another_worker + job = Recommendation::ClassifyDomainJob.new + # Simulate pg_try_advisory_lock returning false (another worker owns it). + job.define_singleton_method(:with_domain_lock) { |_name, &blk| blk.call(false) } + + with_feature_flag(true) do + assert_no_difference -> { DomainClassification.count } do + job.perform('apteek.ee') + end + end + assert_not_requested :post, 'https://api.openai.com/v1/chat/completions' + end + + def test_no_op_for_blank_domain + with_feature_flag(true) do + assert_no_difference -> { DomainClassification.count } do + Recommendation::ClassifyDomainJob.new.perform('') + Recommendation::ClassifyDomainJob.new.perform(nil) + Recommendation::ClassifyDomainJob.new.perform(' ') + end + end + end + + def test_auction_create_enqueues_classification + assert_enqueued_with(job: Recommendation::ClassifyDomainJob) do + Auction.create!( + domain_name: "trigger-test-#{SecureRandom.hex(4)}.ee", + starts_at: 1.hour.ago, + ends_at: 1.day.from_now, + skip_validation: true + ) + end + end + + private + + def stub_openai(domain_name, primary:) + body = { + 'choices' => [{ + 'finish_reason' => 'stop', + 'message' => { + 'content' => { + classifications: [{ + domain_name: domain_name, + primary_category: primary, + tags: [primary], + keywords: %w[example], + audience: 'b2c', + languages: %w[et], + suggested_use_cases: %w[service], + brandability_score: 0.5, + confidence: 0.9 + }] + }.to_json + } + }] + } + stub_request(:post, 'https://api.openai.com/v1/chat/completions') + .to_return_json(status: 200, body: body, headers: {}) + end + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end + end +end diff --git a/test/jobs/recommendation/classify_unclassified_domains_job_test.rb b/test/jobs/recommendation/classify_unclassified_domains_job_test.rb new file mode 100644 index 000000000..41bc26936 --- /dev/null +++ b/test/jobs/recommendation/classify_unclassified_domains_job_test.rb @@ -0,0 +1,169 @@ +require 'test_helper' + +module Recommendation + class ClassifyUnclassifiedDomainsJobTest < ActiveJob::TestCase + def setup + super + DomainClassification.delete_all + end + + def test_no_op_when_openai_integration_disabled + DomainClassification.create!(domain_name: 'pending.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + classified_at: Time.current, + confidence: 0.3) + + with_feature_flag(false) do + assert_nil Recommendation::ClassifyUnclassifiedDomainsJob.new.perform + end + end + + def test_no_op_when_nothing_pending + with_feature_flag(true) do + with_missing_domains([]) do + assert_nil Recommendation::ClassifyUnclassifiedDomainsJob.new.perform + end + end + end + + def test_reclassifies_low_confidence_existing_row + DomainClassification.create!(domain_name: 'weak.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.3, + classified_at: 1.day.ago) + + with_feature_flag(true) do + with_missing_domains([]) do + stub_llm do + processed = Recommendation::ClassifyUnclassifiedDomainsJob.new.perform + assert_equal 1, processed + end + end + end + end + + def test_reclassification_resets_stale_embedding + skip 'embedding column not present' unless DomainClassification.column_names.include?('embedding') + + row = DomainClassification.create!(domain_name: 'weak.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.3, + classified_at: 1.day.ago, + embedding: [0.1, 0.2, 0.3], + embedding_model: 'text-embedding-3-small', + embedded_at: 1.day.ago) + + with_feature_flag(true) do + with_missing_domains([]) do + stub_llm do + Recommendation::ClassifyUnclassifiedDomainsJob.new.perform + end + end + end + + row.reload + assert_nil row.embedding + assert_nil row.embedding_model + assert_nil row.embedded_at + end + + def test_classifies_missing_domains + with_feature_flag(true) do + with_missing_domains(['fresh-domain.ee']) do + stub_llm do + Recommendation::ClassifyUnclassifiedDomainsJob.new.perform + end + end + end + + assert DomainClassification.exists?(domain_name: 'fresh-domain.ee') + end + + def test_scope_picks_low_confidence_and_stale_llm_rows + low_conf_row = DomainClassification.create!( + domain_name: 'l.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.3, + classified_at: 1.day.ago + ) + stale_row = DomainClassification.create!( + domain_name: 's.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.95, + classified_at: 9.months.ago + ) + fresh_llm = DomainClassification.create!( + domain_name: 'f.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.95, + classified_at: 1.day.ago + ) + + scope_ids = Recommendation::ClassifyUnclassifiedDomainsJob.scope.pluck(:id) + assert_includes scope_ids, low_conf_row.id + assert_includes scope_ids, stale_row.id + refute_includes scope_ids, fresh_llm.id + end + + def test_needs_to_run_reflects_feature_and_work + with_feature_flag(false) do + refute Recommendation::ClassifyUnclassifiedDomainsJob.needs_to_run? + end + + DomainClassification.create!( + domain_name: 'pending.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.3, + classified_at: Time.current + ) + + with_feature_flag(true) do + assert Recommendation::ClassifyUnclassifiedDomainsJob.needs_to_run? + end + end + + private + + def stub_llm + original = Recommendation::LlmDomainClassifier.method(:call) + Recommendation::LlmDomainClassifier.define_singleton_method(:call) do |domain_names:, **| + Array(domain_names).map do |d| + { + domain_name: d.to_s.strip.downcase, + primary_category: 'other', + tags: ['other'], + keywords: [], + audience: nil, + languages: [], + suggested_use_cases: [], + brandability_score: nil, + confidence: 0.9, + classification_source: DomainClassification::OPENAI_SOURCE, + classification_model: 'gpt-5', + classified_at: Time.current, + raw_llm_response: {} + } + end + end + yield + ensure + Recommendation::LlmDomainClassifier.define_singleton_method(:call, original) + end + + def with_missing_domains(list) + original = Recommendation::ClassifyUnclassifiedDomainsJob.method(:missing_domains) + Recommendation::ClassifyUnclassifiedDomainsJob.define_singleton_method(:missing_domains) { list } + yield + ensure + Recommendation::ClassifyUnclassifiedDomainsJob.define_singleton_method(:missing_domains, original) + end + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end + end +end diff --git a/test/jobs/recommendation/embed_custom_interests_job_test.rb b/test/jobs/recommendation/embed_custom_interests_job_test.rb new file mode 100644 index 000000000..194c2fc10 --- /dev/null +++ b/test/jobs/recommendation/embed_custom_interests_job_test.rb @@ -0,0 +1,85 @@ +require 'test_helper' + +module Recommendation + class EmbedCustomInterestsJobTest < ActiveJob::TestCase + def setup + super + @user = users(:participant) + end + + def test_noop_when_openai_disabled + profile = build_profile(%w[crypto]) + with_feature_flag(false) do + assert_nil Recommendation::EmbedCustomInterestsJob.new.perform(profile.id) + end + end + + def test_embeds_each_custom_interest_as_its_own_vector + profile = build_profile(['crypto', 'learning platform']) + + with_feature_flag(true) do + stub_embeddings(2) + assert_enqueued_with(job: Recommendation::RefreshSingleUserAuctionScoresJob, args: [@user.id]) do + Recommendation::EmbedCustomInterestsJob.new.perform(profile.id) + end + end + + vectors = profile.reload.custom_interest_vectors + assert_equal %w[crypto learning\ platform].sort, vectors.map { |v| v['text'] }.sort + assert_equal Recommendation::DomainEmbedder::DIMENSIONS, vectors.first['embedding'].size + assert profile.custom_interests_embedded_at.present? + end + + def test_reuses_cached_vectors_for_unchanged_texts + profile = build_profile(['crypto', 'realty']) + cached_vector = Array.new(Recommendation::DomainEmbedder::DIMENSIONS, 0.9) + profile.update_columns(custom_interest_vectors: [{ 'text' => 'crypto', 'embedding' => cached_vector }]) + + with_feature_flag(true) do + stub_embeddings(1) # only 'realty' is missing + Recommendation::EmbedCustomInterestsJob.new.perform(profile.id) + end + + vectors = profile.reload.custom_interest_vectors.index_by { |v| v['text'] } + assert_equal cached_vector, vectors['crypto']['embedding'], 'unchanged text keeps its cached vector' + assert_equal Recommendation::DomainEmbedder::DIMENSIONS, vectors['realty']['embedding'].size + end + + def test_clears_vectors_when_no_custom_interests + profile = build_profile([]) + profile.update_columns(custom_interest_vectors: [{ 'text' => 'stale', 'embedding' => [0.1] }]) + + with_feature_flag(true) do + Recommendation::EmbedCustomInterestsJob.new.perform(profile.id) + end + + assert_equal [], profile.reload.custom_interest_vectors + assert profile.custom_interests_embedded_at.present? + end + + private + + def build_profile(custom_interests) + with_feature_flag(false) do + profile = RecommendationProfile.find_or_initialize_by(user: @user) + profile.custom_interests = custom_interests + profile.save! + profile + end + end + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end + + def stub_embeddings(count) + data = count.times.map { |i| { 'index' => i, 'embedding' => Array.new(Recommendation::DomainEmbedder::DIMENSIONS, 0.1) } } + stub_request(:post, 'https://api.openai.com/v1/embeddings') + .to_return_json(status: 200, body: { 'data' => data }, headers: {}) + end + end +end diff --git a/test/jobs/recommendation/embed_unembedded_domains_job_test.rb b/test/jobs/recommendation/embed_unembedded_domains_job_test.rb new file mode 100644 index 000000000..2f1c2994a --- /dev/null +++ b/test/jobs/recommendation/embed_unembedded_domains_job_test.rb @@ -0,0 +1,91 @@ +require 'test_helper' + +module Recommendation + class EmbedUnembeddedDomainsJobTest < ActiveJob::TestCase + def setup + super + DomainClassification.delete_all + end + + def test_no_op_when_embedding_column_missing + skip 'embedding column present' if DomainClassification.column_names.include?('embedding') + + assert_nothing_raised do + Recommendation::EmbedUnembeddedDomainsJob.new.perform + end + end + + def test_no_op_when_openai_disabled + DomainClassification.create!( + domain_name: 'pending-embed.ee', + keywords: %w[k1], + classification_source: DomainClassification::OPENAI_SOURCE, + classified_at: 1.hour.ago, + confidence: 0.9 + ) + + with_feature_flag(false) do + assert_nil Recommendation::EmbedUnembeddedDomainsJob.new.perform + end + end + + def test_scope_picks_classified_without_embedding + skip 'embedding column missing' unless DomainClassification.column_names.include?('embedding') + + target = DomainClassification.create!( + domain_name: 'embed-me.ee', + keywords: %w[cloud], + classification_source: DomainClassification::OPENAI_SOURCE, + classified_at: 1.hour.ago, + confidence: 0.9 + ) + + unclassified = DomainClassification.create!( + domain_name: 'not-classified.ee', + classification_source: DomainClassification::HEURISTIC_SOURCE + # classified_at left nil — not yet classified, should NOT be picked + ) + + ids = Recommendation::EmbedUnembeddedDomainsJob.scope.pluck(:id) + assert_includes ids, target.id + refute_includes ids, unclassified.id + end + + def test_persist_stamps_the_current_input_version + skip 'version column missing' unless DomainClassification.column_names.include?('embedding_input_version') + + DomainClassification.create!( + domain_name: 'version-me.ee', + keywords: %w[cloud], + classification_source: DomainClassification::OPENAI_SOURCE, + classified_at: 1.hour.ago, + confidence: 0.9 + ) + + with_feature_flag(true) do + stub_embeddings(1) + Recommendation::EmbedUnembeddedDomainsJob.new.perform + end + + record = DomainClassification.find_by(domain_name: 'version-me.ee') + assert_equal Recommendation::DomainEmbedder::INPUT_VERSION, record.embedding_input_version + assert_equal Recommendation::DomainEmbedder::DIMENSIONS, record.embedding.size + end + + private + + def stub_embeddings(count) + data = count.times.map { |i| { 'index' => i, 'embedding' => Array.new(Recommendation::DomainEmbedder::DIMENSIONS, 0.1) } } + stub_request(:post, 'https://api.openai.com/v1/embeddings') + .to_return_json(status: 200, body: { 'data' => data }, headers: {}) + end + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end + end +end diff --git a/test/jobs/recommendation/enrich_interest_categories_job_test.rb b/test/jobs/recommendation/enrich_interest_categories_job_test.rb new file mode 100644 index 000000000..3711738d8 --- /dev/null +++ b/test/jobs/recommendation/enrich_interest_categories_job_test.rb @@ -0,0 +1,93 @@ +require 'test_helper' + +module Recommendation + class EnrichInterestCategoriesJobTest < ActiveJob::TestCase + def setup + super + Setting.find_by(code: 'openai_model')&.update!(value: 'gpt-5') + InterestCategory.delete_all + end + + def test_noop_when_openai_disabled + with_feature_flag(false) do + assert_nil Recommendation::EnrichInterestCategoriesJob.new.perform + end + end + + def test_enriches_and_persists_active_categories + category = create_category('finance', 'Finance and fintech', 'Finants ja fintech') + + with_feature_flag(true) do + stub_enrichment(%w[finance]) + stub_embeddings(1) + Recommendation::EnrichInterestCategoriesJob.new.perform + end + + category.reload + assert_equal 'Money things.', category.description + assert_includes category.keywords, 'finants' + assert_equal Recommendation::DomainEmbedder::DIMENSIONS, category.embedding.size + assert category.embedded_at.present? + end + + def test_single_id_only_enriches_that_category + target = create_category('finance', 'Finance', 'Finants') + create_category('travel', 'Travel', 'Reisimine') + + with_feature_flag(true) do + stub_enrichment(%w[finance]) + stub_embeddings(1) + Recommendation::EnrichInterestCategoriesJob.new.perform(target.id) + end + + assert target.reload.embedded_at.present? + assert_nil InterestCategory.find_by(code: 'travel').embedded_at + end + + def test_needs_to_run_reflects_unembedded_active_categories + create_category('finance', 'Finance', 'Finants') + + with_feature_flag(true) do + assert Recommendation::EnrichInterestCategoriesJob.needs_to_run? + end + with_feature_flag(false) do + refute Recommendation::EnrichInterestCategoriesJob.needs_to_run? + end + end + + private + + def create_category(code, name_en, name_et) + InterestCategory.create!(code: code, name_en: name_en, name_et: name_et, active: true) + end + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end + + def stub_enrichment(codes) + body = { + 'choices' => [{ + 'finish_reason' => 'stop', + 'message' => { + 'content' => { + categories: codes.map { |c| { code: c, description: 'Money things.', keywords: %w[finance finants] } } + }.to_json + } + }] + } + stub_request(:post, 'https://api.openai.com/v1/chat/completions') + .to_return_json(status: 200, body: body, headers: {}) + end + + def stub_embeddings(count) + data = count.times.map { |i| { 'index' => i, 'embedding' => Array.new(Recommendation::DomainEmbedder::DIMENSIONS, 0.1) } } + stub_request(:post, 'https://api.openai.com/v1/embeddings') + .to_return_json(status: 200, body: { 'data' => data }, headers: {}) + end + end +end diff --git a/test/jobs/recommendation/prune_recommendation_events_job_test.rb b/test/jobs/recommendation/prune_recommendation_events_job_test.rb new file mode 100644 index 000000000..a60b055f1 --- /dev/null +++ b/test/jobs/recommendation/prune_recommendation_events_job_test.rb @@ -0,0 +1,52 @@ +require 'test_helper' + +module Recommendation + class PruneRecommendationEventsJobTest < ActiveJob::TestCase + def setup + super + RecommendationEvent.delete_all + end + + def test_deletes_events_older_than_default_retention + old = event('auction_detail_view', 7.months.ago) + recent = event('auction_detail_view', 3.months.ago) + + Recommendation::PruneRecommendationEventsJob.new.perform + + refute RecommendationEvent.exists?(old.id), 'event past retention should be pruned' + assert RecommendationEvent.exists?(recent.id), 'recent scoring event should be kept' + end + + def test_prunes_impressions_on_the_shorter_window + stale_impression = event('auction_impression', 2.months.ago) + fresh_impression = event('auction_impression', 2.weeks.ago) + + Recommendation::PruneRecommendationEventsJob.new.perform + + refute RecommendationEvent.exists?(stale_impression.id), 'impression past 1 month should be pruned' + assert RecommendationEvent.exists?(fresh_impression.id), 'recent impression should be kept' + end + + def test_needs_to_run_reflects_pending_rows + refute Recommendation::PruneRecommendationEventsJob.needs_to_run?, 'empty table needs no run' + + event('auction_detail_view', 7.months.ago) + assert Recommendation::PruneRecommendationEventsJob.needs_to_run? + end + + def test_batching_deletes_everything_past_retention + 3.times { event('auction_detail_view', 8.months.ago) } + + deleted = Recommendation::PruneRecommendationEventsJob.new.perform(batch_size: 2) + + assert_equal 3, deleted + assert_equal 0, RecommendationEvent.where(event_type: 'auction_detail_view').count + end + + private + + def event(type, occurred_at) + RecommendationEvent.create!(event_type: type, occurred_at: occurred_at) + end + end +end diff --git a/test/jobs/recommendation/rebuild_recommendations_job_test.rb b/test/jobs/recommendation/rebuild_recommendations_job_test.rb new file mode 100644 index 000000000..9e12bb3f4 --- /dev/null +++ b/test/jobs/recommendation/rebuild_recommendations_job_test.rb @@ -0,0 +1,39 @@ +require 'test_helper' + +module Recommendation + class RebuildRecommendationsJobTest < ActiveJob::TestCase + def test_needs_to_run_follows_feature_flag + with_feature_flag(false) do + refute Recommendation::RebuildRecommendationsJob.needs_to_run? + end + with_feature_flag(true) do + assert Recommendation::RebuildRecommendationsJob.needs_to_run? + end + end + + def test_perform_runs_pipeline_with_force + captured = nil + original = Recommendation::PipelineRunner.method(:run) + Recommendation::PipelineRunner.define_singleton_method(:run) do |force: false| + captured = force + {} + end + + Recommendation::RebuildRecommendationsJob.new.perform + + assert_equal true, captured + ensure + Recommendation::PipelineRunner.define_singleton_method(:run, original) + end + + private + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end + end +end diff --git a/test/jobs/recommendation/refresh_single_user_auction_scores_job_test.rb b/test/jobs/recommendation/refresh_single_user_auction_scores_job_test.rb new file mode 100644 index 000000000..546a3481d --- /dev/null +++ b/test/jobs/recommendation/refresh_single_user_auction_scores_job_test.rb @@ -0,0 +1,109 @@ +require 'test_helper' + +module Recommendation + class RefreshSingleUserAuctionScoresJobTest < ActiveJob::TestCase + def setup + super + @user = users(:participant) + end + + def test_enqueue_debounced_schedules_job + assert_enqueued_with( + job: Recommendation::RefreshSingleUserAuctionScoresJob, + args: [@user.id] + ) do + Recommendation::RefreshSingleUserAuctionScoresJob.enqueue_debounced(@user.id) + end + end + + def test_perform_skips_when_user_has_fresh_scores + auction = auctions(:valid_without_offers) + UserAuctionScore.create!( + user: @user, + auction: auction, + score: 10, + calculated_at: 5.seconds.ago + ) + + max_updated_at_before = UserAuctionScore.where(user: @user).maximum(:updated_at) + + Recommendation::RefreshSingleUserAuctionScoresJob.new.perform(@user.id) + + max_updated_at_after = UserAuctionScore.where(user: @user).maximum(:updated_at) + + assert_equal max_updated_at_before, max_updated_at_after, + 'job must not touch fresh user_auction_scores' + end + + def test_perform_refreshes_when_scores_are_stale + skip 'embedding column missing' unless DomainClassification.column_names.include?('embedding') + + # v3: a score row only survives a refresh when the candidate earns a magnet + # pull, so give the user one signal and the candidate a matching embedding. + give_user_a_magnet + domain = "stale-refresh-#{SecureRandom.hex(4)}.ee" + auction = Auction.create!( + domain_name: domain, + starts_at: 1.hour.ago, + ends_at: 1.day.from_now, + skip_validation: true + ) + classify(domain, Array.new(8, 1.0)) + UserAuctionScore.create!( + user: @user, + auction: auction, + score: 1, + calculated_at: 2.minutes.ago + ) + + assert_nothing_raised do + Recommendation::RefreshSingleUserAuctionScoresJob.new.perform(@user.id) + end + + reloaded = UserAuctionScore.find_by!(user: @user, auction: auction) + assert reloaded.calculated_at > 1.minute.ago, 'stale scores must be refreshed' + end + + def test_perform_reenqueues_when_recently_refreshed + auction = auctions(:valid_without_offers) + UserAuctionScore.create!( + user: @user, + auction: auction, + score: 10, + calculated_at: 5.seconds.ago + ) + + assert_enqueued_with( + job: Recommendation::RefreshSingleUserAuctionScoresJob, + args: [@user.id] + ) do + Recommendation::RefreshSingleUserAuctionScoresJob.new.perform(@user.id) + end + end + + def test_perform_no_op_for_unknown_user + assert_nothing_raised do + Recommendation::RefreshSingleUserAuctionScoresJob.new.perform(-1) + end + end + + private + + def give_user_a_magnet + history = Auction.create!( + domain_name: "history-#{SecureRandom.hex(4)}.ee", + starts_at: 2.days.ago, ends_at: 1.day.ago, skip_validation: true + ) + Offer.new(user: @user, auction: history, cents: 100, + billing_profile: billing_profiles(:private_person)).save(validate: false) + classify(history.domain_name, Array.new(8, 1.0)) + end + + def classify(domain_name, embedding) + DomainClassification.create!( + domain_name: domain_name, primary_category: 'saas', tags: %w[saas], keywords: %w[cloud], + embedding: embedding, classified_at: 1.hour.ago, confidence: 0.9 + ) + end + end +end diff --git a/test/models/domain_classification_test.rb b/test/models/domain_classification_test.rb new file mode 100644 index 000000000..a1585d18e --- /dev/null +++ b/test/models/domain_classification_test.rb @@ -0,0 +1,106 @@ +require 'test_helper' + +class DomainClassificationTest < ActiveSupport::TestCase + def test_domain_name_is_required + record = DomainClassification.new + refute record.valid? + assert_includes record.errors[:domain_name], "can't be blank" + end + + def test_domain_name_is_normalized_to_lowercase_stripped + record = DomainClassification.create!(domain_name: ' KohViK.ee ') + assert_equal 'kohvik.ee', record.domain_name + end + + def test_domain_name_is_unique + DomainClassification.create!(domain_name: 'unique.ee') + duplicate = DomainClassification.new(domain_name: 'unique.ee') + refute duplicate.valid? + end + + def test_needs_llm_enrichment_scope_picks_heuristic_rows + heuristic_row = DomainClassification.create!( + domain_name: 'fresh-heur.ee', + classification_source: DomainClassification::HEURISTIC_SOURCE, + confidence: 0.8, + classified_at: Time.current + ) + DomainClassification.create!( + domain_name: 'fresh-llm.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.95, + classified_at: 1.day.ago + ) + + assert_includes DomainClassification.needs_llm_enrichment.to_a, heuristic_row + end + + def test_needs_llm_enrichment_picks_low_confidence_rows + weak = DomainClassification.create!( + domain_name: 'weak.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.3, + classified_at: 1.day.ago + ) + assert_includes DomainClassification.needs_llm_enrichment.to_a, weak + end + + def test_needs_llm_enrichment_picks_stale_llm_rows + stale = DomainClassification.create!( + domain_name: 'stale.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.95, + classified_at: 9.months.ago + ) + assert_includes DomainClassification.needs_llm_enrichment.to_a, stale + end + + def test_needs_embedding_picks_rows_without_a_vector + skip 'embedding column missing' unless DomainClassification.column_names.include?('embedding') + + bare = DomainClassification.create!(domain_name: 'bare-embed.ee', classified_at: 1.hour.ago) + assert_includes DomainClassification.needs_embedding.to_a, bare + end + + def test_needs_embedding_picks_rows_built_under_an_older_input_version + skip 'version column missing' unless DomainClassification.column_names.include?('embedding_input_version') + + # Has a vector but NULL version (every pre-versioning row) — must be re-embedded. + legacy = DomainClassification.create!( + domain_name: 'legacy-vec.ee', + classified_at: 1.hour.ago, + embedding: Array.new(3, 0.1), + embedding_input_version: nil + ) + # Has a vector built under an explicitly older version. + older = DomainClassification.create!( + domain_name: 'older-vec.ee', + classified_at: 1.hour.ago, + embedding: Array.new(3, 0.1), + embedding_input_version: Recommendation::DomainEmbedder::INPUT_VERSION - 1 + ) + + scope = DomainClassification.needs_embedding.to_a + assert_includes scope, legacy, 'NULL version must be caught (NULL != current is unknown in SQL)' + assert_includes scope, older + end + + def test_needs_embedding_excludes_rows_at_the_current_input_version + skip 'version column missing' unless DomainClassification.column_names.include?('embedding_input_version') + + current = DomainClassification.create!( + domain_name: 'current-vec.ee', + classified_at: 1.hour.ago, + embedding: Array.new(3, 0.1), + embedding_input_version: Recommendation::DomainEmbedder::INPUT_VERSION + ) + refute_includes DomainClassification.needs_embedding.to_a, current + end + + def test_embedding_reset_attributes_nulls_the_input_version + skip 'version column missing' unless DomainClassification.column_names.include?('embedding_input_version') + + assert_includes DomainClassification.embedding_reset_attributes.keys, :embedding_input_version + assert_nil DomainClassification.embedding_reset_attributes[:embedding_input_version] + end +end diff --git a/test/models/interest_category_test.rb b/test/models/interest_category_test.rb new file mode 100644 index 000000000..4fb13e36d --- /dev/null +++ b/test/models/interest_category_test.rb @@ -0,0 +1,119 @@ +require 'test_helper' + +class InterestCategoryTest < ActiveSupport::TestCase + include ActiveJob::TestHelper + + def test_enqueues_enrichment_on_create_when_openai_enabled + with_feature_flag(true) do + assert_enqueued_with(job: Recommendation::EnrichInterestCategoriesJob) do + InterestCategory.create!(code: 'newcat', name_en: 'New', name_et: 'Uus') + end + end + end + + def test_does_not_enqueue_enrichment_when_openai_disabled + with_feature_flag(false) do + assert_no_enqueued_jobs only: Recommendation::EnrichInterestCategoriesJob do + InterestCategory.create!(code: 'newcat', name_en: 'New', name_et: 'Uus') + end + end + end + + def test_does_not_re_enqueue_on_unrelated_save + category = InterestCategory.create!(code: 'newcat', name_en: 'New', name_et: 'Uus') + category.update_columns(embedding: Array.new(3, 0.1), embedded_at: Time.current) + + with_feature_flag(true) do + assert_no_enqueued_jobs only: Recommendation::EnrichInterestCategoriesJob do + category.update!(position: 99) + end + end + end + + def test_requires_code_and_names + category = InterestCategory.new + refute category.valid? + assert category.errors.key?(:code) + assert category.errors.key?(:name_en) + assert category.errors.key?(:name_et) + end + + def test_normalizes_code_to_lowercase + category = InterestCategory.create!(code: ' SaaS ', name_en: 'SaaS', name_et: 'SaaS') + assert_equal 'saas', category.code + end + + def test_code_uniqueness_is_case_insensitive + InterestCategory.create!(code: 'saas', name_en: 'SaaS', name_et: 'SaaS') + dup = InterestCategory.new(code: 'SAAS', name_en: 'Other', name_et: 'Muu') + refute dup.valid? + end + + def test_name_is_locale_aware + category = InterestCategory.new(code: 'health', name_en: 'Health', name_et: 'Tervis') + + I18n.with_locale(:en) { assert_equal 'Health', category.name } + I18n.with_locale(:et) { assert_equal 'Tervis', category.name } + end + + def test_name_falls_back_to_english_when_estonian_blank + category = InterestCategory.new(code: 'health', name_en: 'Health', name_et: '') + I18n.with_locale(:et) { assert_equal 'Health', category.name } + end + + def test_seed_defaults_is_idempotent + InterestCategory.delete_all + + assert_difference -> { InterestCategory.count }, InterestCategory::DEFAULTS.size do + InterestCategory.seed_defaults! + end + + assert_no_difference -> { InterestCategory.count } do + InterestCategory.seed_defaults! + end + end + + def test_destroy_purges_orphaned_code_from_profiles_and_classifications + category = InterestCategory.create!(code: 'temp_cat', name_en: 'Temp', name_et: 'Temp') + + profile = users(:participant).recommendation_profile || + RecommendationProfile.create!(user: users(:participant)) + profile.update_columns(interest_keywords: %w[saas temp_cat]) # skip normalize + + classification = DomainClassification.create!( + domain_name: 'temp-cat-domain.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.9, + classified_at: 1.hour.ago, + primary_category: 'temp_cat', + tags: %w[temp_cat other] + ) + + category.destroy + + assert_equal %w[saas], profile.reload.interest_keywords + classification.reload + assert_equal %w[other], classification.tags + assert_nil classification.primary_category + end + + def test_seed_defaults_does_not_clobber_admin_edits + InterestCategory.delete_all + InterestCategory.seed_defaults! + InterestCategory.find_by(code: 'saas').update!(name_en: 'Custom SaaS') + + InterestCategory.seed_defaults! + + assert_equal 'Custom SaaS', InterestCategory.find_by(code: 'saas').name_en + end + + private + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end +end diff --git a/test/models/job_test.rb b/test/models/job_test.rb index 3e58710af..316e81f9d 100644 --- a/test/models/job_test.rb +++ b/test/models/job_test.rb @@ -12,4 +12,18 @@ def test_instance_methods_correspond_with_the_class assert_equal(InvoiceCreationJob, @instance.job_class) assert_equal(true, @instance.needs_to_run?) end + + def test_namespaced_job_is_allowed + job = Job.new('Recommendation::ClassifyUnclassifiedDomainsJob') + + assert job.valid? + assert_equal Recommendation::ClassifyUnclassifiedDomainsJob, job.job_class + end + + def test_embed_job_is_allowed + job = Job.new('Recommendation::EmbedUnembeddedDomainsJob') + + assert job.valid? + assert_equal Recommendation::EmbedUnembeddedDomainsJob, job.job_class + end end diff --git a/test/models/recommendation_profile_test.rb b/test/models/recommendation_profile_test.rb new file mode 100644 index 000000000..e78ece2b1 --- /dev/null +++ b/test/models/recommendation_profile_test.rb @@ -0,0 +1,93 @@ +require 'test_helper' + +class RecommendationProfileTest < ActiveSupport::TestCase + include ActiveJob::TestHelper + + def setup + super + @profile = RecommendationProfile.new(user: users(:participant)) + end + + def test_custom_interest_change_enqueues_embedding_when_openai_enabled + with_feature_flag(true) do + assert_enqueued_with(job: Recommendation::EmbedCustomInterestsJob) do + @profile.custom_interests = ['crypto'] + @profile.save! + end + end + end + + def test_category_only_change_does_not_enqueue_embedding + with_feature_flag(false) do + @profile.interest_categories = %w[saas] + @profile.save! + end + + with_feature_flag(true) do + assert_no_enqueued_jobs only: Recommendation::EmbedCustomInterestsJob do + @profile.interest_categories = %w[saas finance] + @profile.save! + end + end + end + + def test_no_embedding_enqueued_when_openai_disabled + with_feature_flag(false) do + assert_no_enqueued_jobs only: Recommendation::EmbedCustomInterestsJob do + @profile.custom_interests = ['crypto'] + @profile.save! + end + end + end + + def test_promptable_until_completed + assert @profile.promptable? + + @profile.completed_at = Time.current + refute @profile.promptable? + end + + def test_promptable_again_after_dismiss_interval + @profile.prompt_dismissed_at = 1.day.ago + refute @profile.promptable? + + @profile.prompt_dismissed_at = 20.days.ago + assert @profile.promptable? + end + + def test_normalizes_interest_categories + @profile.interest_categories = %w[saas legal saas numeric] + @profile.valid? + + assert_equal(%w[saas legal numeric], @profile.interest_categories) + end + + def test_stores_custom_interests_under_other + @profile.custom_interests = ['marketplace', 'marketplace', 'premium names'] + @profile.valid? + + assert_equal(['other'], @profile.interest_categories) + assert_equal(['marketplace', 'premium names'], @profile.custom_interests) + end + + def test_other_is_dropped_when_custom_interests_are_cleared + @profile.update!(interest_keywords: %w[legal other custom:crypto]) + + @profile.custom_interests = [] + @profile.valid? + + assert_empty @profile.custom_interests + # `other` is a derived marker of "has custom interests" — it must not linger. + assert_equal %w[legal], @profile.interest_categories + end + + private + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end +end diff --git a/test/models/user_test.rb b/test/models/user_test.rb index ccd0aa21c..19d124259 100644 --- a/test/models/user_test.rb +++ b/test/models/user_test.rb @@ -436,6 +436,20 @@ def test_reference_number_is_not_assigned_to_user_when_user_is_admin end + def test_user_can_accept_nested_recommendation_profile_attributes + user = boilerplate_user + user.mobile_phone = '+372500100300' + user.country_code = 'EE' + user.recommendation_profile_attributes = { + interest_categories: %w[saas], + custom_interests: ['marketplace'] + } + + assert user.save + assert_equal(%w[other saas], user.recommendation_profile.interest_categories.sort) + assert_equal(['marketplace'], user.recommendation_profile.custom_interests) + end + def boilerplate_user stub_request(:any, "https://eis_billing_system:3000/api/v1/invoice_generator/reference_number_generator") .to_return(status: 200, body: "{\"reference_number\":\"#{rand(100000..999999)}\"}", headers: {}) diff --git a/test/services/openai_structured_output_support_test.rb b/test/services/openai_structured_output_support_test.rb new file mode 100644 index 000000000..6452c2c03 --- /dev/null +++ b/test/services/openai_structured_output_support_test.rb @@ -0,0 +1,21 @@ +require 'test_helper' + +class OpenaiStructuredOutputSupportTest < ActiveSupport::TestCase + def test_returns_configured_model_when_it_supports_structured_output + assert_equal 'gpt-4o-mini', OpenaiStructuredOutputSupport.model('gpt-4o-mini') + assert_equal 'gpt-4.1', OpenaiStructuredOutputSupport.model('gpt-4.1') + assert_equal 'gpt-5', OpenaiStructuredOutputSupport.model('gpt-5') + end + + def test_falls_back_when_model_does_not_support_structured_output + assert_equal 'gpt-5', OpenaiStructuredOutputSupport.model('gpt-3.5-turbo') + end + + def test_omits_temperature_for_gpt5_family + assert_equal({}, OpenaiStructuredOutputSupport.temperature_options('gpt-5', 0.2)) + end + + def test_keeps_temperature_for_supported_non_gpt5_models + assert_equal({ temperature: 0.2 }, OpenaiStructuredOutputSupport.temperature_options('gpt-4o-mini', 0.2)) + end +end diff --git a/test/services/recommendation/domain_embedder_test.rb b/test/services/recommendation/domain_embedder_test.rb new file mode 100644 index 000000000..b19c3a495 --- /dev/null +++ b/test/services/recommendation/domain_embedder_test.rb @@ -0,0 +1,91 @@ +require 'test_helper' + +module Recommendation + class DomainEmbedderTest < ActiveSupport::TestCase + def test_returns_aligned_embeddings_for_each_row + rows = [ + { domain_name: 'a.ee', keywords: %w[one] }, + { domain_name: 'b.ee', keywords: %w[two] } + ] + + stub_embedding_request(2) + + result = Recommendation::DomainEmbedder.call(rows: rows) + assert_equal 2, result.size + assert_equal 'a.ee', result.first[:domain_name] + assert_equal Recommendation::DomainEmbedder::DIMENSIONS, result.first[:embedding].size + assert_equal Recommendation::DomainEmbedder::MODEL, result.first[:embedding_model] + assert result.first[:embedded_at].is_a?(Time) + assert_equal Recommendation::DomainEmbedder::INPUT_VERSION, result.first[:embedding_input_version] + end + + def test_build_input_includes_classification_fields + row = { + domain_name: 'petshop.ee', + description: 'An online store selling pet food.', + primary_category: 'ecommerce', + tags: %w[shop_brand pet_care], + suggested_use_cases: %w[shop marketplace], + audience: 'b2c', + keywords: %w[pets food] + } + + input = Recommendation::DomainEmbedder.new(rows: []).send(:build_input, row) + + assert_includes input, 'petshop.ee' + assert_includes input, 'An online store selling pet food.' + assert_includes input, 'Category: ecommerce' + assert_includes input, 'Tags: shop brand, pet care' # underscores humanized + assert_includes input, 'Use cases: shop, marketplace' + assert_includes input, 'Audience: b2c' + assert_includes input, 'Keywords: pets, food' + end + + def test_build_input_skips_blank_fields_without_dangling_labels + row = { domain_name: 'bare.ee', keywords: %w[minimal] } + + input = Recommendation::DomainEmbedder.new(rows: []).send(:build_input, row) + + assert_equal 'bare.ee. Keywords: minimal', input + refute_includes input, 'Category:' + refute_includes input, 'Tags:' + refute_includes input, ': .' + end + + def test_handles_active_record_rows + classification = DomainClassification.create!( + domain_name: 'ar.ee', + keywords: %w[active record] + ) + stub_embedding_request(1) + + result = Recommendation::DomainEmbedder.call(rows: [classification]) + assert_equal 'ar.ee', result.first[:domain_name] + assert_equal Recommendation::DomainEmbedder::DIMENSIONS, result.first[:embedding].size + end + + def test_empty_input_returns_empty + result = Recommendation::DomainEmbedder.call(rows: []) + assert_equal [], result + end + + def test_raises_on_openai_error + stub_request(:post, 'https://api.openai.com/v1/embeddings') + .to_return_json(status: 200, body: { 'error' => { 'message' => 'oops' } }, headers: {}) + + assert_raises(StandardError) do + Recommendation::DomainEmbedder.call(rows: [{ domain_name: 'x.ee', keywords: [] }]) + end + end + + private + + def stub_embedding_request(count) + data = count.times.map do |i| + { 'index' => i, 'embedding' => Array.new(Recommendation::DomainEmbedder::DIMENSIONS, 0.1) } + end + stub_request(:post, 'https://api.openai.com/v1/embeddings') + .to_return_json(status: 200, body: { 'data' => data, 'model' => 'text-embedding-3-small' }, headers: {}) + end + end +end diff --git a/test/services/recommendation/embedding_test.rb b/test/services/recommendation/embedding_test.rb new file mode 100644 index 000000000..cb269a0dc --- /dev/null +++ b/test/services/recommendation/embedding_test.rb @@ -0,0 +1,39 @@ +require 'test_helper' + +module Recommendation + class EmbeddingTest < ActiveSupport::TestCase + def test_cosine_similarity_of_identical_vectors_is_one + assert_in_delta 1.0, Recommendation::Embedding.cosine_similarity([1.0, 2.0, 3.0], [1.0, 2.0, 3.0]), 1e-9 + end + + def test_cosine_similarity_of_orthogonal_vectors_is_zero + assert_in_delta 0.0, Recommendation::Embedding.cosine_similarity([1.0, 0.0], [0.0, 1.0]), 1e-9 + end + + def test_cosine_similarity_is_nil_for_mismatched_sizes + assert_nil Recommendation::Embedding.cosine_similarity([1.0, 2.0], [1.0]) + end + + def test_cosine_similarity_is_nil_for_zero_magnitude_vector + assert_nil Recommendation::Embedding.cosine_similarity([0.0, 0.0], [1.0, 1.0]) + end + + def test_weighted_centroid_averages_by_weight + centroid = Recommendation::Embedding.weighted_centroid([[[0.0, 0.0], 1.0], [[4.0, 8.0], 3.0]]) + + # (0*1 + 4*3) / 4 = 3.0 ; (0*1 + 8*3) / 4 = 6.0 + assert_in_delta 3.0, centroid[0], 1e-9 + assert_in_delta 6.0, centroid[1], 1e-9 + end + + def test_weighted_centroid_is_nil_when_empty + assert_nil Recommendation::Embedding.weighted_centroid([]) + end + + def test_weighted_centroid_skips_vectors_of_a_different_size + centroid = Recommendation::Embedding.weighted_centroid([[[1.0, 1.0], 1.0], [[9.9, 9.9, 9.9], 1.0]]) + + assert_equal [1.0, 1.0], centroid + end + end +end diff --git a/test/services/recommendation/event_tracker_test.rb b/test/services/recommendation/event_tracker_test.rb new file mode 100644 index 000000000..611ec8e95 --- /dev/null +++ b/test/services/recommendation/event_tracker_test.rb @@ -0,0 +1,67 @@ +require 'test_helper' + +module Recommendation + class EventTrackerTest < ActiveSupport::TestCase + def setup + super + @user = users(:participant) + @auction_a = auctions(:valid_without_offers) + @auction_b = auctions(:valid_with_offers) + end + + def test_track_impressions_inserts_all_events_in_single_query + RecommendationEvent.where(user: @user, event_type: 'auction_impression').delete_all + + assert_difference -> { RecommendationEvent.count }, 2 do + Recommendation::EventTracker.track_impressions( + user: @user, + auctions: [@auction_a, @auction_b], + source: 'test_index' + ) + end + + events = RecommendationEvent.where(user: @user, event_type: 'auction_impression').to_a + assert_equal %w[auction_impression auction_impression], events.map(&:event_type) + assert_equal [@auction_a.id, @auction_b.id].sort, events.map(&:auction_id).sort + assert events.all? { |e| e.source == 'test_index' } + end + + def test_track_impressions_uses_single_insert_query + query_count = 0 + counter = ->(_name, _start, _finish, _id, payload) do + sql = payload[:sql].to_s + query_count += 1 if sql.start_with?('INSERT INTO "recommendation_events"') + end + + ActiveSupport::Notifications.subscribed(counter, 'sql.active_record') do + Recommendation::EventTracker.track_impressions( + user: @user, + auctions: [@auction_a, @auction_b], + source: 'test_index' + ) + end + + assert_equal 1, query_count, 'track_impressions must batch all rows into one INSERT' + end + + def test_track_impressions_skips_empty_input + assert_no_difference -> { RecommendationEvent.count } do + Recommendation::EventTracker.track_impressions( + user: @user, + auctions: [], + source: 'test_index' + ) + end + end + + def test_track_impressions_skips_nil_auctions + assert_difference -> { RecommendationEvent.count }, 1 do + Recommendation::EventTracker.track_impressions( + user: @user, + auctions: [nil, @auction_a, nil], + source: 'test_index' + ) + end + end + end +end diff --git a/test/services/recommendation/interest_catalog_test.rb b/test/services/recommendation/interest_catalog_test.rb new file mode 100644 index 000000000..a2faf351e --- /dev/null +++ b/test/services/recommendation/interest_catalog_test.rb @@ -0,0 +1,33 @@ +require 'test_helper' + +module Recommendation + class InterestCatalogTest < ActiveSupport::TestCase + def test_falls_back_to_constant_when_table_empty + InterestCategory.delete_all + assert_equal Recommendation::InterestCatalog::FALLBACK_CATEGORIES, + Recommendation::InterestCatalog.categories + end + + def test_reads_active_codes_from_db_in_position_order + InterestCategory.delete_all + InterestCategory.create!(code: 'beta', name_en: 'Beta', name_et: 'Beeta', position: 2) + InterestCategory.create!(code: 'alpha', name_en: 'Alpha', name_et: 'Alfa', position: 1) + InterestCategory.create!(code: 'hidden', name_en: 'Hidden', name_et: 'Peidetud', position: 3, active: false) + + assert_equal %w[alpha beta], Recommendation::InterestCatalog.categories + end + + def test_label_for_returns_locale_name_from_db + InterestCategory.delete_all + InterestCategory.create!(code: 'health', name_en: 'Health', name_et: 'Tervis', position: 1) + + I18n.with_locale(:en) { assert_equal 'Health', Recommendation::InterestCatalog.label_for('health') } + I18n.with_locale(:et) { assert_equal 'Tervis', Recommendation::InterestCatalog.label_for('health') } + end + + def test_label_for_unknown_code_falls_back_to_code + InterestCategory.delete_all + assert_equal 'mystery', Recommendation::InterestCatalog.label_for('mystery') + end + end +end diff --git a/test/services/recommendation/interest_category_enricher_test.rb b/test/services/recommendation/interest_category_enricher_test.rb new file mode 100644 index 000000000..64ef2cb67 --- /dev/null +++ b/test/services/recommendation/interest_category_enricher_test.rb @@ -0,0 +1,66 @@ +require 'test_helper' + +module Recommendation + class InterestCategoryEnricherTest < ActiveSupport::TestCase + def setup + super + Setting.find_by(code: 'openai_model')&.update!(value: 'gpt-5') + end + + def test_returns_description_keywords_and_embedding_per_category + categories = [ + InterestCategory.new(code: 'finance', name_en: 'Finance and fintech', name_et: 'Finants ja fintech'), + InterestCategory.new(code: 'real_estate', name_en: 'Real estate', name_et: 'Kinnisvara') + ] + + stub_enrichment_response(%w[finance real_estate]) + stub_embedding_request(2) + + result = Recommendation::InterestCategoryEnricher.call(categories: categories) + + assert_equal 2, result.size + finance = result.find { |r| r[:code] == 'finance' } + assert_equal 'Money things.', finance[:description] + assert_includes finance[:keywords], 'finants' + assert_equal Recommendation::DomainEmbedder::DIMENSIONS, finance[:embedding].size + assert_equal Recommendation::DomainEmbedder::MODEL, finance[:embedding_model] + assert finance[:embedded_at].is_a?(Time) + end + + def test_empty_input_returns_empty + assert_equal [], Recommendation::InterestCategoryEnricher.call(categories: []) + end + + def test_skips_blank_codes + categories = [InterestCategory.new(code: ' ', name_en: 'x', name_et: 'x')] + assert_equal [], Recommendation::InterestCategoryEnricher.call(categories: categories) + end + + private + + def stub_enrichment_response(codes) + body = { + 'choices' => [{ + 'finish_reason' => 'stop', + 'message' => { + 'content' => { + categories: codes.map do |code| + { code: code, description: 'Money things.', keywords: %w[finance finants loan laen] } + end + }.to_json + } + }] + } + stub_request(:post, 'https://api.openai.com/v1/chat/completions') + .to_return_json(status: 200, body: body, headers: {}) + end + + def stub_embedding_request(count) + data = count.times.map do |i| + { 'index' => i, 'embedding' => Array.new(Recommendation::DomainEmbedder::DIMENSIONS, 0.1) } + end + stub_request(:post, 'https://api.openai.com/v1/embeddings') + .to_return_json(status: 200, body: { 'data' => data, 'model' => 'text-embedding-3-small' }, headers: {}) + end + end +end diff --git a/test/services/recommendation/llm_domain_classifier_test.rb b/test/services/recommendation/llm_domain_classifier_test.rb new file mode 100644 index 000000000..ce0235ecd --- /dev/null +++ b/test/services/recommendation/llm_domain_classifier_test.rb @@ -0,0 +1,115 @@ +require 'test_helper' + +module Recommendation + class LlmDomainClassifierTest < ActiveSupport::TestCase + def setup + super + @openai_model = Setting.find_by(code: 'openai_model') + @openai_model.update!(value: 'gpt-5') + end + + def test_parses_structured_response_into_attribute_hashes + stub_request(:post, 'https://api.openai.com/v1/chat/completions') + .to_return_json(status: 200, body: ai_response_for(%w[cloudstack.ee marketflow.ee]), headers: {}) + + result = Recommendation::LlmDomainClassifier.call(domain_names: %w[cloudstack.ee marketflow.ee]) + + assert_equal 2, result.size + + cloud = result.find { |r| r[:domain_name] == 'cloudstack.ee' } + assert_equal 'saas', cloud[:primary_category] + assert_equal 'A cloudstack.ee SaaS platform for teams.', cloud[:description] + assert_equal %w[saas b2b_service], cloud[:tags] + assert_equal 'b2b', cloud[:audience] + assert_includes cloud[:keywords], 'cloud' + assert_equal DomainClassification::OPENAI_SOURCE, cloud[:classification_source] + assert cloud[:confidence] >= 0.0 + assert cloud[:confidence] <= 1.0 + end + + def test_filters_unknown_categories_from_tags + stub_request(:post, 'https://api.openai.com/v1/chat/completions') + .to_return_json(status: 200, body: ai_response_with_bogus_tag('cloudstack.ee'), headers: {}) + + result = Recommendation::LlmDomainClassifier.call(domain_names: %w[cloudstack.ee]) + assert_equal %w[saas], result.first[:tags] + assert_equal 'saas', result.first[:primary_category] + end + + def test_empty_input_returns_empty_array + result = Recommendation::LlmDomainClassifier.call(domain_names: []) + assert_equal [], result + end + + def test_truncates_above_batch_limit + huge = (1..(Recommendation::LlmDomainClassifier::BATCH_LIMIT + 25)).map { |i| "x#{i}.ee" } + classifier = Recommendation::LlmDomainClassifier.new(domain_names: huge) + assert_equal Recommendation::LlmDomainClassifier::BATCH_LIMIT, + classifier.instance_variable_get(:@domain_names).size + end + + def test_raises_on_incomplete_response + stub_request(:post, 'https://api.openai.com/v1/chat/completions') + .to_return_json(status: 200, + body: { 'choices' => [{ 'finish_reason' => 'length', 'message' => { 'content' => '{}' } }] }, + headers: {}) + + assert_raises(StandardError) do + Recommendation::LlmDomainClassifier.call(domain_names: %w[cloudstack.ee]) + end + end + + private + + def ai_response_for(domain_names) + { + 'choices' => [{ + 'finish_reason' => 'stop', + 'message' => { + 'content' => { + classifications: domain_names.map { |name| classification_for(name) } + }.to_json + } + }] + } + end + + def ai_response_with_bogus_tag(domain_name) + { + 'choices' => [{ + 'finish_reason' => 'stop', + 'message' => { + 'content' => { + classifications: [{ + domain_name: domain_name, + primary_category: 'not_a_real_category', + tags: %w[saas absolutely_made_up], + keywords: %w[cloud], + audience: 'b2b', + languages: %w[en], + suggested_use_cases: %w[platform], + brandability_score: 0.5, + confidence: 0.9 + }] + }.to_json + } + }] + } + end + + def classification_for(name) + { + domain_name: name, + description: "A #{name} SaaS platform for teams.", + primary_category: 'saas', + tags: %w[saas b2b_service], + keywords: %w[cloud platform], + audience: 'b2b', + languages: %w[en], + suggested_use_cases: %w[platform agency], + brandability_score: 0.75, + confidence: 0.92 + } + end + end +end diff --git a/test/services/recommendation/magnet_scorer_test.rb b/test/services/recommendation/magnet_scorer_test.rb new file mode 100644 index 000000000..f5bd66947 --- /dev/null +++ b/test/services/recommendation/magnet_scorer_test.rb @@ -0,0 +1,173 @@ +require 'test_helper' + +module Recommendation + # v3 shadow-mode scorer. Skipped before the embedding column is migrated so + # the suite stays green on older schemas (mirrors ScorerEmbeddingTest). + class MagnetScorerTest < ActiveSupport::TestCase + ALIGNED = Array.new(8, 1.0).freeze + ORTHOGONAL = [1.0, 0, 0, 0, 0, 0, 0, 0].freeze + + def setup + super + skip 'embedding column missing' unless DomainClassification.column_names.include?('embedding') + + @user = users(:participant) + travel_to Time.zone.parse('2026-05-27 12:00:00 UTC') + end + + def teardown + super + travel_back + end + + def test_category_magnet_ranks_aligned_domain_first + seed_category('saas', ALIGNED) + @user.create_recommendation_profile!(interest_keywords: %w[saas]) + + aligned = classified_auction('aligned.ee', ALIGNED) + orthogonal = classified_auction('orthogonal.ee', ORTHOGONAL) + + ranking = rank([aligned, orthogonal]) + + assert_equal aligned.id, ranking.first.first.id + assert ranking.first.last > ranking.last.last + end + + def test_custom_interest_vector_acts_as_a_magnet + profile = @user.create_recommendation_profile!(interest_keywords: %w[other custom:crypto]) + profile.update_columns(custom_interest_vectors: [{ 'text' => 'crypto', 'embedding' => ALIGNED }]) + + aligned = classified_auction('coin.ee', ALIGNED) + orthogonal = classified_auction('paper.ee', ORTHOGONAL) + + ranking = rank([aligned, orthogonal]) + + assert_equal aligned.id, ranking.first.first.id + end + + def test_wishlist_domain_acts_as_a_magnet + add_wishlist('wish.ee') + classify('wish.ee', ALIGNED) + + aligned = classified_auction('similar-to-wish.ee', ALIGNED) + orthogonal = classified_auction('unrelated.ee', ORTHOGONAL) + + ranking = rank([aligned, orthogonal]) + + assert_equal aligned.id, ranking.first.first.id, + 'a domain semantically close to a wishlisted domain must rank above an orthogonal one' + assert ranking.first.last > ranking.last.last + end + + def test_bid_history_acts_as_a_magnet + history = ended_auction('history.ee') + place_offer(history) + classify('history.ee', ALIGNED) + + aligned = classified_auction('similar.ee', ALIGNED) + orthogonal = classified_auction('different.ee', ORTHOGONAL) + + ranking = rank([aligned, orthogonal]) + + assert_equal aligned.id, ranking.first.first.id + end + + def test_score_is_nil_without_auction_embedding + seed_category('saas', ALIGNED) + @user.create_recommendation_profile!(interest_keywords: %w[saas]) + + bare = classified_auction('bare.ee', nil) + + scores = Recommendation::MagnetScorer.new(user: @user, scope: Auction.where(id: bare.id)).scores + assert_nil scores[bare.id] + end + + def test_score_is_nil_without_any_magnet + @user.recommendation_profile&.destroy + candidate = classified_auction('lonely.ee', ALIGNED) + + scores = Recommendation::MagnetScorer.new(user: @user, scope: Auction.where(id: candidate.id)).scores + assert_nil scores[candidate.id] + end + + def test_defaults_scope_to_active_auctions + seed_category('saas', ALIGNED) + @user.create_recommendation_profile!(interest_keywords: %w[saas]) + classified_auction('default-scope.ee', ALIGNED) + + # No scope: argument — must fall back to Auction.active without raising. + assert_nothing_raised do + Recommendation::MagnetScorer.new(user: @user).ranked(limit: 1) + end + end + + private + + def rank(auctions) + Recommendation::MagnetScorer + .new(user: @user, scope: Auction.where(id: auctions.map(&:id))) + .ranked + end + + def seed_category(code, embedding) + category = InterestCategory.find_or_create_by!(code: code) do |c| + c.name_en = code.upcase + c.name_et = code.upcase + c.active = true + end + category.update_columns(embedding: embedding, embedded_at: Time.current) + category + end + + def classified_auction(domain_name, embedding) + auction = Auction.create!( + domain_name: domain_name, + starts_at: 1.hour.ago, + ends_at: 1.day.from_now, + classification_tags: %w[saas], + primary_category: 'saas', + skip_validation: true + ) + classify(domain_name, embedding) unless embedding.nil? + auction + end + + def ended_auction(domain_name) + Auction.create!( + domain_name: domain_name, + starts_at: 2.days.ago, + ends_at: 1.day.ago, + skip_validation: true + ) + end + + def classify(domain_name, embedding) + DomainClassification.create!( + domain_name: domain_name, + primary_category: 'saas', + tags: %w[saas], + keywords: %w[cloud], + embedding: embedding, + classified_at: 1.hour.ago, + confidence: 0.9 + ) + end + + def add_wishlist(domain_name) + item = WishlistItem.new(user: @user, domain_name: domain_name) + item.save(validate: false) + item + end + + def place_offer(auction, cents: 100) + offer = Offer.new( + user: @user, + auction: auction, + cents: cents, + billing_profile: billing_profiles(:private_person) + ) + offer.save(validate: false) + offer + end + end +end diff --git a/test/services/recommendation/pipeline_runner_test.rb b/test/services/recommendation/pipeline_runner_test.rb new file mode 100644 index 000000000..7b29e7fe4 --- /dev/null +++ b/test/services/recommendation/pipeline_runner_test.rb @@ -0,0 +1,154 @@ +require 'test_helper' + +module Recommendation + class PipelineRunnerTest < ActiveSupport::TestCase + def test_no_op_when_openai_disabled + with_feature_flag(false) do + assert_equal({}, Recommendation::PipelineRunner.run) + end + end + + def test_force_invalidates_openai_classifications + classification = DomainClassification.create!( + domain_name: 'force-me.ee', + classification_source: DomainClassification::OPENAI_SOURCE, + confidence: 0.9, + classified_at: 1.hour.ago + ) + + with_feature_flag(true) do + with_no_pending do + with_placeholder_prompt do + stub_scorer do + Recommendation::PipelineRunner.run(force: true) + end + end + end + end + + assert classification.reload.classified_at < 6.months.ago, + 'force must push openai classifications past the staleness threshold' + end + + def test_skips_ai_score_when_prompt_is_placeholder + with_feature_flag(true) do + with_no_pending do + with_placeholder_prompt do + stub_scorer do + refute_ai_sorting_called do + summary = Recommendation::PipelineRunner.run + assert_equal :skipped, summary[:ai_score] + end + end + end + end + end + end + + def test_backfills_custom_interest_vectors_for_existing_profiles + skip 'column missing' unless RecommendationProfile.column_names.include?('custom_interest_vectors') + + profile = with_feature_flag(false) do + p = RecommendationProfile.create!(user: users(:participant), interest_keywords: %w[other custom:crypto]) + p.update_columns(custom_interest_vectors: [], custom_interests_embedded_at: nil) + p + end + + with_feature_flag(true) do + with_no_pending do + with_placeholder_prompt do + stub_scorer do + stub_embeddings(1) + Recommendation::PipelineRunner.run + end + end + end + end + + profile.reload + assert profile.custom_interests_embedded_at.present?, 'existing custom interests must get embedded on init' + assert_equal 1, profile.custom_interest_vectors.size + end + + def test_scores_every_participant + with_feature_flag(true) do + with_no_pending do + with_placeholder_prompt do + stub_scorer do + Recommendation::PipelineRunner.run + end + end + end + end + + expected = User.where('? = ANY (roles)', User::PARTICIPANT_ROLE).pluck(:id).sort + assert_equal expected, @scored.sort + end + + private + + def with_feature_flag(enabled) + original = Feature.method(:open_ai_integration_enabled?) + Feature.define_singleton_method(:open_ai_integration_enabled?) { enabled } + yield + ensure + Feature.define_singleton_method(:open_ai_integration_enabled?, original) + end + + # Make both batch stages see an empty backlog so they don't hit the LLM. + def with_no_pending + classify = Recommendation::ClassifyUnclassifiedDomainsJob + embed = Recommendation::EmbedUnembeddedDomainsJob + originals = { + scope: classify.method(:scope), + missing: classify.method(:missing_domains), + embed_scope: embed.method(:scope) + } + classify.define_singleton_method(:scope) { DomainClassification.none } + classify.define_singleton_method(:missing_domains) { [] } + embed.define_singleton_method(:scope) { DomainClassification.none } + yield + ensure + classify.define_singleton_method(:scope, originals[:scope]) + classify.define_singleton_method(:missing_domains, originals[:missing]) + embed.define_singleton_method(:scope, originals[:embed_scope]) + end + + def with_placeholder_prompt + setting = Setting.find_by(code: 'openai_domains_evaluation_prompt') + original = setting.value + setting.update!(value: Recommendation::PipelineRunner::AI_SCORE_PLACEHOLDER_PROMPT) + yield + ensure + setting.update!(value: original) + end + + def stub_scorer + original = Recommendation::Scorer.method(:refresh_for) + @scored = [] + collected = @scored + Recommendation::Scorer.define_singleton_method(:refresh_for) do |user:, **| + collected << user.id + end + yield + ensure + Recommendation::Scorer.define_singleton_method(:refresh_for, original) + end + + def stub_embeddings(count) + data = count.times.map { |i| { 'index' => i, 'embedding' => Array.new(Recommendation::DomainEmbedder::DIMENSIONS, 0.1) } } + stub_request(:post, 'https://api.openai.com/v1/embeddings') + .to_return_json(status: 200, body: { 'data' => data }, headers: {}) + end + + def refute_ai_sorting_called + called = false + original = ActiveAuctionsAiSortingJob.method(:perform_now) + ActiveAuctionsAiSortingJob.define_singleton_method(:perform_now) { |*| called = true } + yield + refute called, 'ai_score job must not run when the prompt is the placeholder' + ensure + ActiveAuctionsAiSortingJob.define_singleton_method(:perform_now, original) + end + end +end diff --git a/test/services/recommendation/scorer_test.rb b/test/services/recommendation/scorer_test.rb new file mode 100644 index 000000000..934495aff --- /dev/null +++ b/test/services/recommendation/scorer_test.rb @@ -0,0 +1,117 @@ +require 'test_helper' + +module Recommendation + # v3 Scorer: score = magnet_base × MAGNET_SCALE + structural nudges. Skipped + # before the embedding column is migrated (magnet base needs it). + class ScorerTest < ActiveSupport::TestCase + ALIGNED = Array.new(8, 1.0).freeze + ORTHOGONAL = [1.0, 0, 0, 0, 0, 0, 0, 0].freeze + + def setup + super + skip 'embedding column missing' unless DomainClassification.column_names.include?('embedding') + + @user = users(:participant) + travel_to Time.zone.parse('2026-05-27 12:00:00 UTC') + give_user_a_magnet + end + + def teardown + super + travel_back + end + + def test_writes_scaled_magnet_score_ranking_aligned_above_orthogonal + aligned = classified_auction('aligned.ee', ALIGNED) + orthogonal = classified_auction('orthogonal.ee', ORTHOGONAL) + + Recommendation::Scorer.refresh_for(user: @user, scope: Auction.where(id: [aligned.id, orthogonal.id])) + + aligned_score = UserAuctionScore.find_by!(user: @user, auction: aligned).score + orthogonal_score = UserAuctionScore.find_by!(user: @user, auction: orthogonal).score + + assert aligned_score > orthogonal_score + # magnet base (~3.0) scaled by 100 dominates the score. + assert aligned_score > 100, "Expected scaled magnet score, got #{aligned_score}" + end + + def test_no_row_written_when_candidate_has_no_embedding + bare = classified_auction('bare.ee', nil) + + count = Recommendation::Scorer.refresh_for(user: @user, scope: Auction.where(id: bare.id)) + + assert_equal 0, count + assert_nil UserAuctionScore.find_by(user: @user, auction: bare) + end + + def test_stale_row_deleted_when_magnet_disappears + candidate = classified_auction('lonely.ee', ALIGNED) + remove_user_magnets # user no longer has any signal → no pull + + UserAuctionScore.create!(user: @user, auction: candidate, score: 42, calculated_at: 1.day.ago) + + Recommendation::Scorer.refresh_for(user: @user, scope: Auction.where(id: candidate.id)) + + assert_nil UserAuctionScore.find_by(user: @user, auction: candidate), + 'stale score row should be deleted once the auction no longer earns a magnet score' + end + + def test_structural_length_bonus_applied_on_top_of_magnet + profile = @user.recommendation_profile || @user.create_recommendation_profile! + profile.update!(preferred_length_min: 1, preferred_length_max: 3) + short = classified_auction('abc.ee', ALIGNED) # normalized 'abc' → length 3, in range + long = classified_auction('abcdefgh.ee', ALIGNED) + + Recommendation::Scorer.refresh_for(user: @user, scope: Auction.where(id: [short.id, long.id])) + + short_score = UserAuctionScore.find_by!(user: @user, auction: short).score + long_score = UserAuctionScore.find_by!(user: @user, auction: long).score + + assert_in_delta Recommendation::Scorer::LENGTH_MATCH_BONUS, (short_score - long_score).to_f, 0.01 + end + + def test_features_version_marker_present + auction = classified_auction('fv.ee', ALIGNED) + Recommendation::Scorer.refresh_for(user: @user, scope: Auction.where(id: auction.id)) + score = UserAuctionScore.find_by!(user: @user, auction: auction) + + assert_equal Recommendation::Scorer::FEATURES_VERSION, score.features_version + assert_equal Recommendation::Scorer::SCORER_NAME, score.scorer_name + end + + private + + # A single behavioural magnet in the ALIGNED direction (a past bid on an + # embedded domain), so ALIGNED candidates get a strong pull. + def give_user_a_magnet + history = Auction.create!( + domain_name: 'history.ee', starts_at: 2.days.ago, ends_at: 1.day.ago, skip_validation: true + ) + Offer.new(user: @user, auction: history, cents: 100, + billing_profile: billing_profiles(:private_person)).save(validate: false) + classify('history.ee', ALIGNED) + end + + def remove_user_magnets + Offer.where(user: @user).delete_all + @user.wishlist_items.delete_all + @user.recommendation_profile&.update_columns(interest_keywords: [], custom_interest_vectors: []) + end + + def classified_auction(domain_name, embedding) + auction = Auction.create!( + domain_name: domain_name, starts_at: 1.hour.ago, ends_at: 1.day.from_now, + classification_tags: %w[saas], primary_category: 'saas', skip_validation: true + ) + classify(domain_name, embedding) unless embedding.nil? + auction + end + + def classify(domain_name, embedding) + DomainClassification.create!( + domain_name: domain_name, primary_category: 'saas', tags: %w[saas], keywords: %w[cloud], + embedding: embedding, classified_at: 1.hour.ago, confidence: 0.9 + ) + end + end +end diff --git a/test/services/recommendation/text_embedder_test.rb b/test/services/recommendation/text_embedder_test.rb new file mode 100644 index 000000000..c4df90562 --- /dev/null +++ b/test/services/recommendation/text_embedder_test.rb @@ -0,0 +1,30 @@ +require 'test_helper' + +module Recommendation + class TextEmbedderTest < ActiveSupport::TestCase + def test_returns_vectors_aligned_to_input_order + stub_request(:post, 'https://api.openai.com/v1/embeddings') + .to_return_json(status: 200, body: { + 'data' => [ + { 'index' => 1, 'embedding' => [0.2, 0.2] }, + { 'index' => 0, 'embedding' => [0.1, 0.1] } + ] + }, headers: {}) + + result = Recommendation::TextEmbedder.embed(%w[first second]) + + assert_equal [[0.1, 0.1], [0.2, 0.2]], result + end + + def test_empty_input_returns_empty_without_calling_openai + assert_equal [], Recommendation::TextEmbedder.embed([]) + end + + def test_raises_on_openai_error + stub_request(:post, 'https://api.openai.com/v1/embeddings') + .to_return_json(status: 200, body: { 'error' => { 'message' => 'boom' } }, headers: {}) + + assert_raises(StandardError) { Recommendation::TextEmbedder.embed(%w[x]) } + end + end +end