fix: apply Algolia settings regardless of clear_index value

[PedroAntunesCosta]

Summary

Fixes a critical bug where Algolia index settings (including attributesForFaceting) were not being applied when clear_index: false, causing configuration to be reset on every scrape when new documentation is added.

Problem

When clear_index is set to false in the config, the scraper copies the production index to a temporary index but skips applying the settings from the config JSON. This means:

• Custom settings from custom_settings in config files were ignored
• Index configuration like attributesForFaceting would get lost
• Each scrape would use whatever settings were previously on the index, not the config

Solution

Modified scraper/src/algolia_helper.py to call set_settings() in both branches (when clear_index is true OR false), ensuring configuration from JSON files is always applied consistently.

Changes

• File: scraper/src/algolia_helper.py
  • Added self.algolia_index_tmp.set_settings(settings) to the else block (line 36)
  • Now settings are applied regardless of clear_index value

Impact

• Index configuration will remain consistent across scrapes
• attributesForFaceting and other custom settings will be properly maintained
• Config JSON files become the true source of truth for index settings