Improve ranking engine for token efficiency

[coderooz]

Context:
Ranking system does not consider token cost.

Problem:
Long or low-value entries may be returned.

Tasks:

• Boost high-importance entries
• Penalize long content
• Prioritize recent and frequently accessed entries

Goal:
Return fewer, higher-quality results.