From a949d1b6967a64e4e27efba61e57500c6aded7bd Mon Sep 17 00:00:00 2001 From: Mario Date: Mon, 30 Mar 2026 14:24:28 -0500 Subject: [PATCH 01/55] docs: Insights from ReAct paper --- docs/learnings.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/learnings.md b/docs/learnings.md index af791f7..b4df38d 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -7,6 +7,31 @@ ## Semana 1 — Setup + RAG básico +### Día 1 **Fecha:** 30/03/2026 +#### REACT +- Chain of Taought como estrategia para prompting es una caja negra estática ya que el MODELO USA SUS REPRESENTACIONES INTERNAS PARA GENERAR EL PENSAMIENTO Y NO LO ALIMENTA DEL MUNDO EXTERIOR. ReAct propone "Razonar para actuar" al tiempo que se "Actúa para razonar". Los modelos de tipo "acción" carecen de la capacidad de llevar objetivos de alto nivel o complejos por lo que es difícil un reflexión profunda. + +- Idea central REACT: Aumentar el espacio de acción A del agente de manera que A = A U L, donde L es el espacio del lenguaje. Una acción en L será referida como "pensamiento" o "traza de razonamiento". Esta acción en el espacio L no afecta el exterior y por tanto no obtiene observación como feedback. Lo que hace es obtener información para razonar sobre el contexto c_t y actualizarlo. + +- Modelo: PalM-540B con pesos congelados es solicitado with few-shot in-context examples + +- Se abordaron tipos de pensamiento como: descomponer metas, inyectar conocimiento de sentido común, extraer partes importantes, rastrear el progreso y manejar excepciones. Además, en función de la pregunta, se tienen razonamiento para tareas de razonmiento en cada paso (pensar-accion-observacion) o pensamientos dispersos para tareas de toma de decisiones (los pensamientos solo están en posiicones relevantes de la trayectoria). Durante las pruebas, un humano podía ir modificanco y controlando el pensameinto del modelo + +- Los resultados demostraron una mejor trabajo con ReAct VS Act, especialmente sintetizando la respuesta final + +- ReAct VS CoT: Mejor en HotpotQA, y levemente inferior en Fever + - Alucinaciones es un problema serio en CoT + - Si bien la intercalación de pasos de razonamiento, acción y observación mejora la solidez y la fiabilidad de ReAct, dicha restricción estructural también reduce su flexibilidad a la hora de formular pasos de razonamiento, lo que da lugar a una tasa de errores de razonamiento mayor que la de CoT. Observamos que existe un patrón de error frecuente específico de ReAct, en el que el modelo genera repetidamente los pensamientos y acciones anteriores, y lo clasificamos como parte de los «errores de razonamiento», ya que el modelo no logra razonar sobre cuál es la siguiente acción adecuada a tomar y salir del bucle. + - La recuperación de información es crítica para ReAct: Cuanod no la obitiene se descarrilla el razonamiento y le cuesta recuperarse + - En el fine tunning, con la estrategia ReAct se obtuvo significativamente mejor desempeño que haciendo ajuste fino con las otras estrategias + +Insights: +1. Costo de la Autonomía: La flexibilidad de ReAct tiene un "impuesto" de tokens y tiempo. Úsalo solo cuando el camino a la respuesta no sea previsible. + +2. Observación como Correctivo: La gran ventaja de ReAct no es que "piense mejor", sino que "escucha" lo que el mundo (las herramientas) le devuelve y corrige su rumbo. + +3. Determinismo vs. Agencia: Si la tarea es clasificar (PQRS), el determinismo del RAG gana. Si la tarea es diagnosticar/decidir (Pensiones), la agencia de ReAct es superior. + **Fecha:** _[completar]_ ### ¿Qué aprendí? From 432733e119889ed3c56ba0a9be99027726327542 Mon Sep 17 00:00:00 2001 From: Mario Date: Mon, 30 Mar 2026 16:42:08 -0500 Subject: [PATCH 02/55] feat(llm): implement AnthropicLLM provider with generate and stream --- notebooks/001-jmmz-LLMProvider.ipynb | 219 ++++++++++++++++++ pyproject.toml | 1 + src/researchos/__init__.py | 3 + src/researchos/config.py | 2 + src/researchos/domain/__init__.py | 3 + .../infrastructure/llm/anthropic_llm.py | 66 ++++++ 6 files changed, 294 insertions(+) create mode 100644 notebooks/001-jmmz-LLMProvider.ipynb create mode 100644 src/researchos/infrastructure/llm/anthropic_llm.py diff --git a/notebooks/001-jmmz-LLMProvider.ipynb b/notebooks/001-jmmz-LLMProvider.ipynb new file mode 100644 index 0000000..9db2e71 --- /dev/null +++ b/notebooks/001-jmmz-LLMProvider.ipynb @@ -0,0 +1,219 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "2e7d5420", + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "source": [ + "Tarea 1: Cliente de Claude API (infrastructure/llm/anthropic.py)\n", + "Implementa la clase que cumple el Protocol LLMProvider de domain/interfaces.py. Debe poder recibir una lista de Message y devolver una respuesta. Usa el SDK de Anthropic (anthropic package). Usa settings.default_model para el modelo y settings.anthropic_api_key para la key. Escribe un test unitario con el MockLLMProvider que ya tienes en conftest, y un test de integración que haga una llamada real a la API (con @pytest.mark.integration).\n", + "\n", + "Tarea 2: Cliente de arXiv API (infrastructure/data/arxiv.py)\n", + "arXiv tiene una API REST gratuita que devuelve XML. Implementa una función que reciba un query string (ej. \"LLM agents\") y un número máximo de resultados, haga la request con httpx, parsee el XML, y devuelva una lista de Paper (tu modelo de dominio). El endpoint es http://export.arxiv.org/api/query. Escribe un test de integración que busque 3 papers sobre \"LLM agents\" y verifique que devuelve objetos Paper válidos.\n", + "\n", + "Tarea 3: Descarga y extracción de PDFs (application/services/ingestion_service.py)\n", + "Crea la función de ingesta que toma un Paper, descarga su PDF usando paper.pdf_url, extrae el texto con PyMuPDF (fitz), y devuelve el texto crudo. No hagas chunking todavía — eso es tarea de semana 2. Solo descarga + extracción de texto. Guarda los PDFs en data/papers/ (crea la carpeta si no existe, agrégala a .gitignore)." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "dd622e22", + "metadata": {}, + "outputs": [], + "source": [ + "# Use this initial code to work in the notebook as if it were a module, that\n", + "# is, to be able to export classes and functions from other subpackages.\n", + "\n", + "import os\n", + "import sys\n", + "\n", + "package_path = os.path.abspath(\".\").split(os.sep + \"notebooks\")[0]\n", + "if package_path not in sys.path:\n", + " sys.path.append(package_path)\n", + "\n", + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "markdown", + "id": "1445c7b8", + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "source": [ + "# Task 1" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6879f46f", + "metadata": {}, + "outputs": [], + "source": [ + "from collections.abc import AsyncIterator\n", + "from typing import Protocol\n", + "\n", + "\n", + "# Protocolo estableciod en domain/interfaces\n", + "class LLMProvider(Protocol):\n", + " \"\"\"Contract for any LLM provider (Claude, Gemini, etc.).\"\"\"\n", + "\n", + " async def generate(self, messages: list[Message]) -> str:\n", + " \"\"\"Generate a response from a list of messages.\"\"\"\n", + " ...\n", + "\n", + " async def stream(self, messages: list[Message]) -> AsyncIterator[str]:\n", + " \"\"\"Stream a response token by token.\"\"\"\n", + " ...\n", + "\n", + "\n", + "from anthropic import AsyncAnthropic\n", + "from src.researchos.domain import Message\n", + "\n", + "from researchos.config import settings\n", + "from researchos.domain.exceptions import GenerationError\n", + "\n", + "\n", + "class AnthropicLLM:\n", + " \"\"\"Contract for Claude provider.\"\"\"\n", + "\n", + " def __init__(self):\n", + " self.client = AsyncAnthropic(api_key=settings.anthropic_api_key)\n", + " self.model_id = settings.default_model\n", + " self.temperature = settings.temperature\n", + " self.max_tokens = settings.max_tokens\n", + "\n", + " def _format_messages(self, messages: list[Message]) -> tuple[str | None, list[dict]]:\n", + " \"\"\"\n", + " Separa el system prompt (si existe) y formatea los mensajes\n", + " para el esquema que espera Anthropic.\n", + " \"\"\"\n", + " system_prompt = None\n", + " formatted = []\n", + "\n", + " for msg in messages:\n", + " if msg.role == \"system\":\n", + " system_prompt = msg.content\n", + " else:\n", + " formatted.append({\"role\": msg.role, \"content\": msg.content})\n", + "\n", + " return system_prompt, formatted\n", + "\n", + " async def generate(self, messages: list[Message]) -> str:\n", + " \"\"\"Generate a response from a list of messages.\"\"\"\n", + "\n", + " system, formatted_msgs = self._format_messages(messages)\n", + "\n", + " response = await self.client.messages.create(\n", + " model=self.model_id,\n", + " max_tokens=self.max_tokens,\n", + " system=system or \"\",\n", + " messages=formatted_msgs,\n", + " temperature=self.temperature,\n", + " )\n", + "\n", + " if response.content and len(response.content) > 0:\n", + " return response.content[0].text\n", + " else:\n", + " raise GenerationError(\"Claude returned empty response\")\n", + "\n", + " async def stream(self, messages: list[Message]) -> AsyncIterator[str]:\n", + " \"\"\"Stream a response token by token.\"\"\"\n", + "\n", + " system, formatted_msgs = self._format_messages(messages)\n", + "\n", + " async with self.client.messages.stream(\n", + " model=self.model_id,\n", + " max_tokens=self.max_tokens,\n", + " system=system or \"\",\n", + " messages=formatted_msgs,\n", + " temperature=self.temperature,\n", + " ) as stream:\n", + " async for text in stream.text_stream:\n", + " yield text" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3f7e601d", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "901381aa", + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "source": [ + "# Task 2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cd6474ab", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cce908aa", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7e141a0a", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3398c559", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "researchos", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/pyproject.toml b/pyproject.toml index 3ef3a39..d3ca78f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -46,6 +46,7 @@ packages = ["src/researchos"] target-version = "py311" line-length = 100 src = ["src", "tests"] +exclude = ["notebooks/"] [tool.ruff.lint] select = ["E", "W", "F", "I", "UP", "B", "SIM", "RUF"] diff --git a/src/researchos/__init__.py b/src/researchos/__init__.py index 4b7a33d..231c21c 100644 --- a/src/researchos/__init__.py +++ b/src/researchos/__init__.py @@ -1,3 +1,6 @@ """ResearchOS — Open-source research engine powered by LLMs.""" __version__ = "0.1.0" + +from .domain import Document as Document +from .domain import Message as Message diff --git a/src/researchos/config.py b/src/researchos/config.py index ddf0386..216bf96 100644 --- a/src/researchos/config.py +++ b/src/researchos/config.py @@ -26,6 +26,8 @@ class Settings(BaseSettings): anthropic_api_key: str = "" default_model: str = "claude-sonnet-4-20250514" fast_model: str = "claude-haiku-4-5-20251001" + temperature: float = 0.5 + max_tokens: int = 1024 # ── Vector store ── vector_store: Literal["chroma", "vertex"] = "chroma" diff --git a/src/researchos/domain/__init__.py b/src/researchos/domain/__init__.py index a15f135..00f9fd6 100644 --- a/src/researchos/domain/__init__.py +++ b/src/researchos/domain/__init__.py @@ -11,3 +11,6 @@ - All other layers depend on Domain, never the reverse - NEVER import from application/ or infrastructure/ here """ + +from .models import Document as Document +from .models import Message as Message diff --git a/src/researchos/infrastructure/llm/anthropic_llm.py b/src/researchos/infrastructure/llm/anthropic_llm.py new file mode 100644 index 0000000..f441396 --- /dev/null +++ b/src/researchos/infrastructure/llm/anthropic_llm.py @@ -0,0 +1,66 @@ +from collections.abc import AsyncIterator + +from anthropic import AsyncAnthropic + +from researchos.config import settings +from researchos.domain.exceptions import GenerationError +from researchos.domain.models import Message + + +class AnthropicLLM: + """Contract for Claude provider.""" + + def __init__(self): + self.client = AsyncAnthropic(api_key=settings.anthropic_api_key) + self.model_id = settings.default_model + self.temperature = settings.temperature + self.max_tokens = settings.max_tokens + + def _format_messages(self, messages: list[Message]) -> tuple[str | None, list[dict]]: + """ + Separa el system prompt (si existe) y formatea los mensajes + para el esquema que espera Anthropic. + """ + system_prompt = None + formatted = [] + + for msg in messages: + if msg.role == "system": + system_prompt = msg.content + else: + formatted.append({"role": msg.role, "content": msg.content}) + + return system_prompt, formatted + + async def generate(self, messages: list[Message]) -> str: + """Generate a response from a list of messages.""" + + system, formatted_msgs = self._format_messages(messages) + + response = await self.client.messages.create( + model=self.model_id, + max_tokens=self.max_tokens, + system=system or "", + messages=formatted_msgs, + temperature=self.temperature, + ) + + if response.content and len(response.content) > 0: + return response.content[0].text + else: + raise GenerationError("Claude returned empty response") + + async def stream(self, messages: list[Message]) -> AsyncIterator[str]: + """Stream a response token by token.""" + + system, formatted_msgs = self._format_messages(messages) + + async with self.client.messages.stream( + model=self.model_id, + max_tokens=self.max_tokens, + system=system or "", + messages=formatted_msgs, + temperature=self.temperature, + ) as stream: + async for text in stream.text_stream: + yield text From c9cb181211cc9c9b695d3048327a4d0a3dfab44e Mon Sep 17 00:00:00 2001 From: Mario Date: Mon, 30 Mar 2026 16:43:20 -0500 Subject: [PATCH 03/55] test(llm): add unit and integration tests for AnthropicLLM --- tests/integration/test_anthropic_llm.py | 33 ++++++++++ .../unit/infrastructure/test_anthropic_llm.py | 66 +++++++++++++++++++ 2 files changed, 99 insertions(+) create mode 100644 tests/integration/test_anthropic_llm.py create mode 100644 tests/unit/infrastructure/test_anthropic_llm.py diff --git a/tests/integration/test_anthropic_llm.py b/tests/integration/test_anthropic_llm.py new file mode 100644 index 0000000..fc4dccc --- /dev/null +++ b/tests/integration/test_anthropic_llm.py @@ -0,0 +1,33 @@ +import pytest +from src.researchos.domain import Message +from src.researchos.infrastructure.llm.anthropic_llm import AnthropicLLM + + +@pytest.mark.integration +@pytest.mark.asyncio +async def test_anthropic_generate_integration(): + llm = AnthropicLLM() + + # 4. Ejecutar + msgs = [Message(role="user", content="Hola, este es mi primer llamado")] + + result = await llm.generate(messages=msgs) + + # 5. Verificar + assert isinstance(result, str) + assert len(result) > 0 + + +@pytest.mark.integration +@pytest.mark.asyncio +async def test_anthropic_stream_integration(): + llm = AnthropicLLM() + + msgs = [Message(role="user", content="Di hola en una palabra")] + + result = "" + async for token in llm.stream(messages=msgs): + result += token + + assert isinstance(result, str) + assert len(result) > 0 diff --git a/tests/unit/infrastructure/test_anthropic_llm.py b/tests/unit/infrastructure/test_anthropic_llm.py new file mode 100644 index 0000000..120028c --- /dev/null +++ b/tests/unit/infrastructure/test_anthropic_llm.py @@ -0,0 +1,66 @@ +from unittest.mock import AsyncMock, MagicMock + +import pytest +from src.researchos.domain import Message +from src.researchos.infrastructure.llm.anthropic_llm import AnthropicLLM + + +@pytest.mark.unit +@pytest.mark.asyncio +async def test_anthropic_generate_unit(): + # 1. Crear la estructura falsa que devuelve Anthropic + fake_content_block = MagicMock() + fake_content_block.text = "respuesta falsa" + + fake_response = MagicMock() + fake_response.content = [fake_content_block] + + # 2. Crear el cliente falso + mock_client = MagicMock() + mock_client.messages.create = AsyncMock(return_value=fake_response) + + # 3. Instanciar AnthropicLLM e inyectar el cliente falso + llm = AnthropicLLM() + llm.client = mock_client + + # 4. Ejecutar + msgs = [Message(role="user", content="Hola, este es mi primer llamado")] + + result = await llm.generate(messages=msgs) + + # 5. Verificar + assert isinstance(result, str) + assert result == "respuesta falsa" + assert len(llm.client.messages.create.call_args_list) == 1 + + +@pytest.mark.unit +@pytest.mark.asyncio +async def test_anthropic_stream_unit(): + # 1. Simular el objeto stream con text_stream iterable + async def fake_text_stream(): + for token in ["hola ", "mundo "]: + yield token + + mock_stream = MagicMock() + mock_stream.text_stream = fake_text_stream() + mock_stream.__aenter__ = AsyncMock(return_value=mock_stream) + mock_stream.__aexit__ = AsyncMock(return_value=False) + + # 2. Crear el cliente falso + mock_client = MagicMock() + mock_client.messages.stream.return_value = mock_stream + + # 3. Instanciar AnthropicLLM e inyectar el cliente falso + llm = AnthropicLLM() + llm.client = mock_client + + # 4. Ejecutar y acumular tokens + msgs = [Message(role="user", content="Hola")] + result = "" + async for token in llm.stream(messages=msgs): + result += token + + # 5. Verificar + assert isinstance(result, str) + assert len(result) > 0 From 6c4bfb19277375cf486d4da964d1779327df6fd4 Mon Sep 17 00:00:00 2001 From: Mario Date: Mon, 30 Mar 2026 16:55:25 -0500 Subject: [PATCH 04/55] docs: Summary of key points from March 30, 2026 --- docs/learnings.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/docs/learnings.md b/docs/learnings.md index b4df38d..5b184e1 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -32,6 +32,41 @@ Insights: 3. Determinismo vs. Agencia: Si la tarea es clasificar (PQRS), el determinismo del RAG gana. Si la tarea es diagnosticar/decidir (Pensiones), la agencia de ReAct es superior. +### Lo que construí +Cliente de Claude API (`AnthropicLLM`) que implementa el Protocol `LLMProvider` con dos métodos: `generate()` para respuestas completas y `stream()` para respuestas token a token. + +### Conceptos aprendidos + +**async/await** +`async def` declara una función que puede pausarse. `await` es el punto de pausa: el event loop atiende otras tareas mientras espera la respuesta externa. Sin async, el programa se bloquea esperando. + +**AsyncMock vs MagicMock** +- `MagicMock` — simula objetos y atributos síncronos +- `AsyncMock` — simula funciones async (las que necesitan `await`) +- Para `async with` hay que mockear `__aenter__` y `__aexit__` +- Para `async for` se necesita un generador async (`async def` + `yield`), no un `iter()` normal + +**Tests unitarios vs integración** +- Unitario: sin llamadas reales, cliente reemplazado por mock, rápido, sin costo +- Integración: llamada real a la API, marcado con `@pytest.mark.integration`, consume tokens +- `make test` corre solo `@pytest.mark.unit`. `make test-all` corre todo. + +**pre-commit** +Guardián que corre antes de cada commit. Si encuentra errores autocorregibles, los corrige y bloquea el commit. Solo hay que volver a hacer el commit con los archivos ya corregidos. + +**Claude Pro vs Anthropic API** +Son productos separados. Claude Pro cubre claude.ai (interfaz web). La API requiere créditos independientes en console.anthropic.com. + +### Decisiones técnicas +- `_format_messages()` separa el system prompt de los mensajes de conversación antes de enviar a la API +- Re-exports explícitos (`Document as Document`) requeridos por ruff para imports públicos en `__init__.py` +- Notebooks excluidos del linting de ruff en `pyproject.toml` + +### ¿Qué no entendí bien? +- Cuándo usar async-await: debo reforzar este concepto porque veo que está muy rlacionado con el uso de APIs +- Protocol: Entiendo que es más como una maqueta que le dice a python que el método debe cumplir X cosas: Eso hace que cuando alguien quiera implementar un nuevo proveedor, mínimamente debe ajustarse al contrato? +- test: el uso de mocks es complejo, seguir profundizando y tal vez buscar hacer ejercicios? + **Fecha:** _[completar]_ ### ¿Qué aprendí? From f7824073acfda245344a2facb9de3f6dcf5f88a6 Mon Sep 17 00:00:00 2001 From: Mario Date: Wed, 1 Apr 2026 16:23:22 -0500 Subject: [PATCH 05/55] feat(domain): align with clean-agents-template (PromptTemplate, AgentInput/Output, agent_utils) - Add PromptTemplate class to domain/prompts/__init__.py (str.format, no Jinja2) - Add AgentInput and AgentOutput models to domain/models.py - Add load_history_and_append() and wrap_output() to agent_utils.py --- .claude/settings.local.json | 7 + .github/workflows/ci.yml | 36 +++++ .github/workflows/deploy.yml | 20 +++ docs/runbooks/incident_response.md | 146 ++++++++++++++++++ .../application/agents/agent_utils.py | 109 +++++++++++-- .../application/workflows/__init__.py | 1 + src/researchos/domain/models.py | 16 ++ src/researchos/domain/prompts/__init__.py | 79 +++++++++- 8 files changed, 399 insertions(+), 15 deletions(-) create mode 100644 .claude/settings.local.json create mode 100644 .github/workflows/ci.yml create mode 100644 .github/workflows/deploy.yml create mode 100644 docs/runbooks/incident_response.md create mode 100644 src/researchos/application/workflows/__init__.py diff --git a/.claude/settings.local.json b/.claude/settings.local.json new file mode 100644 index 0000000..48a5ea0 --- /dev/null +++ b/.claude/settings.local.json @@ -0,0 +1,7 @@ +{ + "permissions": { + "allow": [ + "Read(//c/Users/mario/Documentos/personal_projects/becomes_ai_engineer/clean-agents-template/**)" + ] + } +} diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..388ef61 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,36 @@ +name: CI + +on: + push: + branches: [main, develop] + pull_request: + branches: [main] + +jobs: + test: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Install uv + uses: astral-sh/setup-uv@v3 + with: + version: "latest" + + - name: Set up Python 3.11 + run: uv python install 3.11 + + - name: Install dependencies + run: uv sync --all-extras + + - name: Lint + run: uv run ruff check src/ tests/ + + - name: Format check + run: uv run ruff format --check src/ tests/ + + - name: Run unit tests + run: uv run pytest -v -m unit + # Integration tests are excluded from CI by default. + # Add a separate job with service containers when you're ready. diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml new file mode 100644 index 0000000..6123cd6 --- /dev/null +++ b/.github/workflows/deploy.yml @@ -0,0 +1,20 @@ +name: Deploy + +on: + push: + branches: [main] + tags: ["v*"] + +jobs: + deploy: + runs-on: ubuntu-latest + # TODO: Implement the deploy job for your chosen platform. + # Examples: + # - Google Cloud Run: use google-github-actions/deploy-cloudrun + # - AWS ECS: use aws-actions/amazon-ecs-deploy-task-definition + # - Azure Container Apps: use azure/container-apps-deploy-action + # - Fly.io: use superfly/flyctl-actions + steps: + - uses: actions/checkout@v4 + - name: Deploy (not yet implemented) + run: echo "Configure your deployment target in .github/workflows/deploy.yml" diff --git a/docs/runbooks/incident_response.md b/docs/runbooks/incident_response.md new file mode 100644 index 0000000..47c85ac --- /dev/null +++ b/docs/runbooks/incident_response.md @@ -0,0 +1,146 @@ +# Incident Response Runbook — ResearchOS + +> **Owner:** +> **Last reviewed:** +> **Severity levels:** S1 (service down) · S2 (degraded) · S3 (minor) · S4 (cosmetic) + +--- + +## 1. Triage — First 5 minutes + +| Step | Action | +|------|--------| +| 1 | **Acknowledge** the alert in the on-call channel. | +| 2 | Open the monitoring dashboard (`/monitoring/tracer.py` traces or your APM). | +| 3 | Classify severity (S1–S4) based on user impact. | +| 4 | If S1/S2 → start a dedicated incident channel and page the lead. | +| 5 | Post an initial status update to stakeholders. | + +--- + +## 2. LLM Provider Outage (S1) + +**Symptoms:** API calls to Claude return `5xx`, timeouts spike, agent responses stop arriving. + +### Diagnosis + +```bash +# Review recent error logs +make logs | grep -i "LLMProvider\|anthropic" | tail -50 +``` + +### Mitigation + +1. **Activate fallback provider** (if configured): + - Update `configs/.yaml` → `llm.model` to the backup model. + - Redeploy or restart the service: + ```bash + docker compose restart api + ``` +2. **If no fallback is available:** + - Enable **graceful degradation mode**: return a cached / templated response + explaining temporary unavailability. +3. **Communicate** estimated recovery time to stakeholders. + +### Recovery + +- Monitor Anthropic status page until resolved. +- Verify with a smoke test: + ```bash + curl -X POST http://localhost:8000/agent/ask \ + -H "Content-Type: application/json" \ + -d '{"question": "ping"}' + ``` + +--- + +## 3. Prompt Degradation / Hallucinations (S2) + +**Symptoms:** Agent accuracy drops sharply, users report incorrect or invented +information, evaluation scores fall below threshold. + +### Diagnosis + +```bash +# Run the evaluation suite against the current model + prompts +python scripts/evaluate_agent.py --config configs/production.yaml +``` + +### Mitigation + +1. **Rollback prompts** if a recent prompt change is the suspected cause: + ```bash + git log --oneline -- src/researchos/domain/prompts/ + git checkout -- src/researchos/domain/prompts/ + ``` +2. Redeploy and re-run evaluations to confirm improvement. + +### Recovery + +- Update the prompt test suite (`tests/unit/domain/test_prompts.py`) with the + failing case so it doesn't recur. + +--- + +## 4. Cost Spike / Runaway Agent (S1) + +**Symptoms:** Billing alerts fire, token usage jumps 10×+, a single agent +invocation generates an unusually long chain of LLM calls. + +### Mitigation + +1. **Kill the runaway process immediately:** + ```bash + docker compose stop api + ``` +2. **Set hard limits** in `.env`: + - `LLM_MAX_TOKENS_PER_REQUEST` + - Loop guard in `application/agents/agent_utils.py` + +--- + +## 5. Memory / Vector Store Corruption (S2) + +**Symptoms:** Agent returns stale or contradictory context, retrieval scores +drop, Chroma returns empty results. + +### Mitigation + +1. **Re-ingest documents** from the source of truth: + ```bash + python scripts/ingest_documents.py --config configs/production.yaml --force + ``` + +--- + +## 6. Post-Incident — After Every S1/S2 + +| Step | Owner | Deadline | +|------|-------|----------| +| Write a blameless post-mortem | Incident lead | +48 hours | +| Identify action items with owners | Team | +48 hours | +| Update this runbook if a new scenario was discovered | On-call | +1 week | +| Add missing monitoring / alerts | Infra | +1 sprint | + +--- + +## Quick Reference — Key Commands + +```bash +# Health check +curl http://localhost:8000/health + +# Restart service +docker compose restart api + +# View logs (real-time) +docker compose logs -f api + +# Run evaluation suite +python scripts/evaluate_agent.py --config configs/production.yaml + +# Re-ingest documents +python scripts/ingest_documents.py --config configs/production.yaml --force +``` + +> **Remember:** Update this runbook every time you handle an incident. diff --git a/src/researchos/application/agents/agent_utils.py b/src/researchos/application/agents/agent_utils.py index 836598a..4a49ceb 100644 --- a/src/researchos/application/agents/agent_utils.py +++ b/src/researchos/application/agents/agent_utils.py @@ -1,16 +1,32 @@ -"""Agent utilities — Shared functions for agents via composition. +"""Shared agent utilities — building blocks for agents via composition. -Instead of a base class with inheritance, agents import and use these -functions as building blocks. This is more flexible and testable. +Instead of a base class with inheritance, agents import and compose these +functions. This pattern keeps each agent self-contained, independently +testable, and free from hidden behavior inherited from a parent class. -Usage: - from researchos.application.agents.agent_utils import retrieve_and_generate +All functions here: + - Accept domain interfaces (Protocols) as parameters, never concrete types. + - Return domain models (Pydantic), never raw dicts or SDK objects. + - Are async to support non-blocking I/O throughout the call stack. - answer = await retrieve_and_generate(query, llm, store, system_prompt) +How to use: + Import only the functions your agent needs:: + + from researchos.application.agents.agent_utils import ( + retrieve_context, + build_rag_messages, + retrieve_and_generate, + load_history_and_append, + wrap_output, + ) + +How to extend: + Add new utility functions here when the same pattern appears in two or + more agents. Do not add agent-specific logic here — keep it in the agent. """ -from researchos.domain.interfaces import LLMProvider, VectorStore -from researchos.domain.models import Document, Message +from researchos.domain.interfaces import LLMProvider, MemoryStore, VectorStore +from researchos.domain.models import AgentOutput, Document, Message async def retrieve_context( @@ -18,7 +34,16 @@ async def retrieve_context( store: VectorStore, top_k: int = 5, ) -> list[Document]: - """Retrieve relevant documents from the vector store.""" + """Retrieve relevant documents from the vector store for a given query. + + Args: + query: The search string to use for retrieval. + store: A VectorStore implementation (injected, never instantiated here). + top_k: Number of documents to retrieve. + + Returns: + List of documents ranked by relevance score, descending. + """ return await store.search(query, k=top_k) @@ -27,10 +52,17 @@ def build_rag_messages( documents: list[Document], system_prompt: str, ) -> list[Message]: - """Build a message list for RAG: system prompt + context + query.""" - context = "\n\n".join( - f"[{i + 1}] {doc.text}" for i, doc in enumerate(documents) - ) + """Build the message list for a RAG call: system prompt + context + query. + + Args: + query: The user's question. + documents: Retrieved documents to use as context. + system_prompt: The rendered system prompt for the agent. + + Returns: + A two-element list: [system message, user message with embedded context]. + """ + context = "\n\n".join(f"[{i + 1}] {doc.text}" for i, doc in enumerate(documents)) return [ Message(role="system", content=system_prompt), Message( @@ -47,7 +79,56 @@ async def retrieve_and_generate( system_prompt: str, top_k: int = 5, ) -> str: - """Full RAG pattern: retrieve documents, build context, generate answer.""" + """Execute the full RAG pattern: retrieve → build context → generate. + + This is the most common single-turn agent pattern. For multi-turn + conversations or tool-use loops, build a dedicated agent function instead. + + Args: + query: The user's question. + llm: An LLMProvider implementation (injected). + store: A VectorStore implementation (injected). + system_prompt: The rendered system prompt for the agent. + top_k: Number of documents to retrieve. + + Returns: + The LLM's response as a plain string. + """ docs = await retrieve_context(query, store, top_k) messages = build_rag_messages(query, docs, system_prompt) return await llm.generate(messages) + + +async def load_history_and_append( + session_id: str, + new_message: Message, + memory: MemoryStore, +) -> list[Message]: + """Load conversation history and append the new message. + + Use this when building multi-turn agents that need conversational context. + + Args: + session_id: The session identifier for the conversation. + new_message: The message to append to the history. + memory: A MemoryStore implementation (injected). + + Returns: + The full conversation history including the new message. + """ + history = await memory.get(session_id) + await memory.append(session_id, new_message) + return [*history, new_message] + + +def wrap_output(answer: str, sources: list[Document] | None = None) -> AgentOutput: + """Wrap a raw LLM response string into a typed AgentOutput. + + Args: + answer: The raw text response from the LLM. + sources: Optional list of documents used as context. + + Returns: + A typed AgentOutput ready to return from the agent function. + """ + return AgentOutput(answer=answer, sources=sources or []) diff --git a/src/researchos/application/workflows/__init__.py b/src/researchos/application/workflows/__init__.py new file mode 100644 index 0000000..bed4250 --- /dev/null +++ b/src/researchos/application/workflows/__init__.py @@ -0,0 +1 @@ +"""Application workflows — multi-agent / multi-service orchestration.""" diff --git a/src/researchos/domain/models.py b/src/researchos/domain/models.py index 4f72c6a..4d492c8 100644 --- a/src/researchos/domain/models.py +++ b/src/researchos/domain/models.py @@ -75,3 +75,19 @@ class GeneratedAnswer(BaseModel): sources: list[SearchResult] = Field(default_factory=list) model_used: str = "" tokens_used: int = 0 + + +class AgentInput(BaseModel): + """Input payload for an agent invocation.""" + + query: str = Field(description="The user's request or question") + session_id: str = Field(default="default") + metadata: dict = Field(default_factory=dict) + + +class AgentOutput(BaseModel): + """Typed output from an agent, with optional source attribution.""" + + answer: str + sources: list[Document] = Field(default_factory=list) + metadata: dict = Field(default_factory=dict) diff --git a/src/researchos/domain/prompts/__init__.py b/src/researchos/domain/prompts/__init__.py index be164a2..546730b 100644 --- a/src/researchos/domain/prompts/__init__.py +++ b/src/researchos/domain/prompts/__init__.py @@ -1 +1,78 @@ -"""Prompt templates — versionable .txt files with str.format() rendering.""" +"""Prompt template loader for ResearchOS. + +Prompts are stored as plain .txt files using Python str.format() syntax +and organized into subdirectories by purpose: + + domain/prompts/ + ├── system/ ← system prompts that define agent persona/behavior + └── tasks/ ← task-specific prompts (extraction, summarization, etc.) + +No external dependencies — only Python stdlib. This keeps domain/ pure. + +Variables in templates use str.format() syntax: {variable_name} + +Keeping prompts as .txt files means they are: + - Versionable and diffable in git + - Editable without touching Python code + - Testable independently from the rest of the system + +Usage: + from researchos.domain.prompts import PromptTemplate + + template = PromptTemplate("tasks", "extraction") + rendered = template.render(topic="LLM agents", paper_text="...") +""" + +from pathlib import Path + +from researchos.domain.exceptions import PromptNotFoundError + +_PROMPTS_DIR = Path(__file__).parent + + +class PromptTemplate: + """Loads and renders a prompt template from a .txt file. + + Templates live in subdirectories under domain/prompts/: + - system/ -> agent system prompts + - tasks/ -> task-specific prompts + + Variables in templates use Python str.format() syntax: {variable_name} + + Example: + template = PromptTemplate("tasks", "extraction") + rendered = template.render(topic="LLM agents", paper_text="...") + + Args: + category: Subdirectory name (e.g., "system", "tasks"). + name: Template filename without the .txt extension. + + Raises: + PromptNotFoundError: If the template file does not exist. + """ + + def __init__(self, category: str, name: str) -> None: + self._path = _PROMPTS_DIR / category / f"{name}.txt" + if not self._path.exists(): + raise PromptNotFoundError( + f"Prompt template not found: {category}/{name}.txt. " + f"Expected file at: {self._path}" + ) + self._template = self._path.read_text(encoding="utf-8") + + def render(self, **kwargs: str) -> str: + """Render the template with the provided variables. + + Args: + **kwargs: Variables referenced in the template as {variable_name}. + + Returns: + The rendered prompt string. + + Raises: + KeyError: If a variable referenced in the template is missing from kwargs. + """ + return self._template.format(**kwargs) if kwargs else self._template + + def __repr__(self) -> str: + return f"PromptTemplate('{self._path.relative_to(_PROMPTS_DIR)}')" From 14d1114709880ebef39b210244dac7eb73614929 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 9 Apr 2026 16:30:59 -0500 Subject: [PATCH 06/55] docs: Update CLAUDE.md with GIT workflow, add learning and add new ROADMAP.md to work --- CLAUDE.md | 93 +++++++++++++++++++++++++++++++++++++++++++++++ ROADMAP.md | 20 ++++++++++ docs/learnings.md | 19 +++++++++- 3 files changed, 131 insertions(+), 1 deletion(-) create mode 100644 ROADMAP.md diff --git a/CLAUDE.md b/CLAUDE.md index ef55899..4bae313 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -97,3 +97,96 @@ make format # ruff format make run-api # FastAPI server make run-bot # Telegram bot ``` + +--- + +## Adding a dependency + +```bash +uv add # Producción +uv add --dev # Solo desarrollo +``` + +Siempre agregar el campo correspondiente en `config.py` y `.env.example` +cuando el SDK requiera credenciales. + +--- + +## Work log y Resumen de Jornada + +El registro histórico del proyecto vive en `docs/work_log.md` en la raíz del repositorio. + +**Protocolo de cierre de jornada:** +Cuando el usuario solicite un resumen de la jornada de trabajo (o use comandos similares como "Genera el resumen del día...", "Resume el trabajo que hicimos ..."), DEBES ejecutar este flujo exacto para actualizar el archivo `work_log.md`: + +1. **Analizar el Contexto Manual:** Extrae y resume cualquier información explícita que el usuario te haya dado en ese mismo prompt (ej. reuniones externas, investigación paralela, conversaciones con otros modelos). +2. **Analizar la Sesión de Claude Code:** Revisa tu propia memoria de la sesión actual: ¿Qué archivos de la Clean Architecture exploramos? ¿Qué problemas de código o dependencias resolvimos juntos? ¿Qué nuevas implementaciones se desarrollaron? ¿Qué tareas quedaron pendientes? +3. **Analizar el Repositorio (Git):** Usa tus herramientas de terminal para revisar los commits de las últimas 24 horas (`git log --since="1 day ago"`) y los cambios actuales sin commitear (`git status` o `git diff`). +4. **Redactar y Guardar:** Crea una nueva entrada al final de `work_log.md` con la fecha de hoy. El formato DEBE ser: + + ### [Fecha en formato YYYY-MM-DD] + - **Contexto del Desarrollador:** [Resumen del input manual del usuario] + - **Trabajo con Claude Code:** [Resumen de los archivos tocados, bugs arreglados o lógica discutida en la sesión] + - **Historial de Git:** [Resumen de los commits realizados y estado actual del repo] + - **Tareas Pendientes:** [Resumen de las tareas pendientes para trabajar la próxima sesión] + +--- + +## Git workflow + +### Commit cadence +- Claude Code debe sugerir hacer commit al finalizar cada tarea lógica completa, + no cada archivo modificado. +- Una "tarea lógica" es: un feature implementado, un test que pasa, un bug + corregido, un refactor terminado, una sección de docs completa. +- Al terminar una tarea, Claude debe decir: "Esta tarea está completa. + Sugiero commit: ``. ¿Procedo?" +- Claude NUNCA hace commit automático sin confirmación del usuario. + +### Commit message format +Utiliza Conventional Commits. Los mensajes deben estar escritos en inglés. + + **Format:** `(): ` + + **Types:** + - `feat` — new feature or capability + - `fix` — bug fix + - `refactor` — code change that neither fixes a bug nor adds a feature + - `test` — adding or updating tests + - `docs` — documentation only + - `chore` — tooling, dependencies, CI, config + - `style` — formatting, whitespace (no logic change) + - `perf` — performance improvement + + **Scopes** match the project's architectural layers and components. + Use lowercase, one word. Common scopes for agent projects: + `domain`, `application`, `infrastructure`, `llm`, `retrieval`, `memory`, + `api`, `agent`, `prompts`, `config`, `deps`, `ci`, `tests`, `docs`. + + **Examples:** + feat(llm): add streaming support to provider + fix(retrieval): handle empty search results + refactor(agent): switch from inheritance to composition + test(domain): add unit tests for Protocol implementations + docs(architecture): add ADR for prompt loading decision + chore(deps): upgrade pydantic to v2.9 + +### Commit body (optional) +Utiliza el cuerpo del texto para explicar **por qué**, no **qué**. El «diff» muestra el «qué». +Deja una línea en blanco entre el título y el cuerpo del texto. Ejemplo en triple backticks: + + ``` + refactor(agent): switch from inheritance to composition + + Base class was creating coupling between ResearchAgent and PQRSAgent + because streaming behavior differed. Composition via agent_utils.py + keeps each agent self-contained. + ``` + +#### Cadencia de commits +Commit por tarea lógica, no por archivo. Si implementas AnthropicLLM y sus tests, es UN commit con ambos archivos, no dos. Si implementas el cliente arXiv y además arreglas un typo en el README, son DOS commits separados (feat + docs). La regla es: un commit debe poder revertirse sin romper otras cosas y debe tener un propósito claro. +Para tu flujo típico de desarrollo, apunta a 3-6 commits por jornada de trabajo. Menos de eso y los commits son demasiado grandes (difíciles de revisar); más y son micro-commits que ensucian el historial. + +#### Dos reglas adicionales importantes: +1. Primera: el mensaje de commit se escribe en inglés aunque el código del proyecto tenga comentarios en español. Es convención estándar en la industria y te ayuda a mantener profesionalismo en el repo. +2. Segunda: el cuerpo del mensaje (opcional, después del título) se usa para explicar el por qué, no el qué. El diff ya muestra el qué. Si la decisión no es obvia, explícala en el cuerpo diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..c3529e6 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,20 @@ +# ResearchOS — Roadmap de aprendizaje + +## Abril 2026 — V1 pipeline RAG básico +- [x] T1: AnthropicLLM provider (completado 30 mar) +- [ ] T2: Cliente arXiv API +- [ ] T3: Servicio de ingesta PDFs +- [ ] T4: Chunking fijo +- [ ] T5: Integración end-to-end + +## Mayo 2026 — V1 hybrid search y evaluación +- [ ] T6: BM25 retrieval +- [ ] T7: Hybrid search +- [ ] T8: Reranker básico +- [ ] T9: Dataset de evaluación (20 preguntas) +- [ ] T10: Script de evaluación + +## Junio 2026 — V2 LangGraph +- [ ] T11: Refactor a LangGraph +- [ ] T12: Primer briefing matutino +- [ ] T13: Comparativa V1 vs V2 \ No newline at end of file diff --git a/docs/learnings.md b/docs/learnings.md index 5b184e1..a2d2de6 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -7,7 +7,7 @@ ## Semana 1 — Setup + RAG básico -### Día 1 **Fecha:** 30/03/2026 +### **Fecha:** 30/03/2026 #### REACT - Chain of Taought como estrategia para prompting es una caja negra estática ya que el MODELO USA SUS REPRESENTACIONES INTERNAS PARA GENERAR EL PENSAMIENTO Y NO LO ALIMENTA DEL MUNDO EXTERIOR. ReAct propone "Razonar para actuar" al tiempo que se "Actúa para razonar". Los modelos de tipo "acción" carecen de la capacidad de llevar objetivos de alto nivel o complejos por lo que es difícil un reflexión profunda. @@ -67,6 +67,23 @@ Son productos separados. Claude Pro cubre claude.ai (interfaz web). La API requi - Protocol: Entiendo que es más como una maqueta que le dice a python que el método debe cumplir X cosas: Eso hace que cuando alguien quiera implementar un nuevo proveedor, mínimamente debe ajustarse al contrato? - test: el uso de mocks es complejo, seguir profundizando y tal vez buscar hacer ejercicios? +**Fecha:** 09/04/2026 + +### ¿Qué aprendí? +- En la creación de los repos hermanos debo customizar el CLAUDE.md para que sepa hacia dónde apunta el proyecto y sus pormenores +- También aprendía acerca de .pre-commit y solucioné algunos inconvenientes con su uso, entendí que usa ruff y linter para mantener el código limpio y ordenado +- Establecí una rutina para hacer commits que se basa en hacer cerca de 4 a 6 commits diarios de manera que el avance sea continuo pero contenido, y además se estableció Conventional Commits con una estructura (): y se incluyó en los CLAUDE.md para que el asistente de código ayude a hacer commits y avice cuando note que ya es hora. +- Aprendía sobre uv y su uso para la gestión de dependecias: actualmente es un estandar en python porque permite mantener ambienestes aislados, disminuye el consumo de recursos ya que trabaja como apuntador a librerías que ya se han descargado en lugar de descargar cada una en el ambiente particular; y como bonues es mucho más rápido que la estrategia pip + venv. + +### Que no entendí bien +- Aún tengo dudas sobre el uso de arquitectura limpia y sus beneficios +- También debo de ahondar en cuál es la importnacia de establecer modelos y protocolos que además están aislados de la infraestructura + +### Decisiones de diseño +- Se modificó el CLAUDE.md +- Se replanteó el desarrollo de múltiples proyectos al tiempo +- Hay que actualizar algunas cosas en el template clean-agents-template (NO URGENTE) + **Fecha:** _[completar]_ ### ¿Qué aprendí? From 777e0531c6455b9533ab53f99f482425abfac6b2 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 9 Apr 2026 16:35:24 -0500 Subject: [PATCH 07/55] test: Test pre-commit tootl --- docs/learnings.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/learnings.md b/docs/learnings.md index a2d2de6..48c12d5 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -84,6 +84,9 @@ Son productos separados. Claude Pro cubre claude.ai (interfaz web). La API requi - Se replanteó el desarrollo de múltiples proyectos al tiempo - Hay que actualizar algunas cosas en el template clean-agents-template (NO URGENTE) +### Errores interesantes +- Pensé que no podía trabajar con uv en el server pero descrubrí que sí + **Fecha:** _[completar]_ ### ¿Qué aprendí? From e2b1b2bf117f84db93e34c752e5e6db170a4c927 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 9 Apr 2026 16:36:42 -0500 Subject: [PATCH 08/55] update: update pre-commit file to no commit to master or certifications brances --- .pre-commit-config.yaml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 1bafc4a..49b1f65 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -12,6 +12,9 @@ repos: - id: trailing-whitespace - id: end-of-file-fixer - id: check-yaml + - id: check-toml - id: check-added-large-files args: ['--maxkb=500'] - id: detect-private-key + - id: no-commit-to-branch + args: ['--branch', 'master', '--branch', 'main', '--branch', 'certification'] From a90db6e0bf116cba2bec5783f966c6181520392a Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 9 Apr 2026 16:54:01 -0500 Subject: [PATCH 09/55] docs: update workflow to end session --- CLAUDE.md | 75 ++++++++++++++++++++++++++---------------------- docs/work_log.md | 0 2 files changed, 41 insertions(+), 34 deletions(-) create mode 100644 docs/work_log.md diff --git a/CLAUDE.md b/CLAUDE.md index 4bae313..2343be0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,6 +2,12 @@ > Read automatically by Claude Code at the start of every session. +## Project Overview +ResearchOS is an open-source research engine that monitors scientific sources +(arXiv, PubMed, RSS), answers questions in natural language, and sends +personalized morning briefings. Primary channel: Telegram bot. +Target domains: ML/AI, health/biomedicine, tech news. + ## Current Phase - **Version:** V1 — RAG Robusto + Telegram @@ -103,48 +109,48 @@ make run-bot # Telegram bot ## Adding a dependency ```bash -uv add # Producción -uv add --dev # Solo desarrollo +uv add # Production +uv add --dev # Development only ``` -Siempre agregar el campo correspondiente en `config.py` y `.env.example` -cuando el SDK requiera credenciales. +Always add the corresponding field in `config.py` and `.env.example` +when the SDK requires credentials. --- -## Work log y Resumen de Jornada +## Work Log and Session Summary -El registro histórico del proyecto vive en `docs/work_log.md` en la raíz del repositorio. +The project's historical record lives in `docs/work_log.md` at the repo root. -**Protocolo de cierre de jornada:** -Cuando el usuario solicite un resumen de la jornada de trabajo (o use comandos similares como "Genera el resumen del día...", "Resume el trabajo que hicimos ..."), DEBES ejecutar este flujo exacto para actualizar el archivo `work_log.md`: +**End-of-session protocol:** +When the user requests a session summary (or uses similar commands like "Generate today's summary...", "Summarize the work we did..."), you MUST execute this exact flow to update `work_log.md`: -1. **Analizar el Contexto Manual:** Extrae y resume cualquier información explícita que el usuario te haya dado en ese mismo prompt (ej. reuniones externas, investigación paralela, conversaciones con otros modelos). -2. **Analizar la Sesión de Claude Code:** Revisa tu propia memoria de la sesión actual: ¿Qué archivos de la Clean Architecture exploramos? ¿Qué problemas de código o dependencias resolvimos juntos? ¿Qué nuevas implementaciones se desarrollaron? ¿Qué tareas quedaron pendientes? -3. **Analizar el Repositorio (Git):** Usa tus herramientas de terminal para revisar los commits de las últimas 24 horas (`git log --since="1 day ago"`) y los cambios actuales sin commitear (`git status` o `git diff`). -4. **Redactar y Guardar:** Crea una nueva entrada al final de `work_log.md` con la fecha de hoy. El formato DEBE ser: +1. **Analyze Manual Context:** Extract and summarize any explicit information the user provided in that same prompt (e.g., external meetings, parallel research, conversations with other models). +2. **Analyze the Claude Code Session:** Review your own memory of the current session: Which Clean Architecture files did we explore? What code or dependency problems did we solve together? What new implementations were developed? What tasks remain pending? +3. **Analyze the Repository (Git):** Use your terminal tools to review commits from the last 24 hours (`git log --since="1 day ago"`) and current uncommitted changes (`git status` or `git diff`). +4. **Draft and Save:** Create a new entry at the end of `work_log.md` with today's date. The format MUST be: - ### [Fecha en formato YYYY-MM-DD] - - **Contexto del Desarrollador:** [Resumen del input manual del usuario] - - **Trabajo con Claude Code:** [Resumen de los archivos tocados, bugs arreglados o lógica discutida en la sesión] - - **Historial de Git:** [Resumen de los commits realizados y estado actual del repo] - - **Tareas Pendientes:** [Resumen de las tareas pendientes para trabajar la próxima sesión] + ### [Date in YYYY-MM-DD format] + - **Developer Context:** [Summary of the user's manual input] + - **Work with Claude Code:** [Summary of files touched, bugs fixed, or logic discussed in the session] + - **Git History:** [Summary of commits made and current repo state] + - **Pending Tasks:** [Summary of tasks pending for the next session] --- -## Git workflow +## Git Workflow ### Commit cadence -- Claude Code debe sugerir hacer commit al finalizar cada tarea lógica completa, - no cada archivo modificado. -- Una "tarea lógica" es: un feature implementado, un test que pasa, un bug - corregido, un refactor terminado, una sección de docs completa. -- Al terminar una tarea, Claude debe decir: "Esta tarea está completa. - Sugiero commit: ``. ¿Procedo?" -- Claude NUNCA hace commit automático sin confirmación del usuario. +- Claude Code should suggest a commit after each complete logical task, + not after each modified file. +- A "logical task" is: an implemented feature, a passing test, a fixed bug, + a finished refactor, a completed docs section. +- When finishing a task, Claude should say: "This task is complete. + Suggested commit: ``. Shall I proceed?" +- Claude NEVER commits automatically without user confirmation. ### Commit message format -Utiliza Conventional Commits. Los mensajes deben estar escritos en inglés. +Use Conventional Commits. Messages must be written in English. **Format:** `(): ` @@ -172,8 +178,8 @@ Utiliza Conventional Commits. Los mensajes deben estar escritos en inglés. chore(deps): upgrade pydantic to v2.9 ### Commit body (optional) -Utiliza el cuerpo del texto para explicar **por qué**, no **qué**. El «diff» muestra el «qué». -Deja una línea en blanco entre el título y el cuerpo del texto. Ejemplo en triple backticks: +Use the body to explain **why**, not **what**. The diff already shows the what. +Leave a blank line between the title and the body. Example: ``` refactor(agent): switch from inheritance to composition @@ -183,10 +189,11 @@ Deja una línea en blanco entre el título y el cuerpo del texto. Ejemplo en tri keeps each agent self-contained. ``` -#### Cadencia de commits -Commit por tarea lógica, no por archivo. Si implementas AnthropicLLM y sus tests, es UN commit con ambos archivos, no dos. Si implementas el cliente arXiv y además arreglas un typo en el README, son DOS commits separados (feat + docs). La regla es: un commit debe poder revertirse sin romper otras cosas y debe tener un propósito claro. -Para tu flujo típico de desarrollo, apunta a 3-6 commits por jornada de trabajo. Menos de eso y los commits son demasiado grandes (difíciles de revisar); más y son micro-commits que ensucian el historial. +#### Commit cadence +One commit per logical task, not per file. If you implement AnthropicLLM and its tests, that is ONE commit with both files, not two. If you implement the arXiv client and also fix a typo in the README, those are TWO separate commits (feat + docs). The rule: a commit should be revertable without breaking other things and must have a clear purpose. + +For a typical development session, aim for 3–6 commits per day. Fewer means commits are too large (hard to review); more means micro-commits that clutter the history. -#### Dos reglas adicionales importantes: -1. Primera: el mensaje de commit se escribe en inglés aunque el código del proyecto tenga comentarios en español. Es convención estándar en la industria y te ayuda a mantener profesionalismo en el repo. -2. Segunda: el cuerpo del mensaje (opcional, después del título) se usa para explicar el por qué, no el qué. El diff ya muestra el qué. Si la decisión no es obvia, explícala en el cuerpo +#### Two additional important rules +1. Commit messages are written in English even if the project code has comments in Spanish. This is industry standard convention and keeps the repo professional. +2. The commit body (optional, after the title) is used to explain the why, not the what. The diff already shows the what. If the decision is not obvious, explain it in the body. diff --git a/docs/work_log.md b/docs/work_log.md new file mode 100644 index 0000000..e69de29 From 0a2208c1f00c7b33113f1abd12f3fb11b984880e Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 9 Apr 2026 16:58:27 -0500 Subject: [PATCH 10/55] docs(work_log): initialize work log with sessions 2026-04-01 and 2026-04-09. --- docs/work_log.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/work_log.md b/docs/work_log.md index e69de29..98ef304 100644 --- a/docs/work_log.md +++ b/docs/work_log.md @@ -0,0 +1,17 @@ +# Work Log — ResearchOS + +--- + +### 2026-04-01 +- **Developer Context:** Sesión retomando la rama `feature/v1-infrastructure-setup` tras finalizar el cookiecutter de agentes. Se completaron los tests de streaming de `AnthropicLLM` (unitario con `AsyncMock`, `__aenter__`/`__aexit__` y generador async; integración contra la API real). Se revisó y enriqueció `CLAUDE.md` (Project Overview, traducción al inglés de las secciones Work Log y Git Workflow). Se aclararon conceptos clave: `async/await`, `AsyncMock` vs `MagicMock`, test unitario vs integración y comportamiento de pre-commit. +- **Work with Claude Code:** Exploración de `infrastructure/llm/anthropic.py` y sus tests. Corrección de setup de `AsyncMock` para simular el context manager de `stream()`. Alineación del `CLAUDE.md` con la estructura clean-agents-template. +- **Git History:** Commits relevantes — `feat(llm)`: implement AnthropicLLM provider; `test(llm)`: add unit and integration tests for AnthropicLLM; `feat(domain)`: align with clean-agents-template. +- **Pending Tasks:** Iniciar Tarea 2: cliente arXiv en `infrastructure/data/arxiv.py` (httpx + XML → `list[Paper]`). Verificar que se usa `None` (no `""`) cuando no hay system prompt en `AnthropicLLM`. + +--- + +### 2026-04-09 +- **Developer Context:** Sesión de configuración y refuerzo de la estructura del proyecto. No se desarrolló nueva funcionalidad de negocio; el foco estuvo en formalizar flujos de trabajo, proteger ramas críticas y dejar el entorno listo para la siguiente iteración de desarrollo. +- **Work with Claude Code:** Se revisó y enriqueció `CLAUDE.md` con la sección completa de **Git Workflow** (convenciones de commits, cadencia, formato Conventional Commits, cuerpo del commit). Se creó `ROADMAP.md` con el plan de versiones V1–V3. Se actualizó `docs/learnings.md` con notas de aprendizaje sobre pre-commit. Se ajustó el flujo de fin de sesión (`docs/work_log.md` creado, protocolo documentado en CLAUDE.md). El `work_log.md` se inicializó como archivo vacío en el commit `a90db6e`. +- **Git History:** 4 commits hoy — `docs: Update CLAUDE.md with GIT workflow, add learning and add new ROADMAP.md to work` · `test: Test pre-commit tootl` · `update: update pre-commit file to no commit to master or certifications brances` · `docs: update workflow to end session`. Rama `feature/v1-infrastructure-setup` adelantada 1 commit respecto a origin. Sin cambios sin commitear. +- **Pending Tasks:** Iniciar Tarea 2: cliente arXiv en `infrastructure/data/arxiv.py` (httpx, parseo XML con `xml.etree.ElementTree`, retorna `list[Paper]`, test de integración con 3 papers sobre "LLM agents"). Verificar uso de `None` vs `""` en system prompt de `AnthropicLLM`. Hacer push de la rama al remoto. From e1ff6dd3e7c8d74b090c4d70c083d9b9503d811d Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 13 Apr 2026 12:55:38 -0500 Subject: [PATCH 11/55] feat(data): add arXiv API client with XML parsing and integration test --- notebooks/002-jmmz-ArxivApi.ipynb | 314 ++++++++++++++++++++ src/researchos/infrastructure/data/arxiv.py | 55 ++++ tests/integration/test_arxiv.py | 22 ++ 3 files changed, 391 insertions(+) create mode 100644 notebooks/002-jmmz-ArxivApi.ipynb create mode 100644 src/researchos/infrastructure/data/arxiv.py create mode 100644 tests/integration/test_arxiv.py diff --git a/notebooks/002-jmmz-ArxivApi.ipynb b/notebooks/002-jmmz-ArxivApi.ipynb new file mode 100644 index 0000000..f66a56d --- /dev/null +++ b/notebooks/002-jmmz-ArxivApi.ipynb @@ -0,0 +1,314 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "id": "a8d11dbe", + "metadata": {}, + "outputs": [], + "source": [ + "# Use this initial code to work in the notebook as if it were a module, that\n", + "# is, to be able to export classes and functions from other subpackages.\n", + "\n", + "import os\n", + "import sys\n", + "\n", + "package_path = os.path.abspath(\".\").split(os.sep + \"notebooks\")[0]\n", + "if package_path not in sys.path:\n", + " sys.path.append(package_path)\n", + "\n", + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "fa888971", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + " https://arxiv.org/api/cHxbiOdZaP56ODnBPIenZhzg5f8\n", + " arXiv Query: search_query=all:electron&id_list=&start=0&max_results=1\n", + " 2026-04-13T16:31:38Z\n", + " \n", + " 1\n", + " 180125\n", + " 0\n", + " \n", + " http://arxiv.org/abs/cond-mat/0011267v1\n", + " The electronic structure of cuprates from high energy spectroscopy\n", + " 2000-11-15T16:19:15Z\n", + " \n", + " \n", + " We report studies of the electronic structure and elementary excitations of doped and undoped cuprate chains, ladders and planes. Using high energy spectroscopies such as x-ray absorption, core level photoemission and angle resolved photoemission spectroscopy, important information regarding the charge distribution and hole dynamics can be obtained. The comparison of the experimental data with suitable theoretical models sets constraints on the parameters entering into the model calculations, and offers insight into the important physical quantities governing the electronic structure of these materials. Recurring themes include the importance of the dimensionality of the Cu-O network (1D->2D) and the crucial role played by the spin-background in determining the dynamics of low-lying excitations in these strongly correlated systems.\n", + " \n", + " \n", + " 2000-11-15T16:19:15Z\n", + " J. Electron Spec. Relat. Phenom.: special issue on electron correlation, in press\n", + " \n", + " J. Electron Spectr. Relat. Phenom. 117-118, 203 (2001)\n", + " \n", + " Mark S. Golden\n", + " \n", + " \n", + " Christian Duerr\n", + " \n", + " \n", + " Andreas Koitzsch\n", + " \n", + " \n", + " Sibylle Legner\n", + " \n", + " \n", + " Zhiwei Hu\n", + " \n", + " \n", + " Sergey Borisenko\n", + " \n", + " \n", + " Martin Knupfer\n", + " \n", + " \n", + " Joerg Fink\n", + " \n", + " \n", + "\n", + "\n" + ] + } + ], + "source": [ + "import urllib, urllib.request\n", + "url = 'http://export.arxiv.org/api/query?search_query=all:electron&start=0&max_results=1'\n", + "data = urllib.request.urlopen(url)\n", + "print(data.read().decode('utf-8'))\n" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "af448fd1", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Paper(source_id='http://arxiv.org/abs/cond-mat/0011267v1', source='arxiv', title='The electronic structure of cuprates from high energy spectroscopy', authors=['Mark S. Golden', 'Christian Duerr', 'Andreas Koitzsch', 'Sibylle Legner', 'Zhiwei Hu', 'Sergey Borisenko', 'Martin Knupfer', 'Joerg Fink'], abstract=' We report studies of the electronic structure and elementary excitations of doped and undoped cuprate chains, ladders and planes. Using high energy spectroscopies such as x-ray absorption, core level photoemission and angle resolved photoemission spectroscopy, important information regarding the charge distribution and hole dynamics can be obtained. The comparison of the experimental data with suitable theoretical models sets constraints on the parameters entering into the model calculations, and offers insight into the important physical quantities governing the electronic structure of these materials. Recurring themes include the importance of the dimensionality of the Cu-O network (1D->2D) and the crucial role played by the spin-background in determining the dynamics of low-lying excitations in these strongly correlated systems.', published_date=datetime.datetime(2000, 11, 15, 16, 19, 15, tzinfo=TzInfo(0)), url='https://arxiv.org/abs/cond-mat/0011267v1', pdf_url='https://arxiv.org/pdf/cond-mat/0011267v1', categories=['cond-mat.supr-con', 'cond-mat.str-el'])]" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import httpx\n", + "import xml.etree.ElementTree as ET\n", + "from src.researchos.domain.models import Paper\n", + "\n", + "BASE_URL = 'https://export.arxiv.org/api/query'\n", + "NS = {\n", + " \"atom\": \"http://www.w3.org/2005/Atom\",\n", + " \"arxiv\": \"http://arxiv.org/schemas/atom\",\n", + " }\n", + "\n", + "async def search_papers(query: str, max_results: int) -> list[Paper]:\n", + " params = {\n", + " \"search_query\": query,\n", + " \"start\": 0,\n", + " \"max_results\": max_results\n", + " }\n", + "\n", + " async with httpx.AsyncClient() as client:\n", + " response = await client.get(BASE_URL, params=params)\n", + " return _parse_entries(response)\n", + "\n", + "def _parse_entries(results: httpx.Response) -> list[Paper]:\n", + "\n", + " root = ET.fromstring(results.text)\n", + "\n", + " papers = []\n", + "\n", + " for entry in root.findall(\"atom:entry\", NS):\n", + " id = entry.findtext(\"atom:id\", namespaces=NS)\n", + " title = entry.findtext(\"atom:title\", namespaces=NS)\n", + " summary = entry.findtext(\"atom:summary\", namespaces=NS)\n", + " published = entry.findtext(\"atom:published\", namespaces=NS)\n", + " authors = [author.findtext(\"atom:name\", namespaces=NS) for author in entry.findall(\"atom:author\", NS)]\n", + " categories = [cat.get('term') for cat in entry.findall(\"atom:category\", NS)]\n", + " for link in entry.findall(\"atom:link\", NS):\n", + " if link.get('title') == 'pdf': \n", + " pdf_url = link.get('href')\n", + " elif link.get(\"rel\") == \"alternate\":\n", + " url = link.get(\"href\")\n", + " \n", + "\n", + " paper = Paper(\n", + " source_id=id,\n", + " source=\"arxiv\",\n", + " title=title,\n", + " abstract=summary,\n", + " authors=authors,\n", + " published_date=published,\n", + " url=url,\n", + " pdf_url=pdf_url,\n", + " categories=categories\n", + " )\n", + " papers.append(paper)\n", + "\n", + " return papers\n", + "\n", + "\n", + "q= \"electron\"\n", + "m = 1\n", + "results = await search_papers(query=q, max_results=m)\n", + "\n", + "results" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "785d4f86", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Paper(source_id='http://arxiv.org/abs/cond-mat/0011267v1', source='arxiv', title='The electronic structure of cuprates from high energy spectroscopy', authors=['Mark S. Golden', 'Christian Duerr', 'Andreas Koitzsch', 'Sibylle Legner', 'Zhiwei Hu', 'Sergey Borisenko', 'Martin Knupfer', 'Joerg Fink'], abstract=' We report studies of the electronic structure and elementary excitations of doped and undoped cuprate chains, ladders and planes. Using high energy spectroscopies such as x-ray absorption, core level photoemission and angle resolved photoemission spectroscopy, important information regarding the charge distribution and hole dynamics can be obtained. The comparison of the experimental data with suitable theoretical models sets constraints on the parameters entering into the model calculations, and offers insight into the important physical quantities governing the electronic structure of these materials. Recurring themes include the importance of the dimensionality of the Cu-O network (1D->2D) and the crucial role played by the spin-background in determining the dynamics of low-lying excitations in these strongly correlated systems.', published_date=datetime.datetime(2000, 11, 15, 16, 19, 15, tzinfo=TzInfo(0)), url='https://arxiv.org/abs/cond-mat/0011267v1', pdf_url='https://arxiv.org/pdf/cond-mat/0011267v1', categories=['cond-mat.supr-con', 'cond-mat.str-el'])]" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "fields" + ] + }, + { + "cell_type": "markdown", + "id": "ee5e0961", + "metadata": {}, + "source": [ + "# Prueba desde módulo" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "6b0bd3f6", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Paper(source_id='http://arxiv.org/abs/cond-mat/0011267v1', source='arxiv', title='The electronic structure of cuprates from high energy spectroscopy', authors=['Mark S. Golden', 'Christian Duerr', 'Andreas Koitzsch', 'Sibylle Legner', 'Zhiwei Hu', 'Sergey Borisenko', 'Martin Knupfer', 'Joerg Fink'], abstract=' We report studies of the electronic structure and elementary excitations of doped and undoped cuprate chains, ladders and planes. Using high energy spectroscopies such as x-ray absorption, core level photoemission and angle resolved photoemission spectroscopy, important information regarding the charge distribution and hole dynamics can be obtained. The comparison of the experimental data with suitable theoretical models sets constraints on the parameters entering into the model calculations, and offers insight into the important physical quantities governing the electronic structure of these materials. Recurring themes include the importance of the dimensionality of the Cu-O network (1D->2D) and the crucial role played by the spin-background in determining the dynamics of low-lying excitations in these strongly correlated systems.', published_date=datetime.datetime(2000, 11, 15, 16, 19, 15, tzinfo=TzInfo(0)), url='https://arxiv.org/abs/cond-mat/0011267v1', pdf_url='https://arxiv.org/pdf/cond-mat/0011267v1', categories=['cond-mat.supr-con', 'cond-mat.str-el'])]" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from src.researchos.infrastructure.data.arxiv import search_papers\n", + "\n", + "q= \"electron\"\n", + "m = 1\n", + "results = await search_papers(query=q, max_results=m)\n", + "\n", + "results" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "8ef4434b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Paper(source_id='http://arxiv.org/abs/cond-mat/0011267v1', source='arxiv', title='The electronic structure of cuprates from high energy spectroscopy', authors=['Mark S. Golden', 'Christian Duerr', 'Andreas Koitzsch', 'Sibylle Legner', 'Zhiwei Hu', 'Sergey Borisenko', 'Martin Knupfer', 'Joerg Fink'], abstract=' We report studies of the electronic structure and elementary excitations of doped and undoped cuprate chains, ladders and planes. Using high energy spectroscopies such as x-ray absorption, core level photoemission and angle resolved photoemission spectroscopy, important information regarding the charge distribution and hole dynamics can be obtained. The comparison of the experimental data with suitable theoretical models sets constraints on the parameters entering into the model calculations, and offers insight into the important physical quantities governing the electronic structure of these materials. Recurring themes include the importance of the dimensionality of the Cu-O network (1D->2D) and the crucial role played by the spin-background in determining the dynamics of low-lying excitations in these strongly correlated systems.', published_date=datetime.datetime(2000, 11, 15, 16, 19, 15, tzinfo=TzInfo(0)), url='https://arxiv.org/abs/cond-mat/0011267v1', pdf_url='https://arxiv.org/pdf/cond-mat/0011267v1', categories=['cond-mat.supr-con', 'cond-mat.str-el']),\n", + " Paper(source_id='http://arxiv.org/abs/cond-mat/0211289v1', source='arxiv', title='Surface effects on the electronic energy loss of charged particles entering a metal surface', authors=['A. Garcia-Lekue', 'J. M. Pitarke'], abstract=' Surface effects on the electronic energy loss of charged particles entering a metal surface are investigated within linear-response theory, in the framework of time-dependent density functional theory. Interesting phenomena occur in the loss spectra originated by the boundary (bregenzung) effect, which is as a consequence of the orthogonality of surface and bulk excitation modes. Our calculations indicate that the presence of a non-abrupt electron-density profile at the surface severely affects the nature of surface excitations, as deduced from comparison with simplified models.', published_date=datetime.datetime(2002, 11, 14, 15, 23, 14, tzinfo=TzInfo(0)), url='https://arxiv.org/abs/cond-mat/0211289v1', pdf_url='https://arxiv.org/pdf/cond-mat/0211289v1', categories=['cond-mat.mtrl-sci'])]" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "q= \"electron\"\n", + "m = 2\n", + "results = await search_papers(query=q, max_results=m)\n", + "\n", + "results" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "3f3737f4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Paper(source_id='http://arxiv.org/abs/2604.08224v1', source='arxiv', title='Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering', authors=['Chenyu Zhou', 'Huacan Chai', 'Wenteng Chen', 'Zihan Guo', 'Rong Shan', 'Yuanyi Song', 'Tianyi Xu', 'Yingxuan Yang', 'Aofan Yu', 'Weiming Zhang', 'Congming Zheng', 'Jiachen Zhu', 'Zeyu Zheng', 'Zhuosheng Zhang', 'Xingyu Lou', 'Changwang Zhang', 'Zhihui Fu', 'Jun Wang', 'Weiwen Liu', 'Jianghao Lin', 'Weinan Zhang'], abstract='Large language model (LLM) agents are increasingly built less by changing model weights than by reorganizing the runtime around them. Capabilities that earlier systems expected the model to recover internally are now externalized into memory stores, reusable skills, interaction protocols, and the surrounding harness that makes these modules reliable in practice. This paper reviews that shift through the lens of externalization. Drawing on the idea of cognitive artifacts, we argue that agent infrastructure matters not merely because it adds auxiliary components, but because it transforms hard cognitive burdens into forms that the model can solve more reliably. Under this view, memory externalizes state across time, skills externalize procedural expertise, protocols externalize interaction structure, and harness engineering serves as the unification layer that coordinates them into governed execution. We trace a historical progression from weights to context to harness, analyze memory, skills, and protocols as three distinct but coupled forms of externalization, and examine how they interact inside a larger agent system. We further discuss the trade-off between parametric and externalized capability, identify emerging directions such as self-evolving harnesses and shared agent infrastructure, and discuss open challenges in evaluation, governance, and the long-term co-evolution of models and external infrastructure. The result is a systems-level framework for explaining why practical agent progress increasingly depends not only on stronger models, but on better external cognitive infrastructure.', published_date=datetime.datetime(2026, 4, 9, 13, 19, 41, tzinfo=TzInfo(0)), url='https://arxiv.org/abs/2604.08224v1', pdf_url='https://arxiv.org/pdf/2604.08224v1', categories=['cs.SE', 'cs.MA']),\n", + " Paper(source_id='http://arxiv.org/abs/2602.13713v2', source='arxiv', title='On Theoretically-Driven LLM Agents for Multi-Dimensional Discourse Analysis', authors=['Maciej Uberna', 'Michał Wawer', 'Jarosław A. Chudziak', 'Marcin Koszowy'], abstract='Identifying the strategic uses of reformulation in discourse remains a key challenge for computational argumentation. While LLMs can detect surface-level similarity, they often fail to capture the pragmatic functions of rephrasing, such as its role within rhetorical discourse. This paper presents a comparative multi-agent framework designed to quantify the benefits of incorporating explicit theoretical knowledge for this task. We utilise an dataset of annotated political debates to establish a new standard encompassing four distinct rephrase functions: Deintensification, Intensification, Specification, Generalisation, and Other, which covers all remaining types (D-I-S-G-O). We then evaluate two parallel LLM-based agent systems: one enhanced by argumentation theory via Retrieval-Augmented Generation (RAG), and an identical zero-shot baseline. The results reveal a clear performance gap: the RAG-enhanced agents substantially outperform the baseline across the board, with particularly strong advantages in detecting Intensification and Generalisation context, yielding an overall Macro F1-score improvement of nearly 30\\\\%. Our findings provide evidence that theoretical grounding is not only beneficial but essential for advancing beyond mere paraphrase detection towards function-aware analysis of argumentative discourse. This comparative multi-agent architecture represents a step towards scalable, theoretically informed computational tools capable of identifying rhetorical strategies in contemporary discourse.', published_date=datetime.datetime(2026, 2, 14, 10, 30, 39, tzinfo=TzInfo(0)), url='https://arxiv.org/abs/2602.13713v2', pdf_url='https://arxiv.org/pdf/2602.13713v2', categories=['cs.CL']),\n", + " Paper(source_id='http://arxiv.org/abs/2506.18783v1', source='arxiv', title='TRIZ Agents: A Multi-Agent LLM Approach for TRIZ-Based Innovation', authors=['Kamil Szczepanik', 'Jarosław A. Chudziak'], abstract='TRIZ, the Theory of Inventive Problem Solving, is a structured, knowledge-based framework for innovation and abstracting problems to find inventive solutions. However, its application is often limited by the complexity and deep interdisciplinary knowledge required. Advancements in Large Language Models (LLMs) have revealed new possibilities for automating parts of this process. While previous studies have explored single LLMs in TRIZ applications, this paper introduces a multi-agent approach. We propose an LLM-based multi-agent system, called TRIZ agents, each with specialized capabilities and tool access, collaboratively solving inventive problems based on the TRIZ methodology. This multi-agent system leverages agents with various domain expertise to efficiently navigate TRIZ steps. The aim is to model and simulate an inventive process with language agents. We assess the effectiveness of this team of agents in addressing complex innovation challenges based on a selected case study in engineering. We demonstrate the potential of agent collaboration to produce diverse, inventive solutions. This research contributes to the future of AI-driven innovation, showcasing the advantages of decentralized problem-solving in complex ideation tasks.', published_date=datetime.datetime(2025, 6, 23, 15, 53, 14, tzinfo=TzInfo(0)), url='https://arxiv.org/abs/2506.18783v1', pdf_url='https://arxiv.org/pdf/2506.18783v1', categories=['cs.AI', 'cs.MA'])]" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "q= \"LLM agents\"\n", + "m = 3\n", + "results = await search_papers(query=q, max_results=m)\n", + "\n", + "results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "66741eaf", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/src/researchos/infrastructure/data/arxiv.py b/src/researchos/infrastructure/data/arxiv.py new file mode 100644 index 0000000..9c421f3 --- /dev/null +++ b/src/researchos/infrastructure/data/arxiv.py @@ -0,0 +1,55 @@ +import xml.etree.ElementTree as ET + +import httpx +from src.researchos.domain.models import Paper + +BASE_URL = "https://export.arxiv.org/api/query" +NS = { + "atom": "http://www.w3.org/2005/Atom", + "arxiv": "http://arxiv.org/schemas/atom", +} + + +async def search_papers(query: str, max_results: int) -> list[Paper]: + params = {"search_query": query, "start": 0, "max_results": max_results} + + async with httpx.AsyncClient() as client: + response = await client.get(BASE_URL, params=params) + return _parse_entries(response) + + +def _parse_entries(results: httpx.Response) -> list[Paper]: + root = ET.fromstring(results.text) + + papers = [] + + for entry in root.findall("atom:entry", NS): + id = entry.findtext("atom:id", namespaces=NS) + title = entry.findtext("atom:title", namespaces=NS) + summary = entry.findtext("atom:summary", namespaces=NS) + published = entry.findtext("atom:published", namespaces=NS) + authors = [ + author.findtext("atom:name", namespaces=NS) + for author in entry.findall("atom:author", NS) + ] + categories = [cat.get("term") for cat in entry.findall("atom:category", NS)] + for link in entry.findall("atom:link", NS): + if link.get("title") == "pdf": + pdf_url = link.get("href") + elif link.get("rel") == "alternate": + url = link.get("href") + + paper = Paper( + source_id=id, + source="arxiv", + title=title, + abstract=summary, + authors=authors, + published_date=published, + url=url, + pdf_url=pdf_url, + categories=categories, + ) + papers.append(paper) + + return papers diff --git a/tests/integration/test_arxiv.py b/tests/integration/test_arxiv.py new file mode 100644 index 0000000..8ce3187 --- /dev/null +++ b/tests/integration/test_arxiv.py @@ -0,0 +1,22 @@ +import pytest +from src.researchos.domain.models import Paper +from src.researchos.infrastructure.data.arxiv import search_papers + + +@pytest.mark.integration +@pytest.mark.asyncio +async def test_search_papers_integration(): + results = await search_papers(query="LLM agents", max_results=3) + + assert isinstance(results, list) + assert len(results) == 3 + assert all(isinstance(paper, Paper) for paper in results) + assert all(paper.title for paper in results) + assert all(paper.abstract for paper in results) + assert all(paper.authors for paper in results) + assert all(paper.published_date for paper in results) + assert all(paper.url for paper in results) + assert all(paper.pdf_url for paper in results) + assert all(paper.categories for paper in results) + assert all(paper.source_id for paper in results) + assert all(paper.source == "arxiv" for paper in results) From f9fe56980aecfb56314d3df249854d746fd62b71 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 13 Apr 2026 16:00:25 -0500 Subject: [PATCH 12/55] feat(notebooks): add ignore notebooks in pre-commit tool --- .pre-commit-config.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 49b1f65..47aeeef 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -4,7 +4,9 @@ repos: hooks: - id: ruff args: [--fix] + exclude: "notebooks/" - id: ruff-format + exclude: "notebooks/" - repo: https://github.com/pre-commit/pre-commit-hooks rev: v4.6.0 From dfc2e21078bf49d15588e05585c75eda8144d57a Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 13 Apr 2026 16:01:10 -0500 Subject: [PATCH 13/55] feat(ingestion): add PDF download and text extraction service --- notebooks/002-jmmz-ArxivApi.ipynb | 46 ++++- notebooks/003-jmmz-ingestion_service.ipynb | 187 ++++++++++++++++++ .../application/services/ingestion_service.py | 36 ++++ .../application/test_ingestion_service.py | 36 ++++ 4 files changed, 303 insertions(+), 2 deletions(-) create mode 100644 notebooks/003-jmmz-ingestion_service.ipynb create mode 100644 src/researchos/application/services/ingestion_service.py create mode 100644 tests/unit/application/test_ingestion_service.py diff --git a/notebooks/002-jmmz-ArxivApi.ipynb b/notebooks/002-jmmz-ArxivApi.ipynb index f66a56d..a921420 100644 --- a/notebooks/002-jmmz-ArxivApi.ipynb +++ b/notebooks/002-jmmz-ArxivApi.ipynb @@ -203,7 +203,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 2, "id": "6b0bd3f6", "metadata": {}, "outputs": [ @@ -213,7 +213,7 @@ "[Paper(source_id='http://arxiv.org/abs/cond-mat/0011267v1', source='arxiv', title='The electronic structure of cuprates from high energy spectroscopy', authors=['Mark S. Golden', 'Christian Duerr', 'Andreas Koitzsch', 'Sibylle Legner', 'Zhiwei Hu', 'Sergey Borisenko', 'Martin Knupfer', 'Joerg Fink'], abstract=' We report studies of the electronic structure and elementary excitations of doped and undoped cuprate chains, ladders and planes. Using high energy spectroscopies such as x-ray absorption, core level photoemission and angle resolved photoemission spectroscopy, important information regarding the charge distribution and hole dynamics can be obtained. The comparison of the experimental data with suitable theoretical models sets constraints on the parameters entering into the model calculations, and offers insight into the important physical quantities governing the electronic structure of these materials. Recurring themes include the importance of the dimensionality of the Cu-O network (1D->2D) and the crucial role played by the spin-background in determining the dynamics of low-lying excitations in these strongly correlated systems.', published_date=datetime.datetime(2000, 11, 15, 16, 19, 15, tzinfo=TzInfo(0)), url='https://arxiv.org/abs/cond-mat/0011267v1', pdf_url='https://arxiv.org/pdf/cond-mat/0011267v1', categories=['cond-mat.supr-con', 'cond-mat.str-el'])]" ] }, - "execution_count": 3, + "execution_count": 2, "metadata": {}, "output_type": "execute_result" } @@ -228,6 +228,48 @@ "results" ] }, + { + "cell_type": "code", + "execution_count": 15, + "id": "e88a66dc", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'mark_s._golden_2000'" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "results[0].authors[0].lower().replace(' ', '_').strip() + '_' + results[0].published_date.strftime('%Y')" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "09e8659f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'2000'" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "results[0].published_date.strftime('%Y')" + ] + }, { "cell_type": "code", "execution_count": 5, diff --git a/notebooks/003-jmmz-ingestion_service.ipynb b/notebooks/003-jmmz-ingestion_service.ipynb new file mode 100644 index 0000000..672ca0b --- /dev/null +++ b/notebooks/003-jmmz-ingestion_service.ipynb @@ -0,0 +1,187 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "073804dc", + "metadata": {}, + "outputs": [], + "source": [ + "# Use this initial code to work in the notebook as if it were a module, that\n", + "# is, to be able to export classes and functions from other subpackages.\n", + "\n", + "import os\n", + "import sys\n", + "\n", + "package_path = os.path.abspath(\".\").split(os.sep + \"notebooks\")[0]\n", + "if package_path not in sys.path:\n", + " sys.path.append(package_path)\n", + "\n", + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a2df8505", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import re\n", + "from pathlib import Path\n", + "import httpx\n", + "import pymupdf\n", + "from src.researchos.domain.models import Paper\n", + "\n", + "# PAPERS_DIR = Path(__file__).parent.parent.parent.parent / \"data\" / \"papers\"\n", + "PAPERS_DIR = Path(\"../data/papers\")\n", + "\n", + "def extract_text_pdf(paper: Paper) -> str:\n", + " url = paper.pdf_url\n", + " pdf_name = paper.authors[0].lower().strip()\n", + " pdf_name = re.sub(r'[^a-z0-9_]', '_', pdf_name)\n", + " pdf_name = pdf_name + '_' + paper.published_date.strftime('%Y')\n", + " local_pdf_path = PAPERS_DIR / f\"{pdf_name}.pdf\"\n", + "\n", + " PAPERS_DIR.mkdir(parents=True, exist_ok=True)\n", + "\n", + " response = httpx.get(url)\n", + " response.raise_for_status()\n", + "\n", + " # save pdf in local as .pdf\n", + " with open(local_pdf_path, 'wb') as f:\n", + " \n", + " f.write(response.content)\n", + "\n", + " # extract text\n", + " full_text = \"\"\n", + " doc = pymupdf.open(local_pdf_path)\n", + " for page in doc:\n", + " full_text += page.get_text()\n", + " \n", + " if not full_text.strip():\n", + " raise ValueError(f\"PDF has no extractable text: {url}\")\n", + "\n", + " return full_text \n", + "\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2e60f4b6", + "metadata": {}, + "outputs": [], + "source": [ + "from src.researchos.infrastructure.data.arxiv import search_papers\n", + "\n", + "q= \"LLM-agents\"\n", + "m = 4\n", + "results = await search_papers(query=q, max_results=m)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c2918718", + "metadata": {}, + "outputs": [], + "source": [ + "results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "34e79115", + "metadata": {}, + "outputs": [], + "source": [ + "[extract_text_pdf(paper=results[i]) for i in range(4)]" + ] + }, + { + "cell_type": "markdown", + "id": "53dc9e17", + "metadata": {}, + "source": [ + "# Probar desde módulo" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3705692d", + "metadata": {}, + "outputs": [], + "source": [ + "from src.researchos.infrastructure.data.arxiv import search_papers\n", + "\n", + "q= \"chaotic\"\n", + "m = 1\n", + "results = await search_papers(query=q, max_results=m)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2546b964", + "metadata": {}, + "outputs": [], + "source": [ + "from src.researchos.application.services.ingestion_service import extract_text_pdf\n", + "\n", + "extract_text_pdf(paper=results[0])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3142687b", + "metadata": {}, + "outputs": [], + "source": [ + "results[0].authors" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6553fbee", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/src/researchos/application/services/ingestion_service.py b/src/researchos/application/services/ingestion_service.py new file mode 100644 index 0000000..43ccc94 --- /dev/null +++ b/src/researchos/application/services/ingestion_service.py @@ -0,0 +1,36 @@ +import re +from pathlib import Path + +import httpx +import pymupdf +from src.researchos.domain.models import Paper + +PAPERS_DIR = Path(__file__).parent.parent.parent.parent.parent / "data" / "papers" + + +def extract_text_pdf(paper: Paper) -> str: + url = paper.pdf_url + pdf_name = paper.authors[0].lower().strip() + pdf_name = re.sub(r"[^a-z0-9_]", "_", pdf_name) + pdf_name = pdf_name + "_" + paper.published_date.strftime("%Y") + local_pdf_path = PAPERS_DIR / f"{pdf_name}.pdf" + + PAPERS_DIR.mkdir(parents=True, exist_ok=True) + + response = httpx.get(url) + response.raise_for_status() + + # save pdf in local as .pdf + with open(local_pdf_path, "wb") as f: + f.write(response.content) + + # extract text + full_text = "" + doc = pymupdf.open(local_pdf_path) + for page in doc: + full_text += page.get_text() + + if not full_text.strip(): + raise ValueError(f"PDF has no extractable text: {url}") + + return full_text diff --git a/tests/unit/application/test_ingestion_service.py b/tests/unit/application/test_ingestion_service.py new file mode 100644 index 0000000..af0e70f --- /dev/null +++ b/tests/unit/application/test_ingestion_service.py @@ -0,0 +1,36 @@ +from datetime import datetime +from pathlib import Path +from unittest.mock import MagicMock, patch + +import pytest +from src.researchos.application.services.ingestion_service import extract_text_pdf +from src.researchos.domain.models import Paper + +PAPERS_SAMPLE_DIR = Path(__file__).parent.parent.parent.parent / "data" / "samples" + + +@pytest.mark.unit +def test_extract_text_pdf(): + local_pdf = PAPERS_SAMPLE_DIR / "sample_pdf.pdf" + + paper = Paper( + source_id="1", + source="arxiv", + title="Test Title", + authors=["Test Author"], + published_date=datetime(2022, 1, 1), + pdf_url="https://fake-url/paper.pdf", + abstract="Test Abstract", + categories=["test"], + ) + + mock_response = MagicMock() + mock_response.content = local_pdf.read_bytes() + + with patch( + "researchos.application.services.ingestion_service.httpx.get", return_value=mock_response + ): + result = extract_text_pdf(paper=paper) + + assert isinstance(result, str) + assert len(result) > 0 From 44fe06903bc69da7cc01372fe6e2ad989b9c2bf4 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 13 Apr 2026 16:28:01 -0500 Subject: [PATCH 14/55] refactor(ingestion): centralize filesystem paths in paths.py --- pyproject.toml | 1 + .../application/services/ingestion_service.py | 5 ++--- src/researchos/config.py | 5 +++++ src/researchos/infrastructure/data/arxiv.py | 3 ++- src/researchos/paths.py | 9 +++++++++ tests/integration/test_anthropic_llm.py | 5 +++-- tests/integration/test_arxiv.py | 5 +++-- tests/unit/application/test_ingestion_service.py | 5 +++-- tests/unit/infrastructure/test_anthropic_llm.py | 5 +++-- uv.lock | 14 ++++++++++++++ 10 files changed, 45 insertions(+), 12 deletions(-) create mode 100644 src/researchos/paths.py diff --git a/pyproject.toml b/pyproject.toml index d3ca78f..90a794d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -23,6 +23,7 @@ dependencies = [ "python-dotenv>=1.0.0", "httpx>=0.27.0", "rank-bm25>=0.2.2", + "pyprojroot>=0.3.0", ] [project.optional-dependencies] diff --git a/src/researchos/application/services/ingestion_service.py b/src/researchos/application/services/ingestion_service.py index 43ccc94..ac6a1fa 100644 --- a/src/researchos/application/services/ingestion_service.py +++ b/src/researchos/application/services/ingestion_service.py @@ -1,11 +1,10 @@ import re -from pathlib import Path import httpx import pymupdf -from src.researchos.domain.models import Paper -PAPERS_DIR = Path(__file__).parent.parent.parent.parent.parent / "data" / "papers" +from researchos.domain.models import Paper +from researchos.paths import PAPERS_DIR def extract_text_pdf(paper: Paper) -> str: diff --git a/src/researchos/config.py b/src/researchos/config.py index 216bf96..eaa52c0 100644 --- a/src/researchos/config.py +++ b/src/researchos/config.py @@ -5,6 +5,7 @@ client = Anthropic(api_key=settings.anthropic_api_key) """ +from pathlib import Path from typing import Literal from pydantic_settings import BaseSettings, SettingsConfigDict @@ -59,5 +60,9 @@ class Settings(BaseSettings): # ── GCP (V4) ── google_cloud_project: str = "" + @property + def project_root(self) -> Path: + return Path(__file__).resolve().parent.parent.parent + settings = Settings() diff --git a/src/researchos/infrastructure/data/arxiv.py b/src/researchos/infrastructure/data/arxiv.py index 9c421f3..b73eac0 100644 --- a/src/researchos/infrastructure/data/arxiv.py +++ b/src/researchos/infrastructure/data/arxiv.py @@ -1,7 +1,8 @@ import xml.etree.ElementTree as ET import httpx -from src.researchos.domain.models import Paper + +from researchos.domain.models import Paper BASE_URL = "https://export.arxiv.org/api/query" NS = { diff --git a/src/researchos/paths.py b/src/researchos/paths.py new file mode 100644 index 0000000..2dbe07e --- /dev/null +++ b/src/researchos/paths.py @@ -0,0 +1,9 @@ +from pathlib import Path + +import pyprojroot + +PROJECT_ROOT: Path = pyprojroot.here() +DATA_DIR = PROJECT_ROOT / "data" +PAPERS_DIR = DATA_DIR / "papers" +SAMPLES_DIR = DATA_DIR / "samples" +CHROMA_DIR = DATA_DIR / "chroma" diff --git a/tests/integration/test_anthropic_llm.py b/tests/integration/test_anthropic_llm.py index fc4dccc..047a03c 100644 --- a/tests/integration/test_anthropic_llm.py +++ b/tests/integration/test_anthropic_llm.py @@ -1,6 +1,7 @@ import pytest -from src.researchos.domain import Message -from src.researchos.infrastructure.llm.anthropic_llm import AnthropicLLM + +from researchos.domain import Message +from researchos.infrastructure.llm.anthropic_llm import AnthropicLLM @pytest.mark.integration diff --git a/tests/integration/test_arxiv.py b/tests/integration/test_arxiv.py index 8ce3187..2cfef7a 100644 --- a/tests/integration/test_arxiv.py +++ b/tests/integration/test_arxiv.py @@ -1,6 +1,7 @@ import pytest -from src.researchos.domain.models import Paper -from src.researchos.infrastructure.data.arxiv import search_papers + +from researchos.domain.models import Paper +from researchos.infrastructure.data.arxiv import search_papers @pytest.mark.integration diff --git a/tests/unit/application/test_ingestion_service.py b/tests/unit/application/test_ingestion_service.py index af0e70f..bfe20be 100644 --- a/tests/unit/application/test_ingestion_service.py +++ b/tests/unit/application/test_ingestion_service.py @@ -3,8 +3,9 @@ from unittest.mock import MagicMock, patch import pytest -from src.researchos.application.services.ingestion_service import extract_text_pdf -from src.researchos.domain.models import Paper + +from researchos.application.services.ingestion_service import extract_text_pdf +from researchos.domain.models import Paper PAPERS_SAMPLE_DIR = Path(__file__).parent.parent.parent.parent / "data" / "samples" diff --git a/tests/unit/infrastructure/test_anthropic_llm.py b/tests/unit/infrastructure/test_anthropic_llm.py index 120028c..3cac523 100644 --- a/tests/unit/infrastructure/test_anthropic_llm.py +++ b/tests/unit/infrastructure/test_anthropic_llm.py @@ -1,8 +1,9 @@ from unittest.mock import AsyncMock, MagicMock import pytest -from src.researchos.domain import Message -from src.researchos.infrastructure.llm.anthropic_llm import AnthropicLLM + +from researchos.domain import Message +from researchos.infrastructure.llm.anthropic_llm import AnthropicLLM @pytest.mark.unit diff --git a/uv.lock b/uv.lock index 9d16f79..c18efa1 100644 --- a/uv.lock +++ b/uv.lock @@ -2266,6 +2266,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/bd/24/12818598c362d7f300f18e74db45963dbcb85150324092410c8b49405e42/pyproject_hooks-1.2.0-py3-none-any.whl", hash = "sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913", size = 10216, upload-time = "2024-09-29T09:24:11.978Z" }, ] +[[package]] +name = "pyprojroot" +version = "0.3.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ec/7f/d04044efe4acc4185db1174209fadac33cc21c015ed0d6bef8884c9fa808/pyprojroot-0.3.0.tar.gz", hash = "sha256:109705bb790968704958efcfc5ccce85d8e3dafa054897cc81371fcbbf56cb10", size = 6287, upload-time = "2023-03-13T05:39:30.318Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/53/9b/eef01392be945c0fe86a8d084ba9188b1e2b22af037d7109b9f40a962cd0/pyprojroot-0.3.0-py3-none-any.whl", hash = "sha256:c426b51b17ab4f4d4f95b479cf5b6c22df59bb58fbd4f01b37a6977d29b99888", size = 7558, upload-time = "2023-03-13T05:39:28.707Z" }, +] + [[package]] name = "pytest" version = "9.0.2" @@ -2531,6 +2543,7 @@ dependencies = [ { name = "pydantic" }, { name = "pydantic-settings" }, { name = "pymupdf" }, + { name = "pyprojroot" }, { name = "python-dotenv" }, { name = "python-telegram-bot" }, { name = "rank-bm25" }, @@ -2565,6 +2578,7 @@ requires-dist = [ { name = "pydantic", specifier = ">=2.0.0" }, { name = "pydantic-settings", specifier = ">=2.0.0" }, { name = "pymupdf", specifier = ">=1.24.0" }, + { name = "pyprojroot", specifier = ">=0.3.0" }, { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0.0" }, { name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.24.0" }, { name = "python-dotenv", specifier = ">=1.0.0" }, From 63b2454460d5f040b879ba37ab58845e55d185cc Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 13 Apr 2026 16:30:14 -0500 Subject: [PATCH 15/55] docs: update work_log.md --- docs/work_log.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/work_log.md b/docs/work_log.md index 98ef304..bb1a500 100644 --- a/docs/work_log.md +++ b/docs/work_log.md @@ -15,3 +15,11 @@ - **Work with Claude Code:** Se revisó y enriqueció `CLAUDE.md` con la sección completa de **Git Workflow** (convenciones de commits, cadencia, formato Conventional Commits, cuerpo del commit). Se creó `ROADMAP.md` con el plan de versiones V1–V3. Se actualizó `docs/learnings.md` con notas de aprendizaje sobre pre-commit. Se ajustó el flujo de fin de sesión (`docs/work_log.md` creado, protocolo documentado en CLAUDE.md). El `work_log.md` se inicializó como archivo vacío en el commit `a90db6e`. - **Git History:** 4 commits hoy — `docs: Update CLAUDE.md with GIT workflow, add learning and add new ROADMAP.md to work` · `test: Test pre-commit tootl` · `update: update pre-commit file to no commit to master or certifications brances` · `docs: update workflow to end session`. Rama `feature/v1-infrastructure-setup` adelantada 1 commit respecto a origin. Sin cambios sin commitear. - **Pending Tasks:** Iniciar Tarea 2: cliente arXiv en `infrastructure/data/arxiv.py` (httpx, parseo XML con `xml.etree.ElementTree`, retorna `list[Paper]`, test de integración con 3 papers sobre "LLM agents"). Verificar uso de `None` vs `""` en system prompt de `AnthropicLLM`. Hacer push de la rama al remoto. + +--- + +### 2026-04-13 +- **Developer Context:** Sesión de implementación de las Tareas 2 y 3. Se completó el cliente arXiv, el servicio de ingesta y la centralización de rutas. Se recibió feedback del tutor sobre `ingestion_service.py` con refactors pendientes para la próxima sesión. +- **Work with Claude Code:** Implementado `infrastructure/data/arxiv.py` con `search_papers()` usando `httpx` + parseo XML con namespaces vía `xml.etree.ElementTree`, y `_parse_entries()` privada para separar responsabilidades. Creado `application/services/ingestion_service.py` con `extract_text_pdf()` (descarga con `httpx`, extrae texto con PyMuPDF, limpieza de nombres con `re.sub`, validación con `ValueError`). Creado `src/researchos/paths.py` como módulo transversal usando `pyprojroot` (`PROJECT_ROOT`, `DATA_DIR`, `PAPERS_DIR`, `SAMPLES_DIR`, `CHROMA_DIR`) e importado desde `ingestion_service.py`. `CLAUDE.md` traducido completamente al inglés. 16/16 tests unitarios pasando. +- **Git History:** 4 commits hoy — `feat(data)`: add arXiv API client with XML parsing and integration test · `feat(notebooks)`: add ignore notebooks in pre-commit tool · `feat(ingestion)`: add PDF download and text extraction service · `refactor(ingestion)`: centralize filesystem paths in paths.py. Rama adelantada 3 commits respecto a origin. Árbol limpio. +- **Pending Tasks:** Refactorizar `ingestion_service.py` según feedback del tutor: hacer `extract_text_pdf` async con `httpx.AsyncClient`, cambiar `ValueError` por `IngestionError`, partir en `download_pdf()` + `extract_text()` + orquestadora, cambiar `import pymupdf` por `import fitz`. Tarea 4: chunking fijo en `application/services/retrieval_service.py` (función que recibe texto y devuelve `list[Chunk]`, 500 chars con 50 de overlap, test unitario). From 23c3ef9e90dccf7dbeb14719776d440308ecca11 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Wed, 15 Apr 2026 08:57:52 -0500 Subject: [PATCH 16/55] refactor(ingestion): split into async download/extract, add IngestionError, use fitz --- .../application/services/ingestion_service.py | 31 ++++++++++++------- .../application/test_ingestion_service.py | 22 ++++++++----- 2 files changed, 34 insertions(+), 19 deletions(-) diff --git a/src/researchos/application/services/ingestion_service.py b/src/researchos/application/services/ingestion_service.py index ac6a1fa..014de29 100644 --- a/src/researchos/application/services/ingestion_service.py +++ b/src/researchos/application/services/ingestion_service.py @@ -1,35 +1,44 @@ import re +from pathlib import Path +import fitz import httpx -import pymupdf +from researchos.domain.exceptions import IngestionError from researchos.domain.models import Paper from researchos.paths import PAPERS_DIR +PAPERS_DIR.mkdir(parents=True, exist_ok=True) -def extract_text_pdf(paper: Paper) -> str: + +async def extract_text_pdf(paper: Paper) -> str: + pdf_path = await _download_pdf(paper=paper) + return _extract_text(pdf_path=pdf_path) + + +async def _download_pdf(paper: Paper) -> Path: url = paper.pdf_url pdf_name = paper.authors[0].lower().strip() pdf_name = re.sub(r"[^a-z0-9_]", "_", pdf_name) pdf_name = pdf_name + "_" + paper.published_date.strftime("%Y") local_pdf_path = PAPERS_DIR / f"{pdf_name}.pdf" - PAPERS_DIR.mkdir(parents=True, exist_ok=True) + async with httpx.AsyncClient() as client: + response = await client.get(url) + response.raise_for_status() + with open(local_pdf_path, "wb") as f: + f.write(response.content) - response = httpx.get(url) - response.raise_for_status() + return local_pdf_path - # save pdf in local as .pdf - with open(local_pdf_path, "wb") as f: - f.write(response.content) - # extract text +def _extract_text(pdf_path: Path) -> str: full_text = "" - doc = pymupdf.open(local_pdf_path) + doc = fitz.open(pdf_path) for page in doc: full_text += page.get_text() if not full_text.strip(): - raise ValueError(f"PDF has no extractable text: {url}") + raise IngestionError(f"PDF has no extractable text: {pdf_path}") return full_text diff --git a/tests/unit/application/test_ingestion_service.py b/tests/unit/application/test_ingestion_service.py index bfe20be..ce9da90 100644 --- a/tests/unit/application/test_ingestion_service.py +++ b/tests/unit/application/test_ingestion_service.py @@ -1,18 +1,17 @@ from datetime import datetime -from pathlib import Path -from unittest.mock import MagicMock, patch +from unittest.mock import AsyncMock, MagicMock, patch import pytest from researchos.application.services.ingestion_service import extract_text_pdf from researchos.domain.models import Paper - -PAPERS_SAMPLE_DIR = Path(__file__).parent.parent.parent.parent / "data" / "samples" +from researchos.paths import SAMPLES_DIR @pytest.mark.unit -def test_extract_text_pdf(): - local_pdf = PAPERS_SAMPLE_DIR / "sample_pdf.pdf" +@pytest.mark.asyncio +async def test_extract_text_pdf(): + local_pdf = SAMPLES_DIR / "sample_pdf.pdf" paper = Paper( source_id="1", @@ -27,11 +26,18 @@ def test_extract_text_pdf(): mock_response = MagicMock() mock_response.content = local_pdf.read_bytes() + mock_response.raise_for_status = MagicMock() + + mock_client = MagicMock() + mock_client.get = AsyncMock(return_value=mock_response) + mock_client.__aenter__ = AsyncMock(return_value=mock_client) + mock_client.__aexit__ = AsyncMock(return_value=False) with patch( - "researchos.application.services.ingestion_service.httpx.get", return_value=mock_response + "researchos.application.services.ingestion_service.httpx.AsyncClient", + return_value=mock_client, ): - result = extract_text_pdf(paper=paper) + result = await extract_text_pdf(paper=paper) assert isinstance(result, str) assert len(result) > 0 From 4cecf232b1eb7973e05763ca6e5e53f4e80e1ba8 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Wed, 15 Apr 2026 16:21:04 -0500 Subject: [PATCH 17/55] docs: update learnings and work log for april 15 session --- docs/learnings.md | 54 +++++++++++++++++++++++++++++++++++++++++++++++ docs/work_log.md | 24 +++++++++++++++++++++ 2 files changed, 78 insertions(+) diff --git a/docs/learnings.md b/docs/learnings.md index 48c12d5..d87cb74 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -87,6 +87,60 @@ Son productos separados. Claude Pro cubre claude.ai (interfaz web). La API requi ### Errores interesantes - Pensé que no podía trabajar con uv en el server pero descrubrí que sí +**Fecha:** 15/04/2026 + +### ¿Qué aprendí? +- AsyncIO: La analogía es "imagina una persona jugando ajedrez contra otras 15 personas, cada partida toma cerca de 30 minutos y la persona principal mueve en 5 segundos. Si los procesos fueran síncronos, es decir, la persona jugara la partida 1 completa, luego la 2 completa, luego la 3, etc. se demoría en terminar cerca de 7.5 horas. Sin embargo, si la persona juega de manera asíncrona, es decir, mueve en cada partida y va atentiendo cada mesa según su contrincante vaya realizando su movimiento, entonces la misma persona podría terminar en más rápido (supón 5 seg por mesa son 15*5=75 seg y luego ese tiempo en promedio sería 75*30=2250 seg = 37.5 minutos) +- Está relacionado con los conceptos de paralelismos, multihilo, multiproceso y concurrencia: + - El paralelismo implica que varias tareas se ejecutan al mismo tiempo, cada un en un núcleo diferente. Suelen ser tareas liminatas por CPU, es decir, se realizan cálculos. + - El multiproceso es una manera de lograr paralelismo + - La concurrencia es más amplio que el paralelismo y sugiere que multiples tareas tienen la habilidad de correr traslapándose. Concurrencia no necesarimanete implica paralelismo. + - El miltihilo es una manera de lograr concurrencia en la que múltiples hilos toman turnos para ejecutar tareas. +Estos enfoques tienen sus propias librerías como multiprocessing, concurrent.futures y threading. +- Se puede usar async para crear una función asíncrona (función corrutina) o un generados asíncrono (usando yield) +- También usar async con with para un contexto asíncrono o async con for para iterar sobre un generador asíncrono +- El event loop es el que se encarga de ejecutar las tareas asíncronas. Normalmente se dispara con un asyncio.run() pero también se puede obtener una instancia con asyncio.get_running_loop() para interactur con el objet, como por ejemplo cuando quieres programar un callback pasando el loop como un argumento +- El patrón, que consiste en esperar una corrutina y pasar su resultado a la siguiente, crea una cadena de corrutinas + - Otros patrones importantes: + - Integración de corrutinas y colas +- Async iterators, loops and comprehensions: + - iterador: async for + - generador asíncrono: async def ... yield + - comprehension: [x async for x in f() if ...] +- También es importatne los statements async with ya que garantizan que los recursos se liberen correctamente +- asyncio.create_task() para iniciar corrutinas sin esperar el await +- asyncio.gather() para ejecutar múltiples corrutinas al mismo tiempo y esperar resultados en el orden que las corrutinas son pasadas +- asyncio.as_completed() para ejecutar múltiples corrutinas al mismo tiempo y esperar resultados en el orden que las corrutinas son completadas +- Puedes agrupar los errores que suceden en llamadas asíncronas con un ExceptionGroup y puedes manejarlas desde un bloque try usando except* para cada tipo de excepción generada + +Usa async cuando: + +- Haces llamadas HTTP — arXiv, Anthropic API, NewsAPI (esperas respuesta de red) +- Lees/escribes archivos en volumen — descargar múltiples PDFs en paralelo +- Telegram bot — recibes mensajes mientras procesas otros + +No uses async cuando: + +- Procesas texto en memoria — chunking, parseo XML, re.sub() +- Operaciones CPU intensivas — embeddings, modelos ML (ahí es multiprocessing) + +Regla simple para ResearchOS: + +- ¿Esperas algo externo (API, disco, red)? → async def +- ¿Solo calculas en memoria? → def normal + +### ¿Qué no entendí bien? +- No entiendo también la importancia del event loop a nivel práctico. Entiendo que es quien orquesta las ejecuciones pero a nivel de programación no veo la necesidad de interactuar con él directamente +- Tengo que ahondar en el patrón de integración de corrutinas y colas +- también en el entendimiento de asyncio.create_task() + +### Decisiones de diseño +- Solo patrón de cadena para versión 1, en versión 2 podríamos implementa patrones de integración entre corrutinas y colas. También puedo usar gather, task, as_completed y el manejo de errores +- En V1 todas las funciones que hacen I/O externo son async def. Las que solo procesan datos en memoria son def síncronas. Esta distinción se aplica consistentemente en todo el proyecto. + +### Errores interesantes +- + **Fecha:** _[completar]_ ### ¿Qué aprendí? diff --git a/docs/work_log.md b/docs/work_log.md index bb1a500..3c8a0da 100644 --- a/docs/work_log.md +++ b/docs/work_log.md @@ -23,3 +23,27 @@ - **Work with Claude Code:** Implementado `infrastructure/data/arxiv.py` con `search_papers()` usando `httpx` + parseo XML con namespaces vía `xml.etree.ElementTree`, y `_parse_entries()` privada para separar responsabilidades. Creado `application/services/ingestion_service.py` con `extract_text_pdf()` (descarga con `httpx`, extrae texto con PyMuPDF, limpieza de nombres con `re.sub`, validación con `ValueError`). Creado `src/researchos/paths.py` como módulo transversal usando `pyprojroot` (`PROJECT_ROOT`, `DATA_DIR`, `PAPERS_DIR`, `SAMPLES_DIR`, `CHROMA_DIR`) e importado desde `ingestion_service.py`. `CLAUDE.md` traducido completamente al inglés. 16/16 tests unitarios pasando. - **Git History:** 4 commits hoy — `feat(data)`: add arXiv API client with XML parsing and integration test · `feat(notebooks)`: add ignore notebooks in pre-commit tool · `feat(ingestion)`: add PDF download and text extraction service · `refactor(ingestion)`: centralize filesystem paths in paths.py. Rama adelantada 3 commits respecto a origin. Árbol limpio. - **Pending Tasks:** Refactorizar `ingestion_service.py` según feedback del tutor: hacer `extract_text_pdf` async con `httpx.AsyncClient`, cambiar `ValueError` por `IngestionError`, partir en `download_pdf()` + `extract_text()` + orquestadora, cambiar `import pymupdf` por `import fitz`. Tarea 4: chunking fijo en `application/services/retrieval_service.py` (función que recibe texto y devuelve `list[Chunk]`, 500 chars con 50 de overlap, test unitario). + +--- + +## 2026-04-15 + +### Trabajo desarrollado +- Refactorización de `ingestion_service.py` según feedback del tutor: + - `extract_text_pdf` convertida a `async def` con `httpx.AsyncClient` + - Partida en `_download_pdf()` + `_extract_text()` + orquestadora + - `ValueError` reemplazado por `IngestionError` + - `import pymupdf` reemplazado por `import fitz` + - Test unitario actualizado con `AsyncMock` y mock de context manager +- Estudio de AsyncIO: event loop, coroutines, gather, as_completed, create_task +- 16/16 tests unitarios pasando + +### Próximos pasos +- Tarea 4: chunking fijo en `application/services/retrieval_service.py` + - Función que recibe texto crudo y devuelve `list[Chunk]` + - 500 caracteres con 50 de overlap + - Test unitario con texto de prueba +- Completar benchmark `scripts/benchmark_arxiv.py` (ejercicio async del plan de estudio) +- Continuar plan de estudio: jueves 16 abril — primera mitad del artículo async de Real Python + +--- From b2144d3313cefc9b0d6204a324ec9b9b50637e71 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Fri, 17 Apr 2026 08:19:59 -0500 Subject: [PATCH 18/55] feat(scripts): add arxiv download benchmark sequential vs parallel --- scripts/betchmark_arxiv.py | 65 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 scripts/betchmark_arxiv.py diff --git a/scripts/betchmark_arxiv.py b/scripts/betchmark_arxiv.py new file mode 100644 index 0000000..4ec9752 --- /dev/null +++ b/scripts/betchmark_arxiv.py @@ -0,0 +1,65 @@ +import asyncio +import re +import time + +import httpx + +from researchos.domain.models import Paper +from researchos.infrastructure.data.arxiv import search_papers +from researchos.paths import PAPERS_DIR + + +async def sequential_benchmark(query: str, max_results: int): + start_time = time.perf_counter() + + papers = await search_papers(query, max_results) + + PAPERS_DIR.mkdir(parents=True, exist_ok=True) + + for paper in papers: + url = paper.pdf_url + pdf_name = paper.authors[0].lower().strip() + pdf_name = re.sub(r"[^a-z0-9_]", "_", pdf_name) + pdf_name = pdf_name + "_" + paper.published_date.strftime("%Y") + local_pdf_path = PAPERS_DIR / f"{pdf_name}.pdf" + + with httpx.Client() as client: + response = client.get(url) + response.raise_for_status() + with open(local_pdf_path, "wb") as f: + f.write(response.content) + + end_time = time.perf_counter() + print(f"Sequential benchmark completed in {end_time - start_time} seconds") + + +async def parallel_benchmark(query: str, max_results: int): + start_time = time.perf_counter() + + papers = await search_papers(query, max_results) + + PAPERS_DIR.mkdir(parents=True, exist_ok=True) + + await asyncio.gather(*(_download_one_paper(paper) for paper in papers)) + + end_time = time.perf_counter() + print(f"Parallel betchmark completed in {end_time - start_time} seconds") + + +async def _download_one_paper(paper: Paper): + url = paper.pdf_url + pdf_name = paper.authors[0].lower().strip() + pdf_name = re.sub(r"[^a-z0-9_]", "_", pdf_name) + pdf_name = pdf_name + "_" + paper.published_date.strftime("%Y") + local_pdf_path = PAPERS_DIR / f"{pdf_name}.pdf" + + async with httpx.AsyncClient() as client: + response = await client.get(url) + response.raise_for_status() + with open(local_pdf_path, "wb") as f: + f.write(response.content) + + +if __name__ == "__main__": + asyncio.run(sequential_benchmark("LLM agents", 10)) + asyncio.run(parallel_benchmark("LLM Agents", 10)) From 65556a7da6f11e0a58e279ac1409aaabd984484d Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Fri, 17 Apr 2026 11:19:28 -0500 Subject: [PATCH 19/55] feat(retrieval): add fixed chunking with overlap and parametrized tests --- docs/learnings.md | 21 +- notebooks/001-jmmz-LLMProvider.ipynb | 28 +- notebooks/002-jmmz-ArxivApi.ipynb | 9 + notebooks/003-jmmz-ingestion_service.ipynb | 16 +- notebooks/004-jmmz-retriever_service.ipynb | 288 ++++++++++++++++++ scripts/betchmark_arxiv.py | 5 +- scripts/temporal.py | 34 +++ .../application/services/retrieval_service.py | 31 ++ .../application/test_retrieval_service.py | 33 ++ 9 files changed, 427 insertions(+), 38 deletions(-) create mode 100644 notebooks/004-jmmz-retriever_service.ipynb create mode 100644 scripts/temporal.py create mode 100644 src/researchos/application/services/retrieval_service.py create mode 100644 tests/unit/application/test_retrieval_service.py diff --git a/docs/learnings.md b/docs/learnings.md index d87cb74..fda6a17 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -104,7 +104,7 @@ Estos enfoques tienen sus propias librerías como multiprocessing, concurrent.fu - Otros patrones importantes: - Integración de corrutinas y colas - Async iterators, loops and comprehensions: - - iterador: async for + - iterador: async for -> el async for solo funciona con objetos que implementa '__alter__' y '__anext__' como un generador o **Strams de datos que llegan por partes** como stream.text_stream, por ejemplo. - generador asíncrono: async def ... yield - comprehension: [x async for x in f() if ...] - También es importatne los statements async with ya que garantizan que los recursos se liberen correctamente @@ -130,7 +130,7 @@ Regla simple para ResearchOS: - ¿Solo calculas en memoria? → def normal ### ¿Qué no entendí bien? -- No entiendo también la importancia del event loop a nivel práctico. Entiendo que es quien orquesta las ejecuciones pero a nivel de programación no veo la necesidad de interactuar con él directamente +- No entiendo la importancia del event loop a nivel práctico. Entiendo que es quien orquesta las ejecuciones pero a nivel de programación no veo la necesidad de interactuar con él directamente - Tengo que ahondar en el patrón de integración de corrutinas y colas - también en el entendimiento de asyncio.create_task() @@ -139,8 +139,25 @@ Regla simple para ResearchOS: - En V1 todas las funciones que hacen I/O externo son async def. Las que solo procesan datos en memoria son def síncronas. Esta distinción se aplica consistentemente en todo el proyecto. ### Errores interesantes +- async for -> Solo usado con iteradores o con un stream objetc +- usar un for y al interior llamar una función async sigue siendo un sistema secuencial. El patró busca que el for esté dentro de un async.gather() para disparar la cadena de asincronismos y así ir trayendo o procesando la I/O al mismo tiempo. Si el orden de las corrutinas no importa se podría usar async.as_completed() + +**Fecha:** 17/04/2026 + +### ¿Qué aprendí? +- Overlap en un chuncking es para no perder contexto dentro del mismo documento. Imaginar oración en un chunck sin overlap, queda partida y ningún chunk tiene la idea completa. Con un overlap (de 50 caracteres por ejemplo) la oración qeuda en 2 chunks y el retriever puede encontrarla. + +### ¿Qué no entendí bien? - +### Decisiones de diseño +- + +### Errores interesantes +- + + + **Fecha:** _[completar]_ ### ¿Qué aprendí? diff --git a/notebooks/001-jmmz-LLMProvider.ipynb b/notebooks/001-jmmz-LLMProvider.ipynb index 9db2e71..4be7e09 100644 --- a/notebooks/001-jmmz-LLMProvider.ipynb +++ b/notebooks/001-jmmz-LLMProvider.ipynb @@ -10,13 +10,7 @@ }, "source": [ "Tarea 1: Cliente de Claude API (infrastructure/llm/anthropic.py)\n", - "Implementa la clase que cumple el Protocol LLMProvider de domain/interfaces.py. Debe poder recibir una lista de Message y devolver una respuesta. Usa el SDK de Anthropic (anthropic package). Usa settings.default_model para el modelo y settings.anthropic_api_key para la key. Escribe un test unitario con el MockLLMProvider que ya tienes en conftest, y un test de integración que haga una llamada real a la API (con @pytest.mark.integration).\n", - "\n", - "Tarea 2: Cliente de arXiv API (infrastructure/data/arxiv.py)\n", - "arXiv tiene una API REST gratuita que devuelve XML. Implementa una función que reciba un query string (ej. \"LLM agents\") y un número máximo de resultados, haga la request con httpx, parsee el XML, y devuelva una lista de Paper (tu modelo de dominio). El endpoint es http://export.arxiv.org/api/query. Escribe un test de integración que busque 3 papers sobre \"LLM agents\" y verifique que devuelve objetos Paper válidos.\n", - "\n", - "Tarea 3: Descarga y extracción de PDFs (application/services/ingestion_service.py)\n", - "Crea la función de ingesta que toma un Paper, descarga su PDF usando paper.pdf_url, extrae el texto con PyMuPDF (fitz), y devuelve el texto crudo. No hagas chunking todavía — eso es tarea de semana 2. Solo descarga + extracción de texto. Guarda los PDFs en data/papers/ (crea la carpeta si no existe, agrégala a .gitignore)." + "Implementa la clase que cumple el Protocol LLMProvider de domain/interfaces.py. Debe poder recibir una lista de Message y devolver una respuesta. Usa el SDK de Anthropic (anthropic package). Usa settings.default_model para el modelo y settings.anthropic_api_key para la key. Escribe un test unitario con el MockLLMProvider que ya tienes en conftest, y un test de integración que haga una llamada real a la API (con @pytest.mark.integration)." ] }, { @@ -142,26 +136,6 @@ " yield text" ] }, - { - "cell_type": "code", - "execution_count": null, - "id": "3f7e601d", - "metadata": {}, - "outputs": [], - "source": [] - }, - { - "cell_type": "markdown", - "id": "901381aa", - "metadata": { - "vscode": { - "languageId": "plaintext" - } - }, - "source": [ - "# Task 2" - ] - }, { "cell_type": "code", "execution_count": null, diff --git a/notebooks/002-jmmz-ArxivApi.ipynb b/notebooks/002-jmmz-ArxivApi.ipynb index a921420..37736d1 100644 --- a/notebooks/002-jmmz-ArxivApi.ipynb +++ b/notebooks/002-jmmz-ArxivApi.ipynb @@ -1,5 +1,14 @@ { "cells": [ + { + "cell_type": "markdown", + "id": "8bce43da", + "metadata": {}, + "source": [ + "Tarea 2: Cliente de arXiv API (infrastructure/data/arxiv.py)\n", + "arXiv tiene una API REST gratuita que devuelve XML. Implementa una función que reciba un query string (ej. \"LLM agents\") y un número máximo de resultados, haga la request con httpx, parsee el XML, y devuelva una lista de Paper (tu modelo de dominio). El endpoint es http://export.arxiv.org/api/query. Escribe un test de integración que busque 3 papers sobre \"LLM agents\" y verifique que devuelve objetos Paper válidos." + ] + }, { "cell_type": "code", "execution_count": 1, diff --git a/notebooks/003-jmmz-ingestion_service.ipynb b/notebooks/003-jmmz-ingestion_service.ipynb index 672ca0b..2916b53 100644 --- a/notebooks/003-jmmz-ingestion_service.ipynb +++ b/notebooks/003-jmmz-ingestion_service.ipynb @@ -1,5 +1,14 @@ { "cells": [ + { + "cell_type": "markdown", + "id": "8c753850", + "metadata": {}, + "source": [ + "Tarea 3: Descarga y extracción de PDFs (application/services/ingestion_service.py)\n", + "Crea la función de ingesta que toma un Paper, descarga su PDF usando paper.pdf_url, extrae el texto con PyMuPDF (fitz), y devuelve el texto crudo. No hagas chunking todavía — eso es tarea de semana 2. Solo descarga + extracción de texto. Guarda los PDFs en data/papers/ (crea la carpeta si no existe, agrégala a .gitignore)." + ] + }, { "cell_type": "code", "execution_count": null, @@ -21,13 +30,6 @@ "%autoreload 2" ] }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] - }, { "cell_type": "code", "execution_count": null, diff --git a/notebooks/004-jmmz-retriever_service.ipynb b/notebooks/004-jmmz-retriever_service.ipynb new file mode 100644 index 0000000..dc7ca64 --- /dev/null +++ b/notebooks/004-jmmz-retriever_service.ipynb @@ -0,0 +1,288 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "6a8ac5f1", + "metadata": {}, + "source": [ + "Tarea 4: Chunking fijo (application/services/retrieval_service.py)\n", + "Implementa la función `overlap_chunking` que recibe texto crudo y parámetros de chunking,\n", + "y devuelve una `list[Chunk]` (modelo de dominio). Chunking fijo: 500 caracteres con 50 de overlap.\n", + "Maneja el caso borde de texto más corto que chunk_size (retorna 1 solo chunk).\n", + "Escribe tests unitarios parametrizados con múltiples escenarios:\n", + "texto corto, texto largo, texto de exactamente chunk_size, y texto de chunk_size+1." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "910dc7f7", + "metadata": {}, + "outputs": [], + "source": [ + "# Use this initial code to work in the notebook as if it were a module, that\n", + "# is, to be able to export classes and functions from other subpackages.\n", + "\n", + "import os\n", + "import sys\n", + "\n", + "package_path = os.path.abspath(\".\").split(os.sep + \"notebooks\")[0]\n", + "if package_path not in sys.path:\n", + " sys.path.append(package_path)\n", + "\n", + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from researchos.domain.models import Chunk\n", + "\n", + "def overlap_chunking(\n", + " text: str, paper_id: str, \n", + " chunk_size: int = 500, overlap: int = 50\n", + " ) -> list[Chunk]:\n", + "\n", + " \n", + " chunks = []\n", + " \n", + " for i, initial_car in enumerate(range(0, len(text), chunk_size)):\n", + "\n", + " start_chunk = initial_car - overlap*i\n", + " end_chunk = start_chunk + chunk_size\n", + " text_chunk = text[start_chunk:end_chunk]\n", + " chunk_id = f\"{paper_id}_{i}\"\n", + " start_char = start_chunk\n", + " end_char = min(end_chunk, len(text))\n", + "\n", + " chunk = Chunk(\n", + " chunk_id=chunk_id,\n", + " paper_id=paper_id,\n", + " text=text_chunk,\n", + " metadata={\n", + " 'chunk_size': chunk_size,\n", + " 'overlap': overlap,\n", + " 'start_char': start_char,\n", + " 'end_char': end_char\n", + " },\n", + " chunk_index=i\n", + " )\n", + "\n", + " chunks.append(chunk)\n", + " return chunks" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "id": "75a8adb3", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "1630" + ] + }, + "execution_count": 63, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "full_text = \"The rapid advancement of artificial intelligence has transformed numerous fields, from healthcare to finance, enabling unprecedented capabilities in data analysis and decision-making. Machine learning algorithms, particularly deep neural networks, have demonstrated remarkable performance in tasks such as image recognition, natural language processing, and strategic game playing. These developments have sparked both excitement and concern among researchers, policymakers, and the general public. The integration of AI systems into critical infrastructure raises important questions about reliability, security, and accountability. Researchers are actively working on methods to make AI systems more interpretable and explainable, addressing the black box problem that has long plagued complex models. Federated learning and differential privacy techniques are being developed to enable AI training on sensitive data while preserving user privacy. Meanwhile, the environmental impact of training large models has prompted investigation into more efficient architectures and training procedures. Reinforcement learning from human feedback has emerged as a promising approach for aligning AI behavior with human values and preferences. The development of foundation models trained on massive datasets has enabled few-shot and zero-shot learning across diverse tasks. As these systems become more capable, the importance of robust evaluation frameworks and safety measures continues to grow. The scientific community is increasingly focused on developing AI that is not only powerful but also trustworthy and beneficial to society.\"\n", + "\n", + "len(full_text)" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "id": "aee8c95d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "The rapid advancement of artificial intelligence has transformed numerous fields, from healthcare to finance, enabling unprecedented capabilities in data analysis and decision-making. Machine learning algorithms, particularly deep neural networks, have demonstrated remarkable performance in tasks such as image recognition, natural language processing, and strategic game playing. These developments have sparked both excitement and concern among researchers, policymakers, and the general public. The integration of AI systems into critical infrastructure raises important questions about reliability, security, and accountability. Researchers are actively working on methods to make AI systems more interpretable and explainable, addressing the black box problem that has long plagued complex models. Federated learning and differential privacy techniques are being developed to enable AI training on sensitive data while preserving user privacy. Meanwhile, the environmental impact of training large models has prompted investigation into more efficient architectures and training procedures. Reinforcement learning from human feedback has emerged as a promising approach for aligning AI behavior with human values and preferences. The development of foundation models trained on massive datasets has enabled few-shot and zero-shot learning across diverse tasks. As these systems become more capable, the importance of robust evaluation frameworks and safety measures continues to grow. The scientific community is increasingly focused on developing AI that is not only powerful but also trustworthy and beneficial to society." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from IPython.display import display, Markdown\n", + "display(Markdown(full_text))\n" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "id": "0a4c45e9", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0 0 500\n", + "500 450 950\n", + "1000 900 1400\n", + "1500 1350 1850\n" + ] + }, + { + "data": { + "text/plain": [ + "[Chunk(chunk_id='kamil_szczepanik_2025.pdf_0', paper_id='kamil_szczepanik_2025.pdf', text='The rapid advancement of artificial intelligence has transformed numerous fields, from healthcare to finance, enabling unprecedented capabilities in data analysis and decision-making. Machine learning algorithms, particularly deep neural networks, have demonstrated remarkable performance in tasks such as image recognition, natural language processing, and strategic game playing. These developments have sparked both excitement and concern among researchers, policymakers, and the general public. T', metadata={'chunk_size': 500, 'overlap': 50, 'start_char': 0, 'end_char': 500}, chunk_index=0),\n", + " Chunk(chunk_id='kamil_szczepanik_2025.pdf_1', paper_id='kamil_szczepanik_2025.pdf', text='searchers, policymakers, and the general public. The integration of AI systems into critical infrastructure raises important questions about reliability, security, and accountability. Researchers are actively working on methods to make AI systems more interpretable and explainable, addressing the black box problem that has long plagued complex models. Federated learning and differential privacy techniques are being developed to enable AI training on sensitive data while preserving user privacy. ', metadata={'chunk_size': 500, 'overlap': 50, 'start_char': 450, 'end_char': 950}, chunk_index=1),\n", + " Chunk(chunk_id='kamil_szczepanik_2025.pdf_2', paper_id='kamil_szczepanik_2025.pdf', text=' on sensitive data while preserving user privacy. Meanwhile, the environmental impact of training large models has prompted investigation into more efficient architectures and training procedures. Reinforcement learning from human feedback has emerged as a promising approach for aligning AI behavior with human values and preferences. The development of foundation models trained on massive datasets has enabled few-shot and zero-shot learning across diverse tasks. As these systems become more capa', metadata={'chunk_size': 500, 'overlap': 50, 'start_char': 900, 'end_char': 1400}, chunk_index=2),\n", + " Chunk(chunk_id='kamil_szczepanik_2025.pdf_3', paper_id='kamil_szczepanik_2025.pdf', text='s diverse tasks. As these systems become more capable, the importance of robust evaluation frameworks and safety measures continues to grow. The scientific community is increasingly focused on developing AI that is not only powerful but also trustworthy and beneficial to society.', metadata={'chunk_size': 500, 'overlap': 50, 'start_char': 1350, 'end_char': 1630}, chunk_index=3)]" + ] + }, + "execution_count": 66, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import os\n", + "import fitz\n", + "\n", + "from researchos.paths import PAPERS_DIR\n", + "\n", + "# entries = os.listdir(PAPERS_DIR)\n", + "# local_pdf_path = PAPERS_DIR / entries[0]\n", + "\n", + "# full_text = \"\"\n", + "# doc = fitz.open(local_pdf_path)\n", + "# for page in doc:\n", + "# full_text += page.get_text()\n", + "\n", + "chunks = overlap_chunking(text=full_text, paper_id=entries[0])\n", + "chunks" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "id": "6d7d0793", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.'" + ] + }, + "execution_count": 62, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chunks[0].text" + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "id": "e6fede35", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'searchers, policymakers, and the general public. The integration of AI systems into critical infrastructure raises important questions about reliability, security, and accountability. Researchers are actively working on methods to make AI systems more interpretable and explainable, addressing the black box problem that has long plagued complex models. Federated learning and differential privacy techniques are being developed to enable AI training on sensitive data while preserving user privacy. '" + ] + }, + "execution_count": 56, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chunks[1].text" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "id": "21bafa0f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "' on sensitive data while preserving user privacy. Meanwhile, the environmental impact of training large models has prompted investigation into more efficient architectures and training procedures. Reinforcement learning from human feedback has emerged as a promising approach for aligning AI behavior with human values and preferences. The development of foundation models trained on massive datasets has enabled few-shot and zero-shot learning across diverse tasks. As these systems become more capa'" + ] + }, + "execution_count": 57, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chunks[2].text" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "id": "df089988", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'s diverse tasks. As these systems become more capable, the importance of robust evaluation frameworks and safety measures continues to grow. The scientific community is increasingly focused on developing AI that is not only powerful but also trustworthy and beneficial to society.'" + ] + }, + "execution_count": 58, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "chunks[3].text" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0a09ac5f", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/scripts/betchmark_arxiv.py b/scripts/betchmark_arxiv.py index 4ec9752..d711602 100644 --- a/scripts/betchmark_arxiv.py +++ b/scripts/betchmark_arxiv.py @@ -61,5 +61,6 @@ async def _download_one_paper(paper: Paper): if __name__ == "__main__": - asyncio.run(sequential_benchmark("LLM agents", 10)) - asyncio.run(parallel_benchmark("LLM Agents", 10)) + q = "LLM agents" + asyncio.run(sequential_benchmark(q, 10)) + asyncio.run(parallel_benchmark(q, 10)) diff --git a/scripts/temporal.py b/scripts/temporal.py new file mode 100644 index 0000000..1b380dc --- /dev/null +++ b/scripts/temporal.py @@ -0,0 +1,34 @@ +# import pytest + +from researchos.application.services.retriever_service import overlap_chunking + + +# @pytest.mark.unit +def test_overlap_chunking(): + text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, \ + sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. \ + Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris \ + nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in \ + reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla \ + pariatur. Excepteur sint occaecat cupidatat non proident, sunt in \ + culpa qui officia deserunt mollit anim id est laborum." + + chunk_size = 30 + overlap = 5 + chunks = overlap_chunking( + text=text, paper_id="10205035", chunk_size=chunk_size, overlap=overlap + ) + + if len(chunks) > 1: + assert all(len(chunk.text) == chunk_size for chunk in chunks[:-1]) + assert all( + chunks[i].text[-overlap:] == chunks[i + 1].text[:overlap] + for i in range(len(chunks)) + if i < len(chunks) - 1 + ) + else: + assert chunks[0].text == text + + +if __name__ == "__main__": + test_overlap_chunking() diff --git a/src/researchos/application/services/retrieval_service.py b/src/researchos/application/services/retrieval_service.py new file mode 100644 index 0000000..d12724b --- /dev/null +++ b/src/researchos/application/services/retrieval_service.py @@ -0,0 +1,31 @@ +from researchos.domain.models import Chunk + + +def overlap_chunking( + text: str, paper_id: str, chunk_size: int = 500, overlap: int = 50 +) -> list[Chunk]: + chunks = [] + + for i, initial_car in enumerate(range(0, len(text), chunk_size)): + start_chunk = initial_car - overlap * i + end_chunk = start_chunk + chunk_size + text_chunk = text[start_chunk:end_chunk] + chunk_id = f"{paper_id}_{i}" + start_char = start_chunk + end_char = min(end_chunk, len(text)) + + chunk = Chunk( + chunk_id=chunk_id, + paper_id=paper_id, + text=text_chunk, + metadata={ + "chunk_size": chunk_size, + "overlap": overlap, + "start_char": start_char, + "end_char": end_char, + }, + chunk_index=i, + ) + + chunks.append(chunk) + return chunks diff --git a/tests/unit/application/test_retrieval_service.py b/tests/unit/application/test_retrieval_service.py new file mode 100644 index 0000000..7d58802 --- /dev/null +++ b/tests/unit/application/test_retrieval_service.py @@ -0,0 +1,33 @@ +import pytest + +from researchos.application.services.retrieval_service import overlap_chunking +from researchos.domain.models import Chunk + + +@pytest.mark.parametrize( + "text,chunk_size,overlap,expected_single_chunk", + [ + ("texto corto", 500, 50, True), + ("Lorem ipsum dolor sit amet. " * 50, 500, 50, False), + ("a" * 500, 500, 50, True), # exactamente chunk_size + ("a" * 501, 500, 50, False), # un char más que chunk_size + ], +) +@pytest.mark.unit +def test_overlap_chunking(text, chunk_size, overlap, expected_single_chunk): + chunks = overlap_chunking(text=text, paper_id="test", chunk_size=chunk_size, overlap=overlap) + + assert isinstance(chunks, list) + assert len(chunks) > 0 + assert all(isinstance(chunk, Chunk) for chunk in chunks) + + if expected_single_chunk: + assert len(chunks) == 1 + assert chunks[0].text == text + else: + assert len(chunks) > 1 + assert all(len(chunk.text) == chunk_size for chunk in chunks[:-1]) + assert all( + chunks[i].text[-overlap:] == chunks[i + 1].text[:overlap] + for i in range(len(chunks) - 1) + ) From 053336fb434c46b1cc1249bd500615872f20fb30 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Fri, 17 Apr 2026 11:54:31 -0500 Subject: [PATCH 20/55] feat(retrieval): add chunk_to_document converter with unit test --- docs/learnings.md | 1 + notebooks/004-jmmz-retriever_service.ipynb | 125 ++++++++++++++---- pyproject.toml | 1 + .../application/services/retrieval_service.py | 22 +-- .../infrastructure/retrieval/chroma.py | 4 + .../application/test_retrieval_service.py | 16 ++- uv.lock | 11 ++ 7 files changed, 140 insertions(+), 40 deletions(-) create mode 100644 src/researchos/infrastructure/retrieval/chroma.py diff --git a/docs/learnings.md b/docs/learnings.md index fda6a17..f193b86 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -146,6 +146,7 @@ Regla simple para ResearchOS: ### ¿Qué aprendí? - Overlap en un chuncking es para no perder contexto dentro del mismo documento. Imaginar oración en un chunck sin overlap, queda partida y ningún chunk tiene la idea completa. Con un overlap (de 50 caracteres por ejemplo) la oración qeuda en 2 chunks y el retriever puede encontrarla. +- Cuando requiero que un mismo test haga varias pruebas, puedo utilizar el concepto de test parametrizado, así básicamente establezco con tuplas diferentes variaciones de la entrada del test (ver tests/unit/application/test_retrieval_service.py) ### ¿Qué no entendí bien? - diff --git a/notebooks/004-jmmz-retriever_service.ipynb b/notebooks/004-jmmz-retriever_service.ipynb index dc7ca64..b460776 100644 --- a/notebooks/004-jmmz-retriever_service.ipynb +++ b/notebooks/004-jmmz-retriever_service.ipynb @@ -10,15 +10,26 @@ "y devuelve una `list[Chunk]` (modelo de dominio). Chunking fijo: 500 caracteres con 50 de overlap.\n", "Maneja el caso borde de texto más corto que chunk_size (retorna 1 solo chunk).\n", "Escribe tests unitarios parametrizados con múltiples escenarios:\n", - "texto corto, texto largo, texto de exactamente chunk_size, y texto de chunk_size+1." + "texto corto, texto largo, texto de exactamente chunk_size, y texto de chunk_size+1.\n", + "\n", + "Una vez se tienen los chunks se debe modificar hacia el objeto Document que es estandar en el uso de bases vectoriales y que está declarado como el objeto genérico para el protocolo VectorStore" ] }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 67, "id": "910dc7f7", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The autoreload extension is already loaded. To reload it, use:\n", + " %reload_ext autoreload\n" + ] + } + ], "source": [ "# Use this initial code to work in the notebook as if it were a module, that\n", "# is, to be able to export classes and functions from other subpackages.\n", @@ -36,7 +47,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 68, "metadata": {}, "outputs": [], "source": [ @@ -78,7 +89,7 @@ }, { "cell_type": "code", - "execution_count": 63, + "execution_count": 69, "id": "75a8adb3", "metadata": {}, "outputs": [ @@ -88,7 +99,7 @@ "1630" ] }, - "execution_count": 63, + "execution_count": 69, "metadata": {}, "output_type": "execute_result" } @@ -101,7 +112,7 @@ }, { "cell_type": "code", - "execution_count": 64, + "execution_count": 70, "id": "aee8c95d", "metadata": {}, "outputs": [ @@ -125,20 +136,10 @@ }, { "cell_type": "code", - "execution_count": 66, + "execution_count": 71, "id": "0a4c45e9", "metadata": {}, "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0 0 500\n", - "500 450 950\n", - "1000 900 1400\n", - "1500 1350 1850\n" - ] - }, { "data": { "text/plain": [ @@ -148,7 +149,7 @@ " Chunk(chunk_id='kamil_szczepanik_2025.pdf_3', paper_id='kamil_szczepanik_2025.pdf', text='s diverse tasks. As these systems become more capable, the importance of robust evaluation frameworks and safety measures continues to grow. The scientific community is increasingly focused on developing AI that is not only powerful but also trustworthy and beneficial to society.', metadata={'chunk_size': 500, 'overlap': 50, 'start_char': 1350, 'end_char': 1630}, chunk_index=3)]" ] }, - "execution_count": 66, + "execution_count": 71, "metadata": {}, "output_type": "execute_result" } @@ -173,17 +174,17 @@ }, { "cell_type": "code", - "execution_count": 62, + "execution_count": 72, "id": "6d7d0793", "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.'" + "'The rapid advancement of artificial intelligence has transformed numerous fields, from healthcare to finance, enabling unprecedented capabilities in data analysis and decision-making. Machine learning algorithms, particularly deep neural networks, have demonstrated remarkable performance in tasks such as image recognition, natural language processing, and strategic game playing. These developments have sparked both excitement and concern among researchers, policymakers, and the general public. T'" ] }, - "execution_count": 62, + "execution_count": 72, "metadata": {}, "output_type": "execute_result" } @@ -194,7 +195,7 @@ }, { "cell_type": "code", - "execution_count": 56, + "execution_count": 73, "id": "e6fede35", "metadata": {}, "outputs": [ @@ -204,7 +205,7 @@ "'searchers, policymakers, and the general public. The integration of AI systems into critical infrastructure raises important questions about reliability, security, and accountability. Researchers are actively working on methods to make AI systems more interpretable and explainable, addressing the black box problem that has long plagued complex models. Federated learning and differential privacy techniques are being developed to enable AI training on sensitive data while preserving user privacy. '" ] }, - "execution_count": 56, + "execution_count": 73, "metadata": {}, "output_type": "execute_result" } @@ -215,7 +216,7 @@ }, { "cell_type": "code", - "execution_count": 57, + "execution_count": 74, "id": "21bafa0f", "metadata": {}, "outputs": [ @@ -225,7 +226,7 @@ "' on sensitive data while preserving user privacy. Meanwhile, the environmental impact of training large models has prompted investigation into more efficient architectures and training procedures. Reinforcement learning from human feedback has emerged as a promising approach for aligning AI behavior with human values and preferences. The development of foundation models trained on massive datasets has enabled few-shot and zero-shot learning across diverse tasks. As these systems become more capa'" ] }, - "execution_count": 57, + "execution_count": 74, "metadata": {}, "output_type": "execute_result" } @@ -236,7 +237,7 @@ }, { "cell_type": "code", - "execution_count": 58, + "execution_count": 75, "id": "df089988", "metadata": {}, "outputs": [ @@ -246,7 +247,7 @@ "'s diverse tasks. As these systems become more capable, the importance of robust evaluation frameworks and safety measures continues to grow. The scientific community is increasingly focused on developing AI that is not only powerful but also trustworthy and beneficial to society.'" ] }, - "execution_count": 58, + "execution_count": 75, "metadata": {}, "output_type": "execute_result" } @@ -255,10 +256,76 @@ "chunks[3].text" ] }, + { + "cell_type": "markdown", + "id": "0a09ac5f", + "metadata": {}, + "source": [ + "# Transformar de chunks a Document" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "id": "3d21c398", + "metadata": {}, + "outputs": [], + "source": [ + "from researchos.domain.models import Document\n", + "\n", + "def chunk_to_document(chunk: Chunk) -> Document:\n", + " return Document(\n", + " doc_id=chunk.chunk_id,\n", + " text=chunk.text,\n", + " metadata={**chunk.metadata, \"paper_id\": chunk.paper_id, \"chunk_index\": chunk.chunk_index},\n", + " )\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "id": "50b0eb7f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Document(doc_id='kamil_szczepanik_2025.pdf_0', text='The rapid advancement of artificial intelligence has transformed numerous fields, from healthcare to finance, enabling unprecedented capabilities in data analysis and decision-making. Machine learning algorithms, particularly deep neural networks, have demonstrated remarkable performance in tasks such as image recognition, natural language processing, and strategic game playing. These developments have sparked both excitement and concern among researchers, policymakers, and the general public. T', metadata={'chunk_size': 500, 'overlap': 50, 'start_char': 0, 'end_char': 500, 'paper_id': 'kamil_szczepanik_2025.pdf', 'chunk_index': 0}, score=0.0),\n", + " Document(doc_id='kamil_szczepanik_2025.pdf_1', text='searchers, policymakers, and the general public. The integration of AI systems into critical infrastructure raises important questions about reliability, security, and accountability. Researchers are actively working on methods to make AI systems more interpretable and explainable, addressing the black box problem that has long plagued complex models. Federated learning and differential privacy techniques are being developed to enable AI training on sensitive data while preserving user privacy. ', metadata={'chunk_size': 500, 'overlap': 50, 'start_char': 450, 'end_char': 950, 'paper_id': 'kamil_szczepanik_2025.pdf', 'chunk_index': 1}, score=0.0),\n", + " Document(doc_id='kamil_szczepanik_2025.pdf_2', text=' on sensitive data while preserving user privacy. Meanwhile, the environmental impact of training large models has prompted investigation into more efficient architectures and training procedures. Reinforcement learning from human feedback has emerged as a promising approach for aligning AI behavior with human values and preferences. The development of foundation models trained on massive datasets has enabled few-shot and zero-shot learning across diverse tasks. As these systems become more capa', metadata={'chunk_size': 500, 'overlap': 50, 'start_char': 900, 'end_char': 1400, 'paper_id': 'kamil_szczepanik_2025.pdf', 'chunk_index': 2}, score=0.0),\n", + " Document(doc_id='kamil_szczepanik_2025.pdf_3', text='s diverse tasks. As these systems become more capable, the importance of robust evaluation frameworks and safety measures continues to grow. The scientific community is increasingly focused on developing AI that is not only powerful but also trustworthy and beneficial to society.', metadata={'chunk_size': 500, 'overlap': 50, 'start_char': 1350, 'end_char': 1630, 'paper_id': 'kamil_szczepanik_2025.pdf', 'chunk_index': 3}, score=0.0)]" + ] + }, + "execution_count": 77, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "[chunk_to_document(c) for c in chunks]" + ] + }, { "cell_type": "code", "execution_count": null, - "id": "0a09ac5f", + "id": "d029c9e8", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1c45ca95", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "83a4b3a6", "metadata": {}, "outputs": [], "source": [] diff --git a/pyproject.toml b/pyproject.toml index 90a794d..c482507 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -24,6 +24,7 @@ dependencies = [ "httpx>=0.27.0", "rank-bm25>=0.2.2", "pyprojroot>=0.3.0", + "pysqlite3-binary>=0.5.4.post2", ] [project.optional-dependencies] diff --git a/src/researchos/application/services/retrieval_service.py b/src/researchos/application/services/retrieval_service.py index d12724b..1367b49 100644 --- a/src/researchos/application/services/retrieval_service.py +++ b/src/researchos/application/services/retrieval_service.py @@ -1,4 +1,4 @@ -from researchos.domain.models import Chunk +from researchos.domain.models import Chunk, Document def overlap_chunking( @@ -9,23 +9,27 @@ def overlap_chunking( for i, initial_car in enumerate(range(0, len(text), chunk_size)): start_chunk = initial_car - overlap * i end_chunk = start_chunk + chunk_size - text_chunk = text[start_chunk:end_chunk] - chunk_id = f"{paper_id}_{i}" - start_char = start_chunk - end_char = min(end_chunk, len(text)) chunk = Chunk( - chunk_id=chunk_id, + chunk_id=f"{paper_id}_{i}", paper_id=paper_id, - text=text_chunk, + text=text[start_chunk:end_chunk], metadata={ "chunk_size": chunk_size, "overlap": overlap, - "start_char": start_char, - "end_char": end_char, + "start_char": start_chunk, + "end_char": min(end_chunk, len(text)), }, chunk_index=i, ) chunks.append(chunk) return chunks + + +def chunk_to_document(chunk: Chunk) -> Document: + return Document( + doc_id=chunk.chunk_id, + text=chunk.text, + metadata={**chunk.metadata, "paper_id": chunk.paper_id, "chunk_index": chunk.chunk_index}, + ) diff --git a/src/researchos/infrastructure/retrieval/chroma.py b/src/researchos/infrastructure/retrieval/chroma.py new file mode 100644 index 0000000..c22fb5a --- /dev/null +++ b/src/researchos/infrastructure/retrieval/chroma.py @@ -0,0 +1,4 @@ +import sys + +__import__("pysqlite3") +sys.modules["sqlite3"] = sys.modules.pop("pysqlite3") diff --git a/tests/unit/application/test_retrieval_service.py b/tests/unit/application/test_retrieval_service.py index 7d58802..6be001f 100644 --- a/tests/unit/application/test_retrieval_service.py +++ b/tests/unit/application/test_retrieval_service.py @@ -1,7 +1,7 @@ import pytest -from researchos.application.services.retrieval_service import overlap_chunking -from researchos.domain.models import Chunk +from researchos.application.services.retrieval_service import chunk_to_document, overlap_chunking +from researchos.domain.models import Chunk, Document @pytest.mark.parametrize( @@ -31,3 +31,15 @@ def test_overlap_chunking(text, chunk_size, overlap, expected_single_chunk): chunks[i].text[-overlap:] == chunks[i + 1].text[:overlap] for i in range(len(chunks) - 1) ) + + +@pytest.mark.unit +def test_chunk_to_document(): + chunk = Chunk(chunk_id="test_0", paper_id="test", text="Hello world!", chunk_index=0) + + doc = chunk_to_document(chunk=chunk) + + assert isinstance(doc, Document) + assert doc.doc_id == chunk.chunk_id + assert doc.text == chunk.text + assert doc.metadata["paper_id"] == chunk.paper_id diff --git a/uv.lock b/uv.lock index c18efa1..5f383a2 100644 --- a/uv.lock +++ b/uv.lock @@ -2278,6 +2278,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/53/9b/eef01392be945c0fe86a8d084ba9188b1e2b22af037d7109b9f40a962cd0/pyprojroot-0.3.0-py3-none-any.whl", hash = "sha256:c426b51b17ab4f4d4f95b479cf5b6c22df59bb58fbd4f01b37a6977d29b99888", size = 7558, upload-time = "2023-03-13T05:39:28.707Z" }, ] +[[package]] +name = "pysqlite3-binary" +version = "0.5.4.post2" +source = { registry = "https://pypi.org/simple" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/6b/40/abd5dc39b7c4a9961f831efb5b8c2f68d6c39499f3b23ea014a592fe8a59/pysqlite3_binary-0.5.4.post2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:3060a56666ede382c9af3e4b086e30c9ffb65133b3fa606c2d1b9fbff512f241", size = 4936341, upload-time = "2025-12-03T18:36:23.328Z" }, + { url = "https://files.pythonhosted.org/packages/35/e8/292e14aa4ed1ef3d4a70703c0103823fcd4b7d9701d9462e52ef88c2cc10/pysqlite3_binary-0.5.4.post2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b6162cd966fa563fe85b5372c3e61d11dd7903bd0f09cc185cb0a4c9125f4a0f", size = 4951088, upload-time = "2025-12-03T18:36:39.786Z" }, +] + [[package]] name = "pytest" version = "9.0.2" @@ -2544,6 +2553,7 @@ dependencies = [ { name = "pydantic-settings" }, { name = "pymupdf" }, { name = "pyprojroot" }, + { name = "pysqlite3-binary" }, { name = "python-dotenv" }, { name = "python-telegram-bot" }, { name = "rank-bm25" }, @@ -2579,6 +2589,7 @@ requires-dist = [ { name = "pydantic-settings", specifier = ">=2.0.0" }, { name = "pymupdf", specifier = ">=1.24.0" }, { name = "pyprojroot", specifier = ">=0.3.0" }, + { name = "pysqlite3-binary", specifier = ">=0.5.4.post2" }, { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0.0" }, { name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.24.0" }, { name = "python-dotenv", specifier = ">=1.0.0" }, From 467d99565b682933086d20698e9f293b341e9b26 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Fri, 17 Apr 2026 14:27:24 -0500 Subject: [PATCH 21/55] feat(retrieval): add ChromaVectorStore and LocalEmbedder with integration test --- notebooks/005-jmmz-chroma-VectorStore.ipynb | 461 ++++++++++++++++++ .../infrastructure/retrieval/chroma.py | 70 ++- .../infrastructure/retrieval/embedder.py | 13 + tests/conftest.py | 6 + tests/integration/test_chroma.py | 31 ++ 5 files changed, 578 insertions(+), 3 deletions(-) create mode 100644 notebooks/005-jmmz-chroma-VectorStore.ipynb create mode 100644 src/researchos/infrastructure/retrieval/embedder.py create mode 100644 tests/integration/test_chroma.py diff --git a/notebooks/005-jmmz-chroma-VectorStore.ipynb b/notebooks/005-jmmz-chroma-VectorStore.ipynb new file mode 100644 index 0000000..f67305d --- /dev/null +++ b/notebooks/005-jmmz-chroma-VectorStore.ipynb @@ -0,0 +1,461 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "3b76c9d1", + "metadata": {}, + "source": [ + "Tarea 4: Chunking fijo (application/services/retrieval_service.py)\n", + "Implementa la función `overlap_chunking` que recibe texto crudo y parámetros de chunking,\n", + "y devuelve una `list[Chunk]` (modelo de dominio). Chunking fijo: 500 caracteres con 50 de overlap.\n", + "Maneja el caso borde de texto más corto que chunk_size (retorna 1 solo chunk).\n", + "Escribe tests unitarios parametrizados con múltiples escenarios:\n", + "texto corto, texto largo, texto de exactamente chunk_size, y texto de chunk_size+1.\n", + "\n", + "Una vez se tienen los chunks se debe modificar hacia el objeto Document que es estandar en el uso de bases vectoriales y que está declarado como el objeto genérico para el protocolo VectorStore" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "dbd7c7f3", + "metadata": {}, + "outputs": [], + "source": [ + "# Use this initial code to work in the notebook as if it were a module, that\n", + "# is, to be able to export classes and functions from other subpackages.\n", + "\n", + "import os\n", + "import sys\n", + "\n", + "package_path = os.path.abspath(\".\").split(os.sep + \"notebooks\")[0]\n", + "if package_path not in sys.path:\n", + " sys.path.append(package_path)\n", + "\n", + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "86ac4dc8", + "metadata": {}, + "outputs": [], + "source": [ + "import sys\n", + "\n", + "__import__(\"pysqlite3\")\n", + "sys.modules[\"sqlite3\"] = sys.modules.pop(\"pysqlite3\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Failed to reload module 'sqlite3' from file '/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/pysqlite3/__init__.py'\n", + "Traceback (most recent call last):\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 325, in check\n", + " superreload(m, reload, self.old_objects)\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 584, in superreload\n", + " module = reload(module)\n", + " ^^^^^^^^^^^^^^\n", + " File \"/home/.pyenv/versions/3.11.8/lib/python3.11/importlib/__init__.py\", line 148, in reload\n", + " raise ImportError(msg.format(name), name=name)\n", + "ImportError: module pysqlite3 not in sys.modules\n", + "[autoreload of sqlite3 failed: Traceback (most recent call last):\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 325, in check\n", + " superreload(m, reload, self.old_objects)\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 584, in superreload\n", + " module = reload(module)\n", + " ^^^^^^^^^^^^^^\n", + " File \"/home/.pyenv/versions/3.11.8/lib/python3.11/importlib/__init__.py\", line 148, in reload\n", + " raise ImportError(msg.format(name), name=name)\n", + "ImportError: module pysqlite3 not in sys.modules\n", + "]\n" + ] + } + ], + "source": [ + "import chromadb\n", + "\n", + "from sentence_transformers import SentenceTransformer\n", + "\n", + "from researchos.paths import CHROMA_DIR\n", + "from researchos.domain.models import Document\n", + "from researchos.domain.interfaces import VectorStore\n", + "\n", + "from researchos.infrastructure.retrieval.embedder import LocalEmbedder\n", + "from researchos.infrastructure.retrieval.chroma import ChromaVectorStore\n", + "\n", + "class ChromaVectorStore:\n", + " def __init__(self, embedder: LocalEmbedder, collection_name: str = \"papers\", embedder_metadata: dict = {\"hnsw:space\": \"cosine\"}):\n", + " self.embedder = embedder\n", + " self.embedder_metadata = embedder_metadata\n", + " self.client = chromadb.PersistentClient(path=str(CHROMA_DIR))\n", + " self.collection = self.client.get_or_create_collection(name=collection_name, metadata=self.embedder_metadata) \n", + "\n", + " async def search(self, query: str, k: int) -> list[Document]:\n", + " \"\"\"Search for the top-k most relevant documents.\"\"\"\n", + "\n", + " query_embedding = self.embedder.embed(query)\n", + " retrieved_docs = self.collection.query(\n", + " query_embeddings=[query_embedding],\n", + " n_results=k,\n", + " include=[\"documents\", \"metadatas\", \"distances\"]\n", + " )\n", + "\n", + " results = [\n", + " Document(\n", + " doc_id=id,\n", + " text=text,\n", + " metadata=metadata,\n", + " score=self._distance_to_score(distance)\n", + " )\n", + "\n", + " for id, text, metadata, distance in zip(\n", + " retrieved_docs[\"ids\"][0],\n", + " retrieved_docs[\"documents\"][0],\n", + " retrieved_docs[\"metadatas\"][0],\n", + " retrieved_docs[\"distances\"][0]\n", + " )\n", + " ]\n", + "\n", + " return results\n", + "\n", + " async def upsert(self, documents: list[Document]) -> None:\n", + " \"\"\"Insert or update documents in the store.\"\"\"\n", + "\n", + " vectors = self.embedder.embed_batch([doc.text for doc in documents])\n", + "\n", + " self.collection.upsert(\n", + " ids=[doc.doc_id for doc in documents],\n", + " embeddings=vectors,\n", + " documents=[doc.text for doc in documents], # ← guarda el texto\n", + " metadatas=[doc.metadata if doc.metadata else {'source': 'unknown'} for doc in documents]\n", + " )\n", + "\n", + " def _distance_to_score(self, distance: float) -> float:\n", + " space = self.embedder_metadata.get(\"hnsw:space\", \"cosine\")\n", + " if space == \"cosine\":\n", + " return 1 - (distance / 2)\n", + " elif space == \"l2\":\n", + " return 1 / (1 + distance)\n", + " else:\n", + " return 1 - distance" + ] + }, + { + "cell_type": "markdown", + "id": "9815ebeb", + "metadata": {}, + "source": [ + "# Pruebas" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "23cc97ea", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "89554995abd24e9abbc90ba1ea262481", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Loading weights: 0%| | 0/103 [00:00 list[Document]: + """Search for the top-k most relevant documents.""" + + query_embedding = self.embedder.embed(query) + retrieved_docs = self.collection.query( + query_embeddings=[query_embedding], + n_results=k, + include=["documents", "metadatas", "distances"], + ) + + results = [ + Document( + doc_id=id, text=text, metadata=metadata, score=self._distance_to_score(distance) + ) + for id, text, metadata, distance in zip( + retrieved_docs["ids"][0], + retrieved_docs["documents"][0], + retrieved_docs["metadatas"][0], + retrieved_docs["distances"][0], + strict=False, + ) + ] + + return results + + async def upsert(self, documents: list[Document]) -> None: + """Insert or update documents in the store.""" + + vectors = self.embedder.embed_batch([doc.text for doc in documents]) + + self.collection.upsert( + ids=[doc.doc_id for doc in documents], + embeddings=vectors, + documents=[doc.text for doc in documents], # ← guarda el texto + metadatas=[ + doc.metadata if doc.metadata else {"source": "unknown"} for doc in documents + ], + ) + + def _distance_to_score(self, distance: float) -> float: + space = self.embedder_metadata.get("hnsw:space", "cosine") + if space == "cosine": + return 1 - (distance / 2) + elif space == "l2": + return 1 / (1 + distance) + else: + return 1 - distance diff --git a/src/researchos/infrastructure/retrieval/embedder.py b/src/researchos/infrastructure/retrieval/embedder.py new file mode 100644 index 0000000..6ede1a9 --- /dev/null +++ b/src/researchos/infrastructure/retrieval/embedder.py @@ -0,0 +1,13 @@ +# infrastructure/retrieval/embedder.py +from sentence_transformers import SentenceTransformer + + +class LocalEmbedder: + def __init__(self, model_name: str = "all-MiniLM-L6-v2"): + self.model = SentenceTransformer(model_name) + + def embed(self, text: str) -> list[float]: + return self.model.encode(text).tolist() + + def embed_batch(self, texts: list[str]) -> list[list[float]]: + return self.model.encode(texts).tolist() diff --git a/tests/conftest.py b/tests/conftest.py index 882cc81..7b87e0c 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -1,12 +1,17 @@ """Shared test fixtures — mocks for Protocols and sample data.""" +import sys + import pytest from researchos.domain.models import Chunk, Document, Message, Paper +__import__("pysqlite3") +sys.modules["sqlite3"] = sys.modules.pop("pysqlite3") # ── Sample data fixtures ── + @pytest.fixture def sample_paper() -> Paper: return Paper( @@ -51,6 +56,7 @@ def sample_documents() -> list[Document]: # ── Mock implementations of Protocols ── + class MockLLMProvider: """Mock LLM that returns a fixed response. Implements LLMProvider Protocol.""" diff --git a/tests/integration/test_chroma.py b/tests/integration/test_chroma.py new file mode 100644 index 0000000..50e4b52 --- /dev/null +++ b/tests/integration/test_chroma.py @@ -0,0 +1,31 @@ +import pytest + +from researchos.domain.models import Document +from researchos.infrastructure.retrieval.chroma import ChromaVectorStore +from researchos.infrastructure.retrieval.embedder import LocalEmbedder + + +@pytest.mark.integration +@pytest.mark.asyncio +async def test_chroma_upsert_and_search(): + embedder = LocalEmbedder() + store = ChromaVectorStore(embedder=embedder, collection_name="test_collection") + + docs = [ + Document( + doc_id="t1", text="LLM agents use reasoning to solve tasks", metadata={"source": "test"} + ), + Document( + doc_id="t2", text="RAG combines retrieval with generation", metadata={"source": "test"} + ), + Document(doc_id="t3", text="Python is a programming language", metadata={"source": "test"}), + ] + + await store.upsert(docs) + results = await store.search("how do agents reason?", k=2) + + assert isinstance(results, list) + assert len(results) == 2 + assert all(isinstance(r, Document) for r in results) + assert all(0 <= r.score <= 1 for r in results) + assert results[0].doc_id == "t1" From 50ad8664ba9bce57f8bde945d9a2ccdca5dce1a3 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Fri, 17 Apr 2026 14:36:14 -0500 Subject: [PATCH 22/55] docs: update learnings and work_log files --- docs/learnings.md | 17 ++++++++++------- docs/work_log.md | 26 ++++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 7 deletions(-) diff --git a/docs/learnings.md b/docs/learnings.md index f193b86..fef8042 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -145,17 +145,20 @@ Regla simple para ResearchOS: **Fecha:** 17/04/2026 ### ¿Qué aprendí? -- Overlap en un chuncking es para no perder contexto dentro del mismo documento. Imaginar oración en un chunck sin overlap, queda partida y ningún chunk tiene la idea completa. Con un overlap (de 50 caracteres por ejemplo) la oración qeuda en 2 chunks y el retriever puede encontrarla. -- Cuando requiero que un mismo test haga varias pruebas, puedo utilizar el concepto de test parametrizado, así básicamente establezco con tuplas diferentes variaciones de la entrada del test (ver tests/unit/application/test_retrieval_service.py) +- Benchmark async confirmado en código real: asyncio.gather() fue 11x más rápido que secuencial descargando 10 papers. La segunda ejecución fue más rápida por caché HTTP y reutilización de conexiones TCP — no significa que async sea menos útil, sino que el caché redujo el tiempo de espera. +- `async for` solo funciona con objetos que implementan `__aiter__` y `__anext__`. Una lista normal usa `for` común. `asyncio.gather()` es lo que genera el paralelismo, no el tipo de loop. +- Overlap en chunking es para no perder contexto dentro del mismo documento. Una oración que cae en el borde entre dos chunks queda partida sin overlap. Con 50 chars de overlap, esa oración aparece en ambos chunks y el retriever puede encontrarla completa. +- Tests parametrizados con `@pytest.mark.parametrize`: permiten probar múltiples escenarios con una sola función de test usando tuplas de parámetros (ver `tests/unit/application/test_retrieval_service.py`). +- Chroma solo maneja ids, embeddings, textos y metadatas — no sabe nada de `Document`. `ChromaVectorStore` actúa como adaptador que traduce en ambas direcciones. +- Mutable default arguments en Python son un antipatrón (`B006` en ruff): usar `dict = {}` como default puede causar bugs sutiles. Siempre usar `None` e inicializar dentro de la función. ### ¿Qué no entendí bien? -- +- por qué el score se cambia en función de la distancia cuando uso coseno o l2? ### Decisiones de diseño -- - -### Errores interesantes -- +- `LocalEmbedder` separado de `ChromaVectorStore` — si cambia el modelo de embeddings, no se toca el vector store. +- `embedder_metadata` configurable en `ChromaVectorStore` para soportar diferentes métricas de distancia (coseno, L2). +- Hack de pysqlite3 en `conftest.py`, no en código de producción. Se resolverá en Dockerfile en V4. diff --git a/docs/work_log.md b/docs/work_log.md index 3c8a0da..a644521 100644 --- a/docs/work_log.md +++ b/docs/work_log.md @@ -47,3 +47,29 @@ - Continuar plan de estudio: jueves 16 abril — primera mitad del artículo async de Real Python --- + +## 2026-04-17 + +### Trabajo desarrollado +- Completado ejercicio de benchmark async: `scripts/benchmark_arxiv.py` + - Descarga secuencial vs paralela de 10 papers con `asyncio.gather()` + - Resultado: 13.6s secuencial vs 1.2s paralelo (11x más rápido sin caché) +- Tarea 4 completada: chunking fijo en `application/services/retrieval_service.py` + - `overlap_chunking()` con 500 chars y 50 de overlap + - Manejo de caso borde: texto más corto que chunk_size retorna 1 chunk + - Tests parametrizados con 4 escenarios (texto corto, largo, exactamente chunk_size, chunk_size+1) +- `chunk_to_document()` implementada y testeada +- `infrastructure/retrieval/embedder.py` creado con `LocalEmbedder` (sentence-transformers) +- `infrastructure/retrieval/chroma.py` implementado con `ChromaVectorStore`: + - Cliente persistente local con pysqlite3 workaround + - `upsert()` con `embed_batch` para eficiencia + - `search()` con score normalizado según métrica configurable + - Separación de responsabilidades: embedder inyectado como dependencia +- Test de integración para Chroma: upsert + search verificados + +### Próximos pasos +- Tarea 5: integración end-to-end del pipeline (arXiv → PDF → chunking → Chroma) +- Plan de estudio semana 2: pytest (artículo Real Python + libro Okken caps 1-5, 7) +- Resolver sqlite3 en Docker cuando llegue V4 + +--- From bf2bbb19b9043bb8eec1593c7b4ebd971d1f6587 Mon Sep 17 00:00:00 2001 From: Mario Date: Sat, 18 Apr 2026 07:39:50 -0500 Subject: [PATCH 23/55] chore(deps): make pysqlite3-binary linux-only dependency --- pyproject.toml | 2 +- uv.lock | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index c482507..da331b5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -24,7 +24,7 @@ dependencies = [ "httpx>=0.27.0", "rank-bm25>=0.2.2", "pyprojroot>=0.3.0", - "pysqlite3-binary>=0.5.4.post2", + "pysqlite3-binary>=0.5.4; sys_platform == 'linux'", ] [project.optional-dependencies] diff --git a/uv.lock b/uv.lock index 5f383a2..d05317d 100644 --- a/uv.lock +++ b/uv.lock @@ -2553,7 +2553,7 @@ dependencies = [ { name = "pydantic-settings" }, { name = "pymupdf" }, { name = "pyprojroot" }, - { name = "pysqlite3-binary" }, + { name = "pysqlite3-binary", marker = "sys_platform == 'linux'" }, { name = "python-dotenv" }, { name = "python-telegram-bot" }, { name = "rank-bm25" }, @@ -2589,7 +2589,7 @@ requires-dist = [ { name = "pydantic-settings", specifier = ">=2.0.0" }, { name = "pymupdf", specifier = ">=1.24.0" }, { name = "pyprojroot", specifier = ">=0.3.0" }, - { name = "pysqlite3-binary", specifier = ">=0.5.4.post2" }, + { name = "pysqlite3-binary", marker = "sys_platform == 'linux'", specifier = ">=0.5.4" }, { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0.0" }, { name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.24.0" }, { name = "python-dotenv", specifier = ">=1.0.0" }, From 530334288221811c5f1181a3d86eca76d3d5c6c1 Mon Sep 17 00:00:00 2001 From: Mario Date: Sat, 18 Apr 2026 10:00:28 -0500 Subject: [PATCH 24/55] feat(ingestion): add ingest_papers pipeline orchestrating arxiv, pdf extraction, chunking and chroma --- data/samples/labels.json | 143 ++++ ...2-jmmz-langchain-course-deepLearning.ipynb | 23 + notebooks/003-jmmz-ingestion_service.ipynb | 365 +++++++- pyproject.toml | 26 +- scripts/temporal.py | 34 - .../application/services/ingestion_service.py | 41 +- src/researchos/config.py | 2 +- src/researchos/infrastructure/data/arxiv.py | 5 + src/researchos/paths.py | 7 + tests/conftest.py | 5 +- uv.lock | 777 +++++++++--------- 11 files changed, 978 insertions(+), 450 deletions(-) create mode 100644 data/samples/labels.json create mode 100644 notebooks/002-jmmz-langchain-course-deepLearning.ipynb delete mode 100644 scripts/temporal.py diff --git a/data/samples/labels.json b/data/samples/labels.json new file mode 100644 index 0000000..b6708fa --- /dev/null +++ b/data/samples/labels.json @@ -0,0 +1,143 @@ +{ +"paper": "Attention Is All You Need (1706.03762v7.pdf)", +"preguntas": [ + { + "pregunta": "¿Qué arquitectura de red propone este estudio que abandona por completo la recurrencia y las convoluciones?", + "respuesta": "Propone el Transformer, una arquitectura basada únicamente en mecanismos de atención [1, 2]." + }, + { + "pregunta": "¿Cómo se define la función de atención 'Scaled Dot-Product Attention' utilizada en el modelo?", + "respuesta": "Es una función que calcula el producto punto de la consulta (query) con todas las claves (keys), divide cada uno por la raíz cuadrada de la dimensión de la clave (dk), y aplica una función softmax para obtener los pesos de los valores [3, 4]." + } +] +}, +{ +"paper": "Fine-tuning Causal LLMs for Text Classification (2512.12677v1.pdf)", +"preguntas": [ + { + "pregunta": "¿Cuáles son los dos enfoques principales comparados para el ajuste fino de modelos de lenguaje causal en tareas de clasificación?", + "respuesta": "El enfoque basado en embeddings (añadir un cabezal de clasificación sobre el embedding del token final) y el enfoque basado en instrucciones (formatear la tarea como prompt -> respuesta) [5, 6]." + }, + { + "pregunta": "¿Qué técnicas se combinan para permitir el ajuste fino de modelos de hasta 8B parámetros en una sola GPU?", + "respuesta": "Se combina la cuantificación del modelo de 4 bits con la Adaptación de Bajo Rango (LoRA), técnica conocida como QLoRA [5, 7]." + } +] +}, +{ +"paper": "Constructing Multi-label Hierarchical Classification Models for MITRE ATT&CK (2601.14556v1.pdf)", +"preguntas": [ + { + "pregunta": "¿Qué precisión alcanzó el enfoque jerárquico multietiqueta propuesto a nivel de táctica de ciberseguridad?", + "respuesta": "Alcanzó una precisión aproximada del 94% a nivel de táctica [8, 9]." + }, + { + "pregunta": "¿Cómo superó el modelo baseline de descenso de gradiente estocástico (SGD) al modelo GPT-4o en el estudio piloto?", + "respuesta": "El modelo SGD alcanzó una precisión de 0.8195 frente al 0.59 obtenido por GPT-4o en la clasificación de tácticas a partir de oraciones de ciberinteligencia [10, 11]." + } +] +}, +{ +"paper": "VOICEAGENTRAG (2603.02206v2.pdf)", +"preguntas": [ + { + "pregunta": "¿Qué agentes componen la arquitectura dual diseñada para resolver el cuello de botella de latencia en agentes de voz?", + "respuesta": "Se compone de un 'Slow Thinker' (agente de fondo que predice temas y pre-recupera datos) y un 'Fast Talker' (agente de primer plano que responde desde una caché semántica) [12, 13]." + }, + { + "pregunta": "¿Qué mejora de velocidad de recuperación se logró en las consultas que resultaron en un acierto de caché (cache hit)?", + "respuesta": "Se logró una aceleración de 316 veces, reduciendo la latencia de recuperación de una media de 110 ms a 0.35 ms [12, 14]." + } +] +}, +{ +"paper": "Specification-Driven Generation of Discrete-Event World Models (2603.03784v1.pdf)", +"preguntas": [ + { + "pregunta": "¿Qué formalismo se utiliza para descomponer los sistemas en componentes atómicos y acoplados con semántica de temporización explícita?", + "respuesta": "Se utiliza el formalismo DEVS (Discrete Event System Specification) [15, 16]." + }, + { + "pregunta": "¿En qué consiste el marco de evaluación propuesto para validar los simuladores generados por LLM?", + "respuesta": "Es un marco basado en trazas que valida las trazas de eventos estructuradas emitidas por el simulador contra restricciones temporales y semánticas derivadas de la especificación original [15, 17]." + } +] +}, +{ +"paper": "Agentics 2.0 (2603.04241v1.pdf)", +"preguntas": [ + { + "pregunta": "¿Qué concepto central de Agentics 2.0 formaliza una llamada de inferencia de LLM como una transformación semántica tipada?", + "respuesta": "La 'función transducible', basada en el álgebra de transducción lógica [18-20]." + }, + { + "pregunta": "¿Qué modelo de programación asíncrona utiliza el marco para garantizar la escalabilidad en flujos de trabajo de datos agénticos?", + "respuesta": "Utiliza una semántica de Map-Reduce asíncrona que permite procesar colecciones de estados de tipo en paralelo [18, 21, 22]." + } +] +}, +{ +"paper": "AI Agents, Language, Deep Learning and the Next Revolution in Science (2603.07940v1.pdf)", +"preguntas": [ + { + "pregunta": "¿Qué sistema multi-agente se menciona como ejemplo de aplicación en la investigación de colisionadores de partículas en el CEPC?", + "respuesta": "El sistema Dr. Sai, desarrollado en el Instituto de Física de Altas Energías (IHEP) [23, 24]." + }, + { + "pregunta": "¿Cuál es el nombre del lenguaje de dominio específico (DSL) utilizado en dicho sistema para describir objetivos analíticos?", + "respuesta": "Se denomina SaiScript [25]." + } +] +}, +{ +"paper": "Autonomous AI Agent for Clinical Triage in Remote Patient Monitoring (2603.09052v1.pdf)", +"preguntas": [ + { + "pregunta": "¿Cómo se denomina el agente de IA desarrollado para realizar el triaje clínico contextual de signos vitales?", + "respuesta": "Se llama Sentinel [26, 27]." + }, + { + "pregunta": "¿Qué sensibilidad alcanzó el agente para clasificaciones de emergencia en comparación con el promedio de clínicos humanos?", + "respuesta": "El agente alcanzó una sensibilidad del 97.5% frente al 60.0% agregado de los clínicos individuales en el análisis 'leave-one-out' [28, 29]." + } +] +}, +{ +"paper": "AI Act Evaluation Benchmark (2603.09435v1.pdf)", +"preguntas": [ + { + "pregunta": "¿Qué tareas de aprendizaje automático se incluyen en el conjunto de datos para evaluar el cumplimiento de la Ley de IA de la UE?", + "respuesta": "Clasificación del nivel de riesgo, recuperación de artículos, generación de obligaciones y respuesta a preguntas (QA) [30, 31]." + }, + { + "pregunta": "¿Qué modelo de código abierto se utilizó para generar los escenarios del dataset debido a su capacidad de ejecución en una sola GPU?", + "respuesta": "Se utilizó el modelo gpt-oss-120b [32]." + } +] +}, +{ +"paper": "Extreme Multi-label Text Classification (XMTC) Library Dataset (2603.10876v1.pdf)", +"preguntas": [ + { + "pregunta": "¿Cómo se llama el corpus bilingüe de registros bibliográficos presentado para la indexación automatizada de materias?", + "respuesta": "Se denomina TIB-SID (TIB Subject Indexing Dataset) [33]." + }, + { + "pregunta": "¿Qué archivo de autoridad se utiliza como taxonomía para las anotaciones de materias en este conjunto de datos?", + "respuesta": "Se utiliza el GND (Gemeinsame Normdatei / Integrated Authority File) de la Biblioteca Nacional Alemana [34, 35]." + } +] +}, +{ +"paper": "FinReflectKG - HalluBench (2603.20252v1.pdf)", +"preguntas": [ + { + "pregunta": "¿Cuál es el objetivo principal del benchmark FinReflectKG - HalluBench?", + "respuesta": "Evaluar métodos de detección de alucinaciones en sistemas de respuesta a preguntas financieras aumentados con Grafos de Conocimiento (KG) sobre informes SEC 10-K [36, 37]." + }, + { + "pregunta": "¿Qué tipo de método de detección demostró mayor robustez frente a señales de KG ruidosas o erróneas?", + "respuesta": "Los enfoques basados en embeddings, que mostraron solo un 9% de degradación en comparación con las caídas significativas de otros métodos [36, 38]." + } +] +} diff --git a/notebooks/002-jmmz-langchain-course-deepLearning.ipynb b/notebooks/002-jmmz-langchain-course-deepLearning.ipynb new file mode 100644 index 0000000..4c7532d --- /dev/null +++ b/notebooks/002-jmmz-langchain-course-deepLearning.ipynb @@ -0,0 +1,23 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "id": "8b21f954", + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/notebooks/003-jmmz-ingestion_service.ipynb b/notebooks/003-jmmz-ingestion_service.ipynb index 2916b53..112e30b 100644 --- a/notebooks/003-jmmz-ingestion_service.ipynb +++ b/notebooks/003-jmmz-ingestion_service.ipynb @@ -11,7 +11,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "id": "073804dc", "metadata": {}, "outputs": [], @@ -122,18 +122,41 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "id": "3705692d", "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "ParseError", + "evalue": "syntax error: line 1, column 0 ()", + "output_type": "error", + "traceback": [ + "Traceback \u001b[36m(most recent call last)\u001b[39m:\n", + " File \u001b[92mc:\\Users\\mario\\Documentos\\personal_projects\\becomes_ai_engineer\\researchos\\.venv\\Lib\\site-packages\\IPython\\core\\interactiveshell.py:3699\u001b[39m in \u001b[95mrun_code\u001b[39m\n await eval(code_obj, self.user_global_ns, self.user_ns)\n", + " Cell \u001b[92mIn[2]\u001b[39m\u001b[92m, line 5\u001b[39m\n results = await search_papers(query=q, max_results=m)\n", + " File \u001b[92m~\\Documentos\\personal_projects\\becomes_ai_engineer\\researchos\\src\\researchos\\infrastructure\\data\\arxiv.py:19\u001b[39m in \u001b[95msearch_papers\u001b[39m\n return _parse_entries(response)\n", + " File \u001b[92m~\\Documentos\\personal_projects\\becomes_ai_engineer\\researchos\\src\\researchos\\infrastructure\\data\\arxiv.py:23\u001b[39m in \u001b[95m_parse_entries\u001b[39m\n root = ET.fromstring(results.text)\n", + "\u001b[36m \u001b[39m\u001b[36mFile \u001b[39m\u001b[32m~\\AppData\\Roaming\\uv\\python\\cpython-3.11-windows-x86_64-none\\Lib\\xml\\etree\\ElementTree.py:1350\u001b[39m\u001b[36m in \u001b[39m\u001b[35mXML\u001b[39m\n\u001b[31m \u001b[39m\u001b[31mparser.feed(text)\u001b[39m\n", + " \u001b[36mFile \u001b[39m\u001b[32m\u001b[39m\n\u001b[31m \u001b[39m\n\u001b[31mParseError\u001b[39m\u001b[31m:\u001b[39m syntax error: line 1, column 0\n" + ] + } + ], "source": [ - "from src.researchos.infrastructure.data.arxiv import search_papers\n", + "from researchos.infrastructure.data.arxiv import search_papers\n", "\n", - "q= \"chaotic\"\n", + "q= \"chaos\"\n", "m = 1\n", "results = await search_papers(query=q, max_results=m)" ] }, + { + "cell_type": "code", + "execution_count": null, + "id": "c573a1e4", + "metadata": {}, + "outputs": [], + "source": [] + }, { "cell_type": "code", "execution_count": null, @@ -156,10 +179,338 @@ "results[0].authors" ] }, + { + "cell_type": "markdown", + "id": "6553fbee", + "metadata": {}, + "source": [ + "# Test full ingestion pipeline" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "9a0c262a", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "37f2323c4ba24631959198b051e24831", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Loading weights: 0%| | 0/103 [00:00=3.11,<3.13" authors = [ - { name = "Tu Nombre", email = "tu@email.com" }, + { name = "John Mario Montoya Zapata", email = "jmmontoyaz13@gmail.com" }, ] keywords = ["rag", "llm", "agents", "research", "arxiv", "pubmed"] @@ -27,23 +27,24 @@ dependencies = [ "pysqlite3-binary>=0.5.4; sys_platform == 'linux'", ] -[project.optional-dependencies] +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.hatch.build.targets.wheel] +packages = ["src/researchos"] + +[dependency-groups] dev = [ "pytest>=8.0.0", - "pytest-asyncio>=0.24.0", + "pytest-asyncio>=1.3.0", "ruff>=0.6.0", "mypy>=1.11.0", "pre-commit>=3.8.0", "jupyter>=1.0.0", + "ipykernel>=6.0.0", ] -[build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" - -[tool.hatch.build.targets.wheel] -packages = ["src/researchos"] - [tool.ruff] target-version = "py311" line-length = 100 @@ -70,8 +71,3 @@ python_version = "3.11" warn_return_any = true warn_unused_configs = true disallow_untyped_defs = false - -[dependency-groups] -dev = [ - "pre-commit>=4.5.1", -] diff --git a/scripts/temporal.py b/scripts/temporal.py deleted file mode 100644 index 1b380dc..0000000 --- a/scripts/temporal.py +++ /dev/null @@ -1,34 +0,0 @@ -# import pytest - -from researchos.application.services.retriever_service import overlap_chunking - - -# @pytest.mark.unit -def test_overlap_chunking(): - text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, \ - sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. \ - Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris \ - nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in \ - reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla \ - pariatur. Excepteur sint occaecat cupidatat non proident, sunt in \ - culpa qui officia deserunt mollit anim id est laborum." - - chunk_size = 30 - overlap = 5 - chunks = overlap_chunking( - text=text, paper_id="10205035", chunk_size=chunk_size, overlap=overlap - ) - - if len(chunks) > 1: - assert all(len(chunk.text) == chunk_size for chunk in chunks[:-1]) - assert all( - chunks[i].text[-overlap:] == chunks[i + 1].text[:overlap] - for i in range(len(chunks)) - if i < len(chunks) - 1 - ) - else: - assert chunks[0].text == text - - -if __name__ == "__main__": - test_overlap_chunking() diff --git a/src/researchos/application/services/ingestion_service.py b/src/researchos/application/services/ingestion_service.py index 014de29..b3b8c1e 100644 --- a/src/researchos/application/services/ingestion_service.py +++ b/src/researchos/application/services/ingestion_service.py @@ -1,19 +1,22 @@ +import asyncio import re from pathlib import Path import fitz import httpx +from researchos.application.services.retrieval_service import chunk_to_document, overlap_chunking from researchos.domain.exceptions import IngestionError from researchos.domain.models import Paper +from researchos.infrastructure.data.arxiv import search_papers +from researchos.infrastructure.retrieval.chroma import ChromaVectorStore +from researchos.infrastructure.retrieval.embedder import LocalEmbedder from researchos.paths import PAPERS_DIR -PAPERS_DIR.mkdir(parents=True, exist_ok=True) - -async def extract_text_pdf(paper: Paper) -> str: +async def extract_text_pdf(paper: Paper) -> tuple[str, Path]: pdf_path = await _download_pdf(paper=paper) - return _extract_text(pdf_path=pdf_path) + return _extract_text(pdf_path=pdf_path), pdf_path async def _download_pdf(paper: Paper) -> Path: @@ -26,6 +29,7 @@ async def _download_pdf(paper: Paper) -> Path: async with httpx.AsyncClient() as client: response = await client.get(url) response.raise_for_status() + # save pdf in local system with open(local_pdf_path, "wb") as f: f.write(response.content) @@ -42,3 +46,32 @@ def _extract_text(pdf_path: Path) -> str: raise IngestionError(f"PDF has no extractable text: {pdf_path}") return full_text + + +async def ingest_papers( + query: str, + max_results: int, + chunk_size: int = 500, + overlap: int = 50, + collection_name: str = "papers", + embedder_metadata: dict | None = None, +) -> None: + embedder = LocalEmbedder() + store = ChromaVectorStore( + embedder=embedder, collection_name=collection_name, embedder_metadata=embedder_metadata + ) + + papers = await search_papers(query, max_results) + + results = await asyncio.gather(*(extract_text_pdf(paper) for paper in papers)) + texts = [r[0] for r in results] + local_paths = [r[1] for r in results] + + for text, pdf_path in zip(texts, local_paths, strict=False): + chunks = overlap_chunking( + text=text, paper_id=pdf_path.stem, chunk_size=chunk_size, overlap=overlap + ) + + docs = [chunk_to_document(chunk) for chunk in chunks] + + await store.upsert(docs) diff --git a/src/researchos/config.py b/src/researchos/config.py index eaa52c0..6a2ad72 100644 --- a/src/researchos/config.py +++ b/src/researchos/config.py @@ -25,7 +25,7 @@ class Settings(BaseSettings): # ── LLM ── anthropic_api_key: str = "" - default_model: str = "claude-sonnet-4-20250514" + default_model: str = "claude-haiku-4-5-20251001" fast_model: str = "claude-haiku-4-5-20251001" temperature: float = 0.5 max_tokens: int = 1024 diff --git a/src/researchos/infrastructure/data/arxiv.py b/src/researchos/infrastructure/data/arxiv.py index b73eac0..439688e 100644 --- a/src/researchos/infrastructure/data/arxiv.py +++ b/src/researchos/infrastructure/data/arxiv.py @@ -2,6 +2,7 @@ import httpx +from researchos.domain.exceptions import IngestionError from researchos.domain.models import Paper BASE_URL = "https://export.arxiv.org/api/query" @@ -16,10 +17,14 @@ async def search_papers(query: str, max_results: int) -> list[Paper]: async with httpx.AsyncClient() as client: response = await client.get(BASE_URL, params=params) + response.raise_for_status() return _parse_entries(response) def _parse_entries(results: httpx.Response) -> list[Paper]: + if not results.text.strip().startswith("<"): + raise IngestionError(f"arXiv returned unexpected response: {results.text[:100]}") + root = ET.fromstring(results.text) papers = [] diff --git a/src/researchos/paths.py b/src/researchos/paths.py index 2dbe07e..7545092 100644 --- a/src/researchos/paths.py +++ b/src/researchos/paths.py @@ -7,3 +7,10 @@ PAPERS_DIR = DATA_DIR / "papers" SAMPLES_DIR = DATA_DIR / "samples" CHROMA_DIR = DATA_DIR / "chroma" + + +def ensure_dirs() -> None: + """Create all data directories if they don't exist.""" + dirs = [v for v in globals().values() if isinstance(v, Path) and v != PROJECT_ROOT] + for d in dirs: + d.mkdir(parents=True, exist_ok=True) diff --git a/tests/conftest.py b/tests/conftest.py index 7b87e0c..c1ffcb5 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -6,8 +6,9 @@ from researchos.domain.models import Chunk, Document, Message, Paper -__import__("pysqlite3") -sys.modules["sqlite3"] = sys.modules.pop("pysqlite3") +if sys.platform == "linux": + __import__("pysqlite3") + sys.modules["sqlite3"] = sys.modules.pop("pysqlite3") # ── Sample data fixtures ── diff --git a/uv.lock b/uv.lock index d05317d..6aaf6df 100644 --- a/uv.lock +++ b/uv.lock @@ -26,7 +26,7 @@ wheels = [ [[package]] name = "anthropic" -version = "0.86.0" +version = "0.96.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "anyio" }, @@ -38,9 +38,9 @@ dependencies = [ { name = "sniffio" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/37/7a/8b390dc47945d3169875d342847431e5f7d5fa716b2e37494d57cfc1db10/anthropic-0.86.0.tar.gz", hash = "sha256:60023a7e879aa4fbb1fed99d487fe407b2ebf6569603e5047cfe304cebdaa0e5", size = 583820, upload-time = "2026-03-18T18:43:08.017Z" } +sdist = { url = "https://files.pythonhosted.org/packages/b9/7e/672f533dee813028d2c699bfd2a7f52c9118d7353680d9aa44b9e23f717f/anthropic-0.96.0.tar.gz", hash = "sha256:9de947b737f39452f68aa520f1c2239d44119c9b73b0fb6d4e6ca80f00279ee6", size = 658210, upload-time = "2026-04-16T14:28:02.846Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/63/5f/67db29c6e5d16c8c9c4652d3efb934d89cb750cad201539141781d8eae14/anthropic-0.86.0-py3-none-any.whl", hash = "sha256:9d2bbd339446acce98858c5627d33056efe01f70435b22b63546fe7edae0cd57", size = 469400, upload-time = "2026-03-18T18:43:06.526Z" }, + { url = "https://files.pythonhosted.org/packages/48/5a/72f33204064b6e87601a71a6baf8d855769f8a0c1eaae8d06a1094872371/anthropic-0.96.0-py3-none-any.whl", hash = "sha256:9a6e335a354602a521cd9e777e92bfd46ba6e115bf9bbfe6135311e8fb2015b2", size = 635930, upload-time = "2026-04-16T14:28:01.436Z" }, ] [[package]] @@ -221,16 +221,16 @@ css = [ [[package]] name = "build" -version = "1.4.2" +version = "1.4.3" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "colorama", marker = "os_name == 'nt'" }, { name = "packaging" }, { name = "pyproject-hooks" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/6c/1d/ab15c8ac57f4ee8778d7633bc6685f808ab414437b8644f555389cdc875e/build-1.4.2.tar.gz", hash = "sha256:35b14e1ee329c186d3f08466003521ed7685ec15ecffc07e68d706090bf161d1", size = 83433, upload-time = "2026-03-25T14:20:27.659Z" } +sdist = { url = "https://files.pythonhosted.org/packages/3f/16/4b272700dea44c1d2e8ca963ebb3c684efe22b3eba8cfa31c5fdb60de707/build-1.4.3.tar.gz", hash = "sha256:5aa4231ae0e807efdf1fd0623e07366eca2ab215921345a2e38acdd5d0fa0a74", size = 89314, upload-time = "2026-04-10T21:25:40.857Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/4a/57/3b7d4dd193ade4641c865bc2b93aeeb71162e81fc348b8dad020215601ed/build-1.4.2-py3-none-any.whl", hash = "sha256:7a4d8651ea877cb2a89458b1b198f2e69f536c95e89129dbf5d448045d60db88", size = 24643, upload-time = "2026-03-25T14:20:26.568Z" }, + { url = "https://files.pythonhosted.org/packages/b2/30/f169e1d8b2071beaf8b97088787e30662b1d8fb82f8c0941d14678c0cbf1/build-1.4.3-py3-none-any.whl", hash = "sha256:1bc22b19b383303de8f2c8554c9a32894a58d3f185fe3756b0b20d255bee9a38", size = 26171, upload-time = "2026-04-10T21:25:39.671Z" }, ] [[package]] @@ -289,48 +289,48 @@ wheels = [ [[package]] name = "charset-normalizer" -version = "3.4.6" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/7b/60/e3bec1881450851b087e301bedc3daa9377a4d45f1c26aa90b0b235e38aa/charset_normalizer-3.4.6.tar.gz", hash = "sha256:1ae6b62897110aa7c79ea2f5dd38d1abca6db663687c0b1ad9aed6f6bae3d9d6", size = 143363, upload-time = "2026-03-15T18:53:25.478Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/62/28/ff6f234e628a2de61c458be2779cb182bc03f6eec12200d4a525bbfc9741/charset_normalizer-3.4.6-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:82060f995ab5003a2d6e0f4ad29065b7672b6593c8c63559beefe5b443242c3e", size = 293582, upload-time = "2026-03-15T18:50:25.454Z" }, - { url = "https://files.pythonhosted.org/packages/1c/b7/b1a117e5385cbdb3205f6055403c2a2a220c5ea80b8716c324eaf75c5c95/charset_normalizer-3.4.6-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:60c74963d8350241a79cb8feea80e54d518f72c26db618862a8f53e5023deaf9", size = 197240, upload-time = "2026-03-15T18:50:27.196Z" }, - { url = "https://files.pythonhosted.org/packages/a1/5f/2574f0f09f3c3bc1b2f992e20bce6546cb1f17e111c5be07308dc5427956/charset_normalizer-3.4.6-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f6e4333fb15c83f7d1482a76d45a0818897b3d33f00efd215528ff7c51b8e35d", size = 217363, upload-time = "2026-03-15T18:50:28.601Z" }, - { url = "https://files.pythonhosted.org/packages/4a/d1/0ae20ad77bc949ddd39b51bf383b6ca932f2916074c95cad34ae465ab71f/charset_normalizer-3.4.6-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:bc72863f4d9aba2e8fd9085e63548a324ba706d2ea2c83b260da08a59b9482de", size = 212994, upload-time = "2026-03-15T18:50:30.102Z" }, - { url = "https://files.pythonhosted.org/packages/60/ac/3233d262a310c1b12633536a07cde5ddd16985e6e7e238e9f3f9423d8eb9/charset_normalizer-3.4.6-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9cc4fc6c196d6a8b76629a70ddfcd4635a6898756e2d9cac5565cf0654605d73", size = 204697, upload-time = "2026-03-15T18:50:31.654Z" }, - { url = "https://files.pythonhosted.org/packages/25/3c/8a18fc411f085b82303cfb7154eed5bd49c77035eb7608d049468b53f87c/charset_normalizer-3.4.6-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:0c173ce3a681f309f31b87125fecec7a5d1347261ea11ebbb856fa6006b23c8c", size = 191673, upload-time = "2026-03-15T18:50:33.433Z" }, - { url = "https://files.pythonhosted.org/packages/ff/a7/11cfe61d6c5c5c7438d6ba40919d0306ed83c9ab957f3d4da2277ff67836/charset_normalizer-3.4.6-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c907cdc8109f6c619e6254212e794d6548373cc40e1ec75e6e3823d9135d29cc", size = 201120, upload-time = "2026-03-15T18:50:35.105Z" }, - { url = "https://files.pythonhosted.org/packages/b5/10/cf491fa1abd47c02f69687046b896c950b92b6cd7337a27e6548adbec8e4/charset_normalizer-3.4.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:404a1e552cf5b675a87f0651f8b79f5f1e6fd100ee88dc612f89aa16abd4486f", size = 200911, upload-time = "2026-03-15T18:50:36.819Z" }, - { url = "https://files.pythonhosted.org/packages/28/70/039796160b48b18ed466fde0af84c1b090c4e288fae26cd674ad04a2d703/charset_normalizer-3.4.6-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:e3c701e954abf6fc03a49f7c579cc80c2c6cc52525340ca3186c41d3f33482ef", size = 192516, upload-time = "2026-03-15T18:50:38.228Z" }, - { url = "https://files.pythonhosted.org/packages/ff/34/c56f3223393d6ff3124b9e78f7de738047c2d6bc40a4f16ac0c9d7a1cb3c/charset_normalizer-3.4.6-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:7a6967aaf043bceabab5412ed6bd6bd26603dae84d5cb75bf8d9a74a4959d398", size = 218795, upload-time = "2026-03-15T18:50:39.664Z" }, - { url = "https://files.pythonhosted.org/packages/e8/3b/ce2d4f86c5282191a041fdc5a4ce18f1c6bd40a5bd1f74cf8625f08d51c1/charset_normalizer-3.4.6-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:5feb91325bbceade6afab43eb3b508c63ee53579fe896c77137ded51c6b6958e", size = 201833, upload-time = "2026-03-15T18:50:41.552Z" }, - { url = "https://files.pythonhosted.org/packages/3b/9b/b6a9f76b0fd7c5b5ec58b228ff7e85095370282150f0bd50b3126f5506d6/charset_normalizer-3.4.6-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:f820f24b09e3e779fe84c3c456cb4108a7aa639b0d1f02c28046e11bfcd088ed", size = 213920, upload-time = "2026-03-15T18:50:43.33Z" }, - { url = "https://files.pythonhosted.org/packages/ae/98/7bc23513a33d8172365ed30ee3a3b3fe1ece14a395e5fc94129541fc6003/charset_normalizer-3.4.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:b35b200d6a71b9839a46b9b7fff66b6638bb52fc9658aa58796b0326595d3021", size = 206951, upload-time = "2026-03-15T18:50:44.789Z" }, - { url = "https://files.pythonhosted.org/packages/32/73/c0b86f3d1458468e11aec870e6b3feac931facbe105a894b552b0e518e79/charset_normalizer-3.4.6-cp311-cp311-win32.whl", hash = "sha256:9ca4c0b502ab399ef89248a2c84c54954f77a070f28e546a85e91da627d1301e", size = 143703, upload-time = "2026-03-15T18:50:46.103Z" }, - { url = "https://files.pythonhosted.org/packages/c6/e3/76f2facfe8eddee0bbd38d2594e709033338eae44ebf1738bcefe0a06185/charset_normalizer-3.4.6-cp311-cp311-win_amd64.whl", hash = "sha256:a9e68c9d88823b274cf1e72f28cb5dc89c990edf430b0bfd3e2fb0785bfeabf4", size = 153857, upload-time = "2026-03-15T18:50:47.563Z" }, - { url = "https://files.pythonhosted.org/packages/e2/dc/9abe19c9b27e6cd3636036b9d1b387b78c40dedbf0b47f9366737684b4b0/charset_normalizer-3.4.6-cp311-cp311-win_arm64.whl", hash = "sha256:97d0235baafca5f2b09cf332cc275f021e694e8362c6bb9c96fc9a0eb74fc316", size = 142751, upload-time = "2026-03-15T18:50:49.234Z" }, - { url = "https://files.pythonhosted.org/packages/e5/62/c0815c992c9545347aeea7859b50dc9044d147e2e7278329c6e02ac9a616/charset_normalizer-3.4.6-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:2ef7fedc7a6ecbe99969cd09632516738a97eeb8bd7258bf8a0f23114c057dab", size = 295154, upload-time = "2026-03-15T18:50:50.88Z" }, - { url = "https://files.pythonhosted.org/packages/a8/37/bdca6613c2e3c58c7421891d80cc3efa1d32e882f7c4a7ee6039c3fc951a/charset_normalizer-3.4.6-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a4ea868bc28109052790eb2b52a9ab33f3aa7adc02f96673526ff47419490e21", size = 199191, upload-time = "2026-03-15T18:50:52.658Z" }, - { url = "https://files.pythonhosted.org/packages/6c/92/9934d1bbd69f7f398b38c5dae1cbf9cc672e7c34a4adf7b17c0a9c17d15d/charset_normalizer-3.4.6-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:836ab36280f21fc1a03c99cd05c6b7af70d2697e374c7af0b61ed271401a72a2", size = 218674, upload-time = "2026-03-15T18:50:54.102Z" }, - { url = "https://files.pythonhosted.org/packages/af/90/25f6ab406659286be929fd89ab0e78e38aa183fc374e03aa3c12d730af8a/charset_normalizer-3.4.6-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f1ce721c8a7dfec21fcbdfe04e8f68174183cf4e8188e0645e92aa23985c57ff", size = 215259, upload-time = "2026-03-15T18:50:55.616Z" }, - { url = "https://files.pythonhosted.org/packages/4e/ef/79a463eb0fff7f96afa04c1d4c51f8fc85426f918db467854bfb6a569ce3/charset_normalizer-3.4.6-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0e28d62a8fc7a1fa411c43bd65e346f3bce9716dc51b897fbe930c5987b402d5", size = 207276, upload-time = "2026-03-15T18:50:57.054Z" }, - { url = "https://files.pythonhosted.org/packages/f7/72/d0426afec4b71dc159fa6b4e68f868cd5a3ecd918fec5813a15d292a7d10/charset_normalizer-3.4.6-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:530d548084c4a9f7a16ed4a294d459b4f229db50df689bfe92027452452943a0", size = 195161, upload-time = "2026-03-15T18:50:58.686Z" }, - { url = "https://files.pythonhosted.org/packages/bf/18/c82b06a68bfcb6ce55e508225d210c7e6a4ea122bfc0748892f3dc4e8e11/charset_normalizer-3.4.6-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:30f445ae60aad5e1f8bdbb3108e39f6fbc09f4ea16c815c66578878325f8f15a", size = 203452, upload-time = "2026-03-15T18:51:00.196Z" }, - { url = "https://files.pythonhosted.org/packages/44/d6/0c25979b92f8adafdbb946160348d8d44aa60ce99afdc27df524379875cb/charset_normalizer-3.4.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:ac2393c73378fea4e52aa56285a3d64be50f1a12395afef9cce47772f60334c2", size = 202272, upload-time = "2026-03-15T18:51:01.703Z" }, - { url = "https://files.pythonhosted.org/packages/2e/3d/7fea3e8fe84136bebbac715dd1221cc25c173c57a699c030ab9b8900cbb7/charset_normalizer-3.4.6-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:90ca27cd8da8118b18a52d5f547859cc1f8354a00cd1e8e5120df3e30d6279e5", size = 195622, upload-time = "2026-03-15T18:51:03.526Z" }, - { url = "https://files.pythonhosted.org/packages/57/8a/d6f7fd5cb96c58ef2f681424fbca01264461336d2a7fc875e4446b1f1346/charset_normalizer-3.4.6-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:8e5a94886bedca0f9b78fecd6afb6629142fd2605aa70a125d49f4edc6037ee6", size = 220056, upload-time = "2026-03-15T18:51:05.269Z" }, - { url = "https://files.pythonhosted.org/packages/16/50/478cdda782c8c9c3fb5da3cc72dd7f331f031e7f1363a893cdd6ca0f8de0/charset_normalizer-3.4.6-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:695f5c2823691a25f17bc5d5ffe79fa90972cc34b002ac6c843bb8a1720e950d", size = 203751, upload-time = "2026-03-15T18:51:06.858Z" }, - { url = "https://files.pythonhosted.org/packages/75/fc/cc2fcac943939c8e4d8791abfa139f685e5150cae9f94b60f12520feaa9b/charset_normalizer-3.4.6-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:231d4da14bcd9301310faf492051bee27df11f2bc7549bc0bb41fef11b82daa2", size = 216563, upload-time = "2026-03-15T18:51:08.564Z" }, - { url = "https://files.pythonhosted.org/packages/a8/b7/a4add1d9a5f68f3d037261aecca83abdb0ab15960a3591d340e829b37298/charset_normalizer-3.4.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:a056d1ad2633548ca18ffa2f85c202cfb48b68615129143915b8dc72a806a923", size = 209265, upload-time = "2026-03-15T18:51:10.312Z" }, - { url = "https://files.pythonhosted.org/packages/6c/18/c094561b5d64a24277707698e54b7f67bd17a4f857bbfbb1072bba07c8bf/charset_normalizer-3.4.6-cp312-cp312-win32.whl", hash = "sha256:c2274ca724536f173122f36c98ce188fd24ce3dad886ec2b7af859518ce008a4", size = 144229, upload-time = "2026-03-15T18:51:11.694Z" }, - { url = "https://files.pythonhosted.org/packages/ab/20/0567efb3a8fd481b8f34f739ebddc098ed062a59fed41a8d193a61939e8f/charset_normalizer-3.4.6-cp312-cp312-win_amd64.whl", hash = "sha256:c8ae56368f8cc97c7e40a7ee18e1cedaf8e780cd8bc5ed5ac8b81f238614facb", size = 154277, upload-time = "2026-03-15T18:51:13.004Z" }, - { url = "https://files.pythonhosted.org/packages/15/57/28d79b44b51933119e21f65479d0864a8d5893e494cf5daab15df0247c17/charset_normalizer-3.4.6-cp312-cp312-win_arm64.whl", hash = "sha256:899d28f422116b08be5118ef350c292b36fc15ec2daeb9ea987c89281c7bb5c4", size = 142817, upload-time = "2026-03-15T18:51:14.408Z" }, - { url = "https://files.pythonhosted.org/packages/2a/68/687187c7e26cb24ccbd88e5069f5ef00eba804d36dde11d99aad0838ab45/charset_normalizer-3.4.6-py3-none-any.whl", hash = "sha256:947cf925bc916d90adba35a64c82aace04fa39b46b52d4630ece166655905a69", size = 61455, upload-time = "2026-03-15T18:53:23.833Z" }, +version = "3.4.7" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/a1/67fe25fac3c7642725500a3f6cfe5821ad557c3abb11c9d20d12c7008d3e/charset_normalizer-3.4.7.tar.gz", hash = "sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5", size = 144271, upload-time = "2026-04-02T09:28:39.342Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c2/d7/b5b7020a0565c2e9fa8c09f4b5fa6232feb326b8c20081ccded47ea368fd/charset_normalizer-3.4.7-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7", size = 309705, upload-time = "2026-04-02T09:26:02.191Z" }, + { url = "https://files.pythonhosted.org/packages/5a/53/58c29116c340e5456724ecd2fff4196d236b98f3da97b404bc5e51ac3493/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7", size = 206419, upload-time = "2026-04-02T09:26:03.583Z" }, + { url = "https://files.pythonhosted.org/packages/b2/02/e8146dc6591a37a00e5144c63f29fb7c97a734ea8a111190783c0e60ab63/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e", size = 227901, upload-time = "2026-04-02T09:26:04.738Z" }, + { url = "https://files.pythonhosted.org/packages/fb/73/77486c4cd58f1267bf17db420e930c9afa1b3be3fe8c8b8ebbebc9624359/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c", size = 222742, upload-time = "2026-04-02T09:26:06.36Z" }, + { url = "https://files.pythonhosted.org/packages/a1/fa/f74eb381a7d94ded44739e9d94de18dc5edc9c17fb8c11f0a6890696c0a9/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df", size = 214061, upload-time = "2026-04-02T09:26:08.347Z" }, + { url = "https://files.pythonhosted.org/packages/dc/92/42bd3cefcf7687253fb86694b45f37b733c97f59af3724f356fa92b8c344/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265", size = 199239, upload-time = "2026-04-02T09:26:09.823Z" }, + { url = "https://files.pythonhosted.org/packages/4c/3d/069e7184e2aa3b3cddc700e3dd267413dc259854adc3380421c805c6a17d/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4", size = 210173, upload-time = "2026-04-02T09:26:10.953Z" }, + { url = "https://files.pythonhosted.org/packages/62/51/9d56feb5f2e7074c46f93e0ebdbe61f0848ee246e2f0d89f8e20b89ebb8f/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:e060d01aec0a910bdccb8be71faf34e7799ce36950f8294c8bf612cba65a2c9e", size = 209841, upload-time = "2026-04-02T09:26:12.142Z" }, + { url = "https://files.pythonhosted.org/packages/d2/59/893d8f99cc4c837dda1fe2f1139079703deb9f321aabcb032355de13b6c7/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38", size = 200304, upload-time = "2026-04-02T09:26:13.711Z" }, + { url = "https://files.pythonhosted.org/packages/7d/1d/ee6f3be3464247578d1ed5c46de545ccc3d3ff933695395c402c21fa6b77/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c", size = 229455, upload-time = "2026-04-02T09:26:14.941Z" }, + { url = "https://files.pythonhosted.org/packages/54/bb/8fb0a946296ea96a488928bdce8ef99023998c48e4713af533e9bb98ef07/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b", size = 210036, upload-time = "2026-04-02T09:26:16.478Z" }, + { url = "https://files.pythonhosted.org/packages/9a/bc/015b2387f913749f82afd4fcba07846d05b6d784dd16123cb66860e0237d/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:d635aab80466bc95771bb78d5370e74d36d1fe31467b6b29b8b57b2a3cd7d22c", size = 224739, upload-time = "2026-04-02T09:26:17.751Z" }, + { url = "https://files.pythonhosted.org/packages/17/ab/63133691f56baae417493cba6b7c641571a2130eb7bceba6773367ab9ec5/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ae196f021b5e7c78e918242d217db021ed2a6ace2bc6ae94c0fc596221c7f58d", size = 216277, upload-time = "2026-04-02T09:26:18.981Z" }, + { url = "https://files.pythonhosted.org/packages/06/6d/3be70e827977f20db77c12a97e6a9f973631a45b8d186c084527e53e77a4/charset_normalizer-3.4.7-cp311-cp311-win32.whl", hash = "sha256:adb2597b428735679446b46c8badf467b4ca5f5056aae4d51a19f9570301b1ad", size = 147819, upload-time = "2026-04-02T09:26:20.295Z" }, + { url = "https://files.pythonhosted.org/packages/20/d9/5f67790f06b735d7c7637171bbfd89882ad67201891b7275e51116ed8207/charset_normalizer-3.4.7-cp311-cp311-win_amd64.whl", hash = "sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00", size = 159281, upload-time = "2026-04-02T09:26:21.74Z" }, + { url = "https://files.pythonhosted.org/packages/ca/83/6413f36c5a34afead88ce6f66684d943d91f233d76dd083798f9602b75ae/charset_normalizer-3.4.7-cp311-cp311-win_arm64.whl", hash = "sha256:d4a48e5b3c2a489fae013b7589308a40146ee081f6f509e047e0e096084ceca1", size = 147843, upload-time = "2026-04-02T09:26:22.901Z" }, + { url = "https://files.pythonhosted.org/packages/0c/eb/4fc8d0a7110eb5fc9cc161723a34a8a6c200ce3b4fbf681bc86feee22308/charset_normalizer-3.4.7-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46", size = 311328, upload-time = "2026-04-02T09:26:24.331Z" }, + { url = "https://files.pythonhosted.org/packages/f8/e3/0fadc706008ac9d7b9b5be6dc767c05f9d3e5df51744ce4cc9605de7b9f4/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2", size = 208061, upload-time = "2026-04-02T09:26:25.568Z" }, + { url = "https://files.pythonhosted.org/packages/42/f0/3dd1045c47f4a4604df85ec18ad093912ae1344ac706993aff91d38773a2/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b", size = 229031, upload-time = "2026-04-02T09:26:26.865Z" }, + { url = "https://files.pythonhosted.org/packages/dc/67/675a46eb016118a2fbde5a277a5d15f4f69d5f3f5f338e5ee2f8948fcf43/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a", size = 225239, upload-time = "2026-04-02T09:26:28.044Z" }, + { url = "https://files.pythonhosted.org/packages/4b/f8/d0118a2f5f23b02cd166fa385c60f9b0d4f9194f574e2b31cef350ad7223/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116", size = 216589, upload-time = "2026-04-02T09:26:29.239Z" }, + { url = "https://files.pythonhosted.org/packages/b1/f1/6d2b0b261b6c4ceef0fcb0d17a01cc5bc53586c2d4796fa04b5c540bc13d/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb", size = 202733, upload-time = "2026-04-02T09:26:30.5Z" }, + { url = "https://files.pythonhosted.org/packages/6f/c0/7b1f943f7e87cc3db9626ba17807d042c38645f0a1d4415c7a14afb5591f/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1", size = 212652, upload-time = "2026-04-02T09:26:31.709Z" }, + { url = "https://files.pythonhosted.org/packages/38/dd/5a9ab159fe45c6e72079398f277b7d2b523e7f716acc489726115a910097/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15", size = 211229, upload-time = "2026-04-02T09:26:33.282Z" }, + { url = "https://files.pythonhosted.org/packages/d5/ff/531a1cad5ca855d1c1a8b69cb71abfd6d85c0291580146fda7c82857caa1/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5", size = 203552, upload-time = "2026-04-02T09:26:34.845Z" }, + { url = "https://files.pythonhosted.org/packages/c1/4c/a5fb52d528a8ca41f7598cb619409ece30a169fbdf9cdce592e53b46c3a6/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d", size = 230806, upload-time = "2026-04-02T09:26:36.152Z" }, + { url = "https://files.pythonhosted.org/packages/59/7a/071feed8124111a32b316b33ae4de83d36923039ef8cf48120266844285b/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7", size = 212316, upload-time = "2026-04-02T09:26:37.672Z" }, + { url = "https://files.pythonhosted.org/packages/fd/35/f7dba3994312d7ba508e041eaac39a36b120f32d4c8662b8814dab876431/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464", size = 227274, upload-time = "2026-04-02T09:26:38.93Z" }, + { url = "https://files.pythonhosted.org/packages/8a/2d/a572df5c9204ab7688ec1edc895a73ebded3b023bb07364710b05dd1c9be/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49", size = 218468, upload-time = "2026-04-02T09:26:40.17Z" }, + { url = "https://files.pythonhosted.org/packages/86/eb/890922a8b03a568ca2f336c36585a4713c55d4d67bf0f0c78924be6315ca/charset_normalizer-3.4.7-cp312-cp312-win32.whl", hash = "sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c", size = 148460, upload-time = "2026-04-02T09:26:41.416Z" }, + { url = "https://files.pythonhosted.org/packages/35/d9/0e7dffa06c5ab081f75b1b786f0aefc88365825dfcd0ac544bdb7b2b6853/charset_normalizer-3.4.7-cp312-cp312-win_amd64.whl", hash = "sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6", size = 159330, upload-time = "2026-04-02T09:26:42.554Z" }, + { url = "https://files.pythonhosted.org/packages/9e/5d/481bcc2a7c88ea6b0878c299547843b2521ccbc40980cb406267088bc701/charset_normalizer-3.4.7-cp312-cp312-win_arm64.whl", hash = "sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d", size = 147828, upload-time = "2026-04-02T09:26:44.075Z" }, + { url = "https://files.pythonhosted.org/packages/db/8f/61959034484a4a7c527811f4721e75d02d653a35afb0b6054474d8185d4c/charset_normalizer-3.4.7-py3-none-any.whl", hash = "sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d", size = 61958, upload-time = "2026-04-02T09:28:37.794Z" }, ] [[package]] name = "chromadb" -version = "1.5.5" +version = "1.5.8" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "bcrypt" }, @@ -361,25 +361,25 @@ dependencies = [ { name = "typing-extensions" }, { name = "uvicorn", extra = ["standard"] }, ] -sdist = { url = "https://files.pythonhosted.org/packages/3a/6d/ab03e16be3ec663e353166f38be082efb51c0988687f8c8eee1416a7e732/chromadb-1.5.5.tar.gz", hash = "sha256:8d669285b77cc288db27583a57b2f85ba451a9b8e3bef85a260cd78e6b57be35", size = 2411397, upload-time = "2026-03-10T09:30:01.987Z" } +sdist = { url = "https://files.pythonhosted.org/packages/98/45/39696984553ede2bed1c54550f194111c66ef6ba86c0cd5bd8099d39b2c3/chromadb-1.5.8.tar.gz", hash = "sha256:9f5dfaea989128793c4e1928de6a150d70ae55e44403d85448be94ff32f2c962", size = 2515918, upload-time = "2026-04-16T23:35:11.731Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/f0/62/ee578f8ccd62928257558b13a3e7c236e402cfb319c9b201b6a75897d644/chromadb-1.5.5-cp39-abi3-macosx_10_12_x86_64.whl", hash = "sha256:d590998ed81164afbfb1734bb534b25ec2c9810fc1c5ce53bf8f7ac644a79887", size = 20800888, upload-time = "2026-03-10T09:29:59.546Z" }, - { url = "https://files.pythonhosted.org/packages/f8/ce/430a87d906f79cdc7e23efcd89dd237e3dbedaf6704b40ce1da127993bf8/chromadb-1.5.5-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:5ff2912d20a82fdbf4e27ff3e1c91dab25e2ba2c629f9739bc12c11a3151aac7", size = 20091810, upload-time = "2026-03-10T09:29:56.044Z" }, - { url = "https://files.pythonhosted.org/packages/a8/5a/11543a76ab25c55bec6133bb98ce0dc0f4850acb36600344d8286734a051/chromadb-1.5.5-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2f54e7736ae0eeec436a1c1fb04b77b2c6c4108996790ef16f88327e38ad13cd", size = 20740649, upload-time = "2026-03-10T09:29:49.346Z" }, - { url = "https://files.pythonhosted.org/packages/d3/66/e0b35c41be7c02d6fa37f6c8f61a16b7b20607ddc847574e9a5503fe853b/chromadb-1.5.5-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bb238ae508a6ce68fdd7875e040d7e5aa29d6e40fb651b51f5537b7cda789762", size = 21589423, upload-time = "2026-03-10T09:29:52.724Z" }, - { url = "https://files.pythonhosted.org/packages/a2/df/ce1ffcc0ad3eef8bd35b920809b990e6925ba94b2580dc5bd7ccde0fc06a/chromadb-1.5.5-cp39-abi3-win_amd64.whl", hash = "sha256:3953403b63bb1c05405d10db36d183c4d19a027938c15898510d11943499046f", size = 21915873, upload-time = "2026-03-10T09:30:21.349Z" }, + { url = "https://files.pythonhosted.org/packages/e9/20/5c0218014f1dd8f39e48e9e7d4dc15bd36039510747a167a29e5999d1d68/chromadb-1.5.8-cp39-abi3-macosx_10_12_x86_64.whl", hash = "sha256:397331d749aefc95222733509172b4598688d7306659459a59421f8763ba83b7", size = 22482099, upload-time = "2026-04-16T23:35:09.27Z" }, + { url = "https://files.pythonhosted.org/packages/82/90/4d8ebfc4c168295b2e903dd385f4b76b155980d3c8db69e7383a6770e178/chromadb-1.5.8-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:852aae44eac81d73e9c349e8daa2eaa34873576afeb2604765aeebd01eb03346", size = 21595370, upload-time = "2026-04-16T23:35:06.091Z" }, + { url = "https://files.pythonhosted.org/packages/7f/de/3c4c0661152cef02b5d0684fa8d0195aa36d3242e56903d211ff2acd3424/chromadb-1.5.8-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:33976c781a9d6e6386a15481a58f5bde88e68fce87925aa5f30bca4f142c8303", size = 22568047, upload-time = "2026-04-16T23:34:59.376Z" }, + { url = "https://files.pythonhosted.org/packages/4d/45/53556edcbbd273c54b81e07c1b21c79f7209ab4edf9603f94babb4ecf4f6/chromadb-1.5.8-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9494f0cf734dcfd2976890f9e3a446fdd5ca565375ffebe0de388ce994ac1f5e", size = 23214104, upload-time = "2026-04-16T23:35:02.583Z" }, + { url = "https://files.pythonhosted.org/packages/5e/ea/d6df4e0718bde638563225c8690be110f53874d5b5fb74f347d9a7747a1f/chromadb-1.5.8-cp39-abi3-win_amd64.whl", hash = "sha256:b7490b26843c9851e1e9bb0333a1d584e5e77b5a49a78713424600af9b3a28e6", size = 23413244, upload-time = "2026-04-16T23:35:14.328Z" }, ] [[package]] name = "click" -version = "8.3.1" +version = "8.3.2" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "colorama", marker = "sys_platform == 'win32'" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/3d/fa/656b739db8587d7b5dfa22e22ed02566950fbfbcdc20311993483657a5c0/click-8.3.1.tar.gz", hash = "sha256:12ff4785d337a1bb490bb7e9c2b1ee5da3112e94a8622f26a6c77f5d2fc6842a", size = 295065, upload-time = "2025-11-15T20:45:42.706Z" } +sdist = { url = "https://files.pythonhosted.org/packages/57/75/31212c6bf2503fdf920d87fee5d7a86a2e3bcf444984126f13d8e4016804/click-8.3.2.tar.gz", hash = "sha256:14162b8b3b3550a7d479eafa77dfd3c38d9dc8951f6f69c78913a8f9a7540fd5", size = 302856, upload-time = "2026-04-03T19:14:45.118Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/98/78/01c019cdb5d6498122777c1a43056ebb3ebfeef2076d9d026bfe15583b2b/click-8.3.1-py3-none-any.whl", hash = "sha256:981153a64e25f12d547d3426c367a4857371575ee7ad18df2a6183ab0545b2a6", size = 108274, upload-time = "2025-11-15T20:45:41.139Z" }, + { url = "https://files.pythonhosted.org/packages/e4/20/71885d8b97d4f3dde17b1fdb92dbd4908b00541c5a3379787137285f602e/click-8.3.2-py3-none-any.whl", hash = "sha256:1924d2c27c5653561cd2cae4548d1406039cb79b858b747cfea24924bbc1616d", size = 108379, upload-time = "2026-04-03T19:14:43.505Z" }, ] [[package]] @@ -416,10 +416,10 @@ wheels = [ [[package]] name = "cuda-pathfinder" -version = "1.5.0" +version = "1.5.3" source = { registry = "https://pypi.org/simple" } wheels = [ - { url = "https://files.pythonhosted.org/packages/93/66/0c02bd330e7d976f83fa68583d6198d76f23581bcbb5c0e98a6148f326e5/cuda_pathfinder-1.5.0-py3-none-any.whl", hash = "sha256:498f90a9e9de36044a7924742aecce11c50c49f735f1bc53e05aa46de9ea4110", size = 49739, upload-time = "2026-03-24T21:14:30.869Z" }, + { url = "https://files.pythonhosted.org/packages/d3/d6/ac63065d33dd700fee7ebd7d287332401b54e31b9346e142f871e1f0b116/cuda_pathfinder-1.5.3-py3-none-any.whl", hash = "sha256:dff021123aedbb4117cc7ec81717bbfe198fb4e8b5f1ee57e0e084fec5c8577d", size = 49991, upload-time = "2026-04-14T20:09:27.037Z" }, ] [[package]] @@ -520,11 +520,11 @@ wheels = [ [[package]] name = "docstring-parser" -version = "0.17.0" +version = "0.18.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/b2/9d/c3b43da9515bd270df0f80548d9944e389870713cc1fe2b8fb35fe2bcefd/docstring_parser-0.17.0.tar.gz", hash = "sha256:583de4a309722b3315439bb31d64ba3eebada841f2e2cee23b99df001434c912", size = 27442, upload-time = "2025-07-21T07:35:01.868Z" } +sdist = { url = "https://files.pythonhosted.org/packages/e0/4d/f332313098c1de1b2d2ff91cf2674415cc7cddab2ca1b01ae29774bd5fdf/docstring_parser-0.18.0.tar.gz", hash = "sha256:292510982205c12b1248696f44959db3cdd1740237a968ea1e2e7a900eeb2015", size = 29341, upload-time = "2026-04-14T04:09:19.867Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/55/e2/2537ebcff11c1ee1ff17d8d0b6f4db75873e3b0fb32c2d4a2ee31ecb310a/docstring_parser-0.17.0-py3-none-any.whl", hash = "sha256:cf2569abd23dce8099b300f9b4fa8191e9582dda731fd533daf54c4551658708", size = 36896, upload-time = "2025-07-21T07:35:00.684Z" }, + { url = "https://files.pythonhosted.org/packages/a7/5f/ed01f9a3cdffbd5a008556fc7b2a08ddb1cc6ace7effa7340604b1d16699/docstring_parser-0.18.0-py3-none-any.whl", hash = "sha256:b3fcbed555c47d8479be0796ef7e19c2670d428d72e96da63f3a40122860374b", size = 22484, upload-time = "2026-04-14T04:09:18.638Z" }, ] [[package]] @@ -547,7 +547,7 @@ wheels = [ [[package]] name = "fastapi" -version = "0.135.2" +version = "0.136.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "annotated-doc" }, @@ -556,9 +556,9 @@ dependencies = [ { name = "typing-extensions" }, { name = "typing-inspection" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/c4/73/5903c4b13beae98618d64eb9870c3fac4f605523dd0312ca5c80dadbd5b9/fastapi-0.135.2.tar.gz", hash = "sha256:88a832095359755527b7f63bb4c6bc9edb8329a026189eed83d6c1afcf419d56", size = 395833, upload-time = "2026-03-23T14:12:41.697Z" } +sdist = { url = "https://files.pythonhosted.org/packages/4e/d9/e66315807e41e69e7f6a1b42a162dada2f249c5f06ad3f1a95f84ab336ef/fastapi-0.136.0.tar.gz", hash = "sha256:cf08e067cc66e106e102d9ba659463abfac245200752f8a5b7b1e813de4ff73e", size = 396607, upload-time = "2026-04-16T11:47:13.623Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/8f/ea/18f6d0457f9efb2fc6fa594857f92810cadb03024975726db6546b3d6fcf/fastapi-0.135.2-py3-none-any.whl", hash = "sha256:0af0447d541867e8db2a6a25c23a8c4bd80e2394ac5529bd87501bbb9e240ca5", size = 117407, upload-time = "2026-03-23T14:12:43.284Z" }, + { url = "https://files.pythonhosted.org/packages/26/a3/0bd5f0cdb0bbc92650e8dc457e9250358411ee5d1b65e42b6632387daf81/fastapi-0.136.0-py3-none-any.whl", hash = "sha256:8793d44ec7378e2be07f8a013cf7f7aa47d6327d0dfe9804862688ec4541a6b4", size = 117556, upload-time = "2026-04-16T11:47:11.922Z" }, ] [[package]] @@ -572,11 +572,11 @@ wheels = [ [[package]] name = "filelock" -version = "3.25.2" +version = "3.28.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/94/b8/00651a0f559862f3bb7d6f7477b192afe3f583cc5e26403b44e59a55ab34/filelock-3.25.2.tar.gz", hash = "sha256:b64ece2b38f4ca29dd3e810287aa8c48182bbecd1ae6e9ae126c9b35f1382694", size = 40480, upload-time = "2026-03-11T20:45:38.487Z" } +sdist = { url = "https://files.pythonhosted.org/packages/d6/17/6e8890271880903e3538660a21d63a6c1fea969ac71d0d6b608b78727fa9/filelock-3.28.0.tar.gz", hash = "sha256:4ed1010aae813c4ee8d9c660e4792475ee60c4a0ba76073ceaf862bd317e3ca6", size = 56474, upload-time = "2026-04-14T22:54:33.625Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/a4/a5/842ae8f0c08b61d6484b52f99a03510a3a72d23141942d216ebe81fefbce/filelock-3.25.2-py3-none-any.whl", hash = "sha256:ca8afb0da15f229774c9ad1b455ed96e85a81373065fb10446672f64444ddf70", size = 26759, upload-time = "2026-03-11T20:45:37.437Z" }, + { url = "https://files.pythonhosted.org/packages/3b/21/2f728888c45033d34a417bfcd248ea2564c9e08ab1bfd301377cf05d5586/filelock-3.28.0-py3-none-any.whl", hash = "sha256:de9af6712788e7171df1b28b15eba2446c69721433fa427a9bee07b17820a9db", size = 39189, upload-time = "2026-04-14T22:54:32.037Z" }, ] [[package]] @@ -607,14 +607,14 @@ wheels = [ [[package]] name = "googleapis-common-protos" -version = "1.73.1" +version = "1.74.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "protobuf" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/a1/c0/4a54c386282c13449eca8bbe2ddb518181dc113e78d240458a68856b4d69/googleapis_common_protos-1.73.1.tar.gz", hash = "sha256:13114f0e9d2391756a0194c3a8131974ed7bffb06086569ba193364af59163b6", size = 147506, upload-time = "2026-03-26T22:17:38.451Z" } +sdist = { url = "https://files.pythonhosted.org/packages/20/18/a746c8344152d368a5aac738d4c857012f2c5d1fd2eac7e17b647a7861bd/googleapis_common_protos-1.74.0.tar.gz", hash = "sha256:57971e4eeeba6aad1163c1f0fc88543f965bb49129b8bb55b2b7b26ecab084f1", size = 151254, upload-time = "2026-04-02T21:23:26.679Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/dc/82/fcb6520612bec0c39b973a6c0954b6a0d948aadfe8f7e9487f60ceb8bfa6/googleapis_common_protos-1.73.1-py3-none-any.whl", hash = "sha256:e51f09eb0a43a8602f5a915870972e6b4a394088415c79d79605a46d8e826ee8", size = 297556, upload-time = "2026-03-26T22:15:58.455Z" }, + { url = "https://files.pythonhosted.org/packages/b6/b0/be5d3329badb9230b765de6eea66b73abd5944bdeb5afb3562ddcd80ae84/googleapis_common_protos-1.74.0-py3-none-any.whl", hash = "sha256:702216f78610bb510e3f12ac3cafd281b7ac45cc5d86e90ad87e4d301a3426b5", size = 300743, upload-time = "2026-04-02T21:22:49.108Z" }, ] [[package]] @@ -659,18 +659,18 @@ wheels = [ [[package]] name = "hf-xet" -version = "1.4.2" +version = "1.4.3" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/09/08/23c84a26716382c89151b5b447b4beb19e3345f3a93d3b73009a71a57ad3/hf_xet-1.4.2.tar.gz", hash = "sha256:b7457b6b482d9e0743bd116363239b1fa904a5e65deede350fbc0c4ea67c71ea", size = 672357, upload-time = "2026-03-13T06:58:51.077Z" } +sdist = { url = "https://files.pythonhosted.org/packages/53/92/ec9ad04d0b5728dca387a45af7bc98fbb0d73b2118759f5f6038b61a57e8/hf_xet-1.4.3.tar.gz", hash = "sha256:8ddedb73c8c08928c793df2f3401ec26f95be7f7e516a7bee2fbb546f6676113", size = 670477, upload-time = "2026-03-31T22:40:07.874Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/b4/86/b40b83a2ff03ef05c4478d2672b1fc2b9683ff870e2b25f4f3af240f2e7b/hf_xet-1.4.2-cp37-abi3-macosx_10_12_x86_64.whl", hash = "sha256:71f02d6e4cdd07f344f6844845d78518cc7186bd2bc52d37c3b73dc26a3b0bc5", size = 3800339, upload-time = "2026-03-13T06:58:36.245Z" }, - { url = "https://files.pythonhosted.org/packages/64/2e/af4475c32b4378b0e92a587adb1aa3ec53e3450fd3e5fe0372a874531c00/hf_xet-1.4.2-cp37-abi3-macosx_11_0_arm64.whl", hash = "sha256:e9b38d876e94d4bdcf650778d6ebbaa791dd28de08db9736c43faff06ede1b5a", size = 3559664, upload-time = "2026-03-13T06:58:34.787Z" }, - { url = "https://files.pythonhosted.org/packages/3c/4c/781267da3188db679e601de18112021a5cb16506fe86b246e22c5401a9c4/hf_xet-1.4.2-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:77e8c180b7ef12d8a96739a4e1e558847002afe9ea63b6f6358b2271a8bdda1c", size = 4217422, upload-time = "2026-03-13T06:58:27.472Z" }, - { url = "https://files.pythonhosted.org/packages/68/47/d6cf4a39ecf6c7705f887a46f6ef5c8455b44ad9eb0d391aa7e8a2ff7fea/hf_xet-1.4.2-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:c3b3c6a882016b94b6c210957502ff7877802d0dbda8ad142c8595db8b944271", size = 3992847, upload-time = "2026-03-13T06:58:25.989Z" }, - { url = "https://files.pythonhosted.org/packages/2d/ef/e80815061abff54697239803948abc665c6b1d237102c174f4f7a9a5ffc5/hf_xet-1.4.2-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:9d9a634cc929cfbaf2e1a50c0e532ae8c78fa98618426769480c58501e8c8ac2", size = 4193843, upload-time = "2026-03-13T06:58:44.59Z" }, - { url = "https://files.pythonhosted.org/packages/54/75/07f6aa680575d9646c4167db6407c41340cbe2357f5654c4e72a1b01ca14/hf_xet-1.4.2-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:6b0932eb8b10317ea78b7da6bab172b17be03bbcd7809383d8d5abd6a2233e04", size = 4432751, upload-time = "2026-03-13T06:58:46.533Z" }, - { url = "https://files.pythonhosted.org/packages/cd/71/193eabd7e7d4b903c4aa983a215509c6114915a5a237525ec562baddb868/hf_xet-1.4.2-cp37-abi3-win_amd64.whl", hash = "sha256:ad185719fb2e8ac26f88c8100562dbf9dbdcc3d9d2add00faa94b5f106aea53f", size = 3671149, upload-time = "2026-03-13T06:58:57.07Z" }, - { url = "https://files.pythonhosted.org/packages/b4/7e/ccf239da366b37ba7f0b36095450efae4a64980bdc7ec2f51354205fdf39/hf_xet-1.4.2-cp37-abi3-win_arm64.whl", hash = "sha256:32c012286b581f783653e718c1862aea5b9eb140631685bb0c5e7012c8719a87", size = 3533426, upload-time = "2026-03-13T06:58:55.46Z" }, + { url = "https://files.pythonhosted.org/packages/ac/9f/9c23e4a447b8f83120798f9279d0297a4d1360bdbf59ef49ebec78fe2545/hf_xet-1.4.3-cp37-abi3-macosx_10_12_x86_64.whl", hash = "sha256:d0da85329eaf196e03e90b84c2d0aca53bd4573d097a75f99609e80775f98025", size = 3805048, upload-time = "2026-03-31T22:39:53.105Z" }, + { url = "https://files.pythonhosted.org/packages/0b/f8/7aacb8e5f4a7899d39c787b5984e912e6c18b11be136ef13947d7a66d265/hf_xet-1.4.3-cp37-abi3-macosx_11_0_arm64.whl", hash = "sha256:e23717ce4186b265f69afa66e6f0069fe7efbf331546f5c313d00e123dc84583", size = 3562178, upload-time = "2026-03-31T22:39:51.295Z" }, + { url = "https://files.pythonhosted.org/packages/df/9a/a24b26dc8a65f0ecc0fe5be981a19e61e7ca963b85e062c083f3a9100529/hf_xet-1.4.3-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:fc360b70c815bf340ed56c7b8c63aacf11762a4b099b2fe2c9bd6d6068668c08", size = 4212320, upload-time = "2026-03-31T22:39:42.922Z" }, + { url = "https://files.pythonhosted.org/packages/53/60/46d493db155d2ee2801b71fb1b0fd67696359047fdd8caee2c914cc50c79/hf_xet-1.4.3-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:39f2d2e9654cd9b4319885733993807aab6de9dfbd34c42f0b78338d6617421f", size = 3991546, upload-time = "2026-03-31T22:39:41.335Z" }, + { url = "https://files.pythonhosted.org/packages/bc/f5/067363e1c96c6b17256910830d1b54099d06287e10f4ec6ec4e7e08371fc/hf_xet-1.4.3-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:49ad8a8cead2b56051aa84d7fce3e1335efe68df3cf6c058f22a65513885baac", size = 4193200, upload-time = "2026-03-31T22:40:01.936Z" }, + { url = "https://files.pythonhosted.org/packages/42/4b/53951592882d9c23080c7644542fda34a3813104e9e11fa1a7d82d419cb8/hf_xet-1.4.3-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:7716d62015477a70ea272d2d68cd7cad140f61c52ee452e133e139abfe2c17ba", size = 4429392, upload-time = "2026-03-31T22:40:03.492Z" }, + { url = "https://files.pythonhosted.org/packages/8a/21/75a6c175b4e79662ad8e62f46a40ce341d8d6b206b06b4320d07d55b188c/hf_xet-1.4.3-cp37-abi3-win_amd64.whl", hash = "sha256:6b591fcad34e272a5b02607485e4f2a1334aebf1bc6d16ce8eb1eb8978ac2021", size = 3677359, upload-time = "2026-03-31T22:40:13.619Z" }, + { url = "https://files.pythonhosted.org/packages/8a/7c/44314ecd0e89f8b2b51c9d9e5e7a60a9c1c82024ac471d415860557d3cd8/hf_xet-1.4.3-cp37-abi3-win_arm64.whl", hash = "sha256:7c2c7e20bcfcc946dc67187c203463f5e932e395845d098cc2a93f5b67ca0b47", size = 3533664, upload-time = "2026-03-31T22:40:12.152Z" }, ] [[package]] @@ -725,7 +725,7 @@ wheels = [ [[package]] name = "huggingface-hub" -version = "1.8.0" +version = "1.11.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "filelock" }, @@ -738,18 +738,18 @@ dependencies = [ { name = "typer" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/8e/2a/a847fd02261cd051da218baf99f90ee7c7040c109a01833db4f838f25256/huggingface_hub-1.8.0.tar.gz", hash = "sha256:c5627b2fd521e00caf8eff4ac965ba988ea75167fad7ee72e17f9b7183ec63f3", size = 735839, upload-time = "2026-03-25T16:01:28.152Z" } +sdist = { url = "https://files.pythonhosted.org/packages/dc/89/e7aa12d8a6b9259bed10671abb25ae6fa437c0f88a86ecbf59617bae7759/huggingface_hub-1.11.0.tar.gz", hash = "sha256:15fb3713c7f9cdff7b808a94fd91664f661ab142796bb48c9cd9493e8d166278", size = 761749, upload-time = "2026-04-16T13:07:39.73Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/a9/ae/8a3a16ea4d202cb641b51d2681bdd3d482c1c592d7570b3fa264730829ce/huggingface_hub-1.8.0-py3-none-any.whl", hash = "sha256:d3eb5047bd4e33c987429de6020d4810d38a5bef95b3b40df9b17346b7f353f2", size = 625208, upload-time = "2026-03-25T16:01:26.603Z" }, + { url = "https://files.pythonhosted.org/packages/37/02/4f3f8997d1ea7fe0146b343e5e14bd065fa87af790d07e5576d31b31cc18/huggingface_hub-1.11.0-py3-none-any.whl", hash = "sha256:42a6de0afbfeb5e022222d36398f029679db4eb4778801aafda32257ae9131ab", size = 645499, upload-time = "2026-04-16T13:07:37.716Z" }, ] [[package]] name = "identify" -version = "2.6.18" +version = "2.6.19" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/46/c4/7fb4db12296cdb11893d61c92048fe617ee853f8523b9b296ac03b43757e/identify-2.6.18.tar.gz", hash = "sha256:873ac56a5e3fd63e7438a7ecbc4d91aca692eb3fefa4534db2b7913f3fc352fd", size = 99580, upload-time = "2026-03-15T18:39:50.319Z" } +sdist = { url = "https://files.pythonhosted.org/packages/52/63/51723b5f116cc04b061cb6f5a561790abf249d25931d515cd375e063e0f4/identify-2.6.19.tar.gz", hash = "sha256:6be5020c38fcb07da56c53733538a3081ea5aa70d36a156f83044bfbf9173842", size = 99567, upload-time = "2026-04-17T18:39:50.265Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/46/33/92ef41c6fad0233e41d3d84ba8e8ad18d1780f1e5d99b3c683e6d7f98b63/identify-2.6.18-py2.py3-none-any.whl", hash = "sha256:8db9d3c8ea9079db92cafb0ebf97abdc09d52e97f4dcf773a2e694048b7cd737", size = 99394, upload-time = "2026-03-15T18:39:48.915Z" }, + { url = "https://files.pythonhosted.org/packages/94/84/d9273cd09688070a6523c4aee4663a8538721b2b755c4962aafae0011e72/identify-2.6.19-py2.py3-none-any.whl", hash = "sha256:20e6a87f786f768c092a721ad107fc9df0eb89347be9396cadf3f4abbd1fb78a", size = 99397, upload-time = "2026-04-17T18:39:49.221Z" }, ] [[package]] @@ -775,11 +775,11 @@ wheels = [ [[package]] name = "importlib-resources" -version = "6.5.2" +version = "7.1.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/cf/8c/f834fbf984f691b4f7ff60f50b514cc3de5cc08abfc3295564dd89c5e2e7/importlib_resources-6.5.2.tar.gz", hash = "sha256:185f87adef5bcc288449d98fb4fba07cea78bc036455dd44c5fc4a2fe78fed2c", size = 44693, upload-time = "2025-01-03T18:51:56.698Z" } +sdist = { url = "https://files.pythonhosted.org/packages/e4/06/b56dfa750b44e86157093bc8fca0ab81dccbf5260510de4eaf1cb69b5b99/importlib_resources-7.1.0.tar.gz", hash = "sha256:0722d4c6212489c530f2a145a34c0a7a3b4721bc96a15fada5930e2a0b760708", size = 44985, upload-time = "2026-04-12T16:36:09.232Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/a4/ed/1f1afb2e9e7f38a545d628f864d562a5ae64fe6f7a10e28ffb9b185b4e89/importlib_resources-6.5.2-py3-none-any.whl", hash = "sha256:789cfdc3ed28c78b67a06acb8126751ced69a3d5f79c095a98298cd8a760ccec", size = 37461, upload-time = "2025-01-03T18:51:54.306Z" }, + { url = "https://files.pythonhosted.org/packages/8a/db/55a262f3606bebcae07cc14095338471ad7c0bbcaa37707e6f0ee49725b7/importlib_resources-7.1.0-py3-none-any.whl", hash = "sha256:1bd7b48b4088eddb2cd16382150bb515af0bd2c70128194392725f82ad2c96a1", size = 37232, upload-time = "2026-04-12T16:36:08.219Z" }, ] [[package]] @@ -932,44 +932,46 @@ wheels = [ [[package]] name = "jiter" -version = "0.13.0" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/0d/5e/4ec91646aee381d01cdb9974e30882c9cd3b8c5d1079d6b5ff4af522439a/jiter-0.13.0.tar.gz", hash = "sha256:f2839f9c2c7e2dffc1bc5929a510e14ce0a946be9365fd1219e7ef342dae14f4", size = 164847, upload-time = "2026-02-02T12:37:56.441Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/71/29/499f8c9eaa8a16751b1c0e45e6f5f1761d180da873d417996cc7bddc8eef/jiter-0.13.0-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:ea026e70a9a28ebbdddcbcf0f1323128a8db66898a06eaad3a4e62d2f554d096", size = 311157, upload-time = "2026-02-02T12:35:37.758Z" }, - { url = "https://files.pythonhosted.org/packages/50/f6/566364c777d2ab450b92100bea11333c64c38d32caf8dc378b48e5b20c46/jiter-0.13.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:66aa3e663840152d18cc8ff1e4faad3dd181373491b9cfdc6004b92198d67911", size = 319729, upload-time = "2026-02-02T12:35:39.246Z" }, - { url = "https://files.pythonhosted.org/packages/73/dd/560f13ec5e4f116d8ad2658781646cca91b617ae3b8758d4a5076b278f70/jiter-0.13.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c3524798e70655ff19aec58c7d05adb1f074fecff62da857ea9be2b908b6d701", size = 354766, upload-time = "2026-02-02T12:35:40.662Z" }, - { url = "https://files.pythonhosted.org/packages/7c/0d/061faffcfe94608cbc28a0d42a77a74222bdf5055ccdbe5fd2292b94f510/jiter-0.13.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ec7e287d7fbd02cb6e22f9a00dd9c9cd504c40a61f2c61e7e1f9690a82726b4c", size = 362587, upload-time = "2026-02-02T12:35:42.025Z" }, - { url = "https://files.pythonhosted.org/packages/92/c9/c66a7864982fd38a9773ec6e932e0398d1262677b8c60faecd02ffb67bf3/jiter-0.13.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:47455245307e4debf2ce6c6e65a717550a0244231240dcf3b8f7d64e4c2f22f4", size = 487537, upload-time = "2026-02-02T12:35:43.459Z" }, - { url = "https://files.pythonhosted.org/packages/6c/86/84eb4352cd3668f16d1a88929b5888a3fe0418ea8c1dfc2ad4e7bf6e069a/jiter-0.13.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ee9da221dca6e0429c2704c1b3655fe7b025204a71d4d9b73390c759d776d165", size = 373717, upload-time = "2026-02-02T12:35:44.928Z" }, - { url = "https://files.pythonhosted.org/packages/6e/09/9fe4c159358176f82d4390407a03f506a8659ed13ca3ac93a843402acecf/jiter-0.13.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:24ab43126d5e05f3d53a36a8e11eb2f23304c6c1117844aaaf9a0aa5e40b5018", size = 362683, upload-time = "2026-02-02T12:35:46.636Z" }, - { url = "https://files.pythonhosted.org/packages/c9/5e/85f3ab9caca0c1d0897937d378b4a515cae9e119730563572361ea0c48ae/jiter-0.13.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:9da38b4fedde4fb528c740c2564628fbab737166a0e73d6d46cb4bb5463ff411", size = 392345, upload-time = "2026-02-02T12:35:48.088Z" }, - { url = "https://files.pythonhosted.org/packages/12/4c/05b8629ad546191939e6f0c2f17e29f542a398f4a52fb987bc70b6d1eb8b/jiter-0.13.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:0b34c519e17658ed88d5047999a93547f8889f3c1824120c26ad6be5f27b6cf5", size = 517775, upload-time = "2026-02-02T12:35:49.482Z" }, - { url = "https://files.pythonhosted.org/packages/4d/88/367ea2eb6bc582c7052e4baf5ddf57ebe5ab924a88e0e09830dfb585c02d/jiter-0.13.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:d2a6394e6af690d462310a86b53c47ad75ac8c21dc79f120714ea449979cb1d3", size = 551325, upload-time = "2026-02-02T12:35:51.104Z" }, - { url = "https://files.pythonhosted.org/packages/f3/12/fa377ffb94a2f28c41afaed093e0d70cfe512035d5ecb0cad0ae4792d35e/jiter-0.13.0-cp311-cp311-win32.whl", hash = "sha256:0f0c065695f616a27c920a56ad0d4fc46415ef8b806bf8fc1cacf25002bd24e1", size = 204709, upload-time = "2026-02-02T12:35:52.467Z" }, - { url = "https://files.pythonhosted.org/packages/cb/16/8e8203ce92f844dfcd3d9d6a5a7322c77077248dbb12da52d23193a839cd/jiter-0.13.0-cp311-cp311-win_amd64.whl", hash = "sha256:0733312953b909688ae3c2d58d043aa040f9f1a6a75693defed7bc2cc4bf2654", size = 204560, upload-time = "2026-02-02T12:35:53.925Z" }, - { url = "https://files.pythonhosted.org/packages/44/26/97cc40663deb17b9e13c3a5cf29251788c271b18ee4d262c8f94798b8336/jiter-0.13.0-cp311-cp311-win_arm64.whl", hash = "sha256:5d9b34ad56761b3bf0fbe8f7e55468704107608512350962d3317ffd7a4382d5", size = 189608, upload-time = "2026-02-02T12:35:55.304Z" }, - { url = "https://files.pythonhosted.org/packages/2e/30/7687e4f87086829955013ca12a9233523349767f69653ebc27036313def9/jiter-0.13.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:0a2bd69fc1d902e89925fc34d1da51b2128019423d7b339a45d9e99c894e0663", size = 307958, upload-time = "2026-02-02T12:35:57.165Z" }, - { url = "https://files.pythonhosted.org/packages/c3/27/e57f9a783246ed95481e6749cc5002a8a767a73177a83c63ea71f0528b90/jiter-0.13.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f917a04240ef31898182f76a332f508f2cc4b57d2b4d7ad2dbfebbfe167eb505", size = 318597, upload-time = "2026-02-02T12:35:58.591Z" }, - { url = "https://files.pythonhosted.org/packages/cf/52/e5719a60ac5d4d7c5995461a94ad5ef962a37c8bf5b088390e6fad59b2ff/jiter-0.13.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c1e2b199f446d3e82246b4fd9236d7cb502dc2222b18698ba0d986d2fecc6152", size = 348821, upload-time = "2026-02-02T12:36:00.093Z" }, - { url = "https://files.pythonhosted.org/packages/61/db/c1efc32b8ba4c740ab3fc2d037d8753f67685f475e26b9d6536a4322bcdd/jiter-0.13.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:04670992b576fa65bd056dbac0c39fe8bd67681c380cb2b48efa885711d9d726", size = 364163, upload-time = "2026-02-02T12:36:01.937Z" }, - { url = "https://files.pythonhosted.org/packages/55/8a/fb75556236047c8806995671a18e4a0ad646ed255276f51a20f32dceaeec/jiter-0.13.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5a1aff1fbdb803a376d4d22a8f63f8e7ccbce0b4890c26cc7af9e501ab339ef0", size = 483709, upload-time = "2026-02-02T12:36:03.41Z" }, - { url = "https://files.pythonhosted.org/packages/7e/16/43512e6ee863875693a8e6f6d532e19d650779d6ba9a81593ae40a9088ff/jiter-0.13.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3b3fb8c2053acaef8580809ac1d1f7481a0a0bdc012fd7f5d8b18fb696a5a089", size = 370480, upload-time = "2026-02-02T12:36:04.791Z" }, - { url = "https://files.pythonhosted.org/packages/f8/4c/09b93e30e984a187bc8aaa3510e1ec8dcbdcd71ca05d2f56aac0492453aa/jiter-0.13.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bdaba7d87e66f26a2c45d8cbadcbfc4bf7884182317907baf39cfe9775bb4d93", size = 360735, upload-time = "2026-02-02T12:36:06.994Z" }, - { url = "https://files.pythonhosted.org/packages/1a/1b/46c5e349019874ec5dfa508c14c37e29864ea108d376ae26d90bee238cd7/jiter-0.13.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:7b88d649135aca526da172e48083da915ec086b54e8e73a425ba50999468cc08", size = 391814, upload-time = "2026-02-02T12:36:08.368Z" }, - { url = "https://files.pythonhosted.org/packages/15/9e/26184760e85baee7162ad37b7912797d2077718476bf91517641c92b3639/jiter-0.13.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:e404ea551d35438013c64b4f357b0474c7abf9f781c06d44fcaf7a14c69ff9e2", size = 513990, upload-time = "2026-02-02T12:36:09.993Z" }, - { url = "https://files.pythonhosted.org/packages/e9/34/2c9355247d6debad57a0a15e76ab1566ab799388042743656e566b3b7de1/jiter-0.13.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:1f4748aad1b4a93c8bdd70f604d0f748cdc0e8744c5547798acfa52f10e79228", size = 548021, upload-time = "2026-02-02T12:36:11.376Z" }, - { url = "https://files.pythonhosted.org/packages/ac/4a/9f2c23255d04a834398b9c2e0e665382116911dc4d06b795710503cdad25/jiter-0.13.0-cp312-cp312-win32.whl", hash = "sha256:0bf670e3b1445fc4d31612199f1744f67f889ee1bbae703c4b54dc097e5dd394", size = 203024, upload-time = "2026-02-02T12:36:12.682Z" }, - { url = "https://files.pythonhosted.org/packages/09/ee/f0ae675a957ae5a8f160be3e87acea6b11dc7b89f6b7ab057e77b2d2b13a/jiter-0.13.0-cp312-cp312-win_amd64.whl", hash = "sha256:15db60e121e11fe186c0b15236bd5d18381b9ddacdcf4e659feb96fc6c969c92", size = 205424, upload-time = "2026-02-02T12:36:13.93Z" }, - { url = "https://files.pythonhosted.org/packages/1b/02/ae611edf913d3cbf02c97cdb90374af2082c48d7190d74c1111dde08bcdd/jiter-0.13.0-cp312-cp312-win_arm64.whl", hash = "sha256:41f92313d17989102f3cb5dd533a02787cdb99454d494344b0361355da52fcb9", size = 186818, upload-time = "2026-02-02T12:36:15.308Z" }, - { url = "https://files.pythonhosted.org/packages/79/b3/3c29819a27178d0e461a8571fb63c6ae38be6dc36b78b3ec2876bbd6a910/jiter-0.13.0-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:b1cbfa133241d0e6bdab48dcdc2604e8ba81512f6bbd68ec3e8e1357dd3c316c", size = 307016, upload-time = "2026-02-02T12:37:42.755Z" }, - { url = "https://files.pythonhosted.org/packages/eb/ae/60993e4b07b1ac5ebe46da7aa99fdbb802eb986c38d26e3883ac0125c4e0/jiter-0.13.0-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:db367d8be9fad6e8ebbac4a7578b7af562e506211036cba2c06c3b998603c3d2", size = 305024, upload-time = "2026-02-02T12:37:44.774Z" }, - { url = "https://files.pythonhosted.org/packages/77/fa/2227e590e9cf98803db2811f172b2d6460a21539ab73006f251c66f44b14/jiter-0.13.0-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:45f6f8efb2f3b0603092401dc2df79fa89ccbc027aaba4174d2d4133ed661434", size = 339337, upload-time = "2026-02-02T12:37:46.668Z" }, - { url = "https://files.pythonhosted.org/packages/2d/92/015173281f7eb96c0ef580c997da8ef50870d4f7f4c9e03c845a1d62ae04/jiter-0.13.0-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:597245258e6ad085d064780abfb23a284d418d3e61c57362d9449c6c7317ee2d", size = 346395, upload-time = "2026-02-02T12:37:48.09Z" }, - { url = "https://files.pythonhosted.org/packages/80/60/e50fa45dd7e2eae049f0ce964663849e897300433921198aef94b6ffa23a/jiter-0.13.0-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:3d744a6061afba08dd7ae375dcde870cffb14429b7477e10f67e9e6d68772a0a", size = 305169, upload-time = "2026-02-02T12:37:50.376Z" }, - { url = "https://files.pythonhosted.org/packages/d2/73/a009f41c5eed71c49bec53036c4b33555afcdee70682a18c6f66e396c039/jiter-0.13.0-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:ff732bd0a0e778f43d5009840f20b935e79087b4dc65bd36f1cd0f9b04b8ff7f", size = 303808, upload-time = "2026-02-02T12:37:52.092Z" }, - { url = "https://files.pythonhosted.org/packages/c4/10/528b439290763bff3d939268085d03382471b442f212dca4ff5f12802d43/jiter-0.13.0-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ab44b178f7981fcaea7e0a5df20e773c663d06ffda0198f1a524e91b2fde7e59", size = 337384, upload-time = "2026-02-02T12:37:53.582Z" }, - { url = "https://files.pythonhosted.org/packages/67/8a/a342b2f0251f3dac4ca17618265d93bf244a2a4d089126e81e4c1056ac50/jiter-0.13.0-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7bb00b6d26db67a05fe3e12c76edc75f32077fb51deed13822dc648fa373bc19", size = 343768, upload-time = "2026-02-02T12:37:55.055Z" }, +version = "0.14.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/6e/c1/0cddc6eb17d4c53a99840953f95dd3accdc5cfc7a337b0e9b26476276be9/jiter-0.14.0.tar.gz", hash = "sha256:e8a39e66dac7153cf3f964a12aad515afa8d74938ec5cc0018adcdae5367c79e", size = 165725, upload-time = "2026-04-10T14:28:42.01Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8a/1f/198ae537fccb7080a0ed655eb56abf64a92f79489dfbf79f40fa34225bcd/jiter-0.14.0-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:7e791e247b8044512e070bd1f3633dc08350d32776d2d6e7473309d0edf256a2", size = 316896, upload-time = "2026-04-10T14:26:01.986Z" }, + { url = "https://files.pythonhosted.org/packages/cf/34/da67cff3fce964a36d03c3e365fb0f8726ade2a6cfd4d3c70107e216ead6/jiter-0.14.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:71527ce13fd5a0c4e40ad37331f8c547177dbb2dd0a93e5278b6a5eecf748804", size = 321085, upload-time = "2026-04-10T14:26:03.364Z" }, + { url = "https://files.pythonhosted.org/packages/ed/36/4c72e67180d4e71a4f5dcf7886d0840e83c49ab11788172177a77570326e/jiter-0.14.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:02c4a7ab56f746014874f2c525584c0daca1dec37f66fd707ecef3b7e5c2228c", size = 347393, upload-time = "2026-04-10T14:26:05.314Z" }, + { url = "https://files.pythonhosted.org/packages/bc/db/9b39e09ceafa9878235c0fc29e3e3f9b12a4c6a98ea3085b998cadf3accc/jiter-0.14.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:376e9dafff914253bb9d46cdc5f7965607fbe7feb0a491c34e35f92b2770702e", size = 372937, upload-time = "2026-04-10T14:26:06.884Z" }, + { url = "https://files.pythonhosted.org/packages/b0/96/0dcba1d7a82c1b720774b48ef239376addbaf30df24c34742ac4a57b67b2/jiter-0.14.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:23ad2a7a9da1935575c820428dd8d2490ce4d23189691ce33da1fc0a58e14e1c", size = 463646, upload-time = "2026-04-10T14:26:08.345Z" }, + { url = "https://files.pythonhosted.org/packages/f1/e3/f61b71543e746e6b8b805e7755814fc242715c16f1dba58e1cbccb8032c2/jiter-0.14.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:54b3ddf5786bc7732d293bba3411ac637ecfa200a39983166d1df86a59a43c9f", size = 380225, upload-time = "2026-04-10T14:26:10.161Z" }, + { url = "https://files.pythonhosted.org/packages/ad/5e/0ddeb7096aca099114abe36c4921016e8d251e6f35f5890240b31f1f60ae/jiter-0.14.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5c001d5a646c2a50dc055dd526dad5d5245969e8234d2b1131d0451e81f3a373", size = 358682, upload-time = "2026-04-10T14:26:11.574Z" }, + { url = "https://files.pythonhosted.org/packages/e9/d1/fe0c46cd7fda9cad8f1ff9ad217dc61f1e4280b21052ec6dfe88c1446ef2/jiter-0.14.0-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:834bb5bdabca2e91592a03d373838a8d0a1b8bbde7077ae6913fd2fc51812d00", size = 359973, upload-time = "2026-04-10T14:26:13.316Z" }, + { url = "https://files.pythonhosted.org/packages/ac/21/f5317f91729b501019184771c80d60abd89907009e7bfa6c7e348c5bdd44/jiter-0.14.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:4e9178be60e229b1b2b0710f61b9e24d1f4f8556985a83ff4c4f95920eea7314", size = 397568, upload-time = "2026-04-10T14:26:15.212Z" }, + { url = "https://files.pythonhosted.org/packages/e9/05/79d8f33fb2bf168db0df5c9cd16fe440a8ada57e929d3677b22712c2568f/jiter-0.14.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:a7e4ccff04ec03614e62c613e976a3a5860dc9714ce8266f44328bdc8b1cab2c", size = 522535, upload-time = "2026-04-10T14:26:16.956Z" }, + { url = "https://files.pythonhosted.org/packages/5c/00/d1e3ff3d2a465e67f08507d74bafb2dcd29eba91dc939820e39e8dea38b8/jiter-0.14.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:69539d936fb5d55caf6ecd33e2e884de083ff0ea28579780d56c4403094bb8d9", size = 556709, upload-time = "2026-04-10T14:26:18.5Z" }, + { url = "https://files.pythonhosted.org/packages/60/5b/bbb2189f62ace8d95e869aa4c84c9946616f301e2d02895a6f20dcc3bba3/jiter-0.14.0-cp311-cp311-win32.whl", hash = "sha256:4927d09b3e572787cc5e0a5318601448e1ab9391bcef95677f5840c2d00eaa6d", size = 208660, upload-time = "2026-04-10T14:26:20.511Z" }, + { url = "https://files.pythonhosted.org/packages/b8/86/c500b53dcbf08575f5963e536ebd757a1f7c568272ba5d180b212c9a87fb/jiter-0.14.0-cp311-cp311-win_amd64.whl", hash = "sha256:42d6ed359ac49eb922fdd565f209c57340aa06d589c84c8413e42a0f9ae1b842", size = 204659, upload-time = "2026-04-10T14:26:22.152Z" }, + { url = "https://files.pythonhosted.org/packages/75/4a/a676249049d42cb29bef82233e4fe0524d414cbe3606c7a4b311193c2f77/jiter-0.14.0-cp311-cp311-win_arm64.whl", hash = "sha256:6dd689f5f4a5a33747b28686e051095beb214fe28cfda5e9fe58a295a788f593", size = 194772, upload-time = "2026-04-10T14:26:23.458Z" }, + { url = "https://files.pythonhosted.org/packages/5a/68/7390a418f10897da93b158f2d5a8bd0bcd73a0f9ec3bb36917085bb759ef/jiter-0.14.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:2fb2ce3a7bc331256dfb14cefc34832366bb28a9aca81deaf43bbf2a5659e607", size = 316295, upload-time = "2026-04-10T14:26:24.887Z" }, + { url = "https://files.pythonhosted.org/packages/60/a0/5854ac00ff63551c52c6c89534ec6aba4b93474e7924d64e860b1c94165b/jiter-0.14.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:5252a7ca23785cef5d02d4ece6077a1b556a410c591b379f82091c3001e14844", size = 315898, upload-time = "2026-04-10T14:26:26.601Z" }, + { url = "https://files.pythonhosted.org/packages/41/a1/4f44832650a16b18e8391f1bf1d6ca4909bc738351826bcc198bba4357f4/jiter-0.14.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c409578cbd77c338975670ada777add4efd53379667edf0aceea730cabede6fb", size = 343730, upload-time = "2026-04-10T14:26:28.326Z" }, + { url = "https://files.pythonhosted.org/packages/48/64/a329e9d469f86307203594b1707e11ae51c3348d03bfd514a5f997870012/jiter-0.14.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:7ede4331a1899d604463369c730dbb961ffdc5312bc7f16c41c2896415b1304a", size = 370102, upload-time = "2026-04-10T14:26:30.089Z" }, + { url = "https://files.pythonhosted.org/packages/94/c1/5e3dfc59635aa4d4c7bd20a820ac1d09b8ed851568356802cf1c08edb3cf/jiter-0.14.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:92cd8b6025981a041f5310430310b55b25ca593972c16407af8837d3d7d2ca01", size = 461335, upload-time = "2026-04-10T14:26:31.911Z" }, + { url = "https://files.pythonhosted.org/packages/e3/1b/dd157009dbc058f7b00108f545ccb72a2d56461395c4fc7b9cfdccb00af4/jiter-0.14.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:351bf6eda4e3a7ceb876377840c702e9a3e4ecc4624dbfb2d6463c67ae52637d", size = 378536, upload-time = "2026-04-10T14:26:33.595Z" }, + { url = "https://files.pythonhosted.org/packages/91/78/256013667b7c10b8834f8e6e54cd3e562d4c6e34227a1596addccc05e38c/jiter-0.14.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c1dcfbeb93d9ecd9ca128bbf8910120367777973fa193fb9a39c31237d8df165", size = 353859, upload-time = "2026-04-10T14:26:35.098Z" }, + { url = "https://files.pythonhosted.org/packages/de/d9/137d65ade9093a409fe80955ce60b12bb753722c986467aeda47faf450ad/jiter-0.14.0-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:ae039aaef8de3f8157ecc1fdd4d85043ac4f57538c245a0afaecb8321ec951c3", size = 357626, upload-time = "2026-04-10T14:26:36.685Z" }, + { url = "https://files.pythonhosted.org/packages/2e/48/76750835b87029342727c1a268bea8878ab988caf81ee4e7b880900eeb5a/jiter-0.14.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:7d9d51eb96c82a9652933bd769fe6de66877d6eb2b2440e281f2938c51b5643e", size = 393172, upload-time = "2026-04-10T14:26:38.097Z" }, + { url = "https://files.pythonhosted.org/packages/a6/60/456c4e81d5c8045279aefe60e9e483be08793828800a4e64add8fdde7f2a/jiter-0.14.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:d824ca4148b705970bf4e120924a212fdfca9859a73e42bd7889a63a4ea6bb98", size = 520300, upload-time = "2026-04-10T14:26:39.532Z" }, + { url = "https://files.pythonhosted.org/packages/a8/9f/2020e0984c235f678dced38fe4eec3058cf528e6af36ebf969b410305941/jiter-0.14.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:ff3a6465b3a0f54b1a430f45c3c0ba7d61ceb45cbc3e33f9e1a7f638d690baf3", size = 553059, upload-time = "2026-04-10T14:26:40.991Z" }, + { url = "https://files.pythonhosted.org/packages/ef/32/e2d298e1a22a4bbe6062136d1c7192db7dba003a6975e51d9a9eecabc4c2/jiter-0.14.0-cp312-cp312-win32.whl", hash = "sha256:5dec7c0a3e98d2a3f8a2e67382d0d7c3ac60c69103a4b271da889b4e8bb1e129", size = 206030, upload-time = "2026-04-10T14:26:42.517Z" }, + { url = "https://files.pythonhosted.org/packages/36/ac/96369141b3d8a4a8e4590e983085efe1c436f35c0cda940dd76d942e3e40/jiter-0.14.0-cp312-cp312-win_amd64.whl", hash = "sha256:fc7e37b4b8bc7e80a63ad6cfa5fc11fab27dbfea4cc4ae644b1ab3f273dc348f", size = 201603, upload-time = "2026-04-10T14:26:44.328Z" }, + { url = "https://files.pythonhosted.org/packages/01/c3/75d847f264647017d7e3052bbcc8b1e24b95fa139c320c5f5066fa7a0bdd/jiter-0.14.0-cp312-cp312-win_arm64.whl", hash = "sha256:ee4a72f12847ef29b072aee9ad5474041ab2924106bdca9fcf5d7d965853e057", size = 191525, upload-time = "2026-04-10T14:26:46Z" }, + { url = "https://files.pythonhosted.org/packages/32/a1/ef34ca2cab2962598591636a1804b93645821201cc0095d4a93a9a329c9d/jiter-0.14.0-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:a25ffa2dbbdf8721855612f6dca15c108224b12d0c4024d0ac3d7902132b4211", size = 311366, upload-time = "2026-04-10T14:28:27.943Z" }, + { url = "https://files.pythonhosted.org/packages/60/bb/520576a532a6b8a6f42747afed289c8448c879a34d7802fe2c832d4fd38f/jiter-0.14.0-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:0ac9cbaa86c10996b92bd12c91659b60f939f8e28fcfa6bc11a0e90a774ce95b", size = 309873, upload-time = "2026-04-10T14:28:29.688Z" }, + { url = "https://files.pythonhosted.org/packages/b2/7c/c16db114ea1f2f532f198aa8dc39585026af45af362c69a0492f31bc4821/jiter-0.14.0-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:844e73b6c56b505e9e169234ea3bdea2ea43f769f847f47ac559ba1d2361ebea", size = 344816, upload-time = "2026-04-10T14:28:31.348Z" }, + { url = "https://files.pythonhosted.org/packages/99/8f/15e7741ff19e9bcd4d753f7ff22f988fd54592f134ca13701c13ea8c20e0/jiter-0.14.0-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e52c076f187405fc21523c746c04399c9af8ece566077ed147b2126f2bcba577", size = 351445, upload-time = "2026-04-10T14:28:33.093Z" }, + { url = "https://files.pythonhosted.org/packages/21/42/9042c3f3019de4adcb8c16591c325ec7255beea9fcd33a42a43f3b0b1000/jiter-0.14.0-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:fbd9e482663ca9d005d051330e4d2d8150bb208a209409c10f7e7dfdf7c49da9", size = 308810, upload-time = "2026-04-10T14:28:34.673Z" }, + { url = "https://files.pythonhosted.org/packages/60/cf/a7e19b308bd86bb04776803b1f01a5f9a287a4c55205f4708827ee487fbf/jiter-0.14.0-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:33a20d838b91ef376b3a56896d5b04e725c7df5bc4864cc6569cf046a8d73b6d", size = 308443, upload-time = "2026-04-10T14:28:36.658Z" }, + { url = "https://files.pythonhosted.org/packages/ca/44/e26ede3f0caeff93f222559cb0cc4ca68579f07d009d7b6010c5b586f9b1/jiter-0.14.0-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:432c4db5255d86a259efde91e55cb4c8d18c0521d844c9e2e7efcce3899fb016", size = 343039, upload-time = "2026-04-10T14:28:38.356Z" }, + { url = "https://files.pythonhosted.org/packages/da/e9/1f9ada30cef7b05e74bb06f52127e7a724976c225f46adb65c37b1dadfb6/jiter-0.14.0-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:67f00d94b281174144d6532a04b66a12cb866cbdc47c3af3bfe2973677f9861a", size = 349613, upload-time = "2026-04-10T14:28:40.066Z" }, ] [[package]] @@ -1126,14 +1128,14 @@ wheels = [ [[package]] name = "jupyter-lsp" -version = "2.3.0" +version = "2.3.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "jupyter-server" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/eb/5a/9066c9f8e94ee517133cd98dba393459a16cd48bba71a82f16a65415206c/jupyter_lsp-2.3.0.tar.gz", hash = "sha256:458aa59339dc868fb784d73364f17dbce8836e906cd75fd471a325cba02e0245", size = 54823, upload-time = "2025-08-27T17:47:34.671Z" } +sdist = { url = "https://files.pythonhosted.org/packages/36/ff/1e4a61f5170a9a1d978f3ac3872449de6c01fc71eaf89657824c878b1549/jupyter_lsp-2.3.1.tar.gz", hash = "sha256:fdf8a4aa7d85813976d6e29e95e6a2c8f752701f926f2715305249a3829805a6", size = 55677, upload-time = "2026-04-02T08:10:06.749Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/1a/60/1f6cee0c46263de1173894f0fafcb3475ded276c472c14d25e0280c18d6d/jupyter_lsp-2.3.0-py3-none-any.whl", hash = "sha256:e914a3cb2addf48b1c7710914771aaf1819d46b2e5a79b0f917b5478ec93f34f", size = 76687, upload-time = "2025-08-27T17:47:33.15Z" }, + { url = "https://files.pythonhosted.org/packages/23/e8/9d61dcbd1dce8ef418f06befd4ac084b4720429c26b0b1222bc218685eff/jupyter_lsp-2.3.1-py3-none-any.whl", hash = "sha256:71b954d834e85ff3096400554f2eefaf7fe37053036f9a782b0f7c5e42dadb81", size = 77513, upload-time = "2026-04-02T08:10:01.753Z" }, ] [[package]] @@ -1270,36 +1272,36 @@ wheels = [ [[package]] name = "librt" -version = "0.8.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/56/9c/b4b0c54d84da4a94b37bd44151e46d5e583c9534c7e02250b961b1b6d8a8/librt-0.8.1.tar.gz", hash = "sha256:be46a14693955b3bd96014ccbdb8339ee8c9346fbe11c1b78901b55125f14c73", size = 177471, upload-time = "2026-02-17T16:13:06.101Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/1d/01/0e748af5e4fee180cf7cd12bd12b0513ad23b045dccb2a83191bde82d168/librt-0.8.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:681dc2451d6d846794a828c16c22dc452d924e9f700a485b7ecb887a30aad1fd", size = 65315, upload-time = "2026-02-17T16:11:25.152Z" }, - { url = "https://files.pythonhosted.org/packages/9d/4d/7184806efda571887c798d573ca4134c80ac8642dcdd32f12c31b939c595/librt-0.8.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:a3b4350b13cc0e6f5bec8fa7caf29a8fb8cdc051a3bae45cfbfd7ce64f009965", size = 68021, upload-time = "2026-02-17T16:11:26.129Z" }, - { url = "https://files.pythonhosted.org/packages/ae/88/c3c52d2a5d5101f28d3dc89298444626e7874aa904eed498464c2af17627/librt-0.8.1-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:ac1e7817fd0ed3d14fd7c5df91daed84c48e4c2a11ee99c0547f9f62fdae13da", size = 194500, upload-time = "2026-02-17T16:11:27.177Z" }, - { url = "https://files.pythonhosted.org/packages/d6/5d/6fb0a25b6a8906e85b2c3b87bee1d6ed31510be7605b06772f9374ca5cb3/librt-0.8.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:747328be0c5b7075cde86a0e09d7a9196029800ba75a1689332348e998fb85c0", size = 205622, upload-time = "2026-02-17T16:11:28.242Z" }, - { url = "https://files.pythonhosted.org/packages/b2/a6/8006ae81227105476a45691f5831499e4d936b1c049b0c1feb17c11b02d1/librt-0.8.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f0af2bd2bc204fa27f3d6711d0f360e6b8c684a035206257a81673ab924aa11e", size = 218304, upload-time = "2026-02-17T16:11:29.344Z" }, - { url = "https://files.pythonhosted.org/packages/ee/19/60e07886ad16670aae57ef44dada41912c90906a6fe9f2b9abac21374748/librt-0.8.1-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d480de377f5b687b6b1bc0c0407426da556e2a757633cc7e4d2e1a057aa688f3", size = 211493, upload-time = "2026-02-17T16:11:30.445Z" }, - { url = "https://files.pythonhosted.org/packages/9c/cf/f666c89d0e861d05600438213feeb818c7514d3315bae3648b1fc145d2b6/librt-0.8.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d0ee06b5b5291f609ddb37b9750985b27bc567791bc87c76a569b3feed8481ac", size = 219129, upload-time = "2026-02-17T16:11:32.021Z" }, - { url = "https://files.pythonhosted.org/packages/8f/ef/f1bea01e40b4a879364c031476c82a0dc69ce068daad67ab96302fed2d45/librt-0.8.1-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:9e2c6f77b9ad48ce5603b83b7da9ee3e36b3ab425353f695cba13200c5d96596", size = 213113, upload-time = "2026-02-17T16:11:33.192Z" }, - { url = "https://files.pythonhosted.org/packages/9b/80/cdab544370cc6bc1b72ea369525f547a59e6938ef6863a11ab3cd24759af/librt-0.8.1-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:439352ba9373f11cb8e1933da194dcc6206daf779ff8df0ed69c5e39113e6a99", size = 212269, upload-time = "2026-02-17T16:11:34.373Z" }, - { url = "https://files.pythonhosted.org/packages/9d/9c/48d6ed8dac595654f15eceab2035131c136d1ae9a1e3548e777bb6dbb95d/librt-0.8.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:82210adabbc331dbb65d7868b105185464ef13f56f7f76688565ad79f648b0fe", size = 234673, upload-time = "2026-02-17T16:11:36.063Z" }, - { url = "https://files.pythonhosted.org/packages/16/01/35b68b1db517f27a01be4467593292eb5315def8900afad29fabf56304ba/librt-0.8.1-cp311-cp311-win32.whl", hash = "sha256:52c224e14614b750c0a6d97368e16804a98c684657c7518752c356834fff83bb", size = 54597, upload-time = "2026-02-17T16:11:37.544Z" }, - { url = "https://files.pythonhosted.org/packages/71/02/796fe8f02822235966693f257bf2c79f40e11337337a657a8cfebba5febc/librt-0.8.1-cp311-cp311-win_amd64.whl", hash = "sha256:c00e5c884f528c9932d278d5c9cbbea38a6b81eb62c02e06ae53751a83a4d52b", size = 61733, upload-time = "2026-02-17T16:11:38.691Z" }, - { url = "https://files.pythonhosted.org/packages/28/ad/232e13d61f879a42a4e7117d65e4984bb28371a34bb6fb9ca54ec2c8f54e/librt-0.8.1-cp311-cp311-win_arm64.whl", hash = "sha256:f7cdf7f26c2286ffb02e46d7bac56c94655540b26347673bea15fa52a6af17e9", size = 52273, upload-time = "2026-02-17T16:11:40.308Z" }, - { url = "https://files.pythonhosted.org/packages/95/21/d39b0a87ac52fc98f621fb6f8060efb017a767ebbbac2f99fbcbc9ddc0d7/librt-0.8.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:a28f2612ab566b17f3698b0da021ff9960610301607c9a5e8eaca62f5e1c350a", size = 66516, upload-time = "2026-02-17T16:11:41.604Z" }, - { url = "https://files.pythonhosted.org/packages/69/f1/46375e71441c43e8ae335905e069f1c54febee63a146278bcee8782c84fd/librt-0.8.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:60a78b694c9aee2a0f1aaeaa7d101cf713e92e8423a941d2897f4fa37908dab9", size = 68634, upload-time = "2026-02-17T16:11:43.268Z" }, - { url = "https://files.pythonhosted.org/packages/0a/33/c510de7f93bf1fa19e13423a606d8189a02624a800710f6e6a0a0f0784b3/librt-0.8.1-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:758509ea3f1eba2a57558e7e98f4659d0ea7670bff49673b0dde18a3c7e6c0eb", size = 198941, upload-time = "2026-02-17T16:11:44.28Z" }, - { url = "https://files.pythonhosted.org/packages/dd/36/e725903416409a533d92398e88ce665476f275081d0d7d42f9c4951999e5/librt-0.8.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:039b9f2c506bd0ab0f8725aa5ba339c6f0cd19d3b514b50d134789809c24285d", size = 209991, upload-time = "2026-02-17T16:11:45.462Z" }, - { url = "https://files.pythonhosted.org/packages/30/7a/8d908a152e1875c9f8eac96c97a480df425e657cdb47854b9efaa4998889/librt-0.8.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5bb54f1205a3a6ab41a6fd71dfcdcbd278670d3a90ca502a30d9da583105b6f7", size = 224476, upload-time = "2026-02-17T16:11:46.542Z" }, - { url = "https://files.pythonhosted.org/packages/a8/b8/a22c34f2c485b8903a06f3fe3315341fe6876ef3599792344669db98fcff/librt-0.8.1-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:05bd41cdee35b0c59c259f870f6da532a2c5ca57db95b5f23689fcb5c9e42440", size = 217518, upload-time = "2026-02-17T16:11:47.746Z" }, - { url = "https://files.pythonhosted.org/packages/79/6f/5c6fea00357e4f82ba44f81dbfb027921f1ab10e320d4a64e1c408d035d9/librt-0.8.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:adfab487facf03f0d0857b8710cf82d0704a309d8ffc33b03d9302b4c64e91a9", size = 225116, upload-time = "2026-02-17T16:11:49.298Z" }, - { url = "https://files.pythonhosted.org/packages/f2/a0/95ced4e7b1267fe1e2720a111685bcddf0e781f7e9e0ce59d751c44dcfe5/librt-0.8.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:153188fe98a72f206042be10a2c6026139852805215ed9539186312d50a8e972", size = 217751, upload-time = "2026-02-17T16:11:50.49Z" }, - { url = "https://files.pythonhosted.org/packages/93/c2/0517281cb4d4101c27ab59472924e67f55e375bc46bedae94ac6dc6e1902/librt-0.8.1-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:dd3c41254ee98604b08bd5b3af5bf0a89740d4ee0711de95b65166bf44091921", size = 218378, upload-time = "2026-02-17T16:11:51.783Z" }, - { url = "https://files.pythonhosted.org/packages/43/e8/37b3ac108e8976888e559a7b227d0ceac03c384cfd3e7a1c2ee248dbae79/librt-0.8.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e0d138c7ae532908cbb342162b2611dbd4d90c941cd25ab82084aaf71d2c0bd0", size = 241199, upload-time = "2026-02-17T16:11:53.561Z" }, - { url = "https://files.pythonhosted.org/packages/4b/5b/35812d041c53967fedf551a39399271bbe4257e681236a2cf1a69c8e7fa1/librt-0.8.1-cp312-cp312-win32.whl", hash = "sha256:43353b943613c5d9c49a25aaffdba46f888ec354e71e3529a00cca3f04d66a7a", size = 54917, upload-time = "2026-02-17T16:11:54.758Z" }, - { url = "https://files.pythonhosted.org/packages/de/d1/fa5d5331b862b9775aaf2a100f5ef86854e5d4407f71bddf102f4421e034/librt-0.8.1-cp312-cp312-win_amd64.whl", hash = "sha256:ff8baf1f8d3f4b6b7257fcb75a501f2a5499d0dda57645baa09d4d0d34b19444", size = 62017, upload-time = "2026-02-17T16:11:55.748Z" }, - { url = "https://files.pythonhosted.org/packages/c7/7c/c614252f9acda59b01a66e2ddfd243ed1c7e1deab0293332dfbccf862808/librt-0.8.1-cp312-cp312-win_arm64.whl", hash = "sha256:0f2ae3725904f7377e11cc37722d5d401e8b3d5851fb9273d7f4fe04f6b3d37d", size = 52441, upload-time = "2026-02-17T16:11:56.801Z" }, +version = "0.9.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/eb/6b/3d5c13fb3e3c4f43206c8f9dfed13778c2ed4f000bacaa0b7ce3c402a265/librt-0.9.0.tar.gz", hash = "sha256:a0951822531e7aee6e0dfb556b30d5ee36bbe234faf60c20a16c01be3530869d", size = 184368, upload-time = "2026-04-09T16:06:26.173Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e2/1e/2ec7afcebcf3efea593d13aee18bbcfdd3a243043d848ebf385055e9f636/librt-0.9.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:90904fac73c478f4b83f4ed96c99c8208b75e6f9a8a1910548f69a00f1eaa671", size = 67155, upload-time = "2026-04-09T16:04:42.933Z" }, + { url = "https://files.pythonhosted.org/packages/18/77/72b85afd4435268338ad4ec6231b3da8c77363f212a0227c1ff3b45e4d35/librt-0.9.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:789fff71757facc0738e8d89e3b84e4f0251c1c975e85e81b152cdaca927cc2d", size = 69916, upload-time = "2026-04-09T16:04:44.042Z" }, + { url = "https://files.pythonhosted.org/packages/27/fb/948ea0204fbe2e78add6d46b48330e58d39897e425560674aee302dca81c/librt-0.9.0-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:1bf465d1e5b0a27713862441f6467b5ab76385f4ecf8f1f3a44f8aa3c695b4b6", size = 199635, upload-time = "2026-04-09T16:04:45.5Z" }, + { url = "https://files.pythonhosted.org/packages/ac/cd/894a29e251b296a27957856804cfd21e93c194aa131de8bb8032021be07e/librt-0.9.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f819e0c6413e259a17a7c0d49f97f405abadd3c2a316a3b46c6440b7dbbedbb1", size = 211051, upload-time = "2026-04-09T16:04:47.016Z" }, + { url = "https://files.pythonhosted.org/packages/18/8f/dcaed0bc084a35f3721ff2d081158db569d2c57ea07d35623ddaca5cfc8e/librt-0.9.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e0785c2fb4a81e1aece366aa3e2e039f4a4d7d21aaaded5227d7f3c703427882", size = 224031, upload-time = "2026-04-09T16:04:48.207Z" }, + { url = "https://files.pythonhosted.org/packages/03/44/88f6c1ed1132cd418601cc041fbd92fed28b3a09f39de81978e0822d13ff/librt-0.9.0-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:80b25c7b570a86c03b5da69e665809deb39265476e8e21d96a9328f9762f9990", size = 218069, upload-time = "2026-04-09T16:04:50.025Z" }, + { url = "https://files.pythonhosted.org/packages/a3/90/7d02e981c2db12188d82b4410ff3e35bfdb844b26aecd02233626f46af2b/librt-0.9.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d4d16b608a1c43d7e33142099a75cd93af482dadce0bf82421e91cad077157f4", size = 224857, upload-time = "2026-04-09T16:04:51.684Z" }, + { url = "https://files.pythonhosted.org/packages/ef/c3/c77e706b7215ca32e928d47535cf13dbc3d25f096f84ddf8fbc06693e229/librt-0.9.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:194fc1a32e1e21fe809d38b5faea66cc65eaa00217c8901fbdb99866938adbdb", size = 219865, upload-time = "2026-04-09T16:04:52.949Z" }, + { url = "https://files.pythonhosted.org/packages/52/d1/32b0c1a0eb8461c70c11656c46a29f760b7c7edf3c36d6f102470c17170f/librt-0.9.0-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:8c6bc1384d9738781cfd41d09ad7f6e8af13cfea2c75ece6bd6d2566cdea2076", size = 218451, upload-time = "2026-04-09T16:04:54.174Z" }, + { url = "https://files.pythonhosted.org/packages/74/d1/adfd0f9c44761b1d49b1bec66173389834c33ee2bd3c7fd2e2367f1942d4/librt-0.9.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:15cb151e52a044f06e54ac7f7b47adbfc89b5c8e2b63e1175a9d587c43e8942a", size = 241300, upload-time = "2026-04-09T16:04:55.452Z" }, + { url = "https://files.pythonhosted.org/packages/09/b0/9074b64407712f0003c27f5b1d7655d1438979155f049720e8a1abd9b1a1/librt-0.9.0-cp311-cp311-win32.whl", hash = "sha256:f100bfe2acf8a3689af9d0cc660d89f17286c9c795f9f18f7b62dd1a6b247ae6", size = 55668, upload-time = "2026-04-09T16:04:56.689Z" }, + { url = "https://files.pythonhosted.org/packages/24/19/40b77b77ce80b9389fb03971431b09b6b913911c38d412059e0b3e2a9ef2/librt-0.9.0-cp311-cp311-win_amd64.whl", hash = "sha256:0b73e4266307e51c95e09c0750b7ec383c561d2e97d58e473f6f6a209952fbb8", size = 62976, upload-time = "2026-04-09T16:04:57.733Z" }, + { url = "https://files.pythonhosted.org/packages/70/9d/9fa7a64041e29035cb8c575af5f0e3840be1b97b4c4d9061e0713f171849/librt-0.9.0-cp311-cp311-win_arm64.whl", hash = "sha256:bc5518873822d2faa8ebdd2c1a4d7c8ef47b01a058495ab7924cb65bdbf5fc9a", size = 53502, upload-time = "2026-04-09T16:04:58.806Z" }, + { url = "https://files.pythonhosted.org/packages/bf/90/89ddba8e1c20b0922783cd93ed8e64f34dc05ab59c38a9c7e313632e20ff/librt-0.9.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:9b3e3bc363f71bda1639a4ee593cb78f7fbfeacc73411ec0d4c92f00730010a4", size = 68332, upload-time = "2026-04-09T16:05:00.09Z" }, + { url = "https://files.pythonhosted.org/packages/a8/40/7aa4da1fb08bdeeb540cb07bfc8207cb32c5c41642f2594dbd0098a0662d/librt-0.9.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0a09c2f5869649101738653a9b7ab70cf045a1105ac66cbb8f4055e61df78f2d", size = 70581, upload-time = "2026-04-09T16:05:01.213Z" }, + { url = "https://files.pythonhosted.org/packages/48/ac/73a2187e1031041e93b7e3a25aae37aa6f13b838c550f7e0f06f66766212/librt-0.9.0-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:5ca8e133d799c948db2ab1afc081c333a825b5540475164726dcbf73537e5c2f", size = 203984, upload-time = "2026-04-09T16:05:02.542Z" }, + { url = "https://files.pythonhosted.org/packages/5e/3d/23460d571e9cbddb405b017681df04c142fb1b04cbfce77c54b08e28b108/librt-0.9.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:603138ee838ee1583f1b960b62d5d0007845c5c423feb68e44648b1359014e27", size = 215762, upload-time = "2026-04-09T16:05:04.127Z" }, + { url = "https://files.pythonhosted.org/packages/de/1e/42dc7f8ab63e65b20640d058e63e97fd3e482c1edbda3570d813b4d0b927/librt-0.9.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f4003f70c56a5addd6aa0897f200dd59afd3bf7bcd5b3cce46dd21f925743bc2", size = 230288, upload-time = "2026-04-09T16:05:05.883Z" }, + { url = "https://files.pythonhosted.org/packages/dc/08/ca812b6d8259ad9ece703397f8ad5c03af5b5fedfce64279693d3ce4087c/librt-0.9.0-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:78042f6facfd98ecb25e9829c7e37cce23363d9d7c83bc5f72702c5059eb082b", size = 224103, upload-time = "2026-04-09T16:05:07.148Z" }, + { url = "https://files.pythonhosted.org/packages/b6/3f/620490fb2fa66ffd44e7f900254bc110ebec8dac6c1b7514d64662570e6f/librt-0.9.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:a361c9434a64d70a7dbb771d1de302c0cc9f13c0bffe1cf7e642152814b35265", size = 232122, upload-time = "2026-04-09T16:05:08.386Z" }, + { url = "https://files.pythonhosted.org/packages/e9/83/12864700a1b6a8be458cf5d05db209b0d8e94ae281e7ec261dbe616597b4/librt-0.9.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:dd2c7e082b0b92e1baa4da28163a808672485617bc855cc22a2fd06978fa9084", size = 225045, upload-time = "2026-04-09T16:05:09.707Z" }, + { url = "https://files.pythonhosted.org/packages/fd/1b/845d339c29dc7dbc87a2e992a1ba8d28d25d0e0372f9a0a2ecebde298186/librt-0.9.0-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:7e6274fd33fc5b2a14d41c9119629d3ff395849d8bcbc80cf637d9e8d2034da8", size = 227372, upload-time = "2026-04-09T16:05:10.942Z" }, + { url = "https://files.pythonhosted.org/packages/8d/fe/277985610269d926a64c606f761d58d3db67b956dbbf40024921e95e7fcb/librt-0.9.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5093043afb226ecfa1400120d1ebd4442b4f99977783e4f4f7248879009b227f", size = 248224, upload-time = "2026-04-09T16:05:12.254Z" }, + { url = "https://files.pythonhosted.org/packages/92/1b/ee486d244b8de6b8b5dbaefabe6bfdd4a72e08f6353edf7d16d27114da8d/librt-0.9.0-cp312-cp312-win32.whl", hash = "sha256:9edcc35d1cae9fd5320171b1a838c7da8a5c968af31e82ecc3dff30b4be0957f", size = 55986, upload-time = "2026-04-09T16:05:13.529Z" }, + { url = "https://files.pythonhosted.org/packages/89/7a/ba1737012308c17dc6d5516143b5dce9a2c7ba3474afd54e11f44a4d1ef3/librt-0.9.0-cp312-cp312-win_amd64.whl", hash = "sha256:3cc2917258e131ae5f958a4d872e07555b51cb7466a43433218061c74ef33745", size = 63260, upload-time = "2026-04-09T16:05:14.68Z" }, + { url = "https://files.pythonhosted.org/packages/36/e4/01752c113da15127f18f7bf11142f5640038f062407a611c059d0036c6aa/librt-0.9.0-cp312-cp312-win_arm64.whl", hash = "sha256:90e6d5420fc8a300518d4d2288154ff45005e920425c22cbbfe8330f3f754bd9", size = 53694, upload-time = "2026-04-09T16:05:16.095Z" }, ] [[package]] @@ -1425,7 +1427,7 @@ wheels = [ [[package]] name = "mypy" -version = "1.19.1" +version = "1.20.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "librt", marker = "platform_python_implementation != 'PyPy'" }, @@ -1433,21 +1435,23 @@ dependencies = [ { name = "pathspec" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/f5/db/4efed9504bc01309ab9c2da7e352cc223569f05478012b5d9ece38fd44d2/mypy-1.19.1.tar.gz", hash = "sha256:19d88bb05303fe63f71dd2c6270daca27cb9401c4ca8255fe50d1d920e0eb9ba", size = 3582404, upload-time = "2025-12-15T05:03:48.42Z" } +sdist = { url = "https://files.pythonhosted.org/packages/0b/3d/5b373635b3146264eb7a68d09e5ca11c305bbb058dfffbb47c47daf4f632/mypy-1.20.1.tar.gz", hash = "sha256:6fc3f4ecd52de81648fed1945498bf42fa2993ddfad67c9056df36ae5757f804", size = 3815892, upload-time = "2026-04-13T02:46:51.474Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/ef/47/6b3ebabd5474d9cdc170d1342fbf9dddc1b0ec13ec90bf9004ee6f391c31/mypy-1.19.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:d8dfc6ab58ca7dda47d9237349157500468e404b17213d44fc1cb77bce532288", size = 13028539, upload-time = "2025-12-15T05:03:44.129Z" }, - { url = "https://files.pythonhosted.org/packages/5c/a6/ac7c7a88a3c9c54334f53a941b765e6ec6c4ebd65d3fe8cdcfbe0d0fd7db/mypy-1.19.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:e3f276d8493c3c97930e354b2595a44a21348b320d859fb4a2b9f66da9ed27ab", size = 12083163, upload-time = "2025-12-15T05:03:37.679Z" }, - { url = "https://files.pythonhosted.org/packages/67/af/3afa9cf880aa4a2c803798ac24f1d11ef72a0c8079689fac5cfd815e2830/mypy-1.19.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2abb24cf3f17864770d18d673c85235ba52456b36a06b6afc1e07c1fdcd3d0e6", size = 12687629, upload-time = "2025-12-15T05:02:31.526Z" }, - { url = "https://files.pythonhosted.org/packages/2d/46/20f8a7114a56484ab268b0ab372461cb3a8f7deed31ea96b83a4e4cfcfca/mypy-1.19.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a009ffa5a621762d0c926a078c2d639104becab69e79538a494bcccb62cc0331", size = 13436933, upload-time = "2025-12-15T05:03:15.606Z" }, - { url = "https://files.pythonhosted.org/packages/5b/f8/33b291ea85050a21f15da910002460f1f445f8007adb29230f0adea279cb/mypy-1.19.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f7cee03c9a2e2ee26ec07479f38ea9c884e301d42c6d43a19d20fb014e3ba925", size = 13661754, upload-time = "2025-12-15T05:02:26.731Z" }, - { url = "https://files.pythonhosted.org/packages/fd/a3/47cbd4e85bec4335a9cd80cf67dbc02be21b5d4c9c23ad6b95d6c5196bac/mypy-1.19.1-cp311-cp311-win_amd64.whl", hash = "sha256:4b84a7a18f41e167f7995200a1d07a4a6810e89d29859df936f1c3923d263042", size = 10055772, upload-time = "2025-12-15T05:03:26.179Z" }, - { url = "https://files.pythonhosted.org/packages/06/8a/19bfae96f6615aa8a0604915512e0289b1fad33d5909bf7244f02935d33a/mypy-1.19.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:a8174a03289288c1f6c46d55cef02379b478bfbc8e358e02047487cad44c6ca1", size = 13206053, upload-time = "2025-12-15T05:03:46.622Z" }, - { url = "https://files.pythonhosted.org/packages/a5/34/3e63879ab041602154ba2a9f99817bb0c85c4df19a23a1443c8986e4d565/mypy-1.19.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ffcebe56eb09ff0c0885e750036a095e23793ba6c2e894e7e63f6d89ad51f22e", size = 12219134, upload-time = "2025-12-15T05:03:24.367Z" }, - { url = "https://files.pythonhosted.org/packages/89/cc/2db6f0e95366b630364e09845672dbee0cbf0bbe753a204b29a944967cd9/mypy-1.19.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b64d987153888790bcdb03a6473d321820597ab8dd9243b27a92153c4fa50fd2", size = 12731616, upload-time = "2025-12-15T05:02:44.725Z" }, - { url = "https://files.pythonhosted.org/packages/00/be/dd56c1fd4807bc1eba1cf18b2a850d0de7bacb55e158755eb79f77c41f8e/mypy-1.19.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c35d298c2c4bba75feb2195655dfea8124d855dfd7343bf8b8c055421eaf0cf8", size = 13620847, upload-time = "2025-12-15T05:03:39.633Z" }, - { url = "https://files.pythonhosted.org/packages/6d/42/332951aae42b79329f743bf1da088cd75d8d4d9acc18fbcbd84f26c1af4e/mypy-1.19.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:34c81968774648ab5ac09c29a375fdede03ba253f8f8287847bd480782f73a6a", size = 13834976, upload-time = "2025-12-15T05:03:08.786Z" }, - { url = "https://files.pythonhosted.org/packages/6f/63/e7493e5f90e1e085c562bb06e2eb32cae27c5057b9653348d38b47daaecc/mypy-1.19.1-cp312-cp312-win_amd64.whl", hash = "sha256:b10e7c2cd7870ba4ad9b2d8a6102eb5ffc1f16ca35e3de6bfa390c1113029d13", size = 10118104, upload-time = "2025-12-15T05:03:10.834Z" }, - { url = "https://files.pythonhosted.org/packages/8d/f4/4ce9a05ce5ded1de3ec1c1d96cf9f9504a04e54ce0ed55cfa38619a32b8d/mypy-1.19.1-py3-none-any.whl", hash = "sha256:f1235f5ea01b7db5468d53ece6aaddf1ad0b88d9e7462b86ef96fe04995d7247", size = 2471239, upload-time = "2025-12-15T05:03:07.248Z" }, + { url = "https://files.pythonhosted.org/packages/82/0d/555ab7453cc4a4a8643b7f21c842b1a84c36b15392061ae7b052ee119320/mypy-1.20.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:c01eb9bac2c6a962d00f9d23421cd2913840e65bba365167d057bd0b4171a92e", size = 14336012, upload-time = "2026-04-13T02:45:39.935Z" }, + { url = "https://files.pythonhosted.org/packages/57/26/85a28893f7db8a16ebb41d1e9dfcb4475844d06a88480b6639e32a74d6ef/mypy-1.20.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:55d12ddbd8a9cac5b276878bd534fa39fff5bf543dc6ae18f25d30c8d7d27fca", size = 13224636, upload-time = "2026-04-13T02:45:49.659Z" }, + { url = "https://files.pythonhosted.org/packages/93/41/bd4cd3c2caeb6c448b669222b8cfcbdee4a03b89431527b56fca9e56b6f3/mypy-1.20.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c0aa322c1468b6cdfc927a44ce130f79bb44bcd34eb4a009eb9f96571fd80955", size = 13663471, upload-time = "2026-04-13T02:46:20.276Z" }, + { url = "https://files.pythonhosted.org/packages/3e/56/7ee8c471e10402d64b6517ae10434541baca053cffd81090e4097d5609d4/mypy-1.20.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3f8bc95899cf676b6e2285779a08a998cc3a7b26f1026752df9d2741df3c79e8", size = 14532344, upload-time = "2026-04-13T02:46:44.205Z" }, + { url = "https://files.pythonhosted.org/packages/b5/95/b37d1fa859a433f6156742e12f62b0bb75af658544fb6dada9363918743a/mypy-1.20.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:47c2b90191a870a04041e910277494b0d92f0711be9e524d45c074fe60c00b65", size = 14776670, upload-time = "2026-04-13T02:45:52.481Z" }, + { url = "https://files.pythonhosted.org/packages/03/77/b302e4cb0b80d2bdf6bf4fce5864bb4cbfa461f7099cea544eaf2457df78/mypy-1.20.1-cp311-cp311-win_amd64.whl", hash = "sha256:9857dc8d2ec1a392ffbda518075beb00ac58859979c79f9e6bdcb7277082c2f2", size = 10816524, upload-time = "2026-04-13T02:45:37.711Z" }, + { url = "https://files.pythonhosted.org/packages/7f/21/d969d7a68eb964993ebcc6170d5ecaf0cf65830c58ac3344562e16dc42a9/mypy-1.20.1-cp311-cp311-win_arm64.whl", hash = "sha256:09d8df92bb25b6065ab91b178da843dda67b33eb819321679a6e98a907ce0e10", size = 9750419, upload-time = "2026-04-13T02:45:08.542Z" }, + { url = "https://files.pythonhosted.org/packages/69/1b/75a7c825a02781ca10bc2f2f12fba2af5202f6d6005aad8d2d1f264d8d78/mypy-1.20.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:36ee2b9c6599c230fea89bbd79f401f9f9f8e9fcf0c777827789b19b7da90f51", size = 14494077, upload-time = "2026-04-13T02:45:55.085Z" }, + { url = "https://files.pythonhosted.org/packages/b0/54/5e5a569ea5c2b4d48b729fb32aa936eeb4246e4fc3e6f5b3d36a2dfbefb9/mypy-1.20.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fba3fb0968a7b48806b0c90f38d39296f10766885a94c83bd21399de1e14eb28", size = 13319495, upload-time = "2026-04-13T02:45:29.674Z" }, + { url = "https://files.pythonhosted.org/packages/6f/a4/a1945b19f33e91721b59deee3abb484f2fa5922adc33bb166daf5325d76d/mypy-1.20.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ef1415a637cd3627d6304dfbeddbadd21079dafc2a8a753c477ce4fc0c2af54f", size = 13696948, upload-time = "2026-04-13T02:46:15.006Z" }, + { url = "https://files.pythonhosted.org/packages/b2/c6/75e969781c2359b2f9c15b061f28ec6d67c8b61865ceda176e85c8e7f2de/mypy-1.20.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ef3461b1ad5cd446e540016e90b5984657edda39f982f4cc45ca317b628f5a37", size = 14706744, upload-time = "2026-04-13T02:46:00.482Z" }, + { url = "https://files.pythonhosted.org/packages/a8/6e/b221b1de981fc4262fe3e0bf9ec272d292dfe42394a689c2d49765c144c4/mypy-1.20.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:542dd63c9e1339b6092eb25bd515f3a32a1453aee8c9521d2ddb17dacd840237", size = 14949035, upload-time = "2026-04-13T02:45:06.021Z" }, + { url = "https://files.pythonhosted.org/packages/ca/4b/298ba2de0aafc0da3ff2288da06884aae7ba6489bc247c933f87847c41b3/mypy-1.20.1-cp312-cp312-win_amd64.whl", hash = "sha256:1d55c7cd8ca22e31f93af2a01160a9e95465b5878de23dba7e48116052f20a8d", size = 10883216, upload-time = "2026-04-13T02:45:47.232Z" }, + { url = "https://files.pythonhosted.org/packages/c7/f9/5e25b8f0b8cb92f080bfed9c21d3279b2a0b6a601cdca369a039ba84789d/mypy-1.20.1-cp312-cp312-win_arm64.whl", hash = "sha256:f5b84a79070586e0d353ee07b719d9d0a4aa7c8ee90c0ea97747e98cbe193019", size = 9814299, upload-time = "2026-04-13T02:45:21.934Z" }, + { url = "https://files.pythonhosted.org/packages/d8/28/926bd972388e65a39ee98e188ccf67e81beb3aacfd5d6b310051772d974b/mypy-1.20.1-py3-none-any.whl", hash = "sha256:1aae28507f253fe82d883790d1c0a0d35798a810117c88184097fe8881052f06", size = 2636553, upload-time = "2026-04-13T02:46:30.45Z" }, ] [[package]] @@ -1476,7 +1480,7 @@ wheels = [ [[package]] name = "nbconvert" -version = "7.17.0" +version = "7.17.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "beautifulsoup4" }, @@ -1494,9 +1498,9 @@ dependencies = [ { name = "pygments" }, { name = "traitlets" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/38/47/81f886b699450d0569f7bc551df2b1673d18df7ff25cc0c21ca36ed8a5ff/nbconvert-7.17.0.tar.gz", hash = "sha256:1b2696f1b5be12309f6c7d707c24af604b87dfaf6d950794c7b07acab96dda78", size = 862855, upload-time = "2026-01-29T16:37:48.478Z" } +sdist = { url = "https://files.pythonhosted.org/packages/01/b1/708e53fe2e429c103c6e6e159106bcf0357ac41aa4c28772bd8402339051/nbconvert-7.17.1.tar.gz", hash = "sha256:34d0d0a7e73ce3cbab6c5aae8f4f468797280b01fd8bd2ca746da8569eddd7d2", size = 865311, upload-time = "2026-04-08T00:44:14.914Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/0d/4b/8d5f796a792f8a25f6925a96032f098789f448571eb92011df1ae59e8ea8/nbconvert-7.17.0-py3-none-any.whl", hash = "sha256:4f99a63b337b9a23504347afdab24a11faa7d86b405e5c8f9881cd313336d518", size = 261510, upload-time = "2026-01-29T16:37:46.322Z" }, + { url = "https://files.pythonhosted.org/packages/67/f8/bb0a9d5f46819c821dc1f004aa2cc29b1d91453297dbf5ff20470f00f193/nbconvert-7.17.1-py3-none-any.whl", hash = "sha256:aa85c087b435e7bf1ffd03319f658e285f2b89eccab33bc1ba7025495ab3e7c8", size = 261927, upload-time = "2026-04-08T00:44:12.845Z" }, ] [[package]] @@ -1790,32 +1794,32 @@ wheels = [ [[package]] name = "opentelemetry-api" -version = "1.40.0" +version = "1.41.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "importlib-metadata" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/2c/1d/4049a9e8698361cc1a1aa03a6c59e4fa4c71e0c0f94a30f988a6876a2ae6/opentelemetry_api-1.40.0.tar.gz", hash = "sha256:159be641c0b04d11e9ecd576906462773eb97ae1b657730f0ecf64d32071569f", size = 70851, upload-time = "2026-03-04T14:17:21.555Z" } +sdist = { url = "https://files.pythonhosted.org/packages/47/8e/3778a7e87801d994869a9396b9fc2a289e5f9be91ff54a27d41eace494b0/opentelemetry_api-1.41.0.tar.gz", hash = "sha256:9421d911326ec12dee8bc933f7839090cad7a3f13fcfb0f9e82f8174dc003c09", size = 71416, upload-time = "2026-04-09T14:38:34.544Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/5f/bf/93795954016c522008da367da292adceed71cca6ee1717e1d64c83089099/opentelemetry_api-1.40.0-py3-none-any.whl", hash = "sha256:82dd69331ae74b06f6a874704be0cfaa49a1650e1537d4a813b86ecef7d0ecf9", size = 68676, upload-time = "2026-03-04T14:17:01.24Z" }, + { url = "https://files.pythonhosted.org/packages/58/ee/99ab786653b3bda9c37ade7e24a7b607a1b1f696063172768417539d876d/opentelemetry_api-1.41.0-py3-none-any.whl", hash = "sha256:0e77c806e6a89c9e4f8d372034622f3e1418a11bdbe1c80a50b3d3397ad0fa4f", size = 69007, upload-time = "2026-04-09T14:38:11.833Z" }, ] [[package]] name = "opentelemetry-exporter-otlp-proto-common" -version = "1.40.0" +version = "1.41.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-proto" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/51/bc/1559d46557fe6eca0b46c88d4c2676285f1f3be2e8d06bb5d15fbffc814a/opentelemetry_exporter_otlp_proto_common-1.40.0.tar.gz", hash = "sha256:1cbee86a4064790b362a86601ee7934f368b81cd4cc2f2e163902a6e7818a0fa", size = 20416, upload-time = "2026-03-04T14:17:23.801Z" } +sdist = { url = "https://files.pythonhosted.org/packages/8c/28/e8eca94966fe9a1465f6094dc5ddc5398473682180279c94020bc23b4906/opentelemetry_exporter_otlp_proto_common-1.41.0.tar.gz", hash = "sha256:966bbce537e9edb166154779a7c4f8ab6b8654a03a28024aeaf1a3eacb07d6ee", size = 20411, upload-time = "2026-04-09T14:38:36.572Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/8b/ca/8f122055c97a932311a3f640273f084e738008933503d0c2563cd5d591fc/opentelemetry_exporter_otlp_proto_common-1.40.0-py3-none-any.whl", hash = "sha256:7081ff453835a82417bf38dccf122c827c3cbc94f2079b03bba02a3165f25149", size = 18369, upload-time = "2026-03-04T14:17:04.796Z" }, + { url = "https://files.pythonhosted.org/packages/26/c4/78b9bf2d9c1d5e494f44932988d9d91c51a66b9a7b48adf99b62f7c65318/opentelemetry_exporter_otlp_proto_common-1.41.0-py3-none-any.whl", hash = "sha256:7a99177bf61f85f4f9ed2072f54d676364719c066f6d11f515acc6c745c7acf0", size = 18366, upload-time = "2026-04-09T14:38:15.135Z" }, ] [[package]] name = "opentelemetry-exporter-otlp-proto-grpc" -version = "1.40.0" +version = "1.41.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "googleapis-common-protos" }, @@ -1826,86 +1830,86 @@ dependencies = [ { name = "opentelemetry-sdk" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/8f/7f/b9e60435cfcc7590fa87436edad6822240dddbc184643a2a005301cc31f4/opentelemetry_exporter_otlp_proto_grpc-1.40.0.tar.gz", hash = "sha256:bd4015183e40b635b3dab8da528b27161ba83bf4ef545776b196f0fb4ec47740", size = 25759, upload-time = "2026-03-04T14:17:24.4Z" } +sdist = { url = "https://files.pythonhosted.org/packages/42/46/d75a3f8c91915f2e58f61d0a2e4ada63891e7c7a37a20ff7949ba184a6b2/opentelemetry_exporter_otlp_proto_grpc-1.41.0.tar.gz", hash = "sha256:f704201251c6f65772b11bddea1c948000554459101bdbb0116e0a01b70592f6", size = 25754, upload-time = "2026-04-09T14:38:37.423Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/96/6f/7ee0980afcbdcd2d40362da16f7f9796bd083bf7f0b8e038abfbc0300f5d/opentelemetry_exporter_otlp_proto_grpc-1.40.0-py3-none-any.whl", hash = "sha256:2aa0ca53483fe0cf6405087a7491472b70335bc5c7944378a0a8e72e86995c52", size = 20304, upload-time = "2026-03-04T14:17:05.942Z" }, + { url = "https://files.pythonhosted.org/packages/81/f6/b09e2e0c9f0b5750cebc6eaf31527b910821453cef40a5a0fe93550422b2/opentelemetry_exporter_otlp_proto_grpc-1.41.0-py3-none-any.whl", hash = "sha256:3a1a86bd24806ccf136ec9737dbfa4c09b069f9130ff66b0acb014f9c5255fd1", size = 20299, upload-time = "2026-04-09T14:38:17.01Z" }, ] [[package]] name = "opentelemetry-proto" -version = "1.40.0" +version = "1.41.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "protobuf" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/4c/77/dd38991db037fdfce45849491cb61de5ab000f49824a00230afb112a4392/opentelemetry_proto-1.40.0.tar.gz", hash = "sha256:03f639ca129ba513f5819810f5b1f42bcb371391405d99c168fe6937c62febcd", size = 45667, upload-time = "2026-03-04T14:17:31.194Z" } +sdist = { url = "https://files.pythonhosted.org/packages/e0/d9/08e3dc6156878713e8c811682bc76151f5fe1a3cb7f3abda3966fd56e71e/opentelemetry_proto-1.41.0.tar.gz", hash = "sha256:95d2e576f9fb1800473a3e4cfcca054295d06bdb869fda4dc9f4f779dc68f7b6", size = 45669, upload-time = "2026-04-09T14:38:45.978Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/b9/b2/189b2577dde745b15625b3214302605b1353436219d42b7912e77fa8dc24/opentelemetry_proto-1.40.0-py3-none-any.whl", hash = "sha256:266c4385d88923a23d63e353e9761af0f47a6ed0d486979777fe4de59dc9b25f", size = 72073, upload-time = "2026-03-04T14:17:16.673Z" }, + { url = "https://files.pythonhosted.org/packages/49/8c/65ef7a9383a363864772022e822b5d5c6988e6f9dabeebb9278f5b86ebc3/opentelemetry_proto-1.41.0-py3-none-any.whl", hash = "sha256:b970ab537309f9eed296be482c3e7cca05d8aca8165346e929f658dbe153b247", size = 72074, upload-time = "2026-04-09T14:38:29.38Z" }, ] [[package]] name = "opentelemetry-sdk" -version = "1.40.0" +version = "1.41.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, { name = "opentelemetry-semantic-conventions" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/58/fd/3c3125b20ba18ce2155ba9ea74acb0ae5d25f8cd39cfd37455601b7955cc/opentelemetry_sdk-1.40.0.tar.gz", hash = "sha256:18e9f5ec20d859d268c7cb3c5198c8d105d073714db3de50b593b8c1345a48f2", size = 184252, upload-time = "2026-03-04T14:17:31.87Z" } +sdist = { url = "https://files.pythonhosted.org/packages/f8/0e/a586df1186f9f56b5a0879d52653effc40357b8e88fc50fe300038c3c08b/opentelemetry_sdk-1.41.0.tar.gz", hash = "sha256:7bddf3961131b318fc2d158947971a8e37e38b1cd23470cfb72b624e7cc108bd", size = 230181, upload-time = "2026-04-09T14:38:47.225Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/2c/c5/6a852903d8bfac758c6dc6e9a68b015d3c33f2f1be5e9591e0f4b69c7e0a/opentelemetry_sdk-1.40.0-py3-none-any.whl", hash = "sha256:787d2154a71f4b3d81f20524a8ce061b7db667d24e46753f32a7bc48f1c1f3f1", size = 141951, upload-time = "2026-03-04T14:17:17.961Z" }, + { url = "https://files.pythonhosted.org/packages/2c/13/a7825118208cb32e6a4edcd0a99f925cbef81e77b3b0aedfd9125583c543/opentelemetry_sdk-1.41.0-py3-none-any.whl", hash = "sha256:a596f5687964a3e0d7f8edfdcf5b79cbca9c93c7025ebf5fb00f398a9443b0bd", size = 180214, upload-time = "2026-04-09T14:38:30.657Z" }, ] [[package]] name = "opentelemetry-semantic-conventions" -version = "0.61b0" +version = "0.62b0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/6d/c0/4ae7973f3c2cfd2b6e321f1675626f0dab0a97027cc7a297474c9c8f3d04/opentelemetry_semantic_conventions-0.61b0.tar.gz", hash = "sha256:072f65473c5d7c6dc0355b27d6c9d1a679d63b6d4b4b16a9773062cb7e31192a", size = 145755, upload-time = "2026-03-04T14:17:32.664Z" } +sdist = { url = "https://files.pythonhosted.org/packages/a3/b0/c14f723e86c049b7bf8ff431160d982519b97a7be2857ed2247377397a24/opentelemetry_semantic_conventions-0.62b0.tar.gz", hash = "sha256:cbfb3c8fc259575cf68a6e1b94083cc35adc4a6b06e8cf431efa0d62606c0097", size = 145753, upload-time = "2026-04-09T14:38:48.274Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/b2/37/cc6a55e448deaa9b27377d087da8615a3416d8ad523d5960b78dbeadd02a/opentelemetry_semantic_conventions-0.61b0-py3-none-any.whl", hash = "sha256:fa530a96be229795f8cef353739b618148b0fe2b4b3f005e60e262926c4d38e2", size = 231621, upload-time = "2026-03-04T14:17:19.33Z" }, + { url = "https://files.pythonhosted.org/packages/58/6c/5e86fa1759a525ef91c2d8b79d668574760ff3f900d114297765eb8786cb/opentelemetry_semantic_conventions-0.62b0-py3-none-any.whl", hash = "sha256:0ddac1ce59eaf1a827d9987ab60d9315fb27aea23304144242d1fcad9e16b489", size = 231619, upload-time = "2026-04-09T14:38:32.394Z" }, ] [[package]] name = "orjson" -version = "3.11.7" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/53/45/b268004f745ede84e5798b48ee12b05129d19235d0e15267aa57dcdb400b/orjson-3.11.7.tar.gz", hash = "sha256:9b1a67243945819ce55d24a30b59d6a168e86220452d2c96f4d1f093e71c0c49", size = 6144992, upload-time = "2026-02-02T15:38:49.29Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/37/02/da6cb01fc6087048d7f61522c327edf4250f1683a58a839fdcc435746dd5/orjson-3.11.7-cp311-cp311-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:9487abc2c2086e7c8eb9a211d2ce8855bae0e92586279d0d27b341d5ad76c85c", size = 228664, upload-time = "2026-02-02T15:37:25.542Z" }, - { url = "https://files.pythonhosted.org/packages/c1/c2/5885e7a5881dba9a9af51bc564e8967225a642b3e03d089289a35054e749/orjson-3.11.7-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:79cacb0b52f6004caf92405a7e1f11e6e2de8bdf9019e4f76b44ba045125cd6b", size = 125344, upload-time = "2026-02-02T15:37:26.92Z" }, - { url = "https://files.pythonhosted.org/packages/a4/1d/4e7688de0a92d1caf600dfd5fb70b4c5bfff51dfa61ac555072ef2d0d32a/orjson-3.11.7-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c2e85fe4698b6a56d5e2ebf7ae87544d668eb6bde1ad1226c13f44663f20ec9e", size = 128404, upload-time = "2026-02-02T15:37:28.108Z" }, - { url = "https://files.pythonhosted.org/packages/2f/b2/ec04b74ae03a125db7bd69cffd014b227b7f341e3261bf75b5eb88a1aa92/orjson-3.11.7-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b8d14b71c0b12963fe8a62aac87119f1afdf4cb88a400f61ca5ae581449efcb5", size = 123677, upload-time = "2026-02-02T15:37:30.287Z" }, - { url = "https://files.pythonhosted.org/packages/4c/69/f95bdf960605f08f827f6e3291fe243d8aa9c5c9ff017a8d7232209184c3/orjson-3.11.7-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:91c81ef070c8f3220054115e1ef468b1c9ce8497b4e526cb9f68ab4dc0a7ac62", size = 128950, upload-time = "2026-02-02T15:37:31.595Z" }, - { url = "https://files.pythonhosted.org/packages/a4/1b/de59c57bae1d148ef298852abd31909ac3089cff370dfd4cd84cc99cbc42/orjson-3.11.7-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:411ebaf34d735e25e358a6d9e7978954a9c9d58cfb47bc6683cdc3964cd2f910", size = 141756, upload-time = "2026-02-02T15:37:32.985Z" }, - { url = "https://files.pythonhosted.org/packages/ee/9e/9decc59f4499f695f65c650f6cfa6cd4c37a3fbe8fa235a0a3614cb54386/orjson-3.11.7-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a16bcd08ab0bcdfc7e8801d9c4a9cc17e58418e4d48ddc6ded4e9e4b1a94062b", size = 130812, upload-time = "2026-02-02T15:37:34.204Z" }, - { url = "https://files.pythonhosted.org/packages/28/e6/59f932bcabd1eac44e334fe8e3281a92eacfcb450586e1f4bde0423728d8/orjson-3.11.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9c0b51672e466fd7e56230ffbae7f1639e18d0ce023351fb75da21b71bc2c960", size = 133444, upload-time = "2026-02-02T15:37:35.446Z" }, - { url = "https://files.pythonhosted.org/packages/f1/36/b0f05c0eaa7ca30bc965e37e6a2956b0d67adb87a9872942d3568da846ae/orjson-3.11.7-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:136dcd6a2e796dfd9ffca9fc027d778567b0b7c9968d092842d3c323cef88aa8", size = 138609, upload-time = "2026-02-02T15:37:36.657Z" }, - { url = "https://files.pythonhosted.org/packages/b8/03/58ec7d302b8d86944c60c7b4b82975d5161fcce4c9bc8c6cb1d6741b6115/orjson-3.11.7-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:7ba61079379b0ae29e117db13bda5f28d939766e410d321ec1624afc6a0b0504", size = 408918, upload-time = "2026-02-02T15:37:38.076Z" }, - { url = "https://files.pythonhosted.org/packages/06/3a/868d65ef9a8b99be723bd510de491349618abd9f62c826cf206d962db295/orjson-3.11.7-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:0527a4510c300e3b406591b0ba69b5dc50031895b0a93743526a3fc45f59d26e", size = 143998, upload-time = "2026-02-02T15:37:39.706Z" }, - { url = "https://files.pythonhosted.org/packages/5b/c7/1e18e1c83afe3349f4f6dc9e14910f0ae5f82eac756d1412ea4018938535/orjson-3.11.7-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:a709e881723c9b18acddcfb8ba357322491ad553e277cf467e1e7e20e2d90561", size = 134802, upload-time = "2026-02-02T15:37:41.002Z" }, - { url = "https://files.pythonhosted.org/packages/d4/0b/ccb7ee1a65b37e8eeb8b267dc953561d72370e85185e459616d4345bab34/orjson-3.11.7-cp311-cp311-win32.whl", hash = "sha256:c43b8b5bab288b6b90dac410cca7e986a4fa747a2e8f94615aea407da706980d", size = 127828, upload-time = "2026-02-02T15:37:42.241Z" }, - { url = "https://files.pythonhosted.org/packages/af/9e/55c776dffda3f381e0f07d010a4f5f3902bf48eaba1bb7684d301acd4924/orjson-3.11.7-cp311-cp311-win_amd64.whl", hash = "sha256:6543001328aa857187f905308a028935864aefe9968af3848401b6fe80dbb471", size = 124941, upload-time = "2026-02-02T15:37:43.444Z" }, - { url = "https://files.pythonhosted.org/packages/aa/8e/424a620fa7d263b880162505fb107ef5e0afaa765b5b06a88312ac291560/orjson-3.11.7-cp311-cp311-win_arm64.whl", hash = "sha256:1ee5cc7160a821dfe14f130bc8e63e7611051f964b463d9e2a3a573204446a4d", size = 126245, upload-time = "2026-02-02T15:37:45.18Z" }, - { url = "https://files.pythonhosted.org/packages/80/bf/76f4f1665f6983385938f0e2a5d7efa12a58171b8456c252f3bae8a4cf75/orjson-3.11.7-cp312-cp312-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:bd03ea7606833655048dab1a00734a2875e3e86c276e1d772b2a02556f0d895f", size = 228545, upload-time = "2026-02-02T15:37:46.376Z" }, - { url = "https://files.pythonhosted.org/packages/79/53/6c72c002cb13b5a978a068add59b25a8bdf2800ac1c9c8ecdb26d6d97064/orjson-3.11.7-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:89e440ebc74ce8ab5c7bc4ce6757b4a6b1041becb127df818f6997b5c71aa60b", size = 125224, upload-time = "2026-02-02T15:37:47.697Z" }, - { url = "https://files.pythonhosted.org/packages/2c/83/10e48852865e5dd151bdfe652c06f7da484578ed02c5fca938e3632cb0b8/orjson-3.11.7-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5ede977b5fe5ac91b1dffc0a517ca4542d2ec8a6a4ff7b2652d94f640796342a", size = 128154, upload-time = "2026-02-02T15:37:48.954Z" }, - { url = "https://files.pythonhosted.org/packages/6e/52/a66e22a2b9abaa374b4a081d410edab6d1e30024707b87eab7c734afe28d/orjson-3.11.7-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b7b1dae39230a393df353827c855a5f176271c23434cfd2db74e0e424e693e10", size = 123548, upload-time = "2026-02-02T15:37:50.187Z" }, - { url = "https://files.pythonhosted.org/packages/de/38/605d371417021359f4910c496f764c48ceb8997605f8c25bf1dfe58c0ebe/orjson-3.11.7-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ed46f17096e28fb28d2975834836a639af7278aa87c84f68ab08fbe5b8bd75fa", size = 129000, upload-time = "2026-02-02T15:37:51.426Z" }, - { url = "https://files.pythonhosted.org/packages/44/98/af32e842b0ffd2335c89714d48ca4e3917b42f5d6ee5537832e069a4b3ac/orjson-3.11.7-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3726be79e36e526e3d9c1aceaadbfb4a04ee80a72ab47b3f3c17fefb9812e7b8", size = 141686, upload-time = "2026-02-02T15:37:52.607Z" }, - { url = "https://files.pythonhosted.org/packages/96/0b/fc793858dfa54be6feee940c1463370ece34b3c39c1ca0aa3845f5ba9892/orjson-3.11.7-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:0724e265bc548af1dedebd9cb3d24b4e1c1e685a343be43e87ba922a5c5fff2f", size = 130812, upload-time = "2026-02-02T15:37:53.944Z" }, - { url = "https://files.pythonhosted.org/packages/dc/91/98a52415059db3f374757d0b7f0f16e3b5cd5976c90d1c2b56acaea039e6/orjson-3.11.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e7745312efa9e11c17fbd3cb3097262d079da26930ae9ae7ba28fb738367cbad", size = 133440, upload-time = "2026-02-02T15:37:55.615Z" }, - { url = "https://files.pythonhosted.org/packages/dc/b6/cb540117bda61791f46381f8c26c8f93e802892830a6055748d3bb1925ab/orjson-3.11.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:f904c24bdeabd4298f7a977ef14ca2a022ca921ed670b92ecd16ab6f3d01f867", size = 138386, upload-time = "2026-02-02T15:37:56.814Z" }, - { url = "https://files.pythonhosted.org/packages/63/1a/50a3201c334a7f17c231eee5f841342190723794e3b06293f26e7cf87d31/orjson-3.11.7-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:b9fc4d0f81f394689e0814617aadc4f2ea0e8025f38c226cbf22d3b5ddbf025d", size = 408853, upload-time = "2026-02-02T15:37:58.291Z" }, - { url = "https://files.pythonhosted.org/packages/87/cd/8de1c67d0be44fdc22701e5989c0d015a2adf391498ad42c4dc589cd3013/orjson-3.11.7-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:849e38203e5be40b776ed2718e587faf204d184fc9a008ae441f9442320c0cab", size = 144130, upload-time = "2026-02-02T15:38:00.163Z" }, - { url = "https://files.pythonhosted.org/packages/0f/fe/d605d700c35dd55f51710d159fc54516a280923cd1b7e47508982fbb387d/orjson-3.11.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:4682d1db3bcebd2b64757e0ddf9e87ae5f00d29d16c5cdf3a62f561d08cc3dd2", size = 134818, upload-time = "2026-02-02T15:38:01.507Z" }, - { url = "https://files.pythonhosted.org/packages/e4/e4/15ecc67edb3ddb3e2f46ae04475f2d294e8b60c1825fbe28a428b93b3fbd/orjson-3.11.7-cp312-cp312-win32.whl", hash = "sha256:f4f7c956b5215d949a1f65334cf9d7612dde38f20a95f2315deef167def91a6f", size = 127923, upload-time = "2026-02-02T15:38:02.75Z" }, - { url = "https://files.pythonhosted.org/packages/34/70/2e0855361f76198a3965273048c8e50a9695d88cd75811a5b46444895845/orjson-3.11.7-cp312-cp312-win_amd64.whl", hash = "sha256:bf742e149121dc5648ba0a08ea0871e87b660467ef168a3a5e53bc1fbd64bb74", size = 125007, upload-time = "2026-02-02T15:38:04.032Z" }, - { url = "https://files.pythonhosted.org/packages/68/40/c2051bd19fc467610fed469dc29e43ac65891571138f476834ca192bc290/orjson-3.11.7-cp312-cp312-win_arm64.whl", hash = "sha256:26c3b9132f783b7d7903bf1efb095fed8d4a3a85ec0d334ee8beff3d7a4749d5", size = 126089, upload-time = "2026-02-02T15:38:05.297Z" }, +version = "3.11.8" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/9d/1b/2024d06792d0779f9dbc51531b61c24f76c75b9f4ce05e6f3377a1814cea/orjson-3.11.8.tar.gz", hash = "sha256:96163d9cdc5a202703e9ad1b9ae757d5f0ca62f4fa0cc93d1f27b0e180cc404e", size = 5603832, upload-time = "2026-03-31T16:16:27.878Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/67/41/5aa7fa3b0f4dc6b47dcafc3cea909299c37e40e9972feabc8b6a74e2730d/orjson-3.11.8-cp311-cp311-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:003646067cc48b7fcab2ae0c562491c9b5d2cbd43f1e5f16d98fd118c5522d34", size = 229229, upload-time = "2026-03-31T16:14:50.424Z" }, + { url = "https://files.pythonhosted.org/packages/0a/d7/57e7f2458e0a2c41694f39fc830030a13053a84f837a5b73423dca1f0938/orjson-3.11.8-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:ed193ce51d77a3830cad399a529cd4ef029968761f43ddc549e1bc62b40d88f8", size = 128871, upload-time = "2026-03-31T16:14:51.888Z" }, + { url = "https://files.pythonhosted.org/packages/53/4a/e0fdb9430983e6c46e0299559275025075568aad5d21dd606faee3703924/orjson-3.11.8-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f30491bc4f862aa15744b9738517454f1e46e56c972a2be87d70d727d5b2a8f8", size = 132104, upload-time = "2026-03-31T16:14:53.142Z" }, + { url = "https://files.pythonhosted.org/packages/08/4a/2025a60ff3f5c8522060cda46612d9b1efa653de66ed2908591d8d82f22d/orjson-3.11.8-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:6eda5b8b6be91d3f26efb7dc6e5e68ee805bc5617f65a328587b35255f138bf4", size = 130483, upload-time = "2026-03-31T16:14:54.605Z" }, + { url = "https://files.pythonhosted.org/packages/2d/3c/b9cde05bdc7b2385c66014e0620627da638d3d04e4954416ab48c31196c5/orjson-3.11.8-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ee8db7bfb6fe03581bbab54d7c4124a6dd6a7f4273a38f7267197890f094675f", size = 135481, upload-time = "2026-03-31T16:14:55.901Z" }, + { url = "https://files.pythonhosted.org/packages/ff/f2/a8238e7734de7cb589fed319857a8025d509c89dc52fdcc88f39c6d03d5a/orjson-3.11.8-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5d8b5231de76c528a46b57010bbd83fb51e056aa0220a372fd5065e978406f1c", size = 146819, upload-time = "2026-03-31T16:14:57.548Z" }, + { url = "https://files.pythonhosted.org/packages/db/10/dbf1e2a3cafea673b1b4350e371877b759060d6018a998643b7040e5de48/orjson-3.11.8-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:58a4a208a6fbfdb7a7327b8f201c6014f189f721fd55d047cafc4157af1bc62a", size = 132846, upload-time = "2026-03-31T16:14:58.91Z" }, + { url = "https://files.pythonhosted.org/packages/f8/fc/55e667ec9c85694038fcff00573d221b085d50777368ee3d77f38668bf3c/orjson-3.11.8-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5f8952d6d2505c003e8f0224ff7858d341fa4e33fef82b91c4ff0ef070f2393c", size = 133580, upload-time = "2026-03-31T16:15:00.519Z" }, + { url = "https://files.pythonhosted.org/packages/7e/a6/c08c589a9aad0cb46c4831d17de212a2b6901f9d976814321ff8e69e8785/orjson-3.11.8-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:0022bb50f90da04b009ce32c512dc1885910daa7cb10b7b0cba4505b16db82a8", size = 142042, upload-time = "2026-03-31T16:15:01.906Z" }, + { url = "https://files.pythonhosted.org/packages/5c/cc/2f78ea241d52b717d2efc38878615fe80425bf2beb6e68c984dde257a766/orjson-3.11.8-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:ff51f9d657d1afb6f410cb435792ce4e1fe427aab23d2fcd727a2876e21d4cb6", size = 423845, upload-time = "2026-03-31T16:15:03.703Z" }, + { url = "https://files.pythonhosted.org/packages/70/07/c17dcf05dd8045457538428a983bf1f1127928df5bf328cb24d2b7cddacb/orjson-3.11.8-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:6dbe9a97bdb4d8d9d5367b52a7c32549bba70b2739c58ef74a6964a6d05ae054", size = 147729, upload-time = "2026-03-31T16:15:05.203Z" }, + { url = "https://files.pythonhosted.org/packages/90/6c/0fb6e8a24e682e0958d71711ae6f39110e4b9cd8cab1357e2a89cb8e1951/orjson-3.11.8-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:a5c370674ebabe16c6ccac33ff80c62bf8a6e59439f5e9d40c1f5ab8fd2215b7", size = 136425, upload-time = "2026-03-31T16:15:07.052Z" }, + { url = "https://files.pythonhosted.org/packages/b2/35/4d3cc3a3d616035beb51b24a09bb872942dc452cf2df0c1d11ab35046d9f/orjson-3.11.8-cp311-cp311-win32.whl", hash = "sha256:0e32f7154299f42ae66f13488963269e5eccb8d588a65bc839ed986919fc9fac", size = 131870, upload-time = "2026-03-31T16:15:08.678Z" }, + { url = "https://files.pythonhosted.org/packages/13/26/9fe70f81d16b702f8c3a775e8731b50ad91d22dacd14c7599b60a0941cd1/orjson-3.11.8-cp311-cp311-win_amd64.whl", hash = "sha256:25e0c672a2e32348d2eb33057b41e754091f2835f87222e4675b796b92264f06", size = 127440, upload-time = "2026-03-31T16:15:09.994Z" }, + { url = "https://files.pythonhosted.org/packages/e8/c6/b038339f4145efd2859c1ca53097a52c0bb9cbdd24f947ebe146da1ad067/orjson-3.11.8-cp311-cp311-win_arm64.whl", hash = "sha256:9185589c1f2a944c17e26c9925dcdbc2df061cc4a145395c57f0c51f9b5dbfcd", size = 127399, upload-time = "2026-03-31T16:15:11.412Z" }, + { url = "https://files.pythonhosted.org/packages/01/f6/8d58b32ab32d9215973a1688aebd098252ee8af1766c0e4e36e7831f0295/orjson-3.11.8-cp312-cp312-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:1cd0b77e77c95758f8e1100139844e99f3ccc87e71e6fc8e1c027e55807c549f", size = 229233, upload-time = "2026-03-31T16:15:12.762Z" }, + { url = "https://files.pythonhosted.org/packages/a9/8b/2ffe35e71f6b92622e8ea4607bf33ecf7dfb51b3619dcfabfd36cbe2d0a5/orjson-3.11.8-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:6a3d159d5ffa0e3961f353c4b036540996bf8b9697ccc38261c0eac1fd3347a6", size = 128772, upload-time = "2026-03-31T16:15:14.237Z" }, + { url = "https://files.pythonhosted.org/packages/27/d2/1f8682ae50d5c6897a563cb96bc106da8c9cb5b7b6e81a52e4cc086679b9/orjson-3.11.8-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:76070a76e9c5ae661e2d9848f216980d8d533e0f8143e6ed462807b242e3c5e8", size = 131946, upload-time = "2026-03-31T16:15:15.607Z" }, + { url = "https://files.pythonhosted.org/packages/52/4b/5500f76f0eece84226e0689cb48dcde081104c2fa6e2483d17ca13685ffb/orjson-3.11.8-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:54153d21520a71a4c82a0dbb4523e468941d549d221dc173de0f019678cf3813", size = 130368, upload-time = "2026-03-31T16:15:17.066Z" }, + { url = "https://files.pythonhosted.org/packages/da/4e/58b927e08fbe9840e6c920d9e299b051ea667463b1f39a56e668669f8508/orjson-3.11.8-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:469ac2125611b7c5741a0b3798cd9e5786cbad6345f9f400c77212be89563bec", size = 135540, upload-time = "2026-03-31T16:15:18.404Z" }, + { url = "https://files.pythonhosted.org/packages/56/7c/ba7cb871cba1bcd5cd02ee34f98d894c6cea96353ad87466e5aef2429c60/orjson-3.11.8-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:14778ffd0f6896aa613951a7fbf4690229aa7a543cb2bfbe9f358e08aafa9546", size = 146877, upload-time = "2026-03-31T16:15:19.833Z" }, + { url = "https://files.pythonhosted.org/packages/0b/5d/eb9c25fc1386696c6a342cd361c306452c75e0b55e86ad602dd4827a7fd7/orjson-3.11.8-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ea56a955056a6d6c550cf18b3348656a9d9a4f02e2d0c02cabf3c73f1055d506", size = 132837, upload-time = "2026-03-31T16:15:21.282Z" }, + { url = "https://files.pythonhosted.org/packages/37/87/5ddeb7fc1fbd9004aeccab08426f34c81a5b4c25c7061281862b015fce2b/orjson-3.11.8-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:53a0f57e59a530d18a142f4d4ba6dfc708dc5fdedce45e98ff06b44930a2a48f", size = 133624, upload-time = "2026-03-31T16:15:22.641Z" }, + { url = "https://files.pythonhosted.org/packages/22/09/90048793db94ee4b2fcec4ac8e5ddb077367637d6650be896b3494b79bb7/orjson-3.11.8-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:9b48e274f8824567d74e2158199e269597edf00823a1b12b63d48462bbf5123e", size = 141904, upload-time = "2026-03-31T16:15:24.435Z" }, + { url = "https://files.pythonhosted.org/packages/c0/cf/eb284847487821a5d415e54149a6449ba9bfc5872ce63ab7be41b8ec401c/orjson-3.11.8-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:3f262401086a3960586af06c054609365e98407151f5ea24a62893a40d80dbbb", size = 423742, upload-time = "2026-03-31T16:15:26.155Z" }, + { url = "https://files.pythonhosted.org/packages/44/09/e12423d327071c851c13e76936f144a96adacfc037394dec35ac3fc8d1e8/orjson-3.11.8-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:8e8c6218b614badf8e229b697865df4301afa74b791b6c9ade01d19a9953a942", size = 147806, upload-time = "2026-03-31T16:15:27.909Z" }, + { url = "https://files.pythonhosted.org/packages/b3/6d/37c2589ba864e582ffe7611643314785c6afb1f83c701654ef05daa8fcc7/orjson-3.11.8-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:093d489fa039ddade2db541097dbb484999fcc65fc2b0ff9819141e2ab364f25", size = 136485, upload-time = "2026-03-31T16:15:29.749Z" }, + { url = "https://files.pythonhosted.org/packages/be/c9/135194a02ab76b04ed9a10f68624b7ebd238bbe55548878b11ff15a0f352/orjson-3.11.8-cp312-cp312-win32.whl", hash = "sha256:e0950ed1bcb9893f4293fd5c5a7ee10934fbf82c4101c70be360db23ce24b7d2", size = 131966, upload-time = "2026-03-31T16:15:31.687Z" }, + { url = "https://files.pythonhosted.org/packages/ed/9a/9796f8fbe3cf30ce9cb696748dbb535e5c87be4bf4fe2e9ca498ef1fa8cf/orjson-3.11.8-cp312-cp312-win_amd64.whl", hash = "sha256:3cf17c141617b88ced4536b2135c552490f07799f6ad565948ea07bef0dcb9a6", size = 127441, upload-time = "2026-03-31T16:15:33.333Z" }, + { url = "https://files.pythonhosted.org/packages/cc/47/5aaf54524a7a4a0dd09dd778f3fa65dd2108290615b652e23d944152bc8e/orjson-3.11.8-cp312-cp312-win_arm64.whl", hash = "sha256:48854463b0572cc87dac7d981aa72ed8bf6deedc0511853dc76b8bbd5482d36d", size = 127364, upload-time = "2026-03-31T16:15:34.748Z" }, ] [[package]] @@ -1919,11 +1923,11 @@ wheels = [ [[package]] name = "packaging" -version = "26.0" +version = "26.1" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/65/ee/299d360cdc32edc7d2cf530f3accf79c4fca01e96ffc950d8a52213bd8e4/packaging-26.0.tar.gz", hash = "sha256:00243ae351a257117b6a241061796684b084ed1c516a08c48a3f7e147a9d80b4", size = 143416, upload-time = "2026-01-21T20:50:39.064Z" } +sdist = { url = "https://files.pythonhosted.org/packages/df/de/0d2b39fb4af88a0258f3bac87dfcbb48e73fbdea4a2ed0e2213f9a4c2f9a/packaging-26.1.tar.gz", hash = "sha256:f042152b681c4bfac5cae2742a55e103d27ab2ec0f3d88037136b6bfe7c9c5de", size = 215519, upload-time = "2026-04-14T21:12:49.362Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/b7/b9/c538f279a4e237a006a2c98387d081e9eb060d203d8ed34467cc0f0b9b53/packaging-26.0-py3-none-any.whl", hash = "sha256:b36f1fef9334a5588b4166f8bcd26a14e521f2b55e6b9de3aaa80d3ff7a37529", size = 74366, upload-time = "2026-01-21T20:50:37.788Z" }, + { url = "https://files.pythonhosted.org/packages/7a/c2/920ef838e2f0028c8262f16101ec09ebd5969864e5a64c4c05fad0617c56/packaging-26.1-py3-none-any.whl", hash = "sha256:5d9c0669c6285e491e0ced2eee587eaf67b670d94a19e94e3984a481aba6802f", size = 95831, upload-time = "2026-04-14T21:12:47.56Z" }, ] [[package]] @@ -1967,11 +1971,11 @@ wheels = [ [[package]] name = "platformdirs" -version = "4.9.4" +version = "4.9.6" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/19/56/8d4c30c8a1d07013911a8fdbd8f89440ef9f08d07a1b50ab8ca8be5a20f9/platformdirs-4.9.4.tar.gz", hash = "sha256:1ec356301b7dc906d83f371c8f487070e99d3ccf9e501686456394622a01a934", size = 28737, upload-time = "2026-03-05T18:34:13.271Z" } +sdist = { url = "https://files.pythonhosted.org/packages/9f/4a/0883b8e3802965322523f0b200ecf33d31f10991d0401162f4b23c698b42/platformdirs-4.9.6.tar.gz", hash = "sha256:3bfa75b0ad0db84096ae777218481852c0ebc6c727b3168c1b9e0118e458cf0a", size = 29400, upload-time = "2026-04-09T00:04:10.812Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/63/d7/97f7e3a6abb67d8080dd406fd4df842c2be0efaf712d1c899c32a075027c/platformdirs-4.9.4-py3-none-any.whl", hash = "sha256:68a9a4619a666ea6439f2ff250c12a853cd1cbd5158d258bd824a7df6be2f868", size = 21216, upload-time = "2026-03-05T18:34:12.172Z" }, + { url = "https://files.pythonhosted.org/packages/75/a6/a0a304dc33b49145b21f4808d763822111e67d1c3a32b524a1baf947b6e1/platformdirs-4.9.6-py3-none-any.whl", hash = "sha256:e61adb1d5e5cb3441b4b7710bea7e4c12250ca49439228cc1021c00dcfac0917", size = 21348, upload-time = "2026-04-09T00:04:09.463Z" }, ] [[package]] @@ -2001,11 +2005,11 @@ wheels = [ [[package]] name = "prometheus-client" -version = "0.24.1" +version = "0.25.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/f0/58/a794d23feb6b00fc0c72787d7e87d872a6730dd9ed7c7b3e954637d8f280/prometheus_client-0.24.1.tar.gz", hash = "sha256:7e0ced7fbbd40f7b84962d5d2ab6f17ef88a72504dcf7c0b40737b43b2a461f9", size = 85616, upload-time = "2026-01-14T15:26:26.965Z" } +sdist = { url = "https://files.pythonhosted.org/packages/1b/fb/d9aa83ffe43ce1f19e557c0971d04b90561b0cfd50762aafb01968285553/prometheus_client-0.25.0.tar.gz", hash = "sha256:5e373b75c31afb3c86f1a52fa1ad470c9aace18082d39ec0d2f918d11cc9ba28", size = 86035, upload-time = "2026-04-09T19:53:42.359Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/74/c3/24a2f845e3917201628ecaba4f18bab4d18a337834c1df2a159ee9d22a42/prometheus_client-0.24.1-py3-none-any.whl", hash = "sha256:150db128af71a5c2482b36e588fc8a6b95e498750da4b17065947c16070f4055", size = 64057, upload-time = "2026-01-14T15:26:24.42Z" }, + { url = "https://files.pythonhosted.org/packages/8d/9b/d4b1e644385499c8346fa9b622a3f030dce14cd6ef8a1871c221a17a67e7/prometheus_client-0.25.0-py3-none-any.whl", hash = "sha256:d5aec89e349a6ec230805d0df882f3807f74fd6c1a2fa86864e3c2279059fed1", size = 64154, upload-time = "2026-04-09T19:53:41.324Z" }, ] [[package]] @@ -2142,7 +2146,7 @@ wheels = [ [[package]] name = "pydantic" -version = "2.12.5" +version = "2.13.2" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "annotated-types" }, @@ -2150,64 +2154,66 @@ dependencies = [ { name = "typing-extensions" }, { name = "typing-inspection" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/69/44/36f1a6e523abc58ae5f928898e4aca2e0ea509b5aa6f6f392a5d882be928/pydantic-2.12.5.tar.gz", hash = "sha256:4d351024c75c0f085a9febbb665ce8c0c6ec5d30e903bdb6394b7ede26aebb49", size = 821591, upload-time = "2025-11-26T15:11:46.471Z" } +sdist = { url = "https://files.pythonhosted.org/packages/09/e5/06d23afac9973109d1e3c8ad38e1547a12e860610e327c05ee686827dc37/pydantic-2.13.2.tar.gz", hash = "sha256:b418196607e61081c3226dcd4f0672f2a194828abb9109e9cfb84026564df2d1", size = 843836, upload-time = "2026-04-17T09:31:59.636Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/5a/87/b70ad306ebb6f9b585f114d0ac2137d792b48be34d732d60e597c2f8465a/pydantic-2.12.5-py3-none-any.whl", hash = "sha256:e561593fccf61e8a20fc46dfc2dfe075b8be7d0188df33f221ad1f0139180f9d", size = 463580, upload-time = "2025-11-26T15:11:44.605Z" }, + { url = "https://files.pythonhosted.org/packages/77/ca/b45c378e6e8d0b90577288b533e04e95b7afd61bb1d51b6c263176435489/pydantic-2.13.2-py3-none-any.whl", hash = "sha256:a525087f4c03d7e7456a3de89b64cd693d2229933bb1068b9af6befd5563694e", size = 471947, upload-time = "2026-04-17T09:31:57.541Z" }, ] [[package]] name = "pydantic-core" -version = "2.41.5" +version = "2.46.2" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/71/70/23b021c950c2addd24ec408e9ab05d59b035b39d97cdc1130e1bce647bb6/pydantic_core-2.41.5.tar.gz", hash = "sha256:08daa51ea16ad373ffd5e7606252cc32f07bc72b28284b6bc9c6df804816476e", size = 460952, upload-time = "2025-11-04T13:43:49.098Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/e8/72/74a989dd9f2084b3d9530b0915fdda64ac48831c30dbf7c72a41a5232db8/pydantic_core-2.41.5-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:a3a52f6156e73e7ccb0f8cced536adccb7042be67cb45f9562e12b319c119da6", size = 2105873, upload-time = "2025-11-04T13:39:31.373Z" }, - { url = "https://files.pythonhosted.org/packages/12/44/37e403fd9455708b3b942949e1d7febc02167662bf1a7da5b78ee1ea2842/pydantic_core-2.41.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:7f3bf998340c6d4b0c9a2f02d6a400e51f123b59565d74dc60d252ce888c260b", size = 1899826, upload-time = "2025-11-04T13:39:32.897Z" }, - { url = "https://files.pythonhosted.org/packages/33/7f/1d5cab3ccf44c1935a359d51a8a2a9e1a654b744b5e7f80d41b88d501eec/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:378bec5c66998815d224c9ca994f1e14c0c21cb95d2f52b6021cc0b2a58f2a5a", size = 1917869, upload-time = "2025-11-04T13:39:34.469Z" }, - { url = "https://files.pythonhosted.org/packages/6e/6a/30d94a9674a7fe4f4744052ed6c5e083424510be1e93da5bc47569d11810/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e7b576130c69225432866fe2f4a469a85a54ade141d96fd396dffcf607b558f8", size = 2063890, upload-time = "2025-11-04T13:39:36.053Z" }, - { url = "https://files.pythonhosted.org/packages/50/be/76e5d46203fcb2750e542f32e6c371ffa9b8ad17364cf94bb0818dbfb50c/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6cb58b9c66f7e4179a2d5e0f849c48eff5c1fca560994d6eb6543abf955a149e", size = 2229740, upload-time = "2025-11-04T13:39:37.753Z" }, - { url = "https://files.pythonhosted.org/packages/d3/ee/fed784df0144793489f87db310a6bbf8118d7b630ed07aa180d6067e653a/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:88942d3a3dff3afc8288c21e565e476fc278902ae4d6d134f1eeda118cc830b1", size = 2350021, upload-time = "2025-11-04T13:39:40.94Z" }, - { url = "https://files.pythonhosted.org/packages/c8/be/8fed28dd0a180dca19e72c233cbf58efa36df055e5b9d90d64fd1740b828/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f31d95a179f8d64d90f6831d71fa93290893a33148d890ba15de25642c5d075b", size = 2066378, upload-time = "2025-11-04T13:39:42.523Z" }, - { url = "https://files.pythonhosted.org/packages/b0/3b/698cf8ae1d536a010e05121b4958b1257f0b5522085e335360e53a6b1c8b/pydantic_core-2.41.5-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c1df3d34aced70add6f867a8cf413e299177e0c22660cc767218373d0779487b", size = 2175761, upload-time = "2025-11-04T13:39:44.553Z" }, - { url = "https://files.pythonhosted.org/packages/b8/ba/15d537423939553116dea94ce02f9c31be0fa9d0b806d427e0308ec17145/pydantic_core-2.41.5-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:4009935984bd36bd2c774e13f9a09563ce8de4abaa7226f5108262fa3e637284", size = 2146303, upload-time = "2025-11-04T13:39:46.238Z" }, - { url = "https://files.pythonhosted.org/packages/58/7f/0de669bf37d206723795f9c90c82966726a2ab06c336deba4735b55af431/pydantic_core-2.41.5-cp311-cp311-musllinux_1_1_armv7l.whl", hash = "sha256:34a64bc3441dc1213096a20fe27e8e128bd3ff89921706e83c0b1ac971276594", size = 2340355, upload-time = "2025-11-04T13:39:48.002Z" }, - { url = "https://files.pythonhosted.org/packages/e5/de/e7482c435b83d7e3c3ee5ee4451f6e8973cff0eb6007d2872ce6383f6398/pydantic_core-2.41.5-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:c9e19dd6e28fdcaa5a1de679aec4141f691023916427ef9bae8584f9c2fb3b0e", size = 2319875, upload-time = "2025-11-04T13:39:49.705Z" }, - { url = "https://files.pythonhosted.org/packages/fe/e6/8c9e81bb6dd7560e33b9053351c29f30c8194b72f2d6932888581f503482/pydantic_core-2.41.5-cp311-cp311-win32.whl", hash = "sha256:2c010c6ded393148374c0f6f0bf89d206bf3217f201faa0635dcd56bd1520f6b", size = 1987549, upload-time = "2025-11-04T13:39:51.842Z" }, - { url = "https://files.pythonhosted.org/packages/11/66/f14d1d978ea94d1bc21fc98fcf570f9542fe55bfcc40269d4e1a21c19bf7/pydantic_core-2.41.5-cp311-cp311-win_amd64.whl", hash = "sha256:76ee27c6e9c7f16f47db7a94157112a2f3a00e958bc626e2f4ee8bec5c328fbe", size = 2011305, upload-time = "2025-11-04T13:39:53.485Z" }, - { url = "https://files.pythonhosted.org/packages/56/d8/0e271434e8efd03186c5386671328154ee349ff0354d83c74f5caaf096ed/pydantic_core-2.41.5-cp311-cp311-win_arm64.whl", hash = "sha256:4bc36bbc0b7584de96561184ad7f012478987882ebf9f9c389b23f432ea3d90f", size = 1972902, upload-time = "2025-11-04T13:39:56.488Z" }, - { url = "https://files.pythonhosted.org/packages/5f/5d/5f6c63eebb5afee93bcaae4ce9a898f3373ca23df3ccaef086d0233a35a7/pydantic_core-2.41.5-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:f41a7489d32336dbf2199c8c0a215390a751c5b014c2c1c5366e817202e9cdf7", size = 2110990, upload-time = "2025-11-04T13:39:58.079Z" }, - { url = "https://files.pythonhosted.org/packages/aa/32/9c2e8ccb57c01111e0fd091f236c7b371c1bccea0fa85247ac55b1e2b6b6/pydantic_core-2.41.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:070259a8818988b9a84a449a2a7337c7f430a22acc0859c6b110aa7212a6d9c0", size = 1896003, upload-time = "2025-11-04T13:39:59.956Z" }, - { url = "https://files.pythonhosted.org/packages/68/b8/a01b53cb0e59139fbc9e4fda3e9724ede8de279097179be4ff31f1abb65a/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e96cea19e34778f8d59fe40775a7a574d95816eb150850a85a7a4c8f4b94ac69", size = 1919200, upload-time = "2025-11-04T13:40:02.241Z" }, - { url = "https://files.pythonhosted.org/packages/38/de/8c36b5198a29bdaade07b5985e80a233a5ac27137846f3bc2d3b40a47360/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ed2e99c456e3fadd05c991f8f437ef902e00eedf34320ba2b0842bd1c3ca3a75", size = 2052578, upload-time = "2025-11-04T13:40:04.401Z" }, - { url = "https://files.pythonhosted.org/packages/00/b5/0e8e4b5b081eac6cb3dbb7e60a65907549a1ce035a724368c330112adfdd/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:65840751b72fbfd82c3c640cff9284545342a4f1eb1586ad0636955b261b0b05", size = 2208504, upload-time = "2025-11-04T13:40:06.072Z" }, - { url = "https://files.pythonhosted.org/packages/77/56/87a61aad59c7c5b9dc8caad5a41a5545cba3810c3e828708b3d7404f6cef/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e536c98a7626a98feb2d3eaf75944ef6f3dbee447e1f841eae16f2f0a72d8ddc", size = 2335816, upload-time = "2025-11-04T13:40:07.835Z" }, - { url = "https://files.pythonhosted.org/packages/0d/76/941cc9f73529988688a665a5c0ecff1112b3d95ab48f81db5f7606f522d3/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:eceb81a8d74f9267ef4081e246ffd6d129da5d87e37a77c9bde550cb04870c1c", size = 2075366, upload-time = "2025-11-04T13:40:09.804Z" }, - { url = "https://files.pythonhosted.org/packages/d3/43/ebef01f69baa07a482844faaa0a591bad1ef129253ffd0cdaa9d8a7f72d3/pydantic_core-2.41.5-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:d38548150c39b74aeeb0ce8ee1d8e82696f4a4e16ddc6de7b1d8823f7de4b9b5", size = 2171698, upload-time = "2025-11-04T13:40:12.004Z" }, - { url = "https://files.pythonhosted.org/packages/b1/87/41f3202e4193e3bacfc2c065fab7706ebe81af46a83d3e27605029c1f5a6/pydantic_core-2.41.5-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:c23e27686783f60290e36827f9c626e63154b82b116d7fe9adba1fda36da706c", size = 2132603, upload-time = "2025-11-04T13:40:13.868Z" }, - { url = "https://files.pythonhosted.org/packages/49/7d/4c00df99cb12070b6bccdef4a195255e6020a550d572768d92cc54dba91a/pydantic_core-2.41.5-cp312-cp312-musllinux_1_1_armv7l.whl", hash = "sha256:482c982f814460eabe1d3bb0adfdc583387bd4691ef00b90575ca0d2b6fe2294", size = 2329591, upload-time = "2025-11-04T13:40:15.672Z" }, - { url = "https://files.pythonhosted.org/packages/cc/6a/ebf4b1d65d458f3cda6a7335d141305dfa19bdc61140a884d165a8a1bbc7/pydantic_core-2.41.5-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:bfea2a5f0b4d8d43adf9d7b8bf019fb46fdd10a2e5cde477fbcb9d1fa08c68e1", size = 2319068, upload-time = "2025-11-04T13:40:17.532Z" }, - { url = "https://files.pythonhosted.org/packages/49/3b/774f2b5cd4192d5ab75870ce4381fd89cf218af999515baf07e7206753f0/pydantic_core-2.41.5-cp312-cp312-win32.whl", hash = "sha256:b74557b16e390ec12dca509bce9264c3bbd128f8a2c376eaa68003d7f327276d", size = 1985908, upload-time = "2025-11-04T13:40:19.309Z" }, - { url = "https://files.pythonhosted.org/packages/86/45/00173a033c801cacf67c190fef088789394feaf88a98a7035b0e40d53dc9/pydantic_core-2.41.5-cp312-cp312-win_amd64.whl", hash = "sha256:1962293292865bca8e54702b08a4f26da73adc83dd1fcf26fbc875b35d81c815", size = 2020145, upload-time = "2025-11-04T13:40:21.548Z" }, - { url = "https://files.pythonhosted.org/packages/f9/22/91fbc821fa6d261b376a3f73809f907cec5ca6025642c463d3488aad22fb/pydantic_core-2.41.5-cp312-cp312-win_arm64.whl", hash = "sha256:1746d4a3d9a794cacae06a5eaaccb4b8643a131d45fbc9af23e353dc0a5ba5c3", size = 1976179, upload-time = "2025-11-04T13:40:23.393Z" }, - { url = "https://files.pythonhosted.org/packages/11/72/90fda5ee3b97e51c494938a4a44c3a35a9c96c19bba12372fb9c634d6f57/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:b96d5f26b05d03cc60f11a7761a5ded1741da411e7fe0909e27a5e6a0cb7b034", size = 2115441, upload-time = "2025-11-04T13:42:39.557Z" }, - { url = "https://files.pythonhosted.org/packages/1f/53/8942f884fa33f50794f119012dc6a1a02ac43a56407adaac20463df8e98f/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:634e8609e89ceecea15e2d61bc9ac3718caaaa71963717bf3c8f38bfde64242c", size = 1930291, upload-time = "2025-11-04T13:42:42.169Z" }, - { url = "https://files.pythonhosted.org/packages/79/c8/ecb9ed9cd942bce09fc888ee960b52654fbdbede4ba6c2d6e0d3b1d8b49c/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:93e8740d7503eb008aa2df04d3b9735f845d43ae845e6dcd2be0b55a2da43cd2", size = 1948632, upload-time = "2025-11-04T13:42:44.564Z" }, - { url = "https://files.pythonhosted.org/packages/2e/1b/687711069de7efa6af934e74f601e2a4307365e8fdc404703afc453eab26/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f15489ba13d61f670dcc96772e733aad1a6f9c429cc27574c6cdaed82d0146ad", size = 2138905, upload-time = "2025-11-04T13:42:47.156Z" }, - { url = "https://files.pythonhosted.org/packages/09/32/59b0c7e63e277fa7911c2fc70ccfb45ce4b98991e7ef37110663437005af/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:7da7087d756b19037bc2c06edc6c170eeef3c3bafcb8f532ff17d64dc427adfd", size = 2110495, upload-time = "2025-11-04T13:42:49.689Z" }, - { url = "https://files.pythonhosted.org/packages/aa/81/05e400037eaf55ad400bcd318c05bb345b57e708887f07ddb2d20e3f0e98/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:aabf5777b5c8ca26f7824cb4a120a740c9588ed58df9b2d196ce92fba42ff8dc", size = 1915388, upload-time = "2025-11-04T13:42:52.215Z" }, - { url = "https://files.pythonhosted.org/packages/6e/0d/e3549b2399f71d56476b77dbf3cf8937cec5cd70536bdc0e374a421d0599/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c007fe8a43d43b3969e8469004e9845944f1a80e6acd47c150856bb87f230c56", size = 1942879, upload-time = "2025-11-04T13:42:56.483Z" }, - { url = "https://files.pythonhosted.org/packages/f7/07/34573da085946b6a313d7c42f82f16e8920bfd730665de2d11c0c37a74b5/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:76d0819de158cd855d1cbb8fcafdf6f5cf1eb8e470abe056d5d161106e38062b", size = 2139017, upload-time = "2025-11-04T13:42:59.471Z" }, - { url = "https://files.pythonhosted.org/packages/5f/9b/1b3f0e9f9305839d7e84912f9e8bfbd191ed1b1ef48083609f0dabde978c/pydantic_core-2.41.5-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:b2379fa7ed44ddecb5bfe4e48577d752db9fc10be00a6b7446e9663ba143de26", size = 2101980, upload-time = "2025-11-04T13:43:25.97Z" }, - { url = "https://files.pythonhosted.org/packages/a4/ed/d71fefcb4263df0da6a85b5d8a7508360f2f2e9b3bf5814be9c8bccdccc1/pydantic_core-2.41.5-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:266fb4cbf5e3cbd0b53669a6d1b039c45e3ce651fd5442eff4d07c2cc8d66808", size = 1923865, upload-time = "2025-11-04T13:43:28.763Z" }, - { url = "https://files.pythonhosted.org/packages/ce/3a/626b38db460d675f873e4444b4bb030453bbe7b4ba55df821d026a0493c4/pydantic_core-2.41.5-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:58133647260ea01e4d0500089a8c4f07bd7aa6ce109682b1426394988d8aaacc", size = 2134256, upload-time = "2025-11-04T13:43:31.71Z" }, - { url = "https://files.pythonhosted.org/packages/83/d9/8412d7f06f616bbc053d30cb4e5f76786af3221462ad5eee1f202021eb4e/pydantic_core-2.41.5-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:287dad91cfb551c363dc62899a80e9e14da1f0e2b6ebde82c806612ca2a13ef1", size = 2174762, upload-time = "2025-11-04T13:43:34.744Z" }, - { url = "https://files.pythonhosted.org/packages/55/4c/162d906b8e3ba3a99354e20faa1b49a85206c47de97a639510a0e673f5da/pydantic_core-2.41.5-pp311-pypy311_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:03b77d184b9eb40240ae9fd676ca364ce1085f203e1b1256f8ab9984dca80a84", size = 2143141, upload-time = "2025-11-04T13:43:37.701Z" }, - { url = "https://files.pythonhosted.org/packages/1f/f2/f11dd73284122713f5f89fc940f370d035fa8e1e078d446b3313955157fe/pydantic_core-2.41.5-pp311-pypy311_pp73-musllinux_1_1_armv7l.whl", hash = "sha256:a668ce24de96165bb239160b3d854943128f4334822900534f2fe947930e5770", size = 2330317, upload-time = "2025-11-04T13:43:40.406Z" }, - { url = "https://files.pythonhosted.org/packages/88/9d/b06ca6acfe4abb296110fb1273a4d848a0bfb2ff65f3ee92127b3244e16b/pydantic_core-2.41.5-pp311-pypy311_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:f14f8f046c14563f8eb3f45f499cc658ab8d10072961e07225e507adb700e93f", size = 2316992, upload-time = "2025-11-04T13:43:43.602Z" }, - { url = "https://files.pythonhosted.org/packages/36/c7/cfc8e811f061c841d7990b0201912c3556bfeb99cdcb7ed24adc8d6f8704/pydantic_core-2.41.5-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:56121965f7a4dc965bff783d70b907ddf3d57f6eba29b6d2e5dabfaf07799c51", size = 2145302, upload-time = "2025-11-04T13:43:46.64Z" }, +sdist = { url = "https://files.pythonhosted.org/packages/43/bb/4742f05b739b2478459bb16fa8470549518c802e06ddcf3f106c5081315e/pydantic_core-2.46.2.tar.gz", hash = "sha256:37bb079f9ee3f1a519392b73fda2a96379b31f2013c6b467fe693e7f2987f596", size = 471269, upload-time = "2026-04-17T09:10:07.017Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/89/91/089f517a725f29084364169437833ab0ae4da4d7a6ed9d4474db7f1412e6/pydantic_core-2.46.2-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:d8060f42db3cd204871db0afd51fef54a13fa544c4dd48cdcae2e174ef40c8ba", size = 2106218, upload-time = "2026-04-17T09:10:48.023Z" }, + { url = "https://files.pythonhosted.org/packages/a0/92/23858ed1b58f2a134e50c2fdd0e34ea72721ccb257e1e9346514e1ccb5b9/pydantic_core-2.46.2-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:73a9d2809bd8d4a7cda4d336dc996a565eb4feaaa39932f9d85a65fa18382f28", size = 1948087, upload-time = "2026-04-17T09:11:58.639Z" }, + { url = "https://files.pythonhosted.org/packages/5d/ac/e2240fccb4794e965817593d5a46cf5ea22f2001b73fe360b7578925b7d8/pydantic_core-2.46.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3b0a2dee92dfaabcfb93629188c3e9cf74fdfc0f22e7c369cb444a98814a1e50", size = 1972931, upload-time = "2026-04-17T09:13:13.304Z" }, + { url = "https://files.pythonhosted.org/packages/1a/da/3b11dab2aa15c5c8ed20a01eb7aa432a78b8e3a4713659f7e58490a020a5/pydantic_core-2.46.2-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3098446ba8cf774f61cb8d4008c1dba14a30426a15169cd95ac3392a461193b1", size = 2040454, upload-time = "2026-04-17T09:13:47.895Z" }, + { url = "https://files.pythonhosted.org/packages/d7/39/c4cf5e1f1c6c34c53c0902039c95d81dc15cdd1f03634bd1a93f33e70a72/pydantic_core-2.46.2-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:57c584af6c375ea3f826d8131a94cb212b3d9926eaff67117e3711bbff3a83a5", size = 2221320, upload-time = "2026-04-17T09:13:08.568Z" }, + { url = "https://files.pythonhosted.org/packages/c7/46/891035bc9e93538e754c3188424d24b5a69ec3ae5210fa01d483e99b3302/pydantic_core-2.46.2-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:547381cca999be88b4715a0ed7afa11f07fc7e53cb1883687b190d25a92c56cf", size = 2274559, upload-time = "2026-04-17T09:11:10.257Z" }, + { url = "https://files.pythonhosted.org/packages/ab/d0/7af0b905b3148152c159c9caf203e7ecd9b90b76389f0862e6ab0cf1b2a3/pydantic_core-2.46.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:caeed15dcb1233a5a94bc6ff37ef5393cf5b33a45e4bdfb2d6042f3d24e1cb27", size = 2089239, upload-time = "2026-04-17T09:13:06.326Z" }, + { url = "https://files.pythonhosted.org/packages/c5/bc/566afe02ba2de37712eece74ac7bfba322abd7916410bf90504f1b17ddad/pydantic_core-2.46.2-cp311-cp311-manylinux_2_31_riscv64.whl", hash = "sha256:c05f53362568c75476b5c96659377a5dfd982cfbe5a5c07de5106d08a04efc4f", size = 2116182, upload-time = "2026-04-17T09:11:33.738Z" }, + { url = "https://files.pythonhosted.org/packages/4e/5b/3fcb3a229bbfa23b0e3c65014057af0f9d51ec7a2d9f7adb282f41ff5ac8/pydantic_core-2.46.2-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2643ac7eae296200dbd48762a1c852cf2cad5f5e3eba34e652053cebf03becf8", size = 2172346, upload-time = "2026-04-17T09:10:46.472Z" }, + { url = "https://files.pythonhosted.org/packages/43/9a/baa9e3aa70ea7bbcb9db0f87162a371649ac80c03e43eb54af193390cf17/pydantic_core-2.46.2-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:dc4620a47c6fe6a39f89392c00833a82fc050ce90169798f78a25a8d4df03b6e", size = 2179540, upload-time = "2026-04-17T09:11:21.881Z" }, + { url = "https://files.pythonhosted.org/packages/bd/46/912047a5427f949c909495704b3c8b9ead9d1c66f87e96606011beab1fcb/pydantic_core-2.46.2-cp311-cp311-musllinux_1_1_armv7l.whl", hash = "sha256:78cb0d2453b50bf2035f85fd0d9cfabdb98c47f9c53ddb7c23873cd83da9560b", size = 2327423, upload-time = "2026-04-17T09:13:40.291Z" }, + { url = "https://files.pythonhosted.org/packages/e9/bf/c5e661451dc9411c2ab88a244c1ba57644950c971486040dc200f77b69f4/pydantic_core-2.46.2-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:f0c1cbb7d6112932cc188c6be007a5e2867005a069e47f42fe67bf5f122b0908", size = 2348652, upload-time = "2026-04-17T09:10:37.76Z" }, + { url = "https://files.pythonhosted.org/packages/77/b3/3219e7c522af54b010cf7422dcb11cc6616a4414d1ccd628b0d3f61c6af6/pydantic_core-2.46.2-cp311-cp311-win32.whl", hash = "sha256:c1ce5b2366f85cfdbf7f0907755043707f86d09a5b1b1acebbb7bf1600d75c64", size = 1974410, upload-time = "2026-04-17T09:13:27.392Z" }, + { url = "https://files.pythonhosted.org/packages/e5/29/e5cfac8a74c59873dfd47d3a1477c39ad9247639a7120d3e251a9ff12417/pydantic_core-2.46.2-cp311-cp311-win_amd64.whl", hash = "sha256:f1a6197eadff5bd0bb932f12bb038d403cb75db5b0b391e70e816a647745ddaf", size = 2071158, upload-time = "2026-04-17T09:09:57.69Z" }, + { url = "https://files.pythonhosted.org/packages/6f/8b/b7b19b717cdb3675cb109de143f62d4dc62f5d4a0b9879b6f1ace62c6654/pydantic_core-2.46.2-cp311-cp311-win_arm64.whl", hash = "sha256:15e42885b283f87846ee79e161002c5c496ef747a73f6e47054f45a13d9035bc", size = 2043507, upload-time = "2026-04-17T09:09:51.828Z" }, + { url = "https://files.pythonhosted.org/packages/97/ec/2fafa4c86f5d2a69372c7cddef30925fd0e370b1efaf556609c1a0196d8a/pydantic_core-2.46.2-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:ea1ad8c89da31512fe2d249cf0638fb666925bda341901541bc5f3311c6fcc9e", size = 2101729, upload-time = "2026-04-17T09:12:30.042Z" }, + { url = "https://files.pythonhosted.org/packages/cf/55/be5386c2c4b49af346e8a26b748194ff25757bbb6cf544130854e997af7a/pydantic_core-2.46.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b308da17b92481e0587244631c5529e5d91d04cb2b08194825627b1eca28e21e", size = 1951546, upload-time = "2026-04-17T09:10:10.585Z" }, + { url = "https://files.pythonhosted.org/packages/29/92/89e273a055ce440e6636c756379af35ad86da9d336a560049c3ba5e41c80/pydantic_core-2.46.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d333a50bdd814a917d8d6a7ee35ba2395d53ddaa882613bc24e54a9d8b129095", size = 1976178, upload-time = "2026-04-17T09:11:49.619Z" }, + { url = "https://files.pythonhosted.org/packages/91/b3/e4664469cf70c0cb0f7b2f5719d64e5968bb6f38217042c2afa3d3c4ba17/pydantic_core-2.46.2-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:1d00b99590c5bd1fabbc5d28b170923e32c1b1071b1f1de1851a4d14d89eb192", size = 2051697, upload-time = "2026-04-17T09:12:04.917Z" }, + { url = "https://files.pythonhosted.org/packages/98/58/dbf68213ee06ce51cdd6d8c95f97980e646858c45bd96bd2dfb40433be73/pydantic_core-2.46.2-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9f0e686960ffe9e65066395af856ac2d52c159043144433602c50c221d81c1ba", size = 2233160, upload-time = "2026-04-17T09:12:00.956Z" }, + { url = "https://files.pythonhosted.org/packages/f5/d3/68092aa0ee6c60ff4de4740eb82db3d4ce338ec89b3cecb978c532472f12/pydantic_core-2.46.2-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2d1128da41c9cb474e0a4701f9c363ec645c9d1a02229904c76bf4e0a194fde2", size = 2298398, upload-time = "2026-04-17T09:10:29.694Z" }, + { url = "https://files.pythonhosted.org/packages/e4/51/5d6155eb737db55b0ad354ca5f333ef009f75feb67df2d79a84bace45af6/pydantic_core-2.46.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:48649cf2d8c358d79586e9fb2f8235902fcaa2d969ec1c5301f2d1873b2f8321", size = 2094058, upload-time = "2026-04-17T09:12:10.995Z" }, + { url = "https://files.pythonhosted.org/packages/6b/f3/eb4a986197d71319430464ff181226c95adc8f06d932189b158bae5a82f5/pydantic_core-2.46.2-cp312-cp312-manylinux_2_31_riscv64.whl", hash = "sha256:b902f0fc7c2cf503865a05718b68147c6cd5d0a3867af38c527be574a9fa6e9d", size = 2130388, upload-time = "2026-04-17T09:12:41.159Z" }, + { url = "https://files.pythonhosted.org/packages/56/00/44a9c4fe6d0f64b5786d6a8c649d6f0e34ba6c89b3663add1066e54451a2/pydantic_core-2.46.2-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e80011f808b03d1d87a8f1e76ae3da19a18eb706c823e17981dcf1fae43744fc", size = 2184245, upload-time = "2026-04-17T09:12:36.532Z" }, + { url = "https://files.pythonhosted.org/packages/78/6b/685b98a834d5e3d1c34a1bde1627525559dd223b75075bc7490cdb24eb33/pydantic_core-2.46.2-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:b839d5c802e31348b949b6473f8190cddbf7d47475856d8ac995a373ee16ec59", size = 2186842, upload-time = "2026-04-17T09:13:04.054Z" }, + { url = "https://files.pythonhosted.org/packages/22/64/caa2f5a2ac8b6113adaa410ccdf31ba7f54897a6e54cd0d726fc7e780c88/pydantic_core-2.46.2-cp312-cp312-musllinux_1_1_armv7l.whl", hash = "sha256:c6b1064f3f9cf9072e1d59dd2936f9f3b668bec1c37039708c9222db703c0d5b", size = 2336066, upload-time = "2026-04-17T09:12:13.006Z" }, + { url = "https://files.pythonhosted.org/packages/ee/f9/7d2701bf82945b5b9e7df8347be97ef6a36da2846bfe5b4afec299ffe27b/pydantic_core-2.46.2-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:37a68e6f2ac95578ce3c0564802404b27b24988649616e556c07e77111ed3f1d", size = 2363691, upload-time = "2026-04-17T09:13:42.972Z" }, + { url = "https://files.pythonhosted.org/packages/3b/65/0dab11574101522941055109419db3cc09db871643dc3fc74e2413215e5b/pydantic_core-2.46.2-cp312-cp312-win32.whl", hash = "sha256:d9ffa75a7ef4b97d6e5e205fabd4304ef01fec09e6f1bdde04b9ad1b07d20289", size = 1958801, upload-time = "2026-04-17T09:11:31.981Z" }, + { url = "https://files.pythonhosted.org/packages/13/2b/df84baa609c676f6450b8ecad44ea59146c805e3371b7b52443c0899f989/pydantic_core-2.46.2-cp312-cp312-win_amd64.whl", hash = "sha256:0551f2d2ddb68af5a00e26497f8025c538f73ef3cb698f8e5a487042cd2792a8", size = 2072634, upload-time = "2026-04-17T09:11:02.407Z" }, + { url = "https://files.pythonhosted.org/packages/d1/4e/e1ce8029fc438086a946739bf9d596f70ff470aad4a8345555920618cabe/pydantic_core-2.46.2-cp312-cp312-win_arm64.whl", hash = "sha256:83aef30f106edcc21a6a4cc44b82d3169a1dbe255508db788e778f3c804d3583", size = 2026188, upload-time = "2026-04-17T09:13:11.083Z" }, + { url = "https://files.pythonhosted.org/packages/25/ec/e91aa08df1c33d5e3c2b60c07a1eca9f21809728a824c7b467bb3bda68b5/pydantic_core-2.46.2-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:7c5a5b3dbb9e8918e223be6580da5ffcf861c0505bbc196ebed7176ce05b7b4e", size = 2105046, upload-time = "2026-04-17T09:10:55.614Z" }, + { url = "https://files.pythonhosted.org/packages/f0/73/27112400a0452e375290e7c40aef5cc9844ac0920fb1029238cfc68121fa/pydantic_core-2.46.2-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:bc1e8ce33d5a337f2ba862e0719b8201cd54aaed967406c748e009191d47efdd", size = 1940029, upload-time = "2026-04-17T09:12:21.5Z" }, + { url = "https://files.pythonhosted.org/packages/b1/44/3d39f782bc82ddd0b2d82bde83b408aa40a332cdf6f3018acb34e3d4dcfc/pydantic_core-2.46.2-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b737c0b280f41143266445de2689c0e49c79307e51c44ce3a77fef2bedad4994", size = 1987772, upload-time = "2026-04-17T09:10:02.357Z" }, + { url = "https://files.pythonhosted.org/packages/c4/1a/0242e5b7b6cf51dbccc065029f0420107b6bf7e191fcb918f5cb71218acf/pydantic_core-2.46.2-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1b877d597afb82b4898e35354bba55de6f7f048421ae0edadbb9886ec137b532", size = 2138468, upload-time = "2026-04-17T09:11:51.546Z" }, + { url = "https://files.pythonhosted.org/packages/f3/d2/66c146f421178641bda880b0267c0d57dd84f5fec9ecc8e46be17b480742/pydantic_core-2.46.2-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:e9fcabd1857492b5bf16f90258babde50f618f55d046b1309972da2396321ff9", size = 2091621, upload-time = "2026-04-17T09:12:47.501Z" }, + { url = "https://files.pythonhosted.org/packages/ee/b2/c28419aa9fc8055f4ac8e801d1d11c6357351bfa4321ed9bafab3eb98087/pydantic_core-2.46.2-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:fb3ec2c7f54c07b30d89983ce78dc32c37dd06a972448b8716d609493802d628", size = 1937059, upload-time = "2026-04-17T09:10:53.554Z" }, + { url = "https://files.pythonhosted.org/packages/30/ce/cd0824a2db213dc17113291b7a09b9b0ccd9fbf97daa4b81548703341baf/pydantic_core-2.46.2-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:130a6c837d819ef33e8c2bf702ed2c3429237ea69807f1140943d6f4bdaf52fa", size = 1997278, upload-time = "2026-04-17T09:12:23.784Z" }, + { url = "https://files.pythonhosted.org/packages/c9/69/47283fe3c0c967d3e9e9cd6c42b70907610c8a6f8d6e8381f1bb55f8006c/pydantic_core-2.46.2-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c2e25417cec5cd9bddb151e33cb08c50160f317479ecc02b22a95ec18f8fe004", size = 2147096, upload-time = "2026-04-17T09:12:43.124Z" }, + { url = "https://files.pythonhosted.org/packages/16/d5/dec7c127fa722ff56e1ccf1e960ae1318a9f66742135e97bf9771447216f/pydantic_core-2.46.2-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:c3ad79ed32004d9de91cacd4b5faaff44d56051392fe1d5526feda596f01af25", size = 2107613, upload-time = "2026-04-17T09:10:36.269Z" }, + { url = "https://files.pythonhosted.org/packages/bc/35/975c109b337260a71c93198baf663982b6b39fe3e584e279548a0969e5d4/pydantic_core-2.46.2-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:d157c48d28eebe5d46906de06a6a2f2c9e00b67d3e42de1f1b9c2d42b810f77c", size = 1947099, upload-time = "2026-04-17T09:12:15.304Z" }, + { url = "https://files.pythonhosted.org/packages/4e/11/52a971a0f9218631690274be533f05e5ddde5547f0823bb3e9dfd1be49f6/pydantic_core-2.46.2-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7b42c6471288dedc979ac8400d9c9770f03967dd187db1f8d3405d4d182cc714", size = 2133866, upload-time = "2026-04-17T09:12:27.994Z" }, + { url = "https://files.pythonhosted.org/packages/fe/7a/33d94d0698602b2d1712e78c703a33952eb2ca69e02e8e4b208e7f6602b5/pydantic_core-2.46.2-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:4f27bc4801358dc070d6697b41237fce9923d8e69a1ce1e95606ac36c1552dc1", size = 2161721, upload-time = "2026-04-17T09:11:16.111Z" }, + { url = "https://files.pythonhosted.org/packages/b0/cb/0df7ee0a148e9ce0968a80787967ddca9f6b3f8a49152a881b88da262701/pydantic_core-2.46.2-pp311-pypy311_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:e094a8f85db41aa7f6a45c5dac2950afc9862e66832934231962252b5d284eed", size = 2180175, upload-time = "2026-04-17T09:11:41.577Z" }, + { url = "https://files.pythonhosted.org/packages/8e/a8/258a32878140347532be4e44c6f3b1ace3b52b9c9ca7548a65ce18adf4b4/pydantic_core-2.46.2-pp311-pypy311_pp73-musllinux_1_1_armv7l.whl", hash = "sha256:807eeda5551f6884d3b4421578be37be50ddb7a58832348e99617a6714a73748", size = 2319882, upload-time = "2026-04-17T09:10:21.872Z" }, + { url = "https://files.pythonhosted.org/packages/13/b9/5071c298a0f91314a5402b8c56e0efbcebe77085327d0b4df7dc9cb0b674/pydantic_core-2.46.2-pp311-pypy311_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:fcaa1c3c846a7f6686b38fe493d1b2e8007380e293bfef6a9354563c026cbf36", size = 2348065, upload-time = "2026-04-17T09:11:08.263Z" }, + { url = "https://files.pythonhosted.org/packages/75/f3/0a7087e5f861d66ca64ce927230b397cc264c87b712156e6a93b26a459c8/pydantic_core-2.46.2-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:154dbfdfb11b8cbd8ff4d00d0b81e3d19f4cb4bedd5aa9f091060ba071474c6a", size = 2192159, upload-time = "2026-04-17T09:11:20.123Z" }, ] [[package]] @@ -2289,7 +2295,7 @@ wheels = [ [[package]] name = "pytest" -version = "9.0.2" +version = "9.0.3" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "colorama", marker = "sys_platform == 'win32'" }, @@ -2298,9 +2304,9 @@ dependencies = [ { name = "pluggy" }, { name = "pygments" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/d1/db/7ef3487e0fb0049ddb5ce41d3a49c235bf9ad299b6a25d5780a89f19230f/pytest-9.0.2.tar.gz", hash = "sha256:75186651a92bd89611d1d9fc20f0b4345fd827c41ccd5c299a868a05d70edf11", size = 1568901, upload-time = "2025-12-06T21:30:51.014Z" } +sdist = { url = "https://files.pythonhosted.org/packages/7d/0d/549bd94f1a0a402dc8cf64563a117c0f3765662e2e668477624baeec44d5/pytest-9.0.3.tar.gz", hash = "sha256:b86ada508af81d19edeb213c681b1d48246c1a91d304c6c81a427674c17eb91c", size = 1572165, upload-time = "2026-04-07T17:16:18.027Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/3b/ab/b3226f0bd7cdcf710fbede2b3548584366da3b19b5021e74f5bde2a8fa3f/pytest-9.0.2-py3-none-any.whl", hash = "sha256:711ffd45bf766d5264d487b917733b453d917afd2b0ad65223959f59089f875b", size = 374801, upload-time = "2025-12-06T21:30:49.154Z" }, + { url = "https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9", size = 375249, upload-time = "2026-04-07T17:16:16.13Z" }, ] [[package]] @@ -2330,15 +2336,15 @@ wheels = [ [[package]] name = "python-discovery" -version = "1.2.1" +version = "1.2.2" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "filelock" }, { name = "platformdirs" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/b9/88/815e53084c5079a59df912825a279f41dd2e0df82281770eadc732f5352c/python_discovery-1.2.1.tar.gz", hash = "sha256:180c4d114bff1c32462537eac5d6a332b768242b76b69c0259c7d14b1b680c9e", size = 58457, upload-time = "2026-03-26T22:30:44.496Z" } +sdist = { url = "https://files.pythonhosted.org/packages/de/ef/3bae0e537cfe91e8431efcba4434463d2c5a65f5a89edd47c6cf2f03c55f/python_discovery-1.2.2.tar.gz", hash = "sha256:876e9c57139eb757cb5878cbdd9ae5379e5d96266c99ef731119e04fffe533bb", size = 58872, upload-time = "2026-04-07T17:28:49.249Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/67/0f/019d3949a40280f6193b62bc010177d4ce702d0fce424322286488569cd3/python_discovery-1.2.1-py3-none-any.whl", hash = "sha256:b6a957b24c1cd79252484d3566d1b49527581d46e789aaf43181005e56201502", size = 31674, upload-time = "2026-03-26T22:30:43.396Z" }, + { url = "https://files.pythonhosted.org/packages/d8/db/795879cc3ddfe338599bddea6388cc5100b088db0a4caf6e6c1af1c27e04/python_discovery-1.2.2-py3-none-any.whl", hash = "sha256:e1ae95d9af875e78f15e19aed0c6137ab1bb49c200f21f5061786490c9585c7a", size = 31894, upload-time = "2026-04-07T17:28:48.09Z" }, ] [[package]] @@ -2474,47 +2480,47 @@ wheels = [ [[package]] name = "regex" -version = "2026.3.32" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/81/93/5ab3e899c47fa7994e524447135a71cd121685a35c8fe35029005f8b236f/regex-2026.3.32.tar.gz", hash = "sha256:f1574566457161678297a116fa5d1556c5a4159d64c5ff7c760e7c564bf66f16", size = 415605, upload-time = "2026-03-28T21:49:22.012Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/92/c1/c68163a6ce455996db71e249a65234b1c9f79a914ea2108c6c9af9e1812a/regex-2026.3.32-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:0d7855f5e59fcf91d0c9f4a51dc5d8847813832a2230c3e8e35912ccf20baaa2", size = 489568, upload-time = "2026-03-28T21:45:58.791Z" }, - { url = "https://files.pythonhosted.org/packages/96/9c/0bdd47733b832b5caa11e63df14dccdb311b41ab33c1221e249af4421f8f/regex-2026.3.32-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:18eb45f711e942c27dbed4109830bd070d8d618e008d0db39705f3f57070a4c6", size = 291287, upload-time = "2026-03-28T21:46:00.46Z" }, - { url = "https://files.pythonhosted.org/packages/e1/ff/1977a595f15f8dc355f9cebd875dab67f3faeca1f36b905fe53305bbcaed/regex-2026.3.32-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:ed3b8281c5d0944d939c82db4ec2300409dd69ee087f7a75a94f2e301e855fb4", size = 289325, upload-time = "2026-03-28T21:46:02.285Z" }, - { url = "https://files.pythonhosted.org/packages/0a/68/dfa21aef5af4a144702befeb5ff20ea9f9fbe40a4dfd08d56148b5b48b0a/regex-2026.3.32-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ad5c53f2e8fcae9144009435ebe3d9832003508cf8935c04542a1b3b8deefa15", size = 790898, upload-time = "2026-03-28T21:46:04.079Z" }, - { url = "https://files.pythonhosted.org/packages/36/26/9424e43e0e31ac3ce1ba0e7232ee91e113a04a579c53331bc0f16a4a5bf7/regex-2026.3.32-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:70c634e39c5cda0da05c93d6747fdc957599f7743543662b6dbabdd8d3ba8a96", size = 862462, upload-time = "2026-03-28T21:46:05.923Z" }, - { url = "https://files.pythonhosted.org/packages/63/a8/06573154ac891c6b55b74a88e0fb7c10081c20916b82dd0abc8cef938e13/regex-2026.3.32-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:1e0f6648fd48f4c73d801c55ab976cd602e2da87de99c07bff005b131f269c6a", size = 906522, upload-time = "2026-03-28T21:46:07.988Z" }, - { url = "https://files.pythonhosted.org/packages/e7/26/46673bb18448c51222c6272c850484a0092f364fae8d0315be9aa1e4baa7/regex-2026.3.32-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c5e0fdb5744caf1036dec5510f543164f2144cb64932251f6dfd42fa872b7f9c", size = 798289, upload-time = "2026-03-28T21:46:09.959Z" }, - { url = "https://files.pythonhosted.org/packages/4d/cb/804f1bd5ff08687258e6a92b040aba9b770e626b8d3ba21fffdfa21db2db/regex-2026.3.32-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:dab4178a0bc1ef13178832b12db7bc7f562e8f028b2b5be186e370090dc50652", size = 774823, upload-time = "2026-03-28T21:46:12.049Z" }, - { url = "https://files.pythonhosted.org/packages/e5/94/28a58258f8d822fb949c8ff87fc7e5f2a346922360ec084c193b3c95e51c/regex-2026.3.32-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:f95bd07f301135771559101c060f558e2cf896c7df00bec050ca7f93bf11585a", size = 781381, upload-time = "2026-03-28T21:46:13.746Z" }, - { url = "https://files.pythonhosted.org/packages/c4/f3/71e69dbe0543586a3e3532cf36e8c9b38d6d93033161a9799c1e9090eb78/regex-2026.3.32-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:2dcca2bceb823c9cc610e57b86a265d7ffc30e9fe98548c609eba8bd3c0c2488", size = 855968, upload-time = "2026-03-28T21:46:15.762Z" }, - { url = "https://files.pythonhosted.org/packages/6d/99/850feec404a02b62e048718ec1b4b98b5c3848cd9ca2316d0bdb65a53f6a/regex-2026.3.32-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:567b57eb987547a23306444e4f6f85d4314f83e65c71d320d898aa7550550443", size = 762785, upload-time = "2026-03-28T21:46:17.394Z" }, - { url = "https://files.pythonhosted.org/packages/40/04/808ab0462a2d19b295a3b42134f5183692f798addfe6a8b6aa5f7c7a35b2/regex-2026.3.32-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:b6acb765e7c1f2fa08ac9057a33595e26104d7d67046becae184a8f100932dd9", size = 845797, upload-time = "2026-03-28T21:46:19.269Z" }, - { url = "https://files.pythonhosted.org/packages/06/53/8afcf0fd4bd55440b48442c86cddfe61b0d21c92d96e384c0c47d769f4c3/regex-2026.3.32-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:c1ed17104d1be7f807fdec35ec99777168dd793a09510d753f8710590ba54cdd", size = 785200, upload-time = "2026-03-28T21:46:20.939Z" }, - { url = "https://files.pythonhosted.org/packages/99/4d/23d992ab4115456fec520d6c3aae39e0e33739b244ddb39aa4102a0f7ef0/regex-2026.3.32-cp311-cp311-win32.whl", hash = "sha256:c60f1de066eb5a0fd8ee5974de4194bb1c2e7692941458807162ffbc39887303", size = 266351, upload-time = "2026-03-28T21:46:22.515Z" }, - { url = "https://files.pythonhosted.org/packages/62/74/27c3cdb3a3fbbf67f7231b872877416ec817ae84271573d2fd14bf8723d3/regex-2026.3.32-cp311-cp311-win_amd64.whl", hash = "sha256:8fe14e24124ef41220e5992a0f09432f890037df6f93fd3d6b7a0feff2db16b2", size = 278639, upload-time = "2026-03-28T21:46:24.016Z" }, - { url = "https://files.pythonhosted.org/packages/0a/12/6a67bd509f38aec021d63096dbc884f39473e92adeb1e35d6fb6d89cbd59/regex-2026.3.32-cp311-cp311-win_arm64.whl", hash = "sha256:ded4fc0edf3de792850cb8b04bbf3c5bd725eeaf9df4c27aad510f6eed9c4e19", size = 270594, upload-time = "2026-03-28T21:46:25.857Z" }, - { url = "https://files.pythonhosted.org/packages/38/94/69492c45b0e61b027109d8433a5c3d4f7a90709184c057c7cfc60acb1bfa/regex-2026.3.32-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:ad8d372587e659940568afd009afeb72be939c769c552c9b28773d0337251391", size = 490572, upload-time = "2026-03-28T21:46:28.031Z" }, - { url = "https://files.pythonhosted.org/packages/92/0a/7dcffeebe0fcac45a1f9caf80712002d3cbd66d7d69d719315ee142b280f/regex-2026.3.32-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3f5747501b69299c6b0b047853771e4ed390510bada68cb16da9c9c2078343f7", size = 292078, upload-time = "2026-03-28T21:46:29.789Z" }, - { url = "https://files.pythonhosted.org/packages/e3/ec/988486058ef49eb931476419bae00f164c4ceb44787c45dc7a54b7de0ea4/regex-2026.3.32-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:db976be51375bca900e008941639448d148c655c9545071965d0571ecc04f5d0", size = 289786, upload-time = "2026-03-28T21:46:31.415Z" }, - { url = "https://files.pythonhosted.org/packages/4a/cf/1955bb5567bc491bd63068e17f75ab0c9ff5e9d08466beec7e347f5e768d/regex-2026.3.32-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:66a5083c3ffe5a5a95f8281ea47a88072d4f24001d562d1d9d28d4cdc005fec5", size = 796431, upload-time = "2026-03-28T21:46:33.101Z" }, - { url = "https://files.pythonhosted.org/packages/27/8a/67fcbca511b792107540181ee0690df6de877bfbcb41b7ecae7028025ca5/regex-2026.3.32-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e83ce8008b48762be296f1401f19afd9ea29f3d035d1974e0cecb74e9afbd1df", size = 865785, upload-time = "2026-03-28T21:46:35.053Z" }, - { url = "https://files.pythonhosted.org/packages/c2/59/0677bc44f2c28305edcabc11933777b9ad34e9e8ded7ba573d24e4bc3ee7/regex-2026.3.32-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3aa21bad31db904e0b9055e12c8282df62d43169c4a9d2929407060066ebc74", size = 913593, upload-time = "2026-03-28T21:46:36.835Z" }, - { url = "https://files.pythonhosted.org/packages/0a/fe/661043d1c263b0d9d10c6ff4e9c9745f3df9641c62b51f96a3473638e7ce/regex-2026.3.32-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f54840bea73541652f1170dc63402a5b776fc851ad36a842da9e5163c1f504a0", size = 801512, upload-time = "2026-03-28T21:46:38.587Z" }, - { url = "https://files.pythonhosted.org/packages/ff/27/74c986061380e1811a46cf04cdf9c939db9f8c0e63953eddfe37ffd633ea/regex-2026.3.32-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:2ffbadc647325dd4e3118269bda93ded1eb5f5b0c3b7ba79a3da9fbd04f248e9", size = 776182, upload-time = "2026-03-28T21:46:40.69Z" }, - { url = "https://files.pythonhosted.org/packages/b6/c8/d833397b70cd1bacfcdc0a611f0e2c1f5b91fee8eedd88affcee770cbbb6/regex-2026.3.32-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:66d3126afe7eac41759cd5f0b3b246598086e88e70527c0d68c9e615b81771c4", size = 785837, upload-time = "2026-03-28T21:46:42.926Z" }, - { url = "https://files.pythonhosted.org/packages/e0/53/fa226b72989b5b93db6926fab5478115e085dfcf077e18d2cb386be0fd23/regex-2026.3.32-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:f785f44a44702dea89b28bce5bc82552490694ce4e144e21a4f0545e364d2150", size = 860612, upload-time = "2026-03-28T21:46:44.8Z" }, - { url = "https://files.pythonhosted.org/packages/04/28/bdd2fc0c055a1b15702bd4084829bbb6b06095f27990e5bee52b2898ea03/regex-2026.3.32-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:b7836aa13721dbdef658aebd11f60d00de633a95726521860fe1f6be75fa225a", size = 765285, upload-time = "2026-03-28T21:46:46.625Z" }, - { url = "https://files.pythonhosted.org/packages/b4/da/21f5e2a35a191b27e5a47cccb3914c99e139b49b1342d3f36e64e8cc60f7/regex-2026.3.32-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:5336b1506142eb0f23c96fb4a34b37c4fefd4fed2a7042069f3c8058efe17855", size = 851963, upload-time = "2026-03-28T21:46:48.341Z" }, - { url = "https://files.pythonhosted.org/packages/18/f4/04ed04ebf335a44083695c22772be6a42efa31900415555563acf02cb4de/regex-2026.3.32-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:b56993a7aeb4140c4770f4f7965c9e5af4f024457d06e23c01b0d47501cb18ed", size = 788332, upload-time = "2026-03-28T21:46:50.454Z" }, - { url = "https://files.pythonhosted.org/packages/21/25/5355908f479d0dc13d044f88270cdcabc8723efc12e4c2b19e5a94ff1a96/regex-2026.3.32-cp312-cp312-win32.whl", hash = "sha256:d363660f9ef8c734495598d2f3e527fb41f745c73159dc0d743402f049fb6836", size = 266847, upload-time = "2026-03-28T21:46:52.125Z" }, - { url = "https://files.pythonhosted.org/packages/00/e5/3be71c781a031db5df00735b613895ad5fdbf86c6e3bbea5fbbd7bfb5902/regex-2026.3.32-cp312-cp312-win_amd64.whl", hash = "sha256:c9f261ad3cd97257dc1d9355bfbaa7dd703e06574bffa0fa8fe1e31da915ee38", size = 278034, upload-time = "2026-03-28T21:46:54.096Z" }, - { url = "https://files.pythonhosted.org/packages/31/5f/27f1e0b1eea4faa99c66daca34130af20c44fae0237bbc98b87999dbc4a8/regex-2026.3.32-cp312-cp312-win_arm64.whl", hash = "sha256:89e50667e7e8c0e7903e4d644a2764fffe9a3a5d6578f72ab7a7b4205bf204b7", size = 270673, upload-time = "2026-03-28T21:46:56.046Z" }, +version = "2026.4.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cb/0e/3a246dbf05666918bd3664d9d787f84a9108f6f43cc953a077e4a7dfdb7e/regex-2026.4.4.tar.gz", hash = "sha256:e08270659717f6973523ce3afbafa53515c4dc5dcad637dc215b6fd50f689423", size = 416000, upload-time = "2026-04-03T20:56:28.155Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e0/7a/617356cbecdb452812a5d42f720d6d5096b360d4a4c1073af700ea140ad2/regex-2026.4.4-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:b4c36a85b00fadb85db9d9e90144af0a980e1a3d2ef9cd0f8a5bef88054657c6", size = 489415, upload-time = "2026-04-03T20:53:11.645Z" }, + { url = "https://files.pythonhosted.org/packages/20/e6/bf057227144d02e3ba758b66649e87531d744dda5f3254f48660f18ae9d8/regex-2026.4.4-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:dcb5453ecf9cd58b562967badd1edbf092b0588a3af9e32ee3d05c985077ce87", size = 291205, upload-time = "2026-04-03T20:53:13.289Z" }, + { url = "https://files.pythonhosted.org/packages/eb/3b/637181b787dd1a820ba1c712cee2b4144cd84a32dc776ca067b12b2d70c8/regex-2026.4.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:6aa809ed4dc3706cc38594d67e641601bd2f36d5555b2780ff074edfcb136cf8", size = 289225, upload-time = "2026-04-03T20:53:16.002Z" }, + { url = "https://files.pythonhosted.org/packages/05/21/bac05d806ed02cd4b39d9c8e5b5f9a2998c94c3a351b7792e80671fa5315/regex-2026.4.4-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:33424f5188a7db12958246a54f59a435b6cb62c5cf9c8d71f7cc49475a5fdada", size = 792434, upload-time = "2026-04-03T20:53:17.414Z" }, + { url = "https://files.pythonhosted.org/packages/d9/17/c65d1d8ae90b772d5758eb4014e1e011bb2db353fc4455432e6cc9100df7/regex-2026.4.4-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7d346fccdde28abba117cc9edc696b9518c3307fbfcb689e549d9b5979018c6d", size = 861730, upload-time = "2026-04-03T20:53:18.903Z" }, + { url = "https://files.pythonhosted.org/packages/ad/64/933321aa082a2c6ee2785f22776143ba89840189c20d3b6b1d12b6aae16b/regex-2026.4.4-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:415a994b536440f5011aa77e50a4274d15da3245e876e5c7f19da349caaedd87", size = 906495, upload-time = "2026-04-03T20:53:20.561Z" }, + { url = "https://files.pythonhosted.org/packages/01/ea/4c8d306e9c36ac22417336b1e02e7b358152c34dc379673f2d331143725f/regex-2026.4.4-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:21e5eb86179b4c67b5759d452ea7c48eb135cd93308e7a260aa489ed2eb423a4", size = 799810, upload-time = "2026-04-03T20:53:22.961Z" }, + { url = "https://files.pythonhosted.org/packages/29/ce/7605048f00e1379eba89d610c7d644d8f695dc9b26d3b6ecfa3132b872ff/regex-2026.4.4-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:312ec9dd1ae7d96abd8c5a36a552b2139931914407d26fba723f9e53c8186f86", size = 774242, upload-time = "2026-04-03T20:53:25.015Z" }, + { url = "https://files.pythonhosted.org/packages/e9/77/283e0d5023fde22cd9e86190d6d9beb21590a452b195ffe00274de470691/regex-2026.4.4-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:a0d2b28aa1354c7cd7f71b7658c4326f7facac106edd7f40eda984424229fd59", size = 781257, upload-time = "2026-04-03T20:53:26.918Z" }, + { url = "https://files.pythonhosted.org/packages/8b/fb/7f3b772be101373c8626ed34c5d727dcbb8abd42a7b1219bc25fd9a3cc04/regex-2026.4.4-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:349d7310eddff40429a099c08d995c6d4a4bfaf3ff40bd3b5e5cb5a5a3c7d453", size = 854490, upload-time = "2026-04-03T20:53:29.065Z" }, + { url = "https://files.pythonhosted.org/packages/85/30/56547b80f34f4dd2986e1cdd63b1712932f63b6c4ce2f79c50a6cd79d1c2/regex-2026.4.4-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:e7ab63e9fe45a9ec3417509e18116b367e89c9ceb6219222a3396fa30b147f80", size = 763544, upload-time = "2026-04-03T20:53:30.917Z" }, + { url = "https://files.pythonhosted.org/packages/ac/2f/ce060fdfea8eff34a8997603532e44cdb7d1f35e3bc253612a8707a90538/regex-2026.4.4-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:fe896e07a5a2462308297e515c0054e9ec2dd18dfdc9427b19900b37dfe6f40b", size = 844442, upload-time = "2026-04-03T20:53:32.463Z" }, + { url = "https://files.pythonhosted.org/packages/e5/44/810cb113096a1dacbe82789fbfab2823f79d19b7f1271acecb7009ba9b88/regex-2026.4.4-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:eb59c65069498dbae3c0ef07bbe224e1eaa079825a437fb47a479f0af11f774f", size = 789162, upload-time = "2026-04-03T20:53:34.039Z" }, + { url = "https://files.pythonhosted.org/packages/20/96/9647dd7f2ecf6d9ce1fb04dfdb66910d094e10d8fe53e9c15096d8aa0bd2/regex-2026.4.4-cp311-cp311-win32.whl", hash = "sha256:2a5d273181b560ef8397c8825f2b9d57013de744da9e8257b8467e5da8599351", size = 266227, upload-time = "2026-04-03T20:53:35.601Z" }, + { url = "https://files.pythonhosted.org/packages/33/80/74e13262460530c3097ff343a17de9a34d040a5dc4de9cf3a8241faab51c/regex-2026.4.4-cp311-cp311-win_amd64.whl", hash = "sha256:9542ccc1e689e752594309444081582f7be2fdb2df75acafea8a075108566735", size = 278399, upload-time = "2026-04-03T20:53:37.021Z" }, + { url = "https://files.pythonhosted.org/packages/1c/3c/39f19f47f19dcefa3403f09d13562ca1c0fd07ab54db2bc03148f3f6b46a/regex-2026.4.4-cp311-cp311-win_arm64.whl", hash = "sha256:b5f9fb784824a042be3455b53d0b112655686fdb7a91f88f095f3fee1e2a2a54", size = 270473, upload-time = "2026-04-03T20:53:38.633Z" }, + { url = "https://files.pythonhosted.org/packages/e5/28/b972a4d3df61e1d7bcf1b59fdb3cddef22f88b6be43f161bb41ebc0e4081/regex-2026.4.4-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:c07ab8794fa929e58d97a0e1796b8b76f70943fa39df225ac9964615cf1f9d52", size = 490434, upload-time = "2026-04-03T20:53:40.219Z" }, + { url = "https://files.pythonhosted.org/packages/84/20/30041446cf6dc3e0eab344fc62770e84c23b6b68a3b657821f9f80cb69b4/regex-2026.4.4-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:2c785939dc023a1ce4ec09599c032cc9933d258a998d16ca6f2b596c010940eb", size = 292061, upload-time = "2026-04-03T20:53:41.862Z" }, + { url = "https://files.pythonhosted.org/packages/62/c8/3baa06d75c98c46d4cc4262b71fd2edb9062b5665e868bca57859dadf93a/regex-2026.4.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1b1ce5c81c9114f1ce2f9288a51a8fd3aeea33a0cc440c415bf02da323aa0a76", size = 289628, upload-time = "2026-04-03T20:53:43.701Z" }, + { url = "https://files.pythonhosted.org/packages/31/87/3accf55634caad8c0acab23f5135ef7d4a21c39f28c55c816ae012931408/regex-2026.4.4-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:760ef21c17d8e6a4fe8cf406a97cf2806a4df93416ccc82fc98d25b1c20425be", size = 796651, upload-time = "2026-04-03T20:53:45.379Z" }, + { url = "https://files.pythonhosted.org/packages/f6/0c/aaa2c83f34efedbf06f61cb1942c25f6cf1ee3b200f832c4d05f28306c2e/regex-2026.4.4-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7088fcdcb604a4417c208e2169715800d28838fefd7455fbe40416231d1d47c1", size = 865916, upload-time = "2026-04-03T20:53:47.064Z" }, + { url = "https://files.pythonhosted.org/packages/d9/f6/8c6924c865124643e8f37823eca845dc27ac509b2ee58123685e71cd0279/regex-2026.4.4-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:07edca1ba687998968f7db5bc355288d0c6505caa7374f013d27356d93976d13", size = 912287, upload-time = "2026-04-03T20:53:49.422Z" }, + { url = "https://files.pythonhosted.org/packages/11/0e/a9f6f81013e0deaf559b25711623864970fe6a098314e374ccb1540a4152/regex-2026.4.4-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:993f657a7c1c6ec51b5e0ba97c9817d06b84ea5fa8d82e43b9405de0defdc2b9", size = 801126, upload-time = "2026-04-03T20:53:51.096Z" }, + { url = "https://files.pythonhosted.org/packages/71/61/3a0cc8af2dc0c8deb48e644dd2521f173f7e6513c6e195aad9aa8dd77ac5/regex-2026.4.4-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:2b69102a743e7569ebee67e634a69c4cb7e59d6fa2e1aa7d3bdbf3f61435f62d", size = 776788, upload-time = "2026-04-03T20:53:52.889Z" }, + { url = "https://files.pythonhosted.org/packages/64/0b/8bb9cbf21ef7dee58e49b0fdb066a7aded146c823202e16494a36777594f/regex-2026.4.4-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:6dac006c8b6dda72d86ea3d1333d45147de79a3a3f26f10c1cf9287ca4ca0ac3", size = 785184, upload-time = "2026-04-03T20:53:55.627Z" }, + { url = "https://files.pythonhosted.org/packages/99/c2/d3e80e8137b25ee06c92627de4e4d98b94830e02b3e6f81f3d2e3f504cf5/regex-2026.4.4-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:50a766ee2010d504554bfb5f578ed2e066898aa26411d57e6296230627cdefa0", size = 859913, upload-time = "2026-04-03T20:53:57.249Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e6/9d5d876157d969c804622456ef250017ac7a8f83e0e14f903b9e6df5ce95/regex-2026.4.4-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:9e2f5217648f68e3028c823df58663587c1507a5ba8419f4fdfc8a461be76043", size = 765732, upload-time = "2026-04-03T20:53:59.428Z" }, + { url = "https://files.pythonhosted.org/packages/82/80/b568935b4421388561c8ed42aff77247285d3ae3bb2a6ca22af63bae805e/regex-2026.4.4-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:39d8de85a08e32632974151ba59c6e9140646dcc36c80423962b1c5c0a92e244", size = 852152, upload-time = "2026-04-03T20:54:01.505Z" }, + { url = "https://files.pythonhosted.org/packages/39/29/f0f81217e21cd998245da047405366385d5c6072048038a3d33b37a79dc0/regex-2026.4.4-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:55d9304e0e7178dfb1e106c33edf834097ddf4a890e2f676f6c5118f84390f73", size = 789076, upload-time = "2026-04-03T20:54:03.323Z" }, + { url = "https://files.pythonhosted.org/packages/49/1d/1d957a61976ab9d4e767dd4f9d04b66cc0c41c5e36cf40e2d43688b5ae6f/regex-2026.4.4-cp312-cp312-win32.whl", hash = "sha256:04bb679bc0bde8a7bfb71e991493d47314e7b98380b083df2447cda4b6edb60f", size = 266700, upload-time = "2026-04-03T20:54:05.639Z" }, + { url = "https://files.pythonhosted.org/packages/c5/5c/bf575d396aeb58ea13b06ef2adf624f65b70fafef6950a80fc3da9cae3bc/regex-2026.4.4-cp312-cp312-win_amd64.whl", hash = "sha256:db0ac18435a40a2543dbb3d21e161a6c78e33e8159bd2e009343d224bb03bb1b", size = 277768, upload-time = "2026-04-03T20:54:07.312Z" }, + { url = "https://files.pythonhosted.org/packages/c9/27/049df16ec6a6828ccd72add3c7f54b4df029669bea8e9817df6fff58be90/regex-2026.4.4-cp312-cp312-win_arm64.whl", hash = "sha256:4ce255cc05c1947a12989c6db801c96461947adb7a59990f1360b5983fab4983", size = 270568, upload-time = "2026-04-03T20:54:09.484Z" }, ] [[package]] name = "requests" -version = "2.33.0" +version = "2.33.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "certifi" }, @@ -2522,9 +2528,9 @@ dependencies = [ { name = "idna" }, { name = "urllib3" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/34/64/8860370b167a9721e8956ae116825caff829224fbca0ca6e7bf8ddef8430/requests-2.33.0.tar.gz", hash = "sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652", size = 134232, upload-time = "2026-03-25T15:10:41.586Z" } +sdist = { url = "https://files.pythonhosted.org/packages/5f/a4/98b9c7c6428a668bf7e42ebb7c79d576a1c3c1e3ae2d47e674b468388871/requests-2.33.1.tar.gz", hash = "sha256:18817f8c57c6263968bc123d237e3b8b08ac046f5456bd1e307ee8f4250d3517", size = 134120, upload-time = "2026-03-30T16:09:15.531Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/56/5d/c814546c2333ceea4ba42262d8c4d55763003e767fa169adc693bd524478/requests-2.33.0-py3-none-any.whl", hash = "sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b", size = 65017, upload-time = "2026-03-25T15:10:40.382Z" }, + { url = "https://files.pythonhosted.org/packages/d7/8e/7540e8a2036f79a125c1d2ebadf69ed7901608859186c856fa0388ef4197/requests-2.33.1-py3-none-any.whl", hash = "sha256:4e6d1ef462f3626a1f0a0a9c42dd93c63bad33f9f1c1937509b8c5c8718ab56a", size = 64947, upload-time = "2026-03-30T16:09:13.83Z" }, ] [[package]] @@ -2561,8 +2567,9 @@ dependencies = [ { name = "uvicorn" }, ] -[package.optional-dependencies] +[package.dev-dependencies] dev = [ + { name = "ipykernel" }, { name = "jupyter" }, { name = "mypy" }, { name = "pre-commit" }, @@ -2571,38 +2578,34 @@ dev = [ { name = "ruff" }, ] -[package.dev-dependencies] -dev = [ - { name = "pre-commit" }, -] - [package.metadata] requires-dist = [ { name = "anthropic", specifier = ">=0.40.0" }, { name = "chromadb", specifier = ">=0.5.0" }, { name = "fastapi", specifier = ">=0.115.0" }, { name = "httpx", specifier = ">=0.27.0" }, - { name = "jupyter", marker = "extra == 'dev'", specifier = ">=1.0.0" }, - { name = "mypy", marker = "extra == 'dev'", specifier = ">=1.11.0" }, - { name = "pre-commit", marker = "extra == 'dev'", specifier = ">=3.8.0" }, { name = "pydantic", specifier = ">=2.0.0" }, { name = "pydantic-settings", specifier = ">=2.0.0" }, { name = "pymupdf", specifier = ">=1.24.0" }, { name = "pyprojroot", specifier = ">=0.3.0" }, { name = "pysqlite3-binary", marker = "sys_platform == 'linux'", specifier = ">=0.5.4" }, - { name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0.0" }, - { name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.24.0" }, { name = "python-dotenv", specifier = ">=1.0.0" }, { name = "python-telegram-bot", specifier = ">=21.0" }, { name = "rank-bm25", specifier = ">=0.2.2" }, - { name = "ruff", marker = "extra == 'dev'", specifier = ">=0.6.0" }, { name = "sentence-transformers", specifier = ">=3.0.0" }, { name = "uvicorn", specifier = ">=0.30.0" }, ] -provides-extras = ["dev"] [package.metadata.requires-dev] -dev = [{ name = "pre-commit", specifier = ">=4.5.1" }] +dev = [ + { name = "ipykernel", specifier = ">=6.0.0" }, + { name = "jupyter", specifier = ">=1.0.0" }, + { name = "mypy", specifier = ">=1.11.0" }, + { name = "pre-commit", specifier = ">=3.8.0" }, + { name = "pytest", specifier = ">=8.0.0" }, + { name = "pytest-asyncio", specifier = ">=1.3.0" }, + { name = "ruff", specifier = ">=0.6.0" }, +] [[package]] name = "rfc3339-validator" @@ -2639,15 +2642,15 @@ wheels = [ [[package]] name = "rich" -version = "14.3.3" +version = "15.0.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "markdown-it-py" }, { name = "pygments" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/b3/c6/f3b320c27991c46f43ee9d856302c70dc2d0fb2dba4842ff739d5f46b393/rich-14.3.3.tar.gz", hash = "sha256:b8daa0b9e4eef54dd8cf7c86c03713f53241884e814f4e2f5fb342fe520f639b", size = 230582, upload-time = "2026-02-19T17:23:12.474Z" } +sdist = { url = "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz", hash = "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36", size = 230680, upload-time = "2026-04-12T08:24:00.75Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/14/25/b208c5683343959b670dc001595f2f3737e051da617f66c31f7c4fa93abc/rich-14.3.3-py3-none-any.whl", hash = "sha256:793431c1f8619afa7d3b52b2cdec859562b950ea0d4b6b505397612db8d5362d", size = 310458, upload-time = "2026-02-19T17:23:13.732Z" }, + { url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" }, ] [[package]] @@ -2702,27 +2705,27 @@ wheels = [ [[package]] name = "ruff" -version = "0.15.8" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/14/b0/73cf7550861e2b4824950b8b52eebdcc5adc792a00c514406556c5b80817/ruff-0.15.8.tar.gz", hash = "sha256:995f11f63597ee362130d1d5a327a87cb6f3f5eae3094c620bcc632329a4d26e", size = 4610921, upload-time = "2026-03-26T18:39:38.675Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/4a/92/c445b0cd6da6e7ae51e954939cb69f97e008dbe750cfca89b8cedc081be7/ruff-0.15.8-py3-none-linux_armv6l.whl", hash = "sha256:cbe05adeba76d58162762d6b239c9056f1a15a55bd4b346cfd21e26cd6ad7bc7", size = 10527394, upload-time = "2026-03-26T18:39:41.566Z" }, - { url = "https://files.pythonhosted.org/packages/eb/92/f1c662784d149ad1414cae450b082cf736430c12ca78367f20f5ed569d65/ruff-0.15.8-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:d3e3d0b6ba8dca1b7ef9ab80a28e840a20070c4b62e56d675c24f366ef330570", size = 10905693, upload-time = "2026-03-26T18:39:30.364Z" }, - { url = "https://files.pythonhosted.org/packages/ca/f2/7a631a8af6d88bcef997eb1bf87cc3da158294c57044aafd3e17030613de/ruff-0.15.8-py3-none-macosx_11_0_arm64.whl", hash = "sha256:6ee3ae5c65a42f273f126686353f2e08ff29927b7b7e203b711514370d500de3", size = 10323044, upload-time = "2026-03-26T18:39:33.37Z" }, - { url = "https://files.pythonhosted.org/packages/67/18/1bf38e20914a05e72ef3b9569b1d5c70a7ef26cd188d69e9ca8ef588d5bf/ruff-0.15.8-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fdce027ada77baa448077ccc6ebb2fa9c3c62fd110d8659d601cf2f475858d94", size = 10629135, upload-time = "2026-03-26T18:39:44.142Z" }, - { url = "https://files.pythonhosted.org/packages/d2/e9/138c150ff9af60556121623d41aba18b7b57d95ac032e177b6a53789d279/ruff-0.15.8-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:12e617fc01a95e5821648a6df341d80456bd627bfab8a829f7cfc26a14a4b4a3", size = 10348041, upload-time = "2026-03-26T18:39:52.178Z" }, - { url = "https://files.pythonhosted.org/packages/02/f1/5bfb9298d9c323f842c5ddeb85f1f10ef51516ac7a34ba446c9347d898df/ruff-0.15.8-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:432701303b26416d22ba696c39f2c6f12499b89093b61360abc34bcc9bf07762", size = 11121987, upload-time = "2026-03-26T18:39:55.195Z" }, - { url = "https://files.pythonhosted.org/packages/10/11/6da2e538704e753c04e8d86b1fc55712fdbdcc266af1a1ece7a51fff0d10/ruff-0.15.8-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d910ae974b7a06a33a057cb87d2a10792a3b2b3b35e33d2699fdf63ec8f6b17a", size = 11951057, upload-time = "2026-03-26T18:39:19.18Z" }, - { url = "https://files.pythonhosted.org/packages/83/f0/c9208c5fd5101bf87002fed774ff25a96eea313d305f1e5d5744698dc314/ruff-0.15.8-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2033f963c43949d51e6fdccd3946633c6b37c484f5f98c3035f49c27395a8ab8", size = 11464613, upload-time = "2026-03-26T18:40:06.301Z" }, - { url = "https://files.pythonhosted.org/packages/f8/22/d7f2fabdba4fae9f3b570e5605d5eb4500dcb7b770d3217dca4428484b17/ruff-0.15.8-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0f29b989a55572fb885b77464cf24af05500806ab4edf9a0fd8977f9759d85b1", size = 11257557, upload-time = "2026-03-26T18:39:57.972Z" }, - { url = "https://files.pythonhosted.org/packages/71/8c/382a9620038cf6906446b23ce8632ab8c0811b8f9d3e764f58bedd0c9a6f/ruff-0.15.8-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:ac51d486bf457cdc985a412fb1801b2dfd1bd8838372fc55de64b1510eff4bec", size = 11169440, upload-time = "2026-03-26T18:39:22.205Z" }, - { url = "https://files.pythonhosted.org/packages/4d/0d/0994c802a7eaaf99380085e4e40c845f8e32a562e20a38ec06174b52ef24/ruff-0.15.8-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:c9861eb959edab053c10ad62c278835ee69ca527b6dcd72b47d5c1e5648964f6", size = 10605963, upload-time = "2026-03-26T18:39:46.682Z" }, - { url = "https://files.pythonhosted.org/packages/19/aa/d624b86f5b0aad7cef6bbf9cd47a6a02dfdc4f72c92a337d724e39c9d14b/ruff-0.15.8-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:8d9a5b8ea13f26ae90838afc33f91b547e61b794865374f114f349e9036835fb", size = 10357484, upload-time = "2026-03-26T18:39:49.176Z" }, - { url = "https://files.pythonhosted.org/packages/35/c3/e0b7835d23001f7d999f3895c6b569927c4d39912286897f625736e1fd04/ruff-0.15.8-py3-none-musllinux_1_2_i686.whl", hash = "sha256:c2a33a529fb3cbc23a7124b5c6ff121e4d6228029cba374777bd7649cc8598b8", size = 10830426, upload-time = "2026-03-26T18:40:03.702Z" }, - { url = "https://files.pythonhosted.org/packages/f0/51/ab20b322f637b369383adc341d761eaaa0f0203d6b9a7421cd6e783d81b9/ruff-0.15.8-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:75e5cd06b1cf3f47a3996cfc999226b19aa92e7cce682dcd62f80d7035f98f49", size = 11345125, upload-time = "2026-03-26T18:39:27.799Z" }, - { url = "https://files.pythonhosted.org/packages/37/e6/90b2b33419f59d0f2c4c8a48a4b74b460709a557e8e0064cf33ad894f983/ruff-0.15.8-py3-none-win32.whl", hash = "sha256:bc1f0a51254ba21767bfa9a8b5013ca8149dcf38092e6a9eb704d876de94dc34", size = 10571959, upload-time = "2026-03-26T18:39:36.117Z" }, - { url = "https://files.pythonhosted.org/packages/1f/a2/ef467cb77099062317154c63f234b8a7baf7cb690b99af760c5b68b9ee7f/ruff-0.15.8-py3-none-win_amd64.whl", hash = "sha256:04f79eff02a72db209d47d665ba7ebcad609d8918a134f86cb13dd132159fc89", size = 11743893, upload-time = "2026-03-26T18:39:25.01Z" }, - { url = "https://files.pythonhosted.org/packages/15/e2/77be4fff062fa78d9b2a4dea85d14785dac5f1d0c1fb58ed52331f0ebe28/ruff-0.15.8-py3-none-win_arm64.whl", hash = "sha256:cf891fa8e3bb430c0e7fac93851a5978fc99c8fa2c053b57b118972866f8e5f2", size = 11048175, upload-time = "2026-03-26T18:40:01.06Z" }, +version = "0.15.11" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e4/8d/192f3d7103816158dfd5ea50d098ef2aec19194e6cbccd4b3485bdb2eb2d/ruff-0.15.11.tar.gz", hash = "sha256:f092b21708bf0e7437ce9ada249dfe688ff9a0954fc94abab05dcea7dcd29c33", size = 4637264, upload-time = "2026-04-16T18:46:26.58Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/02/1e/6aca3427f751295ab011828e15e9bf452200ac74484f1db4be0197b8170b/ruff-0.15.11-py3-none-linux_armv6l.whl", hash = "sha256:e927cfff503135c558eb581a0c9792264aae9507904eb27809cdcff2f2c847b7", size = 10607943, upload-time = "2026-04-16T18:46:05.967Z" }, + { url = "https://files.pythonhosted.org/packages/e7/26/1341c262e74f36d4e84f3d6f4df0ac68cd53331a66bfc5080daa17c84c0b/ruff-0.15.11-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:7a1b5b2938d8f890b76084d4fa843604d787a912541eae85fd7e233398bbb73e", size = 10988592, upload-time = "2026-04-16T18:46:00.742Z" }, + { url = "https://files.pythonhosted.org/packages/03/71/850b1d6ffa9564fbb6740429bad53df1094082fe515c8c1e74b6d8d05f18/ruff-0.15.11-py3-none-macosx_11_0_arm64.whl", hash = "sha256:d4176f3d194afbdaee6e41b9ccb1a2c287dba8700047df474abfbe773825d1cb", size = 10338501, upload-time = "2026-04-16T18:46:03.723Z" }, + { url = "https://files.pythonhosted.org/packages/f2/11/cc1284d3e298c45a817a6aadb6c3e1d70b45c9b36d8d9cce3387b495a03a/ruff-0.15.11-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3b17c886fb88203ced3afe7f14e8d5ae96e9d2f4ccc0ee66aa19f2c2675a27e4", size = 10670693, upload-time = "2026-04-16T18:46:41.941Z" }, + { url = "https://files.pythonhosted.org/packages/ce/9e/f8288b034ab72b371513c13f9a41d9ba3effac54e24bfb467b007daee2ca/ruff-0.15.11-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:49fafa220220afe7758a487b048de4c8f9f767f37dfefad46b9dd06759d003eb", size = 10416177, upload-time = "2026-04-16T18:46:21.717Z" }, + { url = "https://files.pythonhosted.org/packages/85/71/504d79abfd3d92532ba6bbe3d1c19fada03e494332a59e37c7c2dabae427/ruff-0.15.11-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f2ab8427e74a00d93b8bda1307b1e60970d40f304af38bccb218e056c220120d", size = 11221886, upload-time = "2026-04-16T18:46:15.086Z" }, + { url = "https://files.pythonhosted.org/packages/43/5a/947e6ab7a5ad603d65b474be15a4cbc6d29832db5d762cd142e4e3a74164/ruff-0.15.11-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:195072c0c8e1fc8f940652073df082e37a5d9cb43b4ab1e4d0566ab8977a13b7", size = 12075183, upload-time = "2026-04-16T18:46:07.944Z" }, + { url = "https://files.pythonhosted.org/packages/9f/a1/0b7bb6268775fdd3a0818aee8efd8f5b4e231d24dd4d528ced2534023182/ruff-0.15.11-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a3a0996d486af3920dec930a2e7daed4847dfc12649b537a9335585ada163e9e", size = 11516575, upload-time = "2026-04-16T18:46:31.687Z" }, + { url = "https://files.pythonhosted.org/packages/30/c3/bb5168fc4d233cc06e95f482770d0f3c87945a0cd9f614b90ea8dc2f2833/ruff-0.15.11-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1bef2cb556d509259f1fe440bb9cd33c756222cf0a7afe90d15edf0866702431", size = 11306537, upload-time = "2026-04-16T18:46:36.988Z" }, + { url = "https://files.pythonhosted.org/packages/e4/92/4cfae6441f3967317946f3b788136eecf093729b94d6561f963ed810c82e/ruff-0.15.11-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:030d921a836d7d4a12cf6e8d984a88b66094ccb0e0f17ddd55067c331191bf19", size = 11296813, upload-time = "2026-04-16T18:46:24.182Z" }, + { url = "https://files.pythonhosted.org/packages/43/26/972784c5dde8313acde8ac71ba8ac65475b85db4a2352a76c9934361f9bc/ruff-0.15.11-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:0e783b599b4577788dbbb66b9addcef87e9a8832f4ce0c19e34bf55543a2f890", size = 10633136, upload-time = "2026-04-16T18:46:39.802Z" }, + { url = "https://files.pythonhosted.org/packages/5b/53/3985a4f185020c2f367f2e08a103032e12564829742a1b417980ce1514a0/ruff-0.15.11-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:ae90592246625ba4a34349d68ec28d4400d75182b71baa196ddb9f82db025ef5", size = 10424701, upload-time = "2026-04-16T18:46:10.381Z" }, + { url = "https://files.pythonhosted.org/packages/d3/57/bf0dfb32241b56c83bb663a826133da4bf17f682ba8c096973065f6e6a68/ruff-0.15.11-py3-none-musllinux_1_2_i686.whl", hash = "sha256:1f111d62e3c983ed20e0ca2e800f8d77433a5b1161947df99a5c2a3fb60514f0", size = 10873887, upload-time = "2026-04-16T18:46:29.157Z" }, + { url = "https://files.pythonhosted.org/packages/02/05/e48076b2a57dc33ee8c7a957296f97c744ca891a8ffb4ffb1aaa3b3f517d/ruff-0.15.11-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:06f483d6646f59eaffba9ae30956370d3a886625f511a3108994000480621d1c", size = 11404316, upload-time = "2026-04-16T18:46:19.462Z" }, + { url = "https://files.pythonhosted.org/packages/88/27/0195d15fe7a897cbcba0904792c4b7c9fdd958456c3a17d2ea6093716a9a/ruff-0.15.11-py3-none-win32.whl", hash = "sha256:476a2aa56b7da0b73a3ee80b6b2f0e19cce544245479adde7baa65466664d5f3", size = 10655535, upload-time = "2026-04-16T18:46:12.47Z" }, + { url = "https://files.pythonhosted.org/packages/3a/5e/c927b325bd4c1d3620211a4b96f47864633199feed60fa936025ab27e090/ruff-0.15.11-py3-none-win_amd64.whl", hash = "sha256:8b6756d88d7e234fb0c98c91511aae3cd519d5e3ed271cae31b20f39cb2a12a3", size = 11779692, upload-time = "2026-04-16T18:46:17.268Z" }, + { url = "https://files.pythonhosted.org/packages/63/b6/aeadee5443e49baa2facd51131159fd6301cc4ccfc1541e4df7b021c37dd/ruff-0.15.11-py3-none-win_arm64.whl", hash = "sha256:063fed18cc1bbe0ee7393957284a6fe8b588c6a406a285af3ee3f46da2391ee4", size = 11032614, upload-time = "2026-04-16T18:46:34.487Z" }, ] [[package]] @@ -2815,7 +2818,7 @@ wheels = [ [[package]] name = "sentence-transformers" -version = "5.3.0" +version = "5.4.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "huggingface-hub" }, @@ -2827,9 +2830,9 @@ dependencies = [ { name = "transformers" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/fe/26/448453925b6ce0c29d8b54327caa71ee4835511aef02070467402273079c/sentence_transformers-5.3.0.tar.gz", hash = "sha256:414a0a881f53a4df0e6cbace75f823bfcb6b94d674c42a384b498959b7c065e2", size = 403330, upload-time = "2026-03-12T14:53:40.778Z" } +sdist = { url = "https://files.pythonhosted.org/packages/4d/68/7f98c221940ce783b492ad6140384daf2e2918cd7175009d6a362c22b9ee/sentence_transformers-5.4.1.tar.gz", hash = "sha256:436bcb1182a0ff42a8fb2b1c43498a70d0a75b688d182f2cd0d1dd115af61ddc", size = 428910, upload-time = "2026-04-14T13:34:59.006Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/e2/9c/2fa7224058cad8df68d84bafee21716f30892cecc7ad1ad73bde61d23754/sentence_transformers-5.3.0-py3-none-any.whl", hash = "sha256:dca6b98db790274a68185d27a65801b58b4caf653a4e556b5f62827509347c7d", size = 512390, upload-time = "2026-03-12T14:53:39.035Z" }, + { url = "https://files.pythonhosted.org/packages/c5/d9/3a9b6f2ccdedc9dc00fe37b2fc58f58f8efbff44565cf4bf39d8568bb13a/sentence_transformers-5.4.1-py3-none-any.whl", hash = "sha256:a6d640fc363849b63affb8e140e9d328feabab86f83d58ac3e16b1c28140b790", size = 571311, upload-time = "2026-04-14T13:34:57.731Z" }, ] [[package]] @@ -3057,7 +3060,7 @@ wheels = [ [[package]] name = "transformers" -version = "5.4.0" +version = "5.5.4" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "huggingface-hub" }, @@ -3070,9 +3073,9 @@ dependencies = [ { name = "tqdm" }, { name = "typer" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/0b/4c/42a8e1c7bbe668d8e073941ec3205263afb1cd02683fa5a8a75e615fdfbe/transformers-5.4.0.tar.gz", hash = "sha256:cb34ca89dce345ae3224b290346b9c0fa9694b951d54f3ed16334a4b1bfe3d04", size = 8152836, upload-time = "2026-03-27T00:24:24.692Z" } +sdist = { url = "https://files.pythonhosted.org/packages/a5/1e/1e244ab2ab50a863e6b52cc55761910567fa532b69a6740f6e99c5fdbd98/transformers-5.5.4.tar.gz", hash = "sha256:2e67cadba81fc7608cc07c4dd54f524820bc3d95b1cabd0ef3db7733c4f8b82e", size = 8227649, upload-time = "2026-04-13T16:55:55.181Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/0b/a0/0a87883e564e364baab32adcacb4bec2e200b28a568423c8cf7fde316461/transformers-5.4.0-py3-none-any.whl", hash = "sha256:9fbe50602d2a4e6d0aa8a35a605433dfac72d595ee2192eae192590a6cc2df86", size = 10105556, upload-time = "2026-03-27T00:24:21.735Z" }, + { url = "https://files.pythonhosted.org/packages/29/fb/162a66789c65e5afa3b051309240c26bf37fbc8fea285b4546ae747995a2/transformers-5.5.4-py3-none-any.whl", hash = "sha256:0bd6281b82966fe5a7a16f553ea517a9db1dee6284d7cb224dfd88fc0dd1c167", size = 10236696, upload-time = "2026-04-13T16:55:51.497Z" }, ] [[package]] @@ -3124,11 +3127,11 @@ wheels = [ [[package]] name = "tzdata" -version = "2025.3" +version = "2026.1" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/5e/a7/c202b344c5ca7daf398f3b8a477eeb205cf3b6f32e7ec3a6bac0629ca975/tzdata-2025.3.tar.gz", hash = "sha256:de39c2ca5dc7b0344f2eba86f49d614019d29f060fc4ebc8a417896a620b56a7", size = 196772, upload-time = "2025-12-13T17:45:35.667Z" } +sdist = { url = "https://files.pythonhosted.org/packages/19/f5/cd531b2d15a671a40c0f66cf06bc3570a12cd56eef98960068ebbad1bf5a/tzdata-2026.1.tar.gz", hash = "sha256:67658a1903c75917309e753fdc349ac0efd8c27db7a0cb406a25be4840f87f98", size = 197639, upload-time = "2026-04-03T11:25:22.002Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/c7/b0/003792df09decd6849a5e39c28b513c06e84436a54440380862b5aeff25d/tzdata-2025.3-py2.py3-none-any.whl", hash = "sha256:06a47e5700f3081aab02b2e513160914ff0694bce9947d6b76ebd6bf57cfc5d1", size = 348521, upload-time = "2025-12-13T17:45:33.889Z" }, + { url = "https://files.pythonhosted.org/packages/b0/70/d460bd685a170790ec89317e9bd33047988e4bce507b831f5db771e142de/tzdata-2026.1-py2.py3-none-any.whl", hash = "sha256:4b1d2be7ac37ceafd7327b961aa3a54e467efbdb563a23655fbfe0d39cfc42a9", size = 348952, upload-time = "2026-04-03T11:25:20.313Z" }, ] [[package]] @@ -3151,15 +3154,15 @@ wheels = [ [[package]] name = "uvicorn" -version = "0.42.0" +version = "0.44.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "click" }, { name = "h11" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/e3/ad/4a96c425be6fb67e0621e62d86c402b4a17ab2be7f7c055d9bd2f638b9e2/uvicorn-0.42.0.tar.gz", hash = "sha256:9b1f190ce15a2dd22e7758651d9b6d12df09a13d51ba5bf4fc33c383a48e1775", size = 85393, upload-time = "2026-03-16T06:19:50.077Z" } +sdist = { url = "https://files.pythonhosted.org/packages/5e/da/6eee1ff8b6cbeed47eeb5229749168e81eb4b7b999a1a15a7176e51410c9/uvicorn-0.44.0.tar.gz", hash = "sha256:6c942071b68f07e178264b9152f1f16dfac5da85880c4ce06366a96d70d4f31e", size = 86947, upload-time = "2026-04-06T09:23:22.826Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/0a/89/f8827ccff89c1586027a105e5630ff6139a64da2515e24dafe860bd9ae4d/uvicorn-0.42.0-py3-none-any.whl", hash = "sha256:96c30f5c7abe6f74ae8900a70e92b85ad6613b745d4879eb9b16ccad15645359", size = 68830, upload-time = "2026-03-16T06:19:48.325Z" }, + { url = "https://files.pythonhosted.org/packages/b7/23/a5bbd9600dd607411fa644c06ff4951bec3a4d82c4b852374024359c19c0/uvicorn-0.44.0-py3-none-any.whl", hash = "sha256:ce937c99a2cc70279556967274414c087888e8cec9f9c94644dfca11bd3ced89", size = 69425, upload-time = "2026-04-06T09:23:21.524Z" }, ] [package.optional-dependencies] @@ -3195,7 +3198,7 @@ wheels = [ [[package]] name = "virtualenv" -version = "21.2.0" +version = "21.2.4" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "distlib" }, @@ -3203,9 +3206,9 @@ dependencies = [ { name = "platformdirs" }, { name = "python-discovery" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/aa/92/58199fe10049f9703c2666e809c4f686c54ef0a68b0f6afccf518c0b1eb9/virtualenv-21.2.0.tar.gz", hash = "sha256:1720dc3a62ef5b443092e3f499228599045d7fea4c79199770499df8becf9098", size = 5840618, upload-time = "2026-03-09T17:24:38.013Z" } +sdist = { url = "https://files.pythonhosted.org/packages/0c/98/3a7e644e19cb26133488caff231be390579860bbbb3da35913c49a1d0a46/virtualenv-21.2.4.tar.gz", hash = "sha256:b294ef68192638004d72524ce7ef303e9d0cf5a44c95ce2e54a7500a6381cada", size = 5850742, upload-time = "2026-04-14T22:15:31.438Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/c6/59/7d02447a55b2e55755011a647479041bc92a82e143f96a8195cb33bd0a1c/virtualenv-21.2.0-py3-none-any.whl", hash = "sha256:1bd755b504931164a5a496d217c014d098426cddc79363ad66ac78125f9d908f", size = 5825084, upload-time = "2026-03-09T17:24:35.378Z" }, + { url = "https://files.pythonhosted.org/packages/27/8d/edd0bd910ff803c308ee9a6b7778621af0d10252219ad9f19ef4d4982a61/virtualenv-21.2.4-py3-none-any.whl", hash = "sha256:29d21e941795206138d0f22f4e45ff7050e5da6c6472299fb7103318763861ac", size = 5831232, upload-time = "2026-04-14T22:15:29.342Z" }, ] [[package]] @@ -3328,9 +3331,9 @@ wheels = [ [[package]] name = "zipp" -version = "3.23.0" +version = "3.23.1" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/e3/02/0f2892c661036d50ede074e376733dca2ae7c6eb617489437771209d4180/zipp-3.23.0.tar.gz", hash = "sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166", size = 25547, upload-time = "2025-06-08T17:06:39.4Z" } +sdist = { url = "https://files.pythonhosted.org/packages/30/21/093488dfc7cc8964ded15ab726fad40f25fd3d788fd741cc1c5a17d78ee8/zipp-3.23.1.tar.gz", hash = "sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110", size = 25965, upload-time = "2026-04-13T23:21:46.6Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/2e/54/647ade08bf0db230bfea292f893923872fd20be6ac6f53b2b936ba839d75/zipp-3.23.0-py3-none-any.whl", hash = "sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e", size = 10276, upload-time = "2025-06-08T17:06:38.034Z" }, + { url = "https://files.pythonhosted.org/packages/08/8a/0861bec20485572fbddf3dfba2910e38fe249796cb73ecdeb74e07eeb8d3/zipp-3.23.1-py3-none-any.whl", hash = "sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc", size = 10378, upload-time = "2026-04-13T23:21:45.386Z" }, ] From 52e8ddce67204d4fc82b62a1eb5026f4e3dc3353 Mon Sep 17 00:00:00 2001 From: Mario Date: Sat, 18 Apr 2026 10:54:05 -0500 Subject: [PATCH 25/55] feat(eval): add evaluation dataset with 20 questions and run_eval script --- data/samples/eval_dataset.json | 102 +++ data/samples/labels.json | 143 ---- notebooks/003-jmmz-ingestion_service.ipynb | 57 ++ notebooks/006-jmmz-eval-retrieval.ipynb | 797 +++++++++++++++++++++ scripts/eval_retrieval.py | 41 ++ 5 files changed, 997 insertions(+), 143 deletions(-) create mode 100644 data/samples/eval_dataset.json delete mode 100644 data/samples/labels.json create mode 100644 notebooks/006-jmmz-eval-retrieval.ipynb create mode 100644 scripts/eval_retrieval.py diff --git a/data/samples/eval_dataset.json b/data/samples/eval_dataset.json new file mode 100644 index 0000000..0a8bfc5 --- /dev/null +++ b/data/samples/eval_dataset.json @@ -0,0 +1,102 @@ +[ + { + "question": "What are the three coupled dimensions of externalization in LLM agents as described by Zhou et al. (2026)?", + "reference_answer": "The three coupled dimensions are memory (externalized state), skills (externalized procedural expertise), and protocols (externalized interaction structure).", + "source_paper": "chenyu_zhou_2026.pdf" + }, + { + "question": "What is the primary function of the 'harness' in an externalized agent architecture?", + "reference_answer": "The harness is the engineering layer that coordinates memory, skills, and protocols into governed execution, providing the orchestration logic, constraints, observability, and feedback loops necessary for practical agency.", + "source_paper": "chenyu_zhou_2026.pdf" + }, + { + "question": "How does externalized memory transform the cognitive task for a Large Language Model?", + "reference_answer": "Externalization transforms a difficult internal recall problem (regenerating knowledge from latent weights) into an external recognition and retrieval problem, where the agent identifies relevant information surfaced from a persistent store.", + "source_paper": "chenyu_zhou_2026.pdf" + }, + { + "question": "In the context of skill externalization, what are the three essential components of 'procedural expertise'?", + "reference_answer": "Procedural expertise consists of operational procedures (the task skeleton), decision heuristics (rules for branching points), and normative constraints (safety and compliance boundaries).", + "source_paper": "chenyu_zhou_2026.pdf" + }, + { + "question": "What is the definition of 'cognitive flow' within the framework proposed by Dissanayake and Nanayakkara (2025)?", + "reference_answer": "Cognitive flow is defined as an optimal psychological state of deep focus and intrinsic motivation that occurs when the challenge of a task is perfectly balanced with an individual's skill level.", + "source_paper": "dinithi_dissanayake_2025.pdf" + }, + { + "question": "What are the three key contextual factors that determine the effectiveness of AI interventions for reasoning support?", + "reference_answer": "The three factors are the type of intervention (e.g., direct feedback vs. Socratic questioning), the timing of the intervention (knowing when the user is 'stuck'), and the scale or magnitude of the intervention.", + "source_paper": "dinithi_dissanayake_2025.pdf" + }, + { + "question": "How can AI systems leverage multimodal cues to infer a user's cognitive load in real-time?", + "reference_answer": "AI systems can monitor behavioral cues such as gaze behavior (anticipation patterns), typing hesitation, interaction speed, and physiological signals to dynamically adjust cognitive support.", + "source_paper": "dinithi_dissanayake_2025.pdf" + }, + { + "question": "What is the core objective of the ILR (Interactive Learning for LLM Reasoning) framework?", + "reference_answer": "The core objective of ILR is to investigate whether multi-agent interaction during training can enhance an LLM's independent problem-solving capacity during inference, effectively internalizing insights from peer interaction.", + "source_paper": "hehai_lin_2025.pdf" + }, + { + "question": "Describe the three sequential stages of the 'Idea3' framework for agent communication.", + "reference_answer": "The Idea3 framework consists of Idea Sharing (proposing initial solutions), Idea Analysis (critically evaluating peer contributions), and Idea Fusion (synthesizing insights into a refined final answer).", + "source_paper": "hehai_lin_2025.pdf" + }, + { + "question": "How does the ILR system decide between cooperation and competition strategies during training?", + "reference_answer": "The system utilizes Item Response Theory (IRT) to estimate the probability of the model solving a question independently based on its capability and the question's difficulty; cooperation is selected if the probability is low (Pq < 0.5), while competition is chosen otherwise.", + "source_paper": "hehai_lin_2025.pdf" + }, + { + "question": "What is 'Perception Calibration' in the ILR framework and how is it implemented?", + "reference_answer": "Perception Calibration is an automated mechanism that integrates the reward distribution characteristics (max, min, and average scores) of one LLM into another LLM's reward function using Group Relative Policy Optimization (GRPO) to strengthen interaction cohesion.", + "source_paper": "hehai_lin_2025.pdf" + }, + { + "question": "How does the DeepSeek-R1-1.5B model compare to significantly larger models like CodeLlama-13B as a discriminator for SQL tasks?", + "reference_answer": "Despite having significantly fewer parameters, DeepSeek-R1-1.5B outranks CodeLlama-13B as a discriminator, achieving higher execution accuracy and much higher classification F1 scores in text-to-SQL tasks.", + "source_paper": "md_fahim_anjum_2025.pdf" + }, + { + "question": "What happens to the discrimination performance of reasoning models when the test-time compute budget is increased beyond 1024 tokens?", + "reference_answer": "Increasing the token budget beyond this threshold yields diminishing returns (less than 0.4% gain), and longer outputs tend to become more repetitive and lexically redundant rather than more insightful.", + "source_paper": "md_fahim_anjum_2025.pdf" + }, + { + "question": "Is the reasoning model DeepSeek-R1 more effective as a generator or a discriminator in SQL parsing?", + "reference_answer": "DeepSeek-R1 is far more effective as a discriminator; it performs poorly as a generator, where even smaller non-reasoning models like TinyLlama-1.1B deliver higher-quality SQL outputs.", + "source_paper": "md_fahim_anjum_2025.pdf" + }, + { + "question": "What novel method was proposed to extract soft scores from reasoning models for fine-grained ranking?", + "reference_answer": "The method prompts the model to output a final answer in a specific JSON format with the key 'correct', identifies the logits for the values 'true' or 'false', and normalizes them via a softmax function to obtain a probability score.", + "source_paper": "md_fahim_anjum_2025.pdf" + }, + { + "question": "How is 'analogical reasoning' defined by Musker et al. (2024)?", + "reference_answer": "Analogical reasoning is defined as the capacity to identify and map structural relationships between different domains (source and target) to transfer knowledge and recognize abstract patterns.", + "source_paper": "sam_musker_2024.pdf" + }, + { + "question": "What is 'flexible re-representation' and why is it essential for solving non-trivial analogies?", + "reference_answer": "Flexible re-representation is the ability to dynamically restructure how concepts are encoded based on task-relevant features; it is essential because real-world concepts have many attributes, and the system must identify which ones are relevant to a specific mapping.", + "source_paper": "sam_musker_2024.pdf" + }, + { + "question": "How did human subjects and LLMs differ in their response to misleading semantic structure in the Study 1 'Randoms' condition?", + "reference_answer": "Humans were able to ignore irrelevant words and switch to a strategy relying only on the target domain patterns, whereas LLMs appeared distracted by the random lexical items and showed significant performance drops.", + "source_paper": "sam_musker_2024.pdf" + }, + { + "question": "What specific limitation did GPT-4 demonstrate in the numeric conditions of the Semantic Content experiment?", + "reference_answer": "GPT-4 failed to correctly relate the number of characters in a response to the numeric property of the object (e.g., number of wheels or legs), which is attributed to a deficiency in counting or numeric reasoning rather than analogical reasoning itself.", + "source_paper": "sam_musker_2024.pdf" + }, + { + "question": "How does the analogical reasoning performance of Claude 3 Opus compare to human performance across the studied conditions?", + "reference_answer": "Claude 3 Opus demonstrated robust performance that matched human levels across all conditions of the Semantic Content experiment and exhibited more flexibility than earlier models like GPT-4 in handling complex mapping tasks.", + "source_paper": "sam_musker_2024.pdf" + } +] diff --git a/data/samples/labels.json b/data/samples/labels.json deleted file mode 100644 index b6708fa..0000000 --- a/data/samples/labels.json +++ /dev/null @@ -1,143 +0,0 @@ -{ -"paper": "Attention Is All You Need (1706.03762v7.pdf)", -"preguntas": [ - { - "pregunta": "¿Qué arquitectura de red propone este estudio que abandona por completo la recurrencia y las convoluciones?", - "respuesta": "Propone el Transformer, una arquitectura basada únicamente en mecanismos de atención [1, 2]." - }, - { - "pregunta": "¿Cómo se define la función de atención 'Scaled Dot-Product Attention' utilizada en el modelo?", - "respuesta": "Es una función que calcula el producto punto de la consulta (query) con todas las claves (keys), divide cada uno por la raíz cuadrada de la dimensión de la clave (dk), y aplica una función softmax para obtener los pesos de los valores [3, 4]." - } -] -}, -{ -"paper": "Fine-tuning Causal LLMs for Text Classification (2512.12677v1.pdf)", -"preguntas": [ - { - "pregunta": "¿Cuáles son los dos enfoques principales comparados para el ajuste fino de modelos de lenguaje causal en tareas de clasificación?", - "respuesta": "El enfoque basado en embeddings (añadir un cabezal de clasificación sobre el embedding del token final) y el enfoque basado en instrucciones (formatear la tarea como prompt -> respuesta) [5, 6]." - }, - { - "pregunta": "¿Qué técnicas se combinan para permitir el ajuste fino de modelos de hasta 8B parámetros en una sola GPU?", - "respuesta": "Se combina la cuantificación del modelo de 4 bits con la Adaptación de Bajo Rango (LoRA), técnica conocida como QLoRA [5, 7]." - } -] -}, -{ -"paper": "Constructing Multi-label Hierarchical Classification Models for MITRE ATT&CK (2601.14556v1.pdf)", -"preguntas": [ - { - "pregunta": "¿Qué precisión alcanzó el enfoque jerárquico multietiqueta propuesto a nivel de táctica de ciberseguridad?", - "respuesta": "Alcanzó una precisión aproximada del 94% a nivel de táctica [8, 9]." - }, - { - "pregunta": "¿Cómo superó el modelo baseline de descenso de gradiente estocástico (SGD) al modelo GPT-4o en el estudio piloto?", - "respuesta": "El modelo SGD alcanzó una precisión de 0.8195 frente al 0.59 obtenido por GPT-4o en la clasificación de tácticas a partir de oraciones de ciberinteligencia [10, 11]." - } -] -}, -{ -"paper": "VOICEAGENTRAG (2603.02206v2.pdf)", -"preguntas": [ - { - "pregunta": "¿Qué agentes componen la arquitectura dual diseñada para resolver el cuello de botella de latencia en agentes de voz?", - "respuesta": "Se compone de un 'Slow Thinker' (agente de fondo que predice temas y pre-recupera datos) y un 'Fast Talker' (agente de primer plano que responde desde una caché semántica) [12, 13]." - }, - { - "pregunta": "¿Qué mejora de velocidad de recuperación se logró en las consultas que resultaron en un acierto de caché (cache hit)?", - "respuesta": "Se logró una aceleración de 316 veces, reduciendo la latencia de recuperación de una media de 110 ms a 0.35 ms [12, 14]." - } -] -}, -{ -"paper": "Specification-Driven Generation of Discrete-Event World Models (2603.03784v1.pdf)", -"preguntas": [ - { - "pregunta": "¿Qué formalismo se utiliza para descomponer los sistemas en componentes atómicos y acoplados con semántica de temporización explícita?", - "respuesta": "Se utiliza el formalismo DEVS (Discrete Event System Specification) [15, 16]." - }, - { - "pregunta": "¿En qué consiste el marco de evaluación propuesto para validar los simuladores generados por LLM?", - "respuesta": "Es un marco basado en trazas que valida las trazas de eventos estructuradas emitidas por el simulador contra restricciones temporales y semánticas derivadas de la especificación original [15, 17]." - } -] -}, -{ -"paper": "Agentics 2.0 (2603.04241v1.pdf)", -"preguntas": [ - { - "pregunta": "¿Qué concepto central de Agentics 2.0 formaliza una llamada de inferencia de LLM como una transformación semántica tipada?", - "respuesta": "La 'función transducible', basada en el álgebra de transducción lógica [18-20]." - }, - { - "pregunta": "¿Qué modelo de programación asíncrona utiliza el marco para garantizar la escalabilidad en flujos de trabajo de datos agénticos?", - "respuesta": "Utiliza una semántica de Map-Reduce asíncrona que permite procesar colecciones de estados de tipo en paralelo [18, 21, 22]." - } -] -}, -{ -"paper": "AI Agents, Language, Deep Learning and the Next Revolution in Science (2603.07940v1.pdf)", -"preguntas": [ - { - "pregunta": "¿Qué sistema multi-agente se menciona como ejemplo de aplicación en la investigación de colisionadores de partículas en el CEPC?", - "respuesta": "El sistema Dr. Sai, desarrollado en el Instituto de Física de Altas Energías (IHEP) [23, 24]." - }, - { - "pregunta": "¿Cuál es el nombre del lenguaje de dominio específico (DSL) utilizado en dicho sistema para describir objetivos analíticos?", - "respuesta": "Se denomina SaiScript [25]." - } -] -}, -{ -"paper": "Autonomous AI Agent for Clinical Triage in Remote Patient Monitoring (2603.09052v1.pdf)", -"preguntas": [ - { - "pregunta": "¿Cómo se denomina el agente de IA desarrollado para realizar el triaje clínico contextual de signos vitales?", - "respuesta": "Se llama Sentinel [26, 27]." - }, - { - "pregunta": "¿Qué sensibilidad alcanzó el agente para clasificaciones de emergencia en comparación con el promedio de clínicos humanos?", - "respuesta": "El agente alcanzó una sensibilidad del 97.5% frente al 60.0% agregado de los clínicos individuales en el análisis 'leave-one-out' [28, 29]." - } -] -}, -{ -"paper": "AI Act Evaluation Benchmark (2603.09435v1.pdf)", -"preguntas": [ - { - "pregunta": "¿Qué tareas de aprendizaje automático se incluyen en el conjunto de datos para evaluar el cumplimiento de la Ley de IA de la UE?", - "respuesta": "Clasificación del nivel de riesgo, recuperación de artículos, generación de obligaciones y respuesta a preguntas (QA) [30, 31]." - }, - { - "pregunta": "¿Qué modelo de código abierto se utilizó para generar los escenarios del dataset debido a su capacidad de ejecución en una sola GPU?", - "respuesta": "Se utilizó el modelo gpt-oss-120b [32]." - } -] -}, -{ -"paper": "Extreme Multi-label Text Classification (XMTC) Library Dataset (2603.10876v1.pdf)", -"preguntas": [ - { - "pregunta": "¿Cómo se llama el corpus bilingüe de registros bibliográficos presentado para la indexación automatizada de materias?", - "respuesta": "Se denomina TIB-SID (TIB Subject Indexing Dataset) [33]." - }, - { - "pregunta": "¿Qué archivo de autoridad se utiliza como taxonomía para las anotaciones de materias en este conjunto de datos?", - "respuesta": "Se utiliza el GND (Gemeinsame Normdatei / Integrated Authority File) de la Biblioteca Nacional Alemana [34, 35]." - } -] -}, -{ -"paper": "FinReflectKG - HalluBench (2603.20252v1.pdf)", -"preguntas": [ - { - "pregunta": "¿Cuál es el objetivo principal del benchmark FinReflectKG - HalluBench?", - "respuesta": "Evaluar métodos de detección de alucinaciones en sistemas de respuesta a preguntas financieras aumentados con Grafos de Conocimiento (KG) sobre informes SEC 10-K [36, 37]." - }, - { - "pregunta": "¿Qué tipo de método de detección demostró mayor robustez frente a señales de KG ruidosas o erróneas?", - "respuesta": "Los enfoques basados en embeddings, que mostraron solo un 9% de degradación en comparación con las caídas significativas de otros métodos [36, 38]." - } -] -} diff --git a/notebooks/003-jmmz-ingestion_service.ipynb b/notebooks/003-jmmz-ingestion_service.ipynb index 112e30b..4594910 100644 --- a/notebooks/003-jmmz-ingestion_service.ipynb +++ b/notebooks/003-jmmz-ingestion_service.ipynb @@ -507,6 +507,63 @@ "set([r.metadata['paper_id'] for r in results])" ] }, + { + "cell_type": "markdown", + "id": "ab289e5d", + "metadata": {}, + "source": [ + "## Para crear un dataset de evaluación" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "505f52b4", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "bc2bfa3c36b04e3f9a2298941a79416c", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Loading weights: 0%| | 0/103 [00:00 list[Document]:\n", + " results = await store.search(query=question, k=max_results)\n", + " for r in results:\n", + " print(f\"\\nscore: {r.score:.3f}\")\n", + " print(f\"text: {r.text[:200]}\")\n", + "\n", + " return results\n", + "\n", + "\n", + "embedder = LocalEmbedder()\n", + "\n", + "store = ChromaVectorStore(\n", + " embedder=embedder,\n", + " )" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "385ce22d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "----------What are the three coupled dimensions of externalization in LLM agents as described by Zhou et al. (2026)?**********\n", + "- pdf_ref: chenyu_zhou_2026.pdf\n", + "\n", + "score: 0.786\n", + "text: Externalization in LLM Agents: A Unified Review of\n", + "Memory, Skills, Protocols and Harness Engineering\n", + "Chenyu Zhou1, Huacan Chai1,∗, Wenteng Chen1,∗, Zihan Guo2,3,∗, Rong Shan1,∗, Yuanyi\n", + "Song1,∗, Tianyi\n", + "\n", + "score: 0.770\n", + "text: dscape onto three capability layers—Weights, Context, and Harness. Fig-\n", + "ure 3 complements this view with an architectural overview of the externalized agent, showing the harness\n", + "at the center with the\n", + "\n", + "score: 0.763\n", + "text: the strongest forms of externalization in agent systems, because it removes entire classes of\n", + "reasoning from the critical path. The transformation is analogous to the shift that memory introduces for\n", + "\n", + "- papers in answer: {'chenyu_zhou_2026'}\n", + "\n", + "\n", + "----------What is the primary function of the 'harness' in an externalized agent architecture?**********\n", + "- pdf_ref: chenyu_zhou_2026.pdf\n", + "\n", + "score: 0.884\n", + "text: l points.\n", + "The remainder of the paper proceeds as follows. Section 2 traces the historical path from weights to context\n", + "to harness. Sections 3 through 5 analyze memory, skills, and protocols as three d\n", + "\n", + "score: 0.868\n", + "text: ocedural expertise, so that complex workflows are loaded rather than reinvented. Protocols\n", + "externalize interaction structure, so that tool and agent coordination follows governed contracts rather than\n", + "\n", + "score: 0.866\n", + "text: g, human oversight, observability, configuration,\n", + "and context management—provide an analytical framework for characterizing harness architectures. None of\n", + "them is a form of externalization in its own \n", + "- papers in answer: {'chenyu_zhou_2026'}\n", + "\n", + "\n", + "----------How does externalized memory transform the cognitive task for a Large Language Model?**********\n", + "- pdf_ref: chenyu_zhou_2026.pdf\n", + "\n", + "score: 0.849\n", + "text: g sections examine each in detail.\n", + "3\n", + "Externalized State: Memory\n", + "Memory externalization addresses the temporal burden of agency. A bare language model must carry conti-\n", + "nuity, prior experience, user-sp\n", + "\n", + "score: 0.829\n", + "text: the memory system has already surfaced. This is closely analogous to Norman’s analysis of how\n", + "an external list changes the nature of remembering: the crucial point is not that extra information has b\n", + "\n", + "score: 0.826\n", + "text: le in principle: keep an effectively unbounded\n", + "history available while still reasoning clearly about the present.\n", + "Memory externalization changes the structure of that task. In Norman’s terms, the repr\n", + "- papers in answer: {'chenyu_zhou_2026'}\n", + "\n", + "\n", + "----------In the context of skill externalization, what are the three essential components of 'procedural expertise'?**********\n", + "- pdf_ref: chenyu_zhou_2026.pdf\n", + "\n", + "score: 0.893\n", + "text: the crucial\n", + "issue is whether procedural expertise can be represented in a form that is discoverable, loadable, interpretable,\n", + "bindable, and executable at runtime. Therefore, skill externalization inv\n", + "\n", + "score: 0.883\n", + "text: n therefore marks\n", + "the point at which skills become a genuine capability layer rather than a collection of isolated recipes [Yu\n", + "et al., 2025].\n", + "Overall, skill externalization should not be understood as\n", + "\n", + "score: 0.881\n", + "text: focuses on three linked questions: what burden skills externalize,\n", + "how skills reorganize task execution, and how they become actionable inside a larger harness.\n", + "Figure 5 Skills as externalized expert\n", + "- papers in answer: {'chenyu_zhou_2026'}\n", + "\n", + "\n", + "----------What is the definition of 'cognitive flow' within the framework proposed by Dissanayake and Nanayakkara (2025)?**********\n", + "- pdf_ref: dinithi_dissanayake_2025.pdf\n", + "\n", + "score: 0.826\n", + "text: nces\n", + "4\n", + "Dinithi Dissanayake and Suranga Nanayakkara\n", + "cognition in ways that feel seamless and natural, preserving a sense of agency while fostering deeper intellectual\n", + "engagement.\n", + "4\n", + "Call to Action\n", + "This \n", + "\n", + "score: 0.796\n", + "text: State of Flow –> State of Cognitive Flow\n", + "One of the fundamental questions in positive psychology concerns the nature of a good life. Flow theory offers\n", + "one perspective, emphasizing deep engagement and\n", + "\n", + "score: 0.770\n", + "text: Navigating the State of Cognitive Flow: Context-Aware AI Interventions for\n", + "Effective Reasoning Support\n", + "DINITHI DISSANAYAKE, Augmented Human Lab, National University of Singapore, Singapore\n", + "SURANGA NAN\n", + "- papers in answer: {'dinithi_dissanayake_2025'}\n", + "\n", + "\n", + "----------What are the three key contextual factors that determine the effectiveness of AI interventions for reasoning support?**********\n", + "- pdf_ref: dinithi_dissanayake_2025.pdf\n", + "\n", + "score: 0.824\n", + "text: re is\n", + "particularly relevant in the context of AI-augmented cognitive interventions, where external assistance must be\n", + "carefully calibrated to avoid disrupting or undermining an individual’s sense of a\n", + "\n", + "score: 0.818\n", + "text: to ensure that reasoning tasks remain appropriately challenging, not so difficult as to cause frustration, nor so easy\n", + "Navigating the State of Cognitive Flow: Context-Aware AI Interventions for Effect\n", + "\n", + "score: 0.816\n", + "text: argumentation [4, 5, 7]. The effectiveness of such systems\n", + "depends on several factors, with the type, timing, and scale of the intervention playing crucial roles in determining the\n", + "quality of the aug\n", + "- papers in answer: {'dinithi_dissanayake_2025'}\n", + "\n", + "\n", + "----------How can AI systems leverage multimodal cues to infer a user's cognitive load in real-time?**********\n", + "- pdf_ref: dinithi_dissanayake_2025.pdf\n", + "\n", + "score: 0.854\n", + "text: lored to an individual’s cognitive state. In the context of cognitive flow, AI systems must intelligently map\n", + "multimodal inputs to determine whether an individual is in a state of deep engagement, fac\n", + "\n", + "score: 0.813\n", + "text: reasing availability of large-scale multimodal datasets have made it possible to model\n", + "human behavior with unprecedented granularity. By leveraging behavioral cues—such as gaze anticipation, gesture\n", + "p\n", + "\n", + "score: 0.810\n", + "text: s cognitive state, neither disrupting engagement nor allowing stagnation. Using insights from\n", + "flow theory, we argue that AI systems should infer behavioral cues from multimodal data to sustain an opti\n", + "- papers in answer: {'dinithi_dissanayake_2025'}\n", + "\n", + "\n", + "----------What is the core objective of the ILR (Interactive Learning for LLM Reasoning) framework?**********\n", + "- pdf_ref: hehai_lin_2025.pdf\n", + "\n", + "score: 0.877\n", + "text: ss of stronger LLMs and surpass-\n", + "ing pure strategy), and the scalability of ILR\n", + "beyond two-model interactions. Code is avail-\n", + "able at https://github.com/linhh29/Interactive-\n", + "Learning-for-LLM-Reasoning\n", + "\n", + "score: 0.845\n", + "text: .11\n", + "20.00\n", + "58.00\n", + "ILR-Group3\n", + "95.42\n", + "82.32\n", + "50.07\n", + "43.53\n", + "19.66\n", + "58.20\n", + "Table 1: The quantification comparison (accuracy %) of ILR and other baselines.\n", + "5.2\n", + "Dynamic Interaction Enhances Stronger\n", + "LLMs’ Robustnes\n", + "\n", + "score: 0.843\n", + "text: first to explore its poten-\n", + "tial for enhancing an individual LLM’s inde-\n", + "pendent reasoning capability.\n", + "• Inspired by human interaction, we design a\n", + "novel multi-agent learning framework ILR,\n", + "which comp\n", + "- papers in answer: {'hehai_lin_2025'}\n", + "\n", + "\n", + "----------Describe the three sequential stages of the 'Idea3' framework for agent communication.**********\n", + "- pdf_ref: hehai_lin_2025.pdf\n", + "\n", + "score: 0.827\n", + "text: ion\n", + "q and determine the interaction mode. We study the\n", + "effect of mixed-type interaction for each question\n", + "in Appendix D.5.\n", + "Mode =\n", + "(\n", + "Cooperation\n", + "if Pq < 0.5\n", + "Competition\n", + "if Pq ≥0.5\n", + "(3)\n", + "To simulate human\n", + "\n", + "score: 0.802\n", + "text: dent entity. We emulate human discussion\n", + "dynamics through a novel Idea3 interaction, specifi-\n", + "cally designed to facilitate critical thinking commu-\n", + "nication among agents via its three-stage process:\n", + "I\n", + "\n", + "score: 0.800\n", + "text: e the probability of solving it independently.\n", + "If the probability is low, the model engages in co-\n", + "operation; otherwise, it chooses competition. For\n", + "“Interaction”, we design a novel Idea3 framework,\n", + "c\n", + "- papers in answer: {'hehai_lin_2025'}\n", + "\n", + "\n", + "----------How does the ILR system decide between cooperation and competition strategies during training?**********\n", + "- pdf_ref: hehai_lin_2025.pdf\n", + "\n", + "score: 0.799\n", + "text: her conduct\n", + "multi-agent inference using ILR-trained LLMs as\n", + "base models. Results show that ILR-trained LLMs\n", + "achieve better performance than untrained LLMs in\n", + "the same multi-agent inference setup, whic\n", + "\n", + "score: 0.784\n", + "text: competition. To further inves-\n", + "tigate the influence of cooperation, we vary the\n", + "cooperation ratio (p) from 0.0 to 1.0 in increments\n", + "of 0.2. Here, p = 0.0 corresponds to full compe-\n", + "tition, p = 1.0 to\n", + "\n", + "score: 0.780\n", + "text: results, with IRT highlighted\n", + "in red. Two key findings emerge for dynamic in-\n", + "teraction design: (1) Suboptimality of Extreme\n", + "Strategies. Relying solely on competition or coop-\n", + "eration is suboptimal fo\n", + "- papers in answer: {'hehai_lin_2025'}\n", + "\n", + "\n", + "----------What is 'Perception Calibration' in the ILR framework and how is it implemented?**********\n", + "- pdf_ref: hehai_lin_2025.pdf\n", + "\n", + "score: 0.764\n", + "text: ected to produce\n", + "during inference. Therefore, there is no misalign-\n", + "ment between the training and inference prompts.\n", + "(2) To ensure a fair comparison with single-agent\n", + "learning baselines. Including the\n", + "\n", + "score: 0.744\n", + "text: 07\n", + "Table 3: Ablation Study of ILR. We report the average accuracy (%) of five mathematical evaluation benchmarks.\n", + "DI, PC represent Dynamic Interaction and Perception Calibration.\n", + "by making them less s\n", + "\n", + "score: 0.718\n", + "text: le 8 shows that im-\n", + "plementing z-score normalization within the ILR\n", + "framework yields inferior performance compared\n", + "to our method. We attribute this degradation to the\n", + "aggressive clipping associated wi\n", + "- papers in answer: {'hehai_lin_2025'}\n", + "\n", + "\n", + "----------How does the DeepSeek-R1-1.5B model compare to significantly larger models like CodeLlama-13B as a discriminator for SQL tasks?**********\n", + "- pdf_ref: md_fahim_anjum_2025.pdf\n", + "\n", + "score: 0.874\n", + "text: articular, we find that a 1.5B distilled DeepSeek-R1 model achieves 87% higher F1 as well as\n", + "3.7% better discrimination accuracy than CodeLlama-7B and 3.7% higher execution accuracy than\n", + "CodeLlama-13B\n", + "\n", + "score: 0.815\n", + "text: s are more effec-\n", + "tive discriminators than non-reasoning LLMs. Our results show that distilled\n", + "DeepSeek-R1-1.5B achieves up to 87% higher F1 and 3.7% better discrimination\n", + "accuracy than CodeLlama-7B, \n", + "\n", + "score: 0.807\n", + "text: capabilities?, 2025.\n", + "[48] Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. Tinyllama: An open-source small\n", + "language model. ArXiv, abs/2401.02385, 2024.\n", + "[49] Kaikai Zhao, Zhaoxiang Liu, Xuejiao\n", + "- papers in answer: {'md_fahim_anjum_2025'}\n", + "\n", + "\n", + "----------What happens to the discrimination performance of reasoning models when the test-time compute budget is increased beyond 1024 tokens?**********\n", + "- pdf_ref: md_fahim_anjum_2025.pdf\n", + "\n", + "score: 0.912\n", + "text: investigate the impact of test-time compute budget on its discrimination\n", + "capabilities. Specifically, we explore whether allocating additional computation for reasoning\n", + "enhances its performance. Experi\n", + "\n", + "score: 0.851\n", + "text: andidates. A very low token limit severely restricts the reasoning\n", + "process, leading to extremely low accuracy and a very high failure rate, indicating the necessity of a\n", + "minimum compute threshold. As \n", + "\n", + "score: 0.846\n", + "text: ere access to logit\n", + "values for reasoning models may be restricted or expensive. Together, these results suggest that\n", + "reasoning models hold substantial promise for robust discrimination tasks, and furt\n", + "- papers in answer: {'md_fahim_anjum_2025'}\n", + "\n", + "\n", + "----------Is the reasoning model DeepSeek-R1 more effective as a generator or a discriminator in SQL parsing?**********\n", + "- pdf_ref: md_fahim_anjum_2025.pdf\n", + "\n", + "score: 0.862\n", + "text: chitecture.\n", + "In this work, we examine the role of a reasoning model within a generator-discriminator LLM\n", + "planning agentic framework applied to the text-to-SQL generation task. For this, we use a distil\n", + "\n", + "score: 0.850\n", + "text: ributions in this work are as follows:\n", + "(1) We provide a systematic comparison of reasoning and non-reasoning LLMs within a generator-\n", + "discriminator LLM planning framework for the text-to-SQL task.\n", + "(2)\n", + "\n", + "score: 0.847\n", + "text: y on scaling\n", + "compute and maximizing context window but also on targeted fine-tuning of such reasoning models.\n", + "11\n", + "5.3\n", + "Generation is Harder than Discrimination\n", + "Third, while Distill-R1 shows strong perfo\n", + "- papers in answer: {'md_fahim_anjum_2025'}\n", + "\n", + "\n", + "----------What novel method was proposed to extract soft scores from reasoning models for fine-grained ranking?**********\n", + "- pdf_ref: md_fahim_anjum_2025.pdf\n", + "\n", + "score: 0.829\n", + "text: ning a soft score is not straightforward.\n", + "For this, we in-\n", + "4\n", + "troduce a novel approach: first, we prompt the reasoning model to think and output the fi-\n", + "nal answer in a specific key-value format (JSON)\n", + "\n", + "score: 0.818\n", + "text: ng soft score extraction. By leveraging our approach, it is potentially\n", + "possible to fine-tune such models (with soft score as the cost function) and adjust their reasoning\n", + "approaches to improve perfor\n", + "\n", + "score: 0.814\n", + "text: riminator role is typically more challenging\n", + "and significant than the generator role [5, 36, 10, 31], our results indicate the opposite for reasoning\n", + "models , which is similar to humans [9].\n", + "5.4\n", + "Futur\n", + "- papers in answer: {'md_fahim_anjum_2025'}\n", + "\n", + "\n", + "----------How is 'analogical reasoning' defined by Musker et al. (2024)?**********\n", + "- pdf_ref: sam_musker_2024.pdf\n", + "\n", + "score: 0.872\n", + "text: musker/LLM\n", + "_Analogical_Reasoning\n", + "Keywords:\n", + "Language models\n", + "Analogical reasoning\n", + "Cognitive science\n", + " \n", + "A B S T R A C T\n", + "Analogical reasoning — the capacity to identify and map structural relationships bet\n", + "\n", + "score: 0.866\n", + "text: ch is: what is the \n", + "relevance of LLMs’ analogical reasoning abilities to human cognitive \n", + "theory? By designing a set of analogical reasoning tasks which are \n", + "not readily explained by existing theories\n", + "\n", + "score: 0.864\n", + "text: nomenon of interest.\n", + "Implicitly requiring subjects in our experiments to infer an abstract \n", + "rule or schema that governs the relationship between a source and \n", + "target domain, and to do so based on flex\n", + "- papers in answer: {'sam_musker_2024'}\n", + "\n", + "\n", + "----------What is 'flexible re-representation' and why is it essential for solving non-trivial analogies?**********\n", + "- pdf_ref: sam_musker_2024.pdf\n", + "\n", + "score: 0.845\n", + "text: lations are relevant \n", + "for a given analogy in the first place (Chalmers et al., 1992).\n", + "The first model which attempted to address the question of re-\n", + "representation is Copycat (Hofstadter & Mitchell, 1\n", + "\n", + "score: 0.787\n", + "text: standing analogical reasoning’s role in natural contexts. Cur-\n", + "rent theories either address re-representation without real-world con-\n", + "cepts, or handle real-world concepts without tackling re-represent\n", + "\n", + "score: 0.770\n", + "text: 025). Applying similar methods \n", + "with our tasks could produce how-possibly explanations for the flexible \n", + "re-representation that underlies analogical reasoning in humans.6 In a \n", + "field where available t\n", + "- papers in answer: {'sam_musker_2024'}\n", + "\n", + "\n", + "----------How did human subjects and LLMs differ in their response to misleading semantic structure in the Study 1 'Randoms' condition?**********\n", + "- pdf_ref: sam_musker_2024.pdf\n", + "\n", + "score: 0.852\n", + "text: ’ for \n", + "the same reason.\n", + "In the Semantic Content experiment, each condition (described \n", + "in Table 4) contains two quizzes, with four questions per quiz. Un-\n", + "less otherwise stated, methodological details\n", + "\n", + "score: 0.837\n", + "text: semantic structure when a clear pattern exists (evi-\n", + "denced by the Defaults and Random Finals conditions) but can ignore \n", + "words when structure is lacking (Randoms condition). Models show the \n", + "former b\n", + "\n", + "score: 0.837\n", + "text: he final term, testing whether subjects \n", + "will dynamically shift their strategy when semantic structure appears \n", + "to be ‘‘misleading’’.\n", + "We interpret a performance discrepancy between either the Random\n", + "o\n", + "- papers in answer: {'sam_musker_2024'}\n", + "\n", + "\n", + "----------What specific limitation did GPT-4 demonstrate in the numeric conditions of the Semantic Content experiment?**********\n", + "- pdf_ref: sam_musker_2024.pdf\n", + "\n", + "score: 0.851\n", + "text: nificant difference in \n", + "that condition relative to the Numeric condition (coef = −1.4323, z =\n", + "−2.622, p = 0.009).\n", + "Takeaways\n", + "The Semantic Content experiment confirms that human subjects \n", + "perform robust\n", + "\n", + "score: 0.848\n", + "text: gical \n", + "reasoning can explain poor model performance in some tasks, we find \n", + "that GPT-4’s failure in the numeric conditions of our Semantic Content \n", + "experiment may be due to a deficiency in counting ab\n", + "\n", + "score: 0.827\n", + "text: ed for the likelihood ratio test is \n", + "4.\n", + "As observed in the Semantic Structure experiment, the performance \n", + "of GPT-4 in the Semantic Content experiment is human-comparable in \n", + "some conditions but notab\n", + "- papers in answer: {'sam_musker_2024'}\n", + "\n", + "\n", + "----------How does the analogical reasoning performance of Claude 3 Opus compare to human performance across the studied conditions?**********\n", + "- pdf_ref: sam_musker_2024.pdf\n", + "\n", + "score: 0.854\n", + "text: tasks allow for clear \n", + "discrimination between human performance and that of most models \n", + "prior to Claude 3, further differences in analogical reasoning patterns \n", + "between humans and Claude 3 likely exi\n", + "\n", + "score: 0.808\n", + "text: n human subjects and \n", + "LLMs across task variations are not subject to an auxiliary task demand \n", + "explanation and suggest that the underlying mechanisms of analogical \n", + "reasoning in these systems may diff\n", + "\n", + "score: 0.807\n", + "text: in nu-\n", + "meric conditions is notable, it most likely reflects a failure in numeric \n", + "reasoning rather than a difference in analogical reasoning.\n", + "We find evidence of decreased human performance, but not m\n", + "- papers in answer: {'sam_musker_2024'}\n", + "\n", + "\n" + ] + } + ], + "source": [ + "answers = []\n", + "for dict_question in data:\n", + " print('-'*10 + f\"{dict_question['question']}\" + '*'*10)\n", + " print(f\"- pdf_ref: {dict_question['source_paper']}\")\n", + " answer = await answer_question(dict_question['question'], store=store, max_results=3)\n", + " print(f\"- papers in answer: {set([doc.metadata['paper_id'] for doc in answer])}\")\n", + " print('\\n')\n", + "\n", + " answers.append(answers)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c293f6ad", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import asyncio\n", + "answers = await asyncio.gather(*(answer_question(dict_question['question'], store=store, max_results=3) for dict_question in data))" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "075eb5bd", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "['sam_musker_2024', 'sam_musker_2024', 'sam_musker_2024']" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "[doc.metadata['paper_id'] for doc in answer]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0f6afdc3", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/scripts/eval_retrieval.py b/scripts/eval_retrieval.py new file mode 100644 index 0000000..5d1188c --- /dev/null +++ b/scripts/eval_retrieval.py @@ -0,0 +1,41 @@ +import asyncio +import json + +from researchos.domain.interfaces import VectorStore +from researchos.domain.models import Document +from researchos.infrastructure.retrieval.chroma import ChromaVectorStore +from researchos.infrastructure.retrieval.embedder import LocalEmbedder +from researchos.paths import SAMPLES_DIR + +path_examples = SAMPLES_DIR / "eval_dataset.json" + + +async def answer_question( + question: str, store: VectorStore, max_results: int = 10 +) -> list[Document]: + results = await store.search(query=question, k=max_results) + for r in results: + print(f"\nscore: {r.score:.3f}") + print(f"text: {r.text[:200]}") + return results + + +async def main() -> None: + with open(path_examples, encoding="utf-8") as f: + data = json.load(f) + + embedder = LocalEmbedder() + store = ChromaVectorStore(embedder=embedder) + + answers = [] + for dict_question in data: + print("-" * 10 + dict_question["question"] + "*" * 10) + print(f"- pdf_ref: {dict_question['source_paper']}") + answer = await answer_question(dict_question["question"], store=store, max_results=3) + print(f"- papers in answer: {set([doc.metadata['paper_id'] for doc in answer])}") + print("\n") + answers.append(answer) + + +if __name__ == "__main__": + asyncio.run(main()) From 1a22a89d26ddfba11c41603ea250c816e0cb6216 Mon Sep 17 00:00:00 2001 From: Mario Date: Sat, 18 Apr 2026 10:58:24 -0500 Subject: [PATCH 26/55] docs: update learnings and work_log files --- docs/learnings.md | 20 ++++++++++++++++++++ docs/work_log.md | 24 ++++++++++++++++++++++++ 2 files changed, 44 insertions(+) diff --git a/docs/learnings.md b/docs/learnings.md index fef8042..b0cd3ab 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -161,6 +161,26 @@ Regla simple para ResearchOS: - Hack de pysqlite3 en `conftest.py`, no en código de producción. Se resolverá en Dockerfile en V4. +**Fecha:** 18/04/2026 + +### ¿Qué aprendí? +- en función del nombre de la colección los textos se guardan aisladamente +- el await asyncio.gather no tiene mucha razón de ser si justamente después de éste viene un ciclo for que procesa en orden cada resultado recuperado -> Preguntar a mi tutor +- Cuando estoy en un script .py y quiero disparar una función async, no puedo usar await (esto solo en el REPL o en notebooks), tengo que usar asyncio.run -> Verificar y preguntar cómo sería algo con create_task() + +### ¿Qué no entendí bien? +- Debo consultar si se puede hacer una consulta general en chroma sin importar la colección para que busque en absolutamente toda la base -> Lo que entiendo es que chroma divide los documentos guardamos conforme las colecciones + +### Decisiones de diseño +- + +### Errores interesantes +- `uv run pip list` en Windows mostraba el entorno global de pipx en lugar del `.venv` del proyecto — engañoso. La forma correcta de verificar es `uv run python -c "import ; print('ok')"` o `uv run python -c "import sys; print(sys.executable)"`. +- `ipykernel` no se instala automáticamente con `jupyter` en `uv` — debe agregarse explícitamente como dependencia dev. Sin él, Jupyter no puede registrar el kernel del proyecto. Solución: `uv add --dev ipykernel` y luego `uv run python -m ipykernel install --user --name researchos --display-name "ResearchOS"`. +- `pysqlite3-binary` solo tiene wheels para Linux — en Windows sqlite3 ya viene actualizado con Python. Solución: `"pysqlite3-binary>=0.5.4; sys_platform == 'linux'"` en `pyproject.toml`. +- El hack de sqlite3 en `conftest.py` fallaba en Windows porque `pysqlite3` no existe ahí. Solución: condicional por plataforma `if sys.platform == "linux":` antes del import. +- `asyncio.gather` sin `await` no ejecuta las coroutines — retorna un objeto coroutine sin resolver. Siempre `await asyncio.gather(...)`. +- `Path.stem` retorna el nombre del archivo sin extensión — más limpio que hacer `split(os.sep)[-1].split('.pdf')[0]` sobre un string. **Fecha:** _[completar]_ diff --git a/docs/work_log.md b/docs/work_log.md index a644521..d440350 100644 --- a/docs/work_log.md +++ b/docs/work_log.md @@ -73,3 +73,27 @@ - Resolver sqlite3 en Docker cuando llegue V4 --- + +## 2026-04-18 + +### Trabajo desarrollado +- Configuración del entorno Windows: + - `pysqlite3-binary` marcado como dependencia solo para Linux en `pyproject.toml` + - Fix de `conftest.py` para que el hack de sqlite3 sea condicional por plataforma + - `ipykernel` agregado como dependencia dev y kernel registrado manualmente + - `pyproject.toml` consolidado: dependencias dev unificadas en `[dependency-groups]` + - Autor actualizado: John Mario Montoya Zapata +- `ensure_dirs()` implementada en `paths.py` — centraliza creación de directorios +- Manejo de errores en `arxiv.py`: validación de respuesta antes de parsear XML +- `ingest_papers()` completada en `ingestion_service.py`: + - Orquesta: arXiv → descarga PDF → chunking → Chroma + - Descarga paralela con `asyncio.gather()` + - Parámetros configurables: `chunk_size`, `overlap`, `collection_name` + - Probada en notebook con `max_results=2` — funcionó correctamente + +### Próximos pasos +- Copiar `data/samples/sample_pdf.pdf` desde el servidor Linux a Windows +- Dataset de evaluación: 20 preguntas con respuestas de referencia +- Consultar con tutor: múltiples colecciones en Chroma, parámetros de `ingest_papers` + +--- From 607a9fec85b8f2878137e1879be745108454dd53 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 23 Apr 2026 09:50:37 -0500 Subject: [PATCH 27/55] docs: add and update docstrings --- ...{betchmark_arxiv.py => benchmark_arxiv.py} | 30 ++++++++++ scripts/eval_retrieval.py | 11 ++++ .../application/services/ingestion_service.py | 57 ++++++++++++++++++ .../application/services/retrieval_service.py | 25 ++++++++ src/researchos/domain/interfaces.py | 60 ++++++++++++++++--- src/researchos/infrastructure/data/arxiv.py | 24 ++++++++ .../infrastructure/llm/anthropic_llm.py | 47 +++++++++++++-- .../infrastructure/retrieval/chroma.py | 43 ++++++++++++- .../infrastructure/retrieval/embedder.py | 28 +++++++++ src/researchos/paths.py | 7 ++- 10 files changed, 315 insertions(+), 17 deletions(-) rename scripts/{betchmark_arxiv.py => benchmark_arxiv.py} (65%) diff --git a/scripts/betchmark_arxiv.py b/scripts/benchmark_arxiv.py similarity index 65% rename from scripts/betchmark_arxiv.py rename to scripts/benchmark_arxiv.py index d711602..1d3e843 100644 --- a/scripts/betchmark_arxiv.py +++ b/scripts/benchmark_arxiv.py @@ -10,6 +10,16 @@ async def sequential_benchmark(query: str, max_results: int): + """Download papers one at a time and report total elapsed time. + + Intended as a baseline to compare against the parallel strategy. Uses a + synchronous ``httpx.Client`` inside an async function, so downloads block + the event loop sequentially. + + Args: + query: arXiv search query string. + max_results: Number of papers to fetch and download. + """ start_time = time.perf_counter() papers = await search_papers(query, max_results) @@ -34,6 +44,15 @@ async def sequential_benchmark(query: str, max_results: int): async def parallel_benchmark(query: str, max_results: int): + """Download all papers concurrently and report total elapsed time. + + Uses ``asyncio.gather`` to fire all downloads at the same time, showing the + speedup over the sequential strategy. + + Args: + query: arXiv search query string. + max_results: Number of papers to fetch and download. + """ start_time = time.perf_counter() papers = await search_papers(query, max_results) @@ -47,6 +66,17 @@ async def parallel_benchmark(query: str, max_results: int): async def _download_one_paper(paper: Paper): + """Download a single paper PDF to PAPERS_DIR using an async HTTP client. + + The filename is derived from the first author's name and publication year, + with non-alphanumeric characters replaced by underscores. + + Args: + paper: Paper whose ``pdf_url`` will be fetched. + + Raises: + httpx.HTTPStatusError: If the download request fails. + """ url = paper.pdf_url pdf_name = paper.authors[0].lower().strip() pdf_name = re.sub(r"[^a-z0-9_]", "_", pdf_name) diff --git a/scripts/eval_retrieval.py b/scripts/eval_retrieval.py index 5d1188c..256fa59 100644 --- a/scripts/eval_retrieval.py +++ b/scripts/eval_retrieval.py @@ -13,6 +13,16 @@ async def answer_question( question: str, store: VectorStore, max_results: int = 10 ) -> list[Document]: + """Retrieve and print the top documents for a question. + + Args: + question: Natural-language question to answer. + store: VectorStore to search against. + max_results: Maximum number of documents to retrieve. + + Returns: + List of retrieved Document objects sorted by score. + """ results = await store.search(query=question, k=max_results) for r in results: print(f"\nscore: {r.score:.3f}") @@ -21,6 +31,7 @@ async def answer_question( async def main() -> None: + """Load the evaluation dataset and run retrieval for every question.""" with open(path_examples, encoding="utf-8") as f: data = json.load(f) diff --git a/src/researchos/application/services/ingestion_service.py b/src/researchos/application/services/ingestion_service.py index b3b8c1e..bcc57dc 100644 --- a/src/researchos/application/services/ingestion_service.py +++ b/src/researchos/application/services/ingestion_service.py @@ -15,11 +15,37 @@ async def extract_text_pdf(paper: Paper) -> tuple[str, Path]: + """Download a paper's PDF and extract its full text. + + Args: + paper: Paper whose ``pdf_url`` will be fetched. + + Returns: + A tuple of (extracted_text, local_pdf_path). + + Raises: + httpx.HTTPStatusError: If the PDF download fails. + IngestionError: If no text can be extracted from the PDF. + """ pdf_path = await _download_pdf(paper=paper) return _extract_text(pdf_path=pdf_path), pdf_path async def _download_pdf(paper: Paper) -> Path: + """Download the PDF for a paper and save it to PAPERS_DIR. + + The filename is derived from the first author's name and the publication year, + with non-alphanumeric characters replaced by underscores. + + Args: + paper: Paper to download. + + Returns: + Path to the saved PDF file. + + Raises: + httpx.HTTPStatusError: If the download request fails. + """ url = paper.pdf_url pdf_name = paper.authors[0].lower().strip() pdf_name = re.sub(r"[^a-z0-9_]", "_", pdf_name) @@ -37,6 +63,17 @@ async def _download_pdf(paper: Paper) -> Path: def _extract_text(pdf_path: Path) -> str: + """Extract all text from a PDF file using PyMuPDF. + + Args: + pdf_path: Path to the local PDF file. + + Returns: + Concatenated plain text from all pages. + + Raises: + IngestionError: If the PDF yields no extractable text. + """ full_text = "" doc = fitz.open(pdf_path) for page in doc: @@ -56,6 +93,26 @@ async def ingest_papers( collection_name: str = "papers", embedder_metadata: dict | None = None, ) -> None: + """Search arXiv, download PDFs, chunk text, and upsert into the vector store. + + This is the top-level ingestion pipeline. It orchestrates: + 1. arXiv search → list of Paper objects. + 2. Parallel PDF download + text extraction. + 3. Overlap chunking of each paper. + 4. Batch upsert into ChromaDB. + + Args: + query: arXiv search query string. + max_results: Number of papers to fetch from arXiv. + chunk_size: Character length of each text chunk. + overlap: Number of characters to overlap between consecutive chunks. + collection_name: Target Chroma collection name. + embedder_metadata: Optional HNSW settings forwarded to ChromaVectorStore. + + Raises: + IngestionError: If any PDF cannot be downloaded or yields no text. + httpx.HTTPStatusError: On network failures during download. + """ embedder = LocalEmbedder() store = ChromaVectorStore( embedder=embedder, collection_name=collection_name, embedder_metadata=embedder_metadata diff --git a/src/researchos/application/services/retrieval_service.py b/src/researchos/application/services/retrieval_service.py index 1367b49..204e785 100644 --- a/src/researchos/application/services/retrieval_service.py +++ b/src/researchos/application/services/retrieval_service.py @@ -4,6 +4,20 @@ def overlap_chunking( text: str, paper_id: str, chunk_size: int = 500, overlap: int = 50 ) -> list[Chunk]: + """Split a text into fixed-size chunks with character-level overlap. + + Each chunk's start position is shifted back by ``overlap * chunk_index`` + characters so consecutive chunks share context at their boundaries. + + Args: + text: Full document text to split. + paper_id: Identifier of the source paper, used as a prefix in chunk IDs. + chunk_size: Number of characters per chunk. + overlap: Number of characters of overlap between consecutive chunks. + + Returns: + Ordered list of Chunk objects covering the full text. + """ chunks = [] for i, initial_car in enumerate(range(0, len(text), chunk_size)): @@ -28,6 +42,17 @@ def overlap_chunking( def chunk_to_document(chunk: Chunk) -> Document: + """Convert a Chunk into a Document suitable for vector store indexing. + + Merges chunk-level metadata with ``paper_id`` and ``chunk_index`` so that + retrieved Documents retain full provenance information. + + Args: + chunk: Source Chunk to convert. + + Returns: + Document with the same text and enriched metadata. + """ return Document( doc_id=chunk.chunk_id, text=chunk.text, diff --git a/src/researchos/domain/interfaces.py b/src/researchos/domain/interfaces.py index bdbc0e7..a5336e9 100644 --- a/src/researchos/domain/interfaces.py +++ b/src/researchos/domain/interfaces.py @@ -16,7 +16,8 @@ async def upsert(self, documents: list[Document]) -> None: pass """ -from typing import AsyncIterator, Protocol +from collections.abc import AsyncIterator +from typing import Protocol from .models import Document, Message @@ -25,11 +26,25 @@ class LLMProvider(Protocol): """Contract for any LLM provider (Claude, Gemini, etc.).""" async def generate(self, messages: list[Message]) -> str: - """Generate a response from a list of messages.""" + """Generate a complete response from a conversation history. + + Args: + messages: Ordered list of messages including optional system prompt. + + Returns: + Full text response from the model. + """ ... async def stream(self, messages: list[Message]) -> AsyncIterator[str]: - """Stream a response token by token.""" + """Stream a response token by token. + + Args: + messages: Ordered list of messages including optional system prompt. + + Yields: + Successive text chunks as they arrive from the model. + """ ... @@ -37,11 +52,24 @@ class VectorStore(Protocol): """Contract for any vector store (Chroma, Vertex Search, Qdrant, etc.).""" async def search(self, query: str, k: int) -> list[Document]: - """Search for the top-k most relevant documents.""" + """Search for the top-k most relevant documents. + + Args: + query: Natural-language query string. + k: Number of results to return. + + Returns: + List of Document objects sorted by relevance score (descending). + """ ... async def upsert(self, documents: list[Document]) -> None: - """Insert or update documents in the store.""" + """Insert or update documents in the store. + + Args: + documents: Documents to index. Each must have a unique ``doc_id``. + Existing documents with the same ID are overwritten. + """ ... @@ -49,13 +77,29 @@ class MemoryStore(Protocol): """Contract for conversational memory persistence.""" async def get(self, session_id: str) -> list[Message]: - """Retrieve conversation history for a session.""" + """Retrieve conversation history for a session. + + Args: + session_id: Unique identifier for the conversation session. + + Returns: + Ordered list of messages for the session, oldest first. + """ ... async def append(self, session_id: str, message: Message) -> None: - """Append a message to a session's history.""" + """Append a message to a session's history. + + Args: + session_id: Unique identifier for the conversation session. + message: Message to append. + """ ... async def clear(self, session_id: str) -> None: - """Clear conversation history for a session.""" + """Clear conversation history for a session. + + Args: + session_id: Unique identifier for the conversation session to clear. + """ ... diff --git a/src/researchos/infrastructure/data/arxiv.py b/src/researchos/infrastructure/data/arxiv.py index 439688e..dc57eff 100644 --- a/src/researchos/infrastructure/data/arxiv.py +++ b/src/researchos/infrastructure/data/arxiv.py @@ -13,6 +13,19 @@ async def search_papers(query: str, max_results: int) -> list[Paper]: + """Search for papers on arXiv using the public API. + + Args: + query: Search query string (supports arXiv query syntax). + max_results: Maximum number of papers to return. + + Returns: + List of Paper objects parsed from the arXiv Atom feed. + + Raises: + httpx.HTTPStatusError: If the HTTP request fails. + IngestionError: If the response cannot be parsed as XML. + """ params = {"search_query": query, "start": 0, "max_results": max_results} async with httpx.AsyncClient() as client: @@ -22,6 +35,17 @@ async def search_papers(query: str, max_results: int) -> list[Paper]: def _parse_entries(results: httpx.Response) -> list[Paper]: + """Parse an arXiv Atom XML response into a list of Paper objects. + + Args: + results: The raw HTTP response from the arXiv API. + + Returns: + List of Paper objects, one per ```` element. + + Raises: + IngestionError: If the response body is not valid XML. + """ if not results.text.strip().startswith("<"): raise IngestionError(f"arXiv returned unexpected response: {results.text[:100]}") diff --git a/src/researchos/infrastructure/llm/anthropic_llm.py b/src/researchos/infrastructure/llm/anthropic_llm.py index f441396..ae87ad4 100644 --- a/src/researchos/infrastructure/llm/anthropic_llm.py +++ b/src/researchos/infrastructure/llm/anthropic_llm.py @@ -8,18 +8,30 @@ class AnthropicLLM: - """Contract for Claude provider.""" + """Anthropic Claude implementation of the LLMProvider protocol. + + Wraps the async Anthropic SDK client, reading model configuration + from application settings. Supports both single-shot generation and + token-by-token streaming. + """ def __init__(self): + """Initialize the client using credentials and defaults from settings.""" self.client = AsyncAnthropic(api_key=settings.anthropic_api_key) self.model_id = settings.default_model self.temperature = settings.temperature self.max_tokens = settings.max_tokens def _format_messages(self, messages: list[Message]) -> tuple[str | None, list[dict]]: - """ - Separa el system prompt (si existe) y formatea los mensajes - para el esquema que espera Anthropic. + """Split out the system prompt and convert messages to the Anthropic wire format. + + Args: + messages: Conversation history including optional system message. + + Returns: + A tuple of (system_prompt, formatted_messages) where system_prompt is + the content of the first system-role message (or None), and + formatted_messages is the remaining messages as dicts. """ system_prompt = None formatted = [] @@ -33,7 +45,19 @@ def _format_messages(self, messages: list[Message]) -> tuple[str | None, list[di return system_prompt, formatted async def generate(self, messages: list[Message]) -> str: - """Generate a response from a list of messages.""" + """Generate a complete response from a conversation history. + + Args: + messages: Conversation history. A system-role message, if present, + is extracted and sent as the Anthropic ``system`` parameter. + + Returns: + The text content of the first content block in the response. + + Raises: + GenerationError: If the model returns an empty response. + anthropic.APIError: On network or API-level failures. + """ system, formatted_msgs = self._format_messages(messages) @@ -51,7 +75,18 @@ async def generate(self, messages: list[Message]) -> str: raise GenerationError("Claude returned empty response") async def stream(self, messages: list[Message]) -> AsyncIterator[str]: - """Stream a response token by token.""" + """Stream a response token by token from a conversation history. + + Args: + messages: Conversation history. A system-role message, if present, + is extracted and sent as the Anthropic ``system`` parameter. + + Yields: + Successive text chunks as they arrive from the model. + + Raises: + anthropic.APIError: On network or API-level failures. + """ system, formatted_msgs = self._format_messages(messages) diff --git a/src/researchos/infrastructure/retrieval/chroma.py b/src/researchos/infrastructure/retrieval/chroma.py index 6c07f4e..ef23017 100644 --- a/src/researchos/infrastructure/retrieval/chroma.py +++ b/src/researchos/infrastructure/retrieval/chroma.py @@ -6,12 +6,27 @@ class ChromaVectorStore: + """ChromaDB-backed implementation of the VectorStore protocol. + + Uses a local persistent Chroma database and a LocalEmbedder to convert + text to vectors. The embedding space metric (cosine, L2, etc.) is + configured via ``embedder_metadata``. + """ + def __init__( self, embedder: LocalEmbedder, collection_name: str = "papers", embedder_metadata: dict | None = None, ): + """Initialize the store and open (or create) the Chroma collection. + + Args: + embedder: The embedder used to convert text to dense vectors. + collection_name: Name of the Chroma collection to use. + embedder_metadata: HNSW / distance-space settings passed to Chroma. + Defaults to ``{"hnsw:space": "cosine"}``. + """ self.embedder = embedder self.embedder_metadata = embedder_metadata or {"hnsw:space": "cosine"} self.client = chromadb.PersistentClient(path=str(CHROMA_DIR)) @@ -20,7 +35,15 @@ def __init__( ) async def search(self, query: str, k: int) -> list[Document]: - """Search for the top-k most relevant documents.""" + """Search for the top-k most relevant documents. + + Args: + query: Natural-language query string. + k: Number of results to return. + + Returns: + List of Document objects sorted by relevance score (descending). + """ query_embedding = self.embedder.embed(query) retrieved_docs = self.collection.query( @@ -45,7 +68,14 @@ async def search(self, query: str, k: int) -> list[Document]: return results async def upsert(self, documents: list[Document]) -> None: - """Insert or update documents in the store.""" + """Insert or update documents in the store. + + Embeddings are computed in batch for all documents. Existing documents + with the same ``doc_id`` are overwritten. + + Args: + documents: Documents to index. Each must have a unique ``doc_id``. + """ vectors = self.embedder.embed_batch([doc.text for doc in documents]) @@ -59,6 +89,15 @@ async def upsert(self, documents: list[Document]) -> None: ) def _distance_to_score(self, distance: float) -> float: + """Convert a Chroma distance value to a [0, 1] similarity score. + + Args: + distance: Raw distance returned by Chroma (interpretation depends + on the HNSW space configured in ``embedder_metadata``). + + Returns: + Similarity score in [0, 1] where 1 is a perfect match. + """ space = self.embedder_metadata.get("hnsw:space", "cosine") if space == "cosine": return 1 - (distance / 2) diff --git a/src/researchos/infrastructure/retrieval/embedder.py b/src/researchos/infrastructure/retrieval/embedder.py index 6ede1a9..7e554fb 100644 --- a/src/researchos/infrastructure/retrieval/embedder.py +++ b/src/researchos/infrastructure/retrieval/embedder.py @@ -3,11 +3,39 @@ class LocalEmbedder: + """Wraps a SentenceTransformer model for local CPU/GPU embedding. + + The model is downloaded on first use and cached by the sentence-transformers + library. No network access is required after the initial download. + """ + def __init__(self, model_name: str = "all-MiniLM-L6-v2"): + """Load the embedding model. + + Args: + model_name: HuggingFace model identifier. Defaults to + ``all-MiniLM-L6-v2`` (384-dim, fast, good quality). + """ self.model = SentenceTransformer(model_name) def embed(self, text: str) -> list[float]: + """Embed a single text string. + + Args: + text: Input text to embed. + + Returns: + Dense vector as a list of floats. + """ return self.model.encode(text).tolist() def embed_batch(self, texts: list[str]) -> list[list[float]]: + """Embed multiple texts in a single forward pass. + + Args: + texts: List of input strings to embed. + + Returns: + List of dense vectors, one per input string, preserving order. + """ return self.model.encode(texts).tolist() diff --git a/src/researchos/paths.py b/src/researchos/paths.py index 7545092..5f9deb8 100644 --- a/src/researchos/paths.py +++ b/src/researchos/paths.py @@ -10,7 +10,12 @@ def ensure_dirs() -> None: - """Create all data directories if they don't exist.""" + """Create all data directories if they don't exist. + + Iterates over every Path-typed global in this module (excluding + PROJECT_ROOT) and calls ``mkdir(parents=True, exist_ok=True)`` on each. + Safe to call multiple times. + """ dirs = [v for v in globals().values() if isinstance(v, Path) and v != PROJECT_ROOT] for d in dirs: d.mkdir(parents=True, exist_ok=True) From e40c8d0fa528a3a64013e747ce9b01af923dcfaa Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Tue, 19 May 2026 09:27:11 -0500 Subject: [PATCH 28/55] docs: Update ROADMAP.md --- ROADMAP.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index c3529e6..ef0a842 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,17 +2,17 @@ ## Abril 2026 — V1 pipeline RAG básico - [x] T1: AnthropicLLM provider (completado 30 mar) -- [ ] T2: Cliente arXiv API -- [ ] T3: Servicio de ingesta PDFs -- [ ] T4: Chunking fijo -- [ ] T5: Integración end-to-end +- [x] T2: Cliente arXiv API (completado 13 abr) +- [x] T3: Servicio de ingesta PDFs (completado 15 abr) +- [x] T4: Chunking fijo (completado 17 abr) +- [x] T5: Integración end-to-end (completado 18 abr) ## Mayo 2026 — V1 hybrid search y evaluación - [ ] T6: BM25 retrieval - [ ] T7: Hybrid search - [ ] T8: Reranker básico -- [ ] T9: Dataset de evaluación (20 preguntas) -- [ ] T10: Script de evaluación +- [x] T9: Dataset de evaluación (20 preguntas) (completado 18 abr) +- [x] T10: Script de evaluación (completado 18 abr) ## Junio 2026 — V2 LangGraph - [ ] T11: Refactor a LangGraph From a5fb552398fc8b7cef7796a0b9c1887f1e3e846d Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Tue, 19 May 2026 09:28:29 -0500 Subject: [PATCH 29/55] docs(src): add Google-style docstrings to all src/ modules and scripts/ . And update repo structure to v2 clean agents template --- .gitignore | 3 +- data/raw/.gitkeep | 0 queries/develop/.gitkeep | 0 queries/production/.gitkeep | 0 scripts/betchmark_arxiv.py | 53 ++++++++++- scripts/eval_retrieval.py | 42 +++++++++ scripts/setup_data.py | 16 ++++ .../application/services/ingestion_service.py | 82 +++++++++++++++++ .../application/services/retrieval_service.py | 50 +++++++++++ src/researchos/config.py | 31 ++++++- src/researchos/domain/prompts/__init__.py | 18 ++++ src/researchos/infrastructure/data/arxiv.py | 47 ++++++++++ .../infrastructure/llm/anthropic_llm.py | 88 +++++++++++++++++-- .../infrastructure/retrieval/chroma.py | 80 ++++++++++++++++- .../infrastructure/retrieval/embedder.py | 46 +++++++++- src/researchos/paths.py | 33 ++++++- 16 files changed, 571 insertions(+), 18 deletions(-) create mode 100644 data/raw/.gitkeep create mode 100644 queries/develop/.gitkeep create mode 100644 queries/production/.gitkeep create mode 100644 scripts/setup_data.py diff --git a/.gitignore b/.gitignore index 57e8cc9..c39760f 100644 --- a/.gitignore +++ b/.gitignore @@ -18,7 +18,8 @@ venv/ # ── Data (local persistence — too large for git) ── data/chroma/ -data/raw/ +data/raw/* +!data/raw/.gitkeep *.pdf # ── IDE ── diff --git a/data/raw/.gitkeep b/data/raw/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/queries/develop/.gitkeep b/queries/develop/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/queries/production/.gitkeep b/queries/production/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/scripts/betchmark_arxiv.py b/scripts/betchmark_arxiv.py index d711602..f4780fa 100644 --- a/scripts/betchmark_arxiv.py +++ b/scripts/betchmark_arxiv.py @@ -1,3 +1,17 @@ +"""Benchmark script — Compares sequential vs. concurrent PDF download speed. + +Measures wall-clock time for downloading a batch of arXiv papers using two +strategies: sequential (one request at a time) and parallel (all requests +concurrent via ``asyncio.gather``). Results are printed to stdout. + +Usage: + uv run python scripts/betchmark_arxiv.py + +Note: + The filename contains a deliberate typo (``betchmark`` instead of + ``benchmark``) preserved from the original commit; do not rename without + also updating Makefile references. +""" import asyncio import re import time @@ -9,7 +23,17 @@ from researchos.paths import PAPERS_DIR -async def sequential_benchmark(query: str, max_results: int): +async def sequential_benchmark(query: str, max_results: int) -> None: + """Download arXiv papers sequentially and report elapsed time. + + Fetches paper metadata, then downloads each PDF one after the other + using a synchronous :class:`httpx.Client` inside an ``async`` function. + Use this as the baseline to compare against :func:`parallel_benchmark`. + + Args: + query: arXiv search query string (e.g. ``"LLM agents"``). + max_results: Number of papers to download. + """ start_time = time.perf_counter() papers = await search_papers(query, max_results) @@ -33,7 +57,17 @@ async def sequential_benchmark(query: str, max_results: int): print(f"Sequential benchmark completed in {end_time - start_time} seconds") -async def parallel_benchmark(query: str, max_results: int): +async def parallel_benchmark(query: str, max_results: int) -> None: + """Download arXiv papers concurrently and report elapsed time. + + Fetches paper metadata, then downloads all PDFs simultaneously using + :func:`asyncio.gather` and :func:`_download_one_paper`. Compare + against :func:`sequential_benchmark` to measure the concurrency speedup. + + Args: + query: arXiv search query string (e.g. ``"LLM agents"``). + max_results: Number of papers to download in parallel. + """ start_time = time.perf_counter() papers = await search_papers(query, max_results) @@ -46,7 +80,20 @@ async def parallel_benchmark(query: str, max_results: int): print(f"Parallel betchmark completed in {end_time - start_time} seconds") -async def _download_one_paper(paper: Paper): +async def _download_one_paper(paper: Paper) -> None: + """Download a single paper PDF and save it to ``PAPERS_DIR``. + + Derives the local filename from the first author's name + (lowercased, sanitised) and publication year, matching the convention + used by the ingestion service. + + Args: + paper: A :class:`~researchos.domain.models.Paper` with ``pdf_url``, + ``authors``, and ``published_date`` populated. + + Raises: + httpx.HTTPStatusError: If the download request fails. + """ url = paper.pdf_url pdf_name = paper.authors[0].lower().strip() pdf_name = re.sub(r"[^a-z0-9_]", "_", pdf_name) diff --git a/scripts/eval_retrieval.py b/scripts/eval_retrieval.py index 5d1188c..1472404 100644 --- a/scripts/eval_retrieval.py +++ b/scripts/eval_retrieval.py @@ -1,3 +1,20 @@ +"""Retrieval evaluation script — Manual sanity-check for the vector store. + +Loads a JSON evaluation dataset from ``data/samples/eval_dataset.json``, +runs each question through the :class:`ChromaVectorStore`, and prints the +top retrieved documents with their relevance scores. Useful for quickly +validating retrieval quality after changing chunk size, overlap, or the +embedding model. + +Usage: + uv run python scripts/eval_retrieval.py + +Eval dataset format (``eval_dataset.json``): + [ + {"question": "What is RAG?", "source_paper": "vaswani_2017"}, + ... + ] +""" import asyncio import json @@ -13,6 +30,21 @@ async def answer_question( question: str, store: VectorStore, max_results: int = 10 ) -> list[Document]: + """Retrieve and print the top documents for a given question. + + Searches the vector store for the most relevant chunks and prints each + result's score and a 200-character preview of the text to stdout. + + Args: + question: Natural-language question to search for. + store: A :class:`~researchos.domain.interfaces.VectorStore` + implementation (typically :class:`ChromaVectorStore`). + max_results: Maximum number of documents to retrieve. Defaults to 10. + + Returns: + List of :class:`~researchos.domain.models.Document` objects returned + by the vector store, in descending relevance order. + """ results = await store.search(query=question, k=max_results) for r in results: print(f"\nscore: {r.score:.3f}") @@ -21,6 +53,16 @@ async def answer_question( async def main() -> None: + """Run the full retrieval evaluation loop. + + Reads each question from the eval dataset, calls :func:`answer_question`, + and prints a summary showing which papers appeared in the top results + compared to the expected ``source_paper``. + + Raises: + FileNotFoundError: If ``data/samples/eval_dataset.json`` does not exist. + json.JSONDecodeError: If the dataset file is malformed. + """ with open(path_examples, encoding="utf-8") as f: data = json.load(f) diff --git a/scripts/setup_data.py b/scripts/setup_data.py new file mode 100644 index 0000000..abb4a40 --- /dev/null +++ b/scripts/setup_data.py @@ -0,0 +1,16 @@ +"""Script: Download and prepare data from corporate sources. + +Usage: + uv run python scripts/setup_data.py + +This script handles the initial data acquisition from external sources +(BigQuery, RDBMS, APIs, file shares) and saves raw data to data/raw/. +Optionally transforms and saves development samples to data/samples/. + +This is an operational script, not part of the installable package. +""" + +# TODO: Import your data access client from infrastructure/data/ +# TODO: Download raw data to data/raw/ +# TODO: Transform and save samples to data/samples/ for local development +# TODO: Log what was downloaded and where it was saved diff --git a/src/researchos/application/services/ingestion_service.py b/src/researchos/application/services/ingestion_service.py index b3b8c1e..15555d9 100644 --- a/src/researchos/application/services/ingestion_service.py +++ b/src/researchos/application/services/ingestion_service.py @@ -1,3 +1,16 @@ +"""Ingestion service — Orchestrates the end-to-end paper ingestion pipeline. + +This service coordinates three sequential steps: +1. Fetch paper metadata from arXiv via :func:`search_papers`. +2. Download and extract raw text from each PDF. +3. Chunk the text and upsert the resulting documents into the vector store. + +Design note: + Concrete infrastructure (``ChromaVectorStore``, ``LocalEmbedder``) is + instantiated here rather than injected, because ``ingest_papers`` is an + operational entry-point (called by CLI scripts), not a reusable + application-layer use-case that needs swappable dependencies. +""" import asyncio import re from pathlib import Path @@ -15,11 +28,42 @@ async def extract_text_pdf(paper: Paper) -> tuple[str, Path]: + """Download a paper's PDF and extract its full text. + + Args: + paper: A :class:`~researchos.domain.models.Paper` whose ``pdf_url`` + and ``authors`` fields are populated. + + Returns: + A tuple of ``(full_text, local_pdf_path)`` where ``full_text`` is + the concatenated text extracted from all pages, and ``local_pdf_path`` + is the path where the PDF was saved on disk. + + Raises: + IngestionError: If the PDF contains no extractable text. + httpx.HTTPStatusError: If the download request fails. + """ pdf_path = await _download_pdf(paper=paper) return _extract_text(pdf_path=pdf_path), pdf_path async def _download_pdf(paper: Paper) -> Path: + """Download the PDF for a paper and save it to ``PAPERS_DIR``. + + The local filename is derived from the first author's last name + (lowercased, non-alphanumeric characters replaced with underscores) + and the publication year (e.g. ``vaswani_2017.pdf``). + + Args: + paper: A :class:`~researchos.domain.models.Paper` with a valid + ``pdf_url``, ``authors`` list, and ``published_date``. + + Returns: + The absolute ``Path`` where the PDF was saved. + + Raises: + httpx.HTTPStatusError: If the HTTP request returns a non-2xx status. + """ url = paper.pdf_url pdf_name = paper.authors[0].lower().strip() pdf_name = re.sub(r"[^a-z0-9_]", "_", pdf_name) @@ -37,6 +81,18 @@ async def _download_pdf(paper: Paper) -> Path: def _extract_text(pdf_path: Path) -> str: + """Extract all text from a PDF file using PyMuPDF (fitz). + + Args: + pdf_path: Absolute path to the PDF file on disk. + + Returns: + Concatenated plain-text content of all pages in the PDF. + + Raises: + IngestionError: If the extracted text is empty (e.g. scanned PDF + without OCR or DRM-protected file). + """ full_text = "" doc = fitz.open(pdf_path) for page in doc: @@ -56,6 +112,32 @@ async def ingest_papers( collection_name: str = "papers", embedder_metadata: dict | None = None, ) -> None: + """Fetch, chunk, and index papers from arXiv into the vector store. + + This is the main entry-point for the ingestion pipeline. It runs + PDF downloads concurrently using :func:`asyncio.gather`, then processes + each paper serially to avoid overwhelming the embedder. + + Args: + query: arXiv search query string (e.g. ``"LLM agents"``). + max_results: Maximum number of papers to fetch from arXiv. + chunk_size: Number of characters per text chunk. Defaults to 500. + overlap: Character overlap between consecutive chunks to preserve + context across boundaries. Defaults to 50. + collection_name: Name of the ChromaDB collection to upsert into. + Defaults to ``"papers"``. + embedder_metadata: Optional HNSW metadata dict forwarded to ChromaDB + (e.g. ``{"hnsw:space": "cosine"}``). If ``None``, the + ``ChromaVectorStore`` default is used. + + Returns: + None. Side-effects: PDFs saved to ``PAPERS_DIR``, chunks upserted + into the vector store. + + Raises: + IngestionError: If any PDF cannot be downloaded or has no + extractable text. + """ embedder = LocalEmbedder() store = ChromaVectorStore( embedder=embedder, collection_name=collection_name, embedder_metadata=embedder_metadata diff --git a/src/researchos/application/services/retrieval_service.py b/src/researchos/application/services/retrieval_service.py index 1367b49..29b814f 100644 --- a/src/researchos/application/services/retrieval_service.py +++ b/src/researchos/application/services/retrieval_service.py @@ -1,9 +1,46 @@ +"""Retrieval service — Text chunking utilities for the ingestion pipeline. + +Provides pure, stateless functions for splitting paper text into overlapping +chunks and converting those chunks into the ``Document`` format expected by +the vector store. These utilities are shared by the ingestion service and +can be reused in evaluation scripts. + +Design note: + Functions here have no side-effects and no external dependencies beyond + domain models. They belong in ``application/services/`` (not ``domain/``) +because they encode an operational decision (chunk size, overlap strategy) + rather than a business concept. +""" from researchos.domain.models import Chunk, Document def overlap_chunking( text: str, paper_id: str, chunk_size: int = 500, overlap: int = 50 ) -> list[Chunk]: + """Split a text string into overlapping fixed-size chunks. + + Each chunk starts at ``chunk_size - overlap`` characters after the + previous one, so adjacent chunks share ``overlap`` characters of context. + This sliding-window approach reduces information loss at chunk boundaries. + + Args: + text: Full text content of the paper to be chunked. + paper_id: Identifier used as the prefix for each ``chunk_id`` + (e.g. the PDF stem). + chunk_size: Number of characters in each chunk. Defaults to 500. + overlap: Number of characters shared between consecutive chunks. + Defaults to 50. + + Returns: + Ordered list of :class:`~researchos.domain.models.Chunk` objects + with ``chunk_id``, ``paper_id``, ``text``, ``chunk_index``, and + position metadata (``start_char``, ``end_char``). + + Example: + >>> chunks = overlap_chunking("Hello world " * 100, "paper_abc") + >>> chunks[0].chunk_id + 'paper_abc_0' + """ chunks = [] for i, initial_car in enumerate(range(0, len(text), chunk_size)): @@ -28,6 +65,19 @@ def overlap_chunking( def chunk_to_document(chunk: Chunk) -> Document: + """Convert a :class:`~researchos.domain.models.Chunk` into a :class:`~researchos.domain.models.Document`. + + Merges the chunk's own metadata with ``paper_id`` and ``chunk_index`` + fields so that documents stored in the vector store can be traced back + to their source paper and position. + + Args: + chunk: A populated :class:`~researchos.domain.models.Chunk` object. + + Returns: + A :class:`~researchos.domain.models.Document` ready to be upserted + into the vector store, with ``doc_id == chunk.chunk_id``. + """ return Document( doc_id=chunk.chunk_id, text=chunk.text, diff --git a/src/researchos/config.py b/src/researchos/config.py index 6a2ad72..a54ec04 100644 --- a/src/researchos/config.py +++ b/src/researchos/config.py @@ -12,7 +12,28 @@ class Settings(BaseSettings): - """Application settings loaded from .env and environment variables.""" + """Application settings loaded from .env and environment variables. + + All fields are populated from the ``.env`` file at the project root or + from environment variables (case-insensitive). Each section maps to a + specific external service or runtime concern: + + - **Environment:** controls the execution mode (development / staging / production). + - **LLM:** Anthropic credentials and generation parameters. + - **Vector store:** backend selection (Chroma | Vertex) and persistence path. + - **Memory:** conversation history backend (in-memory | SQLite). + - **Embeddings:** sentence-transformer model used for embedding. + - **Telegram:** bot token and target chat for morning briefings. + - **API:** host and port for the FastAPI server. + - **News (V2):** NewsAPI key for RSS / news sources. + - **Monitoring (V3):** Langfuse credentials for LLM observability. + - **GCP (V4):** Google Cloud project for Vertex AI and BigQuery. + + Example: + >>> from researchos.config import settings + >>> print(settings.default_model) + claude-haiku-4-5-20251001 + """ model_config = SettingsConfigDict( env_file=".env", @@ -62,6 +83,14 @@ class Settings(BaseSettings): @property def project_root(self) -> Path: + """Absolute path to the repository root. + + Computed by walking three levels up from this file + (``src/researchos/config.py`` → ``src/researchos/`` → ``src/`` → root). + + Returns: + Path: Absolute ``Path`` object pointing to the project root directory. + """ return Path(__file__).resolve().parent.parent.parent diff --git a/src/researchos/domain/prompts/__init__.py b/src/researchos/domain/prompts/__init__.py index 546730b..0dee9b2 100644 --- a/src/researchos/domain/prompts/__init__.py +++ b/src/researchos/domain/prompts/__init__.py @@ -52,6 +52,18 @@ class PromptTemplate: """ def __init__(self, category: str, name: str) -> None: + """Load the prompt template file from disk. + + Args: + category: Subdirectory name under ``domain/prompts/`` + (e.g. ``"system"``, ``"tasks"``). + name: Template filename without the ``.txt`` extension + (e.g. ``"extraction"``). + + Raises: + PromptNotFoundError: If the file ``{category}/{name}.txt`` + does not exist inside the prompts directory. + """ self._path = _PROMPTS_DIR / category / f"{name}.txt" if not self._path.exists(): raise PromptNotFoundError( @@ -75,4 +87,10 @@ def render(self, **kwargs: str) -> str: return self._template.format(**kwargs) if kwargs else self._template def __repr__(self) -> str: + """Return an unambiguous string representation of the template. + + Returns: + String of the form ``PromptTemplate('/.txt')`` + relative to the prompts directory root. + """ return f"PromptTemplate('{self._path.relative_to(_PROMPTS_DIR)}')" diff --git a/src/researchos/infrastructure/data/arxiv.py b/src/researchos/infrastructure/data/arxiv.py index 439688e..e93ec76 100644 --- a/src/researchos/infrastructure/data/arxiv.py +++ b/src/researchos/infrastructure/data/arxiv.py @@ -1,3 +1,18 @@ +"""arXiv data client — Fetches paper metadata from the arXiv Atom API. + +Uses the public arXiv query API (``https://export.arxiv.org/api/query``) +to search for papers by keyword and parses the Atom XML response into +:class:`~researchos.domain.models.Paper` domain objects. + +No authentication is required. Rate limits apply: the arXiv ToS asks +callers to stay below 3 requests per second. + +Example: + >>> from researchos.infrastructure.data.arxiv import search_papers + >>> papers = await search_papers("LLM agents", max_results=5) + >>> papers[0].title + 'ReAct: Synergizing Reasoning and Acting in Language Models' +""" import xml.etree.ElementTree as ET import httpx @@ -13,6 +28,25 @@ async def search_papers(query: str, max_results: int) -> list[Paper]: + """Search arXiv for papers matching a query string. + + Sends a GET request to the arXiv Atom API and parses the response + into a list of :class:`~researchos.domain.models.Paper` objects. + + Args: + query: arXiv search query string. Supports the ``all:``, ``ti:``, + ``au:``, ``abs:`` field prefixes (e.g. ``"ti:LLM agents"``). + max_results: Maximum number of papers to return. ArXiv caps this + at 30 000; realistic values are 1–50 for ingestion runs. + + Returns: + List of :class:`~researchos.domain.models.Paper` objects populated + with title, abstract, authors, URL, PDF URL, and categories. + + Raises: + IngestionError: If the API response is not valid XML. + httpx.HTTPStatusError: If the HTTP request fails. + """ params = {"search_query": query, "start": 0, "max_results": max_results} async with httpx.AsyncClient() as client: @@ -22,6 +56,19 @@ async def search_papers(query: str, max_results: int) -> list[Paper]: def _parse_entries(results: httpx.Response) -> list[Paper]: + """Parse an arXiv Atom API response into a list of Paper objects. + + Args: + results: The raw :class:`httpx.Response` from the arXiv API. + + Returns: + List of :class:`~researchos.domain.models.Paper` objects, one per + ```` element found in the Atom feed. + + Raises: + IngestionError: If the response body does not start with ``<`` + (i.e. is not XML), which typically indicates an API error page. + """ if not results.text.strip().startswith("<"): raise IngestionError(f"arXiv returned unexpected response: {results.text[:100]}") diff --git a/src/researchos/infrastructure/llm/anthropic_llm.py b/src/researchos/infrastructure/llm/anthropic_llm.py index f441396..581c673 100644 --- a/src/researchos/infrastructure/llm/anthropic_llm.py +++ b/src/researchos/infrastructure/llm/anthropic_llm.py @@ -1,3 +1,16 @@ +"""Anthropic LLM provider — Concrete implementation of the ``LLMProvider`` Protocol. + +Wraps the ``anthropic`` async SDK to implement both one-shot generation and +token-by-token streaming. All configuration (model, temperature, max_tokens) +is read from :data:`~researchos.config.settings` so no constructor arguments +are required at call sites. + +Example: + >>> from researchos.infrastructure.llm.anthropic_llm import AnthropicLLM + >>> from researchos.domain.models import Message + >>> llm = AnthropicLLM() + >>> answer = await llm.generate([Message(role="user", content="Hello")]) +""" from collections.abc import AsyncIterator from anthropic import AsyncAnthropic @@ -8,18 +21,47 @@ class AnthropicLLM: - """Contract for Claude provider.""" + """Async Claude provider implementing the :class:`~researchos.domain.interfaces.LLMProvider` Protocol. + + Reads model configuration from :data:`~researchos.config.settings` and + delegates to the official ``anthropic`` async SDK. Supports both + single-response generation and streaming. + + Attributes: + client: Authenticated :class:`anthropic.AsyncAnthropic` instance. + model_id: Claude model identifier (e.g. ``claude-haiku-4-5-20251001``). + temperature: Sampling temperature forwarded to the API. + max_tokens: Maximum number of tokens in the generated response. + """ + + def __init__(self) -> None: + """Initialise the provider from application settings. - def __init__(self): + No arguments are required; all credentials and parameters are read + from :data:`~researchos.config.settings` (populated from ``.env``). + """ self.client = AsyncAnthropic(api_key=settings.anthropic_api_key) self.model_id = settings.default_model self.temperature = settings.temperature self.max_tokens = settings.max_tokens def _format_messages(self, messages: list[Message]) -> tuple[str | None, list[dict]]: - """ - Separa el system prompt (si existe) y formatea los mensajes - para el esquema que espera Anthropic. + """Separate the system prompt and format messages for the Anthropic API. + + The Anthropic API expects the system prompt as a top-level ``system`` + parameter rather than as an element of the ``messages`` list. This + method extracts the first ``role=="system"`` message (if any) and + converts the remaining messages to the ``{role, content}`` dict schema. + + Args: + messages: List of :class:`~researchos.domain.models.Message` + objects in the conversation so far. + + Returns: + A tuple of ``(system_prompt, formatted_messages)`` where + ``system_prompt`` is the system instruction string or ``None`` + if no system message was provided, and ``formatted_messages`` is + the list of dicts for the Anthropic ``messages`` parameter. """ system_prompt = None formatted = [] @@ -33,8 +75,24 @@ def _format_messages(self, messages: list[Message]) -> tuple[str | None, list[di return system_prompt, formatted async def generate(self, messages: list[Message]) -> str: - """Generate a response from a list of messages.""" + """Generate a complete response from the Claude API. + + Sends the conversation to the Anthropic ``messages.create`` endpoint + and returns the first text block of the response. + + Args: + messages: Ordered list of :class:`~researchos.domain.models.Message` + objects representing the conversation history. May include a + leading ``role=="system"`` message. + Returns: + The model's reply as a plain string. + + Raises: + GenerationError: If the API returns an empty ``content`` list. + anthropic.APIError: For network or API-level errors propagated from + the SDK. + """ system, formatted_msgs = self._format_messages(messages) response = await self.client.messages.create( @@ -51,8 +109,24 @@ async def generate(self, messages: list[Message]) -> str: raise GenerationError("Claude returned empty response") async def stream(self, messages: list[Message]) -> AsyncIterator[str]: - """Stream a response token by token.""" + """Stream the model response token by token. + Uses the Anthropic streaming context manager so that each text delta + is yielded immediately as it arrives. Suitable for real-time UIs + (e.g. Telegram bot with progressive message updates). + + Args: + messages: Ordered list of :class:`~researchos.domain.models.Message` + objects. Same format as :meth:`generate`. + + Yields: + Successive text fragments (tokens or token groups) from the model + response, in order. + + Raises: + anthropic.APIError: For network or API-level errors propagated from + the SDK. + """ system, formatted_msgs = self._format_messages(messages) async with self.client.messages.stream( diff --git a/src/researchos/infrastructure/retrieval/chroma.py b/src/researchos/infrastructure/retrieval/chroma.py index 6c07f4e..0b75891 100644 --- a/src/researchos/infrastructure/retrieval/chroma.py +++ b/src/researchos/infrastructure/retrieval/chroma.py @@ -1,3 +1,14 @@ +"""Chroma vector store — Concrete implementation of the ``VectorStore`` Protocol. + +Persists document embeddings to disk using ChromaDB's ``PersistentClient``. +All embedding computation is delegated to :class:`LocalEmbedder` so the store +remains agnostic to the embedding model. + +To swap Chroma for another backend (e.g. Qdrant, Vertex Search), create a new +file in ``infrastructure/retrieval/`` that implements the same four-method +interface (``search``, ``upsert``) defined in +:class:`~researchos.domain.interfaces.VectorStore`. +""" import chromadb from researchos.domain.models import Document @@ -6,12 +17,36 @@ class ChromaVectorStore: + """Persistent vector store backed by ChromaDB. + + Implements the :class:`~researchos.domain.interfaces.VectorStore` Protocol. + Embeddings are computed locally via :class:`LocalEmbedder` and stored in a + ChromaDB collection on disk at :data:`~researchos.paths.CHROMA_DIR`. + + Attributes: + embedder: The :class:`LocalEmbedder` used to vectorise queries and documents. + embedder_metadata: HNSW index configuration forwarded to the ChromaDB + collection (e.g. ``{"hnsw:space": "cosine"}``). + client: ChromaDB :class:`chromadb.PersistentClient` instance. + collection: The active ChromaDB collection. + """ def __init__( self, embedder: LocalEmbedder, collection_name: str = "papers", embedder_metadata: dict | None = None, - ): + ) -> None: + """Initialise the persistent vector store. + + Args: + embedder: A :class:`LocalEmbedder` instance used for both query + embedding and batch document embedding. + collection_name: Name of the ChromaDB collection to use or create. + Defaults to ``"papers"``. + embedder_metadata: HNSW metadata for the collection + (e.g. ``{"hnsw:space": "cosine"}``). If ``None``, defaults + to ``{"hnsw:space": "cosine"}``. + """ self.embedder = embedder self.embedder_metadata = embedder_metadata or {"hnsw:space": "cosine"} self.client = chromadb.PersistentClient(path=str(CHROMA_DIR)) @@ -20,8 +55,21 @@ def __init__( ) async def search(self, query: str, k: int) -> list[Document]: - """Search for the top-k most relevant documents.""" + """Search for the top-k most relevant documents using cosine similarity. + + Embeds the query with :class:`LocalEmbedder`, queries the ChromaDB + collection, and converts raw results to typed + :class:`~researchos.domain.models.Document` objects with normalised + relevance scores. + Args: + query: Natural-language search string. + k: Number of top documents to return. + + Returns: + List of :class:`~researchos.domain.models.Document` objects ordered + by descending relevance score (best match first). + """ query_embedding = self.embedder.embed(query) retrieved_docs = self.collection.query( query_embeddings=[query_embedding], @@ -45,8 +93,18 @@ async def search(self, query: str, k: int) -> list[Document]: return results async def upsert(self, documents: list[Document]) -> None: - """Insert or update documents in the store.""" + """Insert or update documents in the ChromaDB collection. + + Embeds all document texts in a single batch call to the embedder, + then calls ChromaDB ``upsert`` (insert-or-replace) so the operation + is idempotent: re-ingesting the same paper does not create duplicates. + Args: + documents: List of :class:`~researchos.domain.models.Document` + objects to persist. Documents with an empty ``metadata`` dict + receive a fallback ``{"source": "unknown"}`` entry to satisfy + ChromaDB's non-null constraint. + """ vectors = self.embedder.embed_batch([doc.text for doc in documents]) self.collection.upsert( @@ -59,6 +117,22 @@ async def upsert(self, documents: list[Document]) -> None: ) def _distance_to_score(self, distance: float) -> float: + """Convert a ChromaDB distance value to a [0, 1] relevance score. + + ChromaDB returns distances whose interpretation depends on the HNSW + distance space configured for the collection: + + - ``cosine``: distance ∈ [0, 2]; score = ``1 - distance / 2``. + - ``l2`` (Euclidean): distance ∈ [0, ∞); score = ``1 / (1 + distance)``. + - Anything else: score = ``1 - distance`` (assumes distance ∈ [0, 1]). + + Args: + distance: Raw distance value returned by ChromaDB. + + Returns: + Normalised relevance score where 1.0 is a perfect match and + 0.0 is maximally dissimilar. + """ space = self.embedder_metadata.get("hnsw:space", "cosine") if space == "cosine": return 1 - (distance / 2) diff --git a/src/researchos/infrastructure/retrieval/embedder.py b/src/researchos/infrastructure/retrieval/embedder.py index 6ede1a9..e4a47e3 100644 --- a/src/researchos/infrastructure/retrieval/embedder.py +++ b/src/researchos/infrastructure/retrieval/embedder.py @@ -1,13 +1,55 @@ -# infrastructure/retrieval/embedder.py +"""Local embedder — Sentence-transformer-based text embedding. + +Provides synchronous embedding of single texts and batches using a locally +downloaded ``sentence-transformers`` model (default: ``all-MiniLM-L6-v2``). +The model is downloaded on first use and cached by the ``sentence-transformers`` +library in the system's HuggingFace cache directory. +""" from sentence_transformers import SentenceTransformer class LocalEmbedder: - def __init__(self, model_name: str = "all-MiniLM-L6-v2"): + """Thin wrapper around a ``sentence-transformers`` model. + + Provides ``embed`` and ``embed_batch`` helpers used by + :class:`~researchos.infrastructure.retrieval.chroma.ChromaVectorStore` + to vectorise documents and queries. + + Attributes: + model: Loaded :class:`sentence_transformers.SentenceTransformer` instance. + """ + + def __init__(self, model_name: str = "all-MiniLM-L6-v2") -> None: + """Load the sentence-transformer model. + + Args: + model_name: Name of the model to load from HuggingFace Hub or + the local cache. Defaults to ``"all-MiniLM-L6-v2"`` + (384-dimensional embeddings, fast inference). + """ self.model = SentenceTransformer(model_name) def embed(self, text: str) -> list[float]: + """Embed a single text string into a dense vector. + + Args: + text: Input string to embed. + + Returns: + List of ``float`` values representing the embedding vector. + """ return self.model.encode(text).tolist() def embed_batch(self, texts: list[str]) -> list[list[float]]: + """Embed a list of text strings in a single forward pass. + + Batching is more efficient than calling :meth:`embed` in a loop + because the underlying model processes all texts in parallel. + + Args: + texts: List of input strings to embed. + + Returns: + List of embedding vectors in the same order as the input. + """ return self.model.encode(texts).tolist() diff --git a/src/researchos/paths.py b/src/researchos/paths.py index 7545092..f4a067e 100644 --- a/src/researchos/paths.py +++ b/src/researchos/paths.py @@ -1,3 +1,20 @@ +"""Project path constants — single source of truth for filesystem locations. + +All paths are derived from the project root discovered by ``pyprojroot`` +(looks for ``pyproject.toml``). Import these constants instead of +building paths manually to stay resilient against directory changes. + +Exported constants: + PROJECT_ROOT: Absolute path to the repository root. + DATA_DIR: Root directory for all persistent data (``data/``). + PAPERS_DIR: Downloaded raw PDF files (``data/papers/``). + SAMPLES_DIR: Evaluation datasets and sample files (``data/samples/``). + CHROMA_DIR: ChromaDB persistence directory (``data/chroma/``). + +Example: + >>> from researchos.paths import PAPERS_DIR + >>> pdf_path = PAPERS_DIR / "attention_2017.pdf" +""" from pathlib import Path import pyprojroot @@ -6,11 +23,25 @@ DATA_DIR = PROJECT_ROOT / "data" PAPERS_DIR = DATA_DIR / "papers" SAMPLES_DIR = DATA_DIR / "samples" +SCHEMAS_DIR = DATA_DIR / "schemas" +RAW_DIR = DATA_DIR / "raw" CHROMA_DIR = DATA_DIR / "chroma" def ensure_dirs() -> None: - """Create all data directories if they don't exist.""" + """Create all data directories if they do not already exist. + + Iterates over every module-level variable whose value is a ``Path`` + instance (excluding ``PROJECT_ROOT`` itself, which is not a data + directory) and calls ``mkdir(parents=True, exist_ok=True)`` on each. + + Typical call site: application startup or the ingestion script entry-point + to guarantee the directory tree is present before writing files. + + Example: + >>> from researchos.paths import ensure_dirs + >>> ensure_dirs() # Creates data/, data/papers/, data/samples/, data/chroma/ + """ dirs = [v for v in globals().values() if isinstance(v, Path) and v != PROJECT_ROOT] for d in dirs: d.mkdir(parents=True, exist_ok=True) From 3090f4ae73e161b4ae0d45975da4296189738c1d Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 08:43:48 -0500 Subject: [PATCH 30/55] feat(embedder): Add option to load local embedder from pre-dowload model --- .env.example | 5 + notebooks/007-jmmz-rag-service.ipynb | 131 ++++++++++++++++++ .../application/services/rag_service.py | 44 ++++++ src/researchos/config.py | 2 + .../domain/prompts/system/agent.txt | 4 +- .../infrastructure/retrieval/embedder.py | 21 +-- 6 files changed, 196 insertions(+), 11 deletions(-) create mode 100644 notebooks/007-jmmz-rag-service.ipynb create mode 100644 src/researchos/application/services/rag_service.py diff --git a/.env.example b/.env.example index 26158d8..21f993f 100644 --- a/.env.example +++ b/.env.example @@ -13,6 +13,11 @@ ANTHROPIC_API_KEY=sk-ant-... TELEGRAM_BOT_TOKEN= TELEGRAM_CHAT_ID= +# ── Embeddings ── +# Leave blank to download from HuggingFace. Set to a local directory path on +# networks where HuggingFace is blocked (e.g. corporate environments). +# EMBEDDING_MODEL_LOCAL_PATH=/absolute/path/to/all-MiniLM-L6-v2 + # ── Vector store ── VECTOR_STORE=chroma diff --git a/notebooks/007-jmmz-rag-service.ipynb b/notebooks/007-jmmz-rag-service.ipynb new file mode 100644 index 0000000..cf6462f --- /dev/null +++ b/notebooks/007-jmmz-rag-service.ipynb @@ -0,0 +1,131 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "id": "1785f3b5", + "metadata": {}, + "outputs": [], + "source": [ + "# Use this initial code to work in the notebook as if it were a module, that\n", + "# is, to be able to export classes and functions from other subpackages.\n", + "\n", + "import os\n", + "import sys\n", + "\n", + "package_path = os.path.abspath(\".\").split(os.sep + \"notebooks\")[0]\n", + "if package_path not in sys.path:\n", + " sys.path.append(package_path)\n", + "\n", + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "268bbb55", + "metadata": {}, + "outputs": [], + "source": [ + "from researchos.application.services.rag_service import answer_query" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "f9783658", + "metadata": {}, + "outputs": [], + "source": [ + "user_query = \"What is the primary function of the 'harness' in an externalized agent architecture?\"" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "30a65cb0", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "'[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:1016)' thrown while requesting HEAD https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/./modules.json\n", + "Retrying in 1s [Retry 1/5].\n", + "'[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure (_ssl.c:1016)' thrown while requesting HEAD https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/adapter_config.json\n", + "Retrying in 1s [Retry 1/5].\n" + ] + }, + { + "ename": "RuntimeError", + "evalue": "Cannot send a request, as the client has been closed.", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mRuntimeError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[4]\u001b[39m\u001b[32m, line 1\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m1\u001b[39m \u001b[43manswer_query\u001b[49m\u001b[43m(\u001b[49m\u001b[43muser_query\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[32;43m5\u001b[39;49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\src\\researchos\\application\\services\\rag_service.py:11\u001b[39m, in \u001b[36manswer_query\u001b[39m\u001b[34m(user_query, max_results, **kwargs)\u001b[39m\n\u001b[32m 10\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34manswer_query\u001b[39m(user_query: \u001b[38;5;28mstr\u001b[39m, max_results: \u001b[38;5;28mint\u001b[39m = \u001b[32m10\u001b[39m, **kwargs) -> \u001b[38;5;28mlist\u001b[39m[Message]:\n\u001b[32m---> \u001b[39m\u001b[32m11\u001b[39m embedder = \u001b[43mLocalEmbedder\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 12\u001b[39m chroma_store = ChromaVectorStore(embedder, **kwargs)\n\u001b[32m 14\u001b[39m docs = asyncio.run(chroma_store.search(query=user_query, k=max_results))\n", + "\u001b[36mFile \u001b[39m\u001b[32m~\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\src\\researchos\\infrastructure\\retrieval\\embedder.py:30\u001b[39m, in \u001b[36mLocalEmbedder.__init__\u001b[39m\u001b[34m(self, model_name)\u001b[39m\n\u001b[32m 22\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34m__init__\u001b[39m(\u001b[38;5;28mself\u001b[39m, model_name: \u001b[38;5;28mstr\u001b[39m = \u001b[33m\"\u001b[39m\u001b[33mall-MiniLM-L6-v2\u001b[39m\u001b[33m\"\u001b[39m) -> \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 23\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Load the sentence-transformer model.\u001b[39;00m\n\u001b[32m 24\u001b[39m \n\u001b[32m 25\u001b[39m \u001b[33;03m Args:\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 28\u001b[39m \u001b[33;03m (384-dimensional embeddings, fast inference).\u001b[39;00m\n\u001b[32m 29\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m---> \u001b[39m\u001b[32m30\u001b[39m \u001b[38;5;28mself\u001b[39m.model = \u001b[43mSentenceTransformer\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel_name\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\util\\decorators.py:41\u001b[39m, in \u001b[36mdeprecated_kwargs..decorator..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 39\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m 40\u001b[39m kwargs.pop(old_name)\n\u001b[32m---> \u001b[39m\u001b[32m41\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\sentence_transformer\\model.py:183\u001b[39m, in \u001b[36mSentenceTransformer.__init__\u001b[39m\u001b[34m(self, model_name_or_path, modules, device, prompts, default_prompt_name, cache_folder, trust_remote_code, revision, local_files_only, token, use_auth_token, model_kwargs, processor_kwargs, config_kwargs, model_card_data, backend, similarity_fn_name, truncate_dim)\u001b[39m\n\u001b[32m 178\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[32m 179\u001b[39m \u001b[33m\"\u001b[39m\u001b[33mBoth `token` and `use_auth_token` are specified. Please only specify the `token` argument.\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 180\u001b[39m )\n\u001b[32m 181\u001b[39m token = use_auth_token\n\u001b[32m--> \u001b[39m\u001b[32m183\u001b[39m \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m.\u001b[49m\u001b[34;43m__init__\u001b[39;49m\u001b[43m(\u001b[49m\n\u001b[32m 184\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 185\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodules\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodules\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 186\u001b[39m \u001b[43m \u001b[49m\u001b[43mdevice\u001b[49m\u001b[43m=\u001b[49m\u001b[43mdevice\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 187\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 188\u001b[39m \u001b[43m \u001b[49m\u001b[43mtrust_remote_code\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtrust_remote_code\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 189\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 190\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 191\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 192\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 193\u001b[39m \u001b[43m \u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 194\u001b[39m \u001b[43m \u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 195\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_card_data\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_card_data\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 196\u001b[39m \u001b[43m \u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m=\u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 197\u001b[39m \u001b[43m \u001b[49m\u001b[43mprompts\u001b[49m\u001b[43m=\u001b[49m\u001b[43mprompts\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 198\u001b[39m \u001b[43m \u001b[49m\u001b[43mdefault_prompt_name\u001b[49m\u001b[43m=\u001b[49m\u001b[43mdefault_prompt_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 199\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 200\u001b[39m \u001b[38;5;28mself\u001b[39m.model_card_data: SentenceTransformerModelCardData\n\u001b[32m 202\u001b[39m \u001b[38;5;66;03m# Handle INSTRUCTOR models\u001b[39;00m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\base\\model.py:198\u001b[39m, in \u001b[36mBaseModel.__init__\u001b[39m\u001b[34m(self, model_name_or_path, modules, device, prompts, default_prompt_name, cache_folder, trust_remote_code, revision, local_files_only, token, model_kwargs, processor_kwargs, config_kwargs, model_card_data, backend)\u001b[39m\n\u001b[32m 195\u001b[39m model_name_or_path = \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m.default_huggingface_organization\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m/\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmodel_name_or_path\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m 197\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m model_name_or_path:\n\u001b[32m--> \u001b[39m\u001b[32m198\u001b[39m modules, \u001b[38;5;28mself\u001b[39m.module_kwargs = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_load_modules\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 199\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 200\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 201\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 202\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 203\u001b[39m \u001b[43m \u001b[49m\u001b[43mtrust_remote_code\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtrust_remote_code\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 204\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 205\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 206\u001b[39m \u001b[43m \u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 207\u001b[39m \u001b[43m \u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 208\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 210\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m modules \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(modules, OrderedDict):\n\u001b[32m 211\u001b[39m modules = OrderedDict([(\u001b[38;5;28mstr\u001b[39m(idx), module) \u001b[38;5;28;01mfor\u001b[39;00m idx, module \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28menumerate\u001b[39m(modules)])\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\base\\model.py:963\u001b[39m, in \u001b[36mBaseModel._load_modules\u001b[39m\u001b[34m(self, model_name_or_path, token, cache_folder, revision, trust_remote_code, local_files_only, model_kwargs, processor_kwargs, config_kwargs)\u001b[39m\n\u001b[32m 961\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m modules_json_path \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 962\u001b[39m logger.info(\u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mNo modules.json found for \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmodel_name_or_path\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m, initializing a new \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m.model_type\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m model.\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m--> \u001b[39m\u001b[32m963\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_load_default_modules\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mload_kwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 965\u001b[39m model_type_being_loaded = \u001b[38;5;28mself\u001b[39m._get_model_type(\n\u001b[32m 966\u001b[39m model_name_or_path,\n\u001b[32m 967\u001b[39m token=token,\n\u001b[32m (...)\u001b[39m\u001b[32m 970\u001b[39m local_files_only=local_files_only,\n\u001b[32m 971\u001b[39m )\n\u001b[32m 972\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m model_type_being_loaded == \u001b[38;5;28mself\u001b[39m.model_type:\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\sentence_transformer\\model.py:1056\u001b[39m, in \u001b[36mSentenceTransformer._load_default_modules\u001b[39m\u001b[34m(self, model_name_or_path, token, cache_folder, revision, trust_remote_code, local_files_only, model_kwargs, processor_kwargs, config_kwargs)\u001b[39m\n\u001b[32m 1053\u001b[39m processor_kwargs = {**shared_kwargs} \u001b[38;5;28;01mif\u001b[39;00m processor_kwargs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m {**shared_kwargs, **processor_kwargs}\n\u001b[32m 1054\u001b[39m config_kwargs = {**shared_kwargs} \u001b[38;5;28;01mif\u001b[39;00m config_kwargs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m {**shared_kwargs, **config_kwargs}\n\u001b[32m-> \u001b[39m\u001b[32m1056\u001b[39m transformer_model = \u001b[43mTransformer\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1057\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1058\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1059\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1060\u001b[39m \u001b[43m \u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1061\u001b[39m \u001b[43m \u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1062\u001b[39m \u001b[43m \u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1063\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1064\u001b[39m modules = [transformer_model]\n\u001b[32m 1065\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m transformer_model.module_output_name == \u001b[33m\"\u001b[39m\u001b[33mtoken_embeddings\u001b[39m\u001b[33m\"\u001b[39m:\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\util\\decorators.py:87\u001b[39m, in \u001b[36mtransformer_kwargs_decorator..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 84\u001b[39m kwargs.setdefault(dict_name, {})\n\u001b[32m 85\u001b[39m kwargs[dict_name].setdefault(\u001b[33m\"\u001b[39m\u001b[33mcache_dir\u001b[39m\u001b[33m\"\u001b[39m, cache_dir)\n\u001b[32m---> \u001b[39m\u001b[32m87\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\base\\modules\\transformer.py:628\u001b[39m, in \u001b[36mTransformer.__init__\u001b[39m\u001b[34m(self, model_name_or_path, transformer_task, model_kwargs, processor_kwargs, config_kwargs, processing_kwargs, backend, modality_config, module_output_name, unpad_inputs, max_seq_length, do_lower_case, tokenizer_name_or_path)\u001b[39m\n\u001b[32m 625\u001b[39m \u001b[38;5;28mself\u001b[39m._prompt_length_mapping = {}\n\u001b[32m 626\u001b[39m \u001b[38;5;28mself\u001b[39m._method_signature_cache: \u001b[38;5;28mdict\u001b[39m[\u001b[38;5;28mstr\u001b[39m, \u001b[38;5;28mset\u001b[39m[\u001b[38;5;28mstr\u001b[39m]] = {}\n\u001b[32m--> \u001b[39m\u001b[32m628\u001b[39m config, is_peft_model = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_load_config\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 630\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m (\n\u001b[32m 631\u001b[39m transformer_task == \u001b[33m\"\u001b[39m\u001b[33msequence-classification\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 632\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[33m\"\u001b[39m\u001b[33mnum_labels\u001b[39m\u001b[33m\"\u001b[39m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;129;01min\u001b[39;00m config_kwargs\n\u001b[32m (...)\u001b[39m\u001b[32m 638\u001b[39m \u001b[38;5;66;03m# If we're loading a model for sequence-classification, but the base architecture is not for sequence-classification,\u001b[39;00m\n\u001b[32m 639\u001b[39m \u001b[38;5;66;03m# and num_labels is not specified, we default to 1 label for CrossEncoder-like behavior\u001b[39;00m\n\u001b[32m 640\u001b[39m config.num_labels = \u001b[32m1\u001b[39m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\base\\modules\\transformer.py:1354\u001b[39m, in \u001b[36mTransformer._load_config\u001b[39m\u001b[34m(self, model_name_or_path, backend, config_kwargs)\u001b[39m\n\u001b[32m 1340\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34m_load_config\u001b[39m(\n\u001b[32m 1341\u001b[39m \u001b[38;5;28mself\u001b[39m, model_name_or_path: \u001b[38;5;28mstr\u001b[39m, backend: \u001b[38;5;28mstr\u001b[39m, config_kwargs: \u001b[38;5;28mdict\u001b[39m[\u001b[38;5;28mstr\u001b[39m, Any]\n\u001b[32m 1342\u001b[39m ) -> \u001b[38;5;28mtuple\u001b[39m[PeftConfig | PretrainedConfig, \u001b[38;5;28mbool\u001b[39m]:\n\u001b[32m 1343\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Loads the transformers or PEFT configuration\u001b[39;00m\n\u001b[32m 1344\u001b[39m \n\u001b[32m 1345\u001b[39m \u001b[33;03m Args:\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 1352\u001b[39m \u001b[33;03m tuple[PeftConfig | PretrainedConfig, bool]: The model configuration and a boolean indicating whether the model is a PEFT model.\u001b[39;00m\n\u001b[32m 1353\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1354\u001b[39m adapter_config_file = \u001b[43mfind_adapter_config_file\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1355\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1356\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mcache_dir\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1357\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtoken\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1358\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mrevision\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1359\u001b[39m \u001b[43m \u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43msubfolder\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1360\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mlocal_files_only\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1361\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1362\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m adapter_config_file \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 1363\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m backend != \u001b[33m\"\u001b[39m\u001b[33mtorch\u001b[39m\u001b[33m\"\u001b[39m:\n\u001b[32m 1364\u001b[39m \u001b[38;5;66;03m# TODO: Consider following these steps automatically so we can load PEFT models with other backends\u001b[39;00m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\transformers\\utils\\peft_utils.py:84\u001b[39m, in \u001b[36mfind_adapter_config_file\u001b[39m\u001b[34m(model_id, cache_dir, force_download, proxies, token, revision, local_files_only, subfolder, _commit_hash)\u001b[39m\n\u001b[32m 82\u001b[39m adapter_cached_filename = os.path.join(model_id, ADAPTER_CONFIG_NAME)\n\u001b[32m 83\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m---> \u001b[39m\u001b[32m84\u001b[39m adapter_cached_filename = \u001b[43mcached_file\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 85\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 86\u001b[39m \u001b[43m \u001b[49m\u001b[43mADAPTER_CONFIG_NAME\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 87\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 88\u001b[39m \u001b[43m \u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m=\u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 89\u001b[39m \u001b[43m \u001b[49m\u001b[43mproxies\u001b[49m\u001b[43m=\u001b[49m\u001b[43mproxies\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 90\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 91\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 92\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 93\u001b[39m \u001b[43m \u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m=\u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 94\u001b[39m \u001b[43m \u001b[49m\u001b[43m_commit_hash\u001b[49m\u001b[43m=\u001b[49m\u001b[43m_commit_hash\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 95\u001b[39m \u001b[43m \u001b[49m\u001b[43m_raise_exceptions_for_gated_repo\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 96\u001b[39m \u001b[43m \u001b[49m\u001b[43m_raise_exceptions_for_missing_entries\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 97\u001b[39m \u001b[43m \u001b[49m\u001b[43m_raise_exceptions_for_connection_errors\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 98\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 100\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m adapter_cached_filename\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\transformers\\utils\\hub.py:278\u001b[39m, in \u001b[36mcached_file\u001b[39m\u001b[34m(path_or_repo_id, filename, **kwargs)\u001b[39m\n\u001b[32m 223\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mcached_file\u001b[39m(\n\u001b[32m 224\u001b[39m path_or_repo_id: \u001b[38;5;28mstr\u001b[39m | os.PathLike,\n\u001b[32m 225\u001b[39m filename: \u001b[38;5;28mstr\u001b[39m,\n\u001b[32m 226\u001b[39m **kwargs,\n\u001b[32m 227\u001b[39m ) -> \u001b[38;5;28mstr\u001b[39m | \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 228\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"\u001b[39;00m\n\u001b[32m 229\u001b[39m \u001b[33;03m Tries to locate a file in a local folder and repo, downloads and cache it if necessary.\u001b[39;00m\n\u001b[32m 230\u001b[39m \n\u001b[32m (...)\u001b[39m\u001b[32m 276\u001b[39m \u001b[33;03m ```\u001b[39;00m\n\u001b[32m 277\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m278\u001b[39m file = \u001b[43mcached_files\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpath_or_repo_id\u001b[49m\u001b[43m=\u001b[49m\u001b[43mpath_or_repo_id\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mfilenames\u001b[49m\u001b[43m=\u001b[49m\u001b[43m[\u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 279\u001b[39m file = file[\u001b[32m0\u001b[39m] \u001b[38;5;28;01mif\u001b[39;00m file \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m file\n\u001b[32m 280\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m file\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\transformers\\utils\\hub.py:512\u001b[39m, in \u001b[36mcached_files\u001b[39m\u001b[34m(path_or_repo_id, filenames, cache_dir, force_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_gated_repo, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, tqdm_class, **deprecated_kwargs)\u001b[39m\n\u001b[32m 509\u001b[39m \u001b[38;5;66;03m# Any other Exception type should now be re-raised, in order to provide helpful error messages and break the execution flow\u001b[39;00m\n\u001b[32m 510\u001b[39m \u001b[38;5;66;03m# (EntryNotFoundError will be treated outside this block and correctly re-raised if needed)\u001b[39;00m\n\u001b[32m 511\u001b[39m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(e, EntryNotFoundError):\n\u001b[32m--> \u001b[39m\u001b[32m512\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m e\n\u001b[32m 514\u001b[39m resolved_files = [\n\u001b[32m 515\u001b[39m _get_cache_file_to_return(path_or_repo_id, filename, cache_dir, revision) \u001b[38;5;28;01mfor\u001b[39;00m filename \u001b[38;5;129;01min\u001b[39;00m full_filenames\n\u001b[32m 516\u001b[39m ]\n\u001b[32m 517\u001b[39m \u001b[38;5;66;03m# If there are any missing file and the flag is active, raise\u001b[39;00m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\transformers\\utils\\hub.py:422\u001b[39m, in \u001b[36mcached_files\u001b[39m\u001b[34m(path_or_repo_id, filenames, cache_dir, force_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_gated_repo, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, tqdm_class, **deprecated_kwargs)\u001b[39m\n\u001b[32m 419\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m 420\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(full_filenames) == \u001b[32m1\u001b[39m:\n\u001b[32m 421\u001b[39m \u001b[38;5;66;03m# This is slightly better for only 1 file\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m422\u001b[39m \u001b[43mhf_hub_download\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 423\u001b[39m \u001b[43m \u001b[49m\u001b[43mpath_or_repo_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 424\u001b[39m \u001b[43m \u001b[49m\u001b[43mfilenames\u001b[49m\u001b[43m[\u001b[49m\u001b[32;43m0\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 425\u001b[39m \u001b[43m \u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mlen\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[43m==\u001b[49m\u001b[43m \u001b[49m\u001b[32;43m0\u001b[39;49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 426\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 427\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 428\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 429\u001b[39m \u001b[43m \u001b[49m\u001b[43muser_agent\u001b[49m\u001b[43m=\u001b[49m\u001b[43muser_agent\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 430\u001b[39m \u001b[43m \u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m=\u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 431\u001b[39m \u001b[43m \u001b[49m\u001b[43mproxies\u001b[49m\u001b[43m=\u001b[49m\u001b[43mproxies\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 432\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 433\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 434\u001b[39m \u001b[43m \u001b[49m\u001b[43mtqdm_class\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtqdm_class\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 435\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 436\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m 437\u001b[39m snapshot_download(\n\u001b[32m 438\u001b[39m path_or_repo_id,\n\u001b[32m 439\u001b[39m allow_patterns=full_filenames,\n\u001b[32m (...)\u001b[39m\u001b[32m 448\u001b[39m tqdm_class=tqdm_class,\n\u001b[32m 449\u001b[39m )\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_validators.py:88\u001b[39m, in \u001b[36mvalidate_hf_hub_args.._inner_fn\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 84\u001b[39m validate_repo_id(arg_value)\n\u001b[32m 86\u001b[39m kwargs = smoothly_deprecate_legacy_arguments(fn_name=fn.\u001b[34m__name__\u001b[39m, kwargs=kwargs)\n\u001b[32m---> \u001b[39m\u001b[32m88\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\file_download.py:997\u001b[39m, in \u001b[36mhf_hub_download\u001b[39m\u001b[34m(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, user_agent, force_download, etag_timeout, token, local_files_only, headers, endpoint, tqdm_class, dry_run)\u001b[39m\n\u001b[32m 976\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m _hf_hub_download_to_local_dir(\n\u001b[32m 977\u001b[39m \u001b[38;5;66;03m# Destination\u001b[39;00m\n\u001b[32m 978\u001b[39m local_dir=local_dir,\n\u001b[32m (...)\u001b[39m\u001b[32m 994\u001b[39m dry_run=dry_run,\n\u001b[32m 995\u001b[39m )\n\u001b[32m 996\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m997\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43m_hf_hub_download_to_cache_dir\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 998\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# Destination\u001b[39;49;00m\n\u001b[32m 999\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1000\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# File info\u001b[39;49;00m\n\u001b[32m 1001\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_id\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1002\u001b[39m \u001b[43m \u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m=\u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1003\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1004\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1005\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# HTTP info\u001b[39;49;00m\n\u001b[32m 1006\u001b[39m \u001b[43m \u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m=\u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1007\u001b[39m \u001b[43m \u001b[49m\u001b[43metag_timeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43metag_timeout\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1008\u001b[39m \u001b[43m \u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m=\u001b[49m\u001b[43mhf_headers\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1009\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1010\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# Additional options\u001b[39;49;00m\n\u001b[32m 1011\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1012\u001b[39m \u001b[43m \u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m=\u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1013\u001b[39m \u001b[43m \u001b[49m\u001b[43mtqdm_class\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtqdm_class\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1014\u001b[39m \u001b[43m \u001b[49m\u001b[43mdry_run\u001b[49m\u001b[43m=\u001b[49m\u001b[43mdry_run\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1015\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\file_download.py:1130\u001b[39m, in \u001b[36m_hf_hub_download_to_cache_dir\u001b[39m\u001b[34m(cache_dir, repo_id, filename, repo_type, revision, endpoint, etag_timeout, headers, token, local_files_only, force_download, tqdm_class, dry_run)\u001b[39m\n\u001b[32m 1124\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(head_call_error, _DEFAULT_RETRY_ON_EXCEPTIONS) \u001b[38;5;129;01mor\u001b[39;00m (\n\u001b[32m 1125\u001b[39m \u001b[38;5;28misinstance\u001b[39m(head_call_error, HfHubHTTPError)\n\u001b[32m 1126\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m head_call_error.response.status_code \u001b[38;5;129;01min\u001b[39;00m _DEFAULT_RETRY_ON_STATUS_CODES\n\u001b[32m 1127\u001b[39m ):\n\u001b[32m 1128\u001b[39m logger.info(\u001b[33m\"\u001b[39m\u001b[33mNo local file found. Retrying..\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 1129\u001b[39m (url_to_download, etag, commit_hash, expected_size, xet_file_data, head_call_error) = (\n\u001b[32m-> \u001b[39m\u001b[32m1130\u001b[39m \u001b[43m_get_metadata_or_catch_error\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1131\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_id\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1132\u001b[39m \u001b[43m \u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m=\u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1133\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1134\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1135\u001b[39m \u001b[43m \u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m=\u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1136\u001b[39m \u001b[43m \u001b[49m\u001b[43metag_timeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43m_ETAG_RETRY_TIMEOUT\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1137\u001b[39m \u001b[43m \u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m=\u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1138\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1139\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1140\u001b[39m \u001b[43m \u001b[49m\u001b[43mstorage_folder\u001b[49m\u001b[43m=\u001b[49m\u001b[43mstorage_folder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1141\u001b[39m \u001b[43m \u001b[49m\u001b[43mrelative_filename\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrelative_filename\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1142\u001b[39m \u001b[43m \u001b[49m\u001b[43mretry_on_errors\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 1143\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1144\u001b[39m )\n\u001b[32m 1146\u001b[39m \u001b[38;5;66;03m# If still error, raise\u001b[39;00m\n\u001b[32m 1147\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m head_call_error \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\file_download.py:1669\u001b[39m, in \u001b[36m_get_metadata_or_catch_error\u001b[39m\u001b[34m(repo_id, filename, repo_type, revision, endpoint, etag_timeout, headers, token, local_files_only, relative_filename, storage_folder, retry_on_errors)\u001b[39m\n\u001b[32m 1667\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m 1668\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m-> \u001b[39m\u001b[32m1669\u001b[39m metadata = \u001b[43mget_hf_file_metadata\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1670\u001b[39m \u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1671\u001b[39m \u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43metag_timeout\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1672\u001b[39m \u001b[43m \u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m=\u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1673\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1674\u001b[39m \u001b[43m \u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m=\u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1675\u001b[39m \u001b[43m \u001b[49m\u001b[43mretry_on_errors\u001b[49m\u001b[43m=\u001b[49m\u001b[43mretry_on_errors\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1676\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1677\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m RemoteEntryNotFoundError \u001b[38;5;28;01mas\u001b[39;00m http_error:\n\u001b[32m 1678\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m storage_folder \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mand\u001b[39;00m relative_filename \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 1679\u001b[39m \u001b[38;5;66;03m# Cache the non-existence of the file\u001b[39;00m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_validators.py:88\u001b[39m, in \u001b[36mvalidate_hf_hub_args.._inner_fn\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 84\u001b[39m validate_repo_id(arg_value)\n\u001b[32m 86\u001b[39m kwargs = smoothly_deprecate_legacy_arguments(fn_name=fn.\u001b[34m__name__\u001b[39m, kwargs=kwargs)\n\u001b[32m---> \u001b[39m\u001b[32m88\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\file_download.py:1591\u001b[39m, in \u001b[36mget_hf_file_metadata\u001b[39m\u001b[34m(url, token, timeout, library_name, library_version, user_agent, headers, endpoint, retry_on_errors)\u001b[39m\n\u001b[32m 1588\u001b[39m hf_headers[\u001b[33m\"\u001b[39m\u001b[33mAccept-Encoding\u001b[39m\u001b[33m\"\u001b[39m] = \u001b[33m\"\u001b[39m\u001b[33midentity\u001b[39m\u001b[33m\"\u001b[39m \u001b[38;5;66;03m# prevent any compression => we want to know the real size of the file\u001b[39;00m\n\u001b[32m 1590\u001b[39m \u001b[38;5;66;03m# Retrieve metadata\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1591\u001b[39m response = \u001b[43m_httpx_follow_relative_redirects_with_backoff\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1592\u001b[39m \u001b[43m \u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m=\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mHEAD\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m=\u001b[49m\u001b[43mhf_headers\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtimeout\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mretry_on_errors\u001b[49m\u001b[43m=\u001b[49m\u001b[43mretry_on_errors\u001b[49m\n\u001b[32m 1593\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1594\u001b[39m hf_raise_for_status(response)\n\u001b[32m 1596\u001b[39m \u001b[38;5;66;03m# Return\u001b[39;00m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_http.py:685\u001b[39m, in \u001b[36m_httpx_follow_relative_redirects_with_backoff\u001b[39m\u001b[34m(method, url, retry_on_errors, **httpx_kwargs)\u001b[39m\n\u001b[32m 680\u001b[39m no_retry_kwargs: \u001b[38;5;28mdict\u001b[39m[\u001b[38;5;28mstr\u001b[39m, Any] = (\n\u001b[32m 681\u001b[39m {} \u001b[38;5;28;01mif\u001b[39;00m retry_on_errors \u001b[38;5;28;01melse\u001b[39;00m {\u001b[33m\"\u001b[39m\u001b[33mretry_on_exceptions\u001b[39m\u001b[33m\"\u001b[39m: (), \u001b[33m\"\u001b[39m\u001b[33mretry_on_status_codes\u001b[39m\u001b[33m\"\u001b[39m: ()}\n\u001b[32m 682\u001b[39m )\n\u001b[32m 684\u001b[39m \u001b[38;5;28;01mwhile\u001b[39;00m \u001b[38;5;28;01mTrue\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m685\u001b[39m response = \u001b[43mhttp_backoff\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 686\u001b[39m \u001b[43m \u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 687\u001b[39m \u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 688\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mhttpx_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 689\u001b[39m \u001b[43m \u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 690\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mno_retry_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 691\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 692\u001b[39m hf_raise_for_status(response)\n\u001b[32m 694\u001b[39m \u001b[38;5;66;03m# Check if response is a relative redirect\u001b[39;00m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_http.py:559\u001b[39m, in \u001b[36mhttp_backoff\u001b[39m\u001b[34m(method, url, max_retries, base_wait_time, max_wait_time, retry_on_exceptions, retry_on_status_codes, **kwargs)\u001b[39m\n\u001b[32m 494\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mhttp_backoff\u001b[39m(\n\u001b[32m 495\u001b[39m method: HTTP_METHOD_T,\n\u001b[32m 496\u001b[39m url: \u001b[38;5;28mstr\u001b[39m,\n\u001b[32m (...)\u001b[39m\u001b[32m 503\u001b[39m **kwargs,\n\u001b[32m 504\u001b[39m ) -> httpx.Response:\n\u001b[32m 505\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Wrapper around httpx to retry calls on an endpoint, with exponential backoff.\u001b[39;00m\n\u001b[32m 506\u001b[39m \n\u001b[32m 507\u001b[39m \u001b[33;03m Endpoint call is retried on exceptions (ex: connection timeout, proxy error,...)\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 557\u001b[39m \u001b[33;03m > issue on [Github](https://github.com/huggingface/huggingface_hub).\u001b[39;00m\n\u001b[32m 558\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m559\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mnext\u001b[39;49m\u001b[43m(\u001b[49m\n\u001b[32m 560\u001b[39m \u001b[43m \u001b[49m\u001b[43m_http_backoff_base\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 561\u001b[39m \u001b[43m \u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 562\u001b[39m \u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 563\u001b[39m \u001b[43m \u001b[49m\u001b[43mmax_retries\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmax_retries\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 564\u001b[39m \u001b[43m \u001b[49m\u001b[43mbase_wait_time\u001b[49m\u001b[43m=\u001b[49m\u001b[43mbase_wait_time\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 565\u001b[39m \u001b[43m \u001b[49m\u001b[43mmax_wait_time\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmax_wait_time\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 566\u001b[39m \u001b[43m \u001b[49m\u001b[43mretry_on_exceptions\u001b[49m\u001b[43m=\u001b[49m\u001b[43mretry_on_exceptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 567\u001b[39m \u001b[43m \u001b[49m\u001b[43mretry_on_status_codes\u001b[49m\u001b[43m=\u001b[49m\u001b[43mretry_on_status_codes\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 568\u001b[39m \u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 569\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 570\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 571\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_http.py:467\u001b[39m, in \u001b[36m_http_backoff_base\u001b[39m\u001b[34m(method, url, max_retries, base_wait_time, max_wait_time, retry_on_exceptions, retry_on_status_codes, stream, **kwargs)\u001b[39m\n\u001b[32m 465\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m\n\u001b[32m 466\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m467\u001b[39m response = \u001b[43mclient\u001b[49m\u001b[43m.\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 468\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m _should_retry(response):\n\u001b[32m 469\u001b[39m \u001b[38;5;28;01myield\u001b[39;00m response\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\httpx\\_client.py:825\u001b[39m, in \u001b[36mClient.request\u001b[39m\u001b[34m(self, method, url, content, data, files, json, params, headers, cookies, auth, follow_redirects, timeout, extensions)\u001b[39m\n\u001b[32m 810\u001b[39m warnings.warn(message, \u001b[38;5;167;01mDeprecationWarning\u001b[39;00m, stacklevel=\u001b[32m2\u001b[39m)\n\u001b[32m 812\u001b[39m request = \u001b[38;5;28mself\u001b[39m.build_request(\n\u001b[32m 813\u001b[39m method=method,\n\u001b[32m 814\u001b[39m url=url,\n\u001b[32m (...)\u001b[39m\u001b[32m 823\u001b[39m extensions=extensions,\n\u001b[32m 824\u001b[39m )\n\u001b[32m--> \u001b[39m\u001b[32m825\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43msend\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mauth\u001b[49m\u001b[43m=\u001b[49m\u001b[43mauth\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m=\u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\httpx\\_client.py:901\u001b[39m, in \u001b[36mClient.send\u001b[39m\u001b[34m(self, request, stream, auth, follow_redirects)\u001b[39m\n\u001b[32m 887\u001b[39m \u001b[38;5;250m\u001b[39m\u001b[33;03m\"\"\"\u001b[39;00m\n\u001b[32m 888\u001b[39m \u001b[33;03mSend a request.\u001b[39;00m\n\u001b[32m 889\u001b[39m \n\u001b[32m (...)\u001b[39m\u001b[32m 898\u001b[39m \u001b[33;03m[0]: /advanced/clients/#request-instances\u001b[39;00m\n\u001b[32m 899\u001b[39m \u001b[33;03m\"\"\"\u001b[39;00m\n\u001b[32m 900\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._state == ClientState.CLOSED:\n\u001b[32m--> \u001b[39m\u001b[32m901\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mRuntimeError\u001b[39;00m(\u001b[33m\"\u001b[39m\u001b[33mCannot send a request, as the client has been closed.\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 903\u001b[39m \u001b[38;5;28mself\u001b[39m._state = ClientState.OPENED\n\u001b[32m 904\u001b[39m follow_redirects = (\n\u001b[32m 905\u001b[39m \u001b[38;5;28mself\u001b[39m.follow_redirects\n\u001b[32m 906\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(follow_redirects, UseClientDefault)\n\u001b[32m 907\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m follow_redirects\n\u001b[32m 908\u001b[39m )\n", + "\u001b[31mRuntimeError\u001b[39m: Cannot send a request, as the client has been closed." + ] + } + ], + "source": [ + "answer_query(user_query, 5)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2f36482f", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "researchos (3.11.15)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/src/researchos/application/services/rag_service.py b/src/researchos/application/services/rag_service.py new file mode 100644 index 0000000..eb7232c --- /dev/null +++ b/src/researchos/application/services/rag_service.py @@ -0,0 +1,44 @@ +import asyncio + +from researchos.infrastructure.retrieval.embedder import LocalEmbedder +from researchos.infrastructure.retrieval.chroma import ChromaVectorStore + +from researchos.infrastructure.llm.anthropic_llm import AnthropicLLM +from researchos.domain.models import Message +from researchos.domain.prompts.registry import load_prompt + +def answer_query(user_query: str, max_results: int = 10, **kwargs) -> list[Message]: + embedder = LocalEmbedder() + chroma_store = ChromaVectorStore(embedder, **kwargs) + + docs = asyncio.run(chroma_store.search(query=user_query, k=max_results)) + print(docs) + context = ''.join([doc.text for doc in docs if doc.score > 0.7]) + + system_prompt = Message( + role="system", + content=load_prompt("system", "agent") + ) + + corpus = Message( + role="user", + content=f""" + Based on the context provided below in triple backticks, answer + the following question: {user_query} + + ``` + {context} + ``` + """ + ) + + messages = [ + system_prompt, + corpus + ] + + llm_model = AnthropicLLM() + answer_model = asyncio.run(llm_model.generate(messages)) + messages.append(Message(role='assistant', content=answer_model)) + + return messages \ No newline at end of file diff --git a/src/researchos/config.py b/src/researchos/config.py index a54ec04..2898141 100644 --- a/src/researchos/config.py +++ b/src/researchos/config.py @@ -61,6 +61,8 @@ class Settings(BaseSettings): # ── Embeddings ── embedding_model: str = "all-MiniLM-L6-v2" + # Set to an absolute local path to load the model from disk (no HuggingFace needed). + embedding_model_local_path: str = "" # ── Telegram ── telegram_bot_token: str = "" diff --git a/src/researchos/domain/prompts/system/agent.txt b/src/researchos/domain/prompts/system/agent.txt index 63d8913..017fdf0 100644 --- a/src/researchos/domain/prompts/system/agent.txt +++ b/src/researchos/domain/prompts/system/agent.txt @@ -1,4 +1,6 @@ -You are ResearchOS, a research assistant that helps users understand scientific papers and stay up to date with the latest research in AI, machine learning, health, and technology. +You are ResearchOS, a research assistant that helps users understand +scientific papers and stay up to date with the latest research in AI, +machine learning, health, and technology. When answering questions: - Always cite your sources with paper titles and authors diff --git a/src/researchos/infrastructure/retrieval/embedder.py b/src/researchos/infrastructure/retrieval/embedder.py index e4a47e3..44c29ba 100644 --- a/src/researchos/infrastructure/retrieval/embedder.py +++ b/src/researchos/infrastructure/retrieval/embedder.py @@ -1,12 +1,13 @@ """Local embedder — Sentence-transformer-based text embedding. -Provides synchronous embedding of single texts and batches using a locally -downloaded ``sentence-transformers`` model (default: ``all-MiniLM-L6-v2``). -The model is downloaded on first use and cached by the ``sentence-transformers`` -library in the system's HuggingFace cache directory. +Loads from a local directory when ``EMBEDDING_MODEL_LOCAL_PATH`` is set in +the environment — useful on networks where HuggingFace is blocked. +Falls back to downloading ``EMBEDDING_MODEL`` from HuggingFace otherwise. """ from sentence_transformers import SentenceTransformer +from researchos.config import settings + class LocalEmbedder: """Thin wrapper around a ``sentence-transformers`` model. @@ -19,15 +20,15 @@ class LocalEmbedder: model: Loaded :class:`sentence_transformers.SentenceTransformer` instance. """ - def __init__(self, model_name: str = "all-MiniLM-L6-v2") -> None: + def __init__(self) -> None: """Load the sentence-transformer model. - Args: - model_name: Name of the model to load from HuggingFace Hub or - the local cache. Defaults to ``"all-MiniLM-L6-v2"`` - (384-dimensional embeddings, fast inference). + Uses ``EMBEDDING_MODEL_LOCAL_PATH`` from settings when set (offline mode). + Falls back to downloading ``EMBEDDING_MODEL`` from HuggingFace Hub. """ - self.model = SentenceTransformer(model_name) + source = settings.embedding_model_local_path or settings.embedding_model + local_only = bool(settings.embedding_model_local_path) + self.model = SentenceTransformer(source, local_files_only=local_only) def embed(self, text: str) -> list[float]: """Embed a single text string into a dense vector. From 123c6eca62a277b828c62c08a25130213b7c215b Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 09:12:22 -0500 Subject: [PATCH 31/55] feat(rag): Method to answer question usin full RAG --- .../application/services/rag_service.py | 13 +++++++ src/researchos/domain/prompts/registry.py | 37 ------------------- 2 files changed, 13 insertions(+), 37 deletions(-) create mode 100644 src/researchos/application/services/rag_service.py delete mode 100644 src/researchos/domain/prompts/registry.py diff --git a/src/researchos/application/services/rag_service.py b/src/researchos/application/services/rag_service.py new file mode 100644 index 0000000..896975a --- /dev/null +++ b/src/researchos/application/services/rag_service.py @@ -0,0 +1,13 @@ +from researchos.application.agents.agent_utils import retrieve_and_generate +from researchos.domain.interfaces import LLMProvider, VectorStore +from researchos.domain.prompts import PromptTemplate + + +async def answer_query( + query: str, + llm: LLMProvider, + store: VectorStore, + top_k: int = 5, +) -> str: + system_prompt = PromptTemplate("system", "agent").render() + return await retrieve_and_generate(query, llm, store, system_prompt, top_k) diff --git a/src/researchos/domain/prompts/registry.py b/src/researchos/domain/prompts/registry.py deleted file mode 100644 index d8d9edd..0000000 --- a/src/researchos/domain/prompts/registry.py +++ /dev/null @@ -1,37 +0,0 @@ -"""Prompt registry — Loads and renders prompt templates from .txt files. - -Prompts are stored as plain .txt files with {variable} placeholders, -rendered using Python's str.format(). No external dependencies (no Jinja2). - -Usage: - from researchos.domain.prompts.registry import load_prompt - - prompt = load_prompt("tasks", "extraction", topic="LLM agents", paper_text="...") -""" - -from pathlib import Path - -from researchos.domain.exceptions import PromptNotFoundError - -PROMPTS_DIR = Path(__file__).parent - - -def load_prompt(category: str, name: str, **kwargs: str) -> str: - """Load a prompt template from file and render variables. - - Args: - category: Subdirectory (e.g., "system", "tasks"). - name: Filename without extension (e.g., "extraction"). - **kwargs: Variables to substitute in the template. - - Returns: - Rendered prompt string. - - Raises: - PromptNotFoundError: If the template file doesn't exist. - """ - path = PROMPTS_DIR / category / f"{name}.txt" - if not path.exists(): - raise PromptNotFoundError(f"Prompt not found: {path}") - template = path.read_text(encoding="utf-8") - return template.format(**kwargs) if kwargs else template From 9a72a18891fce298f73a81916bf0329559ec4d6d Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 10:31:08 -0500 Subject: [PATCH 32/55] data(eval): update eval dataset questions --- data/samples/eval_dataset.json | 102 ++++++++++++++++----------------- 1 file changed, 51 insertions(+), 51 deletions(-) diff --git a/data/samples/eval_dataset.json b/data/samples/eval_dataset.json index 0a8bfc5..934cdf9 100644 --- a/data/samples/eval_dataset.json +++ b/data/samples/eval_dataset.json @@ -1,102 +1,102 @@ [ { - "question": "What are the three coupled dimensions of externalization in LLM agents as described by Zhou et al. (2026)?", - "reference_answer": "The three coupled dimensions are memory (externalized state), skills (externalized procedural expertise), and protocols (externalized interaction structure).", - "source_paper": "chenyu_zhou_2026.pdf" - }, - { - "question": "What is the primary function of the 'harness' in an externalized agent architecture?", - "reference_answer": "The harness is the engineering layer that coordinates memory, skills, and protocols into governed execution, providing the orchestration logic, constraints, observability, and feedback loops necessary for practical agency.", - "source_paper": "chenyu_zhou_2026.pdf" - }, - { - "question": "How does externalized memory transform the cognitive task for a Large Language Model?", - "reference_answer": "Externalization transforms a difficult internal recall problem (regenerating knowledge from latent weights) into an external recognition and retrieval problem, where the agent identifies relevant information surfaced from a persistent store.", - "source_paper": "chenyu_zhou_2026.pdf" - }, - { - "question": "In the context of skill externalization, what are the three essential components of 'procedural expertise'?", - "reference_answer": "Procedural expertise consists of operational procedures (the task skeleton), decision heuristics (rules for branching points), and normative constraints (safety and compliance boundaries).", - "source_paper": "chenyu_zhou_2026.pdf" + "question": "What is the definition of 'cognitive flow' in the context of AI-augmented reasoning systems?", + "reference_answer": "Cognitive flow is an optimal psychological state of deep focus and intrinsic motivation that occurs when the challenge of a task is perfectly matched to an individual's skill level.", + "source_paper": "dinithi_dissanayake_2025.pdf" }, { - "question": "What is the definition of 'cognitive flow' within the framework proposed by Dissanayake and Nanayakkara (2025)?", - "reference_answer": "Cognitive flow is defined as an optimal psychological state of deep focus and intrinsic motivation that occurs when the challenge of a task is perfectly balanced with an individual's skill level.", + "question": "What three key contextual factors determine the effectiveness of AI interventions for reasoning support?", + "reference_answer": "The three critical factors are the type of intervention (e.g., direct feedback vs. Socratic questioning), the timing of the intervention (knowing when the user is 'stuck'), and the scale or magnitude of the intervention.", "source_paper": "dinithi_dissanayake_2025.pdf" }, { - "question": "What are the three key contextual factors that determine the effectiveness of AI interventions for reasoning support?", - "reference_answer": "The three factors are the type of intervention (e.g., direct feedback vs. Socratic questioning), the timing of the intervention (knowing when the user is 'stuck'), and the scale or magnitude of the intervention.", + "question": "How can AI systems leverage multimodal cues to infer a user's cognitive load in real-time?", + "reference_answer": "AI systems can monitor behavioral cues such as gaze behavior (anticipation patterns), typing hesitation, gesture patterns, interaction speed, and physiological signals to dynamically adjust cognitive support.", "source_paper": "dinithi_dissanayake_2025.pdf" }, { - "question": "How can AI systems leverage multimodal cues to infer a user's cognitive load in real-time?", - "reference_answer": "AI systems can monitor behavioral cues such as gaze behavior (anticipation patterns), typing hesitation, interaction speed, and physiological signals to dynamically adjust cognitive support.", + "question": "How should an AI system adjust its intervention when a task is too easy for the user to maintain cognitive flow?", + "reference_answer": "The AI should increase cognitive demand by introducing counterarguments, critiques, or prompts that encourage deeper critical thinking and reflective thought processes.", "source_paper": "dinithi_dissanayake_2025.pdf" }, { - "question": "What is the core objective of the ILR (Interactive Learning for LLM Reasoning) framework?", - "reference_answer": "The core objective of ILR is to investigate whether multi-agent interaction during training can enhance an LLM's independent problem-solving capacity during inference, effectively internalizing insights from peer interaction.", + "question": "What are the two core components of the ILR (Interactive Learning for LLM Reasoning) framework?", + "reference_answer": "The ILR framework integrates two key components: Dynamic Interaction and Perception Calibration.", "source_paper": "hehai_lin_2025.pdf" }, { - "question": "Describe the three sequential stages of the 'Idea3' framework for agent communication.", - "reference_answer": "The Idea3 framework consists of Idea Sharing (proposing initial solutions), Idea Analysis (critically evaluating peer contributions), and Idea Fusion (synthesizing insights into a refined final answer).", + "question": "Describe the three sequential stages of the 'Idea3' framework for multi-agent communication.", + "reference_answer": "The Idea3 framework consists of Idea Sharing (proposing initial solutions), Idea Analysis (critically evaluating and reflecting on peer contributions), and Idea Fusion (synthesizing insights into a refined final answer).", "source_paper": "hehai_lin_2025.pdf" }, { - "question": "How does the ILR system decide between cooperation and competition strategies during training?", - "reference_answer": "The system utilizes Item Response Theory (IRT) to estimate the probability of the model solving a question independently based on its capability and the question's difficulty; cooperation is selected if the probability is low (Pq < 0.5), while competition is chosen otherwise.", + "question": "How does the ILR framework dynamically select between cooperation and competition strategies?", + "reference_answer": "The framework uses Item Response Theory (IRT) to estimate the probability (Pq) of the model solving a question independently based on its capability and the question's difficulty; cooperation is selected if the probability is low (Pq < 0.5), while competition is chosen otherwise.", "source_paper": "hehai_lin_2025.pdf" }, { - "question": "What is 'Perception Calibration' in the ILR framework and how is it implemented?", - "reference_answer": "Perception Calibration is an automated mechanism that integrates the reward distribution characteristics (max, min, and average scores) of one LLM into another LLM's reward function using Group Relative Policy Optimization (GRPO) to strengthen interaction cohesion.", + "question": "What is the purpose of 'Perception Calibration' in the ILR training process?", + "reference_answer": "Perception Calibration is an automated mechanism that integrates the reward distribution characteristics of one LLM into another's reward function using Group Relative Policy Optimization (GRPO), allowing models to perceive the quality of peer solutions and strengthening interaction cohesion.", "source_paper": "hehai_lin_2025.pdf" }, { - "question": "How does the DeepSeek-R1-1.5B model compare to significantly larger models like CodeLlama-13B as a discriminator for SQL tasks?", - "reference_answer": "Despite having significantly fewer parameters, DeepSeek-R1-1.5B outranks CodeLlama-13B as a discriminator, achieving higher execution accuracy and much higher classification F1 scores in text-to-SQL tasks.", - "source_paper": "md_fahim_anjum_2025.pdf" + "question": "What is the primary difference in set-level objectives between 'comparative shopping' and 'bundle shopping' in RecoAtlas?", + "reference_answer": "Comparative shopping emphasizes non-redundancy (providing credible alternatives while avoiding near-duplicates), whereas bundle shopping emphasizes complementarity (providing a coherent set of products that work together or fill different roles).", + "source_paper": "imad_aouali_2026.pdf" + }, + { + "question": "What three behavior-grounded reward models does RecoAtlas use to evaluate recommendation sets?", + "reference_answer": "The three reward models are Relevance Reward (query-item match), Complementarity Reward (item-item compatibility), and Diversity Reward (non-redundancy/substitutability).", + "source_paper": "imad_aouali_2026.pdf" + }, + { + "question": "What did the RecoAtlas study reveal about the alignment between LLM-as-a-judge scoring and behavior-grounded utility (SetHit@20)?", + "reference_answer": "They are poorly aligned; LLM judges often favor semantically plausible reports that do not necessarily capture behavior-grounded utility, and can even reverse the ranking induced by actual shopping objectives.", + "source_paper": "imad_aouali_2026.pdf" }, { - "question": "What happens to the discrimination performance of reasoning models when the test-time compute budget is increased beyond 1024 tokens?", - "reference_answer": "Increasing the token budget beyond this threshold yields diminishing returns (less than 0.4% gain), and longer outputs tend to become more repetitive and lexically redundant rather than more insightful.", + "question": "How does recommendation utility in RecoAtlas scale with model size and reasoning deliberation?", + "reference_answer": "Utility scales strongly with both model capacity and test-time reasoning; larger models benefit significantly more from deliberation to interpret tool outputs and assemble complex 20-item sets effectively.", + "source_paper": "imad_aouali_2026.pdf" + }, + { + "question": "How does the 1.5B distilled DeepSeek-R1 model compare to significantly larger models like CodeLlama-13B as a discriminator for SQL tasks?", + "reference_answer": "Despite its smaller size, DeepSeek-R1-1.5B outranks CodeLlama-13B as a discriminator, achieving higher execution accuracy and classification F1 scores by leveraging its inherent reasoning capabilities.", "source_paper": "md_fahim_anjum_2025.pdf" }, { - "question": "Is the reasoning model DeepSeek-R1 more effective as a generator or a discriminator in SQL parsing?", - "reference_answer": "DeepSeek-R1 is far more effective as a discriminator; it performs poorly as a generator, where even smaller non-reasoning models like TinyLlama-1.1B deliver higher-quality SQL outputs.", + "question": "What phenomenon was observed regarding test-time compute (token budget) for reasoning models in SQL discrimination?", + "reference_answer": "Increasing compute budget yields diminishing returns; performance peaks around 1024 tokens, beyond which additional tokens provide marginal gains (<0.4%) and often lead to repetitive, redundant outputs.", "source_paper": "md_fahim_anjum_2025.pdf" }, { - "question": "What novel method was proposed to extract soft scores from reasoning models for fine-grained ranking?", - "reference_answer": "The method prompts the model to output a final answer in a specific JSON format with the key 'correct', identifies the logits for the values 'true' or 'false', and normalizes them via a softmax function to obtain a probability score.", + "question": "Is the reasoning model DeepSeek-R1 more effective as a generator or a discriminator in text-to-SQL tasks?", + "reference_answer": "DeepSeek-R1 is far more effective as a discriminator; it performs poorly as a generator, where it is outperformed even by smaller non-reasoning models like TinyLlama-1.1B.", "source_paper": "md_fahim_anjum_2025.pdf" }, { - "question": "How is 'analogical reasoning' defined by Musker et al. (2024)?", - "reference_answer": "Analogical reasoning is defined as the capacity to identify and map structural relationships between different domains (source and target) to transfer knowledge and recognize abstract patterns.", - "source_paper": "sam_musker_2024.pdf" + "question": "What method was proposed to extract soft scores from the chain-of-thought (CoT) outputs of reasoning models?", + "reference_answer": "The method prompts the model to output a final answer in a specific JSON format, identifies the logit for the predicted value ('true' or 'false'), and normalizes it using a softmax function to obtain a probability score.", + "source_paper": "md_fahim_anjum_2025.pdf" }, { - "question": "What is 'flexible re-representation' and why is it essential for solving non-trivial analogies?", + "question": "What is 'flexible re-representation' and why is it central to analogical reasoning?", "reference_answer": "Flexible re-representation is the ability to dynamically restructure how concepts are encoded based on task-relevant features; it is essential because real-world concepts have many attributes, and the system must identify which ones are relevant to a specific mapping.", "source_paper": "sam_musker_2024.pdf" }, { - "question": "How did human subjects and LLMs differ in their response to misleading semantic structure in the Study 1 'Randoms' condition?", - "reference_answer": "Humans were able to ignore irrelevant words and switch to a strategy relying only on the target domain patterns, whereas LLMs appeared distracted by the random lexical items and showed significant performance drops.", + "question": "What specific limitation did GPT-4 demonstrate in the numeric conditions of the Semantic Content experiment in Musker et al. (2024)?", + "reference_answer": "GPT-4 failed to correctly relate the number of characters in a response to the numeric property of an object (e.g., number of legs or wheels), likely due to a deficiency in counting or numeric reasoning rather than analogical reasoning itself.", "source_paper": "sam_musker_2024.pdf" }, { - "question": "What specific limitation did GPT-4 demonstrate in the numeric conditions of the Semantic Content experiment?", - "reference_answer": "GPT-4 failed to correctly relate the number of characters in a response to the numeric property of the object (e.g., number of wheels or legs), which is attributed to a deficiency in counting or numeric reasoning rather than analogical reasoning itself.", + "question": "How do humans and LLMs differ when faced with random, unrelated words in the source domain of an analogy task?", + "reference_answer": "Humans are able to identify when the source domain lacks useful structure and employ a strategy relying only on target domain patterns, whereas LLMs appear distracted by the random lexical items and show significant performance drops.", "source_paper": "sam_musker_2024.pdf" }, { - "question": "How does the analogical reasoning performance of Claude 3 Opus compare to human performance across the studied conditions?", - "reference_answer": "Claude 3 Opus demonstrated robust performance that matched human levels across all conditions of the Semantic Content experiment and exhibited more flexibility than earlier models like GPT-4 in handling complex mapping tasks.", + "question": "Which advanced LLM demonstrated the most robust performance across all analogical reasoning conditions, matching human levels in both studies?", + "reference_answer": "Claude 3 Opus matched human performance across all conditions of both the Semantic Structure and Semantic Content experiments.", "source_paper": "sam_musker_2024.pdf" } ] From 4d7384adc5b69e311264d583e48767fc558cb050 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 10:32:26 -0500 Subject: [PATCH 33/55] docs(notebooks): reorder notebooks to reflext dependency order --- ...ipynb => 003-jmmz-retriever_service.ipynb} | 15 ++----- ...pynb => 004-jmmz-chroma-VectorStore.ipynb} | 4 +- ...ipynb => 005-jmmz-ingestion_service.ipynb} | 42 +++++++++++++++++-- 3 files changed, 43 insertions(+), 18 deletions(-) rename notebooks/{004-jmmz-retriever_service.ipynb => 003-jmmz-retriever_service.ipynb} (98%) rename notebooks/{005-jmmz-chroma-VectorStore.ipynb => 004-jmmz-chroma-VectorStore.ipynb} (99%) rename notebooks/{003-jmmz-ingestion_service.ipynb => 005-jmmz-ingestion_service.ipynb} (88%) diff --git a/notebooks/004-jmmz-retriever_service.ipynb b/notebooks/003-jmmz-retriever_service.ipynb similarity index 98% rename from notebooks/004-jmmz-retriever_service.ipynb rename to notebooks/003-jmmz-retriever_service.ipynb index b460776..70b048c 100644 --- a/notebooks/004-jmmz-retriever_service.ipynb +++ b/notebooks/003-jmmz-retriever_service.ipynb @@ -17,19 +17,10 @@ }, { "cell_type": "code", - "execution_count": 67, + "execution_count": 1, "id": "910dc7f7", "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "The autoreload extension is already loaded. To reload it, use:\n", - " %reload_ext autoreload\n" - ] - } - ], + "outputs": [], "source": [ "# Use this initial code to work in the notebook as if it were a module, that\n", "# is, to be able to export classes and functions from other subpackages.\n", @@ -333,7 +324,7 @@ ], "metadata": { "kernelspec": { - "display_name": ".venv", + "display_name": "researchos (3.11.8)", "language": "python", "name": "python3" }, diff --git a/notebooks/005-jmmz-chroma-VectorStore.ipynb b/notebooks/004-jmmz-chroma-VectorStore.ipynb similarity index 99% rename from notebooks/005-jmmz-chroma-VectorStore.ipynb rename to notebooks/004-jmmz-chroma-VectorStore.ipynb index f67305d..aea8c76 100644 --- a/notebooks/005-jmmz-chroma-VectorStore.ipynb +++ b/notebooks/004-jmmz-chroma-VectorStore.ipynb @@ -294,7 +294,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "id": "e4028544", "metadata": {}, "outputs": [ @@ -374,7 +374,7 @@ "from researchos.application.services.retrieval_service import overlap_chunking, chunk_to_document\n", "\n", "# Cargar 3 PDFs locales\n", - "entries = os.listdir(PAPERS_DIR)[:3]\n", + "entries = os.listdir(PAPERS_DIR)\n", "\n", "embedder = LocalEmbedder()\n", "store = ChromaVectorStore(embedder=embedder, collection_name=\"test_papers\")\n", diff --git a/notebooks/003-jmmz-ingestion_service.ipynb b/notebooks/005-jmmz-ingestion_service.ipynb similarity index 88% rename from notebooks/003-jmmz-ingestion_service.ipynb rename to notebooks/005-jmmz-ingestion_service.ipynb index 4594910..9fe88b0 100644 --- a/notebooks/003-jmmz-ingestion_service.ipynb +++ b/notebooks/005-jmmz-ingestion_service.ipynb @@ -30,6 +30,20 @@ "%autoreload 2" ] }, + { + "cell_type": "code", + "execution_count": 3, + "id": "aed1a38c", + "metadata": {}, + "outputs": [], + "source": [ + "# Patch sqlite3 with bundled modern version — required on Linux where system sqlite3 < 3.35.0.\n", + "# Same patch used in tests/conftest.py. Must run before any chromadb import.\n", + "if sys.platform == \"linux\":\n", + " __import__(\"pysqlite3\")\n", + " sys.modules[\"sqlite3\"] = sys.modules.pop(\"pysqlite3\")" + ] + }, { "cell_type": "code", "execution_count": null, @@ -517,7 +531,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 4, "id": "505f52b4", "metadata": {}, "outputs": [ @@ -525,13 +539,33 @@ "name": "stderr", "output_type": "stream", "text": [ + "Failed to reload module 'sqlite3' from file '/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/pysqlite3/__init__.py'\n", + "Traceback (most recent call last):\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 325, in check\n", + " superreload(m, reload, self.old_objects)\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 584, in superreload\n", + " module = reload(module)\n", + " ^^^^^^^^^^^^^^\n", + " File \"/home/.pyenv/versions/3.11.8/lib/python3.11/importlib/__init__.py\", line 148, in reload\n", + " raise ImportError(msg.format(name), name=name)\n", + "ImportError: module pysqlite3 not in sys.modules\n", + "[autoreload of sqlite3 failed: Traceback (most recent call last):\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 325, in check\n", + " superreload(m, reload, self.old_objects)\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 584, in superreload\n", + " module = reload(module)\n", + " ^^^^^^^^^^^^^^\n", + " File \"/home/.pyenv/versions/3.11.8/lib/python3.11/importlib/__init__.py\", line 148, in reload\n", + " raise ImportError(msg.format(name), name=name)\n", + "ImportError: module pysqlite3 not in sys.modules\n", + "]\n", "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "bc2bfa3c36b04e3f9a2298941a79416c", + "model_id": "7d3ff607d7c6460186486b1e8274d6bb", "version_major": 2, "version_minor": 0 }, @@ -575,7 +609,7 @@ ], "metadata": { "kernelspec": { - "display_name": ".venv", + "display_name": "researchos (3.11.8)", "language": "python", "name": "python3" }, @@ -589,7 +623,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.15" + "version": "3.11.8" } }, "nbformat": 4, From 2beffbccc02b5e39b3bb964e45c4eaab1fd098e6 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 10:32:52 -0500 Subject: [PATCH 34/55] chore(deps): add pysqlite3-binary to fix sqlite3 on Linux --- notebooks/007-jmmz-rag-service.ipynb | 171 +++++++++++++++++++-------- pyproject.toml | 1 + uv.lock | 57 +++++---- 3 files changed, 155 insertions(+), 74 deletions(-) diff --git a/notebooks/007-jmmz-rag-service.ipynb b/notebooks/007-jmmz-rag-service.ipynb index cf6462f..0e39b01 100644 --- a/notebooks/007-jmmz-rag-service.ipynb +++ b/notebooks/007-jmmz-rag-service.ipynb @@ -2,10 +2,19 @@ "cells": [ { "cell_type": "code", - "execution_count": 1, + "execution_count": 7, "id": "1785f3b5", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The autoreload extension is already loaded. To reload it, use:\n", + " %reload_ext autoreload\n" + ] + } + ], "source": [ "# Use this initial code to work in the notebook as if it were a module, that\n", "# is, to be able to export classes and functions from other subpackages.\n", @@ -23,85 +32,151 @@ }, { "cell_type": "code", - "execution_count": 2, - "id": "268bbb55", + "execution_count": 8, + "id": "15abec19", "metadata": {}, "outputs": [], + "source": [ + "# Patch sqlite3 with bundled modern version — required on Linux where system sqlite3 < 3.35.0.\n", + "# Same patch used in tests/conftest.py. Must run before any chromadb import.\n", + "if sys.platform == \"linux\":\n", + " __import__(\"pysqlite3\")\n", + " sys.modules[\"sqlite3\"] = sys.modules.pop(\"pysqlite3\")" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "268bbb55", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Failed to reload module 'sqlite3' from file '/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/pysqlite3/__init__.py'\n", + "Traceback (most recent call last):\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 325, in check\n", + " superreload(m, reload, self.old_objects)\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 584, in superreload\n", + " module = reload(module)\n", + " ^^^^^^^^^^^^^^\n", + " File \"/home/.pyenv/versions/3.11.8/lib/python3.11/importlib/__init__.py\", line 148, in reload\n", + " raise ImportError(msg.format(name), name=name)\n", + "ImportError: module pysqlite3 not in sys.modules\n", + "[autoreload of sqlite3 failed: Traceback (most recent call last):\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 325, in check\n", + " superreload(m, reload, self.old_objects)\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 584, in superreload\n", + " module = reload(module)\n", + " ^^^^^^^^^^^^^^\n", + " File \"/home/.pyenv/versions/3.11.8/lib/python3.11/importlib/__init__.py\", line 148, in reload\n", + " raise ImportError(msg.format(name), name=name)\n", + "ImportError: module pysqlite3 not in sys.modules\n", + "]\n" + ] + } + ], "source": [ "from researchos.application.services.rag_service import answer_query" ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 20, "id": "f9783658", "metadata": {}, "outputs": [], "source": [ - "user_query = \"What is the primary function of the 'harness' in an externalized agent architecture?\"" + "user_query = \"What is the ILR (Interactive Learning for LLM Reasoning) framework?\"" ] }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 23, "id": "30a65cb0", "metadata": {}, "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "0db129a6aec84e1fa95035a8c1010567", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Loading weights: 0%| | 0/103 [00:00 \u001b[39m\u001b[32m1\u001b[39m \u001b[43manswer_query\u001b[49m\u001b[43m(\u001b[49m\u001b[43muser_query\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[32;43m5\u001b[39;49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32m~\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\src\\researchos\\application\\services\\rag_service.py:11\u001b[39m, in \u001b[36manswer_query\u001b[39m\u001b[34m(user_query, max_results, **kwargs)\u001b[39m\n\u001b[32m 10\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34manswer_query\u001b[39m(user_query: \u001b[38;5;28mstr\u001b[39m, max_results: \u001b[38;5;28mint\u001b[39m = \u001b[32m10\u001b[39m, **kwargs) -> \u001b[38;5;28mlist\u001b[39m[Message]:\n\u001b[32m---> \u001b[39m\u001b[32m11\u001b[39m embedder = \u001b[43mLocalEmbedder\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 12\u001b[39m chroma_store = ChromaVectorStore(embedder, **kwargs)\n\u001b[32m 14\u001b[39m docs = asyncio.run(chroma_store.search(query=user_query, k=max_results))\n", - "\u001b[36mFile \u001b[39m\u001b[32m~\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\src\\researchos\\infrastructure\\retrieval\\embedder.py:30\u001b[39m, in \u001b[36mLocalEmbedder.__init__\u001b[39m\u001b[34m(self, model_name)\u001b[39m\n\u001b[32m 22\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34m__init__\u001b[39m(\u001b[38;5;28mself\u001b[39m, model_name: \u001b[38;5;28mstr\u001b[39m = \u001b[33m\"\u001b[39m\u001b[33mall-MiniLM-L6-v2\u001b[39m\u001b[33m\"\u001b[39m) -> \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 23\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Load the sentence-transformer model.\u001b[39;00m\n\u001b[32m 24\u001b[39m \n\u001b[32m 25\u001b[39m \u001b[33;03m Args:\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 28\u001b[39m \u001b[33;03m (384-dimensional embeddings, fast inference).\u001b[39;00m\n\u001b[32m 29\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m---> \u001b[39m\u001b[32m30\u001b[39m \u001b[38;5;28mself\u001b[39m.model = \u001b[43mSentenceTransformer\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel_name\u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\util\\decorators.py:41\u001b[39m, in \u001b[36mdeprecated_kwargs..decorator..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 39\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m 40\u001b[39m kwargs.pop(old_name)\n\u001b[32m---> \u001b[39m\u001b[32m41\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\sentence_transformer\\model.py:183\u001b[39m, in \u001b[36mSentenceTransformer.__init__\u001b[39m\u001b[34m(self, model_name_or_path, modules, device, prompts, default_prompt_name, cache_folder, trust_remote_code, revision, local_files_only, token, use_auth_token, model_kwargs, processor_kwargs, config_kwargs, model_card_data, backend, similarity_fn_name, truncate_dim)\u001b[39m\n\u001b[32m 178\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[32m 179\u001b[39m \u001b[33m\"\u001b[39m\u001b[33mBoth `token` and `use_auth_token` are specified. Please only specify the `token` argument.\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 180\u001b[39m )\n\u001b[32m 181\u001b[39m token = use_auth_token\n\u001b[32m--> \u001b[39m\u001b[32m183\u001b[39m \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m.\u001b[49m\u001b[34;43m__init__\u001b[39;49m\u001b[43m(\u001b[49m\n\u001b[32m 184\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 185\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodules\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodules\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 186\u001b[39m \u001b[43m \u001b[49m\u001b[43mdevice\u001b[49m\u001b[43m=\u001b[49m\u001b[43mdevice\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 187\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 188\u001b[39m \u001b[43m \u001b[49m\u001b[43mtrust_remote_code\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtrust_remote_code\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 189\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 190\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 191\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 192\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 193\u001b[39m \u001b[43m \u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 194\u001b[39m \u001b[43m \u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 195\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_card_data\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_card_data\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 196\u001b[39m \u001b[43m \u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m=\u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 197\u001b[39m \u001b[43m \u001b[49m\u001b[43mprompts\u001b[49m\u001b[43m=\u001b[49m\u001b[43mprompts\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 198\u001b[39m \u001b[43m \u001b[49m\u001b[43mdefault_prompt_name\u001b[49m\u001b[43m=\u001b[49m\u001b[43mdefault_prompt_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 199\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 200\u001b[39m \u001b[38;5;28mself\u001b[39m.model_card_data: SentenceTransformerModelCardData\n\u001b[32m 202\u001b[39m \u001b[38;5;66;03m# Handle INSTRUCTOR models\u001b[39;00m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\base\\model.py:198\u001b[39m, in \u001b[36mBaseModel.__init__\u001b[39m\u001b[34m(self, model_name_or_path, modules, device, prompts, default_prompt_name, cache_folder, trust_remote_code, revision, local_files_only, token, model_kwargs, processor_kwargs, config_kwargs, model_card_data, backend)\u001b[39m\n\u001b[32m 195\u001b[39m model_name_or_path = \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m.default_huggingface_organization\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m/\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmodel_name_or_path\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m\n\u001b[32m 197\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m model_name_or_path:\n\u001b[32m--> \u001b[39m\u001b[32m198\u001b[39m modules, \u001b[38;5;28mself\u001b[39m.module_kwargs = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_load_modules\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 199\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 200\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 201\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 202\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 203\u001b[39m \u001b[43m \u001b[49m\u001b[43mtrust_remote_code\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtrust_remote_code\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 204\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 205\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 206\u001b[39m \u001b[43m \u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 207\u001b[39m \u001b[43m \u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 208\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 210\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m modules \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(modules, OrderedDict):\n\u001b[32m 211\u001b[39m modules = OrderedDict([(\u001b[38;5;28mstr\u001b[39m(idx), module) \u001b[38;5;28;01mfor\u001b[39;00m idx, module \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28menumerate\u001b[39m(modules)])\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\base\\model.py:963\u001b[39m, in \u001b[36mBaseModel._load_modules\u001b[39m\u001b[34m(self, model_name_or_path, token, cache_folder, revision, trust_remote_code, local_files_only, model_kwargs, processor_kwargs, config_kwargs)\u001b[39m\n\u001b[32m 961\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m modules_json_path \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 962\u001b[39m logger.info(\u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mNo modules.json found for \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmodel_name_or_path\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m, initializing a new \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m.model_type\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m model.\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m--> \u001b[39m\u001b[32m963\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_load_default_modules\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mload_kwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 965\u001b[39m model_type_being_loaded = \u001b[38;5;28mself\u001b[39m._get_model_type(\n\u001b[32m 966\u001b[39m model_name_or_path,\n\u001b[32m 967\u001b[39m token=token,\n\u001b[32m (...)\u001b[39m\u001b[32m 970\u001b[39m local_files_only=local_files_only,\n\u001b[32m 971\u001b[39m )\n\u001b[32m 972\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m model_type_being_loaded == \u001b[38;5;28mself\u001b[39m.model_type:\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\sentence_transformer\\model.py:1056\u001b[39m, in \u001b[36mSentenceTransformer._load_default_modules\u001b[39m\u001b[34m(self, model_name_or_path, token, cache_folder, revision, trust_remote_code, local_files_only, model_kwargs, processor_kwargs, config_kwargs)\u001b[39m\n\u001b[32m 1053\u001b[39m processor_kwargs = {**shared_kwargs} \u001b[38;5;28;01mif\u001b[39;00m processor_kwargs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m {**shared_kwargs, **processor_kwargs}\n\u001b[32m 1054\u001b[39m config_kwargs = {**shared_kwargs} \u001b[38;5;28;01mif\u001b[39;00m config_kwargs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m {**shared_kwargs, **config_kwargs}\n\u001b[32m-> \u001b[39m\u001b[32m1056\u001b[39m transformer_model = \u001b[43mTransformer\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1057\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1058\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_folder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1059\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmodel_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1060\u001b[39m \u001b[43m \u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mprocessor_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1061\u001b[39m \u001b[43m \u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1062\u001b[39m \u001b[43m \u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1063\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1064\u001b[39m modules = [transformer_model]\n\u001b[32m 1065\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m transformer_model.module_output_name == \u001b[33m\"\u001b[39m\u001b[33mtoken_embeddings\u001b[39m\u001b[33m\"\u001b[39m:\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\util\\decorators.py:87\u001b[39m, in \u001b[36mtransformer_kwargs_decorator..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 84\u001b[39m kwargs.setdefault(dict_name, {})\n\u001b[32m 85\u001b[39m kwargs[dict_name].setdefault(\u001b[33m\"\u001b[39m\u001b[33mcache_dir\u001b[39m\u001b[33m\"\u001b[39m, cache_dir)\n\u001b[32m---> \u001b[39m\u001b[32m87\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\base\\modules\\transformer.py:628\u001b[39m, in \u001b[36mTransformer.__init__\u001b[39m\u001b[34m(self, model_name_or_path, transformer_task, model_kwargs, processor_kwargs, config_kwargs, processing_kwargs, backend, modality_config, module_output_name, unpad_inputs, max_seq_length, do_lower_case, tokenizer_name_or_path)\u001b[39m\n\u001b[32m 625\u001b[39m \u001b[38;5;28mself\u001b[39m._prompt_length_mapping = {}\n\u001b[32m 626\u001b[39m \u001b[38;5;28mself\u001b[39m._method_signature_cache: \u001b[38;5;28mdict\u001b[39m[\u001b[38;5;28mstr\u001b[39m, \u001b[38;5;28mset\u001b[39m[\u001b[38;5;28mstr\u001b[39m]] = {}\n\u001b[32m--> \u001b[39m\u001b[32m628\u001b[39m config, is_peft_model = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_load_config\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mbackend\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 630\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m (\n\u001b[32m 631\u001b[39m transformer_task == \u001b[33m\"\u001b[39m\u001b[33msequence-classification\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 632\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m \u001b[33m\"\u001b[39m\u001b[33mnum_labels\u001b[39m\u001b[33m\"\u001b[39m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;129;01min\u001b[39;00m config_kwargs\n\u001b[32m (...)\u001b[39m\u001b[32m 638\u001b[39m \u001b[38;5;66;03m# If we're loading a model for sequence-classification, but the base architecture is not for sequence-classification,\u001b[39;00m\n\u001b[32m 639\u001b[39m \u001b[38;5;66;03m# and num_labels is not specified, we default to 1 label for CrossEncoder-like behavior\u001b[39;00m\n\u001b[32m 640\u001b[39m config.num_labels = \u001b[32m1\u001b[39m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\sentence_transformers\\base\\modules\\transformer.py:1354\u001b[39m, in \u001b[36mTransformer._load_config\u001b[39m\u001b[34m(self, model_name_or_path, backend, config_kwargs)\u001b[39m\n\u001b[32m 1340\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34m_load_config\u001b[39m(\n\u001b[32m 1341\u001b[39m \u001b[38;5;28mself\u001b[39m, model_name_or_path: \u001b[38;5;28mstr\u001b[39m, backend: \u001b[38;5;28mstr\u001b[39m, config_kwargs: \u001b[38;5;28mdict\u001b[39m[\u001b[38;5;28mstr\u001b[39m, Any]\n\u001b[32m 1342\u001b[39m ) -> \u001b[38;5;28mtuple\u001b[39m[PeftConfig | PretrainedConfig, \u001b[38;5;28mbool\u001b[39m]:\n\u001b[32m 1343\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Loads the transformers or PEFT configuration\u001b[39;00m\n\u001b[32m 1344\u001b[39m \n\u001b[32m 1345\u001b[39m \u001b[33;03m Args:\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 1352\u001b[39m \u001b[33;03m tuple[PeftConfig | PretrainedConfig, bool]: The model configuration and a boolean indicating whether the model is a PEFT model.\u001b[39;00m\n\u001b[32m 1353\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1354\u001b[39m adapter_config_file = \u001b[43mfind_adapter_config_file\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1355\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_name_or_path\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1356\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mcache_dir\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1357\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mtoken\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1358\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mrevision\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1359\u001b[39m \u001b[43m \u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43msubfolder\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1360\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mconfig_kwargs\u001b[49m\u001b[43m.\u001b[49m\u001b[43mget\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mlocal_files_only\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1361\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1362\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m adapter_config_file \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 1363\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m backend != \u001b[33m\"\u001b[39m\u001b[33mtorch\u001b[39m\u001b[33m\"\u001b[39m:\n\u001b[32m 1364\u001b[39m \u001b[38;5;66;03m# TODO: Consider following these steps automatically so we can load PEFT models with other backends\u001b[39;00m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\transformers\\utils\\peft_utils.py:84\u001b[39m, in \u001b[36mfind_adapter_config_file\u001b[39m\u001b[34m(model_id, cache_dir, force_download, proxies, token, revision, local_files_only, subfolder, _commit_hash)\u001b[39m\n\u001b[32m 82\u001b[39m adapter_cached_filename = os.path.join(model_id, ADAPTER_CONFIG_NAME)\n\u001b[32m 83\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m---> \u001b[39m\u001b[32m84\u001b[39m adapter_cached_filename = \u001b[43mcached_file\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 85\u001b[39m \u001b[43m \u001b[49m\u001b[43mmodel_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 86\u001b[39m \u001b[43m \u001b[49m\u001b[43mADAPTER_CONFIG_NAME\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 87\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 88\u001b[39m \u001b[43m \u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m=\u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 89\u001b[39m \u001b[43m \u001b[49m\u001b[43mproxies\u001b[49m\u001b[43m=\u001b[49m\u001b[43mproxies\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 90\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 91\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 92\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 93\u001b[39m \u001b[43m \u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m=\u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 94\u001b[39m \u001b[43m \u001b[49m\u001b[43m_commit_hash\u001b[49m\u001b[43m=\u001b[49m\u001b[43m_commit_hash\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 95\u001b[39m \u001b[43m \u001b[49m\u001b[43m_raise_exceptions_for_gated_repo\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 96\u001b[39m \u001b[43m \u001b[49m\u001b[43m_raise_exceptions_for_missing_entries\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 97\u001b[39m \u001b[43m \u001b[49m\u001b[43m_raise_exceptions_for_connection_errors\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 98\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 100\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m adapter_cached_filename\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\transformers\\utils\\hub.py:278\u001b[39m, in \u001b[36mcached_file\u001b[39m\u001b[34m(path_or_repo_id, filename, **kwargs)\u001b[39m\n\u001b[32m 223\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mcached_file\u001b[39m(\n\u001b[32m 224\u001b[39m path_or_repo_id: \u001b[38;5;28mstr\u001b[39m | os.PathLike,\n\u001b[32m 225\u001b[39m filename: \u001b[38;5;28mstr\u001b[39m,\n\u001b[32m 226\u001b[39m **kwargs,\n\u001b[32m 227\u001b[39m ) -> \u001b[38;5;28mstr\u001b[39m | \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 228\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"\u001b[39;00m\n\u001b[32m 229\u001b[39m \u001b[33;03m Tries to locate a file in a local folder and repo, downloads and cache it if necessary.\u001b[39;00m\n\u001b[32m 230\u001b[39m \n\u001b[32m (...)\u001b[39m\u001b[32m 276\u001b[39m \u001b[33;03m ```\u001b[39;00m\n\u001b[32m 277\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m278\u001b[39m file = \u001b[43mcached_files\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpath_or_repo_id\u001b[49m\u001b[43m=\u001b[49m\u001b[43mpath_or_repo_id\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mfilenames\u001b[49m\u001b[43m=\u001b[49m\u001b[43m[\u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 279\u001b[39m file = file[\u001b[32m0\u001b[39m] \u001b[38;5;28;01mif\u001b[39;00m file \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m file\n\u001b[32m 280\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m file\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\transformers\\utils\\hub.py:512\u001b[39m, in \u001b[36mcached_files\u001b[39m\u001b[34m(path_or_repo_id, filenames, cache_dir, force_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_gated_repo, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, tqdm_class, **deprecated_kwargs)\u001b[39m\n\u001b[32m 509\u001b[39m \u001b[38;5;66;03m# Any other Exception type should now be re-raised, in order to provide helpful error messages and break the execution flow\u001b[39;00m\n\u001b[32m 510\u001b[39m \u001b[38;5;66;03m# (EntryNotFoundError will be treated outside this block and correctly re-raised if needed)\u001b[39;00m\n\u001b[32m 511\u001b[39m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(e, EntryNotFoundError):\n\u001b[32m--> \u001b[39m\u001b[32m512\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m e\n\u001b[32m 514\u001b[39m resolved_files = [\n\u001b[32m 515\u001b[39m _get_cache_file_to_return(path_or_repo_id, filename, cache_dir, revision) \u001b[38;5;28;01mfor\u001b[39;00m filename \u001b[38;5;129;01min\u001b[39;00m full_filenames\n\u001b[32m 516\u001b[39m ]\n\u001b[32m 517\u001b[39m \u001b[38;5;66;03m# If there are any missing file and the flag is active, raise\u001b[39;00m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\transformers\\utils\\hub.py:422\u001b[39m, in \u001b[36mcached_files\u001b[39m\u001b[34m(path_or_repo_id, filenames, cache_dir, force_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_gated_repo, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, tqdm_class, **deprecated_kwargs)\u001b[39m\n\u001b[32m 419\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m 420\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(full_filenames) == \u001b[32m1\u001b[39m:\n\u001b[32m 421\u001b[39m \u001b[38;5;66;03m# This is slightly better for only 1 file\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m422\u001b[39m \u001b[43mhf_hub_download\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 423\u001b[39m \u001b[43m \u001b[49m\u001b[43mpath_or_repo_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 424\u001b[39m \u001b[43m \u001b[49m\u001b[43mfilenames\u001b[49m\u001b[43m[\u001b[49m\u001b[32;43m0\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 425\u001b[39m \u001b[43m \u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mlen\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[43m==\u001b[49m\u001b[43m \u001b[49m\u001b[32;43m0\u001b[39;49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43msubfolder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 426\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 427\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 428\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 429\u001b[39m \u001b[43m \u001b[49m\u001b[43muser_agent\u001b[49m\u001b[43m=\u001b[49m\u001b[43muser_agent\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 430\u001b[39m \u001b[43m \u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m=\u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 431\u001b[39m \u001b[43m \u001b[49m\u001b[43mproxies\u001b[49m\u001b[43m=\u001b[49m\u001b[43mproxies\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 432\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 433\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 434\u001b[39m \u001b[43m \u001b[49m\u001b[43mtqdm_class\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtqdm_class\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 435\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 436\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m 437\u001b[39m snapshot_download(\n\u001b[32m 438\u001b[39m path_or_repo_id,\n\u001b[32m 439\u001b[39m allow_patterns=full_filenames,\n\u001b[32m (...)\u001b[39m\u001b[32m 448\u001b[39m tqdm_class=tqdm_class,\n\u001b[32m 449\u001b[39m )\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_validators.py:88\u001b[39m, in \u001b[36mvalidate_hf_hub_args.._inner_fn\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 84\u001b[39m validate_repo_id(arg_value)\n\u001b[32m 86\u001b[39m kwargs = smoothly_deprecate_legacy_arguments(fn_name=fn.\u001b[34m__name__\u001b[39m, kwargs=kwargs)\n\u001b[32m---> \u001b[39m\u001b[32m88\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\file_download.py:997\u001b[39m, in \u001b[36mhf_hub_download\u001b[39m\u001b[34m(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, user_agent, force_download, etag_timeout, token, local_files_only, headers, endpoint, tqdm_class, dry_run)\u001b[39m\n\u001b[32m 976\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m _hf_hub_download_to_local_dir(\n\u001b[32m 977\u001b[39m \u001b[38;5;66;03m# Destination\u001b[39;00m\n\u001b[32m 978\u001b[39m local_dir=local_dir,\n\u001b[32m (...)\u001b[39m\u001b[32m 994\u001b[39m dry_run=dry_run,\n\u001b[32m 995\u001b[39m )\n\u001b[32m 996\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m997\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43m_hf_hub_download_to_cache_dir\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 998\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# Destination\u001b[39;49;00m\n\u001b[32m 999\u001b[39m \u001b[43m \u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m=\u001b[49m\u001b[43mcache_dir\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1000\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# File info\u001b[39;49;00m\n\u001b[32m 1001\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_id\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1002\u001b[39m \u001b[43m \u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m=\u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1003\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1004\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1005\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# HTTP info\u001b[39;49;00m\n\u001b[32m 1006\u001b[39m \u001b[43m \u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m=\u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1007\u001b[39m \u001b[43m \u001b[49m\u001b[43metag_timeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43metag_timeout\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1008\u001b[39m \u001b[43m \u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m=\u001b[49m\u001b[43mhf_headers\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1009\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1010\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# Additional options\u001b[39;49;00m\n\u001b[32m 1011\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1012\u001b[39m \u001b[43m \u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m=\u001b[49m\u001b[43mforce_download\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1013\u001b[39m \u001b[43m \u001b[49m\u001b[43mtqdm_class\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtqdm_class\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1014\u001b[39m \u001b[43m \u001b[49m\u001b[43mdry_run\u001b[49m\u001b[43m=\u001b[49m\u001b[43mdry_run\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1015\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\file_download.py:1130\u001b[39m, in \u001b[36m_hf_hub_download_to_cache_dir\u001b[39m\u001b[34m(cache_dir, repo_id, filename, repo_type, revision, endpoint, etag_timeout, headers, token, local_files_only, force_download, tqdm_class, dry_run)\u001b[39m\n\u001b[32m 1124\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(head_call_error, _DEFAULT_RETRY_ON_EXCEPTIONS) \u001b[38;5;129;01mor\u001b[39;00m (\n\u001b[32m 1125\u001b[39m \u001b[38;5;28misinstance\u001b[39m(head_call_error, HfHubHTTPError)\n\u001b[32m 1126\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m head_call_error.response.status_code \u001b[38;5;129;01min\u001b[39;00m _DEFAULT_RETRY_ON_STATUS_CODES\n\u001b[32m 1127\u001b[39m ):\n\u001b[32m 1128\u001b[39m logger.info(\u001b[33m\"\u001b[39m\u001b[33mNo local file found. Retrying..\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 1129\u001b[39m (url_to_download, etag, commit_hash, expected_size, xet_file_data, head_call_error) = (\n\u001b[32m-> \u001b[39m\u001b[32m1130\u001b[39m \u001b[43m_get_metadata_or_catch_error\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1131\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_id\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1132\u001b[39m \u001b[43m \u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m=\u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1133\u001b[39m \u001b[43m \u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrepo_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1134\u001b[39m \u001b[43m \u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrevision\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1135\u001b[39m \u001b[43m \u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m=\u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1136\u001b[39m \u001b[43m \u001b[49m\u001b[43metag_timeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43m_ETAG_RETRY_TIMEOUT\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1137\u001b[39m \u001b[43m \u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m=\u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1138\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1139\u001b[39m \u001b[43m \u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m=\u001b[49m\u001b[43mlocal_files_only\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1140\u001b[39m \u001b[43m \u001b[49m\u001b[43mstorage_folder\u001b[49m\u001b[43m=\u001b[49m\u001b[43mstorage_folder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1141\u001b[39m \u001b[43m \u001b[49m\u001b[43mrelative_filename\u001b[49m\u001b[43m=\u001b[49m\u001b[43mrelative_filename\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1142\u001b[39m \u001b[43m \u001b[49m\u001b[43mretry_on_errors\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 1143\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1144\u001b[39m )\n\u001b[32m 1146\u001b[39m \u001b[38;5;66;03m# If still error, raise\u001b[39;00m\n\u001b[32m 1147\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m head_call_error \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\file_download.py:1669\u001b[39m, in \u001b[36m_get_metadata_or_catch_error\u001b[39m\u001b[34m(repo_id, filename, repo_type, revision, endpoint, etag_timeout, headers, token, local_files_only, relative_filename, storage_folder, retry_on_errors)\u001b[39m\n\u001b[32m 1667\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m 1668\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m-> \u001b[39m\u001b[32m1669\u001b[39m metadata = \u001b[43mget_hf_file_metadata\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1670\u001b[39m \u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1671\u001b[39m \u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43metag_timeout\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1672\u001b[39m \u001b[43m \u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m=\u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1673\u001b[39m \u001b[43m \u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtoken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1674\u001b[39m \u001b[43m \u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m=\u001b[49m\u001b[43mendpoint\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1675\u001b[39m \u001b[43m \u001b[49m\u001b[43mretry_on_errors\u001b[49m\u001b[43m=\u001b[49m\u001b[43mretry_on_errors\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1676\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1677\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m RemoteEntryNotFoundError \u001b[38;5;28;01mas\u001b[39;00m http_error:\n\u001b[32m 1678\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m storage_folder \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mand\u001b[39;00m relative_filename \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 1679\u001b[39m \u001b[38;5;66;03m# Cache the non-existence of the file\u001b[39;00m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_validators.py:88\u001b[39m, in \u001b[36mvalidate_hf_hub_args.._inner_fn\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 84\u001b[39m validate_repo_id(arg_value)\n\u001b[32m 86\u001b[39m kwargs = smoothly_deprecate_legacy_arguments(fn_name=fn.\u001b[34m__name__\u001b[39m, kwargs=kwargs)\n\u001b[32m---> \u001b[39m\u001b[32m88\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\file_download.py:1591\u001b[39m, in \u001b[36mget_hf_file_metadata\u001b[39m\u001b[34m(url, token, timeout, library_name, library_version, user_agent, headers, endpoint, retry_on_errors)\u001b[39m\n\u001b[32m 1588\u001b[39m hf_headers[\u001b[33m\"\u001b[39m\u001b[33mAccept-Encoding\u001b[39m\u001b[33m\"\u001b[39m] = \u001b[33m\"\u001b[39m\u001b[33midentity\u001b[39m\u001b[33m\"\u001b[39m \u001b[38;5;66;03m# prevent any compression => we want to know the real size of the file\u001b[39;00m\n\u001b[32m 1590\u001b[39m \u001b[38;5;66;03m# Retrieve metadata\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1591\u001b[39m response = \u001b[43m_httpx_follow_relative_redirects_with_backoff\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1592\u001b[39m \u001b[43m \u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m=\u001b[49m\u001b[33;43m\"\u001b[39;49m\u001b[33;43mHEAD\u001b[39;49m\u001b[33;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mheaders\u001b[49m\u001b[43m=\u001b[49m\u001b[43mhf_headers\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtimeout\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mretry_on_errors\u001b[49m\u001b[43m=\u001b[49m\u001b[43mretry_on_errors\u001b[49m\n\u001b[32m 1593\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1594\u001b[39m hf_raise_for_status(response)\n\u001b[32m 1596\u001b[39m \u001b[38;5;66;03m# Return\u001b[39;00m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_http.py:685\u001b[39m, in \u001b[36m_httpx_follow_relative_redirects_with_backoff\u001b[39m\u001b[34m(method, url, retry_on_errors, **httpx_kwargs)\u001b[39m\n\u001b[32m 680\u001b[39m no_retry_kwargs: \u001b[38;5;28mdict\u001b[39m[\u001b[38;5;28mstr\u001b[39m, Any] = (\n\u001b[32m 681\u001b[39m {} \u001b[38;5;28;01mif\u001b[39;00m retry_on_errors \u001b[38;5;28;01melse\u001b[39;00m {\u001b[33m\"\u001b[39m\u001b[33mretry_on_exceptions\u001b[39m\u001b[33m\"\u001b[39m: (), \u001b[33m\"\u001b[39m\u001b[33mretry_on_status_codes\u001b[39m\u001b[33m\"\u001b[39m: ()}\n\u001b[32m 682\u001b[39m )\n\u001b[32m 684\u001b[39m \u001b[38;5;28;01mwhile\u001b[39;00m \u001b[38;5;28;01mTrue\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m685\u001b[39m response = \u001b[43mhttp_backoff\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 686\u001b[39m \u001b[43m \u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 687\u001b[39m \u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 688\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mhttpx_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 689\u001b[39m \u001b[43m \u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 690\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mno_retry_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 691\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 692\u001b[39m hf_raise_for_status(response)\n\u001b[32m 694\u001b[39m \u001b[38;5;66;03m# Check if response is a relative redirect\u001b[39;00m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_http.py:559\u001b[39m, in \u001b[36mhttp_backoff\u001b[39m\u001b[34m(method, url, max_retries, base_wait_time, max_wait_time, retry_on_exceptions, retry_on_status_codes, **kwargs)\u001b[39m\n\u001b[32m 494\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mhttp_backoff\u001b[39m(\n\u001b[32m 495\u001b[39m method: HTTP_METHOD_T,\n\u001b[32m 496\u001b[39m url: \u001b[38;5;28mstr\u001b[39m,\n\u001b[32m (...)\u001b[39m\u001b[32m 503\u001b[39m **kwargs,\n\u001b[32m 504\u001b[39m ) -> httpx.Response:\n\u001b[32m 505\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Wrapper around httpx to retry calls on an endpoint, with exponential backoff.\u001b[39;00m\n\u001b[32m 506\u001b[39m \n\u001b[32m 507\u001b[39m \u001b[33;03m Endpoint call is retried on exceptions (ex: connection timeout, proxy error,...)\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 557\u001b[39m \u001b[33;03m > issue on [Github](https://github.com/huggingface/huggingface_hub).\u001b[39;00m\n\u001b[32m 558\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m559\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mnext\u001b[39;49m\u001b[43m(\u001b[49m\n\u001b[32m 560\u001b[39m \u001b[43m \u001b[49m\u001b[43m_http_backoff_base\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 561\u001b[39m \u001b[43m \u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 562\u001b[39m \u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 563\u001b[39m \u001b[43m \u001b[49m\u001b[43mmax_retries\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmax_retries\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 564\u001b[39m \u001b[43m \u001b[49m\u001b[43mbase_wait_time\u001b[49m\u001b[43m=\u001b[49m\u001b[43mbase_wait_time\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 565\u001b[39m \u001b[43m \u001b[49m\u001b[43mmax_wait_time\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmax_wait_time\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 566\u001b[39m \u001b[43m \u001b[49m\u001b[43mretry_on_exceptions\u001b[49m\u001b[43m=\u001b[49m\u001b[43mretry_on_exceptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 567\u001b[39m \u001b[43m \u001b[49m\u001b[43mretry_on_status_codes\u001b[49m\u001b[43m=\u001b[49m\u001b[43mretry_on_status_codes\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 568\u001b[39m \u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m=\u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[32m 569\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 570\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 571\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\huggingface_hub\\utils\\_http.py:467\u001b[39m, in \u001b[36m_http_backoff_base\u001b[39m\u001b[34m(method, url, max_retries, base_wait_time, max_wait_time, retry_on_exceptions, retry_on_status_codes, stream, **kwargs)\u001b[39m\n\u001b[32m 465\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m\n\u001b[32m 466\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m467\u001b[39m response = \u001b[43mclient\u001b[49m\u001b[43m.\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m=\u001b[49m\u001b[43mmethod\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43murl\u001b[49m\u001b[43m=\u001b[49m\u001b[43murl\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 468\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m _should_retry(response):\n\u001b[32m 469\u001b[39m \u001b[38;5;28;01myield\u001b[39;00m response\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\httpx\\_client.py:825\u001b[39m, in \u001b[36mClient.request\u001b[39m\u001b[34m(self, method, url, content, data, files, json, params, headers, cookies, auth, follow_redirects, timeout, extensions)\u001b[39m\n\u001b[32m 810\u001b[39m warnings.warn(message, \u001b[38;5;167;01mDeprecationWarning\u001b[39;00m, stacklevel=\u001b[32m2\u001b[39m)\n\u001b[32m 812\u001b[39m request = \u001b[38;5;28mself\u001b[39m.build_request(\n\u001b[32m 813\u001b[39m method=method,\n\u001b[32m 814\u001b[39m url=url,\n\u001b[32m (...)\u001b[39m\u001b[32m 823\u001b[39m extensions=extensions,\n\u001b[32m 824\u001b[39m )\n\u001b[32m--> \u001b[39m\u001b[32m825\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43msend\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrequest\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mauth\u001b[49m\u001b[43m=\u001b[49m\u001b[43mauth\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m=\u001b[49m\u001b[43mfollow_redirects\u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32mc:\\Users\\JMONTOYA\\Documents\\personal_projects\\becomes-ai-engineer\\researchos\\.venv\\Lib\\site-packages\\httpx\\_client.py:901\u001b[39m, in \u001b[36mClient.send\u001b[39m\u001b[34m(self, request, stream, auth, follow_redirects)\u001b[39m\n\u001b[32m 887\u001b[39m \u001b[38;5;250m\u001b[39m\u001b[33;03m\"\"\"\u001b[39;00m\n\u001b[32m 888\u001b[39m \u001b[33;03mSend a request.\u001b[39;00m\n\u001b[32m 889\u001b[39m \n\u001b[32m (...)\u001b[39m\u001b[32m 898\u001b[39m \u001b[33;03m[0]: /advanced/clients/#request-instances\u001b[39;00m\n\u001b[32m 899\u001b[39m \u001b[33;03m\"\"\"\u001b[39;00m\n\u001b[32m 900\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._state == ClientState.CLOSED:\n\u001b[32m--> \u001b[39m\u001b[32m901\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mRuntimeError\u001b[39;00m(\u001b[33m\"\u001b[39m\u001b[33mCannot send a request, as the client has been closed.\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 903\u001b[39m \u001b[38;5;28mself\u001b[39m._state = ClientState.OPENED\n\u001b[32m 904\u001b[39m follow_redirects = (\n\u001b[32m 905\u001b[39m \u001b[38;5;28mself\u001b[39m.follow_redirects\n\u001b[32m 906\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(follow_redirects, UseClientDefault)\n\u001b[32m 907\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m follow_redirects\n\u001b[32m 908\u001b[39m )\n", - "\u001b[31mRuntimeError\u001b[39m: Cannot send a request, as the client has been closed." + "name": "stdout", + "output_type": "stream", + "text": [ + "# Interactive Learning for LLM Reasoning (ILR)\n", + "\n", + "Based on the paper excerpts provided, **ILR is a co-learning framework designed to enhance Large Language Models' independent reasoning capabilities through multi-agent interaction**.\n", + "\n", + "## Key Characteristics\n", + "\n", + "**Core Innovation:**\n", + "ILR addresses a gap in existing multi-agent learning approaches. While traditional multi-agent systems require re-executing multiple agents during inference, ILR enables individual LLMs to internalize insights from peer interactions and solve problems independently afterward—mirroring how humans improve through discussion but later reason autonomously.\n", + "\n", + "**Two Main Components:**\n", + "1. **Dynamic Interaction** - Simulates human discussion and peer collaboration during training\n", + "2. **Perception Calibration** - Helps models reflect on and learn from peer solutions\n", + "\n", + "## Performance Results\n", + "\n", + "According to the paper, ILR demonstrates:\n", + "- Consistent outperformance over both single-agent and multi-agent learning baselines\n", + "- Significant improvements on complex reasoning tasks (particularly on competition-level datasets like AIME24&25)\n", + "- Enhanced robustness of stronger LLMs during inference\n", + "- Benefits for both weaker and stronger models\n", + "\n", + "The framework was evaluated across **nine benchmarks** spanning mathematical reasoning, coding, general question answering, and scientific reasoning.\n", + "\n", + "## Source\n", + "**Paper:** \"Interactive Learning for LLM Reasoning\" by Hehai Lin, Shilei Cao, Sudong Wang, and colleagues from The Hong Kong University of Science and Technology (Guangzhou) and collaborating institutions. Code is available on GitHub." ] } ], "source": [ - "answer_query(user_query, 5)" + "from researchos.infrastructure.llm.anthropic_llm import AnthropicLLM\n", + "from researchos.infrastructure.retrieval.embedder import LocalEmbedder\n", + "from researchos.infrastructure.retrieval.chroma import ChromaVectorStore\n", + "\n", + "llm = AnthropicLLM()\n", + "embedder = LocalEmbedder()\n", + "chorma_store = ChromaVectorStore(\n", + " embedder=embedder, collection_name='papers'\n", + ")\n", + "\n", + "\n", + "answer = await answer_query(query=user_query, llm=llm, store=chorma_store, top_k=20)\n", + "\n", + "print(answer, end=\"\")" ] }, { "cell_type": "code", "execution_count": null, - "id": "2f36482f", + "id": "6f71f38d", "metadata": {}, "outputs": [], "source": [] @@ -109,7 +184,7 @@ ], "metadata": { "kernelspec": { - "display_name": "researchos (3.11.15)", + "display_name": "researchos (3.11.8)", "language": "python", "name": "python3" }, @@ -123,7 +198,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.15" + "version": "3.11.8" } }, "nbformat": 4, diff --git a/pyproject.toml b/pyproject.toml index cf31574..dd65241 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -25,6 +25,7 @@ dependencies = [ "rank-bm25>=0.2.2", "pyprojroot>=0.3.0", "pysqlite3-binary>=0.5.4; sys_platform == 'linux'", + "pysqlite3-binary>=0.5.4.post2", ] [build-system] diff --git a/uv.lock b/uv.lock index 6aaf6df..1f48663 100644 --- a/uv.lock +++ b/uv.lock @@ -2,8 +2,10 @@ version = 1 revision = 3 requires-python = ">=3.11, <3.13" resolution-markers = [ - "python_full_version >= '3.12'", - "python_full_version < '3.12'", + "python_full_version >= '3.12' and sys_platform == 'linux'", + "python_full_version >= '3.12' and sys_platform != 'linux'", + "python_full_version < '3.12' and sys_platform == 'linux'", + "python_full_version < '3.12' and sys_platform != 'linux'", ] [[package]] @@ -224,7 +226,7 @@ name = "build" version = "1.4.3" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "colorama", marker = "os_name == 'nt'" }, + { name = "colorama", marker = "os_name == 'nt' and sys_platform != 'linux'" }, { name = "packaging" }, { name = "pyproject-hooks" }, ] @@ -405,7 +407,7 @@ name = "cuda-bindings" version = "13.2.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "cuda-pathfinder" }, + { name = "cuda-pathfinder", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/e0/a9/3a8241c6e19483ac1f1dcf5c10238205dcb8a6e9d0d4d4709240dff28ff4/cuda_bindings-13.2.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:721104c603f059780d287969be3d194a18d0cc3b713ed9049065a1107706759d", size = 5730273, upload-time = "2026-03-11T00:12:37.18Z" }, @@ -432,37 +434,37 @@ wheels = [ [package.optional-dependencies] cublas = [ - { name = "nvidia-cublas", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-cublas", marker = "sys_platform == 'linux'" }, ] cudart = [ - { name = "nvidia-cuda-runtime", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-cuda-runtime", marker = "sys_platform == 'linux'" }, ] cufft = [ - { name = "nvidia-cufft", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-cufft", marker = "sys_platform == 'linux'" }, ] cufile = [ { name = "nvidia-cufile", marker = "sys_platform == 'linux'" }, ] cupti = [ - { name = "nvidia-cuda-cupti", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-cuda-cupti", marker = "sys_platform == 'linux'" }, ] curand = [ - { name = "nvidia-curand", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-curand", marker = "sys_platform == 'linux'" }, ] cusolver = [ - { name = "nvidia-cusolver", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-cusolver", marker = "sys_platform == 'linux'" }, ] cusparse = [ - { name = "nvidia-cusparse", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-cusparse", marker = "sys_platform == 'linux'" }, ] nvjitlink = [ - { name = "nvidia-nvjitlink", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-nvjitlink", marker = "sys_platform == 'linux'" }, ] nvrtc = [ - { name = "nvidia-cuda-nvrtc", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-cuda-nvrtc", marker = "sys_platform == 'linux'" }, ] nvtx = [ - { name = "nvidia-nvtx", marker = "sys_platform == 'linux' or sys_platform == 'win32'" }, + { name = "nvidia-nvtx", marker = "sys_platform == 'linux'" }, ] [[package]] @@ -821,7 +823,8 @@ name = "ipython" version = "9.10.1" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version < '3.12'", + "python_full_version < '3.12' and sys_platform == 'linux'", + "python_full_version < '3.12' and sys_platform != 'linux'", ] dependencies = [ { name = "colorama", marker = "python_full_version < '3.12' and sys_platform == 'win32'" }, @@ -846,7 +849,8 @@ name = "ipython" version = "9.12.0" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version >= '3.12'", + "python_full_version >= '3.12' and sys_platform == 'linux'", + "python_full_version >= '3.12' and sys_platform != 'linux'", ] dependencies = [ { name = "colorama", marker = "python_full_version >= '3.12' and sys_platform == 'win32'" }, @@ -1155,7 +1159,7 @@ dependencies = [ { name = "overrides", marker = "python_full_version < '3.12'" }, { name = "packaging" }, { name = "prometheus-client" }, - { name = "pywinpty", marker = "os_name == 'nt'" }, + { name = "pywinpty", marker = "os_name == 'nt' and sys_platform != 'linux'" }, { name = "pyzmq" }, { name = "send2trash" }, { name = "terminado" }, @@ -1173,7 +1177,7 @@ name = "jupyter-server-terminals" version = "0.5.4" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "pywinpty", marker = "os_name == 'nt'" }, + { name = "pywinpty", marker = "os_name == 'nt' and sys_platform != 'linux'" }, { name = "terminado" }, ] sdist = { url = "https://files.pythonhosted.org/packages/f4/a7/bcd0a9b0cbba88986fe944aaaf91bfda603e5a50bda8ed15123f381a3b2f/jupyter_server_terminals-0.5.4.tar.gz", hash = "sha256:bbda128ed41d0be9020349f9f1f2a4ab9952a73ed5f5ac9f1419794761fb87f5", size = 31770, upload-time = "2026-01-14T16:53:20.213Z" } @@ -1651,7 +1655,7 @@ name = "nvidia-cudnn-cu13" version = "9.19.0.56" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas" }, + { name = "nvidia-cublas", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/f1/84/26025437c1e6b61a707442184fa0c03d083b661adf3a3eecfd6d21677740/nvidia_cudnn_cu13-9.19.0.56-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:6ed29ffaee1176c612daf442e4dd6cfeb6a0caa43ddcbeb59da94953030b1be4", size = 433781201, upload-time = "2026-02-03T20:40:53.805Z" }, @@ -1663,7 +1667,7 @@ name = "nvidia-cufft" version = "12.0.0.61" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink" }, + { name = "nvidia-nvjitlink", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/8b/ae/f417a75c0259e85c1d2f83ca4e960289a5f814ed0cea74d18c353d3e989d/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2708c852ef8cd89d1d2068bdbece0aa188813a0c934db3779b9b1faa8442e5f5", size = 214053554, upload-time = "2025-09-04T08:31:38.196Z" }, @@ -1693,9 +1697,9 @@ name = "nvidia-cusolver" version = "12.0.4.66" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas" }, - { name = "nvidia-cusparse" }, - { name = "nvidia-nvjitlink" }, + { name = "nvidia-cublas", marker = "sys_platform == 'linux'" }, + { name = "nvidia-cusparse", marker = "sys_platform == 'linux'" }, + { name = "nvidia-nvjitlink", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/c8/c3/b30c9e935fc01e3da443ec0116ed1b2a009bb867f5324d3f2d7e533e776b/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:02c2457eaa9e39de20f880f4bd8820e6a1cfb9f9a34f820eb12a155aa5bc92d2", size = 223467760, upload-time = "2025-09-04T08:33:04.222Z" }, @@ -1707,7 +1711,7 @@ name = "nvidia-cusparse" version = "12.6.3.3" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink" }, + { name = "nvidia-nvjitlink", marker = "sys_platform == 'linux'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/f8/94/5c26f33738ae35276672f12615a64bd008ed5be6d1ebcb23579285d960a9/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:80bcc4662f23f1054ee334a15c72b8940402975e0eab63178fc7e670aa59472c", size = 162155568, upload-time = "2025-09-04T08:33:42.864Z" }, @@ -2559,7 +2563,7 @@ dependencies = [ { name = "pydantic-settings" }, { name = "pymupdf" }, { name = "pyprojroot" }, - { name = "pysqlite3-binary", marker = "sys_platform == 'linux'" }, + { name = "pysqlite3-binary" }, { name = "python-dotenv" }, { name = "python-telegram-bot" }, { name = "rank-bm25" }, @@ -2588,6 +2592,7 @@ requires-dist = [ { name = "pydantic-settings", specifier = ">=2.0.0" }, { name = "pymupdf", specifier = ">=1.24.0" }, { name = "pyprojroot", specifier = ">=0.3.0" }, + { name = "pysqlite3-binary", specifier = ">=0.5.4.post2" }, { name = "pysqlite3-binary", marker = "sys_platform == 'linux'", specifier = ">=0.5.4" }, { name = "python-dotenv", specifier = ">=1.0.0" }, { name = "python-telegram-bot", specifier = ">=21.0" }, @@ -2934,7 +2939,7 @@ version = "0.18.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "ptyprocess", marker = "os_name != 'nt'" }, - { name = "pywinpty", marker = "os_name == 'nt'" }, + { name = "pywinpty", marker = "os_name == 'nt' and sys_platform != 'linux'" }, { name = "tornado" }, ] sdist = { url = "https://files.pythonhosted.org/packages/8a/11/965c6fd8e5cc254f1fe142d547387da17a8ebfd75a3455f637c663fb38a0/terminado-0.18.1.tar.gz", hash = "sha256:de09f2c4b85de4765f7714688fff57d3e75bad1f909b589fde880460c753fd2e", size = 32701, upload-time = "2024-03-12T14:34:39.026Z" } From 2eaf394d5c39fa35be0279a5f1b75c816873c0bf Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 13:54:05 -0500 Subject: [PATCH 35/55] chore(dev): add pytest-cov with term-missing coverage report --- pyproject.toml | 2 ++ uv.lock | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 89 insertions(+) diff --git a/pyproject.toml b/pyproject.toml index dd65241..bfa65c5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -44,6 +44,7 @@ dev = [ "pre-commit>=3.8.0", "jupyter>=1.0.0", "ipykernel>=6.0.0", + "pytest-cov>=7.1.0", ] [tool.ruff] @@ -66,6 +67,7 @@ markers = [ "unit: Unit tests (no external dependencies)", "integration: Integration tests (may require APIs or DB)", ] +addopts = "--cov=src/researchos --cov-report=term-missing" [tool.mypy] python_version = "3.11" diff --git a/uv.lock b/uv.lock index 1f48663..12f1945 100644 --- a/uv.lock +++ b/uv.lock @@ -402,6 +402,50 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/60/97/891a0971e1e4a8c5d2b20bbe0e524dc04548d2307fee33cdeba148fd4fc7/comm-0.2.3-py3-none-any.whl", hash = "sha256:c615d91d75f7f04f095b30d1c1711babd43bdc6419c1be9886a85f2f4e489417", size = 7294, upload-time = "2025-07-25T14:02:02.896Z" }, ] +[[package]] +name = "coverage" +version = "7.14.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/23/7f/d0720730a397a999ffc0fd3f5bebef347338e3a47b727da66fbb228e2ff2/coverage-7.14.0.tar.gz", hash = "sha256:057a6af2f160a85384cde4ab36f0d2777bae1057bae255f95413cdd382aa5c74", size = 919489, upload-time = "2026-05-10T18:02:31.397Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fc/e4/649c8d4f7f1709b6dbfc474358aa1bba02f67bcd52e2fec291a5014006cd/coverage-7.14.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:6a78e2a9d9c5e3b8d4ab9b9d28c985ea66fced0a7d7c2aec1f216e03a2011480", size = 219795, upload-time = "2026-05-10T17:59:48.198Z" }, + { url = "https://files.pythonhosted.org/packages/7f/8d/46692d24b3f395d4cbf17bfcc57136b4f2f9c0c0df864b0bddfc1d71a014/coverage-7.14.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:a1816c505187592dcd1c5a5f226601a549f70365fbd00930ac88b0c225b76bb4", size = 220299, upload-time = "2026-05-10T17:59:49.683Z" }, + { url = "https://files.pythonhosted.org/packages/12/c2/a40f5cb295bbcbb697a76947a56081c494c61950366294ee426ffe261099/coverage-7.14.0-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:d8e1762f0e9cbc26ec315471e7b47855218e833cd5a032d706fbf43845d878c7", size = 250721, upload-time = "2026-05-10T17:59:51.494Z" }, + { url = "https://files.pythonhosted.org/packages/fd/35/202235eb5c3c14c212462cd91d61b7386bf8fc44bc7a77f4742d2a69174b/coverage-7.14.0-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:9336e23e8bb3a3925398261385e2a1533957d3e760e91070dcb0e98bfa514eed", size = 252633, upload-time = "2026-05-10T17:59:53.244Z" }, + { url = "https://files.pythonhosted.org/packages/bb/80/5f596e8995785124ee191c42535664c5e62c65995b66f4ca21e28ae04c81/coverage-7.14.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd1169b2230f9cbe9c638ba38022ed7a2b1e641cc07f7cea0365e4be2a74980", size = 254743, upload-time = "2026-05-10T17:59:55.021Z" }, + { url = "https://files.pythonhosted.org/packages/1e/6d/0d178825be2350f0adb27984d0aa7cf84bbdab201f6fb926b535d23a8f5f/coverage-7.14.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d1bb3543b58fea74d2cd1abc4054cc927e4724687cb4560cd2ed88d2c7d820c0", size = 256700, upload-time = "2026-05-10T17:59:56.511Z" }, + { url = "https://files.pythonhosted.org/packages/19/5b/9e549c2f6e9dfea472adadba06c294e64735dabc2dd19015fac082095013/coverage-7.14.0-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a93bac2cb577ef60074999ed56d8a1535894398e2ed920d4185c3ec0c8864742", size = 250854, upload-time = "2026-05-10T17:59:57.94Z" }, + { url = "https://files.pythonhosted.org/packages/3d/1c/b94f9f5f36396021ee2f62c5834b12e6a3d31f0bed5d6fc6d1c3caec087c/coverage-7.14.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:5904abf7e18cddc463219b17552229650c6b79e061d31a1059283051169cf7d5", size = 252433, upload-time = "2026-05-10T17:59:59.688Z" }, + { url = "https://files.pythonhosted.org/packages/b5/cb/d192cd8e1345eccabc32016f2d39072ecd10cb4f4b983ed8d0ebdeaf00dc/coverage-7.14.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:741f57cddc9004a8c81b084660215f33a6b597dbe62c31386b983ee26310e327", size = 250494, upload-time = "2026-05-10T18:00:01.953Z" }, + { url = "https://files.pythonhosted.org/packages/53/c5/aac9f460a41d835dbddef1d377f105f6ac2311d0f3c1588e9f51046d8813/coverage-7.14.0-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:664123feb0929d7affc135717dbd70d61d98688a08ab1e5ba464739620c6252d", size = 254261, upload-time = "2026-05-10T18:00:03.779Z" }, + { url = "https://files.pythonhosted.org/packages/23/aa/7af7c0081980a9cb3d289c5a435a4b7657dcecbd128e25c580e6a50389b5/coverage-7.14.0-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:c83d2399a51bbec8429266905d33616f04bc5726b1138c35844d5fcd896b2e20", size = 250216, upload-time = "2026-05-10T18:00:05.262Z" }, + { url = "https://files.pythonhosted.org/packages/35/60/a4257538ce2f6b978aeb51870d6c4208c510928a03db7e0339bb625dccb7/coverage-7.14.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:bcb2e855b87321259a037429288ae85216d191c74de3e79bf57cd2bc0761992c", size = 251125, upload-time = "2026-05-10T18:00:06.858Z" }, + { url = "https://files.pythonhosted.org/packages/a1/ab/f91af47642ec1aa53490e835a95847168d9c77fc39aa58527604c051e145/coverage-7.14.0-cp311-cp311-win32.whl", hash = "sha256:731dc15b385ac52289743d476245b61e1a2927e803bef655b52bc3b2a75a21f3", size = 222300, upload-time = "2026-05-10T18:00:08.608Z" }, + { url = "https://files.pythonhosted.org/packages/f0/f0/a71ddbd874431e7a7cd96071f0c331cfbbad07704833c765d24ffbab8a67/coverage-7.14.0-cp311-cp311-win_amd64.whl", hash = "sha256:bfb0ed8ec5d25e93face268115d7964db9df8b9aae8edcde9ec6b16c726a7cc1", size = 223241, upload-time = "2026-05-10T18:00:10.746Z" }, + { url = "https://files.pythonhosted.org/packages/d8/6e/d9d312a5151a96cd110efee32efc3fc97b01ebd86203fe618ccb29cf4c92/coverage-7.14.0-cp311-cp311-win_arm64.whl", hash = "sha256:7ebb1c6df9f78046a1b1e0a89674cd4bf73b7c648914eebcf976a57fd99a5627", size = 221908, upload-time = "2026-05-10T18:00:12.242Z" }, + { url = "https://files.pythonhosted.org/packages/09/1e/2f996b2c8415cbb6f54b0f5ec1ee850c96d7911961afb4fc05f4a89d8c58/coverage-7.14.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7ffd19fc8aed057fd686a17a4935eef5f9859d69208f96310e893e64b9b6ccf5", size = 219967, upload-time = "2026-05-10T18:00:13.756Z" }, + { url = "https://files.pythonhosted.org/packages/34/23/35c7aea1274aef7525bdd2dc92f710bdde6d11652239d71d1ec450067939/coverage-7.14.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:829994cfe1aeb773ca27bf246d4badc1e764893e3bfb98fff820fcecd1ca4662", size = 220329, upload-time = "2026-05-10T18:00:15.264Z" }, + { url = "https://files.pythonhosted.org/packages/75/cf/a8f4b43a16e194b0261257ad28ded5853ec052570afef4a84e1d81189f3b/coverage-7.14.0-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:b4f07cf7edcb7ec39431a5074d7ea83b29a9f71fcfc494f0f40af4e65180420f", size = 251839, upload-time = "2026-05-10T18:00:17.16Z" }, + { url = "https://files.pythonhosted.org/packages/69/ff/6699e7b71e60d3049eb2bdcbc95ee3f35707b2b0e48f32e9e63d3ce30c08/coverage-7.14.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:ca3d9cf2c32b521bd9518385608787fa86f38daf993695307531822c3430ed67", size = 254576, upload-time = "2026-05-10T18:00:18.829Z" }, + { url = "https://files.pythonhosted.org/packages/22/ec/c936d495fcd67f48f03a9c4ad3297ff80d1f222a5df3980f15b34c186c21/coverage-7.14.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92af52828e7f29d827346b0294e5a0853fa206db77db0395b282918d41e28db9", size = 255690, upload-time = "2026-05-10T18:00:20.648Z" }, + { url = "https://files.pythonhosted.org/packages/5c/42/5af63f636cc62a4a2b1b3ba9146f6ee6f53a35a50d5cefc54d5670f60999/coverage-7.14.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7b2bb6c9d7e769360d0f20a0f219603fd64f0c8f97de17ab25853261602be0fb", size = 257949, upload-time = "2026-05-10T18:00:22.28Z" }, + { url = "https://files.pythonhosted.org/packages/26/d3/a225317bd2012132a27e1176d51660b826f99bb975876463c44ea0d7ee5a/coverage-7.14.0-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:1c9ed6ef99f88fb8c14aa8e2bf8eb0fe55fa2edfea68f8675d78741df1a5ac0e", size = 252242, upload-time = "2026-05-10T18:00:24.076Z" }, + { url = "https://files.pythonhosted.org/packages/f1/7f/9e65495298c3ea414742998539c37d048b5e81cc818fb1828cc6b51d10bf/coverage-7.14.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8231ade007f37959fbf58acc677f26b922c02eda6f0428ea307da0fd39681bf3", size = 253608, upload-time = "2026-05-10T18:00:25.588Z" }, + { url = "https://files.pythonhosted.org/packages/94/46/1522b524a35bdad22b2b8c4f9d32d0a104b524726ec380b2db68db1746f5/coverage-7.14.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:d8b013632cc1ce1d09dbe4f32667b4d320ec2f54fc326ebeffcd0b0bcc2bb6c4", size = 251753, upload-time = "2026-05-10T18:00:27.104Z" }, + { url = "https://files.pythonhosted.org/packages/f3/e9/cdf00d38817742c541ade405e115a3f7bf36e6f2a8b99d4f209861b85a2d/coverage-7.14.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:1733198802d71ec4c524f322e2867ee05c62e9e75df86bdca545407a221827d1", size = 255823, upload-time = "2026-05-10T18:00:29.038Z" }, + { url = "https://files.pythonhosted.org/packages/38/fc/5e7877cf5f902d08a17ff1c532511476d87e1bea355bd5028cb97f902e79/coverage-7.14.0-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:72a305291fa8ee01332f1aaf38b348ca34097f6aa0b0ef627eef2837e57bbba5", size = 251323, upload-time = "2026-05-10T18:00:30.647Z" }, + { url = "https://files.pythonhosted.org/packages/18/9d/50f05a72dff8487464fdd4178dda5daed642a060e60afb644e3d45123559/coverage-7.14.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:fcaba850dd317c65423a9d63d88f9573c53b00354d6dd95724576cc98a131595", size = 253197, upload-time = "2026-05-10T18:00:32.211Z" }, + { url = "https://files.pythonhosted.org/packages/00/3f/6f61ffe6439df266c3cf60f5c99cfaa21103d0210d706a42fc6c30683ff8/coverage-7.14.0-cp312-cp312-win32.whl", hash = "sha256:5ac83957a80d0701310e96d8bec68cdcf4f90a7674b7d13f15a344315b41ab27", size = 222515, upload-time = "2026-05-10T18:00:33.717Z" }, + { url = "https://files.pythonhosted.org/packages/85/19/93853133df2cb371083285ef6a93982a0173e7a233b0f61373ba9fd30eb2/coverage-7.14.0-cp312-cp312-win_amd64.whl", hash = "sha256:70390b0da32cb90b501953716302906e8bcce087cb283e70d8c97729f22e92b2", size = 223324, upload-time = "2026-05-10T18:00:35.172Z" }, + { url = "https://files.pythonhosted.org/packages/74/18/9f7fe62f659f24b7a82a0be56bf94c1bd0a89e0ae7ab4c668f6e82404294/coverage-7.14.0-cp312-cp312-win_arm64.whl", hash = "sha256:91b993743d959b8be85b4abf9d5478216a69329c321efe5be0433c1a841d691d", size = 221944, upload-time = "2026-05-10T18:00:37.014Z" }, + { url = "https://files.pythonhosted.org/packages/61/e8/cb8e80d6f9f55b99588625062822bf946cf03ed06315df4bd8397f5632a1/coverage-7.14.0-py3-none-any.whl", hash = "sha256:8de5b61163aee3d05c8a2beab6f47913df7981dad1baf82c414d99158c286ab1", size = 211764, upload-time = "2026-05-10T18:02:29.538Z" }, +] + +[package.optional-dependencies] +toml = [ + { name = "tomli", marker = "python_full_version <= '3.11'" }, +] + [[package]] name = "cuda-bindings" version = "13.2.0" @@ -2326,6 +2370,20 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl", hash = "sha256:611e26147c7f77640e6d0a92a38ed17c3e9848063698d5c93d5aa7aa11cebff5", size = 15075, upload-time = "2025-11-10T16:07:45.537Z" }, ] +[[package]] +name = "pytest-cov" +version = "7.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "coverage", extra = ["toml"] }, + { name = "pluggy" }, + { name = "pytest" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/b1/51/a849f96e117386044471c8ec2bd6cfebacda285da9525c9106aeb28da671/pytest_cov-7.1.0.tar.gz", hash = "sha256:30674f2b5f6351aa09702a9c8c364f6a01c27aae0c1366ae8016160d1efc56b2", size = 55592, upload-time = "2026-03-21T20:11:16.284Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9d/7a/d968e294073affff457b041c2be9868a40c1c71f4a35fcc1e45e5493067b/pytest_cov-7.1.0-py3-none-any.whl", hash = "sha256:a0461110b7865f9a271aa1b51e516c9a95de9d696734a2f71e3e78f46e1d4678", size = 22876, upload-time = "2026-03-21T20:11:14.438Z" }, +] + [[package]] name = "python-dateutil" version = "2.9.0.post0" @@ -2579,6 +2637,7 @@ dev = [ { name = "pre-commit" }, { name = "pytest" }, { name = "pytest-asyncio" }, + { name = "pytest-cov" }, { name = "ruff" }, ] @@ -2609,6 +2668,7 @@ dev = [ { name = "pre-commit", specifier = ">=3.8.0" }, { name = "pytest", specifier = ">=8.0.0" }, { name = "pytest-asyncio", specifier = ">=1.3.0" }, + { name = "pytest-cov", specifier = ">=7.1.0" }, { name = "ruff", specifier = ">=0.6.0" }, ] @@ -2994,6 +3054,33 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/72/f4/0de46cfa12cdcbcd464cc59fde36912af405696f687e53a091fb432f694c/tokenizers-0.22.2-cp39-abi3-win_arm64.whl", hash = "sha256:9ce725d22864a1e965217204946f830c37876eee3b2ba6fc6255e8e903d5fcbc", size = 2612133, upload-time = "2026-01-05T10:45:17.232Z" }, ] +[[package]] +name = "tomli" +version = "2.4.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/22/de/48c59722572767841493b26183a0d1cc411d54fd759c5607c4590b6563a6/tomli-2.4.1.tar.gz", hash = "sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f", size = 17543, upload-time = "2026-03-25T20:22:03.828Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/11/db3d5885d8528263d8adc260bb2d28ebf1270b96e98f0e0268d32b8d9900/tomli-2.4.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30", size = 154704, upload-time = "2026-03-25T20:21:10.473Z" }, + { url = "https://files.pythonhosted.org/packages/6d/f7/675db52c7e46064a9aa928885a9b20f4124ecb9bc2e1ce74c9106648d202/tomli-2.4.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a", size = 149454, upload-time = "2026-03-25T20:21:12.036Z" }, + { url = "https://files.pythonhosted.org/packages/61/71/81c50943cf953efa35bce7646caab3cf457a7d8c030b27cfb40d7235f9ee/tomli-2.4.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076", size = 237561, upload-time = "2026-03-25T20:21:13.098Z" }, + { url = "https://files.pythonhosted.org/packages/48/c1/f41d9cb618acccca7df82aaf682f9b49013c9397212cb9f53219e3abac37/tomli-2.4.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9", size = 243824, upload-time = "2026-03-25T20:21:14.569Z" }, + { url = "https://files.pythonhosted.org/packages/22/e4/5a816ecdd1f8ca51fb756ef684b90f2780afc52fc67f987e3c61d800a46d/tomli-2.4.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c", size = 242227, upload-time = "2026-03-25T20:21:15.712Z" }, + { url = "https://files.pythonhosted.org/packages/6b/49/2b2a0ef529aa6eec245d25f0c703e020a73955ad7edf73e7f54ddc608aa5/tomli-2.4.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc", size = 247859, upload-time = "2026-03-25T20:21:17.001Z" }, + { url = "https://files.pythonhosted.org/packages/83/bd/6c1a630eaca337e1e78c5903104f831bda934c426f9231429396ce3c3467/tomli-2.4.1-cp311-cp311-win32.whl", hash = "sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049", size = 97204, upload-time = "2026-03-25T20:21:18.079Z" }, + { url = "https://files.pythonhosted.org/packages/42/59/71461df1a885647e10b6bb7802d0b8e66480c61f3f43079e0dcd315b3954/tomli-2.4.1-cp311-cp311-win_amd64.whl", hash = "sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e", size = 108084, upload-time = "2026-03-25T20:21:18.978Z" }, + { url = "https://files.pythonhosted.org/packages/b8/83/dceca96142499c069475b790e7913b1044c1a4337e700751f48ed723f883/tomli-2.4.1-cp311-cp311-win_arm64.whl", hash = "sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece", size = 95285, upload-time = "2026-03-25T20:21:20.309Z" }, + { url = "https://files.pythonhosted.org/packages/c1/ba/42f134a3fe2b370f555f44b1d72feebb94debcab01676bf918d0cb70e9aa/tomli-2.4.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a", size = 155924, upload-time = "2026-03-25T20:21:21.626Z" }, + { url = "https://files.pythonhosted.org/packages/dc/c7/62d7a17c26487ade21c5422b646110f2162f1fcc95980ef7f63e73c68f14/tomli-2.4.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085", size = 150018, upload-time = "2026-03-25T20:21:23.002Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/79d13d7c15f13bdef410bdd49a6485b1c37d28968314eabee452c22a7fda/tomli-2.4.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9", size = 244948, upload-time = "2026-03-25T20:21:24.04Z" }, + { url = "https://files.pythonhosted.org/packages/10/90/d62ce007a1c80d0b2c93e02cab211224756240884751b94ca72df8a875ca/tomli-2.4.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5", size = 253341, upload-time = "2026-03-25T20:21:25.177Z" }, + { url = "https://files.pythonhosted.org/packages/1a/7e/caf6496d60152ad4ed09282c1885cca4eea150bfd007da84aea07bcc0a3e/tomli-2.4.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585", size = 248159, upload-time = "2026-03-25T20:21:26.364Z" }, + { url = "https://files.pythonhosted.org/packages/99/e7/c6f69c3120de34bbd882c6fba7975f3d7a746e9218e56ab46a1bc4b42552/tomli-2.4.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1", size = 253290, upload-time = "2026-03-25T20:21:27.46Z" }, + { url = "https://files.pythonhosted.org/packages/d6/2f/4a3c322f22c5c66c4b836ec58211641a4067364f5dcdd7b974b4c5da300c/tomli-2.4.1-cp312-cp312-win32.whl", hash = "sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917", size = 98141, upload-time = "2026-03-25T20:21:28.492Z" }, + { url = "https://files.pythonhosted.org/packages/24/22/4daacd05391b92c55759d55eaee21e1dfaea86ce5c571f10083360adf534/tomli-2.4.1-cp312-cp312-win_amd64.whl", hash = "sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9", size = 108847, upload-time = "2026-03-25T20:21:29.386Z" }, + { url = "https://files.pythonhosted.org/packages/68/fd/70e768887666ddd9e9f5d85129e84910f2db2796f9096aa02b721a53098d/tomli-2.4.1-cp312-cp312-win_arm64.whl", hash = "sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257", size = 95088, upload-time = "2026-03-25T20:21:30.677Z" }, + { url = "https://files.pythonhosted.org/packages/7b/61/cceae43728b7de99d9b847560c262873a1f6c98202171fd5ed62640b494b/tomli-2.4.1-py3-none-any.whl", hash = "sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe", size = 14583, upload-time = "2026-03-25T20:22:03.012Z" }, +] + [[package]] name = "torch" version = "2.11.0" From 3de79d80b76ed12c08bb66ebf20b54d4445517ac Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 13:54:18 -0500 Subject: [PATCH 36/55] refactor(prompts): remove registry.py, consolidate on PromptTemplate --- docs/prompts_guide.md | 2 +- tests/unit/domain/test_prompts.py | 11 +++++++---- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/prompts_guide.md b/docs/prompts_guide.md index 2435a77..6117440 100644 --- a/docs/prompts_guide.md +++ b/docs/prompts_guide.md @@ -10,7 +10,7 @@ All prompts are in `src/researchos/domain/prompts/`: 1. Create a `.txt` file in the appropriate subdirectory 2. Use `{variable_name}` for dynamic content -3. Load with `load_prompt("category", "name", variable=value)` +3. Load with `PromptTemplate("category", "name").render(**kwargs)` where **kwargs are variables referenced in the template as {variable_name}. ## How to version prompts diff --git a/tests/unit/domain/test_prompts.py b/tests/unit/domain/test_prompts.py index 48fb424..548f712 100644 --- a/tests/unit/domain/test_prompts.py +++ b/tests/unit/domain/test_prompts.py @@ -3,20 +3,23 @@ import pytest from researchos.domain.exceptions import PromptNotFoundError -from researchos.domain.prompts.registry import load_prompt +from researchos.domain.prompts import PromptTemplate @pytest.mark.unit class TestPromptRegistry: def test_load_system_prompt(self): - prompt = load_prompt("system", "agent") + prompt = PromptTemplate("system", "agent").render() assert "ResearchOS" in prompt def test_load_task_prompt_with_variables(self): - prompt = load_prompt("tasks", "extraction", topic="LLM agents", paper_text="Test content") + prompt = PromptTemplate("tasks", "extraction").render( + topic="LLM agents", paper_text="Test content" + ) + assert "LLM agents" in prompt assert "Test content" in prompt def test_load_nonexistent_prompt_raises(self): with pytest.raises(PromptNotFoundError): - load_prompt("tasks", "nonexistent_prompt") + PromptTemplate("tasks", "nonexistent_prompt").render() From 19ff6baa69722a4688d58ac9391da4e907b4f438 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 13:54:25 -0500 Subject: [PATCH 37/55] refactor(ingestion): inject VectorStore as optional parameter --- .../application/services/ingestion_service.py | 29 ++++++++++++------- .../application/test_ingestion_service.py | 8 +++-- 2 files changed, 24 insertions(+), 13 deletions(-) diff --git a/src/researchos/application/services/ingestion_service.py b/src/researchos/application/services/ingestion_service.py index 2f80bd2..2102aa9 100644 --- a/src/researchos/application/services/ingestion_service.py +++ b/src/researchos/application/services/ingestion_service.py @@ -21,10 +21,9 @@ from researchos.application.services.retrieval_service import chunk_to_document, overlap_chunking from researchos.domain.exceptions import IngestionError +from researchos.domain.interfaces import VectorStore from researchos.domain.models import Paper from researchos.infrastructure.data.arxiv import search_papers -from researchos.infrastructure.retrieval.chroma import ChromaVectorStore -from researchos.infrastructure.retrieval.embedder import LocalEmbedder from researchos.paths import PAPERS_DIR @@ -110,6 +109,7 @@ async def ingest_papers( max_results: int, chunk_size: int = 500, overlap: int = 50, + store: VectorStore | None = None, collection_name: str = "papers", embedder_metadata: dict | None = None, ) -> None: @@ -125,11 +125,13 @@ async def ingest_papers( chunk_size: Number of characters per text chunk. Defaults to 500. overlap: Character overlap between consecutive chunks to preserve context across boundaries. Defaults to 50. - collection_name: Name of the ChromaDB collection to upsert into. - Defaults to ``"papers"``. - embedder_metadata: Optional HNSW metadata dict forwarded to ChromaDB - (e.g. ``{"hnsw:space": "cosine"}``). If ``None``, the - ``ChromaVectorStore`` default is used. + store: A VectorStore implementation to upsert documents into. + If ``None``, a default ``ChromaVectorStore`` is created using + ``collection_name`` and ``embedder_metadata``. + collection_name: Chroma collection name used only when ``store`` is + ``None``. Defaults to ``"papers"``. + embedder_metadata: HNSW settings forwarded to ``ChromaVectorStore`` + when ``store`` is ``None``. Returns: None. Side-effects: PDFs saved to ``PAPERS_DIR``, chunks upserted @@ -139,10 +141,15 @@ async def ingest_papers( IngestionError: If any PDF cannot be downloaded or has no extractable text. """ - embedder = LocalEmbedder() - store = ChromaVectorStore( - embedder=embedder, collection_name=collection_name, embedder_metadata=embedder_metadata - ) + if store is None: + from researchos.infrastructure.retrieval.chroma import ChromaVectorStore + from researchos.infrastructure.retrieval.embedder import LocalEmbedder + + store = ChromaVectorStore( + embedder=LocalEmbedder(), + collection_name=collection_name, + embedder_metadata=embedder_metadata, + ) papers = await search_papers(query, max_results) diff --git a/tests/unit/application/test_ingestion_service.py b/tests/unit/application/test_ingestion_service.py index ce9da90..533d8da 100644 --- a/tests/unit/application/test_ingestion_service.py +++ b/tests/unit/application/test_ingestion_service.py @@ -1,4 +1,5 @@ from datetime import datetime +from pathlib import Path from unittest.mock import AsyncMock, MagicMock, patch import pytest @@ -39,5 +40,8 @@ async def test_extract_text_pdf(): ): result = await extract_text_pdf(paper=paper) - assert isinstance(result, str) - assert len(result) > 0 + assert isinstance(result, tuple) + assert isinstance(result[0], str) + assert isinstance(result[1], Path) + + assert len(result) == 2 From a0129359cc9edbb668d2bb5917b656fa2cc24372 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 13:54:40 -0500 Subject: [PATCH 38/55] feat(retrieval): add Retriever protocol and BM25Retriever implementation --- notebooks/008-jmmz-bm25-retrieval.ipynb | 340 ++++++++++++++++++ src/researchos/domain/interfaces.py | 16 + .../infrastructure/retrieval/bm25.py | 49 +++ 3 files changed, 405 insertions(+) create mode 100644 notebooks/008-jmmz-bm25-retrieval.ipynb create mode 100644 src/researchos/infrastructure/retrieval/bm25.py diff --git a/notebooks/008-jmmz-bm25-retrieval.ipynb b/notebooks/008-jmmz-bm25-retrieval.ipynb new file mode 100644 index 0000000..63f4d7b --- /dev/null +++ b/notebooks/008-jmmz-bm25-retrieval.ipynb @@ -0,0 +1,340 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "id": "cell-setup", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import sys\n", + "\n", + "package_path = os.path.abspath(\".\").split(os.sep + \"notebooks\")[0]\n", + "if package_path not in sys.path:\n", + " sys.path.append(package_path)\n", + "\n", + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "cell-sqlite-patch", + "metadata": {}, + "outputs": [], + "source": [ + "# Patch sqlite3 with bundled modern version — required on Linux where system sqlite3 < 3.35.0.\n", + "# Same patch used in tests/conftest.py. Must run before any chromadb import.\n", + "if sys.platform == \"linux\":\n", + " __import__(\"pysqlite3\")\n", + " sys.modules[\"sqlite3\"] = sys.modules.pop(\"pysqlite3\")" + ] + }, + { + "cell_type": "markdown", + "id": "cell-md-load", + "metadata": {}, + "source": [ + "## 1. Cargar documentos desde Chroma\n", + "\n", + "BM25Retriever necesita los documentos en memoria. Los cargamos directamente\n", + "desde la colección existente en ChromaDB usando el cliente nativo." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "cell-load-docs", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Failed to reload module 'sqlite3' from file '/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/pysqlite3/__init__.py'\n", + "Traceback (most recent call last):\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 325, in check\n", + " superreload(m, reload, self.old_objects)\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 584, in superreload\n", + " module = reload(module)\n", + " ^^^^^^^^^^^^^^\n", + " File \"/home/.pyenv/versions/3.11.8/lib/python3.11/importlib/__init__.py\", line 148, in reload\n", + " raise ImportError(msg.format(name), name=name)\n", + "ImportError: module pysqlite3 not in sys.modules\n", + "[autoreload of sqlite3 failed: Traceback (most recent call last):\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 325, in check\n", + " superreload(m, reload, self.old_objects)\n", + " File \"/home/jmontoya@proteccion.local/personal_projects/researchos/.venv/lib/python3.11/site-packages/IPython/extensions/autoreload.py\", line 584, in superreload\n", + " module = reload(module)\n", + " ^^^^^^^^^^^^^^\n", + " File \"/home/.pyenv/versions/3.11.8/lib/python3.11/importlib/__init__.py\", line 148, in reload\n", + " raise ImportError(msg.format(name), name=name)\n", + "ImportError: module pysqlite3 not in sys.modules\n", + "]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Documentos cargados: 754\n" + ] + } + ], + "source": [ + "import chromadb\n", + "\n", + "from researchos.domain.models import Document\n", + "from researchos.paths import CHROMA_DIR\n", + "\n", + "COLLECTION_NAME = \"papers\"\n", + "\n", + "client = chromadb.PersistentClient(path=str(CHROMA_DIR))\n", + "collection = client.get_collection(COLLECTION_NAME)\n", + "\n", + "raw = collection.get(include=[\"documents\", \"metadatas\"])\n", + "\n", + "documents = [\n", + " Document(doc_id=doc_id, text=text, metadata=metadata)\n", + " for doc_id, text, metadata in zip(raw[\"ids\"], raw[\"documents\"], raw[\"metadatas\"])\n", + "]\n", + "\n", + "print(f\"Documentos cargados: {len(documents)}\")" + ] + }, + { + "cell_type": "markdown", + "id": "cell-md-bm25", + "metadata": {}, + "source": [ + "## 2. Crear BM25Retriever y buscar\n", + "\n", + "El índice se construye en el constructor — tokenización básica por defecto." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "cell-bm25-search", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "score: 15.5751 | paper: sam_musker_2024\n", + "text: composition, but be aided by an \n", + "increased availability of information in this condition. What appears \n", + "as undiminished model performance in compositional conditions may \n", + "be the result of a balanced n\n", + "\n", + "score: 14.9431 | paper: sam_musker_2024\n", + "text: example, one subject \n", + "attains a below-average match to reference of 50% in the Distracted\n", + "condition despite being able to state that the task involves ‘‘Looking \n", + "at other comparable entries to figure \n", + "\n", + "score: 14.6667 | paper: sam_musker_2024\n", + "text: & Rathkopf, 2025).\n", + "The question of whether we require human-likeness of a mecha-\n", + "nism to declare human-level ‘‘competence’’ is ultimately not empiri-\n", + "cal, but rather demands philosophical consensus am\n", + "\n", + "score: 14.5783 | paper: sam_musker_2024\n", + "text: \n", + "appears significantly smaller (see sections 4.2 and 4.3 of Grattafiori et al. \n", + "(2024)). That said, Llama 405B’s training being mostly text prediction does \n", + "not guarantee that this is what underlies \n", + "\n", + "score: 14.4955 | paper: hehai_lin_2025\n", + "text: +f(2) = 1\n", + "7. This is a contradiction,\n", + "indicating an error in the setup. Revisiting the problem,\n", + "we find:f(2) = 2\n", + "3 Thus, the correct value is: 2\n", + "3 . ×\n", + "LLM1 (Idea Analysis):\n", + "The partner’s contribution \n", + "\n" + ] + } + ], + "source": [ + "from researchos.infrastructure.retrieval.bm25 import BM25Retriever\n", + "\n", + "bm25 = BM25Retriever(documents=documents)\n", + "\n", + "query = \"What is the primary function of the harness in an externalized agent architecture?\"\n", + "\n", + "results = await bm25.search(query=query, k=5)\n", + "\n", + "for r in results:\n", + " print(f\"score: {r.score:.4f} | paper: {r.metadata.get('paper_id', '?')}\")\n", + " print(f\"text: {r.text[:200]}\")\n", + " print()" + ] + }, + { + "cell_type": "markdown", + "id": "cell-md-compare", + "metadata": {}, + "source": [ + "## 3. Comparar BM25 vs búsqueda vectorial\n", + "\n", + "Misma query, mismo k — observa qué documentos recupera cada estrategia." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "cell-vector-search", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "9547c9ae238c4a238395af5268ad5601", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Loading weights: 0%| | 0/103 [00:00 None: ... +class Retriever(Protocol): + """Contract for any retriever strategy (vectorization, BM25, etc.).""" + + async def search(self, query: str, k: int) -> list[Document]: + """Search for the top-k most relevant documents. + + Args: + query: Natural-language query string. + k: Number of results to return. + + Returns: + List of Document objects sorted by relevance score (descending). + """ + ... + + class MemoryStore(Protocol): """Contract for conversational memory persistence.""" diff --git a/src/researchos/infrastructure/retrieval/bm25.py b/src/researchos/infrastructure/retrieval/bm25.py new file mode 100644 index 0000000..83d5b59 --- /dev/null +++ b/src/researchos/infrastructure/retrieval/bm25.py @@ -0,0 +1,49 @@ +from rank_bm25 import BM25Okapi + +from researchos.domain.models import Document + + +class BM25Retriever: + """BM25-based retriever implementing the Retriever protocol. + + Builds an in-memory BM25 index from a list of Documents at construction + time. Retrieval is pure keyword matching — no embeddings, no vector store. + Complements semantic search for exact term and acronym lookups. + """ + + def __init__(self, documents: list[Document], tokenizer=None) -> None: + """Build the BM25 index from the provided documents. + + Args: + documents: Pre-processed documents to index. Typically the same + chunks stored in the vector store. + tokenizer: Optional callable that takes a string and returns a + list of tokens. Defaults to lowercased whitespace splitting. + """ + self.documents = documents + self._tokenize = tokenizer or (lambda text: text.lower().split()) + self._corpus = [self._tokenize(doc.text) for doc in documents] + self.bm25 = BM25Okapi(self._corpus) + + async def search(self, query: str, k: int) -> list[Document]: + """Return the top-k documents ranked by BM25 score. + + Defined as async to satisfy the Retriever protocol and allow uniform + use with asyncio.gather alongside async retrievers like ChromaVectorStore. + Executes synchronously in practice since no I/O is involved. + + Args: + query: Natural-language query string. + k: Number of top results to return. + + Returns: + List of Document objects sorted by BM25 score descending, with + the score field populated. + """ + tokenized_query = self._tokenize(query) + scores = self.bm25.get_scores(tokenized_query) + top_indices = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)[:k] + + return [ + self.documents[i].model_copy(update={"score": float(scores[i])}) for i in top_indices + ] From 8dacca32f77c9498d048fb1cb6156ed99c65896c Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Thu, 21 May 2026 13:54:46 -0500 Subject: [PATCH 39/55] docs: update learnings and work_log for 2026-05-21 --- docs/learnings.md | 38 +++++++++++++++++++++++++++++++++----- docs/work_log.md | 21 +++++++++++++++++++++ 2 files changed, 54 insertions(+), 5 deletions(-) diff --git a/docs/learnings.md b/docs/learnings.md index b0cd3ab..92cd668 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -182,19 +182,47 @@ Regla simple para ResearchOS: - `asyncio.gather` sin `await` no ejecuta las coroutines — retorna un objeto coroutine sin resolver. Siempre `await asyncio.gather(...)`. - `Path.stem` retorna el nombre del archivo sin extensión — más limpio que hacer `split(os.sep)[-1].split('.pdf')[0]` sobre un string. -**Fecha:** _[completar]_ +**Fecha:** 21/05/2026 ### ¿Qué aprendí? -- + +**Protocols y Clean Architecture — el propósito real** +- Recibir `VectorStore` en lugar de `ChromaVectorStore` desacopla la capa de aplicación de la infraestructura. Beneficios concretos: (1) testabilidad — se inyecta un mock sin tocar BD real; (2) intercambiabilidad — cambiar de Chroma a Qdrant solo requiere crear una nueva clase que cumpla el Protocol, sin tocar código de negocio; (3) contrato explícito — el Protocol documenta exactamente qué necesita la aplicación. +- Los Protocols en Python se validan en tiempo de chequeo estático (mypy), NO en runtime. En runtime Python no lanza error si falta un método — solo falla cuando se llama. En proyectos maduros mypy corre en CI como barrera automática. +- Structural subtyping ("duck typing con tipos"): una clase satisface un Protocol simplemente teniendo los métodos con las mismas firmas — no necesita heredar explícitamente ni declararlo. + +**RAG: vector search vs BM25 vs hybrid** +- Búsqueda vectorial: texto → embedding → similitud coseno. Encuentra contenido semánticamente cercano aunque las palabras sean distintas. +- BM25: recuperación por coincidencia exacta de términos, ponderada por frecuencia y rareza. Trabaja en memoria, sin base vectorial. Fuerte donde el vectorial falla: siglas, nombres propios, términos técnicos exactos. +- Hybrid search: combina ambos resultados con Reciprocal Rank Fusion (RRF). El objetivo no es más chunks sino mejor ranking, considerando señales semánticas y de coincidencia exacta simultáneamente. + +**Cobertura de tests por capa** +- Infraestructura (`arxiv.py`, `chroma.py`, `embedder.py`) tiene baja cobertura unitaria por diseño — dependen de sistemas externos y pertenecen a tests de integración. +- Medir cobertura de código sin tests asociados solo genera ruido en el reporte. +- `addopts` en `[tool.pytest.ini_options]` pasa flags automáticamente a cada ejecución de pytest. +- `isinstance` no puede verificar tipos genéricos como `tuple[str, Path]` en runtime — hay que usar assertions separadas por elemento. + +**BM25 y la relación con la base vectorial** +- Chroma cumple dos roles distintos: persistencia de chunks en disco y retrieval vectorial. BM25 solo necesita el primero — usa Chroma como almacén y construye su propio índice en memoria con `collection.get()`. +- Los scores de BM25 y vectorial son incomparables directamente: BM25 retorna frecuencias ponderadas (sin límite superior), vectorial retorna similitud coseno (0 a 1). RRF resuelve esto comparando posiciones en el ranking, no scores absolutos. +- Una función `async` que no contiene `await` es perfectamente válida — se resuelve inmediatamente sin suspenderse. Útil para que BM25Retriever sea uniforme con ChromaVectorStore en `asyncio.gather()`. +- `model_copy(update={...})` en Pydantic crea una copia del objeto con campos modificados sin mutar el original — necesario para asignar el `score` calculado en cada búsqueda. ### ¿Qué no entendí bien? -- +- El rol práctico del event loop a nivel de programación (cuándo y por qué interactuar con él directamente). +- Cómo integrar mypy en el pipeline de CI para que valide Protocols automáticamente. ### Decisiones de diseño -- +- `BM25Retriever` implementa el Protocol `Retriever` (solo `search`) — no `VectorStore` (que exige también `upsert`). `ChromaVectorStore` satisface ambos por structural subtyping. +- `BM25Retriever.search` definido como `async` aunque opera en memoria, para ser uniforme con `ChromaVectorStore` y poder usarse en `asyncio.gather()` en el hybrid search. +- `ingest_papers` acepta `store: VectorStore | None = None` — imports de infraestructura son lazy dentro de la función para no violar la dependencia application → infrastructure a nivel de módulo. +- Tests de cobertura se acumulan en lotes por sesión dedicada, no después de cada feature. +- Tokenizador de BM25 inyectado como callable (`tokenizer=None`) en lugar de string selector — más flexible y Pythónico. ### Errores interesantes -- +- `registry.py` y `PromptTemplate` hacían lo mismo — tener ambos era redundancia. Se eliminó `registry.py` y se consolidó en `PromptTemplate.render()`. +- El reporte de un bug en `overlap_chunking` era incorrecto: la lógica `start = i * (chunk_size - overlap)` produce un paso fijo, no un overlap acumulativo. Verificar con math antes de reportar un bug. +- `isinstance` no puede verificar tipos genéricos como `tuple[str, Path]` en runtime — usar assertions separadas por elemento. --- diff --git a/docs/work_log.md b/docs/work_log.md index d440350..6680d2f 100644 --- a/docs/work_log.md +++ b/docs/work_log.md @@ -97,3 +97,24 @@ - Consultar con tutor: múltiples colecciones en Chroma, parámetros de `ingest_papers` --- + +## 2026-05-21 + +### Trabajo desarrollado +- Docstrings Google-style agregados a 12 módulos: `arxiv.py`, `anthropic_llm.py`, `chroma.py`, `embedder.py`, `ingestion_service.py`, `retrieval_service.py`, `paths.py`, `models.py`, `exceptions.py`, `interfaces.py`, `registry.py`, `benchmark_arxiv.py`, `eval_retrieval.py`. +- `pysqlite3-binary` agregado como dependencia Linux en `pyproject.toml`; celda de patch sqlite3 agregada a notebook 007. +- Notebooks reordenados: 003=retriever_service, 004=chroma-VectorStore, 005=ingestion_service (refleja orden de dependencias). +- `registry.py` eliminado; `PromptTemplate.render()` es ahora el único mecanismo de carga de prompts. +- `ingestion_service.py` refactorizado: `store: VectorStore | None = None` como parámetro, imports de infraestructura lazy dentro de la función. +- `test_ingestion_service.py` actualizado: assertions separadas para verificar `tuple[str, Path]`. +- `pytest-cov` agregado con `addopts = "--cov=src/researchos --cov-report=term-missing"` — cobertura global 76% unit, 83% con integración. +- Protocol `Retriever` creado en `domain/interfaces.py` con solo `search` (sin `upsert`). +- `BM25Retriever` implementado en `infrastructure/retrieval/bm25.py`: índice en memoria con `BM25Okapi`, tokenizador inyectable, `search` async, scores vía `model_copy`. +- Notebook `008-jmmz-bm25-retrieval.ipynb` creado para comparar BM25 vs vectorial sobre la misma query. + +### Próximos pasos +- T7: Hybrid search con Reciprocal Rank Fusion en `retrieval_service.py` +- T8: Reranker con Claude sobre top-10 del hybrid +- Re-ejecutar evaluación comparando las cuatro estrategias + +--- From 56a5b8558fb00a5f7a92aa447365836774ab26cd Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 1 Jun 2026 09:41:00 -0500 Subject: [PATCH 40/55] refactor(services): split chunking from retrieval service --- .../application/services/chunking_service.py | 86 ++++++++++++++++++ .../application/services/ingestion_service.py | 2 +- .../application/services/retrieval_service.py | 89 ++----------------- ...al_service.py => test_chunking_service.py} | 2 +- 4 files changed, 93 insertions(+), 86 deletions(-) create mode 100644 src/researchos/application/services/chunking_service.py rename tests/unit/application/{test_retrieval_service.py => test_chunking_service.py} (93%) diff --git a/src/researchos/application/services/chunking_service.py b/src/researchos/application/services/chunking_service.py new file mode 100644 index 0000000..3aa855c --- /dev/null +++ b/src/researchos/application/services/chunking_service.py @@ -0,0 +1,86 @@ +"""Chunking service — Text chunking utilities for the ingestion pipeline. + +Provides pure, stateless functions for splitting paper text into overlapping +chunks and converting those chunks into the ``Document`` format expected by +the vector store. These utilities are shared by the ingestion service and +can be reused in evaluation scripts. + +Design note: + Functions here have no side-effects and no external dependencies beyond + domain models. They belong in ``application/services/`` (not ``domain/``) +because they encode an operational decision (chunk size, overlap strategy) + rather than a business concept. +""" + +from researchos.domain.models import Chunk, Document + + +def overlap_chunking( + text: str, paper_id: str, chunk_size: int = 500, overlap: int = 50 +) -> list[Chunk]: + """Split a text string into overlapping fixed-size chunks. + + Each chunk starts at ``chunk_size - overlap`` characters after the + previous one, so adjacent chunks share ``overlap`` characters of context. + This sliding-window approach reduces information loss at chunk boundaries. + + Args: + text: Full text content of the paper to be chunked. + paper_id: Identifier used as the prefix for each ``chunk_id`` + (e.g. the PDF stem). + chunk_size: Number of characters in each chunk. Defaults to 500. + overlap: Number of characters shared between consecutive chunks. + Defaults to 50. + + Returns: + Ordered list of :class:`~researchos.domain.models.Chunk` objects + with ``chunk_id``, ``paper_id``, ``text``, ``chunk_index``, and + position metadata (``start_char``, ``end_char``). + + Example: + >>> chunks = overlap_chunking("Hello world " * 100, "paper_abc") + >>> chunks[0].chunk_id + 'paper_abc_0' + """ + chunks = [] + + for i, initial_car in enumerate(range(0, len(text), chunk_size)): + start_chunk = initial_car - overlap * i + end_chunk = start_chunk + chunk_size + + chunk = Chunk( + chunk_id=f"{paper_id}_{i}", + paper_id=paper_id, + text=text[start_chunk:end_chunk], + metadata={ + "chunk_size": chunk_size, + "overlap": overlap, + "start_char": start_chunk, + "end_char": min(end_chunk, len(text)), + }, + chunk_index=i, + ) + + chunks.append(chunk) + return chunks + + +def chunk_to_document(chunk: Chunk) -> Document: + """Convert a Chunk into a Document suitable for vector store indexing. + + Merges the chunk's own metadata with ``paper_id`` and ``chunk_index`` + fields so that documents stored in the vector store can be traced back + to their source paper and position. + + Args: + chunk: A populated :class:`~researchos.domain.models.Chunk` object. + + Returns: + A :class:`~researchos.domain.models.Document` ready to be upserted + into the vector store, with ``doc_id == chunk.chunk_id``. + """ + return Document( + doc_id=chunk.chunk_id, + text=chunk.text, + metadata={**chunk.metadata, "paper_id": chunk.paper_id, "chunk_index": chunk.chunk_index}, + ) diff --git a/src/researchos/application/services/ingestion_service.py b/src/researchos/application/services/ingestion_service.py index 2102aa9..f589cbc 100644 --- a/src/researchos/application/services/ingestion_service.py +++ b/src/researchos/application/services/ingestion_service.py @@ -19,7 +19,7 @@ import fitz import httpx -from researchos.application.services.retrieval_service import chunk_to_document, overlap_chunking +from researchos.application.services.chunking_service import chunk_to_document, overlap_chunking from researchos.domain.exceptions import IngestionError from researchos.domain.interfaces import VectorStore from researchos.domain.models import Paper diff --git a/src/researchos/application/services/retrieval_service.py b/src/researchos/application/services/retrieval_service.py index 770a7df..b1225a9 100644 --- a/src/researchos/application/services/retrieval_service.py +++ b/src/researchos/application/services/retrieval_service.py @@ -1,86 +1,7 @@ -"""Retrieval service — Text chunking utilities for the ingestion pipeline. +"""Retrieval service — Orchestrates document retrieval strategies. -Provides pure, stateless functions for splitting paper text into overlapping -chunks and converting those chunks into the ``Document`` format expected by -the vector store. These utilities are shared by the ingestion service and -can be reused in evaluation scripts. - -Design note: - Functions here have no side-effects and no external dependencies beyond - domain models. They belong in ``application/services/`` (not ``domain/``) -because they encode an operational decision (chunk size, overlap strategy) - rather than a business concept. +Provides hybrid search by combining multiple Retriever implementations +(vector, BM25, etc.) via Reciprocal Rank Fusion (RRF). All retrievers +are queried in parallel using asyncio.gather and results are merged into +a single ranked list of Documents. """ - -from researchos.domain.models import Chunk, Document - - -def overlap_chunking( - text: str, paper_id: str, chunk_size: int = 500, overlap: int = 50 -) -> list[Chunk]: - """Split a text string into overlapping fixed-size chunks. - - Each chunk starts at ``chunk_size - overlap`` characters after the - previous one, so adjacent chunks share ``overlap`` characters of context. - This sliding-window approach reduces information loss at chunk boundaries. - - Args: - text: Full text content of the paper to be chunked. - paper_id: Identifier used as the prefix for each ``chunk_id`` - (e.g. the PDF stem). - chunk_size: Number of characters in each chunk. Defaults to 500. - overlap: Number of characters shared between consecutive chunks. - Defaults to 50. - - Returns: - Ordered list of :class:`~researchos.domain.models.Chunk` objects - with ``chunk_id``, ``paper_id``, ``text``, ``chunk_index``, and - position metadata (``start_char``, ``end_char``). - - Example: - >>> chunks = overlap_chunking("Hello world " * 100, "paper_abc") - >>> chunks[0].chunk_id - 'paper_abc_0' - """ - chunks = [] - - for i, initial_car in enumerate(range(0, len(text), chunk_size)): - start_chunk = initial_car - overlap * i - end_chunk = start_chunk + chunk_size - - chunk = Chunk( - chunk_id=f"{paper_id}_{i}", - paper_id=paper_id, - text=text[start_chunk:end_chunk], - metadata={ - "chunk_size": chunk_size, - "overlap": overlap, - "start_char": start_chunk, - "end_char": min(end_chunk, len(text)), - }, - chunk_index=i, - ) - - chunks.append(chunk) - return chunks - - -def chunk_to_document(chunk: Chunk) -> Document: - """Convert a Chunk into a Document suitable for vector store indexing. - - Merges the chunk's own metadata with ``paper_id`` and ``chunk_index`` - fields so that documents stored in the vector store can be traced back - to their source paper and position. - - Args: - chunk: A populated :class:`~researchos.domain.models.Chunk` object. - - Returns: - A :class:`~researchos.domain.models.Document` ready to be upserted - into the vector store, with ``doc_id == chunk.chunk_id``. - """ - return Document( - doc_id=chunk.chunk_id, - text=chunk.text, - metadata={**chunk.metadata, "paper_id": chunk.paper_id, "chunk_index": chunk.chunk_index}, - ) diff --git a/tests/unit/application/test_retrieval_service.py b/tests/unit/application/test_chunking_service.py similarity index 93% rename from tests/unit/application/test_retrieval_service.py rename to tests/unit/application/test_chunking_service.py index 6be001f..cee78b4 100644 --- a/tests/unit/application/test_retrieval_service.py +++ b/tests/unit/application/test_chunking_service.py @@ -1,6 +1,6 @@ import pytest -from researchos.application.services.retrieval_service import chunk_to_document, overlap_chunking +from researchos.application.services.chunking_service import chunk_to_document, overlap_chunking from researchos.domain.models import Chunk, Document From 51aafcfef5c8b3772783d5b3cd85b79b8bc63c13 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 1 Jun 2026 10:37:44 -0500 Subject: [PATCH 41/55] feat(services): Add hybrid search and its tests --- .../application/services/retrieval_service.py | 33 ++++++++++++++++ .../application/test_retrieval_service.py | 39 +++++++++++++++++++ 2 files changed, 72 insertions(+) create mode 100644 tests/unit/application/test_retrieval_service.py diff --git a/src/researchos/application/services/retrieval_service.py b/src/researchos/application/services/retrieval_service.py index b1225a9..958bb92 100644 --- a/src/researchos/application/services/retrieval_service.py +++ b/src/researchos/application/services/retrieval_service.py @@ -5,3 +5,36 @@ are queried in parallel using asyncio.gather and results are merged into a single ranked list of Documents. """ + +import asyncio + +from researchos.domain.interfaces import Retriever +from researchos.domain.models import Document + + +async def hybrid_search( + query: str, + retrievers: list[Retriever], + k: int = 5, + candidates_per_retriever: int | None = None, + rrf_k: int = 60, +) -> list[Document]: + if not retrievers: + raise ValueError("Without retrievers. Add retrieves, please") + + n = candidates_per_retriever or k * 2 + + results = await asyncio.gather(*[retriever.search(query, n) for retriever in retrievers]) + + rrf_scores = {} + docs_by_id = {} + + for ranking in results: + for rank, doc in enumerate(ranking, start=1): # 1-indexed + rrf_scores[doc.doc_id] = rrf_scores.get(doc.doc_id, 0.0) + 1 / (rrf_k + rank) + docs_by_id.setdefault(doc.doc_id, doc) + + sorted_by_scores = dict(sorted(rrf_scores.items(), key=lambda item: item[1], reverse=True)) + first_k = list(sorted_by_scores.items())[:k] + + return [docs_by_id[doc_id].model_copy(update={"score": score}) for doc_id, score in first_k] diff --git a/tests/unit/application/test_retrieval_service.py b/tests/unit/application/test_retrieval_service.py new file mode 100644 index 0000000..68ad25c --- /dev/null +++ b/tests/unit/application/test_retrieval_service.py @@ -0,0 +1,39 @@ +"""Unit tests for hybrid_search in retrieval_service.""" + +import pytest + +from researchos.application.services.retrieval_service import hybrid_search +from researchos.domain.models import Document +from tests.conftest import MockVectorStore + + +def _make_docs(*ids: str) -> list[Document]: + return [Document(doc_id=doc_id, text=f"text for {doc_id}") for doc_id in ids] + + +@pytest.mark.unit +@pytest.mark.asyncio +async def test_hybrid_search_doc_in_both_retrievers_wins(): + """A document appearing in both retrievers must rank first — RRF accumulates scores.""" + retriever_a = MockVectorStore(_make_docs("doc1", "doc2", "doc3")) + retriever_b = MockVectorStore(_make_docs("doc2", "doc4", "doc5")) + + results = await hybrid_search( + query="test query", + retrievers=[retriever_a, retriever_b], + k=5, + ) + + assert results[0].doc_id == "doc2" + assert results[0].score > results[1].score + doc_ids = [r.doc_id for r in results] + assert len(doc_ids) == len(set(doc_ids)) + assert len(results) <= 5 + + +@pytest.mark.unit +@pytest.mark.asyncio +async def test_hybrid_search_empty_retrievers_raises(): + """Passing an empty retrievers list must raise ValueError immediately.""" + with pytest.raises(ValueError): + await hybrid_search(query="test", retrievers=[]) From 8432ea35d0828ef7bffa70892a43c91f8d993096 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 1 Jun 2026 11:41:13 -0500 Subject: [PATCH 42/55] feat(services): add LLM-based reranker and its tests --- .../application/services/retrieval_service.py | 70 ++++++++++++++++++- .../domain/prompts/tasks/rerank.txt | 18 +++++ .../application/test_retrieval_service.py | 35 +++++++++- 3 files changed, 117 insertions(+), 6 deletions(-) create mode 100644 src/researchos/domain/prompts/tasks/rerank.txt diff --git a/src/researchos/application/services/retrieval_service.py b/src/researchos/application/services/retrieval_service.py index 958bb92..4e92c52 100644 --- a/src/researchos/application/services/retrieval_service.py +++ b/src/researchos/application/services/retrieval_service.py @@ -7,9 +7,11 @@ """ import asyncio +import json -from researchos.domain.interfaces import Retriever -from researchos.domain.models import Document +from researchos.domain.interfaces import LLMProvider, Retriever +from researchos.domain.models import Document, Message +from researchos.domain.prompts import PromptTemplate async def hybrid_search( @@ -19,8 +21,31 @@ async def hybrid_search( candidates_per_retriever: int | None = None, rrf_k: int = 60, ) -> list[Document]: + """Combine multiple retrievers into a single ranked list using Reciprocal Rank Fusion. + + Each retriever is queried in parallel for ``candidates_per_retriever`` documents. + Results are merged with RRF: documents that appear in multiple rankings accumulate + higher scores. The final list is deduplicated and trimmed to top-k. + + Args: + query: Natural-language query string passed to every retriever. + retrievers: List of Retriever implementations to query (e.g. BM25, vector). + Must contain at least one retriever. + k: Number of documents to return in the final ranked list. + candidates_per_retriever: Documents fetched from each retriever before fusion. + Defaults to ``k * 2`` to ensure enough candidates after deduplication. + rrf_k: RRF smoothing constant (default 60, per Cormack et al. 2009). + Higher values reduce the influence of top-ranked documents. + + Returns: + List of up to ``k`` Documents sorted by RRF score descending, with the + ``score`` field set to the accumulated RRF score. + + Raises: + ValueError: If ``retrievers`` is empty. + """ if not retrievers: - raise ValueError("Without retrievers. Add retrieves, please") + raise ValueError("At least one retriever is required.") n = candidates_per_retriever or k * 2 @@ -38,3 +63,42 @@ async def hybrid_search( first_k = list(sorted_by_scores.items())[:k] return [docs_by_id[doc_id].model_copy(update={"score": score}) for doc_id, score in first_k] + + +async def hybrid_rerank_search( + query: str, + llm: LLMProvider, + documents: list[Document], + k: int = 5, +) -> list[Document]: + """Reorder a list of candidate documents by relevance using an LLM as judge. + + Renders the ``tasks/rerank`` prompt with the query and document texts, + asks the LLM to return a JSON-ordered list of document IDs, and maps + those IDs back to the original Document objects. + + Intended to be called after ``hybrid_search`` — pass its output as + ``documents`` and this function returns the top-k reranked subset. + + Args: + query: The original user query used to judge relevance. + llm: An LLMProvider implementation (injected). + documents: Candidate documents to rerank (typically hybrid top-10). + k: Number of documents to return after reranking. + + Returns: + List of up to ``k`` Documents in LLM-ranked order. Scores are not + updated — the ordering itself is the signal. + + Raises: + json.JSONDecodeError: If the LLM response is not valid JSON. + KeyError: If the LLM returns a doc_id not present in ``documents``. + """ + docs_str = "\n".join(f'"{doc.doc_id}": {doc.text}' for doc in documents) + prompt = PromptTemplate("tasks", "rerank").render(query=query, documents=docs_str) + message = Message(role="user", content=prompt) + llm_answer = await llm.generate([message]) + + docs_by_id = {doc.doc_id: doc for doc in documents} + ranked_ids = json.loads(llm_answer) + return [docs_by_id[doc_id] for doc_id in ranked_ids[:k]] diff --git a/src/researchos/domain/prompts/tasks/rerank.txt b/src/researchos/domain/prompts/tasks/rerank.txt new file mode 100644 index 0000000..c82bf32 --- /dev/null +++ b/src/researchos/domain/prompts/tasks/rerank.txt @@ -0,0 +1,18 @@ +You are an expert at evaluating document relevance. + +Your task is to reorder a list of documents by how useful each one is for answering a specific query. + +Query: {query} + +Documents (format: "doc_id": text): +``` +{documents} +``` + +Return ONLY a JSON array of document IDs sorted from most to least relevant. Example: +["doc_id_1", "doc_id_2", "doc_id_3"] + +Rules: +- Output must be a valid JSON array of strings, nothing else — no explanation, no markdown. +- The array must contain exactly as many elements as there are documents above. +- No document ID may be repeated. diff --git a/tests/unit/application/test_retrieval_service.py b/tests/unit/application/test_retrieval_service.py index 68ad25c..b36c410 100644 --- a/tests/unit/application/test_retrieval_service.py +++ b/tests/unit/application/test_retrieval_service.py @@ -1,10 +1,12 @@ -"""Unit tests for hybrid_search in retrieval_service.""" +"""Unit tests for hybrid_search and hybrid_rerank_search in retrieval_service.""" + +import json import pytest -from researchos.application.services.retrieval_service import hybrid_search +from researchos.application.services.retrieval_service import hybrid_rerank_search, hybrid_search from researchos.domain.models import Document -from tests.conftest import MockVectorStore +from tests.conftest import MockLLMProvider, MockVectorStore def _make_docs(*ids: str) -> list[Document]: @@ -37,3 +39,30 @@ async def test_hybrid_search_empty_retrievers_raises(): """Passing an empty retrievers list must raise ValueError immediately.""" with pytest.raises(ValueError): await hybrid_search(query="test", retrievers=[]) + + +@pytest.mark.unit +@pytest.mark.asyncio +async def test_hybrid_rerank_search_respects_llm_order(): + """Documents must be returned in the order the LLM specifies.""" + docs = _make_docs("doc1", "doc2", "doc3") + llm_order = ["doc3", "doc1", "doc2"] + llm = MockLLMProvider(response=json.dumps(llm_order)) + + results = await hybrid_rerank_search(query="test query", llm=llm, documents=docs, k=3) + + assert [r.doc_id for r in results] == llm_order + + +@pytest.mark.unit +@pytest.mark.asyncio +async def test_hybrid_rerank_search_trims_to_k(): + """Only the first k documents from the LLM ranking are returned.""" + docs = _make_docs("doc1", "doc2", "doc3", "doc4", "doc5") + llm_order = ["doc5", "doc3", "doc1", "doc4", "doc2"] + llm = MockLLMProvider(response=json.dumps(llm_order)) + + results = await hybrid_rerank_search(query="test query", llm=llm, documents=docs, k=3) + + assert len(results) == 3 + assert results[0].doc_id == "doc5" From 4ad3016613095c26c0dc49b38a612330048605af Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 1 Jun 2026 13:40:58 -0500 Subject: [PATCH 43/55] feat(services): add hybrid search, reranker, and eval script --- scripts/eval_retrieval.py | 146 ++++++++++++------ .../application/services/retrieval_service.py | 8 +- 2 files changed, 103 insertions(+), 51 deletions(-) diff --git a/scripts/eval_retrieval.py b/scripts/eval_retrieval.py index 9812830..9ec7e03 100644 --- a/scripts/eval_retrieval.py +++ b/scripts/eval_retrieval.py @@ -1,83 +1,131 @@ -"""Retrieval evaluation script — Manual sanity-check for the vector store. +"""Retrieval evaluation script — Compares four retrieval strategies. -Loads a JSON evaluation dataset from ``data/samples/eval_dataset.json``, -runs each question through the :class:`ChromaVectorStore`, and prints the -top retrieved documents with their relevance scores. Useful for quickly -validating retrieval quality after changing chunk size, overlap, or the -embedding model. +Loads the evaluation dataset from ``data/samples/eval_dataset.json`` and +runs each question through four strategies: + 1. Vector search (ChromaVectorStore) + 2. BM25 (BM25Retriever) + 3. Hybrid RRF (vector + BM25) + 4. Hybrid + LLM rerank + +Reports P@k and MRR per strategy across all questions. Usage: uv run python scripts/eval_retrieval.py Eval dataset format (``eval_dataset.json``): [ - {"question": "What is RAG?", "source_paper": "vaswani_2017"}, + {"question": "...", "reference_answer": "...", "source_paper": "paper_stem.pdf"}, ... ] """ import asyncio import json +import sys + +if sys.platform == "linux": + __import__("pysqlite3") + sys.modules["sqlite3"] = sys.modules.pop("pysqlite3") + +import chromadb -from researchos.domain.interfaces import VectorStore +from researchos.application.services.retrieval_service import hybrid_rerank_search, hybrid_search from researchos.domain.models import Document +from researchos.infrastructure.llm.anthropic_llm import AnthropicLLM +from researchos.infrastructure.retrieval.bm25 import BM25Retriever from researchos.infrastructure.retrieval.chroma import ChromaVectorStore from researchos.infrastructure.retrieval.embedder import LocalEmbedder -from researchos.paths import SAMPLES_DIR +from researchos.paths import CHROMA_DIR, SAMPLES_DIR +COLLECTION_NAME = "papers" +K = 5 path_examples = SAMPLES_DIR / "eval_dataset.json" -async def answer_question( - question: str, store: VectorStore, max_results: int = 10 -) -> list[Document]: - """Retrieve and print the top documents for a given question. +def _paper_id(source_paper: str) -> str: + """Normalize source_paper to match the paper_id stored in chunk metadata.""" + return source_paper.replace(".pdf", "") - Searches the vector store for the most relevant chunks and prints each - result's score and a 200-character preview of the text to stdout. - Args: - question: Natural-language question to search for. - store: A :class:`~researchos.domain.interfaces.VectorStore` - implementation (typically :class:`ChromaVectorStore`). - max_results: Maximum number of documents to retrieve. Defaults to 10. +def _precision_at_k(results: list[Document], source_paper: str, k: int) -> float: + """Return 1.0 if source_paper appears in the top-k results, else 0.0.""" + top_k_ids = {doc.metadata.get("paper_id", "") for doc in results[:k]} + return 1.0 if _paper_id(source_paper) in top_k_ids else 0.0 - Returns: - List of :class:`~researchos.domain.models.Document` objects returned - by the vector store, in descending relevance order. - """ - results = await store.search(query=question, k=max_results) - for r in results: - print(f"\nscore: {r.score:.3f}") - print(f"text: {r.text[:200]}") - return results +def _reciprocal_rank(results: list[Document], source_paper: str) -> float: + """Return 1/rank of the first result matching source_paper, or 0.0.""" + target = _paper_id(source_paper) + for rank, doc in enumerate(results, start=1): + if doc.metadata.get("paper_id", "") == target: + return 1.0 / rank + return 0.0 -async def main() -> None: - """Run the full retrieval evaluation loop. - Reads each question from the eval dataset, calls :func:`answer_question`, - and prints a summary showing which papers appeared in the top results - compared to the expected ``source_paper``. - - Raises: - FileNotFoundError: If ``data/samples/eval_dataset.json`` does not exist. - json.JSONDecodeError: If the dataset file is malformed. - """ +async def main() -> None: + """Run the full evaluation loop and print a strategy comparison table.""" with open(path_examples, encoding="utf-8") as f: data = json.load(f) + # ── Build retrievers ── embedder = LocalEmbedder() - store = ChromaVectorStore(embedder=embedder) - - answers = [] - for dict_question in data: - print("-" * 10 + dict_question["question"] + "*" * 10) - print(f"- pdf_ref: {dict_question['source_paper']}") - answer = await answer_question(dict_question["question"], store=store, max_results=3) - print(f"- papers in answer: {set([doc.metadata['paper_id'] for doc in answer])}") - print("\n") - answers.append(answer) + chroma = ChromaVectorStore(embedder=embedder, collection_name=COLLECTION_NAME) + llm = AnthropicLLM() + + raw = ( + chromadb.PersistentClient(path=str(CHROMA_DIR)) + .get_collection(COLLECTION_NAME) + .get(include=["documents", "metadatas"]) + ) + all_docs = [ + Document(doc_id=doc_id, text=text, metadata=metadata) + for doc_id, text, metadata in zip( + raw["ids"], raw["documents"], raw["metadatas"], strict=False + ) + ] + bm25 = BM25Retriever(documents=all_docs) + + strategies = { + "vector": lambda q: chroma.search(q, K), + "bm25": lambda q: bm25.search(q, K), + "hybrid": lambda q: hybrid_search(q, retrievers=[chroma, bm25], k=K), + "hybrid+rerank": lambda q: _rerank(q, chroma, bm25, llm), + } + + scores: dict[str, list[float]] = {s: [] for s in strategies} + mrr: dict[str, list[float]] = {s: [] for s in strategies} + + for item in data: + question = item["question"] + source = item["source_paper"] + print(f"\nQ: {question[:80]}...") + print(f" expected: {_paper_id(source)}") + + for name, fn in strategies.items(): + results = await fn(question) + p = _precision_at_k(results, source, K) + rr = _reciprocal_rank(results, source) + scores[name].append(p) + mrr[name].append(rr) + found = _paper_id(source) in {doc.metadata.get("paper_id", "") for doc in results[:K]} + print(f" {name:15s} P@{K}={'✓' if found else '✗'} RR={rr:.2f}") + + # ── Summary table ── + print("\n" + "=" * 50) + print(f"{'Strategy':<15} {'P@' + str(K):>6} {'MRR':>6}") + print("-" * 30) + for name in strategies: + avg_p = sum(scores[name]) / len(scores[name]) + avg_mrr = sum(mrr[name]) / len(mrr[name]) + print(f"{name:<15} {avg_p:>6.3f} {avg_mrr:>6.3f}") + + +async def _rerank( + query: str, chroma: ChromaVectorStore, bm25: BM25Retriever, llm: AnthropicLLM +) -> list[Document]: + """Run hybrid search then rerank the top-10 with Claude.""" + candidates = await hybrid_search(query, retrievers=[chroma, bm25], k=10) + return await hybrid_rerank_search(query=query, llm=llm, documents=candidates, k=K) if __name__ == "__main__": diff --git a/src/researchos/application/services/retrieval_service.py b/src/researchos/application/services/retrieval_service.py index 4e92c52..54d1354 100644 --- a/src/researchos/application/services/retrieval_service.py +++ b/src/researchos/application/services/retrieval_service.py @@ -99,6 +99,10 @@ async def hybrid_rerank_search( message = Message(role="user", content=prompt) llm_answer = await llm.generate([message]) + # Extract the JSON array from the response — Claude may wrap it in markdown or add text. + start = llm_answer.find("[") + end = llm_answer.rfind("]") + 1 + ranked_ids = json.loads(llm_answer[start:end]) + docs_by_id = {doc.doc_id: doc for doc in documents} - ranked_ids = json.loads(llm_answer) - return [docs_by_id[doc_id] for doc_id in ranked_ids[:k]] + return [docs_by_id[doc_id] for doc_id in ranked_ids[:k] if doc_id in docs_by_id] From 5a54bc8c9e1516aa5d7e3fbff59795079cb7921e Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 1 Jun 2026 13:41:09 -0500 Subject: [PATCH 44/55] docs: update learnings and work_log for 2026-06-01 --- docs/learnings.md | 18 ++++++++++++++++++ docs/work_log.md | 21 +++++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/docs/learnings.md b/docs/learnings.md index 92cd668..e4fc37e 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -224,5 +224,23 @@ Regla simple para ResearchOS: - El reporte de un bug en `overlap_chunking` era incorrecto: la lógica `start = i * (chunk_size - overlap)` produce un paso fijo, no un overlap acumulativo. Verificar con math antes de reportar un bug. - `isinstance` no puede verificar tipos genéricos como `tuple[str, Path]` en runtime — usar assertions separadas por elemento. +**Fecha:** 01/06/2026 + +### ¿Qué aprendí? +- hybrid rerank con LLM com juez busca solucionar el problema de orden de los retrievers que se basan exclusivamente en el score y que no toman en cuenta el contexto de la pregunta como es el caso de BM25 +- en asyncio.gather lo que debo tener en mente es que se pasan argumentos por lo que el operador * lo que hace es desempaquetar tuplas o listas +- los mocks de protocolos personaizados se realizan en conftest.py, los que involucran llamados a serivicos externos son reemplazados por mocks en librerías estandar +- Para que una evaluación de un retriever sea justa es necesario que el set de evaluación no sufra de data leakege, esto es, que no conozco qué hay explicitamente en el corpus actual. Mi conjunto está sesgado porque se hizo exactamente con documentos que sí o sí ya sabíamos que estaban en el vector store + +### ¿Qué no entendí bien? +- Debo profundizar sobre métricas de evaluación de un retriever. Entiendo que el MRR mide en qué posición es recuperado un documento y luego se saca un promedio, pero no me queda claro cómo opera esta métrica en el caso por ejemplo de un RAG donde el corpus son ejemplos históricos como el de PQRs. + +### Decisiones de diseño +- Modificamos retrieval_service para que quedar acomo chunking service y agregamos un retrieval service real que se enfoca justamente en un servicio de retreival + +### Errores interesantes +- Claude puede devolver JSON envuelto en markdown o con texto adicional — `json.loads()` falla. Solución: extraer el array con `find("[")` y `rfind("]")` antes de parsear. +- Las lambdas en un dict de estrategias capturan variables del scope exterior por referencia — en este caso no fue problema, pero es un antipatrón a tener en mente si las variables cambian en el loop. + --- diff --git a/docs/work_log.md b/docs/work_log.md index 6680d2f..d36339d 100644 --- a/docs/work_log.md +++ b/docs/work_log.md @@ -118,3 +118,24 @@ - Re-ejecutar evaluación comparando las cuatro estrategias --- + +## 2026-06-01 + +### Trabajo desarrollado +- Refactor: `retrieval_service.py` renombrado a `chunking_service.py`; nuevo `retrieval_service.py` creado para orquestación de retrieval. +- T7 completado: `hybrid_search` en `retrieval_service.py` con RRF, `asyncio.gather` paralelo, deduplicación y top-k. Test unitario verifica que documento en ambos retrievers gana el ranking. +- T8 completado: `hybrid_rerank_search` en `retrieval_service.py` — usa Claude como juez para reordenar candidatos del hybrid. Prompt en `domain/prompts/tasks/rerank.txt`. Parseo robusto del JSON de respuesta con extracción por `find("[")`. +- T9 completado: `scripts/eval_retrieval.py` refactorizado para comparar las cuatro estrategias (vector, BM25, hybrid, hybrid+rerank) con métricas P@k y MRR. Resultados: vector=1.000/1.000, bm25=0.950/0.925, hybrid=1.000/1.000, hybrid+rerank=1.000/1.000. +- 25/25 tests unitarios pasando. + +### Análisis de resultados +- Resultados altos esperados: el eval dataset fue construido sobre los mismos documentos indexados (data leakage). En producción con queries reales los scores serían menores. +- BM25 levemente inferior al vectorial — falla en una pregunta sobre `imad_aouali_2026` y tiene RR=0.50 en una pregunta sobre Idea3 (lo encuentra en posición 2 en lugar de 1). +- Hybrid y hybrid+rerank igualan al vectorial en este corpus controlado. + +### Próximos pasos +- Merge de `feature/v1-infrastructure-setup` a `main` — V1 completada. +- Nombrar siguiente rama por feature concreta (e.g. `feature/v1-hybrid-rerank` ya hecho, próxima podría ser `feature/v2-telegram-bot`). +- Evaluar con queries reales para obtener métricas más representativas. + +--- From ce415f87598984a02a477935d756a4460d59f969 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 1 Jun 2026 13:42:43 -0500 Subject: [PATCH 45/55] docs(roadmap): mark T6-T10 complete, V1 done --- ROADMAP.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index ef0a842..1b8b228 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -7,14 +7,14 @@ - [x] T4: Chunking fijo (completado 17 abr) - [x] T5: Integración end-to-end (completado 18 abr) -## Mayo 2026 — V1 hybrid search y evaluación -- [ ] T6: BM25 retrieval -- [ ] T7: Hybrid search -- [ ] T8: Reranker básico +## Mayo–Junio 2026 — V1 hybrid search y evaluación +- [x] T6: BM25 retrieval (completado 21 may) +- [x] T7: Hybrid search con RRF (completado 01 jun) +- [x] T8: Reranker con Claude (completado 01 jun) - [x] T9: Dataset de evaluación (20 preguntas) (completado 18 abr) -- [x] T10: Script de evaluación (completado 18 abr) +- [x] T10: Script de evaluación comparativa (completado 01 jun) ## Junio 2026 — V2 LangGraph - [ ] T11: Refactor a LangGraph - [ ] T12: Primer briefing matutino -- [ ] T13: Comparativa V1 vs V2 \ No newline at end of file +- [ ] T13: Comparativa V1 vs V2 From 218ab3f1740180d9daa63c36b82f46ab0e74f413 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 27 Jul 2026 09:08:18 -0500 Subject: [PATCH 46/55] chore(notebooks): Workshop to resume development of the project --- notebooks/taller_retorno_researchos.ipynb | 853 ++++++++++++++++++++++ 1 file changed, 853 insertions(+) create mode 100644 notebooks/taller_retorno_researchos.ipynb diff --git a/notebooks/taller_retorno_researchos.ipynb b/notebooks/taller_retorno_researchos.ipynb new file mode 100644 index 0000000..0b2e05c --- /dev/null +++ b/notebooks/taller_retorno_researchos.ipynb @@ -0,0 +1,853 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "42afc49f", + "metadata": {}, + "source": [ + "# Taller de retorno a ResearchOS\n", + "\n", + "**Objetivo.** Recuperar y afianzar lo que ya construiste en V1 antes de arrancar V2. No es evaluación — es reactivación. Si algo no lo sabes, lo buscas en tu propio código en `src/researchos/` y anotas la duda para discutirla después.\n", + "\n", + "**Duración estimada.** 1.5 a 2 horas si no lo apuras. Si vas más rápido, probablemente estás autocompletando; frena.\n", + "\n", + "**Reglas.**\n", + "\n", + "1. No uses Claude Code, Copilot ni ningún autocompletado LLM para las celdas de código. El punto es forzar el recall, no producir código correcto.\n", + "2. Sí puedes leer tu propio código en `src/researchos/` cuando te atores — esa búsqueda es parte del ejercicio.\n", + "3. Cada sección tiene una celda de \"check\" al final. Corre el check antes de pasar a la siguiente sección.\n", + "4. Las respuestas conceptuales van en celdas markdown propias. Escribe con tus palabras — no copies de docs.\n", + "5. Al final, en la sección de auto-verificación, hay soluciones sugeridas. No las mires antes de terminar cada sección.\n", + "\n", + "**Cobertura.** Ocho secciones:\n", + "\n", + "1. Clean Architecture — conceptos base\n", + "2. Domain — modelos y Protocols\n", + "3. Infrastructure — LLM provider\n", + "4. Retrieval — VectorStore y BM25\n", + "5. Application — RAG y Hybrid Search\n", + "6. Async y concurrencia\n", + "7. Testing con mocks\n", + "8. Meta arquitectural — extender el sistema\n", + "\n", + "Cuando termines, contame en la sesión qué te costó, qué dudas quedaron y con qué sección te sentiste más flojo. Con eso ajustamos el arranque de V2.\n", + "\n", + "---\n" + ] + }, + { + "cell_type": "markdown", + "id": "3e8bcbb2", + "metadata": {}, + "source": [ + "## 1. Clean Architecture — conceptos base\n", + "\n", + "Preguntas para responder con tus palabras en la celda markdown que sigue. Objetivo: 5-10 min.\n" + ] + }, + { + "cell_type": "markdown", + "id": "2434e752", + "metadata": {}, + "source": [ + "**1.1** ¿Cuál es la regla de dependencia entre las tres capas (`domain`, `application`, `infrastructure`)? Es decir, ¿quién puede importar de quién y quién NO puede importar de quién? ¿Por qué esta regla y qué se rompe si no se respeta?\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "application importa de infrastructure, y infrastructure importa de domain, application no puede importar directamente de domain. Esta regla está para mantener separadas las capas y permitir que la capa infrastructure implemente diferentes estructuras y servicios (por ejemplo, conexiones a diferentes tipos de proveedores de LLM o diferentes BD vectoriales) para que las aplicaciones puedan ser cambiadas fácilmente entre esos proveedores, así los refactores son poco costosos y se puede tener un escalamiento fácil. Si se rompe justamente se pierde la facilidad de escalar y de expandir a distintos proveedores porque el código de infraestructura viviría en la aplicación y tocaría tocar mucho código para hacer cambios; además domain establece los contratos de manera que si cualquier implemnetación en infrastructure cumple el contrato debería ser suficiente para implementarse en una application\n", + "\n", + "---\n", + "\n", + "**1.2** ¿Qué es un `Protocol` de Python y en qué se diferencia de una clase abstracta (`ABC`)? Da un ejemplo concreto de tu proyecto (nombre del Protocol y qué implementaciones concretas lo satisfacen).\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "Un Protocol es un un contrato que a diferencia de una clase abstracta no implementa los métodos o propiedades, sino que declara qué métodos y sus firmas debe tener una implementación. Por ejemplo el Protocolo \"LLMProvider\" establece que cualquier implementación debe tener los métodos \"generate\" y \"stream\" sin importar si la conexión que tenemos es con claude api o gemini o gpt. La implementación en la ruta src/researcos/infrastructure/llm/anthropic_llm.py satisface éste protocolo\n", + "\n", + "---\n", + "\n", + "**1.3** En tu proyecto tienes `Retriever` y `VectorStore` como Protocols separados. `VectorStore` tiene `search` y `upsert`; `Retriever` solo tiene `search`. ¿Por qué esta separación? ¿Qué se te complicaría si `BM25Retriever` implementara `VectorStore` directamente?\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n", + "\n", + "---\n", + "\n", + "**1.4** ¿Por qué los agentes en `application/agents/` usan composición (importar funciones de `agent_utils.py`) en lugar de herencia (heredar de una clase `BaseAgent`)? Da al menos dos razones concretas.\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "9f553061", + "metadata": {}, + "source": [ + "## 2. Domain — modelos y Protocols\n", + "\n", + "Objetivo: 10-15 min. Recuerdas cómo definir modelos Pydantic y Protocols mínimos.\n" + ] + }, + { + "cell_type": "markdown", + "id": "db86d995", + "metadata": {}, + "source": [ + "**2.1** Define un modelo Pydantic llamado `Feedback` que representa un feedback de usuario sobre una respuesta del sistema. Debe tener:\n", + "\n", + "- `session_id: str`\n", + "- `query: str` \n", + "- `answer: str`\n", + "- `rating: int` entre 1 y 5 (usa validación de Pydantic)\n", + "- `comment: str` opcional, default vacío\n", + "- `created_at: datetime` con default de la fecha actual\n", + "\n", + "Escribe el modelo. Debe importar solo de la biblioteca estándar y de Pydantic.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "721d7653", + "metadata": {}, + "outputs": [], + "source": [ + "from datetime import datetime\n", + "from pydantic import BaseModel, Field\n", + "\n", + "# Tu código acá\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "dafa15ef", + "metadata": {}, + "source": [ + "**2.2** Define un `Protocol` llamado `FeedbackStore` que abstrae el almacenamiento de feedback. Debe tener dos métodos async:\n", + "\n", + "- `save(feedback: Feedback) -> None`\n", + "- `list_by_session(session_id: str) -> list[Feedback]`\n", + "\n", + "Escribe el Protocol siguiendo el mismo estilo que ves en `src/researchos/domain/interfaces.py`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b0908b2a", + "metadata": {}, + "outputs": [], + "source": [ + "from typing import Protocol\n", + "\n", + "# Tu código acá\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "cb978c9e", + "metadata": {}, + "source": [ + "**2.3 — Check.** Corre la celda siguiente. Debe pasar sin error.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e54a43b6", + "metadata": {}, + "outputs": [], + "source": [ + "# Check sección 2\n", + "try:\n", + " fb = Feedback(session_id=\"s1\", query=\"q\", answer=\"a\", rating=3)\n", + " assert fb.comment == \"\"\n", + " assert isinstance(fb.created_at, datetime)\n", + " print(\"2.1 ✓\")\n", + "except Exception as e:\n", + " print(f\"2.1 ✗ {e}\")\n", + "\n", + "try:\n", + " # Validación de rating fuera de rango debe fallar\n", + " Feedback(session_id=\"s1\", query=\"q\", answer=\"a\", rating=10)\n", + " print(\"2.1 rating validation ✗ (aceptó 10)\")\n", + "except Exception:\n", + " print(\"2.1 rating validation ✓\")\n", + "\n", + "try:\n", + " class DummyStore:\n", + " async def save(self, feedback): pass\n", + " async def list_by_session(self, session_id): return []\n", + " # Structural subtyping: no hay assert directo en runtime, pero debe compilar\n", + " store: FeedbackStore = DummyStore()\n", + " print(\"2.2 ✓ (structural subtyping se validaría con mypy)\")\n", + "except NameError as e:\n", + " print(f\"2.2 ✗ {e}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "a782bd7b", + "metadata": {}, + "source": [ + "## 3. Infrastructure — LLM Provider\n", + "\n", + "Objetivo: 10-15 min. Recuerdas cómo se conecta un LLM externo a través del Protocol.\n", + "\n", + "Tu proyecto tiene `AnthropicLLM` en `src/researchos/infrastructure/llm/anthropic_llm.py`. Vas a implementar una versión simplificada del método `generate`.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4816d02a", + "metadata": {}, + "source": [ + "**3.1** Implementa la clase `SimpleAnthropicLLM` que satisface el Protocol `LLMProvider`. El método `generate` debe:\n", + "\n", + "1. Convertir la lista de `Message` domain al formato que espera el SDK de Anthropic (recuerda: el rol `system` se maneja aparte, no va en `messages`).\n", + "2. Llamar a `client.messages.create(...)` con el modelo `claude-haiku-4-5`, `max_tokens=1024`.\n", + "3. Devolver el texto del primer bloque de la respuesta.\n", + "\n", + "No necesitas implementar `stream` — deja un `raise NotImplementedError`. Tampoco necesitas correr esto contra la API real; solo la estructura.\n", + "\n", + "Pista: mira tu propio código en `infrastructure/llm/anthropic_llm.py` si te atoras. El punto no es memorizar el SDK, es entender el patrón.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ea91bb66", + "metadata": {}, + "outputs": [], + "source": [ + "from anthropic import Anthropic\n", + "# from researchos.domain.interfaces import LLMProvider # descomenta si vas a correr esto\n", + "# from researchos.domain.models import Message\n", + "\n", + "# Para el taller, redefinimos Message localmente si no lo tienes importado:\n", + "from pydantic import BaseModel\n", + "\n", + "class Message(BaseModel):\n", + " role: str\n", + " content: str\n", + "\n", + "\n", + "class SimpleAnthropicLLM:\n", + " def __init__(self, api_key: str | None = None, model: str = \"claude-haiku-4-5\"):\n", + " self.client = Anthropic(api_key=api_key)\n", + " self.model = model\n", + "\n", + " async def generate(self, messages: list[Message]) -> str:\n", + " # 1. Separar el mensaje system (si existe) del resto\n", + " # 2. Construir la lista de messages para el SDK (solo user/assistant)\n", + " # 3. Llamar a self.client.messages.create(...)\n", + " # 4. Devolver el texto del primer bloque\n", + " \n", + " # Tu código acá\n", + " pass\n", + "\n", + " async def stream(self, messages: list[Message]):\n", + " raise NotImplementedError\n" + ] + }, + { + "cell_type": "markdown", + "id": "d429d25d", + "metadata": {}, + "source": [ + "**3.2 — Conceptual.** En el Protocol `LLMProvider`, `generate` es `async def`. Pero el SDK de Anthropic (`self.client.messages.create`) es sincrónico. Explica en tus palabras:\n", + "\n", + "(a) ¿Se rompe algo por declarar `async def generate` aunque adentro llames a un método sincrónico?\n", + "\n", + "(b) ¿Cuándo sí importaría convertir la llamada interna a async (con `httpx.AsyncClient` o `anthropic.AsyncAnthropic`)?\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "c2ec8760", + "metadata": {}, + "source": [ + "## 4. Retrieval — VectorStore y BM25\n", + "\n", + "Objetivo: 15 min. Recuerdas cómo funciona la búsqueda vectorial y BM25, y por qué complementan.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4bb9077b", + "metadata": {}, + "source": [ + "**4.1 — Conceptual.** Da un ejemplo concreto de una query donde BM25 supera a la búsqueda vectorial, y otro ejemplo donde vectorial supera a BM25. Los ejemplos deben ser del dominio de ResearchOS (papers de ML/AI). Un párrafo por caso.\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "268fbce4", + "metadata": {}, + "source": [ + "**4.2** Implementa una clase `SimpleBM25` que satisface el Protocol `Retriever` (solo `search`). Debe:\n", + "\n", + "1. Recibir en `__init__` una `list[Document]` con `doc_id`, `text` y `metadata`.\n", + "2. Construir un índice BM25 en memoria (usa `rank_bm25.BM25Okapi`).\n", + "3. Tokenizar simple: `text.lower().split()`.\n", + "4. En `search(query, k)`, devolver los top-k Documents ordenados por score BM25 descendente, con el campo `score` populado sin mutar los originales.\n", + "\n", + "Para el ejercicio, usa un `Document` local simple (redefinido abajo).\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "86eb7a89", + "metadata": {}, + "outputs": [], + "source": [ + "from rank_bm25 import BM25Okapi\n", + "from pydantic import BaseModel, Field\n", + "\n", + "\n", + "class Document(BaseModel):\n", + " doc_id: str\n", + " text: str\n", + " metadata: dict = Field(default_factory=dict)\n", + " score: float = 0.0\n", + "\n", + "\n", + "class SimpleBM25:\n", + " def __init__(self, documents: list[Document]) -> None:\n", + " # Tu código acá: guardar documents, tokenizar corpus, construir BM25Okapi\n", + " pass\n", + "\n", + " async def search(self, query: str, k: int) -> list[Document]:\n", + " # Tu código acá:\n", + " # 1. Tokenizar la query\n", + " # 2. Obtener scores con self.bm25.get_scores(...)\n", + " # 3. Ordenar índices por score descendente, tomar top-k\n", + " # 4. Devolver los documents correspondientes con score actualizado (SIN MUTAR)\n", + " pass\n" + ] + }, + { + "cell_type": "markdown", + "id": "f7c41aba", + "metadata": {}, + "source": [ + "**4.3 — Check.** Corre la celda. Debe pasar sin error.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2460c382", + "metadata": {}, + "outputs": [], + "source": [ + "import asyncio\n", + "\n", + "async def _check_bm25():\n", + " docs = [\n", + " Document(doc_id=\"d1\", text=\"reinforcement learning from human feedback\"),\n", + " Document(doc_id=\"d2\", text=\"convolutional neural networks for images\"),\n", + " Document(doc_id=\"d3\", text=\"human feedback in language model training\"),\n", + " ]\n", + " bm25 = SimpleBM25(docs)\n", + " results = await bm25.search(\"human feedback\", k=2)\n", + " \n", + " assert len(results) == 2, f\"Expected 2, got {len(results)}\"\n", + " assert results[0].doc_id in {\"d1\", \"d3\"}, f\"Top result should mention 'human feedback', got {results[0].doc_id}\"\n", + " assert results[0].score > 0, \"Score should be populated\"\n", + " # No mutación: doc original de la lista sigue en score=0.0\n", + " assert docs[0].score == 0.0, \"Original docs must NOT be mutated\"\n", + " print(\"4.2 ✓\")\n", + "\n", + "asyncio.run(_check_bm25())\n" + ] + }, + { + "cell_type": "markdown", + "id": "6114f85d", + "metadata": {}, + "source": [ + "## 5. Application — RAG y Hybrid Search\n", + "\n", + "Objetivo: 15-20 min. Recuerdas el patrón RAG y el algoritmo RRF que implementaste en `retrieval_service.py`.\n" + ] + }, + { + "cell_type": "markdown", + "id": "773cdd67", + "metadata": {}, + "source": [ + "**5.1 — Conceptual.** Explica en tus palabras qué es Reciprocal Rank Fusion (RRF) y responde:\n", + "\n", + "(a) ¿Por qué RRF suma las contribuciones cuando un documento aparece en dos rankings, en lugar de promediarlas o quedarse con la mayor?\n", + "\n", + "(b) ¿Por qué el índice del rank arranca en 1 (no en 0)?\n", + "\n", + "(c) ¿Qué controla la constante `rrf_k` (típicamente 60)? Si la subo a 200, ¿qué cambia en la práctica?\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "30ccde06", + "metadata": {}, + "source": [ + "**5.2** Implementa `hybrid_search_simple` sin mirar tu propio código. Debe:\n", + "\n", + "1. Recibir `query: str`, `retrievers: list[Retriever]`, `k: int = 5`, `rrf_k: int = 60`.\n", + "2. Validar que `retrievers` no esté vacío (raise `ValueError`).\n", + "3. Ejecutar los retrievers en paralelo con `asyncio.gather`, cada uno pidiendo `k * 2` candidatos.\n", + "4. Acumular scores RRF por `doc_id` (`1 / (rrf_k + rank)`), con rank 1-indexed.\n", + "5. Devolver los top-k Documents ordenados por RRF descendente, con `score` = RRF score.\n", + "\n", + "Cada documento único aparece una vez en el resultado.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "99c92e62", + "metadata": {}, + "outputs": [], + "source": [ + "import asyncio\n", + "from typing import Protocol\n", + "\n", + "\n", + "class Retriever(Protocol):\n", + " async def search(self, query: str, k: int) -> list[Document]:\n", + " ...\n", + "\n", + "\n", + "async def hybrid_search_simple(\n", + " query: str,\n", + " retrievers: list[Retriever],\n", + " k: int = 5,\n", + " rrf_k: int = 60,\n", + ") -> list[Document]:\n", + " # Tu código acá\n", + " pass\n" + ] + }, + { + "cell_type": "markdown", + "id": "0247317c", + "metadata": {}, + "source": [ + "**5.3 — Check.** Corre la celda. El documento que aparece en ambos retrievers debe quedar primero.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b3d44080", + "metadata": {}, + "outputs": [], + "source": [ + "class _MockRetriever:\n", + " def __init__(self, docs): self.docs = docs\n", + " async def search(self, query, k): return self.docs[:k]\n", + "\n", + "\n", + "async def _check_hybrid():\n", + " r_a = _MockRetriever([\n", + " Document(doc_id=\"doc1\", text=\"t1\"),\n", + " Document(doc_id=\"doc2\", text=\"t2\"),\n", + " Document(doc_id=\"doc3\", text=\"t3\"),\n", + " ])\n", + " r_b = _MockRetriever([\n", + " Document(doc_id=\"doc2\", text=\"t2\"),\n", + " Document(doc_id=\"doc4\", text=\"t4\"),\n", + " Document(doc_id=\"doc5\", text=\"t5\"),\n", + " ])\n", + " \n", + " results = await hybrid_search_simple(\"q\", retrievers=[r_a, r_b], k=5)\n", + " \n", + " assert results[0].doc_id == \"doc2\", f\"doc2 should win, got {results[0].doc_id}\"\n", + " assert results[0].score > results[1].score, \"Top should have higher RRF\"\n", + " ids = [r.doc_id for r in results]\n", + " assert len(ids) == len(set(ids)), \"Docs must be deduplicated\"\n", + " print(\"5.2 ✓ orden correcto, doc2 primero, sin duplicados\")\n", + " \n", + " try:\n", + " await hybrid_search_simple(\"q\", retrievers=[], k=5)\n", + " print(\"5.2 ✗ retrievers vacío no lanza error\")\n", + " except ValueError:\n", + " print(\"5.2 ✓ retrievers vacío lanza ValueError\")\n", + "\n", + "\n", + "asyncio.run(_check_hybrid())\n" + ] + }, + { + "cell_type": "markdown", + "id": "f53e2ade", + "metadata": {}, + "source": [ + "## 6. Async y concurrencia\n", + "\n", + "Objetivo: 10-15 min. Recuerdas cuándo async ayuda y cuándo no, y los antipatrones comunes.\n" + ] + }, + { + "cell_type": "markdown", + "id": "5c7e5ce1", + "metadata": {}, + "source": [ + "**6.1 — Conceptual.** Regla mental que documentaste en learnings el 15/04: \"¿Esperas algo externo (API, disco, red)? → `async def`. ¿Solo calculas en memoria? → `def` normal\". Aplica esa regla a cada función siguiente y justifica:\n", + "\n", + "(a) `overlap_chunking(text, paper_id, chunk_size, overlap) -> list[Chunk]`\n", + "\n", + "(b) `download_pdf(url) -> bytes`\n", + "\n", + "(c) `embed_text(text) -> list[float]` (usa un modelo local en CPU con sentence-transformers)\n", + "\n", + "(d) `send_telegram_message(bot, chat_id, text) -> None`\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "ce45ed39", + "metadata": {}, + "source": [ + "**6.2** El código siguiente pretende descargar tres URLs en paralelo. Tiene un bug conceptual que hace que corra en secuencia. Identifícalo y corrígelo.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb90030b", + "metadata": {}, + "outputs": [], + "source": [ + "import asyncio\n", + "import httpx\n", + "\n", + "\n", + "async def fetch(client: httpx.AsyncClient, url: str) -> str:\n", + " response = await client.get(url)\n", + " return response.text\n", + "\n", + "\n", + "async def fetch_all_broken(urls: list[str]) -> list[str]:\n", + " # Este código está mal — corre secuencial. Explica por qué y arregla.\n", + " async with httpx.AsyncClient() as client:\n", + " results = []\n", + " for url in urls:\n", + " result = await fetch(client, url)\n", + " results.append(result)\n", + " return results\n", + "\n", + "\n", + "# Escribe la versión corregida acá\n", + "async def fetch_all_fixed(urls: list[str]) -> list[str]:\n", + " # Tu código acá\n", + " pass\n", + "\n", + "\n", + "# Explicación del bug (celda markdown abajo):\n" + ] + }, + { + "cell_type": "markdown", + "id": "380747eb", + "metadata": {}, + "source": [ + "*Tu explicación del bug:*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "084dd267", + "metadata": {}, + "source": [ + "## 7. Testing con mocks\n", + "\n", + "Objetivo: 10-15 min. Recuerdas cómo se testea el application layer sin tocar sistemas reales.\n" + ] + }, + { + "cell_type": "markdown", + "id": "448c824c", + "metadata": {}, + "source": [ + "**7.1 — Conceptual.** ¿Cuál es la diferencia entre `@pytest.mark.unit` y `@pytest.mark.integration` en tu proyecto? Da un ejemplo concreto de cada uno y explica qué haría CI si separaras el pipeline en dos etapas.\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "c916c072", + "metadata": {}, + "source": [ + "**7.2** Escribe un `MockFeedbackStore` en memoria que satisfaga el Protocol `FeedbackStore` que definiste en la sección 2. Debe funcionar como el `MockVectorStore` de tu `conftest.py` — sin BD real, guardando en un dict interno.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7cb9935d", + "metadata": {}, + "outputs": [], + "source": [ + "# Recordatorio de tu Feedback y FeedbackStore de la sección 2\n", + "# (los redefiníamos abajo si te acomoda tenerlos a la mano)\n", + "\n", + "\n", + "class MockFeedbackStore:\n", + " def __init__(self):\n", + " # Tu código acá: estructura interna para almacenar por session\n", + " pass\n", + "\n", + " async def save(self, feedback) -> None:\n", + " # Tu código acá\n", + " pass\n", + "\n", + " async def list_by_session(self, session_id: str) -> list:\n", + " # Tu código acá\n", + " pass\n" + ] + }, + { + "cell_type": "markdown", + "id": "6229f35a", + "metadata": {}, + "source": [ + "**7.3** Escribe un test `test_mock_feedback_store_saves_and_lists` que:\n", + "\n", + "1. Crea un `MockFeedbackStore`\n", + "2. Guarda dos `Feedback` de la misma sesión y uno de otra sesión\n", + "3. Verifica que `list_by_session` de la primera sesión devuelva solo dos, en el orden en que se guardaron\n", + "\n", + "No necesitas `@pytest.mark.asyncio` acá — corre con `asyncio.run` como los otros checks del taller.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "181d9386", + "metadata": {}, + "outputs": [], + "source": [ + "async def test_mock_feedback_store_saves_and_lists():\n", + " # Tu código acá\n", + " pass\n", + "\n", + "\n", + "asyncio.run(test_mock_feedback_store_saves_and_lists())\n", + "print(\"7.3 ✓ si llegaste hasta acá sin AssertionError\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "68718dbd", + "metadata": {}, + "source": [ + "## 8. Meta arquitectural — extender el sistema\n", + "\n", + "Objetivo: 15 min. Estas son las preguntas que definen si internalizaste la arquitectura o solo la seguiste. No hay código — solo diseño en palabras.\n" + ] + }, + { + "cell_type": "markdown", + "id": "f794c8a6", + "metadata": {}, + "source": [ + "**8.1** Alguien te pide agregar Qdrant como vector store alternativo a Chroma. En tu proyecto tal cual está hoy, ¿qué archivos habría que crear o modificar? Sé concreto: rutas y qué va en cada uno. ¿Qué archivos NO deberías tocar?\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n", + "\n", + "---\n", + "\n", + "**8.2** Quieres agregar un canal nuevo: Slack. ¿Dónde va el código de Slack? ¿Qué relación tiene con `application/services/rag_service.py`? Traza el flujo de un mensaje entrando por Slack hasta la respuesta.\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n", + "\n", + "---\n", + "\n", + "**8.3** El PO de Pensiones (Paola) te pide un endpoint HTTP que reciba un caso y devuelva una recomendación. En términos de tu arquitectura, ¿en qué capa vive un endpoint HTTP? ¿Cuál es el rol de FastAPI: parte del motor o parte del canal?\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n", + "\n", + "---\n", + "\n", + "**8.4 — La pregunta clave.** En una entrevista de AI Engineering te preguntan: \"Explica cómo evaluarías un sistema RAG en producción, sin data leakage\". Con lo que sabes hoy, escribe una respuesta de 3-5 oraciones. Menciona: qué mides, cómo obtienes ground truth, qué haces con las queries que fallan.\n", + "\n", + "*Tu respuesta:*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "41bba9ba", + "metadata": {}, + "source": [ + "---\n", + "\n", + "## Apéndice: auto-verificación\n", + "\n", + "No mires esta sección antes de terminar. Es orientativa — hay más de una forma correcta.\n", + "\n", + "### 1. Clean Architecture\n", + "\n", + "**1.1.** Dependencia: `infrastructure → application → domain`, nunca al revés. `domain` no importa nada de las otras dos; `application` importa de `domain` (Protocols y modelos) pero no de `infrastructure`; `infrastructure` importa de `domain` para implementar los Protocols. Si `domain` importa de `infrastructure`, cambiar de Chroma a Qdrant te obliga a tocar código de negocio, y los tests unitarios de dominio dejan de correrse sin instalar todo el stack.\n", + "\n", + "**1.2.** Un Protocol define un contrato estructural. Cualquier clase con los métodos de la firma correcta lo satisface, sin heredar. ABC exige herencia explícita (`class Foo(ABC):`). Ejemplo: `VectorStore` con `ChromaVectorStore` implementándolo por duck typing.\n", + "\n", + "**1.3.** `BM25Retriever` no puede persistir (no tiene `upsert` real, es en memoria). Si implementara `VectorStore`, mentiría sobre su contrato. Separar `Retriever` (solo `search`) permite que `BM25Retriever` sea honesto sobre lo que hace, y que `hybrid_search` reciba una lista genérica de retrievers sin distinguir tipos.\n", + "\n", + "**1.4.** (a) Cada agente es autoresponsable — no hay comportamiento oculto heredado; (b) testear una función pura es más simple que testear un método con `super().__init__` de por medio; (c) agregar un método a `agent_utils.py` no obliga a todos los agentes a adoptarlo.\n", + "\n", + "### 2. Domain\n", + "\n", + "```python\n", + "class Feedback(BaseModel):\n", + " session_id: str\n", + " query: str\n", + " answer: str\n", + " rating: int = Field(ge=1, le=5)\n", + " comment: str = \"\"\n", + " created_at: datetime = Field(default_factory=datetime.now)\n", + "\n", + "\n", + "class FeedbackStore(Protocol):\n", + " async def save(self, feedback: Feedback) -> None: ...\n", + " async def list_by_session(self, session_id: str) -> list[Feedback]: ...\n", + "```\n", + "\n", + "### 3. LLM\n", + "\n", + "Estructura mínima del `generate`:\n", + "\n", + "```python\n", + "system_msg = next((m.content for m in messages if m.role == \"system\"), None)\n", + "user_msgs = [{\"role\": m.role, \"content\": m.content} for m in messages if m.role != \"system\"]\n", + "kwargs = {\"model\": self.model, \"max_tokens\": 1024, \"messages\": user_msgs}\n", + "if system_msg: kwargs[\"system\"] = system_msg\n", + "response = self.client.messages.create(**kwargs)\n", + "return response.content[0].text\n", + "```\n", + "\n", + "**3.2.** (a) No se rompe: `async def` con contenido sincrónico se resuelve inmediatamente. Es válido. (b) Importa cuando llamas al LLM en paralelo con otras coroutines (p.ej. varios agentes en `asyncio.gather`), porque un `create` sincrónico bloquea el event loop y anula el paralelismo.\n", + "\n", + "### 4. Retrieval\n", + "\n", + "**4.1.** BM25 supera a vectorial en queries con siglas o nombres propios: `\"BERT vs GPT-3\"` (coincidencia exacta). Vectorial supera a BM25 en queries semánticas: `\"papers on models that reason step by step\"` (encuentra chain-of-thought aunque no diga esas palabras).\n", + "\n", + "Estructura de `SimpleBM25.search`:\n", + "\n", + "```python\n", + "tokens = query.lower().split()\n", + "scores = self.bm25.get_scores(tokens)\n", + "top_idx = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)[:k]\n", + "return [self.documents[i].model_copy(update={\"score\": float(scores[i])}) for i in top_idx]\n", + "```\n", + "\n", + "### 5. Hybrid\n", + "\n", + "**5.1.** (a) La suma premia consenso: docs que dos retrievers coinciden en rankear alto suben más que docs que solo uno rankeó alto. Promediar diluiría la señal; quedarse con la mayor ignoraría el consenso. (b) Con rank 0-indexed, el primer doc daría `1/rrf_k` (relativamente grande) y los demás caerían muy rápido. 1-indexed suaviza la curva y es la convención del paper original. (c) `rrf_k` controla la suavidad. Con `rrf_k=60`, el doc en rank 1 aporta `1/61 ≈ 0.0164`; en rank 10, `1/70 ≈ 0.0143`. Con `rrf_k=200`, la diferencia entre rank 1 y rank 10 se aplasta — todos aportan casi lo mismo. Un `rrf_k` alto hace que la fusión sea más democrática entre retrievers; uno bajo premia más al top.\n", + "\n", + "### 6. Async\n", + "\n", + "**6.1.** (a) `def` normal, memoria pura. (b) `async def`, red. (c) `def` normal, CPU en memoria (o async si delegas a un servidor de embeddings). (d) `async def`, red.\n", + "\n", + "**6.2.** El bug es que `await` dentro de un `for` secuencia las llamadas. La corrección:\n", + "\n", + "```python\n", + "async def fetch_all_fixed(urls):\n", + " async with httpx.AsyncClient() as client:\n", + " return await asyncio.gather(*[fetch(client, u) for u in urls])\n", + "```\n", + "\n", + "### 7. Testing\n", + "\n", + "**7.1.** `unit` no toca red, disco ni servicios externos — corre en milisegundos. `integration` sí — más lento, se corre menos frecuente. En CI, `make test` corre unit en cada commit; `make test-all` corre integración solo en merge a main o nightly.\n", + "\n", + "**7.2.**\n", + "\n", + "```python\n", + "class MockFeedbackStore:\n", + " def __init__(self):\n", + " self._by_session: dict[str, list[Feedback]] = {}\n", + "\n", + " async def save(self, feedback):\n", + " self._by_session.setdefault(feedback.session_id, []).append(feedback)\n", + "\n", + " async def list_by_session(self, session_id):\n", + " return list(self._by_session.get(session_id, []))\n", + "```\n", + "\n", + "### 8. Meta\n", + "\n", + "**8.1.** Crear: `infrastructure/retrieval/qdrant.py` con clase `QdrantVectorStore` que satisface `VectorStore`. Modificar: `config.py` (agregar opción `Literal[\"chroma\", \"qdrant\"]` en settings) y factory/DI donde se instancia el vector store. NO tocar: `domain/`, `application/services/*` (ni siquiera `retrieval_service.py`), tests de aplicación (los mocks siguen sirviendo).\n", + "\n", + "**8.2.** Slack va en `infrastructure/bot/slack.py`. Recibe mensaje → llama a `rag_service.answer_query(text, llm, store)` → devuelve el string → lo envía a Slack. `rag_service` no sabe que existe Slack, es agnóstico al canal.\n", + "\n", + "**8.3.** El endpoint HTTP vive en `infrastructure/api/routers/`. FastAPI es canal, no motor — expone el mismo motor (`rag_service`, agentes) por HTTP. La lógica de recomendación no sabe si vino de FastAPI, Telegram o un CLI.\n", + "\n", + "**8.4.** Referencia (una versión posible): \"Medir faithfulness (¿el LLM inventa cosas no soportadas por los docs?), context precision (¿los docs recuperados son útiles?) y answer relevancy. Ground truth: LLM-as-judge con un modelo distinto al que genera (evita sesgo de auto-evaluación); complementado con eval humano sobre una muestra. Las queries que fallan van a un dataset de regresión: se etiquetan manualmente, se agregan al eval automático, y el pipeline CI/CD bloquea deploys que hagan bajar el score. Diferenciar métricas offline (dataset fijo, corren en cada PR) de online (feedback en producción, thumbs up/down, latencia real).\"\n", + "\n", + "---\n", + "\n", + "## Cierre\n", + "\n", + "Cuando termines, en el chat contame:\n", + "\n", + "- Qué secciones te salieron sin friction.\n", + "- Qué secciones te forzaron a abrir tu propio código en `src/researchos/`.\n", + "- Qué preguntas quedaron sin respuesta clara (esas son los huecos reales).\n", + "- Qué error de tus respuestas te sorprendió al comparar con el apéndice.\n", + "\n", + "Con eso ajustamos el arranque de V2 y decidimos si conviene afianzar algún tema antes de tocar código nuevo.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.11.8" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From c965130d33f827130fbe92582d0e7da17064f618 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Tue, 28 Jul 2026 10:25:17 -0500 Subject: [PATCH 47/55] chore(notebooks): Progress in the review workshop --- notebooks/taller_retorno_researchos.ipynb | 133 +++++++++++++++++----- 1 file changed, 105 insertions(+), 28 deletions(-) diff --git a/notebooks/taller_retorno_researchos.ipynb b/notebooks/taller_retorno_researchos.ipynb index 0b2e05c..44560e0 100644 --- a/notebooks/taller_retorno_researchos.ipynb +++ b/notebooks/taller_retorno_researchos.ipynb @@ -68,9 +68,7 @@ "\n", "**1.3** En tu proyecto tienes `Retriever` y `VectorStore` como Protocols separados. `VectorStore` tiene `search` y `upsert`; `Retriever` solo tiene `search`. ¿Por qué esta separación? ¿Qué se te complicaría si `BM25Retriever` implementara `VectorStore` directamente?\n", "\n", - "*Tu respuesta:*\n", - "\n", - "\n", + "Un retriever es cualquier recuperador, así que la separación se tiene para separar cualquier recuperador de un recuperador que es específicamente una base vectorial. Si se implementara directamente vectorStore habría problemas en la definición de la búsqueda híbrida ya que tendría que hacer 2 implementaciones: una para bases vectoriales y otroa para recuperadores como el BM25\n", "\n", "---\n", "\n", @@ -78,7 +76,7 @@ "\n", "*Tu respuesta:*\n", "\n", - "\n" + "No tengo respuesta" ] }, { @@ -110,7 +108,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 6, "id": "721d7653", "metadata": {}, "outputs": [], @@ -118,7 +116,20 @@ "from datetime import datetime\n", "from pydantic import BaseModel, Field\n", "\n", - "# Tu código acá\n", + "\n", + "class Feedback(BaseModel):\n", + " \"\"\"\n", + " Feedback dado por el usuario acerca de la respuesta que obtuvo del sistema\n", + " \"\"\"\n", + "\n", + " session_id: str\n", + " query: str\n", + " answer: str\n", + " rating: int = Field(ge=1, le=5)\n", + " comment: str = Field(default=\"\")\n", + " created_at: datetime = Field(default_factory=lambda: datetime.now())\n", + "\n", + "\n", "\n", "\n" ] @@ -138,16 +149,29 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "id": "b0908b2a", "metadata": {}, "outputs": [], "source": [ "from typing import Protocol\n", "\n", - "# Tu código acá\n", + "class FeedbackStore(Protocol):\n", + " \"\"\"\n", + " Contrato para cualquier servicio que se encargue de almacenar y gestionar los feedbacks dados por los ususarios\n", + " \"\"\"\n", + "\n", + " async def save(feedback: Feedback) -> None:\n", + " \"\"\"\n", + " Almacena el feedback dado por el usuario\n", + " \"\"\"\n", + " ...\n", "\n", - "\n" + " async def list_by_session(session_id: str) -> list[Feedback]:\n", + " \"\"\"\n", + " Enlista todos los feedback que ha dado un usuario (id) a lo largo de su historia\n", + " \"\"\"\n", + " ...\n" ] }, { @@ -160,10 +184,20 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "id": "e54a43b6", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "2.1 ✓\n", + "2.1 rating validation ✓\n", + "2.2 ✓ (structural subtyping se validaría con mypy)\n" + ] + } + ], "source": [ "# Check sección 2\n", "try:\n", @@ -222,7 +256,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 15, "id": "ea91bb66", "metadata": {}, "outputs": [], @@ -245,13 +279,30 @@ " self.model = model\n", "\n", " async def generate(self, messages: list[Message]) -> str:\n", + "\n", + " formatted_messages = []\n", + "\n", " # 1. Separar el mensaje system (si existe) del resto\n", + " for message in messages:\n", + " if messages.role == 'system': \n", + " system_message = message.content\n", + "\n", " # 2. Construir la lista de messages para el SDK (solo user/assistant)\n", + " else:\n", + " formatted_messages.append({\"role\": message.role,\n", + " \"content\": message.content})\n", + "\n", + "\n", " # 3. Llamar a self.client.messages.create(...)\n", + " response = await self.client.messages.create(\n", + " system=system_message, \n", + " model=self.model,\n", + " max_tokens=1024\n", + " )\n", + "\n", " # 4. Devolver el texto del primer bloque\n", - " \n", - " # Tu código acá\n", - " pass\n", + " if response.content and len(response.content) > 0:\n", + " return response.content[0].text\n", "\n", " async def stream(self, messages: list[Message]):\n", " raise NotImplementedError\n" @@ -265,12 +316,10 @@ "**3.2 — Conceptual.** En el Protocol `LLMProvider`, `generate` es `async def`. Pero el SDK de Anthropic (`self.client.messages.create`) es sincrónico. Explica en tus palabras:\n", "\n", "(a) ¿Se rompe algo por declarar `async def generate` aunque adentro llames a un método sincrónico?\n", + "No se rompe nada, simplemente al ser el método del SDK síncrono trabajará como un proceso iterativo donde se hace de a 1 consulta a la vez\n", "\n", "(b) ¿Cuándo sí importaría convertir la llamada interna a async (con `httpx.AsyncClient` o `anthropic.AsyncAnthropic`)?\n", - "\n", - "*Tu respuesta:*\n", - "\n", - "\n" + "Cuando tengo muchos usuarios preguntando al tiempo, en una aplicación de ese estilo necesito volver asíncrono el proceso para que diferentes usuarios puedan usar mi aplicación al mismo tiempo" ] }, { @@ -292,7 +341,8 @@ "\n", "*Tu respuesta:*\n", "\n", - "\n" + "**BM25 supera al vectorial**: \"What are the specific differences between the XGBoost method and the RandomForest method?\"\n", + "**vectorial supera a BM25**: \"Identify and rank the machine learning methods with the highest accuracy for classifying cars in images\"" ] }, { @@ -312,7 +362,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 16, "id": "86eb7a89", "metadata": {}, "outputs": [], @@ -331,15 +381,24 @@ "class SimpleBM25:\n", " def __init__(self, documents: list[Document]) -> None:\n", " # Tu código acá: guardar documents, tokenizar corpus, construir BM25Okapi\n", - " pass\n", + " self.documents = documents\n", + " self.tokenizer = lambda text: text.lower().split()\n", + " self._corpus = [self.tokenizer(doc.text) for doc in documents]\n", + " self.bm25 = BM25Okapi(self._corpus)\n", "\n", " async def search(self, query: str, k: int) -> list[Document]:\n", " # Tu código acá:\n", " # 1. Tokenizar la query\n", + " tokenized_query = self.tokenizer(query)\n", " # 2. Obtener scores con self.bm25.get_scores(...)\n", + " scores = self.bm25.get_scores(tokenized_query)\n", " # 3. Ordenar índices por score descendente, tomar top-k\n", + " top_indices = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)[:k]\n", " # 4. Devolver los documents correspondientes con score actualizado (SIN MUTAR)\n", - " pass\n" + " return [\n", + " self.documents[i].model_copy(update={\"score\": float(scores[i])})\n", + " for i in top_indices\n", + " ]\n" ] }, { @@ -352,10 +411,18 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 20, "id": "2460c382", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4.2 ✓\n" + ] + } + ], "source": [ "import asyncio\n", "\n", @@ -375,7 +442,8 @@ " assert docs[0].score == 0.0, \"Original docs must NOT be mutated\"\n", " print(\"4.2 ✓\")\n", "\n", - "asyncio.run(_check_bm25())\n" + "# asyncio.run(_check_bm25())\n", + "await _check_bm25()\n" ] }, { @@ -403,7 +471,8 @@ "\n", "*Tu respuesta:*\n", "\n", - "\n" + "a. Porque los rankings pueden tener escalas diferentes lo que podría sesgar el cálculo\n", + "b. " ] }, { @@ -839,12 +908,20 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "researchos (3.11.8)", "language": "python", "name": "python3" }, "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", "version": "3.11.8" } }, From 95404dd5ad9e92c5fd003ac61acd9076980a25fe Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Wed, 29 Jul 2026 14:13:25 -0500 Subject: [PATCH 48/55] chore(skills): The skill to draw, recreate, or define the project's architecture in Drawio format is added --- .claude/skills/arquitectura-drawio/SKILL.md | 103 ++++++++ .../arquitectura-drawio/references/estilo.md | 169 +++++++++++++ .../arquitectura-drawio/references/iconos.md | 80 +++++++ .../arquitectura-drawio/scripts/.gitignore | 3 + .../scripts/check_layout.py | 213 +++++++++++++++++ .../arquitectura-drawio/scripts/drawio_kit.py | 226 ++++++++++++++++++ .../arquitectura-drawio/scripts/ejemplo.py | 81 +++++++ .../arquitectura-drawio/scripts/gcp_icon.py | 98 ++++++++ .../arquitectura-drawio/scripts/glyph.py | 135 +++++++++++ 9 files changed, 1108 insertions(+) create mode 100644 .claude/skills/arquitectura-drawio/SKILL.md create mode 100644 .claude/skills/arquitectura-drawio/references/estilo.md create mode 100644 .claude/skills/arquitectura-drawio/references/iconos.md create mode 100644 .claude/skills/arquitectura-drawio/scripts/.gitignore create mode 100644 .claude/skills/arquitectura-drawio/scripts/check_layout.py create mode 100644 .claude/skills/arquitectura-drawio/scripts/drawio_kit.py create mode 100644 .claude/skills/arquitectura-drawio/scripts/ejemplo.py create mode 100644 .claude/skills/arquitectura-drawio/scripts/gcp_icon.py create mode 100644 .claude/skills/arquitectura-drawio/scripts/glyph.py diff --git a/.claude/skills/arquitectura-drawio/SKILL.md b/.claude/skills/arquitectura-drawio/SKILL.md new file mode 100644 index 0000000..e176e7b --- /dev/null +++ b/.claude/skills/arquitectura-drawio/SKILL.md @@ -0,0 +1,103 @@ +--- +name: arquitectura-drawio +description: Genera y edita diagramas de arquitectura en draw.io (.drawio) con el estilo visual de la organización — paleta, cajas por categoría, edges ortogonales animados theme-aware y logos oficiales de la tecnología que sea (GCP, AWS, Azure, LangChain, React, Node, Python, Docker…) con glifo genérico de fallback. Agnóstica del stack. Usar cuando se pida crear, rediseñar o exportar una arquitectura/diagrama en draw.io, o convertir una descripción o boceto en un .drawio con el look de la casa. +--- + +# Arquitecturas draw.io con el estilo de la organización + +Skill **portable y transversal**: para llevarla a otro repo, copiar la carpeta +`.claude/skills/arquitectura-drawio/` completa. Los scripts son solo stdlib de Python; `gcp_icon.py` +descarga iconos desde URLs públicas de Google (internet una vez, luego cacheado). + +## Principio + +Un `.drawio` es XML de mxGraph (texto plano). Lo mejor es **generarlo directamente** con el motor +`scripts/drawio_kit.py`, que ya trae los tokens de estilo de la casa, asegura IDs únicos, escapa el +XML y valida el resultado. No hace falta ningún MCP ni servicio online. + +## Flujo de trabajo + +1. **Entender la arquitectura**: nodos, capas, flujos, qué herramienta es cada nodo. Si es ambiguo, + preguntar antes de dibujar. +2. **Generar** con `drawio_kit` (ver `scripts/ejemplo.py` como plantilla). Escribir un script corto + en el scratchpad que: + - agregue `scripts/` de esta skill a `sys.path` e importe `Diagram, STYLE, EDGE` y, para + iconos, `glyph.logo` (logo oficial de cualquier tecnología) y `glyph.material` (fallback); + - defina nodos con coordenadas en grilla y edges con **anclajes explícitos** + (`exit=`/`entry=`) — clave para que no se traslapen las flechas; + - llame `Diagram.write(ruta)`, que valida (IDs únicos, edges íntegros, XML) y avisa si supera 500 KB. +3. **Agregar la leyenda** (obligatorio, ver sección *Leyenda*): un bloque que explique qué significa + cada color de flecha y de caja antes de dar por terminado el diagrama. +4. **Revisar** abriendo el `.drawio` en VS Code con la extensión *Draw.io Integration* + (`hediet.vscode-drawio`). Iterar con el usuario y confirmar que los iconos rendericen. + +Alternativa sin scripts: escribir el XML a mano siguiendo `references/estilo.md` (útil para retoques +puntuales). Estructura mínima por página: `` +` … celdas … `. + +## Estilo + +Paleta, strings de estilo, edges y **reglas de layout para evitar traslapes** (fan-out/gather con +anclajes, almacenes pegados a su nodo) están en **`references/estilo.md`**. Rasgo distintivo de la +casa: edges `orthogonalEdgeStyle` con `flowAnimation=1` y color `light-dark(...)` (theme-aware). + +## Iconos (agnóstico de tecnología) + +Las tecnologías varían por proyecto. **Regla: logo oficial de la tecnología; si no existe, glifo +genérico.** Detalle y ejemplos en **`references/iconos.md`**. Orden: + +1. **Logo oficial** — `from glyph import logo; logo(nombre, color)`. Punto de entrada único que + resuelve marcas (simple-icons: `langchain`, `react`, `nodejs`, `python`, `docker`, `awslambda`, + `microsoftazure`…) y productos Google Cloud (`vertex_ai`, `bigquery`, `cloud_run`…). +2. **Glifo genérico de fallback** — si `logo()` no encuentra la tecnología (p. ej. Google ADK, + Langfuse, un componente custom), usa `glyph.material(symbol, color)` en vez de una caja vacía + (`python scripts/glyph.py list-suggested` mapea tipo→glifo). +3. **Librería nativa de draw.io** — más liviana, look distinto, riesgo de icono en blanco. +4. **Imágenes del usuario** — último recurso; al usarla **informar siempre** del peso extra, que + son imágenes pegadas, el riesgo de PII en capturas y las licencias de terceros. + +## Legibilidad (evitar traslapes) — obligatorio + +El flujo debe leerse claro, sin flechas ni textos superpuestos. `references/estilo.md` tiene las +reglas (separación mínima, anclajes `exit`/`entry`, `points=` para rutear alrededor de nodos). Antes +de entregar, corre el linter y **itera hasta que no queden traslapes duros**: + +```bash +python scripts/check_layout.py .drawio # nodos/etiquetas/flechas superpuestos +``` + +`drawio_kit.write()` ya lo ejecuta y avisa. Confirma también visualmente en draw.io. + +## Leyenda (obligatorio) + +Toda arquitectura entregada **debe incluir una leyenda descriptiva** — sin ella el diagrama no está +terminado. Un diagrama codifica significado en el color de las flechas y de las cajas; la leyenda es +lo que hace ese código legible para quien no lo dibujó. Ubícala en una franja al pie (o en una esquina +libre) y cubre, como mínimo: + +- **Color de cada flecha**: qué relación representa. En el estilo de la casa (ver `EDGE` en + `references/estilo.md`) el convenio es: **verde** = flujo de datos; **azul** = orquestación / control + (quién invoca o ejecuta a quién); **gris punteada** = consumo de un servicio o recurso compartido + (lectura/escritura). Si usas otros colores o relaciones, explícalos igual. +- **Color de cada caja**: qué tipo de componente es (p. ej. verde = paso determinista, azul = + servicio/modelo LLM, rojo = orquestador, blanco = almacén de apoyo, amarillo = salida). Describe + solo las categorías que realmente aparezcan en el diagrama. +- **Cualquier otra convención relevante**: iconos sueltos = fuentes/actores externos, que las flechas + van animadas e indican el sentido del flujo, líneas punteadas vs. sólidas, etc. + +Dibuja las muestras de flecha como edges reales (con su mismo `style`) para que el color y la animación +coincidan con el diagrama, y usa chips de color con el `fillColor`/`strokeColor` de cada arquetipo de +caja. Mantén la leyenda dentro del `pageHeight` y pásala por el linter como el resto. + +## Reglas + +- **Todos los conectores van animados** (`flowAnimation=1`); el kit lo garantiza en `EDGE`. +- **Incluye siempre una leyenda descriptiva** de colores de flecha y de caja (ver sección *Leyenda*); + un diagrama sin leyenda está incompleto. +- **Esquinas con radio fijo** (no la curva grande por defecto de `rounded=1`, que tapa el texto): dos + niveles, ya incluidos en `STYLE` (`ARC` y `ARC_ZONE`) — cajas de componente `absoluteArcSize=1` (muy + sutil) y contenedores/zonas azules punteados `absoluteArcSize=2` (esquina algo más marcada). +- Sin PII ni datos internos sensibles en etiquetas, notas o imágenes embebidas. +- Mantener el `.drawio` liviano (iconos SVG, no PNG pesados): el pre-commit típico bloquea > 500 KB. +- No prometer que rendericen los iconos sin que el usuario lo confirme en draw.io; si alguno sale en + blanco, revisar el data URI (ver `references/iconos.md`). diff --git a/.claude/skills/arquitectura-drawio/references/estilo.md b/.claude/skills/arquitectura-drawio/references/estilo.md new file mode 100644 index 0000000..b36ea78 --- /dev/null +++ b/.claude/skills/arquitectura-drawio/references/estilo.md @@ -0,0 +1,169 @@ +# Tokens de estilo — arquitecturas draw.io de la organización + +Destilado de los diagramas de referencia de la organización. `scripts/drawio_kit.py` ya trae estos +tokens en los dicts `STYLE` y `EDGE`; este documento es para consulta o para escribir el XML a mano. + +## Paleta + +| Rol | Color | +|---|---| +| Texto primario | `#4B5259` | +| Texto atenuado | `#9E9E9E` / `#999999` / `#777777` | +| Azul de marca (bordes, edges) | `#4284F3` | +| Azul secundario (acentos/iconos) | `#5184F3` | +| Azul banner (relleno sólido, texto blanco) | `#4DA1F5` | +| Azul icono (actor ios7) | `#0080F0` | +| Verde datos / éxito | `#66CC00` (edges) · `#82b366` (bordes) | +| Borde neutro | `#dddddd` (con `shadow=1`) | + +Pasteles estándar de draw.io para categorizar cajas: azul `#dae8fc` (LLM/servicio), verde `#d5e8d4` +(determinista), rojo `#f8cecc` (crítico/PII), naranja `#ffe6cc` (gate), amarillo `#fff2cc` (salida). + +## Tipografía y tamaños + +Fuente por defecto de draw.io. `fontSize` 12 en nodos, 15 en banners, 10–11 en detalles/almacenes. +Tamaños de caja habituales: tarjeta **160×60** o **170×60**, almacén **≈190×44**, actor **48×60**. + +## Esquinas redondeadas fijas (dos niveles) + +Las cajas redondeadas llevan un **radio fijo en píxeles** (`absoluteArcSize`), no la curva grande y +**proporcional al tamaño** que aplica `rounded=1` a secas — esa curva, en cajas anchas o de poca +altura, tapa el texto alineado a la izquierda o lo saca por las esquinas. Se usan **dos niveles**: + +- **Cajas de componente** (tarjeta, LLM, determinista, crítico, salida): `arcSize=6;absoluteArcSize=1;` + — radio muy sutil. +- **Contenedores / zonas** (los recuadros azules punteados grandes): `arcSize=6;absoluteArcSize=2;` + — un poco más marcado para que la esquina se note en las cajas grandes. + +`drawio_kit` ya lo inyecta en `STYLE` (constantes `ARC` y `ARC_ZONE`); si escribes el XML a mano, añádelo. + +## Strings de estilo (copiar en draw.io con Ctrl+E) + +``` +# Banner / cabecera +rounded=0;whiteSpace=wrap;html=1;fillColor=#4DA1F5;strokeColor=none;shadow=1;fontColor=#ffffff;fontSize=15;fontStyle=1;align=center; + +# Tarjeta neutra +rounded=1;arcSize=6;absoluteArcSize=1;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#dddddd;shadow=1;strokeWidth=1;fontColor=#4B5259;fontSize=12; + +# Servicio / agente LLM (azul) +rounded=1;arcSize=6;absoluteArcSize=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;shadow=1;fontColor=#4B5259;fontSize=12; + +# Núcleo determinista (verde) +rounded=1;arcSize=6;absoluteArcSize=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;shadow=1;fontColor=#4B5259;fontSize=12; + +# Paso crítico / advertencia (rojo) +rounded=1;arcSize=6;absoluteArcSize=1;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;shadow=1;fontColor=#4B5259;fontSize=12;fontStyle=1; + +# Salida / revisión humana (amarillo) +rounded=1;arcSize=6;absoluteArcSize=1;whiteSpace=wrap;html=1;fillColor=#fff2cc;strokeColor=#d6b656;shadow=1;fontColor=#4B5259;fontSize=12; + +# Gate condicional (rombo naranja) +rhombus;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;shadow=1;fontColor=#4B5259;fontSize=11; + +# Contenedor / zona (borde azul punteado, fondo transparente) — esquina algo más marcada (arcSize 2) +rounded=1;arcSize=6;absoluteArcSize=2;whiteSpace=wrap;html=1;fillColor=none;strokeColor=#4284F3;dashed=1;verticalAlign=top;align=left;fontColor=#4284F3;fontSize=13;fontStyle=1;spacingLeft=12;spacingTop=6; + +# Actor / usuario · Almacén-BD (cilindro, etiqueta DEBAJO, estándar ~120x50) · Documento +shape=mxgraph.ios7.icons.user;html=1;strokeColor=#0080F0;strokeWidth=2;verticalLabelPosition=bottom;verticalAlign=top;align=center;fontColor=#4B5259; +shape=cylinder3;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#999999;verticalLabelPosition=bottom;verticalAlign=top;align=center;fontColor=#4B5259;fontSize=10; +shape=document;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#999999;boundedLbl=1;fontColor=#4B5259;fontSize=11; +``` + +## Conexiones (la firma de la casa) + +Edges **ortogonales y animados** con color **theme-aware** (`light-dark(claro,oscuro)`). +**Obligatorio: TODOS los conectores llevan `flowAnimation=1`** (animación de flujo) — es la firma de +la casa; también las variantes punteadas. El kit ya lo garantiza en `EDGE`. + +``` +# Flujo principal (animado) +edgeStyle=orthogonalEdgeStyle;flowAnimation=1;rounded=0;html=1;strokeColor=light-dark(#4284F3,#6671E3);strokeWidth=2; + +# Datos / RAG (verde) · Excepción/humano (rojo punteado) · Auxiliar (gris punteado) — TODOS animados +edgeStyle=orthogonalEdgeStyle;flowAnimation=1;rounded=0;html=1;strokeColor=#66CC00;strokeWidth=2; +edgeStyle=orthogonalEdgeStyle;flowAnimation=1;rounded=0;html=1;dashed=1;strokeColor=#b85450;strokeWidth=2; +edgeStyle=orthogonalEdgeStyle;flowAnimation=1;rounded=0;html=1;dashed=1;strokeColor=#9E9E9E;strokeWidth=1; +``` + +## Leyenda descriptiva (obligatoria) + +Todo diagrama entregado incluye una leyenda que traduce el color a significado. Convenio de la casa +para las **flechas** (mismo `EDGE` del kit): + +| Color | `EDGE` | Significado | +|---|---|---| +| Verde `#66CC00` | `data` | Flujo de datos (lo que se transforma y avanza entre pasos) | +| Azul `light-dark(#4284F3,#6671E3)` | `flow` | Orquestación / control (quién invoca o ejecuta a quién) | +| Gris punteada `#9E9E9E` | `aux` | Consumo de un servicio o recurso compartido (lectura/escritura) | +| Rojo punteado `#b85450` | `warn` | Excepción / intervención humana | + +Y para las **cajas**, describe solo los arquetipos que uses (verde = determinista, azul = servicio/LLM, +rojo = orquestador/crítico, blanco = almacén de apoyo, amarillo = salida). Añade las convenciones +extra que apliquen (iconos sueltos = fuentes externas; flechas animadas = sentido del flujo). + +Dibújala en una franja al pie (dentro del `pageHeight`) con **edges de muestra reales** — mismo `style` +que en el diagrama, con `sourcePoint`/`targetPoint` en la geometría en lugar de `source`/`target` — y +**chips** de color usando el `fillColor`/`strokeColor` de cada arquetipo: + +``` +# Línea de muestra (sin nodos): reutiliza el EDGE real, solo cambia los puntos + + + + + +# Chip de color de caja (usa el fillColor/strokeColor del arquetipo) +rounded=1;arcSize=6;absoluteArcSize=1;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366; +``` + +## Claridad y anti-traslape (lo más importante para que se entienda) + +Reglas de diseño: + +- **Separación mínima**: deja ≥ 40 px entre cajas de nodos distintos. Recuerda que la etiqueta de un + icono se dibuja **debajo** de él (con `verticalLabelPosition=bottom`) y ocupa ~15 px por línea: no + pongas otro nodo pegado abajo o la etiqueta lo pisará. +- **Fijar los anclajes** con `exit=(fx,fy)` y `entry=(fx,fy)` (fracciones 0–1). Es lo que separa un + diagrama legible de un enredo de flechas. +- **Fan-out**: salir del origen a distinta Y por rama (`exit=(1,0.3)`, `(1,0.5)`, `(1,0.7)`); entrar + por el mismo lado del destino → peine paralelo. **Gather**: entrar al destino a distinta Y + (`entry=(0,0.2)`, `(0,0.5)`, `(0,0.8)`). +- **Almacenes de apoyo** pegados **junto** al nodo que los consume (edge corto), no al otro extremo. +- **Flechas que cruzarían un nodo** → usa `points=[(x,y),...]` en `edge()` para sacar la ruta por + encima/alrededor (un carril libre), o reubica los nodos. No dejes una flecha atravesando un icono. +- **Etiquetas de flecha** (`202`, `Post`): ponlas donde el tramo esté libre; si caen sobre un nodo, + mueve el nodo o añade un waypoint para desplazar el punto medio. +- **Contenedor/zona**: dibújalo primero (queda detrás), `fillColor=none`; el título va en una esquina + (`align=left`), no centrado sobre el paso de las flechas. + +### Verificar con el linter (obligatorio antes de entregar) + +`drawio_kit.write()` corre `check_layout` y avisa. Para el detalle: + +```bash +python scripts/check_layout.py .drawio +``` + +Reporta **nodos-superpuestos**, **etiqueta-sobre-nodo**, **flecha-cruza-nodo** y +**etiqueta-flecha-sobre-nodo**. Itera reubicando nodos / añadiendo waypoints hasta que no queden +traslapes **duros** (nodos-superpuestos, flecha-cruza-nodo). Es heurístico (el ruteo real de draw.io +difiere), así que confirma también visualmente en draw.io. + +## Etiquetas: pasar texto crudo (evitar doble escape) + +`drawio_kit` ya escapa el XML por ti. En `label` pasa **texto crudo**, no entidades: + +- salto de línea → `"\n"` (no `" "`); +- `<`, `>`, `&` literales → escríbelos tal cual (`" MicroComponent"`, `"Search & Chat"`), no + `"<"`/`"&"`. + +Si pre-codificas entidades, el kit las vuelve a escapar (`&lt;`, `&#xa;`) y draw.io muestra +el texto literal en vez de interpretarlo. + +## Higiene de git + +- El pre-commit típico **bloquea archivos > 500 KB**. Con iconos SVG embebidos un diagrama pesa + decenas de KB; no embeber PNG pesados ni cientos de imágenes. +- Tratar las arquitecturas como documentación interna; sin PII ni datos sensibles en etiquetas/notas. +- Revisar y editar el `.drawio` en VS Code con la extensión *Draw.io Integration* (`hediet.vscode-drawio`). diff --git a/.claude/skills/arquitectura-drawio/references/iconos.md b/.claude/skills/arquitectura-drawio/references/iconos.md new file mode 100644 index 0000000..3cc0f45 --- /dev/null +++ b/.claude/skills/arquitectura-drawio/references/iconos.md @@ -0,0 +1,80 @@ +# Iconos de tecnología — regla y orden de preferencia + +Cada proyecto usa tecnologías distintas: GCP, AWS, Azure, LangChain, Langfuse, Google ADK, +React, Node, Python, TypeScript, Docker, Kubernetes, PostgreSQL, FastAPI, etc. La skill es +**agnóstica del stack**. + +> **Regla:** para cada componente usa SIEMPRE el **logo oficial de la tecnología** que representa; +> si no existe, cae al **glifo genérico**. Nunca inventes un icono ni dejes una caja vacía. + +## 1ª opción — logo oficial de la tecnología (recomendado) + +Punto de entrada único en `scripts/glyph.py`, sirve para cualquier tecnología: + +```python +from glyph import logo +p.node("LangChain service", x, y, w, h, istyle(logo("langchain"))) +p.node("Frontend React", x, y, w, h, istyle(logo("react", "#61DAFB"))) +p.node("PostgreSQL", x, y, w, h, istyle(logo("postgresql"))) +p.node("Vertex AI", x, y, w, h, istyle(logo("vertex_ai"))) # cae a producto GCP +``` + +`logo(name, color)` prueba, en orden: **marca** (simple-icons, ~3000 logos oficiales) y luego +**producto Google Cloud** (`gcp_icon`). Si ninguno tiene el logo, lanza un error que te guía al +glifo genérico (no inventa iconos). `color` es opcional (respeta el color de marca). + +- Slugs de marca típicos: `langchain`, `react`, `nodejs`, `typescript`, `python`, `docker`, + `kubernetes`, `fastapi`, `postgresql`, `redis`, `awslambda`, `amazonsqs`, `microsoftazure`, + `apache`, `openai`, `huggingface`. Catálogo completo: . +- Productos GCP: `vertex_ai`, `bigquery`, `cloud_run`, `cloud_storage`, `cloud_vision_api`, + `data_loss_prevention_api`, ... (`python scripts/gcp_icon.py --list`). + +CLI para explorar/obtener un data URI: + +```bash +python scripts/glyph.py logo langchain # logo oficial (marca o producto GCP) +python scripts/glyph.py logo vertex_ai +``` + +Detalle técnico: el data URI es **URL-encoded** (no base64) y `image=` va **de último** en el +`style`, **sin `;` final** (así el `;base64` no rompe el parser de estilos de draw.io). `drawio_kit` +ya lo maneja al pasar `icon=`. El mismo patrón aplica a cualquier proveedor: bajar el SVG oficial, +URL-encode, `shape=image`. + +## Fallback — glifo genérico (cuando NO hay logo oficial) + +Muchas tecnologías nuevas o de nicho no tienen logo en las fuentes (p. ej. **Google ADK**, +**Langfuse**), igual que los componentes **custom** (un microservicio propio, una cola interna). +En esos casos NO dejes la caja vacía: rellena con un glifo genérico apropiado de Material Symbols +(Apache-2.0), coloreado según su categoría: + +```python +from glyph import material +p.node("Orquestador ADK", x, y, w, h, istyle(material("smart_toy", "#1a73e8"))) +p.node("Langfuse (observabilidad)", x, y, w, h, istyle(material("visibility"))) +p.node("MicroComponent", x, y, w, h, istyle(material("code", "#cc0000"))) +``` + +Glifos sugeridos por tipo (`python scripts/glyph.py list-suggested`): `code` (código/microservicio), +`deployed_code` (paquete), `smart_toy` (agente/ADK), `neurology` (modelo/IA), `api` (endpoint), +`database` (almacén), `forum` (cola), `bolt` (evento), `hub` (integración/orquestador), `settings` +(proceso), `dns` (servidor), `language`/`globe()` (internet), `lock` (auth), `person` (usuario). + +## 2ª opción — librería nativa de draw.io + +Formas vectoriales de la librería del proveedor en draw.io (p. ej. `mxgraph.gcp2.hexIcon` con +`prIcon=`; *More Shapes → …*). Es lo **más liviano** (cero KB embebidos) y diffeable, +pero el look difiere del arte oficial y hay riesgo de icono en blanco si el nombre/`prIcon` no +existe en esa versión. Usar cuando el peso importe más que la fidelidad, o sin acceso a la fuente. + +## 3ª opción — imágenes provistas por el usuario (informar SIEMPRE) + +Si el usuario aporta sus propios iconos (PNG/SVG: capturas, assets internos, logos de terceros), se +embeben como `shape=image`, pero **siempre hay que informarle** de: + +- el **peso extra** en el `.drawio` (y el límite de 500 KB del pre-commit típico); +- que son **imágenes pegadas** (raster/opacas), no vectores de librería ni logos oficiales; +- que **no deben contener PII ni datos internos sensibles** — una captura puede filtrar información; +- posibles **restricciones de licencia** de logos de terceros. + +Es el último recurso, cuando la tecnología no está en las fuentes oficiales ni aplica un glifo. diff --git a/.claude/skills/arquitectura-drawio/scripts/.gitignore b/.claude/skills/arquitectura-drawio/scripts/.gitignore new file mode 100644 index 0000000..2a62e60 --- /dev/null +++ b/.claude/skills/arquitectura-drawio/scripts/.gitignore @@ -0,0 +1,3 @@ +__pycache__/ +*.pyc +*.drawio diff --git a/.claude/skills/arquitectura-drawio/scripts/check_layout.py b/.claude/skills/arquitectura-drawio/scripts/check_layout.py new file mode 100644 index 0000000..aabdaf9 --- /dev/null +++ b/.claude/skills/arquitectura-drawio/scripts/check_layout.py @@ -0,0 +1,213 @@ +"""check_layout — linter de legibilidad para diagramas .drawio. + +Detecta lo que hace ilegible un diagrama y que draw.io no valida por ti: + 1. NODOS superpuestos (cajas/iconos que se pisan). + 2. ETIQUETAS que invaden otro nodo (incluye la etiqueta al pie de un icono + que cae sobre el nodo de abajo). + 3. FLECHAS que cruzan un nodo que no es su origen/destino (heurístico: ruteo + ortogonal aproximado según exit/entry). + 4. ETIQUETAS DE FLECHA que caen dentro de un nodo ajeno. + +Es una heurística (el ruteo real de draw.io difiere), pero atrapa los problemas +gruesos de traslape. Los contenedores/zonas se excluyen (contienen a otros por diseño). + +CLI: + python check_layout.py archivo.drawio # reporta y sale !=0 si hay traslapes duros + +Como librería: + from check_layout import check + issues = check("archivo.drawio") # -> lista de dicts {tipo, pagina, detalle} +""" + +from __future__ import annotations + +import re +import sys +from xml.etree import ElementTree as ET + +MIN_AREA = 90 # px² de intersección para contar un traslape de cajas +LABEL_LH = 15 # alto estimado por línea de etiqueta al pie de un icono + + +def _sk(style: str) -> dict: + d = {} + for t in (style or "").split(";"): + if "=" in t: + k, v = t.split("=", 1) + d[k] = v + elif t: + d[t] = "" + return d + + +def _is_container(d: dict) -> bool: + return "dashed" in d and d.get("fillColor") == "none" and d.get("shape", "") == "" + + +def _label(cell) -> str: + return re.sub(r"<[^>]+>", "", cell.get("value") or "") + + +def _footprint(cell, d, geo): + """(x,y,w,h) del nodo, ampliado con la banda de etiqueta al pie si es un icono + con verticalLabelPosition=bottom (esa etiqueta ocupa espacio bajo el icono).""" + x, y, w, h = geo + label = _label(cell) + if label and d.get("verticalLabelPosition") == "bottom": + lines = label.count("\n") + (cell.get("value") or "").count(" ") + 1 + lw = max(w * 1.7, 90) + lh = LABEL_LH * lines + nx = x + w / 2 - lw / 2 + return (min(x, nx), y, max(w, lw), h + lh) + return (x, y, w, h) + + +def _inter(a, b) -> float: + ax, ay, aw, ah = a + bx, by, bw, bh = b + ix = max(0, min(ax + aw, bx + bw) - max(ax, bx)) + iy = max(0, min(ay + ah, by + bh) - max(ay, by)) + return ix * iy + + +def _seg_hits_rect(p, q, rect, margin=2.0) -> bool: + """¿El segmento axis-aligned p-q entra en rect (con margen para no contar roces)?""" + rx, ry, rw, rh = rect[0] + margin, rect[1] + margin, rect[2] - 2 * margin, rect[3] - 2 * margin + if rw <= 0 or rh <= 0: + return False + x1, y1 = p + x2, y2 = q + if abs(y1 - y2) < 0.5: # horizontal + lo, hi = sorted((x1, x2)) + return ry <= y1 <= ry + rh and lo < rx + rw and hi > rx + if abs(x1 - x2) < 0.5: # vertical + lo, hi = sorted((y1, y2)) + return rx <= x1 <= rx + rw and lo < ry + rh and hi > ry + return False + + +def _route(p0, p1, ex): + mx, my = (p0[0] + p1[0]) / 2, (p0[1] + p1[1]) / 2 + if ex in (0.0, 1.0): # sale horizontal + return [p0, (mx, p0[1]), (mx, p1[1]), p1] + return [p0, (p0[0], my), (p1[0], my), p1] + + +def check(path: str) -> list[dict]: + t = ET.parse(path) + issues: list[dict] = [] + for diag in t.iter("diagram"): + page = diag.get("name", "?") + geo, sty, val, kind = {}, {}, {}, {} + for c in diag.iter("mxCell"): + g = c.find("mxGeometry") + if c.get("vertex") == "1" and g is not None and g.get("width"): + style = c.get("style", "") + # icono-overlay decorativo (kit node(icon=): imagen sin etiqueta sobre + # una tarjeta) -> no es un nodo propio, no cuenta para traslapes. + if "shape=image" in style and not (c.get("value") or "").strip(): + continue + gid = c.get("id") + geo[gid] = tuple(float(g.get(k, 0)) for k in ("x", "y", "width", "height")) + d = _sk(style) + sty[gid] = d + val[gid] = c + kind[gid] = ( + "container" + if _is_container(d) + else ("text" if style.startswith("text;") else "node") + ) + + # 1 y 2: traslape de nodos / etiquetas (excluye contenedores) + solid = [i for i in geo if kind[i] != "container"] + fp = {i: _footprint(val[i], sty[i], geo[i]) for i in solid} + for i, a in enumerate(solid): + for b in solid[i + 1 :]: + area = _inter(fp[a], fp[b]) + if area > MIN_AREA: + ta, tb = kind[a], kind[b] + typ = "etiqueta-sobre-nodo" if "text" in (ta, tb) else "nodos-superpuestos" + issues.append( + { + "tipo": typ, + "pagina": page, + "detalle": f"«{_label(val[a])[:22]}» ∩ «{_label(val[b])[:22]}» " + f"(~{int(area)} px²)", + } + ) + + # 3 y 4: flechas que cruzan nodos / etiquetas de flecha dentro de un nodo + def point(gid, fx, fy, geo=geo): # geo=geo: enlaza el geo de ESTA página (B023) + x, y, w, h = geo[gid] + return x + fx * w, y + fy * h + + for c in diag.iter("mxCell"): + if c.get("edge") != "1": + continue + s, tg = c.get("source"), c.get("target") + if s not in geo or tg not in geo: + continue + d = _sk(c.get("style", "")) + ex = float(d.get("exitX", 0.5)) + p0 = point(s, ex, float(d.get("exitY", 0.5))) + p1 = point(tg, float(d.get("entryX", 0.5)), float(d.get("entryY", 0.5))) + wps = [ + (float(mp.get("x")), float(mp.get("y"))) + for mp in c.findall("./mxGeometry/Array/mxPoint") + ] + pts = [p0, *wps, p1] if wps else _route(p0, p1, ex) + for gid in geo: + if gid in (s, tg) or kind[gid] == "container": + continue + if any(_seg_hits_rect(pts[k], pts[k + 1], geo[gid]) for k in range(len(pts) - 1)): + issues.append( + { + "tipo": "flecha-cruza-nodo", + "pagina": page, + "detalle": f"edge «{_label(val[s])[:16]}»→«{_label(val[tg])[:16]}» " + f"cruza «{_label(val[gid])[:20]}»", + } + ) + if c.get("value"): + mid = ((p0[0] + p1[0]) / 2, (p0[1] + p1[1]) / 2) + for gid in geo: + if gid in (s, tg) or kind[gid] == "container": + continue + gx, gy, gw, gh = geo[gid] + if gx <= mid[0] <= gx + gw and gy <= mid[1] <= gy + gh: + issues.append( + { + "tipo": "etiqueta-flecha-sobre-nodo", + "pagina": page, + "detalle": f"etiqueta «{_label(c)[:16]}» " + f"cae en «{_label(val[gid])[:20]}»", + } + ) + return issues + + +def summarize(path: str) -> dict: + issues = check(path) + counts: dict[str, int] = {} + for it in issues: + counts[it["tipo"]] = counts.get(it["tipo"], 0) + 1 + return {"total": len(issues), "por_tipo": counts, "issues": issues} + + +if __name__ == "__main__": + if len(sys.argv) < 2: + print("uso: python check_layout.py archivo.drawio") + sys.exit(2) + res = summarize(sys.argv[1]) + if not res["total"]: + print("✓ check_layout: sin traslapes detectados") + sys.exit(0) + print(f"⚠ check_layout: {res['total']} posibles problemas de legibilidad") + for tipo, n in sorted(res["por_tipo"].items(), key=lambda kv: -kv[1]): + print(f" {n:3} {tipo}") + print("---") + for it in res["issues"][:40]: + print(f" [{it['tipo']}] {it['detalle']}") + # traslapes duros -> exit !=0 (útil como gate en iteración) + hard = {"nodos-superpuestos", "flecha-cruza-nodo"} + sys.exit(1 if any(it["tipo"] in hard for it in res["issues"]) else 0) diff --git a/.claude/skills/arquitectura-drawio/scripts/drawio_kit.py b/.claude/skills/arquitectura-drawio/scripts/drawio_kit.py new file mode 100644 index 0000000..e875017 --- /dev/null +++ b/.claude/skills/arquitectura-drawio/scripts/drawio_kit.py @@ -0,0 +1,226 @@ +"""drawio_kit — motor mínimo para generar diagramas .drawio en el estilo de la casa. + +Portable y sin dependencias externas (solo stdlib). Emite XML mxGraph válido con: + - IDs únicos por página (prefijo automático), + - escapado XML y saltos de línea correctos en etiquetas, + - tokens de estilo de la organización (paleta, cajas, edges), + - soporte de iconos embebidos (data URI) con el patrón "caja + icono a la izquierda". + +Uso típico: + + from drawio_kit import Diagram, STYLE, EDGE + from glyph import logo # logo oficial de CUALQUIER tecnología + + d = Diagram() + p = d.page("Arquitectura") + a = p.node("Frontend", 40, 40, 170, 60, STYLE["card"], icon=logo("react")) + b = p.node("Servicio", 260, 40, 170, 60, STYLE["llm"], icon=logo("langchain")) + p.edge(a, b, EDGE["flow"], "procesa", exit=(1, 0.5), entry=(0, 0.5)) + d.write("arquitectura.drawio") + +Convenciones de estilo: ver references/estilo.md. Iconos: ver references/iconos.md. +""" + +from __future__ import annotations + +import collections +from xml.etree import ElementTree as ET +from xml.sax.saxutils import escape + +FONT = "#4B5259" # texto primario de la casa +BLUE = "#4284F3" # azul de marca (bordes / edges) + +# Radio de esquina fijo (px) para las cajas redondeadas. Con `rounded=1` a secas +# draw.io usa una curva grande (proporcional al tamaño) que puede tapar el texto +# alineado a la izquierda o sacarlo por las esquinas; `absoluteArcSize=1` fija el +# radio en píxeles y `arcSize` lo controla. Ver references/estilo.md. +# ARC → cajas de componente: radio muy sutil (1 px). +# ARC_ZONE → contenedores/zonas azules punteados: esquina algo más marcada (2 px) +# para que se note en las cajas grandes. +ARC = "arcSize=6;absoluteArcSize=1;" +ARC_ZONE = "arcSize=6;absoluteArcSize=2;" + +# --- Arquetipos de nodo (copiar-pegar tal cual en draw.io con Ctrl+E) --------- +STYLE = { + "banner": "rounded=0;whiteSpace=wrap;html=1;fillColor=#4DA1F5;strokeColor=none;" + "shadow=1;fontColor=#ffffff;fontSize=15;fontStyle=1;align=center;verticalAlign=middle;", + "card": f"rounded=1;{ARC}whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#dddddd;" + f"shadow=1;strokeWidth=1;fontColor={FONT};fontSize=12;", + "llm": f"rounded=1;{ARC}whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" + f"shadow=1;fontColor={FONT};fontSize=12;", + "det": f"rounded=1;{ARC}whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" + f"shadow=1;fontColor={FONT};fontSize=12;", + "warn": f"rounded=1;{ARC}whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" + f"shadow=1;fontColor={FONT};fontSize=12;fontStyle=1;", + "out": f"rounded=1;{ARC}whiteSpace=wrap;html=1;fillColor=#fff2cc;strokeColor=#d6b656;" + f"shadow=1;fontColor={FONT};fontSize=12;", + "gate": "rhombus;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" + f"shadow=1;fontColor={FONT};fontSize=11;", + # Almacén/BD: cilindro compacto con la ETIQUETA DEBAJO (no dentro; así la elipse + # superior no tacha el texto). Tamaño estándar ~120x50. No usar boundedLbl. + "store": "shape=cylinder3;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#999999;" + "verticalLabelPosition=bottom;verticalAlign=top;labelPosition=center;align=center;" + f"fontColor={FONT};fontSize=10;", + "doc": "shape=document;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#999999;" + f"boundedLbl=1;fontColor={FONT};fontSize=11;", + "user": "shape=mxgraph.ios7.icons.user;html=1;strokeColor=#0080F0;strokeWidth=2;" + "verticalLabelPosition=bottom;verticalAlign=top;labelPosition=center;align=center;" + f"fontColor={FONT};fontSize=11;", + "zone": f"""rounded=1;{ARC_ZONE}whiteSpace=wrap;html=1;fillColor=none; + strokeColor={BLUE};dashed=1;""" + f"verticalAlign=top;align=left;fontColor={BLUE};fontSize=13;fontStyle=1;" + "spacingLeft=12;spacingTop=6;", + "note": "text;whiteSpace=wrap;html=1;fontColor=#9E9E9E;fontSize=11;align=left;", + "caption": f"text;whiteSpace=wrap;html=1;fontColor={FONT};" + "fontSize=11;align=center;fontStyle=1;", + # Icono suelto embebido (image= debe quedar de ÚLTIMO y sin ';' final) + "_icon": "shape=image;html=1;imageAspect=0;aspect=fixed;image=%s", +} + +# --- Edges: la firma de la casa es orthogonal + flowAnimation + theme-aware ---- +# TODOS los conectores van ANIMADOS (flowAnimation=1). Es un rasgo obligatorio del +# estilo de la casa; ver references/estilo.md. Las variantes punteadas siguen animadas. +EDGE = { + "flow": "edgeStyle=orthogonalEdgeStyle;flowAnimation=1;rounded=0;html=1;jettySize=auto;" + "strokeColor=light-dark(#4284F3,#6671E3);strokeWidth=2;", + "data": "edgeStyle=orthogonalEdgeStyle;flowAnimation=1;rounded=0;html=1;jettySize=auto;" + "strokeColor=#66CC00;strokeWidth=2;", + "warn": "edgeStyle=orthogonalEdgeStyle;flowAnimation=1;rounded=0;html=1;" + "jettySize=auto;dashed=1;strokeColor=#b85450;strokeWidth=2;", + "aux": "edgeStyle=orthogonalEdgeStyle;flowAnimation=1;rounded=0;html=1;" + "jettySize=auto;dashed=1;strokeColor=#9E9E9E;strokeWidth=1;", +} + + +def _esc(text: str) -> str: + """Escapa para atributo XML y convierte '\\n' en salto de línea de draw.io.""" + return escape(text or "").replace("\n", " ") + + +class Page: + """Una página () del archivo. No instanciar directo: usar Diagram.page().""" + + def __init__(self, name: str, prefix: str) -> None: + self.name = name + self.prefix = prefix + self.cells: list[str] = [] + self._n = 0 + + def _id(self, hint: str) -> str: + self._n += 1 + return f"{self.prefix}-{hint}-{self._n}" + + def node(self, label, x, y, w, h, style, ident=None, icon=None) -> str: + """Crea un nodo y devuelve su id. Si `icon` (data URI) se pasa, incrusta + un icono 22x22 sobre el borde izquierdo y alinea la etiqueta a su derecha.""" + cid = f"{self.prefix}-{ident}" if ident else self._id("n") + st = style + ("align=left;spacingLeft=32;" if icon else "") + self.cells.append( + f'' + ) + if icon: + iid = self._id("ico") + iy = y + (h - 22) / 2 + self.cells.append( + f'' + ) + return cid + + def edge( + self, src, dst, style=EDGE["flow"], label="", exit=None, entry=None, points=None + ) -> str: + """Conecta dos nodos. `exit`/`entry` son (fx, fy) en [0,1] para fijar el + punto de anclaje y evitar traslapes (fan-out/gather en peine). `points` es + una lista de (x, y) absolutos: waypoints por los que se fuerza el ruteo + (úsalo para sacar una flecha por encima/alrededor y no cruzar nodos).""" + st = style + if exit: + st += f"exitX={exit[0]};exitY={exit[1]};exitDx=0;exitDy=0;" + if entry: + st += f"entryX={entry[0]};entryY={entry[1]};entryDx=0;entryDy=0;" + cid = self._id("e") + if points: + pts = "".join(f'' for x, y in points) + geo = ( + f'' + f'{pts}' + ) + else: + geo = '' + self.cells.append( + f'{geo}' + ) + return cid + + def _xml(self) -> str: + body = "".join(self.cells) + return ( + f'' + f'' + f'{body}' + f"" + ) + + +class Diagram: + """Archivo .drawio con una o más páginas.""" + + def __init__(self) -> None: + self.pages: list[Page] = [] + + def page(self, name: str) -> Page: + p = Page(name, f"p{len(self.pages) + 1}") + self.pages.append(p) + return p + + def xml(self) -> str: + inner = "".join(p._xml() for p in self.pages) + return ( + f'{inner}' + ) + + def write(self, path: str) -> None: + with open(path, "w", encoding="utf-8") as fh: + fh.write(self.xml()) + self.validate(path) + size = len(self.xml()) + print(f"escrito: {path} ({size} bytes, {size // 1024} KB)") + if size > 500_000: + print(" ⚠️ >500 KB: el pre-commit típico lo bloquea. Reduce iconos embebidos.") + try: # chequeo de legibilidad (no fatal) + from check_layout import summarize + + rep = summarize(path) + if rep["total"]: + print(f" ⚠️ check_layout: {rep['total']} posibles traslapes -> {rep['por_tipo']}") + print(f" detalle: python scripts/check_layout.py {path}") + else: + print(" ✓ check_layout: sin traslapes detectados") + except Exception as exc: + print(f" (check_layout no ejecutado: {exc})") + + @staticmethod + def validate(path: str) -> None: + """Falla ruidosamente si el XML es inválido, hay IDs duplicados o edges rotos.""" + t = ET.parse(path) # lanza si no es XML bien formado + ids = [c.get("id") for c in t.iter("mxCell") if c.get("id") not in ("0", "1")] + dups = [k for k, v in collections.Counter(ids).items() if v > 1] + if dups: + raise ValueError(f"IDs duplicados: {dups}") + allids = {c.get("id") for c in t.iter("mxCell")} + broken = [ + (c.get("id"), k, c.get(k)) + for c in t.iter("mxCell") + if c.get("edge") == "1" + for k in ("source", "target") + if c.get(k) and c.get(k) not in allids + ] + if broken: + raise ValueError(f"edges con extremos inexistentes: {broken}") diff --git a/.claude/skills/arquitectura-drawio/scripts/ejemplo.py b/.claude/skills/arquitectura-drawio/scripts/ejemplo.py new file mode 100644 index 0000000..6adddfb --- /dev/null +++ b/.claude/skills/arquitectura-drawio/scripts/ejemplo.py @@ -0,0 +1,81 @@ +"""Plantilla de uso de drawio_kit — AGNÓSTICA de tecnología. + +Muestra el flujo de iconos recomendado: intentar el LOGO OFICIAL de cada tecnología +con `logo(...)` y, si no existe, caer a un GLIFO GENÉRICO con `material(...)`. +Copie este patrón para cualquier stack (JS, Python, LangChain, ADK, cloud, ...). + + python ejemplo.py # escribe ejemplo.drawio, lo valida y chequea el layout +""" + +from drawio_kit import EDGE, STYLE, Diagram + +try: + from glyph import logo, material + + def icon(name, fallback_glyph, color=None): + """Logo oficial de `name`; si no existe, el glifo genérico `fallback_glyph`.""" + try: + return logo(name, color) + except Exception: + return material(fallback_glyph, color or "#5f6368") +except Exception as exc: # sin red -> degradar sin iconos (no romper) + print(f"(aviso: sin iconos, sigo sin ellos: {exc})") + + def icon(name, fallback_glyph, color=None): + return None + + +def istyle(uri, fs=11): + return ( + ( + f"shape=image;html=1;imageAspect=0;aspect=fixed;verticalLabelPosition=bottom;" + f"verticalAlign=top;labelPosition=center;align=center;fontColor=#4B5259;" + f"fontSize={fs};image={uri}" + ) + if uri + else STYLE["card"] + ) + + +d = Diagram() +p = d.page("Ejemplo") +p.node( + "Arquitectura de ejemplo — plantilla drawio_kit (agnóstica de tecnología)", + 40, + 20, + 900, + 40, + STYLE["banner"], +) +p.node("Aplicación", 40, 90, 920, 330, STYLE["zone"], ident="app") + +# Cada nodo usa el logo oficial de su tecnología; los que no lo tienen, un glifo genérico. +front = p.node("Frontend", 70, 150, 150, 60, STYLE["card"], icon=icon("react", "code", "#61DAFB")) +api = p.node("API", 280, 150, 150, 60, STYLE["card"], icon=icon("fastapi", "api", "#009688")) +agent = p.node( + "Agente LangChain", 490, 150, 160, 60, STYLE["llm"], icon=icon("langchain", "smart_toy") +) +# Langfuse NO está en las fuentes de logos -> cae al glifo genérico "visibility" +obs = p.node( + "Langfuse\n(observabilidad)", + 720, + 152, + 160, + 56, + STYLE["card"], + icon=icon("langfuse", "visibility", "#1a73e8"), +) +# Componente custom sin logo -> glifo genérico "code" +worker = p.node( + "Worker propio", 280, 270, 150, 56, STYLE["card"], icon=icon("__custom__", "code", "#cc0000") +) +# Base de datos: cilindro estándar (etiqueta debajo) +db = p.node("PostgreSQL", 520, 280, 120, 50, STYLE["store"]) + +p.edge(front, api, EDGE["flow"], exit=(1, 0.5), entry=(0, 0.5)) +p.edge(api, agent, EDGE["flow"], exit=(1, 0.5), entry=(0, 0.5)) +p.edge(agent, obs, EDGE["flow"], exit=(1, 0.5), entry=(0, 0.5)) +p.edge(agent, db, EDGE["data"], exit=(0.5, 1), entry=(0.5, 0)) +p.edge(api, worker, EDGE["flow"], exit=(0.5, 1), entry=(0.5, 0)) + +d.write("ejemplo.drawio") diff --git a/.claude/skills/arquitectura-drawio/scripts/gcp_icon.py b/.claude/skills/arquitectura-drawio/scripts/gcp_icon.py new file mode 100644 index 0000000..400b32e --- /dev/null +++ b/.claude/skills/arquitectura-drawio/scripts/gcp_icon.py @@ -0,0 +1,98 @@ +"""gcp_icon — resuelve iconos OFICIALES de Google Cloud a data URI para draw.io. + +Descarga los sets públicos de Google (sin login), extrae el SVG del producto, +lo convierte a data URI URL-encoded y lo devuelve. Sin dependencias externas +(solo stdlib). Cachea los ZIP en el temp del sistema para no re-descargar. + +CLI: + python gcp_icon.py vertex_ai # imprime el data URI + python gcp_icon.py --list # lista los nombres disponibles + python gcp_icon.py cloud vision # match difuso por palabras + +Como librería: + from gcp_icon import data_uri + uri = data_uri("cloud_dlp") # -> "data:image/svg+xml,%3Csvg..." + +Fuente: https://cloud.google.com/icons (descargas públicas de Google). +""" + +from __future__ import annotations + +import os +import re +import sys +import tempfile +import urllib.parse +import urllib.request +import zipfile + +# ZIP oficiales. 'core' = estilo actual (productos insignia); 'legacy' = set +# completo (216 SVG, incluye Document AI, Vision, DLP, etc.). +ZIPS = { + "core": "https://services.google.com/fh/files/misc/core-products-icons.zip", + "legacy": "https://services.google.com/fh/files/misc/google-cloud-legacy-icons.zip", +} +CACHE = os.path.join(tempfile.gettempdir(), "gcp_drawio_icons") + + +def _norm(s: str) -> str: + """Normaliza un nombre para comparar: minúsculas, solo alfanumérico.""" + return re.sub(r"[^a-z0-9]", "", s.lower()) + + +def _ensure() -> dict[str, bytes]: + """Descarga (si hace falta) y devuelve {nombre_svg_sin_ext: bytes_svg}. + Prefiere el SVG de 'core' sobre 'legacy' cuando el mismo producto está en ambos.""" + os.makedirs(CACHE, exist_ok=True) + svgs: dict[str, bytes] = {} + for key in ("legacy", "core"): # core al final => pisa a legacy (estilo actual gana) + local = os.path.join(CACHE, f"{key}.zip") + if not os.path.exists(local): + req = urllib.request.Request(ZIPS[key], headers={"User-Agent": "drawio-kit"}) + with urllib.request.urlopen(req, timeout=120) as r: + data = r.read() + with open(local, "wb") as fh: + fh.write(data) + with zipfile.ZipFile(local) as z: + for info in z.infolist(): + if info.filename.lower().endswith(".svg"): + base = os.path.splitext(os.path.basename(info.filename))[0] + # normaliza variantes: "CloudRun-512-color-rgb" -> "cloudrun" + clean = re.sub(r"[-_](512|color|rgb|blue|dark|light)\b", "", base, flags=re.I) + svgs[_norm(clean)] = z.read(info.filename) + return svgs + + +def _find(name: str, svgs: dict[str, bytes]) -> bytes: + key = _norm(name) + if key in svgs: + return svgs[key] + # match difuso: todas las palabras del query aparecen en el nombre del svg + words = [w for w in re.split(r"[^a-z0-9]+", name.lower()) if w] + cands = [k for k in svgs if all(w in k for w in words)] + if len(cands) == 1: + return svgs[cands[0]] + if not cands: + raise KeyError(f"icono no encontrado: {name!r}. Prueba `--list`.") + raise KeyError(f"{name!r} es ambiguo: {sorted(cands)[:10]}") + + +def data_uri(name: str) -> str: + """Devuelve el data URI URL-encoded del SVG del producto (embebible en draw.io).""" + svg = _find(name, _ensure()).decode("utf-8") + return "data:image/svg+xml," + urllib.parse.quote(svg, safe="") + + +def available() -> list[str]: + return sorted(_ensure().keys()) + + +if __name__ == "__main__": + args = sys.argv[1:] + if not args or args[0] in ("-h", "--help"): + print(__doc__) + elif args[0] == "--list": + for n in available(): + print(n) + else: + print(data_uri(" ".join(args))) diff --git a/.claude/skills/arquitectura-drawio/scripts/glyph.py b/.claude/skills/arquitectura-drawio/scripts/glyph.py new file mode 100644 index 0000000..c849493 --- /dev/null +++ b/.claude/skills/arquitectura-drawio/scripts/glyph.py @@ -0,0 +1,135 @@ +"""glyph — resolución de iconos de tecnología para draw.io (AGNÓSTICO de stack). + +Las arquitecturas usan tecnologías variadas según el proyecto (GCP, AWS, Azure, +LangChain, Langfuse, Google ADK, React, Node, Python, Docker, Kubernetes, ...). +Regla: SIEMPRE el logo OFICIAL de la tecnología; si no existe, un glifo genérico. +Todo se resuelve a data URI URL-encoded (embebible con `shape=image`). Solo stdlib; +cachea cada SVG en el temp del sistema. + +Entradas: + - logo(name, color) -> LOGO OFICIAL (punto de entrada recomendado). Prueba, en + orden: marca (simple-icons, ~3000 logos) y producto Google + Cloud (gcp_icon). Sirve para cualquier tecnología. + - simple_icon(slug, color) -> logo de marca puntual (simple-icons). + - material(symbol, color) -> GLIFO GENÉRICO de fallback (code, database, api, hub, + settings, smart_toy...) vía Material Symbols (Apache-2.0). + Úsalo cuando NO haya logo oficial (p. ej. Google ADK, un + componente custom). + - globe(color) -> globo/Internet (atajo de material('language')). + +CLI: + python glyph.py logo langchain # logo oficial (marca o producto GCP) + python glyph.py logo vertex_ai # cae a producto Google Cloud + python glyph.py brand awslambda "#FF9900" + python glyph.py glyph code "#cc0000" # glifo genérico + python glyph.py list-suggested # glifos genéricos por tipo +""" + +from __future__ import annotations + +import hashlib +import os +import sys +import tempfile +import urllib.parse +import urllib.request + +SIMPLE = "https://cdn.jsdelivr.net/npm/simple-icons/icons/{}.svg" +MATERIAL = "https://cdn.jsdelivr.net/npm/@material-symbols/svg-400/outlined/{}.svg" +CACHE = os.path.join(tempfile.gettempdir(), "drawio_glyphs") + +# Glifos genéricos sugeridos según el tipo de componente (Material Symbols). +SUGGESTED = { + "código / microservicio / componente": "code", + "componente desplegable / paquete": "deployed_code", + "agente / kit de agentes (p. ej. ADK)": "smart_toy", + "API / endpoint genérico": "api", + "base de datos / almacén": "database", + "cola / mensajería": "forum", + "evento / trigger": "bolt", + "red / hub / integración / orquestador": "hub", + "configuración / proceso": "settings", + "servidor / DNS": "dns", + "internet / web (globo)": "language", + "seguridad / auth": "lock", + "usuario / actor": "person", + "modelo / IA": "neurology", +} + + +def _fetch(url: str) -> str: + os.makedirs(CACHE, exist_ok=True) + local = os.path.join(CACHE, hashlib.sha256(url.encode()).hexdigest()[:16] + ".svg") + if not os.path.exists(local): + req = urllib.request.Request(url, headers={"User-Agent": "drawio-kit"}) + with urllib.request.urlopen(req, timeout=30) as r: + data = r.read() + with open(local, "wb") as fh: + fh.write(data) + with open(local, encoding="utf-8") as fh: + return fh.read() + + +def _uri(svg: str, color: str | None) -> str: + if color: # el path hereda el fill puesto en el + svg = svg.replace(" str: + """Logo de marca (simple-icons). `slug` p.ej.: awslambda, amazonsqs, microsoftazure, + apache. `color` opcional (hex) para colorizar; por defecto negro.""" + return _uri(_fetch(SIMPLE.format(slug)), color) + + +def material(symbol: str, color: str = "#5f6368") -> str: + """Glifo genérico de fallback (Material Symbols). Ver SUGGESTED para nombres útiles.""" + return _uri(_fetch(MATERIAL.format(symbol)), color) + + +def globe(color: str = "#5f6368") -> str: + return material("language", color) + + +def logo(name: str, color: str | None = None) -> str: + """Logo OFICIAL de una tecnología (punto de entrada agnóstico de stack). + + Prueba, en orden: (1) marca en simple-icons (langchain, langfuse, react, nodejs, + python, docker, kubernetes, awslambda, microsoftazure, ...) y (2) producto Google + Cloud (gcp_icon: vertex_ai, bigquery, cloud_run, ...). Si NO hay logo oficial, lanza + KeyError sugiriendo un glifo genérico con material() — nunca inventa un icono. + """ + try: + return simple_icon(name, color) + except Exception: + pass + try: # gcp_icon vive en la misma carpeta de la skill + from gcp_icon import data_uri as _gcp + + return _gcp(name) + except Exception: + pass + raise KeyError( + f"sin logo oficial para {name!r}. Usa un glifo genérico de fallback, p. ej. " + f"material('code'|'hub'|'database'|'smart_toy', color). Ver `list-suggested`." + ) + + +if __name__ == "__main__": + a = sys.argv[1:] + if not a or a[0] in ("-h", "--help"): + print(__doc__) + elif a[0] == "list-suggested": + for k, v in SUGGESTED.items(): + print(f" {v:16} <- {k}") + elif a[0] == "logo": + print(logo(a[1], a[2] if len(a) > 2 else None)) + elif a[0] == "brand": + print(simple_icon(a[1], a[2] if len(a) > 2 else None)) + elif a[0] == "glyph": + print(material(a[1], a[2] if len(a) > 2 else "#5f6368")) + else: + print( + "uso: glyph.py [logo [color] | brand [color] | " + "glyph [color] | list-suggested]" + ) From 0e9ea8e13b76d13ff2c30a222fcf22de1b7fa33b Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Wed, 29 Jul 2026 14:14:05 -0500 Subject: [PATCH 49/55] chore(skills): SKILL is added to manage the flow of commits --- .claude/skills/git-commits/SKILL.md | 70 +++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 .claude/skills/git-commits/SKILL.md diff --git a/.claude/skills/git-commits/SKILL.md b/.claude/skills/git-commits/SKILL.md new file mode 100644 index 0000000..476870a --- /dev/null +++ b/.claude/skills/git-commits/SKILL.md @@ -0,0 +1,70 @@ +--- +name: git-commits +description: Convenciones de commits del proyecto (Conventional Commits en inglés, scopes válidos, cuerpo del mensaje, cadencia y agrupación de cambios). Usar SIEMPRE antes de redactar un mensaje de commit o de decidir cómo agrupar los cambios de una sesión en commits. +--- + +# Convenciones de commits + +Reglas permanentes (también en CLAUDE.md): commit por tarea lógica, 3–6 por +jornada, mensaje en inglés, y **nunca** commitear sin confirmación del usuario. + +## Formato del mensaje + +`(): ` — Conventional Commits, en inglés. + +**Types:** + +- `feat` — new feature or capability +- `fix` — bug fix +- `refactor` — code change that neither fixes a bug nor adds a feature +- `test` — adding or updating tests +- `docs` — documentation only +- `chore` — tooling, dependencies, CI, config +- `style` — formatting, whitespace (no logic change) +- `perf` — performance improvement + +**Scopes** (lowercase, una palabra, según capas y componentes del proyecto), como: +`domain`, `application`, `infrastructure`, `llm`, `retrieval`, `ocr`, +`privacy`, `estimator`, `api`, `agent`, `workflow`, `prompts`, `config`, +`deps`, `ci`, `tests`, `docs`, `mlops`, `notebooks`, `rag`. + +**Ejemplos:** + +``` +feat(llm): add streaming support to provider +fix(retrieval): handle empty search results +refactor(agent): switch from inheritance to composition +test(domain): add unit tests for Protocol implementations +docs(architecture): add ADR for prompt loading decision +chore(deps): upgrade pydantic to v2.9 +``` + +## Cuerpo del mensaje (opcional) + +Es completamente opcional y debe colocarse únicamente en el caso en que el mensaje +del commit no sea lo suficientemente claro para entender los cambios. Procura solo +emplear el mensaje del commit y el cuerpo solo úsalo cuando los cambios son realmente +grandes o tocan muchos componentes y/o archivos. + +Explica el **por qué**, no el qué — el diff ya muestra el qué. Línea en blanco +entre título y cuerpo. Si la decisión no es obvia, explícala: + +``` +refactor(agent): switch from inheritance to composition + +Base class was creating coupling between ResearchAgent and PQRSAgent +because streaming behavior differed. Composition via agent_utils.py +keeps each agent self-contained. +``` + +En commits que tocan superficie LLM, documentar riesgo OWASP-LLM y mitigación +en el cuerpo (tabla de riesgos en CLAUDE.md). + +## Cadencia y agrupación + +- Commit por tarea lógica, no por archivo: un provider y sus tests son UN + commit; el provider y un typo del README son DOS commits (feat + docs). +- Un commit debe poder revertirse sin romper otras cosas y tener un propósito + claro. +- 3–6 commits por jornada típica: menos = commits demasiado grandes para + revisar; más = micro-commits que ensucian el historial. From 07d93564855c8a52b24aa483b5de849018afd2eb Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Wed, 29 Jul 2026 14:15:22 -0500 Subject: [PATCH 50/55] chore(system): add concepts tutor system with 15 seed questions --- .claude/skills/interview-bank/SKILL.md | 116 +++++ .claude/skills/weekly-essay/SKILL.md | 89 ++++ docs/interview_prep/bank.md | 426 ++++++++++++++++++ .../by_topic/async_concurrencia.md | 88 ++++ .../by_topic/clean_architecture.md | 123 +++++ docs/interview_prep/by_topic/protocols.md | 68 +++ docs/interview_prep/by_topic/rag_retrieval.md | 100 ++++ notebooks/taller_retorno_researchos.ipynb | 185 ++++++-- 8 files changed, 1167 insertions(+), 28 deletions(-) create mode 100644 .claude/skills/interview-bank/SKILL.md create mode 100644 .claude/skills/weekly-essay/SKILL.md create mode 100644 docs/interview_prep/bank.md create mode 100644 docs/interview_prep/by_topic/async_concurrencia.md create mode 100644 docs/interview_prep/by_topic/clean_architecture.md create mode 100644 docs/interview_prep/by_topic/protocols.md create mode 100644 docs/interview_prep/by_topic/rag_retrieval.md diff --git a/.claude/skills/interview-bank/SKILL.md b/.claude/skills/interview-bank/SKILL.md new file mode 100644 index 0000000..0b0812d --- /dev/null +++ b/.claude/skills/interview-bank/SKILL.md @@ -0,0 +1,116 @@ +--- +name: interview-bank +description: Generar candidatos de preguntas de entrevista de AI Engineer basados en el trabajo real de la semana (commits, work_log, learnings), y curar las seleccionadas hacia el banco maestro. Usar cuando el usuario pida "generá el borrador de preguntas de esta semana" o "movés las marcadas al banco". +--- + +# Banco de preguntas de entrevista + +Sistema semanal para generar y curar preguntas de entrevista para AI Engineer, +derivadas del código real del proyecto — no de libros genéricos. El objetivo es +llegar a 80–100 preguntas al final del proyecto, categorizadas por tema, cada +una con respuesta esperada, trampa común y ejemplo del código propio. + +## Archivos involucrados + +- `docs/interview_prep/bank.md` — banco maestro, fuente de verdad +- `docs/interview_prep/by_topic/{tema}.md` — vistas por tema (regenerables desde bank.md) +- `docs/interview_prep/weekly_drafts/YYYY-WNN.md` — borradores semanales antes de curar +- `docs/interview_prep/weekly_drafts/YYYY-WNN.done.md` — draft archivado tras curación + +## Comandos del usuario + +| Comando (aprox.) | Acción del agente | +|---|---| +| `Generá el borrador de preguntas de esta semana` | Crea `weekly_drafts/YYYY-WNN.md` con 8–12 candidatos | +| `Movés las marcadas al banco` | Migra checkboxed al banco maestro, regenera `by_topic/` | +| `Mostrame las preguntas del tema X` | Lee `by_topic/{tema}.md` y responde inline | + +## Generación de candidatos + +Cuando el usuario pide el borrador semanal, el proceso es: + +1. Ejecutar `git log --since='7 days ago' --oneline --stat` para conocer commits + y archivos tocados esta semana. +2. Leer las entradas más recientes de `docs/work_log.md` y `docs/learnings.md`. +3. Leer `docs/interview_prep/bank.md` completo para conocer preguntas existentes + y evitar duplicados. +4. Identificar los temas técnicos que aparecen esta semana. Prefijos válidos: + - `CA` — Clean Architecture + - `PR` — Protocols + - `RG` — RAG y retrieval (BM25, vector, hybrid, RRF, rerank) + - `AS` — Async y concurrencia + - `TS` — Testing con mocks + - `LG` — LangGraph y agentes + - `OB` — Observabilidad y evals + - `GR` — Guardrails y seguridad + - `LM` — LLM providers y SDKs + - `IN` — Ingesta (chunking, embeddings, PDF parsing) +5. Por cada tema tocado, generar 2–4 candidatos con niveles progresivos: + - **básico**: definición del concepto + - **intermedio**: trade-off, "por qué X en vez de Y" + - **avanzado**: extensión, "cómo diseñarías Z" o "qué falla si Y" +6. **Regla anti-duplicado**: si una pregunta candidata cubre el mismo concepto + que una ya presente en `bank.md`, marcarla como `[DUP de ID-XXX]` y omitirla. +7. Cada pregunta debe tener un ejemplo concreto del código del proyecto (ruta + de archivo y línea, o SHA del commit que la origina). No respuestas de libro. + +## Formato de cada candidato en el draft + +```markdown +- [ ] **Tema: {nombre}** — Nivel: básico|intermedio|avanzado + - **Pregunta:** ... + - **Respuesta esperada** (3–5 oraciones): ... + - **Trampa común:** ... + - **Ejemplo en el proyecto:** `src/researchos/.../archivo.py:LN` o commit `abc1234` + - **Generada desde:** commit `abc1234` o "learnings del DD/MM" +``` + +## Curación al banco maestro + +Cuando el usuario dice "movés las marcadas al banco": + +1. Leer `weekly_drafts/YYYY-WNN.md` y filtrar solo los items con `[x]`. +2. Asignar ID incremental por tema usando el prefijo (ej. `CA-006`, `AS-004`). + El próximo número se calcula leyendo el ID más alto existente en `bank.md` + para ese prefijo. +3. Insertar cada pregunta curada en `bank.md` bajo la sección del tema + correspondiente, en orden de ID ascendente. +4. Regenerar los archivos `by_topic/{tema}.md` a partir de `bank.md` + (sobrescribir completamente). +5. Renombrar el draft: `weekly_drafts/YYYY-WNN.md` → `weekly_drafts/YYYY-WNN.done.md`. + Nunca borrar drafts — son histórico. +6. Confirmar al usuario cuántas preguntas se agregaron por tema. + +## Formato del banco maestro (`bank.md`) + +```markdown +# Banco de preguntas — ResearchOS + +## Índice por tema +- [Clean Architecture (CA)](#clean-architecture) — N preguntas +- [Protocols (PR)](#protocols) — N preguntas +- ... + +--- + +## Clean Architecture + +### [CA-001] Nivel: básico +**Pregunta:** ... +**Respuesta esperada:** ... +**Trampa común:** ... +**Ejemplo en el proyecto:** ... + +### [CA-002] Nivel: intermedio +... +``` + +## Anti-patrones + +- No generar preguntas genéricas sin conexión al código del repo +- No superar 12 candidatos por semana (calidad > cantidad) +- No re-generar preguntas ya presentes en `bank.md`, aunque estén en semanas + distintas del historial +- No inventar rutas de archivos o commits que no existan en el repo +- No escribir respuestas esperadas que superen 5 oraciones (rigor sobre extensión) +- No mezclar temas en una sola pregunta — si toca dos temas, cortarla en dos diff --git a/.claude/skills/weekly-essay/SKILL.md b/.claude/skills/weekly-essay/SKILL.md new file mode 100644 index 0000000..300e62d --- /dev/null +++ b/.claude/skills/weekly-essay/SKILL.md @@ -0,0 +1,89 @@ +--- +name: weekly-essay +description: Proponer un tema de ensayo semanal (400–500 palabras) basado en el trabajo real, priorizando huecos conceptuales, decisiones arquitectónicas no triviales, o temas donde el usuario mostró fragilidad en el banco de preguntas. Usar cuando el usuario pida "proponeme un tema de ensayo para esta semana". +--- + +# Ensayo semanal + +Sistema para forzar consolidación conceptual una vez por semana. El usuario +escribe 400–500 palabras sin abrir código, revisa contra la realidad, y +extrae una versión destilada hacia `learnings.md`. El agente **propone el +tema**, no lo escribe. + +## Archivos involucrados + +- `docs/essays/YYYY-WNN-{slug-tema}.md` — el ensayo escrito por el usuario +- `docs/essays/prompts/YYYY-WNN.md` — propuesta de tema generada por el agente +- `docs/learnings.md` — destino final después de revisión (versión destilada) + +## Comando del usuario + +| Comando (aprox.) | Acción del agente | +|---|---| +| `Proponeme un tema de ensayo para esta semana` | Crea `essays/prompts/YYYY-WNN.md` | + +## Selección del tema + +Cuando el usuario pide una propuesta, el proceso es: + +1. Ejecutar `git log --since='7 days ago' --stat` para conocer commits y + archivos tocados. +2. Leer las entradas más recientes de `docs/work_log.md`, `docs/learnings.md`, + y si existe, `docs/interview_prep/weekly_drafts/YYYY-WNN.md`. +3. Identificar 3–4 temas candidatos según estos criterios de prioridad: + - **Alta**: decisión arquitectónica no trivial tomada esta semana + (ej. "por qué composición en vez de herencia en agents"); tema donde + varias preguntas del banco quedaron marcadas como flojas o mal + respondidas. + - **Media**: concepto nuevo que apareció esta semana y aún no está + consolidado (ej. primera vez tocando LangGraph, primera vez usando + LLM-as-judge). + - **Baja**: temas de rutina o cosas ya cubiertas en ensayos previos + (buscar en `docs/essays/` para evitar repetir). +4. Elegir UN tema principal y proponer 1–2 alternativas. + +## Formato de la propuesta + +```markdown +# Tema propuesto — Semana YYYY-WNN + +## Título sugerido +{título específico y técnico, evitar títulos vagos como "aprendizajes de la semana"} + +## Por qué este tema +{2–3 oraciones conectando el tema con commits, decisiones, o huecos concretos +de la semana. Cita archivos o commits específicos.} + +## Preguntas guía para arrancar la escritura +- ¿...? +- ¿...? +- ¿...? + +## Longitud +400–500 palabras. Escribir sin abrir código ni notas. Después de escribir, +comparar contra el código y anotar lo que no se recordó. + +## Alternativas si este no resuena +1. **{título alternativo 1}** — {una oración sobre de qué trataría} +2. **{título alternativo 2}** — {idem} +``` + +## Cierre del ciclo + +El agente **no** escribe el ensayo ni lo revisa. El ciclo completo es: + +1. Viernes: el usuario pide propuesta, escribe ensayo en 25–30 min. +2. Lunes siguiente: el usuario lleva el ensayo a un chat con el tutor humano + o con un modelo externo para revisión estricta (10 min). +3. Aplicar correcciones y extraer versión destilada (5–10 oraciones) hacia + `docs/learnings.md` con formato estándar. + +## Anti-patrones + +- No proponer temas vagos ("qué aprendí esta semana", "reflexiones") +- No proponer temas que ya tienen un ensayo previo (revisar `docs/essays/`) +- No escribir el ensayo por el usuario, ni siquiera un draft +- No proponer más de un tema principal — decisión, no menú de indecisión +- No inventar decisiones o commits que no existan; si la semana fue floja + en commits, decirlo y proponer un tema de repaso conceptual de algo previo + en vez de forzar novedad diff --git a/docs/interview_prep/bank.md b/docs/interview_prep/bank.md new file mode 100644 index 0000000..cbdefc1 --- /dev/null +++ b/docs/interview_prep/bank.md @@ -0,0 +1,426 @@ +# Banco de preguntas — ResearchOS + +Preguntas de entrevista para AI Engineer, derivadas del código real del +proyecto. Meta: 80–100 preguntas al final de V5. + +## Índice por tema + +- [Clean Architecture (CA)](#clean-architecture) — 5 preguntas +- [Protocols (PR)](#protocols) — 3 preguntas +- [RAG y retrieval (RG)](#rag-y-retrieval) — 4 preguntas +- [Async y concurrencia (AS)](#async-y-concurrencia) — 3 preguntas +- [Testing (TS)](#testing) — 0 preguntas +- [LangGraph y agentes (LG)](#langgraph-y-agentes) — 0 preguntas +- [Observabilidad y evals (OB)](#observabilidad-y-evals) — 0 preguntas +- [Guardrails y seguridad (GR)](#guardrails-y-seguridad) — 0 preguntas +- [LLM providers y SDKs (LM)](#llm-providers-y-sdks) — 0 preguntas +- [Ingesta (IN)](#ingesta) — 0 preguntas + +--- + +## Clean Architecture + +#### [CA-001] Nivel: básico +**Pregunta:** Explicá la regla de dependencia entre las tres capas +(`domain`, `application`, `infrastructure`) y qué se rompe si se viola. + +**Respuesta esperada:** La dirección de dependencia es +`infrastructure → application → domain`. Domain no importa de nadie; +application importa solo de domain (Protocols y modelos); infrastructure +importa de domain para implementar los Protocols. Los composition roots +(scripts, factories, `__main__`) son los únicos que conocen tanto +application como infrastructure, típicamente inyectando implementaciones +concretas. Si domain importa de infrastructure, cambiar de Chroma a Qdrant +obliga a tocar código de negocio, y los tests unitarios de dominio dejan +de correr sin instalar el stack completo. + +**Trampa común:** Invertir la dirección diciendo "application importa de +infrastructure". Es al revés — application no sabe qué implementaciones +concretas existen; solo conoce los Protocols de domain. + +**Ejemplo en el proyecto:** +`src/researchos/application/services/ingestion_service.py` recibe +`store: VectorStore | None` — un Protocol de `domain/interfaces.py`, no +`ChromaVectorStore` de `infrastructure/`. + +--- + +#### [CA-002] Nivel: intermedio +**Pregunta:** Dado un archivo cualquiera del proyecto, ¿cómo decidís a qué +capa pertenece? Dame la regla en una oración y tres ejemplos concretos del +repo. + +**Respuesta esperada:** Regla: **domain** contiene lógica pura sin +dependencias externas (modelos Pydantic, Protocols, excepciones de negocio, +prompts .txt); **application** contiene orquestación programada contra +Protocols (services y agents); **infrastructure** contiene implementaciones +concretas con dependencias externas (SDKs, drivers, frameworks web). +Ejemplos del repo: `domain/models.py` (Document, Chunk) es domain porque +solo usa Pydantic; `application/services/retrieval_service.py` +(hybrid_search) es application porque orquesta contra Protocols; +`infrastructure/retrieval/chroma.py` (ChromaVectorStore) es infrastructure +porque depende del SDK de Chroma. + +**Trampa común:** Colocar código con lógica de negocio dentro de +infrastructure porque "toca a Chroma", o colocar código de infrastructure +dentro de application porque "hace algo importante". La distinción no es +por importancia — es por dependencias externas. + +**Ejemplo en el proyecto:** Ver la estructura completa en +`src/researchos/{domain,application,infrastructure}/`. + +--- + +#### [CA-003] Nivel: intermedio +**Pregunta:** Te piden agregar un endpoint HTTP nuevo (con FastAPI) que +reciba un caso y devuelva una recomendación. ¿En qué capa vive el endpoint? +¿Cuál es el rol de FastAPI? + +**Respuesta esperada:** El endpoint vive en `infrastructure/api/routers/`. +FastAPI es canal, no motor — expone el motor (services y agentes de +`application/`) por HTTP. El router es delgado: recibe el payload como +Pydantic model, valida, llama a un service o agente, devuelve el response +model. El router en sí no computa nada — solo traduce HTTP a llamadas +internas y viceversa. + +**Trampa común:** Colocar la lógica de negocio (la "recomendación") dentro +del router o en `application/services`. La lógica de recomendación es +application; el endpoint que la expone es infrastructure. Si mañana +descontinúan FastAPI y aparece un framework nuevo, cambiás el router sin +tocar la lógica. + +**Ejemplo en el proyecto:** Estructura existente en +`src/researchos/infrastructure/api/routers/`. + +--- + +#### [CA-004] Nivel: intermedio +**Pregunta:** Querés agregar un canal nuevo: Slack. ¿Dónde va el código +del bot? Trazá el flujo de un mensaje entrando por Slack hasta la +respuesta. + +**Respuesta esperada:** El bot va en `infrastructure/bot/slack.py`. Flujo: +(1) llega evento HTTP de Slack al webhook expuesto en el bot; (2) el bot +extrae texto y contexto (user_id, channel_id); (3) el bot llama a +`rag_service.answer_query(text=..., llm=..., store=...)` — llamada +agnóstica al canal; (4) `rag_service` orquesta hybrid_search + rerank + +generación; (5) devuelve el string; (6) el bot publica la respuesta en +Slack usando el SDK. Motor no sabe que existe Slack. + +**Trampa común:** Meter el bot en `application/`. Los SDKs de Slack o +Telegram son dependencias externas — pertenecen a infrastructure. El bot +importa del motor, no al revés. + +**Ejemplo en el proyecto:** Planeado para V2 — +`infrastructure/bot/telegram.py` seguirá el mismo patrón. + +--- + +#### [CA-005] Nivel: avanzado +**Pregunta:** Alguien te pide agregar Qdrant como vector store alternativo +a Chroma. ¿Qué archivos crear? ¿Qué archivos modificar? ¿Qué archivos NO +deberías tocar y por qué? + +**Respuesta esperada:** Crear: `infrastructure/retrieval/qdrant.py` con +clase `QdrantVectorStore` que implementa el Protocol `VectorStore`; +`tests/integration/test_qdrant.py`. Modificar: `config.py` para agregar +la opción `Literal["chroma", "qdrant"]` en Settings; el composition root +donde se instancia el store (scripts como `scripts/ingest_papers.py`). +No tocar: nada en `domain/` (las abstracciones no cambian); nada en +`application/services/*` (programan contra Protocols); ni los tests +unitarios de application (los mocks de conftest.py siguen sirviendo). +Este es el pago concreto de haber aplicado Clean Architecture desde el +inicio. + +**Trampa común:** Modificar `application/services/ingestion_service.py` +para "adaptarlo a Qdrant". Eso rompe la razón de existir del Protocol +VectorStore — si tenés que tocar application cada vez que agregás una +implementación nueva, el patrón está mal aplicado. + +**Ejemplo en el proyecto:** Mismo patrón que ya se usó al agregar +`BM25Retriever` como segunda implementación de `Retriever`. + +--- + +## Protocols + +#### [PR-001] Nivel: básico +**Pregunta:** ¿Qué es un `Protocol` de Python y en qué se diferencia de +una clase abstracta (`ABC`)? + +**Respuesta esperada:** Un Protocol define un contrato **estructural** — +cualquier clase con los métodos y firmas correctas lo satisface, sin +necesidad de heredar. La verificación es típicamente estática (mypy). Una +ABC define un contrato **nominal** — requiere herencia explícita +(`class Foo(BaseClass):`) y hace enforcement en runtime (`TypeError` si +un método abstracto no se implementa al instanciar). Protocol permite +que clases de librerías externas cumplan el contrato sin modificarlas; +ABC obliga a controlar el árbol de herencia. + +**Trampa común:** Tratarlos como sinónimos "más modernos" uno del otro. +Son diferentes en runtime behavior y en filosofía de diseño. + +**Ejemplo en el proyecto:** `domain/interfaces.py` define +`LLMProvider(Protocol)` con métodos `generate` y `stream`. Cualquier clase +con esos métodos lo satisface, incluyendo mocks de test que no heredan +de nada. + +--- + +#### [PR-002] Nivel: intermedio +**Pregunta:** ¿Por qué en tu proyecto usás Protocols para las abstracciones +de `application/` en lugar de clases abstractas? + +**Respuesta esperada:** Los Protocols evitan acoplar `application/` a +jerarquías de herencia. Si Anthropic saca un SDK nuevo con estructura +distinta, la nueva implementación de `LLMProvider` vive en infrastructure +sin necesitar herencia común. También facilita testing: los mocks pueden +ser clases simples sin importar nada de infrastructure. Además, structural +subtyping permite que clases de terceros cumplan el contrato sin tocar +su código. + +**Trampa común:** Pensar que "ABC hace lo mismo con más rigor". El +"rigor" adicional de ABC (enforcement al instanciar) es innecesario si +usás mypy en CI — y a cambio pagás con acoplamiento por herencia. + +**Ejemplo en el proyecto:** `application/services/rag_service.py` recibe +`llm: LLMProvider` y `store: VectorStore` — no sabe si son AnthropicLLM, +GeminiLLM, ChromaVectorStore o mocks; solo sabe qué métodos puede llamar. + +--- + +#### [PR-003] Nivel: avanzado +**Pregunta:** ¿Cómo se testea código de `application/` que depende de un +Protocol, sin usar implementaciones reales de infrastructure? + +**Respuesta esperada:** Se crea una clase mock local (en el test o en +`conftest.py`) que implementa los métodos del Protocol con comportamiento +controlado. Como Protocol es structural, no hace falta heredar de nada — +solo tener los métodos con las firmas correctas. Ejemplo: `MockVectorStore` +en `tests/conftest.py` con `search` y `upsert` que operan sobre una lista +en memoria. Los tests de `application/` inyectan estos mocks y verifican +comportamiento sin tocar Chroma real. Alternativa débil: `unittest.mock. +MagicMock` — funciona pero pierde chequeo estático (podés llamar métodos +que el Protocol no declara y no te avisa). + +**Trampa común:** Usar `MagicMock` universalmente por comodidad. Perdés la +señal de mypy sobre si tu test está usando el Protocol correctamente. + +**Ejemplo en el proyecto:** `tests/conftest.py` tiene `MockVectorStore` y +mocks de LLMProvider usados en `tests/unit/application/`. + +--- + +## RAG y retrieval + +#### [RG-001] Nivel: básico +**Pregunta:** Dame un ejemplo concreto de una query donde BM25 supera a +la búsqueda vectorial, y otro donde vectorial supera a BM25. Explicá por qué. + +**Respuesta esperada:** BM25 supera cuando la query contiene siglas, +nombres propios técnicos o términos raros con coincidencia exacta: por +ejemplo `"BERT vs GPT-3"` — un vectorizador puede diluir esas siglas en un +embedding genérico, pero BM25 encuentra ocurrencias exactas. Vectorial +supera cuando la query es semántica y el corpus usa vocabulario distinto: +por ejemplo `"papers on models that reason step by step"` — encuentra +papers de "chain-of-thought" aunque la query no contenga esas palabras. +Son complementarios: por eso hybrid search existe. + +**Trampa común:** Pensar que uno es "mejor" en absoluto. Cada uno tiene +un tipo de query donde brilla. + +**Ejemplo en el proyecto:** `infrastructure/retrieval/bm25.py` y +`infrastructure/retrieval/chroma.py` — ambos en producción, combinados +vía `application/services/retrieval_service.py`. + +--- + +#### [RG-002] Nivel: intermedio +**Pregunta:** Explicá Reciprocal Rank Fusion (RRF) paso a paso. ¿Por qué +suma las contribuciones cuando un documento aparece en dos rankings, en +lugar de promediarlas o quedarse con la mayor? + +**Respuesta esperada:** RRF recibe rankings (no scores) de N retrievers, +cada uno con sus top-K candidatos. Descarta los scores originales — trabaja +solo con posiciones (rank 1, 2, 3...). Por cada documento en cada ranking, +calcula contribución `1 / (rrf_k + rank)` con rank 1-indexed. Cuando un +doc aparece en múltiples rankings, **suma** las contribuciones. Ordena por +score total descendente y devuelve top-K. La suma premia el consenso: +docs que dos retrievers rankean alto suben más que docs rankeados alto +por uno solo. Promediar diluiría la señal (un rank alto se compensaría +con la ausencia); quedarse con la mayor ignoraría el consenso. + +**Trampa común:** Decir que RRF "combina scores". No combina scores — +descarta los scores originales precisamente porque tienen escalas +incomparables (BM25 sin límite superior, coseno entre -1 y 1). Combina +**posiciones**, que sí son comparables. + +**Ejemplo en el proyecto:** `application/services/retrieval_service.py` +implementa `hybrid_search` con RRF. + +--- + +#### [RG-003] Nivel: intermedio +**Pregunta:** En tu hybrid_search, cada retriever devuelve `k*2` +candidatos aunque al final devuelvas solo `k`. ¿Por qué no pedir +exactamente `k`? + +**Respuesta esperada:** Porque después de la fusión hay deduplicación y +reordenamiento. Si dos retrievers devuelven exactamente los mismos `k` +docs, después de deduplicar quedás con `k` únicos y la fusión no aportó +nada — mismo resultado que un solo retriever. Con `k*2`, tenés material +adicional: docs que aparecen en posiciones 4-6 de ambos retrievers son +consensuados aunque ninguno los rankee arriba, y RRF los promueve. En +producción el overlap entre vectorial y BM25 sobre el mismo corpus está +entre 20% y 60%, entonces `k*2` es un colchón razonable. + +**Trampa común:** Pensar que el escenario "cero overlap" invalida `k*2`. +Cierto, ahí `k=candidatos` da lo mismo — pero no sabés el overlap antes +de correr, y el default tiene que servir para el caso peor. + +**Ejemplo en el proyecto:** `hybrid_search` en `retrieval_service.py`, +parámetro `candidates_per_retriever` con default `k*2` cuando es `None`. + +--- + +#### [RG-004] Nivel: avanzado +**Pregunta:** ¿Cómo evaluarías un sistema RAG en producción sin caer en +data leakage? Mencioná qué medís, cómo obtenés ground truth, y qué hacés +con las queries que fallan. + +**Respuesta esperada:** Métricas objetivas: faithfulness (¿el LLM inventa +cosas no soportadas por los docs?), context precision (¿los docs +recuperados son relevantes?), answer relevancy (¿la respuesta contesta la +pregunta?). Todas con LLM-as-judge externo — un modelo distinto al que +genera, para evitar sesgo de auto-evaluación (ej. Gemini Flash mientras +generás con Claude). Ground truth: dos fuentes. (a) curación humana +externa — alguien que NO conoce el corpus escribe queries que representan +cómo un usuario preguntaría, y anota respuesta esperada. (b) feedback en +producción — thumbs up/down, reformulaciones (señal de que la primera +respuesta no sirvió). Queries que fallan: van a un regression dataset, +etiquetadas manualmente, corren en CI/CD, bloquean deploys que degraden +el score. + +**Trampa común:** Construir queries de test conociendo el corpus indexado. +Eso es data leakage — el sistema "acierta" porque las queries están +alineadas con el contenido, no porque sea bueno. Mide qué tan bien recuerda +lo que ya sabías que estaba, no calidad de retrieval. + +**Ejemplo en el proyecto:** El `learnings.md` del 01/06/2026 documenta +exactamente este problema al observar scores 1.000/1.000 en el eval — la +razón fue leakage por construir queries desde el corpus. + +--- + +## Async y concurrencia + +#### [AS-001] Nivel: intermedio +**Pregunta:** ¿Cuál es la diferencia entre async y paralelismo real? +¿Cuándo usás cada uno? + +**Respuesta esperada:** Async es concurrencia cooperativa en un solo +thread. Un event loop rota entre corrutinas cuando alguna hace `await` +sobre I/O — el thread nunca está inactivo, pero solo una cosa se ejecuta +a la vez. Analogía: un mesero muy hábil que nunca se queda parado. +Paralelismo real es múltiples procesos (o threads con caveats por el GIL) +ejecutando código simultáneamente en cores distintos — múltiples cuerpos +haciendo trabajo real al mismo tiempo. Async para I/O bound (HTTP, disco, +red); multiprocessing para CPU bound (embeddings, ML inference). +Combinables: uvicorn con N workers procesos, cada uno con event loop +async, escala bien para APIs con alto tráfico. + +**Trampa común:** Marcar todo `async def` porque "quiero que sea rápido". +Async no acelera CPU — solo aprovecha tiempos muertos de I/O. Sin `await` +adentro, `async def` es una mentira que confunde y bloquea el event loop. + +**Ejemplo en el proyecto:** `infrastructure/llm/anthropic_llm.py` es +async correctamente (HTTP a Anthropic). `infrastructure/retrieval/bm25.py` +es async por excepción consciente — uniformidad con Chroma en +`asyncio.gather`, aunque adentro no haya `await`. + +--- + +#### [AS-002] Nivel: intermedio +**Pregunta:** Tenés una función `embed_text(text) -> list[float]` que usa +`sentence-transformers` con un modelo local en CPU. ¿Va como `def` normal +o `async def`? ¿Por qué? + +**Respuesta esperada:** `def` normal. sentence-transformers en CPU es +cálculo puro — tokeniza, pasa por la red neuronal, devuelve el vector. No +hay I/O esperando en background. Marcarlo `async def` no lo paraleliza; en +realidad lo empeora, porque si adentro no hay `await`, la corrutina +bloquea el event loop mientras calcula, impidiendo que otras corrutinas +de I/O (llamadas HTTP, timers) corran en paralelo. Para paralelizar +embeddings, se usa (a) batching nativo del modelo (`model.encode(lista)` +es vectorizado internamente), o (b) `asyncio.to_thread(embed_text, texto)` +si estás en un pipeline async y necesitás no bloquear el loop. + +**Trampa común:** Marcarlo `async def` "por si acaso" o "porque el +pipeline es async". Ese razonamiento crea deuda técnica y bugs de +performance difíciles de diagnosticar. + +**Ejemplo en el proyecto:** Regla mental documentada en `learnings.md` +del 15/04/2026 — "¿Esperás algo externo? async. ¿Solo calculás en +memoria? def normal." + +--- + +#### [AS-003] Nivel: básico +**Pregunta:** El siguiente código pretende descargar tres URLs en +paralelo pero corre secuencial. Identificá el bug y explicá cómo +diagnosticarlo. + +```python +async def fetch_all(urls): + async with httpx.AsyncClient() as client: + results = [] + for url in urls: + result = await fetch(client, url) + results.append(result) + return results +``` + +**Respuesta esperada:** El bug es que `await` dentro del `for` serializa +las llamadas. Cada `await fetch(...)` espera que la anterior complete +antes de arrancar la siguiente. El event loop no está bloqueado — +simplemente no le dieron trabajo concurrente; nadie tiene múltiples +corrutinas en vuelo. Diagnóstico: tiempo total ≈ suma de tiempos +individuales, cuando debería ser ≈ tiempo del más lento. Corrección: +`results = await asyncio.gather(*[fetch(client, url) for url in urls])`. +Ahora las N corrutinas están en vuelo simultáneamente y el event loop +rota entre ellas mientras esperan I/O. + +**Trampa común:** Decir que "el for detiene el event loop". El for no +detiene nada — un for normal en código async es legítimo. El bug es la +falta de trabajo concurrente. La distinción importa: en producción, +diagnosticar "event loop bloqueado" vs "falta de concurrencia" lleva a +soluciones diferentes. + +**Ejemplo en el proyecto:** Ejercicio 6.2 del taller de retorno a +ResearchOS. + +--- + +## Testing + +_(sin preguntas todavía)_ + +## LangGraph y agentes + +_(sin preguntas todavía)_ + +## Observabilidad y evals + +_(sin preguntas todavía)_ + +## Guardrails y seguridad + +_(sin preguntas todavía)_ + +## LLM providers y SDKs + +_(sin preguntas todavía)_ + +## Ingesta + +_(sin preguntas todavía)_ diff --git a/docs/interview_prep/by_topic/async_concurrencia.md b/docs/interview_prep/by_topic/async_concurrencia.md new file mode 100644 index 0000000..a54a37d --- /dev/null +++ b/docs/interview_prep/by_topic/async_concurrencia.md @@ -0,0 +1,88 @@ +# Async y concurrencia — banco de preguntas + +3 preguntas. Fuente: `docs/interview_prep/bank.md`. + +#### [AS-001] Nivel: intermedio +**Pregunta:** ¿Cuál es la diferencia entre async y paralelismo real? +¿Cuándo usás cada uno? + +**Respuesta esperada:** Async es concurrencia cooperativa en un solo +thread. Un event loop rota entre corrutinas cuando alguna hace `await` +sobre I/O — el thread nunca está inactivo, pero solo una cosa se ejecuta +a la vez. Analogía: un mesero muy hábil que nunca se queda parado. +Paralelismo real es múltiples procesos (o threads con caveats por el GIL) +ejecutando código simultáneamente en cores distintos — múltiples cuerpos +haciendo trabajo real al mismo tiempo. Async para I/O bound (HTTP, disco, +red); multiprocessing para CPU bound (embeddings, ML inference). +Combinables: uvicorn con N workers procesos, cada uno con event loop +async, escala bien para APIs con alto tráfico. + +**Trampa común:** Marcar todo `async def` porque "quiero que sea rápido". +Async no acelera CPU — solo aprovecha tiempos muertos de I/O. Sin `await` +adentro, `async def` es una mentira que confunde y bloquea el event loop. + +**Ejemplo en el proyecto:** `infrastructure/llm/anthropic_llm.py` es +async correctamente (HTTP a Anthropic). `infrastructure/retrieval/bm25.py` +es async por excepción consciente — uniformidad con Chroma en +`asyncio.gather`, aunque adentro no haya `await`. + +--- + +#### [AS-002] Nivel: intermedio +**Pregunta:** Tenés una función `embed_text(text) -> list[float]` que usa +`sentence-transformers` con un modelo local en CPU. ¿Va como `def` normal +o `async def`? ¿Por qué? + +**Respuesta esperada:** `def` normal. sentence-transformers en CPU es +cálculo puro — tokeniza, pasa por la red neuronal, devuelve el vector. No +hay I/O esperando en background. Marcarlo `async def` no lo paraleliza; en +realidad lo empeora, porque si adentro no hay `await`, la corrutina +bloquea el event loop mientras calcula, impidiendo que otras corrutinas +de I/O (llamadas HTTP, timers) corran en paralelo. Para paralelizar +embeddings, se usa (a) batching nativo del modelo (`model.encode(lista)` +es vectorizado internamente), o (b) `asyncio.to_thread(embed_text, texto)` +si estás en un pipeline async y necesitás no bloquear el loop. + +**Trampa común:** Marcarlo `async def` "por si acaso" o "porque el +pipeline es async". Ese razonamiento crea deuda técnica y bugs de +performance difíciles de diagnosticar. + +**Ejemplo en el proyecto:** Regla mental documentada en `learnings.md` +del 15/04/2026 — "¿Esperás algo externo? async. ¿Solo calculás en +memoria? def normal." + +--- + +#### [AS-003] Nivel: básico +**Pregunta:** El siguiente código pretende descargar tres URLs en +paralelo pero corre secuencial. Identificá el bug y explicá cómo +diagnosticarlo. + +```python +async def fetch_all(urls): + async with httpx.AsyncClient() as client: + results = [] + for url in urls: + result = await fetch(client, url) + results.append(result) + return results +``` + +**Respuesta esperada:** El bug es que `await` dentro del `for` serializa +las llamadas. Cada `await fetch(...)` espera que la anterior complete +antes de arrancar la siguiente. El event loop no está bloqueado — +simplemente no le dieron trabajo concurrente; nadie tiene múltiples +corrutinas en vuelo. Diagnóstico: tiempo total ≈ suma de tiempos +individuales, cuando debería ser ≈ tiempo del más lento. Corrección: +`results = await asyncio.gather(*[fetch(client, url) for url in urls])`. +Ahora las N corrutinas están en vuelo simultáneamente y el event loop +rota entre ellas mientras esperan I/O. + +**Trampa común:** Decir que "el for detiene el event loop". El for no +detiene nada — un for normal en código async es legítimo. El bug es la +falta de trabajo concurrente. La distinción importa: en producción, +diagnosticar "event loop bloqueado" vs "falta de concurrencia" lleva a +soluciones diferentes. + +**Ejemplo en el proyecto:** Ejercicio 6.2 del taller de retorno a +ResearchOS. diff --git a/docs/interview_prep/by_topic/clean_architecture.md b/docs/interview_prep/by_topic/clean_architecture.md new file mode 100644 index 0000000..e974125 --- /dev/null +++ b/docs/interview_prep/by_topic/clean_architecture.md @@ -0,0 +1,123 @@ +# Clean Architecture — banco de preguntas + +5 preguntas. Fuente: `docs/interview_prep/bank.md`. + +#### [CA-001] Nivel: básico +**Pregunta:** Explicá la regla de dependencia entre las tres capas +(`domain`, `application`, `infrastructure`) y qué se rompe si se viola. + +**Respuesta esperada:** La dirección de dependencia es +`infrastructure → application → domain`. Domain no importa de nadie; +application importa solo de domain (Protocols y modelos); infrastructure +importa de domain para implementar los Protocols. Los composition roots +(scripts, factories, `__main__`) son los únicos que conocen tanto +application como infrastructure, típicamente inyectando implementaciones +concretas. Si domain importa de infrastructure, cambiar de Chroma a Qdrant +obliga a tocar código de negocio, y los tests unitarios de dominio dejan +de correr sin instalar el stack completo. + +**Trampa común:** Invertir la dirección diciendo "application importa de +infrastructure". Es al revés — application no sabe qué implementaciones +concretas existen; solo conoce los Protocols de domain. + +**Ejemplo en el proyecto:** +`src/researchos/application/services/ingestion_service.py` recibe +`store: VectorStore | None` — un Protocol de `domain/interfaces.py`, no +`ChromaVectorStore` de `infrastructure/`. + +--- + +#### [CA-002] Nivel: intermedio +**Pregunta:** Dado un archivo cualquiera del proyecto, ¿cómo decidís a qué +capa pertenece? Dame la regla en una oración y tres ejemplos concretos del +repo. + +**Respuesta esperada:** Regla: **domain** contiene lógica pura sin +dependencias externas (modelos Pydantic, Protocols, excepciones de negocio, +prompts .txt); **application** contiene orquestación programada contra +Protocols (services y agents); **infrastructure** contiene implementaciones +concretas con dependencias externas (SDKs, drivers, frameworks web). +Ejemplos del repo: `domain/models.py` (Document, Chunk) es domain porque +solo usa Pydantic; `application/services/retrieval_service.py` +(hybrid_search) es application porque orquesta contra Protocols; +`infrastructure/retrieval/chroma.py` (ChromaVectorStore) es infrastructure +porque depende del SDK de Chroma. + +**Trampa común:** Colocar código con lógica de negocio dentro de +infrastructure porque "toca a Chroma", o colocar código de infrastructure +dentro de application porque "hace algo importante". La distinción no es +por importancia — es por dependencias externas. + +**Ejemplo en el proyecto:** Ver la estructura completa en +`src/researchos/{domain,application,infrastructure}/`. + +--- + +#### [CA-003] Nivel: intermedio +**Pregunta:** Te piden agregar un endpoint HTTP nuevo (con FastAPI) que +reciba un caso y devuelva una recomendación. ¿En qué capa vive el endpoint? +¿Cuál es el rol de FastAPI? + +**Respuesta esperada:** El endpoint vive en `infrastructure/api/routers/`. +FastAPI es canal, no motor — expone el motor (services y agentes de +`application/`) por HTTP. El router es delgado: recibe el payload como +Pydantic model, valida, llama a un service o agente, devuelve el response +model. El router en sí no computa nada — solo traduce HTTP a llamadas +internas y viceversa. + +**Trampa común:** Colocar la lógica de negocio (la "recomendación") dentro +del router o en `application/services`. La lógica de recomendación es +application; el endpoint que la expone es infrastructure. Si mañana +descontinúan FastAPI y aparece un framework nuevo, cambiás el router sin +tocar la lógica. + +**Ejemplo en el proyecto:** Estructura existente en +`src/researchos/infrastructure/api/routers/`. + +--- + +#### [CA-004] Nivel: intermedio +**Pregunta:** Querés agregar un canal nuevo: Slack. ¿Dónde va el código +del bot? Trazá el flujo de un mensaje entrando por Slack hasta la +respuesta. + +**Respuesta esperada:** El bot va en `infrastructure/bot/slack.py`. Flujo: +(1) llega evento HTTP de Slack al webhook expuesto en el bot; (2) el bot +extrae texto y contexto (user_id, channel_id); (3) el bot llama a +`rag_service.answer_query(text=..., llm=..., store=...)` — llamada +agnóstica al canal; (4) `rag_service` orquesta hybrid_search + rerank + +generación; (5) devuelve el string; (6) el bot publica la respuesta en +Slack usando el SDK. Motor no sabe que existe Slack. + +**Trampa común:** Meter el bot en `application/`. Los SDKs de Slack o +Telegram son dependencias externas — pertenecen a infrastructure. El bot +importa del motor, no al revés. + +**Ejemplo en el proyecto:** Planeado para V2 — +`infrastructure/bot/telegram.py` seguirá el mismo patrón. + +--- + +#### [CA-005] Nivel: avanzado +**Pregunta:** Alguien te pide agregar Qdrant como vector store alternativo +a Chroma. ¿Qué archivos crear? ¿Qué archivos modificar? ¿Qué archivos NO +deberías tocar y por qué? + +**Respuesta esperada:** Crear: `infrastructure/retrieval/qdrant.py` con +clase `QdrantVectorStore` que implementa el Protocol `VectorStore`; +`tests/integration/test_qdrant.py`. Modificar: `config.py` para agregar +la opción `Literal["chroma", "qdrant"]` en Settings; el composition root +donde se instancia el store (scripts como `scripts/ingest_papers.py`). +No tocar: nada en `domain/` (las abstracciones no cambian); nada en +`application/services/*` (programan contra Protocols); ni los tests +unitarios de application (los mocks de conftest.py siguen sirviendo). +Este es el pago concreto de haber aplicado Clean Architecture desde el +inicio. + +**Trampa común:** Modificar `application/services/ingestion_service.py` +para "adaptarlo a Qdrant". Eso rompe la razón de existir del Protocol +VectorStore — si tenés que tocar application cada vez que agregás una +implementación nueva, el patrón está mal aplicado. + +**Ejemplo en el proyecto:** Mismo patrón que ya se usó al agregar +`BM25Retriever` como segunda implementación de `Retriever`. diff --git a/docs/interview_prep/by_topic/protocols.md b/docs/interview_prep/by_topic/protocols.md new file mode 100644 index 0000000..2d2b844 --- /dev/null +++ b/docs/interview_prep/by_topic/protocols.md @@ -0,0 +1,68 @@ +# Protocols — banco de preguntas + +3 preguntas. Fuente: `docs/interview_prep/bank.md`. + +#### [PR-001] Nivel: básico +**Pregunta:** ¿Qué es un `Protocol` de Python y en qué se diferencia de +una clase abstracta (`ABC`)? + +**Respuesta esperada:** Un Protocol define un contrato **estructural** — +cualquier clase con los métodos y firmas correctas lo satisface, sin +necesidad de heredar. La verificación es típicamente estática (mypy). Una +ABC define un contrato **nominal** — requiere herencia explícita +(`class Foo(BaseClass):`) y hace enforcement en runtime (`TypeError` si +un método abstracto no se implementa al instanciar). Protocol permite +que clases de librerías externas cumplan el contrato sin modificarlas; +ABC obliga a controlar el árbol de herencia. + +**Trampa común:** Tratarlos como sinónimos "más modernos" uno del otro. +Son diferentes en runtime behavior y en filosofía de diseño. + +**Ejemplo en el proyecto:** `domain/interfaces.py` define +`LLMProvider(Protocol)` con métodos `generate` y `stream`. Cualquier clase +con esos métodos lo satisface, incluyendo mocks de test que no heredan +de nada. + +--- + +#### [PR-002] Nivel: intermedio +**Pregunta:** ¿Por qué en tu proyecto usás Protocols para las abstracciones +de `application/` en lugar de clases abstractas? + +**Respuesta esperada:** Los Protocols evitan acoplar `application/` a +jerarquías de herencia. Si Anthropic saca un SDK nuevo con estructura +distinta, la nueva implementación de `LLMProvider` vive en infrastructure +sin necesitar herencia común. También facilita testing: los mocks pueden +ser clases simples sin importar nada de infrastructure. Además, structural +subtyping permite que clases de terceros cumplan el contrato sin tocar +su código. + +**Trampa común:** Pensar que "ABC hace lo mismo con más rigor". El +"rigor" adicional de ABC (enforcement al instanciar) es innecesario si +usás mypy en CI — y a cambio pagás con acoplamiento por herencia. + +**Ejemplo en el proyecto:** `application/services/rag_service.py` recibe +`llm: LLMProvider` y `store: VectorStore` — no sabe si son AnthropicLLM, +GeminiLLM, ChromaVectorStore o mocks; solo sabe qué métodos puede llamar. + +--- + +#### [PR-003] Nivel: avanzado +**Pregunta:** ¿Cómo se testea código de `application/` que depende de un +Protocol, sin usar implementaciones reales de infrastructure? + +**Respuesta esperada:** Se crea una clase mock local (en el test o en +`conftest.py`) que implementa los métodos del Protocol con comportamiento +controlado. Como Protocol es structural, no hace falta heredar de nada — +solo tener los métodos con las firmas correctas. Ejemplo: `MockVectorStore` +en `tests/conftest.py` con `search` y `upsert` que operan sobre una lista +en memoria. Los tests de `application/` inyectan estos mocks y verifican +comportamiento sin tocar Chroma real. Alternativa débil: `unittest.mock. +MagicMock` — funciona pero pierde chequeo estático (podés llamar métodos +que el Protocol no declara y no te avisa). + +**Trampa común:** Usar `MagicMock` universalmente por comodidad. Perdés la +señal de mypy sobre si tu test está usando el Protocol correctamente. + +**Ejemplo en el proyecto:** `tests/conftest.py` tiene `MockVectorStore` y +mocks de LLMProvider usados en `tests/unit/application/`. diff --git a/docs/interview_prep/by_topic/rag_retrieval.md b/docs/interview_prep/by_topic/rag_retrieval.md new file mode 100644 index 0000000..51888b1 --- /dev/null +++ b/docs/interview_prep/by_topic/rag_retrieval.md @@ -0,0 +1,100 @@ +# RAG y retrieval — banco de preguntas + +4 preguntas. Fuente: `docs/interview_prep/bank.md`. + +#### [RG-001] Nivel: básico +**Pregunta:** Dame un ejemplo concreto de una query donde BM25 supera a +la búsqueda vectorial, y otro donde vectorial supera a BM25. Explicá por qué. + +**Respuesta esperada:** BM25 supera cuando la query contiene siglas, +nombres propios técnicos o términos raros con coincidencia exacta: por +ejemplo `"BERT vs GPT-3"` — un vectorizador puede diluir esas siglas en un +embedding genérico, pero BM25 encuentra ocurrencias exactas. Vectorial +supera cuando la query es semántica y el corpus usa vocabulario distinto: +por ejemplo `"papers on models that reason step by step"` — encuentra +papers de "chain-of-thought" aunque la query no contenga esas palabras. +Son complementarios: por eso hybrid search existe. + +**Trampa común:** Pensar que uno es "mejor" en absoluto. Cada uno tiene +un tipo de query donde brilla. + +**Ejemplo en el proyecto:** `infrastructure/retrieval/bm25.py` y +`infrastructure/retrieval/chroma.py` — ambos en producción, combinados +vía `application/services/retrieval_service.py`. + +--- + +#### [RG-002] Nivel: intermedio +**Pregunta:** Explicá Reciprocal Rank Fusion (RRF) paso a paso. ¿Por qué +suma las contribuciones cuando un documento aparece en dos rankings, en +lugar de promediarlas o quedarse con la mayor? + +**Respuesta esperada:** RRF recibe rankings (no scores) de N retrievers, +cada uno con sus top-K candidatos. Descarta los scores originales — trabaja +solo con posiciones (rank 1, 2, 3...). Por cada documento en cada ranking, +calcula contribución `1 / (rrf_k + rank)` con rank 1-indexed. Cuando un +doc aparece en múltiples rankings, **suma** las contribuciones. Ordena por +score total descendente y devuelve top-K. La suma premia el consenso: +docs que dos retrievers rankean alto suben más que docs rankeados alto +por uno solo. Promediar diluiría la señal (un rank alto se compensaría +con la ausencia); quedarse con la mayor ignoraría el consenso. + +**Trampa común:** Decir que RRF "combina scores". No combina scores — +descarta los scores originales precisamente porque tienen escalas +incomparables (BM25 sin límite superior, coseno entre -1 y 1). Combina +**posiciones**, que sí son comparables. + +**Ejemplo en el proyecto:** `application/services/retrieval_service.py` +implementa `hybrid_search` con RRF. + +--- + +#### [RG-003] Nivel: intermedio +**Pregunta:** En tu hybrid_search, cada retriever devuelve `k*2` +candidatos aunque al final devuelvas solo `k`. ¿Por qué no pedir +exactamente `k`? + +**Respuesta esperada:** Porque después de la fusión hay deduplicación y +reordenamiento. Si dos retrievers devuelven exactamente los mismos `k` +docs, después de deduplicar quedás con `k` únicos y la fusión no aportó +nada — mismo resultado que un solo retriever. Con `k*2`, tenés material +adicional: docs que aparecen en posiciones 4-6 de ambos retrievers son +consensuados aunque ninguno los rankee arriba, y RRF los promueve. En +producción el overlap entre vectorial y BM25 sobre el mismo corpus está +entre 20% y 60%, entonces `k*2` es un colchón razonable. + +**Trampa común:** Pensar que el escenario "cero overlap" invalida `k*2`. +Cierto, ahí `k=candidatos` da lo mismo — pero no sabés el overlap antes +de correr, y el default tiene que servir para el caso peor. + +**Ejemplo en el proyecto:** `hybrid_search` en `retrieval_service.py`, +parámetro `candidates_per_retriever` con default `k*2` cuando es `None`. + +--- + +#### [RG-004] Nivel: avanzado +**Pregunta:** ¿Cómo evaluarías un sistema RAG en producción sin caer en +data leakage? Mencioná qué medís, cómo obtenés ground truth, y qué hacés +con las queries que fallan. + +**Respuesta esperada:** Métricas objetivas: faithfulness (¿el LLM inventa +cosas no soportadas por los docs?), context precision (¿los docs +recuperados son relevantes?), answer relevancy (¿la respuesta contesta la +pregunta?). Todas con LLM-as-judge externo — un modelo distinto al que +genera, para evitar sesgo de auto-evaluación (ej. Gemini Flash mientras +generás con Claude). Ground truth: dos fuentes. (a) curación humana +externa — alguien que NO conoce el corpus escribe queries que representan +cómo un usuario preguntaría, y anota respuesta esperada. (b) feedback en +producción — thumbs up/down, reformulaciones (señal de que la primera +respuesta no sirvió). Queries que fallan: van a un regression dataset, +etiquetadas manualmente, corren en CI/CD, bloquean deploys que degraden +el score. + +**Trampa común:** Construir queries de test conociendo el corpus indexado. +Eso es data leakage — el sistema "acierta" porque las queries están +alineadas con el contenido, no porque sea bueno. Mide qué tan bien recuerda +lo que ya sabías que estaba, no calidad de retrieval. + +**Ejemplo en el proyecto:** El `learnings.md` del 01/06/2026 documenta +exactamente este problema al observar scores 1.000/1.000 en el eval — la +razón fue leakage por construir queries desde el corpus. diff --git a/notebooks/taller_retorno_researchos.ipynb b/notebooks/taller_retorno_researchos.ipynb index 44560e0..cca92f0 100644 --- a/notebooks/taller_retorno_researchos.ipynb +++ b/notebooks/taller_retorno_researchos.ipynb @@ -108,7 +108,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 1, "id": "721d7653", "metadata": {}, "outputs": [], @@ -149,7 +149,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 2, "id": "b0908b2a", "metadata": {}, "outputs": [], @@ -184,7 +184,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 3, "id": "e54a43b6", "metadata": {}, "outputs": [ @@ -256,7 +256,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 4, "id": "ea91bb66", "metadata": {}, "outputs": [], @@ -362,7 +362,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 5, "id": "86eb7a89", "metadata": {}, "outputs": [], @@ -411,7 +411,7 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 6, "id": "2460c382", "metadata": {}, "outputs": [ @@ -469,10 +469,16 @@ "\n", "(c) ¿Qué controla la constante `rrf_k` (típicamente 60)? Si la subo a 200, ¿qué cambia en la práctica?\n", "\n", - "*Tu respuesta:*\n", + "RRF es una metodología para convinar los rankings dados por otras estrategias de rankeo, de manera que al fusionar diferentes estrategias se le da peso a la aparición de documentos conforme su relevancia para el query dado y no se sesga el rankeo por las magnitudes que maneje cada retriever, al contrario, RRF lo que busca es reranquear basándose en las posiciones netamente y no en las magnitudes de los scores.\n", "\n", - "a. Porque los rankings pueden tener escalas diferentes lo que podría sesgar el cálculo\n", - "b. " + "a. RRF suma para premiar el consenso entre retrievers; un doc que aparece en múltiples rankings acumula más score y sube al top, que es exactamente lo que quiero de una fusión.\n", + "b. matemáticamente casi no cambia el resultado, pero es la convención estándar y facilita lectura del código\n", + "c. Con rrf_k=60, rank 1 aporta 1/61 ≈ 0.0164, rank 10 aporta 1/70 ≈ 0.0143. Diferencia: 12%.\n", + "Con rrf_k=200, rank 1 aporta 1/201 ≈ 0.00498, rank 10 aporta 1/210 ≈ 0.00476. Diferencia: 5%.\n", + "\n", + "Entonces: rrf_k alto aplasta la curva — hace que la diferencia entre estar en rank 1 y en rank 10 sea pequeña. Consecuencia práctica: con rrf_k alto, la fusión se vuelve más \"democrática\" (todos los rankings pesan casi igual, importa más aparecer que aparecer arriba). Con rrf_k bajo, premias más al top de cada retriever (la punta del ranking domina la fusión).\n", + "\n", + "En producción, 60 es el default estándar. Bajar a 20-40 ayuda si confías mucho en tus retrievers y quieres que el top-3 domine. Subir a 100+ ayuda si tus retrievers son ruidosos y prefieres promediar más." ] }, { @@ -493,7 +499,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "id": "99c92e62", "metadata": {}, "outputs": [], @@ -513,8 +519,34 @@ " k: int = 5,\n", " rrf_k: int = 60,\n", ") -> list[Document]:\n", - " # Tu código acá\n", - " pass\n" + "\n", + " if not retrievers:\n", + " raise ValueError(\"Retrievers is empty. At least one retriever is required.\")\n", + "\n", + " candidates_per_retriever = k * 2\n", + "\n", + " results = await asyncio.gather(*[r.search(query, k=candidates_per_retriever) for r in retrievers])\n", + "\n", + " rrf_scores = {}\n", + " docs_by_id = {}\n", + "\n", + " for ranking in results:\n", + " for pos, doc in enumerate(ranking, start=1):\n", + " rrf_scores[doc.doc_id] = rrf_scores.get(doc.doc_id, 0.0) + (1/(rrf_k + pos))\n", + "\n", + " docs_by_id.setdefault(doc.doc_id, doc)\n", + "\n", + " top_k = sorted(rrf_scores.items(), key=lambda item: item[1], reverse=True)[:k]\n", + "\n", + " return [\n", + " docs_by_id[doc_id].model_copy(update={\"score\": score})\n", + " for doc_id, score in top_k\n", + " ]\n", + "\n", + "\n", + "\n", + "\n", + " \n" ] }, { @@ -527,10 +559,19 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 9, "id": "b3d44080", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "5.2 ✓ orden correcto, doc2 primero, sin duplicados\n", + "5.2 ✓ retrievers vacío lanza ValueError\n" + ] + } + ], "source": [ "class _MockRetriever:\n", " def __init__(self, docs): self.docs = docs\n", @@ -564,7 +605,9 @@ " print(\"5.2 ✓ retrievers vacío lanza ValueError\")\n", "\n", "\n", - "asyncio.run(_check_hybrid())\n" + "# asyncio.run(_check_hybrid())\n", + "\n", + "await _check_hybrid()\n" ] }, { @@ -594,7 +637,94 @@ "\n", "*Tu respuesta:*\n", "\n", - "\n" + "a. def -> todo el cálculo de overlaping (cuántos caracteres traslapar) en los docs ocurre en memoria\n", + "b. async def -> Tengo que esperar que la URL responda, y como pido varios docs quiero que trabaje descargando el que más rápido vaya dadndo, no una cola en espera\n", + "c. async def -> El modelo está en local pero hago llamados a él y lo que quieor es que vaya haciendo los embeddings lo más rápido que pueda, no en una cola 1 a 1\n", + "d. async def -> Uso un servicio externo así que mando todos los mensajes que pueda y espero que el servicio los vaya gestinando" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "48a48dd6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "=== Tres tareas I/O con gather ===\n", + "[10:56:15] A arranca, va a esperar 3.0s\n", + "[10:56:15] B arranca, va a esperar 1.0s\n", + "[10:56:15] C arranca, va a esperar 3.0s\n", + "[10:56:16] B termina\n", + "[10:56:18] A termina\n", + "[10:56:18] C termina\n", + "Total: 3.00s\n", + "['resultado de A', 'resultado de B', 'resultado de C']\n", + "\n", + "=== Tres tareas CPU con gather ===\n", + "[10:56:18] X arranca cálculo\n", + "[10:56:23] X termina cálculo\n", + "[10:56:23] Y arranca cálculo\n", + "[10:56:29] Y termina cálculo\n", + "[10:56:29] Z arranca cálculo\n", + "[10:56:34] Z termina cálculo\n", + "Total: 16.46s\n" + ] + } + ], + "source": [ + "import asyncio\n", + "import time\n", + "\n", + "\n", + "async def io_bound_task(name: str, duration: float) -> str:\n", + " \"\"\"Simula una llamada HTTP: espera 'duration' segundos sin usar CPU.\"\"\"\n", + " print(f\"[{time.strftime('%H:%M:%S')}] {name} arranca, va a esperar {duration}s\")\n", + " await asyncio.sleep(duration) # ← acá el mesero le dice \"avísame en X seg\"\n", + " print(f\"[{time.strftime('%H:%M:%S')}] {name} termina\")\n", + " return f\"resultado de {name}\"\n", + "\n", + "\n", + "async def cpu_bound_task(name: str, iterations: int) -> str:\n", + " \"\"\"Simula un cálculo pesado: usa CPU 'iterations' veces sin dejar respirar.\"\"\"\n", + " print(f\"[{time.strftime('%H:%M:%S')}] {name} arranca cálculo\")\n", + " total = 0\n", + " for i in range(iterations):\n", + " total += i ** 2 # cálculo puro, no hay await\n", + " print(f\"[{time.strftime('%H:%M:%S')}] {name} termina cálculo\")\n", + " return f\"resultado de {name}\"\n", + "\n", + "\n", + "async def main_io():\n", + " \"\"\"Tres tareas I/O en paralelo: total ~2s (el máximo), no 6s.\"\"\"\n", + " print(\"=== Tres tareas I/O con gather ===\")\n", + " inicio = time.time()\n", + " resultados = await asyncio.gather(\n", + " io_bound_task(\"A\", 3.0),\n", + " io_bound_task(\"B\", 1.0),\n", + " io_bound_task(\"C\", 3.0),\n", + " )\n", + " print(f\"Total: {time.time() - inicio:.2f}s\")\n", + " print(resultados)\n", + "\n", + "\n", + "async def main_cpu():\n", + " \"\"\"Tres tareas CPU en gather: total ≈ suma, NO se paralelizan.\"\"\"\n", + " print(\"\\n=== Tres tareas CPU con gather ===\")\n", + " inicio = time.time()\n", + " resultados = await asyncio.gather(\n", + " cpu_bound_task(\"X\", 50_000_000),\n", + " cpu_bound_task(\"Y\", 50_000_000),\n", + " cpu_bound_task(\"Z\", 50_000_000),\n", + " )\n", + " print(f\"Total: {time.time() - inicio:.2f}s\")\n", + "\n", + "await main_io()\n", + "await main_cpu()\n", + "# asyncio.run(main_io())\n", + "# asyncio.run(main_cpu())" ] }, { @@ -633,11 +763,10 @@ "\n", "# Escribe la versión corregida acá\n", "async def fetch_all_fixed(urls: list[str]) -> list[str]:\n", - " # Tu código acá\n", - " pass\n", - "\n", + " async with httpx.AsyncClient() as client:\n", + " return await asyncio.gather(*[fetch(client, url) for url in urls])\n", "\n", - "# Explicación del bug (celda markdown abajo):\n" + "# Explicación del bug (celda markdown abajo)\n" ] }, { @@ -647,7 +776,9 @@ "source": [ "*Tu explicación del bug:*\n", "\n", - "\n" + "El bug no es que el for detenga el event loop. El bug es que el await dentro del for hace que cada llamada a fetch deba completarse antes de arrancar la siguiente. No hay corrutinas \"en vuelo\" simultáneamente — se lanza una, se espera, se lanza la próxima. El event loop está funcionando bien; simplemente no le dieron trabajo concurrente.\n", + "\n", + "asyncio.gather corrige esto: le pasa al event loop una lista de corrutinas para que las inicie todas en paralelo y espere a que todas terminen. Ahora sí hay múltiples HTTP requests \"en vuelo\" al mismo tiempo, y el tiempo total es aproximadamente el de la request más lenta, no la suma." ] }, { @@ -754,7 +885,7 @@ "\n", "*Tu respuesta:*\n", "\n", - "\n", + "1. en src/researchos/infrastructure/retrieval agregar un qdrant.py, luego modificar scripts/eval_retrieval.py o crear uno propio para qdrant. Modificar src/researchos/application/ingestion_service.py para que tome la nueva clase QdrantVectorStore. Agregar un test de integración para test_qdrant.py\n", "\n", "---\n", "\n", @@ -762,23 +893,21 @@ "\n", "*Tu respuesta:*\n", "\n", - "\n", - "\n", + "El código de slack va en application y debe consumir el rag_service que a su vez carga la infra del sistema rag para embeber el query, hacer la búsqueda híbrida y devolver la respuesta por el canal de Slack\n", "---\n", "\n", "**8.3** El PO de Pensiones (Paola) te pide un endpoint HTTP que reciba un caso y devuelva una recomendación. En términos de tu arquitectura, ¿en qué capa vive un endpoint HTTP? ¿Cuál es el rol de FastAPI: parte del motor o parte del canal?\n", "\n", "*Tu respuesta:*\n", "\n", - "\n", - "\n", + "El endpoint vive src/researchos/application/services, es decir, en la capa de aplicaciones. FastAPI es parte del canal ya que solo expone el resultado, no computa nada.\n", "---\n", "\n", "**8.4 — La pregunta clave.** En una entrevista de AI Engineering te preguntan: \"Explica cómo evaluarías un sistema RAG en producción, sin data leakage\". Con lo que sabes hoy, escribe una respuesta de 3-5 oraciones. Menciona: qué mides, cómo obtienes ground truth, qué haces con las queries que fallan.\n", "\n", "*Tu respuesta:*\n", "\n", - "\n" + "Lanzaría 10 queries al sistema: 5 de ellas con frases de información que sé 100% que sí están en el sistema, 3 de ellas son queries que aunque podrían estar en el sistema están escritas de manera sútil para que eventualmente el rag no devuelva nada PERO la intención real es que sí debería devolver ya que la comparación es semántica, y 2 de ellas con preguntas que en definitiva la información no está en la base de datos y que realmente no se debería hacer ninguna recuperación. Si existe data leakage debería ver qué ..." ] }, { @@ -908,7 +1037,7 @@ ], "metadata": { "kernelspec": { - "display_name": "researchos (3.11.8)", + "display_name": "researchos (3.11.8.final.0)", "language": "python", "name": "python3" }, From 10bd08f77962a907b8e48160a4d8ae16174d5a31 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Wed, 29 Jul 2026 14:19:40 -0500 Subject: [PATCH 51/55] chore(docs): add missing weekly_drafts and essays/prompts directories MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Scaffolding for the concepts-tutor system was incomplete — bank generation and essay proposal modes need these dirs to exist before writing drafts. --- docs/essays/prompts/.gitkeep | 0 docs/interview_prep/weekly_drafts/.gitkeep | 0 2 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 docs/essays/prompts/.gitkeep create mode 100644 docs/interview_prep/weekly_drafts/.gitkeep diff --git a/docs/essays/prompts/.gitkeep b/docs/essays/prompts/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/docs/interview_prep/weekly_drafts/.gitkeep b/docs/interview_prep/weekly_drafts/.gitkeep new file mode 100644 index 0000000..e69de29 From 18279f69fdf897c314b7fd1b1099d906db711a2e Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Wed, 29 Jul 2026 15:31:40 -0500 Subject: [PATCH 52/55] docs: close out 2026-07-29 session (learnings + work_log) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Corrects three conceptual errors written in today's learnings.md entry (infrastructure/application dependency direction, business logic location, agent composition wording) and tracks them in "¿Qué no entendí bien?" for follow-up in future sessions. Reconciles the work_log.md draft with the day's actual commits. --- docs/learnings.md | 25 ++++++++++++++++++++++++- docs/work_log.md | 28 ++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+), 1 deletion(-) diff --git a/docs/learnings.md b/docs/learnings.md index e4fc37e..9a1be36 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -224,6 +224,8 @@ Regla simple para ResearchOS: - El reporte de un bug en `overlap_chunking` era incorrecto: la lógica `start = i * (chunk_size - overlap)` produce un paso fijo, no un overlap acumulativo. Verificar con math antes de reportar un bug. - `isinstance` no puede verificar tipos genéricos como `tuple[str, Path]` en runtime — usar assertions separadas por elemento. +--- + **Fecha:** 01/06/2026 ### ¿Qué aprendí? @@ -243,4 +245,25 @@ Regla simple para ResearchOS: - Las lambdas en un dict de estrategias capturan variables del scope exterior por referencia — en este caso no fue problema, pero es un antipatrón a tener en mente si las variables cambian en el loop. --- - + +**Fecha:** 29/07/2026 + +### ¿Qué aprendí? +- Se repasaron conceptos de Clean Architecture: application se codifica contra domain; infrastructure también se codifica contra domain (para implementar sus Protocols) e infrastructure sí importa de application — por ejemplo, un router o un bot llaman a un service como `rag_service.answer_query`. Lo que nunca ocurre es que domain importe de alguien, o que application importe de infrastructure. +- Se refuerza la idea de sumar contribuciones en Reciprocal Rank Fusion (RRF) porque así se premia el consenso entre retrievers. Si se promediara, un rank muy alto se compensaría con la ausencia en otro y se perdería la señal de que 2 o más retrievers coincidieron. -> Aún falta reforzar e interiorizar más este concepto +- las funciones asíncronas existen para permitir que el event loop pueda ejecutar otras corrutinas mientras se realizan operaciones de I/O, por ejemplo cuando se hace una petición HTTP. Es diferente a trabajo paralelo porque ese trabajo sí requiere uso de CPU. La analogía es un mesero que atiende muchas mesas: en lugar de quedarse esperando, pide al chef que prepare un plato y mientras tanto va y toma el pedido en otra mesa o la limpia. +- Los agentes importan funciones de `agent_utils.py` (composición) en lugar de heredar de una clase base, porque así, si se modifica un método de la clase base, todos los agentes heredarían el cambio (y un posible error), mientras que con composición solo se ve afectado el agente que efectivamente importa esa función. Es menos elegante pero más aislado. + +### ¿Qué no entendí bien? +- Me hicieron preguntas sobre cómo evaluar un RAG. Entendí que la manera es con métricas claras y un ground truth que viene de dos fuentes: (a) una entidad externa que no conoce el contenido de la BD vectorial y por tanto no está sesgada, y (b) retroalimentación de producción señalando si una recuperación fue buena o mala. Las queries que fallan no son una tercera fuente de verdad — se etiquetan y se mandan a un dataset de regresión para monitoreo futuro, eso es manejo de fallos, no una fuente adicional de ground truth. No tengo claras las métricas concretas (formulación matemática, en qué se sustentan) ni si existen herramientas de evaluación automática de RAG en el mercado o solo metodologías. +- **[corregido hoy]** Dirección de dependencia infrastructure↔application: escribí dos veces en esta misma entrada que "infrastructure nunca importa de application" — es al revés, infrastructure sí importa de application (routers/bots llaman a services). Es la misma confusión de dirección de dependencias ya fichada como hueco recurrente; revisar en próximas sesiones si ya quedó interiorizada. +- **[corregido hoy]** Ubicación de la lógica de negocio: escribí que infrastructure incluye "lógica propia del negocio" — no es así, la lógica de negocio vive en domain/application; infrastructure es solo detalle técnico (SDKs, llamados a APIs, drivers). +- **[corregido hoy]** Terminología de composición de agentes: escribí "los agentes exportan de agent_utils.py" — es al revés, `agent_utils.py` exporta funciones y los agentes las importan/consumen. + +### Decisiones de diseño +- Se decide modificar el esquema de trabajo: con 5h/semana se van a emplear cerca de 4 en el desarrollo de código y 1h en la revisión conceptual: llenado de un banco de preguntas y revisión de un ensayo sobre un tema específico + +### Errores interesantes +- Confundí la dirección de dependencia entre infrastructure y application: dije que "infrastructure nunca importa de application", cuando es lo opuesto — infrastructure sí importa de application (ej. un router o un bot llaman a un service). Lo que nunca ocurre es que domain importe de alguien o que application importe de infrastructure. Application se encarga de orquestar contra Protocols de domain; infrastructure implementa detalle técnico (SDKs, llamados a APIs, drivers) sin lógica de negocio; domain establece el contrato (requisitos de modelos y Protocols). + +--- diff --git a/docs/work_log.md b/docs/work_log.md index d36339d..b03a1e5 100644 --- a/docs/work_log.md +++ b/docs/work_log.md @@ -139,3 +139,31 @@ - Evaluar con queries reales para obtener métricas más representativas. --- + +## 2026-07-29 + +### Trabajo desarrollado +- Retorno al proyecto tras pausa de ~8 semanas; taller de retorno a ResearchOS + completado (~60%, sección 7 saltada por decisión consciente) +- Sistema de estudio conceptual implementado: skills `interview-bank`, + `weekly-essay`, `git-commits`, `arquitectura-drawio` y `daily-closeout` + agregadas en `.claude/skills/` +- Banco de preguntas poblado (`docs/interview_prep/bank.md`) con 15 preguntas + semilla derivadas del taller — Clean Architecture (5), Protocols (3), RAG y + retrieval (4), Async (3) — con archivos por tema regenerados en `by_topic/` +- Estructura de `docs/interview_prep/weekly_drafts/` y `docs/essays/prompts/` + completada +- Revisión de cierre de jornada sobre `docs/learnings.md`: corregidos tres + errores conceptuales en la entrada de hoy (dirección de dependencia + infrastructure↔application, ubicación de la lógica de negocio, terminología + de composición de agentes), dejando registro en "¿Qué no entendí bien?" + para monitorear en próximas sesiones + +### Próximos pasos +- Jueves 30/07: refuerzo arquitectural (diagramas de flujo + reescritura + de respuestas 8.1–8.4 del taller) +- Viernes 31/07: primer ciclo real del sistema (banco semanal + ensayo) +- Verificar en próximas sesiones si los tres conceptos corregidos hoy en + learnings.md ya quedaron interiorizados + +--- From 4e13ee3a1e30b8bfab05ee5537570db84f7237bc Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Wed, 29 Jul 2026 15:31:50 -0500 Subject: [PATCH 53/55] chore(skills): add daily-closeout skill Portable end-of-day skill: reviews the latest learnings.md entry and synthesizes work_log.md from actual git history, so day closeouts stay disciplined and traceable across projects. --- .claude/skills/daily-closeout/SKILL.md | 218 +++++++++++++++++++++++++ 1 file changed, 218 insertions(+) create mode 100644 .claude/skills/daily-closeout/SKILL.md diff --git a/.claude/skills/daily-closeout/SKILL.md b/.claude/skills/daily-closeout/SKILL.md new file mode 100644 index 0000000..cfc9591 --- /dev/null +++ b/.claude/skills/daily-closeout/SKILL.md @@ -0,0 +1,218 @@ +--- +name: daily-closeout +description: Cerrar la jornada de trabajo — revisar la última entrada de learnings.md por errores conceptuales o de redacción, generar entrada nueva en work_log.md basada en commits reales y archivos modificados del día. Usar cuando el usuario indique "cerremos la jornada", "he finalizado el día", "fin de jornada" o variantes similares. +--- + +# Cierre de jornada + +Skill genérica para cerrar el día de trabajo de forma disciplinada en cualquier +proyecto de código: revisar el último aprendizaje registrado, sintetizar el +work_log desde el trabajo real (git), y dejar todo commiteado. Portable entre +proyectos — no depende de la temática específica del repo. + +## Archivos involucrados + +- `docs/work_log.md` — bitácora diaria del proyecto (obligatorio) +- `docs/learnings.md` — registro de aprendizajes (opcional; si no existe, se + saltea la revisión conceptual) + +Si estos archivos viven en otras rutas del proyecto (ej. `WORK_LOG.md` en la +raíz), leé el `CLAUDE.md` del repo para detectar la ubicación correcta antes +de operar. + +## Comando del usuario + +| Comando (aproximado) | Acción del agente | +|---|---| +| "Cerremos la jornada" / "he finalizado el día" / "fin de jornada" | Ejecutá el flujo completo | +| "Cierre sin revisar learnings" | Salteás el paso 3 (útil si no hubo aprendizajes registrados hoy) | + +## Flujo + +### Paso 1 — Precondiciones + +1. Verificá que existe `docs/work_log.md`. Si no existe, ofrecé crearlo con el + formato base (ver sección "Formato de work_log"). Esperá confirmación. +2. Verificá si existe `docs/learnings.md`. Si no existe, marcá que la revisión + conceptual del paso 3 no aplica y continuá. +3. Ejecutá `git status`. Si hay cambios sin commitear, avisá al usuario y + preguntá si quiere commitearlos antes de cerrar la jornada. No cerrás con + trabajo sin commitear — se pierde la trazabilidad. + +### Paso 2 — Recolección de contexto del día + +Ejecutá en orden y consolidá los resultados: + +1. `git log --since='midnight' --pretty=format:'%h %s%n%b' --stat` — commits + del día con sus estadísticas de cambios. +2. `git log --since='midnight' --name-only --pretty=format:''` — lista de + archivos modificados hoy. +3. Leé las últimas 2–3 entradas de `docs/work_log.md` para conocer el formato + exacto que usa este proyecto (fecha, secciones, viñetas, etc.). +4. Si el proyecto tiene `CLAUDE.md`, leélo para entender contexto general + (dominio, convenciones, roles) que ayude a interpretar los cambios. + +**Regla dura:** si no hay commits del día, avisá al usuario. Un día sin commits +puede ser: (a) legítimo — día de investigación, lectura o reuniones; (b) señal +de problema — trabajo local no commiteado. En cualquier caso, preguntá antes +de escribir en el work_log. Si el día fue legítimamente no-código, el usuario +te va a dar contexto para escribir la entrada. + +### Paso 3 — Revisión de learnings.md (si existe) + +Leé la última entrada de `docs/learnings.md` (la que corresponde a hoy o a la +sesión más reciente). Verificá tres cosas: + +1. **Redacción**: frases confusas, errores ortográficos evidentes, párrafos + que no fluyen. No corregís estilo personal ni preferencias del autor — + solo problemas objetivos de claridad. +2. **Conceptos**: afirmaciones técnicas dudosas o incorrectas. Cruzá contra + tu conocimiento general y contra el código real del repo cuando aplique. + Ejemplos: si el usuario afirma "usé RRF con rank 0-indexed", verificá en + el código. Si dice "async paraleliza CPU en Python", es incorrecto y hay + que señalarlo. +3. **Coherencia**: contradicciones internas en la entrada, o con entradas + previas cercanas. + +**Reporte al usuario** con formato claro: + +``` +Revisión de learnings.md — entrada del {fecha}: + +Redacción: +- {hallazgo 1 o "sin observaciones"} + +Conceptos: +- {hallazgo 1 con cita textual y corrección propuesta, o "sin observaciones"} + +Coherencia: +- {hallazgo 1 o "sin observaciones"} +``` + +**Regla dura:** no editás `learnings.md` automáticamente. El usuario decide +qué corregir y lo hace él mismo. Vos solo señalás. + +Si el usuario pide que hagas la corrección después de leer tu reporte, +entonces sí modificás el archivo — pero solo bajo instrucción explícita. + +### Paso 4 — Propuesta de entrada en work_log.md + +Construí la entrada basándote **exclusivamente** en el contexto recolectado en +el paso 2 (commits reales y archivos modificados). No inventés trabajo que no +está en git. + +La entrada debe tener dos secciones mínimas: + +- **Trabajo desarrollado**: 3–7 viñetas concisas describiendo qué se hizo hoy. + Cada viñeta refleja uno o varios commits agrupados por tema lógico. Referí + archivos específicos cuando ayude a la trazabilidad (ej. + `application/services/retrieval_service.py`). +- **Próximos pasos**: 2–4 viñetas con lo que sigue. Si el usuario mencionó + explícitamente el plan del día siguiente, usalo. Si no, inferí del contexto + (tareas pendientes visibles en el código, TODOs, roadmap del proyecto). + +Otras secciones opcionales, agregar solo si aplican: + +- **Bloqueos**: si detectás en los commits o mensajes que hay algo pendiente + de decisión externa (input de stakeholder, respuesta de compañero, etc.). +- **Análisis de resultados**: si el día incluyó evaluaciones, tests, o + benchmarks con métricas. + +Respetá el formato exacto del work_log existente. Si el proyecto usa `###` +para subsecciones, usá `###`. Si separa entradas con `---`, respetá el +separador. Si los archivos van en backticks, mantené backticks. + +### Paso 5 — Confirmación + +Mostrá la entrada propuesta al usuario con el prefijo: + +``` +Propuesta de entrada para work_log.md (fecha {DD-MM-YYYY}): + +--- +{contenido propuesto} +--- + +¿La escribo tal cual, la ajusto, o querés cambiarla? +``` + +Esperá confirmación explícita. Aceptá pedidos de ajuste (agregar detalle, +recortar, reformular). No escribís en el archivo hasta tener OK claro. + +### Paso 6 — Escritura y commit + +1. Insertá la entrada nueva en `docs/work_log.md` al final del archivo, + respetando el formato de separadores. +2. Preparás el mensaje de commit. + - Si el repo tiene `.claude/skills/git-commits/SKILL.md`, seguí sus + convenciones estrictamente. + - Si no la tiene, usá Conventional Commits: `docs(work-log): update for + YYYY-MM-DD`. +3. **No hacés `git commit` sin confirmación explícita del usuario.** Mostrás + el comando propuesto y esperás. +4. No hacés `git push` — eso queda a criterio del usuario. + +## Formato de work_log (si el archivo no existe) + +Formato base a proponer al usuario si `docs/work_log.md` no existe todavía: + +```markdown +# Work Log + +Bitácora diaria de trabajo del proyecto {nombre}. + +--- + +## YYYY-MM-DD + +### Trabajo desarrollado +- {item 1} +- {item 2} + +### Próximos pasos +- {item 1} +- {item 2} + +--- +``` + +Después de la primera entrada, respetás el formato que el usuario haya +adoptado. + +## Anti-patrones + +- **No inventar trabajo**. Si no está en los commits, no está en el work_log. + Si el usuario hizo algo que no commiteó todavía (ej. lectura, diseño en + papel), pediselo verbalmente y anotálo — pero identificándolo como aporte + del usuario, no como inferencia tuya. +- **No corregir learnings.md automáticamente**. El aprendizaje pertenece al + autor; tu rol es señalar, no editar. +- **No cerrar jornada con trabajo sin commitear**. Rompe la trazabilidad y + el próximo cierre no va a poder reconstruir qué se hizo hoy vs mañana. +- **No inflar la entrada con detalles obvios**. `chore(deps): bump pydantic` + no necesita tres viñetas de explicación. Una línea alcanza. La densidad + de información importa más que la cantidad. +- **No incluir información sensible innecesaria**. En proyectos corporativos, + evitá nombres completos de clientes, cédulas, saldos, o cualquier dato + personal que aparezca en logs o comments. Referí a personas por rol o + nombre de pila si es imprescindible. +- **No mezclar work_log con learnings**. Work_log es "qué hice"; learnings + es "qué aprendí". Si el usuario escribió reflexiones en el work_log, no + las muevas — solo señalá en el próximo cierre que ese contenido va mejor + en learnings. + +## Portabilidad entre proyectos + +Esta skill funciona en cualquier repo con git y con al menos un +`docs/work_log.md`. Para instalarla en otro proyecto: + +1. Copiá el archivo `.claude/skills/daily-closeout/SKILL.md` al nuevo repo. +2. Verificá si el nuevo proyecto tiene `docs/work_log.md` — si no, la skill + ofrecerá crearlo en el primer uso. +3. Opcional: si el nuevo proyecto tiene convenciones de commit distintas, + asegurate de tener también `.claude/skills/git-commits/SKILL.md` con esas + convenciones. Sin esa skill, `daily-closeout` cae al default Conventional + Commits. + +La skill no depende del tema del proyecto (RAG, clasificación, agentes, +MLOps, análisis estadístico). Depende solo de git y del formato de work_log. From c27ae20244cc005f48fa690650efea526fb2cc22 Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 10 Aug 2026 13:43:29 -0500 Subject: [PATCH 54/55] feat(bot): add Telegram bot adapter wired to vector-only RAG TelegramBot depends only on AnswerFn (Callable[[str], Awaitable[str]]), never on LLMProvider/VectorStore/AnthropicLLM/ChromaVectorStore. Wiring of concrete infrastructure lives in the composition root (scripts/run_telegram_bot.py) via a closure, keeping the adapter testable and channel-agnostic. --- scripts/run_telegram_bot.py | 24 ++++++++++++++ .../infrastructure/bot/telegram_bot.py | 31 +++++++++++++++++++ 2 files changed, 55 insertions(+) create mode 100644 scripts/run_telegram_bot.py create mode 100644 src/researchos/infrastructure/bot/telegram_bot.py diff --git a/scripts/run_telegram_bot.py b/scripts/run_telegram_bot.py new file mode 100644 index 0000000..05c9c63 --- /dev/null +++ b/scripts/run_telegram_bot.py @@ -0,0 +1,24 @@ +import sys + +if sys.platform == "linux": + __import__("pysqlite3") + sys.modules["sqlite3"] = sys.modules.pop("pysqlite3") + +from researchos.application.services.rag_service import answer_query +from researchos.config import settings +from researchos.infrastructure.bot.telegram_bot import TelegramBot +from researchos.infrastructure.llm.anthropic_llm import AnthropicLLM +from researchos.infrastructure.retrieval.chroma import ChromaVectorStore +from researchos.infrastructure.retrieval.embedder import LocalEmbedder + +embedder = LocalEmbedder() +chroma = ChromaVectorStore(embedder=embedder, collection_name="papers") +llm = AnthropicLLM() + + +async def answer(query: str) -> str: + return await answer_query(query, llm=llm, store=chroma) + + +bot = TelegramBot(token=settings.telegram_bot_token, answer_fn=answer) +bot.run() diff --git a/src/researchos/infrastructure/bot/telegram_bot.py b/src/researchos/infrastructure/bot/telegram_bot.py new file mode 100644 index 0000000..32bba6e --- /dev/null +++ b/src/researchos/infrastructure/bot/telegram_bot.py @@ -0,0 +1,31 @@ +"""Telegram bot adapter — delivery channel for the RAG engine.""" + +import logging +from collections.abc import Awaitable, Callable + +from telegram import Update +from telegram.ext import ApplicationBuilder, ContextTypes, MessageHandler, filters + +logger = logging.getLogger(__name__) + +AnswerFn = Callable[[str], Awaitable[str]] + + +class TelegramBot: + def __init__(self, token: str, answer_fn: AnswerFn) -> None: + self.token_telegram = token + self.answer_fn = answer_fn + + async def _handle_message(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None: + raw_text = update.message.text + + logger.info("Mensaje recibido: %s", raw_text[:80]) + + answer_llm = await self.answer_fn(raw_text) + await update.message.reply_text(answer_llm) + + def run(self) -> None: + print(self.token_telegram) + app = ApplicationBuilder().token(self.token_telegram).build() + app.add_handler(MessageHandler(filters.TEXT & ~filters.COMMAND, self._handle_message)) + app.run_polling() From f6884629e4a738f8091db44b15e37601bad9e4ce Mon Sep 17 00:00:00 2001 From: John Mario Montoya Zapata Date: Mon, 10 Aug 2026 13:43:40 -0500 Subject: [PATCH 55/55] docs: close out 2026-08-10 session (learnings + work_log) Adds today's learnings entry on AnswerFn as a lightweight function-type contract for the Telegram adapter, and reconciles work_log.md with the actual bot implementation committed today. --- docs/learnings.md | 35 +++++++++++++++++++++++++++++++++++ docs/work_log.md | 26 ++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/docs/learnings.md b/docs/learnings.md index 9a1be36..bab8049 100644 --- a/docs/learnings.md +++ b/docs/learnings.md @@ -267,3 +267,38 @@ Regla simple para ResearchOS: - Confundí la dirección de dependencia entre infrastructure y application: dije que "infrastructure nunca importa de application", cuando es lo opuesto — infrastructure sí importa de application (ej. un router o un bot llaman a un service). Lo que nunca ocurre es que domain importe de alguien o que application importe de infrastructure. Application se encarga de orquestar contra Protocols de domain; infrastructure implementa detalle técnico (SDKs, llamados a APIs, drivers) sin lógica de negocio; domain establece el contrato (requisitos de modelos y Protocols). --- + +**Fecha:** 10/08/2026 + +### ¿Qué aprendí? + +- **`AnswerFn = Callable[[str], Awaitable[str]]` es un alias de tipo que declara un contrato mínimo de comportamiento.** Se lee: "una función que recibe un `str` y devuelve algo que, al esperarlo, produce un `str`". Es el contrato completo que el bot de Telegram necesita conocer del motor RAG: nada más. + +- **Por qué `Awaitable[str]` y no `str`.** Cuando escribo `async def answer(query: str) -> str`, la anotación `-> str` describe lo que la corrutina *resuelve*, no lo que la llamada *retorna*. Llamar `answer("hola")` sin `await` devuelve una corrutina, no un string. Por eso, visto como valor de primera clase, el tipo de esa función es `Callable[[str], Awaitable[str]]`. Si escribiera `Callable[[str], str]`, estaría describiendo una función sincrónica y mypy me marcaría el error al inyectar la async. + +- **Un alias de función es un contrato más liviano que un Protocol.** Un Protocol declara varios métodos con nombre; `AnswerFn` declara un solo comportamiento anónimo: un parámetro, un retorno. Regla que me llevo: cuando lo que inyecto es *un solo comportamiento*, alcanza un tipo de función; cuando son *varios comportamientos relacionados que comparten estado*, ahí sí conviene un Protocol o una clase. Envolver una sola función en una clase es ceremonia sin beneficio. + +- **`AnswerFn` es el mecanismo concreto que hace al bot agnóstico al motor.** El bot no importa `LLMProvider`, ni `VectorStore`, ni `AnthropicLLM`, ni `ChromaVectorStore`. Solo sabe que le dieron algo llamable con esa firma. Mañana puedo cambiar el motor de vector-only a hybrid+rerank, o cambiar Chroma por Qdrant, y `telegram.py` no se entera. El mismo `AnswerFn` va a servir para Slack, para un router de FastAPI y para un CLI. + +- **El closure es lo que llena el contrato.** En `scripts/run_telegram_bot.py` defino `async def answer(query: str) -> str` que captura `llm` y `chroma` del scope exterior. Su firma resultante es exactamente `AnswerFn`. Las dependencias concretas quedan atrapadas en el closure y nunca cruzan la frontera hacia el bot. + +- **El wiring pertenece al composition root, no al adapter.** Si `telegram.py` instanciara `ChromaVectorStore`, no sería una violación de capas (ambos son `infrastructure/`), pero rompería tres cosas: el bot quedaría intesteable sin un Chroma real, agregar Slack duplicaría el wiring, y cambiar la composición del motor obligaría a editar cada canal. + +### ¿Qué no entendí bien? + +- El bot responde bien la primera pregunta y falla en la de seguimiento ("cuál es el mecanismo"). Identifiqué que son **dos** problemas distintos, no uno: (1) no hay memoria conversacional — `answer_query` no recibe historial ni `session_id`, cada mensaje es independiente; (2) aunque hubiera memoria, el retrieval seguiría fallando, porque se hace con la query cruda y "cuál es el mecanismo" es anafórica: su embedding no se parece a ningún chunk. Agregar memoria no arregla retrieval. Falta entender bien cómo se implementa el query rewriting y en qué punto del grafo va. + +### Decisiones de diseño + +- El bot recibe `answer_fn` inyectada en el constructor, no instancia infraestructura. `TelegramBot(token, answer_fn)`. +- `run()` es `def` normal, no `async def`: `app.run_polling()` gestiona su propio event loop, así que el script de arranque es sincrónico de punta a punta. Llamarlo desde `asyncio.run` produciría un conflicto de loops. +- Se descartó usar `context.user_data` de `python-telegram-bot` para guardar historial. Es memoria en RAM que se pierde al reiniciar, no se comparte entre canales, y pondría estado del motor en el adaptador. La memoria pertenece al motor vía el Protocol `MemoryStore`. +- Alcance deliberado: el bot conecta `answer_query` (vector-only). Conectar hybrid search es un cambio en `application/`, no en el bot. + +### Errores interesantes + +- Escribí `self.answer_fn: answer_fn` en vez de `self.answer_fn = answer_fn`. Con dos puntos, Python lo lee como anotación de tipo, no como asignación: es sintácticamente válido, la clase se define sin error, pero **el atributo nunca se crea**. Habría explotado con `AttributeError` en el primer mensaje. Segundo error mecánico de este tipo en dos semanas (el anterior fue omitir `self` en firmas de Protocol) — es un hueco de escritura de Python, no conceptual. +- Usé un f-string innecesario en `.token(f"{self.token}")` cuando `self.token` ya es `str`. +- Guardé el retorno de `await update.message.reply_text(...)` en una variable sin usar. + +--- diff --git a/docs/work_log.md b/docs/work_log.md index b03a1e5..ce869a6 100644 --- a/docs/work_log.md +++ b/docs/work_log.md @@ -167,3 +167,29 @@ learnings.md ya quedaron interiorizados --- + +## 2026-08-10 + +### Trabajo desarrollado +- Implementado el adapter `TelegramBot` en `infrastructure/bot/telegram_bot.py`: + recibe `token` y una función `answer_fn: AnswerFn` + (`Callable[[str], Awaitable[str]]`) inyectada por constructor, sin conocer + `LLMProvider`, `VectorStore`, `AnthropicLLM` ni `ChromaVectorStore` +- Composition root en `scripts/run_telegram_bot.py`: instancia `LocalEmbedder`, + `ChromaVectorStore`, `AnthropicLLM` y arma un closure que satisface `AnswerFn` + llamando a `rag_service.answer_query` (vector-only) +- Detectado en pruebas manuales: el bot falla en preguntas de seguimiento + anafóricas — dos causas distintas identificadas: falta de memoria + conversacional (`answer_query` no recibe historial/`session_id`) y falta + de query rewriting antes del retrieval + +### Próximos pasos +- Diseñar memoria conversacional vía el Protocol `MemoryStore` e inyectarla + en `answer_query` +- Investigar query rewriting para resolver referencias anafóricas antes del + retrieval +- Evaluar si conectar hybrid search al canal de Telegram (hoy es vector-only) +- Manejar el límite de 4096 caracteres por mensaje de Telegram — el bot hoy + no trunca ni divide respuestas largas antes de `reply_text` + +---