diff --git a/python-recipes/gemini_ai_demo/gemini_search_enrichment.ipynb b/python-recipes/gemini_ai_demo/gemini_search_enrichment.ipynb index 4a9760e..8565eaf 100644 --- a/python-recipes/gemini_ai_demo/gemini_search_enrichment.ipynb +++ b/python-recipes/gemini_ai_demo/gemini_search_enrichment.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "29e31474", + "id": "de627238", "metadata": {}, "source": [ "# Grounded data enrichment with the Gemini API and Parallel Web Search\n", @@ -16,47 +16,37 @@ "- Parallel authentication, via either:\n", " - a **Parallel API key** from [platform.parallel.ai](https://platform.parallel.ai) (Bring Your Own Key), or\n", " - an active [Parallel Web Search subscription on the Google Cloud Marketplace](https://console.cloud.google.com/marketplace/product/parallel-web-systems-public/parallel-web-systems) for your project — in that case no API key is needed.\n", - "- The `gemini_parallel` package from this directory (installed in the next cell), which brings `pydantic` with it.\n", + "- Two pip packages, installed in the next cell: `google-genai` (the Gemini SDK) and `pydantic`.\n", "\n", - "> Grounding with Parallel on Gemini Enterprise is currently in **Preview**. See the [Parallel integration docs](https://docs.parallel.ai/integrations/google-gemini-enterprise) and [Google's grounding documentation](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/grounding/grounding-with-parallel) for the current list of supported models.\n", "\n", - "The saved outputs below were generated on July 5, 2026. Because they use the live web, rerunning the notebook may return different sources and answers." + "The saved outputs below were generated on July 14, 2026. Because they use the live web, rerunning the notebook may return different sources and answers." ] }, { "cell_type": "markdown", - "id": "f3b07642", + "id": "a8b7333a", "metadata": {}, "source": [ "## 1. Set up Gemini with Parallel grounding\n", "\n", - "### 1.1 Install the `gemini_parallel` client\n", + "### 1.1 Install dependencies\n", "\n", - "The client lives in this directory (`src/gemini_parallel/`); installing it in editable mode also pulls in `pydantic`, which defines the typed output contracts in sections 2 and 3." + "Parallel grounding is available natively in the official Gemini Python SDK, [`google-genai`](https://googleapis.github.io/python-genai/), as the `parallel_ai_search` tool — no raw REST calls needed. `pydantic` defines the typed output contracts in sections 2–4." ] }, { "cell_type": "code", "execution_count": 1, - "id": "8793572e", + "id": "65f11f3b", "metadata": { "execution": { - "iopub.execute_input": "2026-07-07T02:07:48.525326Z", - "iopub.status.busy": "2026-07-07T02:07:48.525263Z", - "iopub.status.idle": "2026-07-07T02:07:50.436073Z", - "shell.execute_reply": "2026-07-07T02:07:50.435602Z" + "iopub.execute_input": "2026-07-14T21:29:58.128434Z", + "iopub.status.busy": "2026-07-14T21:29:58.128377Z", + "iopub.status.idle": "2026-07-14T21:29:59.477081Z", + "shell.execute_reply": "2026-07-14T21:29:59.476600Z" } }, "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\r\n", - "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m26.1.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m26.1.2\u001b[0m\r\n", - "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49m/Users/ruthvikmukkamala/parallel-cookbook/.scratch/venv/bin/python3.14 -m pip install --upgrade pip\u001b[0m\r\n" - ] - }, { "name": "stdout", "output_type": "stream", @@ -66,17 +56,17 @@ } ], "source": [ - "%pip install --quiet -e ." + "%pip install --quiet --upgrade google-genai pydantic" ] }, { "cell_type": "markdown", - "id": "3eb01d31", + "id": "e4d69c87", "metadata": {}, "source": [ "### 1.2 Configure credentials and create the client\n", "\n", - "`GroundedGeminiClient` resolves Google Cloud auth through application default credentials and caches the OAuth token between requests. Three values configure it:\n", + "The SDK client targets the Gemini API on Google Cloud (`vertexai=True`) and resolves auth through application default credentials. Three values configure it:\n", "\n", "- **Project** — read from `GOOGLE_CLOUD_PROJECT` (must have the Gemini API enabled).\n", "- **Parallel API key** — read from `PARALLEL_API_KEY` for BYOK auth. Leave it unset if your project has a Google Cloud Marketplace subscription; if both are present, the API key takes precedence.\n", @@ -88,13 +78,13 @@ { "cell_type": "code", "execution_count": 2, - "id": "4276ca31", + "id": "1467d5e6", "metadata": { "execution": { - "iopub.execute_input": "2026-07-07T02:07:50.437204Z", - "iopub.status.busy": "2026-07-07T02:07:50.437118Z", - "iopub.status.idle": "2026-07-07T02:07:51.210734Z", - "shell.execute_reply": "2026-07-07T02:07:51.210225Z" + "iopub.execute_input": "2026-07-14T21:29:59.478156Z", + "iopub.status.busy": "2026-07-14T21:29:59.478089Z", + "iopub.status.idle": "2026-07-14T21:29:59.951442Z", + "shell.execute_reply": "2026-07-14T21:29:59.951018Z" } }, "outputs": [ @@ -103,7 +93,7 @@ "output_type": "stream", "text": [ "model : gemini-3.5-flash\n", - "endpoint : https://aiplatform.googleapis.com/v1/projects/your-gcp-project-id/locations/global/publishers/google/models/gemini-3.5-flash:generateContent\n" + "location : global\n" ] } ], @@ -112,7 +102,8 @@ "import os\n", "from getpass import getpass\n", "\n", - "from gemini_parallel import GroundedGeminiClient\n", + "from google import genai\n", + "from google.genai import types\n", "\n", "PROJECT_ID = os.environ.get(\"GOOGLE_CLOUD_PROJECT\")\n", "LOCATION = os.environ.get(\"GOOGLE_CLOUD_LOCATION\", \"global\")\n", @@ -121,61 +112,101 @@ " \"PARALLEL_API_KEY\"\n", ") or getpass(\"Parallel API key (leave blank if using a Marketplace subscription): \").strip()\n", "\n", - "client = GroundedGeminiClient(\n", - " project_id=PROJECT_ID,\n", - " location=LOCATION,\n", - " parallel_api_key=PARALLEL_API_KEY or None,\n", - ")\n", + "client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)\n", "\n", "print(f\"model : {MODEL}\")\n", - "print(f\"endpoint : {client._get_endpoint_url(MODEL)}\")" + "print(f\"location : {LOCATION}\")" ] }, { "cell_type": "markdown", - "id": "a41fc29d", + "id": "8ff0068f", "metadata": {}, "source": [ "### 1.3 How grounding works — and how we'll call it\n", "\n", - "Grounding is enabled by a `parallelAiSearch` entry in the request's `tools` array, which the client builds from its `GroundingConfig`. When it is present, Gemini translates the prompt into web search queries, Parallel retrieves LLM-optimized excerpts from the live web, and the model composes its answer from that retrieved evidence.\n", + "Grounding is enabled by passing a `parallel_ai_search` tool in the request config. When it is present, Gemini translates the prompt into web search queries, Parallel retrieves LLM-optimized excerpts from the live web, and the model composes its answer from that retrieved evidence.\n", "\n", - "All of the grounding configuration is optional, and the defaults are the right starting point: per [Parallel's search best practices](https://docs.parallel.ai/search/best-practices), restrictive retrieval parameters (domain allowlists, low result caps, geo filters) can unnecessarily limit results and reduce quality, so we don't set any. The full parameter list is in the appendix.\n", + "The tool's `custom_configs` accepts any Parallel Search API parameter, but every field is optional and the defaults are the right starting point: per [Parallel's search best practices](https://docs.parallel.ai/search/best-practices). \n", "\n", - "We use `client.generate()` in two modes:\n", + "We call `client.models.generate_content` in two modes:\n", "\n", - "- **Grounded** (the default) for research calls — returns a `GroundedResponse` whose `text` is the evidence prose, `web_search_queries` are the queries Gemini executed through Parallel, and `sources` are the retrieved documents parsed from the response's `groundingMetadata`.\n", - "- **Tool-free** (`grounded=False`) for extraction calls — Gemini cannot combine a grounding tool with `responseSchema` in one request, so structured output must be a separate call, with the schema and related settings passed through `generation_config`." + "- **Grounded** for research calls — the config carries the Parallel tool, and the response's `grounding_metadata` holds the executed `web_search_queries`, the retrieved source documents (`grounding_chunks`), and per-claim attribution spans (`grounding_supports`).\n", + "- **Tool-free** for extraction calls — Gemini cannot combine a grounding tool with `response_schema` in one request, so structured output is a separate call with no tools attached.\n", + "\n", + "The first grounded call below asks a current factual question and prints the queries Gemini ran through Parallel before the answer." ] }, { - "cell_type": "markdown", - "id": "cfc20361", - "metadata": {}, + "cell_type": "code", + "execution_count": 3, + "id": "eda3a5f2", + "metadata": { + "execution": { + "iopub.execute_input": "2026-07-14T21:29:59.953585Z", + "iopub.status.busy": "2026-07-14T21:29:59.953482Z", + "iopub.status.idle": "2026-07-14T21:30:06.522241Z", + "shell.execute_reply": "2026-07-14T21:30:06.521748Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Executed search queries:\n", + " - \"Selling, general and administrative\" Apple Q2 2026 10-Q\n", + " - Apple investor relations financial results\n", + "\n", + "Answer:\n", + "In its most recently reported fiscal quarter—the **second quarter of fiscal year 2026** (which ended on **March 28, 2026**)—Apple's selling, general and administrative (SG&A) expense was **$7.477 billion** ($7,477 million).\n" + ] + } + ], "source": [ - "## 2. Company enrichment, step by step\n", - "\n", - "We enrich one record at a time so the Gemini and Parallel integration stays visible; applying the pattern to many rows is a loop over the same three calls.\n", + "parallel_tool = types.Tool(\n", + " parallel_ai_search=types.ToolParallelAiSearch(\n", + " api_key=PARALLEL_API_KEY or None, # omit for Marketplace-subscription auth\n", + " )\n", + ")\n", "\n", - "### 2.1 Define the output contract\n", + "response = client.models.generate_content(\n", + " model=MODEL,\n", + " contents=(\n", + " \"What was Apple's selling, general and administrative (SG&A) expense in its most \"\n", + " \"recently reported fiscal quarter? Give the exact figure and state which quarter it covers.\"\n", + " ),\n", + " config=types.GenerateContentConfig(tools=[parallel_tool], temperature=0.2),\n", + ")\n", "\n", - "Pydantic gives us a single source of truth for three things: the JSON Schema sent to Gemini, the validation of the model's output, and the typed object handed to downstream code. The field descriptions do real work here — they tell the model what each field means and, critically, the exact format it must use. `headquarters` and `founded_year` are good examples: their descriptions pin the formats (\"City, Region, Country\" and `YYYY`), so values come back machine-comparable rather than free-text like \"the Bay Area\" or \"founded about five years ago\".\n", + "grounding = response.candidates[0].grounding_metadata\n", + "print(\"Executed search queries:\")\n", + "for query in grounding.web_search_queries or []:\n", + " print(f\" - {query}\")\n", "\n", - "Structured output guarantees that the response follows this shape. It does not guarantee that every fact is correct, so the schema also carries per-field `citations` to keep the evidence visible.\n", + "print(f\"\\nAnswer:\\n{response.text.strip()}\")" + ] + }, + { + "cell_type": "markdown", + "id": "51c38fb6", + "metadata": {}, + "source": [ + "### 1.4 Attach inline citations from the grounding metadata\n", "\n", - "Gemini's `responseSchema` accepts an OpenAPI-style subset of JSON Schema, not pydantic's dialect (it rejects `$ref`/`$defs` and unknown keys, and models `Optional` with `nullable` rather than `anyOf`), so `to_gemini_schema` performs that mechanical conversion." + "The response's `grounding_supports` maps segments of the answer text to the indices of the `grounding_chunks` that back them. That lets us attach citations **in code** instead of asking the model to write links: the model never generates a URL, so it cannot invent or rewrite one — every footnote is copied from the grounding metadata.\n" ] }, { "cell_type": "code", - "execution_count": 3, - "id": "1b5cfbf5", + "execution_count": 4, + "id": "f7a7f5ca", "metadata": { "execution": { - "iopub.execute_input": "2026-07-07T02:07:51.212129Z", - "iopub.status.busy": "2026-07-07T02:07:51.211942Z", - "iopub.status.idle": "2026-07-07T02:07:51.217686Z", - "shell.execute_reply": "2026-07-07T02:07:51.217170Z" + "iopub.execute_input": "2026-07-14T21:30:06.527064Z", + "iopub.status.busy": "2026-07-14T21:30:06.526943Z", + "iopub.status.idle": "2026-07-14T21:30:06.530216Z", + "shell.execute_reply": "2026-07-14T21:30:06.529825Z" } }, "outputs": [ @@ -183,27 +214,71 @@ "name": "stdout", "output_type": "stream", "text": [ - "{\n", - " \"required\": [\n", - " \"company_name\",\n", - " \"official_domain\",\n", - " \"ceo_name\",\n", - " \"headquarters\",\n", - " \"headquarters_address\",\n", - " \"founded_year\",\n", - " \"citations\"\n", - " ],\n", - " \"type\": \"object\",\n", - " \"properties\": {\n", - " \"company_name\": {\n", - " \"description\": \"Company name, copied exactly from the input record.\",\n", - " \"type\": \"string\"\n", - " },\n", - " \"official_domain\": {\n", - " \"description\": \"Official domain, copi …\n" + "In its most recently reported fiscal quarter—the **second quarter of fiscal year 2026** (which ended on **March 28, 2026**)—Apple's selling, general and administrative (SG&A) expense was **$7.477 billion** ($7,477 million)[1][2].\n", + "\n", + "Sources:\n", + "[1] Apple SG&A Expenses 2012-2026 | AAPL | MacroTrends — https://www.macrotrends.net/stocks/charts/AAPL/apple/selling-general-administrative-expenses\n", + "[2] aapl-20260328 — https://www.sec.gov/Archives/edgar/data/320193/000032019326000013/aapl-20260328.htm\n" ] } ], + "source": [ + "def with_inline_citations(response) -> str:\n", + " \"\"\"Insert [n] markers after each supported claim and append the numbered source list.\"\"\"\n", + " metadata = response.candidates[0].grounding_metadata\n", + " chunks = metadata.grounding_chunks or []\n", + " supports = metadata.grounding_supports or []\n", + "\n", + " encoded = response.text.encode(\"utf-8\") # segment offsets are byte offsets\n", + " insertions = []\n", + " for support in supports:\n", + " end = support.segment.end_index\n", + " indices = support.grounding_chunk_indices or []\n", + " if end is not None and indices:\n", + " insertions.append((end, \"\".join(f\"[{i + 1}]\" for i in indices)))\n", + " for end, marker in sorted(insertions, key=lambda pair: -pair[0]): # right to left\n", + " encoded = encoded[:end] + marker.encode(\"utf-8\") + encoded[end:]\n", + "\n", + " footnotes = \"\\n\".join(\n", + " f\"[{i}] {' '.join((chunk.web.title or '(untitled)').split())} — {chunk.web.uri}\"\n", + " for i, chunk in enumerate(chunks, start=1)\n", + " if chunk.web\n", + " )\n", + " return f\"{encoded.decode('utf-8').strip()}\\n\\nSources:\\n{footnotes}\"\n", + "\n", + "\n", + "print(with_inline_citations(response))" + ] + }, + { + "cell_type": "markdown", + "id": "d01fc327", + "metadata": {}, + "source": [ + "## 2. Company enrichment, step by step\n", + "\n", + "### 2.1 Define the output contract\n", + "\n", + "Pydantic gives us a single source of truth for three things: the JSON Schema sent to Gemini, the validation of the model's output, and the typed object handed to downstream code. The field descriptions do real work here — they tell the model what each field means and, critically, the exact format it must use. `headquarters` and `founded_year` are good examples: their descriptions pin the formats (\"City, Region, Country\" and `YYYY`), so values come back machine-comparable rather than free-text like \"the Bay Area\" or \"founded about five years ago\".\n", + "\n", + "Structured output guarantees that the response follows this shape. It does not guarantee that every fact is correct, so the schema also carries per-field `citations` to keep the evidence visible.\n", + "\n", + "The SDK accepts a pydantic model directly as `response_schema` — it converts the model to the API's OpenAPI-subset schema and parses the response back into a typed instance on `response.parsed`, so no hand-written schema conversion is needed." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "683494b4", + "metadata": { + "execution": { + "iopub.execute_input": "2026-07-14T21:30:06.532037Z", + "iopub.status.busy": "2026-07-14T21:30:06.531964Z", + "iopub.status.idle": "2026-07-14T21:30:06.535207Z", + "shell.execute_reply": "2026-07-14T21:30:06.534839Z" + } + }, + "outputs": [], "source": [ "from pydantic import BaseModel, Field\n", "\n", @@ -230,79 +305,19 @@ " founded_year: str = Field(\n", " description=\"Year the company was founded, as a four-digit year in YYYY format, or 'unknown'.\"\n", " )\n", - " citations: list[Citation] = Field(description=\"Sources supporting every populated field.\")\n", - "\n", - "\n", - "GEMINI_SCHEMA_KEYS = {\n", - " \"type\", \"format\", \"description\", \"nullable\", \"enum\",\n", - " \"required\", \"minimum\", \"maximum\", \"minItems\", \"maxItems\",\n", - "}\n", - "\n", - "\n", - "def to_gemini_schema(model_cls: type[BaseModel]) -> dict:\n", - " \"\"\"Convert a pydantic JSON Schema to the OpenAPI subset the Gemini API's responseSchema accepts:\n", - " inline $refs, collapse Optional[...] anyOf into nullable, drop unsupported keys.\"\"\"\n", - " schema = model_cls.model_json_schema()\n", - " defs = schema.get(\"$defs\", {})\n", - "\n", - " def clean(node: dict) -> dict:\n", - " if \"$ref\" in node:\n", - " node = defs[node[\"$ref\"].split(\"/\")[-1]]\n", - " if \"anyOf\" in node: # pydantic emits Optional[X] as anyOf [X, null]\n", - " variant = next(v for v in node[\"anyOf\"] if v.get(\"type\") != \"null\")\n", - " node = {**variant, \"nullable\": True, **{k: v for k, v in node.items() if k != \"anyOf\"}}\n", - " cleaned = {key: value for key, value in node.items() if key in GEMINI_SCHEMA_KEYS}\n", - " if \"properties\" in node:\n", - " cleaned[\"properties\"] = {name: clean(sub) for name, sub in node[\"properties\"].items()}\n", - " if \"items\" in node:\n", - " cleaned[\"items\"] = clean(node[\"items\"])\n", - " return cleaned\n", - "\n", - " return clean(schema)\n", - "\n", - "\n", - "company_schema = to_gemini_schema(CompanyEnrichment)\n", - "print(json.dumps(company_schema, indent=2)[:400], \"…\")" + " citations: list[Citation] = Field(description=\"Sources supporting every populated field.\")" ] }, { "cell_type": "markdown", - "id": "a4f510ff", + "id": "ab9b6b0b", "metadata": {}, "source": [ - "### 2.2 Define the input record\n", + "### 2.2 Define the input record and the two instruction blocks\n", "\n", - "The input is deliberately small: it contains what we already know. The workflow's job is to add verified fields without changing the original identity of the record — exactly the shape of a CRM row or a vendor list entry waiting to be filled in." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "183f1d5c", - "metadata": { - "execution": { - "iopub.execute_input": "2026-07-07T02:07:51.218694Z", - "iopub.status.busy": "2026-07-07T02:07:51.218635Z", - "iopub.status.idle": "2026-07-07T02:07:51.220153Z", - "shell.execute_reply": "2026-07-07T02:07:51.219852Z" - } - }, - "outputs": [], - "source": [ - "company_row = {\n", - " \"company_name\": \"Anthropic\",\n", - " \"official_domain\": \"anthropic.com\",\n", - "}" - ] - }, - { - "cell_type": "markdown", - "id": "96ccc836", - "metadata": {}, - "source": [ - "### 2.3 Separate the research objective from the enrichment policy\n", + "The input record is deliberately small: it contains what we already know. The workflow's job is to add verified fields without changing the original identity of the record — exactly the shape of a CRM row or a vendor list entry waiting to be filled in.\n", "\n", - "The two model calls get two different instruction blocks, mirroring the two jobs:\n", + "The two model calls then get two different instruction blocks, mirroring the two jobs:\n", "\n", "- **The research objective** goes to the *grounding* call. Per Parallel's best practices for search objectives, it is a natural-language description of the research goal that names the key entity, states exactly what to find, and carries source guidance (\"prefer the company's official website, press releases, and filings\") in prose. Retrieval itself stays unrestricted — the guidance steers ranking without excluding evidence.\n", "- **The enrichment policy** goes to the *structuring* call. It contains only output rules: copy identity fields exactly, copy citation URLs only from the supplied source list, honor each field's declared format, and represent uncertainty as `\"unknown\"` rather than a guess. It never mentions searching, because the structuring call has no tools.\n", @@ -312,18 +327,24 @@ }, { "cell_type": "code", - "execution_count": 5, - "id": "6c11ee00", + "execution_count": 6, + "id": "3b58b9ec", "metadata": { "execution": { - "iopub.execute_input": "2026-07-07T02:07:51.221043Z", - "iopub.status.busy": "2026-07-07T02:07:51.220990Z", - "iopub.status.idle": "2026-07-07T02:07:51.222664Z", - "shell.execute_reply": "2026-07-07T02:07:51.222371Z" + "iopub.execute_input": "2026-07-14T21:30:06.536263Z", + "iopub.status.busy": "2026-07-14T21:30:06.536180Z", + "iopub.status.idle": "2026-07-14T21:30:06.538045Z", + "shell.execute_reply": "2026-07-14T21:30:06.537760Z" } }, "outputs": [], "source": [ + "company_row = {\n", + " \"company_name\": \"Anthropic\",\n", + " \"official_domain\": \"anthropic.com\",\n", + "}\n", + "\n", + "\n", "def company_objective(record: dict) -> str:\n", " return f\"\"\"Research the company {record[\"company_name\"]} (official website: {record[\"official_domain\"]}).\n", "\n", @@ -348,29 +369,29 @@ }, { "cell_type": "markdown", - "id": "a57501d1", + "id": "4fe95c6a", "metadata": {}, "source": [ - "### 2.4 Ground: gather cited evidence\n", + "### 2.3 Ground: gather cited evidence\n", "\n", - "The grounding call sends the research objective through `client.generate()` with default (unrestricted) retrieval and a low temperature — this is factual research, not creative writing. The returned `GroundedResponse` carries, parsed from the response's `groundingMetadata`:\n", + "The grounding call sends the research objective with the Parallel tool attached, default (unrestricted) retrieval, and a low temperature — this is factual research, not creative writing. From the response's `grounding_metadata` we keep:\n", "\n", "- **`web_search_queries`** — the queries Gemini executed through Parallel, useful for observability.\n", - "- **`sources`** — the retrieved source documents. Each one is a document-level **citable unit**, and its `uri` is the identifier we carry into citations, alongside the page title.\n", + "- **`grounding_chunks`** — the retrieved source documents. Each one is a document-level **citable unit**, and its `web.uri` is the identifier we carry into citations, alongside the page title.\n", "\n", "`normalize_sources` turns those into deduplicated `{url, title}` dicts. That list, produced by retrieval rather than by the model's text, is the trust anchor for the whole enrichment: it defines the only URLs the structuring step is allowed to cite." ] }, { "cell_type": "code", - "execution_count": 6, - "id": "aae130c9", + "execution_count": 7, + "id": "c1d17806", "metadata": { "execution": { - "iopub.execute_input": "2026-07-07T02:07:51.223565Z", - "iopub.status.busy": "2026-07-07T02:07:51.223506Z", - "iopub.status.idle": "2026-07-07T02:08:02.295434Z", - "shell.execute_reply": "2026-07-07T02:08:02.294982Z" + "iopub.execute_input": "2026-07-14T21:30:06.538930Z", + "iopub.status.busy": "2026-07-14T21:30:06.538874Z", + "iopub.status.idle": "2026-07-14T21:30:16.397962Z", + "shell.execute_reply": "2026-07-14T21:30:16.397514Z" } }, "outputs": [ @@ -379,101 +400,74 @@ "output_type": "stream", "text": [ "Executed search queries:\n", - " - Anthropic founded year\n", - " - Anthropic headquarters address\n", + " - \"548 Market Street\"\n", " - Anthropic CEO 2026\n", - " - \"548 Market St\" \"PMB\" Anthropic\n", + " - \"Anthropic\" \"548 Market\" OR \"500 Howard\"\n", "\n", - "Grounded sources (12):\n", - " - Where is Anthropic Located? HQ, Global Offices & Company Insights — https://www.highperformr.ai/company/anthropicresearch\n", + "Grounded sources (6):\n", " - Anthropic - Wikipedia — https://en.wikipedia.org/wiki/Anthropic\n", - " - Company \\ Anthropic — https://www.anthropic.com/company\n", - " - Anthropic’s C.E.O. Says It Could Grow by 80 Times This Year - The New York Times — https://www.nytimes.com/2026/05/06/technology/anthropic-ceo-ai-growth.html\n", - " - 2028: Two scenarios for global AI leadership \\ Anthropic — https://www.anthropic.com/research/2028-ai-leadership\n", - " - description: Learn about Anthropic's headquarters location and see a list of their major offices across different regions. Find out where they operate worldwide. title: Where is Anthropic's Headquarters? Main Office Location and Global Offices | Clay — https://www.clay.com/dossier/anthropic-headquarters-office-locations\n", - " - Who founded Anthropic, the makers of Claude AI, and why? | Britannica — https://www.britannica.com/question/Who-founded-Anthropic-the-makers-of-Claude-AI-and-why\n", - " - Anthropic, PBC — https://www.lobbyfacts.eu/datacard/anthropic-pbc?rid=511227350325-42&sid=178563\n", - " - ANTHROPIC - Anthropic, PBC Trademark Registration — https://uspto.report/TM/98046928\n", - " - 548 Market St., PMB 90375 — https://downloads.regulations.gov/USCIS-2023-0005-1139/attachment_1.pdf\n", - " - Anthropic Corporate Headquarters, Office Locations and Addresses | Craft.co — https://craft.co/anthropic/locations\n", - " - Anthropic 2026 Company Profile: Valuation, Funding & ... — https://pitchbook.com/profiles/company/466959-97\n", - "\n", - "Evidence (1262 chars, first 600):\n", - "Based on the company's official filings, trademark applications, and reputable business publications, here is the researched information for Anthropic:\n", - "\n", - "### 1. Current Chief Executive Officer\n", - "* **Full Name:** Dario Amodei\n", - "* *Source:* Official Anthropic Governance Page, *The New York Times*, and *The Wall Street Journal*.\n", - "\n", - "### 2. Headquarters Location\n", - "* **City:** San Francisco\n", - "* **Region:** California\n", - "* **Country:** United States\n", - "* *Source:* Official Anthropic Research Publications and *Encyclopaedia Britannica*.\n", + " - Anthropic, Pbc, Inc. San Francisco, CA - filing information — https://www.bizprofile.net/ca/san-francisco/anthropic-pbc-inc\n", + " - News | How Anthropic is growing its office empire in downtown San Francisco — https://www.costar.com/article/390354131/how-anthropic-is-growing-its-office-empire-in-downtown-san-francisco\n", + " - What is the mailing address for Anthropic? - Ask and Answer - Glarity — https://askai.glarity.app/search/What-is-the-mailing-address-for-Anthropic\n", + " - anthropic, pbc, inc. - Detail by Entity Name — https://search.sunbiz.org/Inquiry/corporationsearch/SearchResultDetail?aggregateId=forp-f24000001568-aa469358-d133-43d9-9fc6-3c7c00c42c1d&directionType=Initial&inquirytype=EntityName&listNameOrder=ANTHROED%20L200000257930&searchNameOrder=ANTHROPICPBC%20F240000015680&searchTerm=ANTHRO-ED%20LLC\n", + " - Anthropic, Pbc San Francisco, CA - filing information — https://www.bizprofile.net/ca/san-francisco/anthropic-pbc-2\n", "\n", - "### 3. Headquarters Mailing Address\n", - "* **Full Mailing Address:** \n", - " Anthropic, PBC…\n" + "Evidence (1139…\n" ] } ], "source": [ "def normalize_sources(response) -> list[dict]:\n", - " \"\"\"Deduplicated {url, title} dicts from a GroundedResponse, in retrieval order.\"\"\"\n", + " \"\"\"Deduplicated {url, title} dicts from a grounded response, in retrieval order.\"\"\"\n", + " metadata = response.candidates[0].grounding_metadata\n", " sources, seen = [], set()\n", - " for source in response.sources:\n", - " if source.uri and source.uri not in seen:\n", - " seen.add(source.uri)\n", - " sources.append({\"url\": source.uri, \"title\": \" \".join((source.title or \"\").split())})\n", + " for chunk in metadata.grounding_chunks or []:\n", + " if chunk.web and chunk.web.uri and chunk.web.uri not in seen:\n", + " seen.add(chunk.web.uri)\n", + " sources.append({\"url\": chunk.web.uri, \"title\": \" \".join((chunk.web.title or \"\").split())})\n", " return sources\n", "\n", "\n", - "grounding_response = client.generate(\n", - " company_objective(company_row),\n", - " model_id=MODEL,\n", - " temperature=0.2,\n", + "grounding_response = client.models.generate_content(\n", + " model=MODEL,\n", + " contents=company_objective(company_row),\n", + " config=types.GenerateContentConfig(tools=[parallel_tool], temperature=0.2),\n", ")\n", "\n", "evidence = grounding_response.text.strip()\n", "grounded_sources = normalize_sources(grounding_response)\n", "\n", "print(\"Executed search queries:\")\n", - "for query in grounding_response.web_search_queries:\n", + "for query in grounding_response.candidates[0].grounding_metadata.web_search_queries or []:\n", " print(f\" - {query}\")\n", "\n", "print(f\"\\nGrounded sources ({len(grounded_sources)}):\")\n", "for source in grounded_sources:\n", " print(f\" - {source['title'] or '(untitled)'} — {source['url']}\")\n", "\n", - "print(f\"\\nEvidence ({len(evidence)} chars, first 600):\\n{evidence[:600]}…\")" + "print(f\"\\nEvidence ({len(evidence)}…\")" ] }, { "cell_type": "markdown", - "id": "8be0cd9f", + "id": "94210558", "metadata": {}, "source": [ - "### 2.5 Structure: extract the typed record\n", + "### 2.4 Structure: extract the typed record\n", "\n", - "The structuring call is configured for mechanical extraction, the opposite profile from research:\n", - "\n", - "- **`grounded=False`** — no tools are attached, so the model may only reorganize the evidence it is given, never fetch more.\n", - "- **`temperature=0`** and **`responseSchema`** — decoding is deterministic and constrained to `CompanyEnrichment`'s shape, with `responseMimeType: application/json`.\n", - "- **`thinkingBudget: 0`** — extraction doesn't benefit from extended thinking, and disabling it prevents a thinking model from spending the output budget on thoughts.\n", - "\n", - "The prompt stacks the enrichment policy on top of three clearly delimited data blocks: the input record, the grounded evidence, and the sources list. Pydantic then validates the raw JSON — if the model ever returned a malformed or schema-violating object, `model_validate_json` would raise here rather than let a bad record flow downstream." + "The prompt stacks the enrichment policy on top of three clearly delimited data blocks: the input record, the grounded evidence, and the sources list. The SDK validates and parses the JSON into a `CompanyEnrichment` instance on `response.parsed` — if the model ever returned a malformed or schema-violating object, parsing would raise here rather than let a bad record flow downstream." ] }, { "cell_type": "code", - "execution_count": 7, - "id": "54e6e084", + "execution_count": 8, + "id": "3c6f42d7", "metadata": { "execution": { - "iopub.execute_input": "2026-07-07T02:08:02.296585Z", - "iopub.status.busy": "2026-07-07T02:08:02.296517Z", - "iopub.status.idle": "2026-07-07T02:08:04.913991Z", - "shell.execute_reply": "2026-07-07T02:08:04.913469Z" + "iopub.execute_input": "2026-07-14T21:30:16.400137Z", + "iopub.status.busy": "2026-07-14T21:30:16.400042Z", + "iopub.status.idle": "2026-07-14T21:30:18.758019Z", + "shell.execute_reply": "2026-07-14T21:30:18.757530Z" } }, "outputs": [ @@ -486,28 +480,28 @@ " \"official_domain\": \"anthropic.com\",\n", " \"ceo_name\": \"Dario Amodei\",\n", " \"headquarters\": \"San Francisco, California, United States\",\n", - " \"headquarters_address\": \"548 Market St, PMB 90375, San Francisco, CA, 94104, United States\",\n", + " \"headquarters_address\": \"548 Market Street, PMB 90375, San Francisco, CA 94104, United States\",\n", " \"founded_year\": \"2021\",\n", " \"citations\": [\n", " {\n", " \"field\": \"ceo_name\",\n", - " \"url\": \"https://www.nytimes.com/2026/05/06/technology/anthropic-ceo-ai-growth.html\",\n", - " \"note\": \"Dario Amodei is the Chief Executive Officer of Anthropic.\"\n", + " \"url\": \"https://en.wikipedia.org/wiki/Anthropic\",\n", + " \"note\": \"Dario Amodei\"\n", " },\n", " {\n", " \"field\": \"headquarters\",\n", - " \"url\": \"https://www.britannica.com/question/Who-founded-Anthropic-the-makers-of-Claude-AI-and-why\",\n", - " \"note\": \"Anthropic's headquarters is located in San Francisco, California, United States.\"\n", + " \"url\": \"https://en.wikipedia.org/wiki/Anthropic\",\n", + " \"note\": \"San Francisco, California, United States\"\n", " },\n", " {\n", " \"field\": \"headquarters_address\",\n", - " \"url\": \"https://uspto.report/TM/98046928\",\n", - " \"note\": \"The official corporate mailing address of record is 548 Market St, PMB 90375, San Francisco, CA 94104, United States.\"\n", + " \"url\": \"https://askai.glarity.app/search/What-is-the-mailing-address-for-Anthropic\",\n", + " \"note\": \"548 Market Street, PMB 90375, San Francisco, CA 94104, United States\"\n", " },\n", " {\n", " \"field\": \"founded_year\",\n", - " \"url\": \"https://pitchbook.com/profiles/company/466959-97\",\n", - " \"note\": \"Anthropic was founded in the year 2021.\"\n", + " \"url\": \"https://en.wikipedia.org/wiki/Anthropic\",\n", + " \"note\": \"2021\"\n", " }\n", " ]\n", "}\n" @@ -532,45 +526,43 @@ "\"\"\"\n", "\n", "\n", - "structuring_response = client.generate(\n", - " extraction_prompt(company_row, evidence, grounded_sources),\n", - " model_id=MODEL,\n", - " grounded=False,\n", - " temperature=0.0,\n", - " max_output_tokens=8192,\n", - " generation_config={\n", - " \"responseMimeType\": \"application/json\",\n", - " \"responseSchema\": company_schema,\n", - " \"thinkingConfig\": {\"thinkingBudget\": 0},\n", - " },\n", + "structuring_response = client.models.generate_content(\n", + " model=MODEL,\n", + " contents=extraction_prompt(company_row, evidence, grounded_sources),\n", + " config=types.GenerateContentConfig(\n", + " temperature=0.0,\n", + " response_mime_type=\"application/json\",\n", + " response_schema=CompanyEnrichment,\n", + " thinking_config=types.ThinkingConfig(thinking_budget=0),\n", + " ),\n", ")\n", "\n", - "company_enrichment = CompanyEnrichment.model_validate_json(structuring_response.text)\n", + "company_enrichment: CompanyEnrichment = structuring_response.parsed\n", "print(json.dumps(company_enrichment.model_dump(), indent=2))" ] }, { "cell_type": "markdown", - "id": "9ac8f537", + "id": "a7a73685", "metadata": {}, "source": [ - "### 2.6 Verify citations and load the record\n", + "### 2.5 Verify citations and load the record\n", "\n", - "Structured output guaranteed the shape and pydantic validated it; the last step is verifying provenance. Because the policy requires citation URLs to be copied from the grounded source list, we can check every one mechanically against the URLs that came out of the grounding metadata in step 2.4 — and confirm that every populated fact field carries at least one citation. A citation that fails this check would mean the model wrote a URL retrieval never returned, which is exactly the failure mode this pattern exists to catch.\n", + "Structured output guaranteed the shape and pydantic validated it; the last step is verifying provenance. Because the policy requires citation URLs to be copied from the grounded source list, we can check every one mechanically against the URLs that came out of the grounding metadata in step 2.3 — and confirm that every populated fact field carries at least one citation. A citation that fails this check would mean the model wrote a URL retrieval never returned, which is exactly the failure mode this pattern exists to catch.\n", "\n", "`verify_citations` is written once, generically: fact fields are whatever the contract declares beyond the input record's identity fields and the bookkeeping field (`citations`). Section 3 reuses it unchanged. After the checks pass, `model_dump()` turns the record into plain Python data, ready for a dataframe, database, or API response." ] }, { "cell_type": "code", - "execution_count": 8, - "id": "7f1b644a", + "execution_count": 9, + "id": "e190b3db", "metadata": { "execution": { - "iopub.execute_input": "2026-07-07T02:08:04.914994Z", - "iopub.status.busy": "2026-07-07T02:08:04.914925Z", - "iopub.status.idle": "2026-07-07T02:08:04.919001Z", - "shell.execute_reply": "2026-07-07T02:08:04.918569Z" + "iopub.execute_input": "2026-07-14T21:30:18.759616Z", + "iopub.status.busy": "2026-07-14T21:30:18.759530Z", + "iopub.status.idle": "2026-07-14T21:30:18.764761Z", + "shell.execute_reply": "2026-07-14T21:30:18.764115Z" } }, "outputs": [ @@ -578,11 +570,11 @@ "name": "stdout", "output_type": "stream", "text": [ - "field verified against grounded sources url\n", - "ceo_name True https://www.nytimes.com/2026/05/06/technology/anthropic-ceo-ai-growth.html\n", - "headquarters True https://www.britannica.com/question/Who-founded-Anthropic-the-makers-of-Claude-AI-and-why\n", - "headquarters_address True https://uspto.report/TM/98046928\n", - "founded_year True https://pitchbook.com/profiles/company/466959-97\n", + "field url\n", + "ceo_name https://en.wikipedia.org/wiki/Anthropic\n", + "headquarters https://en.wikipedia.org/wiki/Anthropic\n", + "headquarters_address https://askai.glarity.app/search/What-is-the-mailing-address-for-Anthropic\n", + "founded_year https://en.wikipedia.org/wiki/Anthropic\n", "\n", "All citations verified.\n" ] @@ -594,23 +586,23 @@ " 'official_domain': 'anthropic.com',\n", " 'ceo_name': 'Dario Amodei',\n", " 'headquarters': 'San Francisco, California, United States',\n", - " 'headquarters_address': '548 Market St, PMB 90375, San Francisco, CA, 94104, United States',\n", + " 'headquarters_address': '548 Market Street, PMB 90375, San Francisco, CA 94104, United States',\n", " 'founded_year': '2021',\n", " 'citations': [{'field': 'ceo_name',\n", - " 'url': 'https://www.nytimes.com/2026/05/06/technology/anthropic-ceo-ai-growth.html',\n", - " 'note': 'Dario Amodei is the Chief Executive Officer of Anthropic.'},\n", + " 'url': 'https://en.wikipedia.org/wiki/Anthropic',\n", + " 'note': 'Dario Amodei'},\n", " {'field': 'headquarters',\n", - " 'url': 'https://www.britannica.com/question/Who-founded-Anthropic-the-makers-of-Claude-AI-and-why',\n", - " 'note': \"Anthropic's headquarters is located in San Francisco, California, United States.\"},\n", + " 'url': 'https://en.wikipedia.org/wiki/Anthropic',\n", + " 'note': 'San Francisco, California, United States'},\n", " {'field': 'headquarters_address',\n", - " 'url': 'https://uspto.report/TM/98046928',\n", - " 'note': 'The official corporate mailing address of record is 548 Market St, PMB 90375, San Francisco, CA 94104, United States.'},\n", + " 'url': 'https://askai.glarity.app/search/What-is-the-mailing-address-for-Anthropic',\n", + " 'note': '548 Market Street, PMB 90375, San Francisco, CA 94104, United States'},\n", " {'field': 'founded_year',\n", - " 'url': 'https://pitchbook.com/profiles/company/466959-97',\n", - " 'note': 'Anthropic was founded in the year 2021.'}]}" + " 'url': 'https://en.wikipedia.org/wiki/Anthropic',\n", + " 'note': '2021'}]}" ] }, - "execution_count": 8, + "execution_count": 9, "metadata": {}, "output_type": "execute_result" } @@ -624,9 +616,9 @@ " if name not in record and name != \"citations\"\n", " ]\n", "\n", - " print(f\"{'field':<17} {'verified against grounded sources':<36} url\")\n", + " print(f\"{'field':<21} url\")\n", " for citation in enriched.citations:\n", - " print(f\"{citation.field:<17} {str(citation.url in grounded_urls):<36} {citation.url}\")\n", + " print(f\"{citation.field:<21} {citation.url}\")\n", "\n", " unverified = [c.url for c in enriched.citations if c.url not in grounded_urls]\n", " uncited = [\n", @@ -645,34 +637,79 @@ }, { "cell_type": "markdown", - "id": "0b749f31", + "id": "91955542", "metadata": {}, "source": [ "## 3. People enrichment with the same pipeline\n", "\n", - "Nothing in sections 2.4–2.6 was specific to companies: ground, structure, and verify only care about *a record*, *a contract*, and *an objective*. To enrich people instead, we swap the two record-type-specific pieces:\n", + "Nothing in sections 2.3–2.5 was specific to companies: ground, structure, and verify only care about *a record*, *a contract*, and *an objective*. To enrich people instead, we swap the two record-type-specific pieces:\n", "\n", "- **A new contract.** `PersonEnrichment` declares the fields a recruiting or sales list needs — current title, current employer, and location — with the same format-precise descriptions and the same `citations` bookkeeping.\n", "- **A new objective template.** People are harder to disambiguate than companies, so the input record carries a `known_affiliation` field and the objective instructs the model to use it — and to say so if the identification is uncertain, rather than blending two people who share a name.\n", "\n", - "The enrichment policy, the schema converter, and the verification logic are reused verbatim.\n", - "\n", - "### 3.1 Define the person contract and objective" + "The enrichment policy and the verification logic are reused verbatim: `enrich` below packages the three steps exactly as sections 2.3–2.5 ran them, and is the loop body you would run once per row to enrich a whole dataset." ] }, { "cell_type": "code", - "execution_count": 9, - "id": "e3d261c0", + "execution_count": 10, + "id": "604cdd13", "metadata": { "execution": { - "iopub.execute_input": "2026-07-07T02:08:04.920012Z", - "iopub.status.busy": "2026-07-07T02:08:04.919937Z", - "iopub.status.idle": "2026-07-07T02:08:04.922527Z", - "shell.execute_reply": "2026-07-07T02:08:04.922137Z" + "iopub.execute_input": "2026-07-14T21:30:18.766877Z", + "iopub.status.busy": "2026-07-14T21:30:18.766806Z", + "iopub.status.idle": "2026-07-14T21:30:26.917197Z", + "shell.execute_reply": "2026-07-14T21:30:26.916801Z" } }, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Executed search queries:\n", + " - Lisa Su AMD current job title employer website\n", + " - AMD headquarters address city state country\n", + "Grounded sources: 8\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "field url\n", + "current_title https://www.amd.com/en/corporate/leadership/lisa-su.html\n", + "current_employer https://www.amd.com/en/corporate/leadership/lisa-su.html\n", + "location https://www.linkedin.com/in/lisasu-amd\n", + "\n", + "All citations verified.\n" + ] + }, + { + "data": { + "text/plain": [ + "{'full_name': 'Lisa Su',\n", + " 'known_affiliation': 'AMD',\n", + " 'current_title': 'Chair and Chief Executive Officer',\n", + " 'current_employer': 'Advanced Micro Devices, Inc. (AMD)',\n", + " 'location': 'Austin, Texas, United States',\n", + " 'citations': [{'field': 'current_title',\n", + " 'url': 'https://www.amd.com/en/corporate/leadership/lisa-su.html',\n", + " 'note': 'Chair and Chief Executive Officer'},\n", + " {'field': 'current_employer',\n", + " 'url': 'https://www.amd.com/en/corporate/leadership/lisa-su.html',\n", + " 'note': 'Advanced Micro Devices, Inc. (AMD)'},\n", + " {'field': 'location',\n", + " 'url': 'https://www.linkedin.com/in/lisasu-amd',\n", + " 'note': 'Austin, Texas, United States'}]}" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "class PersonEnrichment(BaseModel):\n", " full_name: str = Field(description=\"Person's full name, copied exactly from the input record.\")\n", @@ -708,29 +745,65 @@ "person_row = {\n", " \"full_name\": \"Lisa Su\",\n", " \"known_affiliation\": \"AMD\",\n", - "}" + "}\n", + "\n", + "\n", + "def enrich(record: dict, contract: type[BaseModel], objective: str) -> BaseModel:\n", + " \"\"\"Ground -> structure -> verify one record against a pydantic contract.\"\"\"\n", + " grounding = client.models.generate_content(\n", + " model=MODEL,\n", + " contents=objective,\n", + " config=types.GenerateContentConfig(tools=[parallel_tool], temperature=0.2),\n", + " )\n", + " evidence = grounding.text.strip()\n", + " sources = normalize_sources(grounding)\n", + "\n", + " print(\"Executed search queries:\")\n", + " for query in grounding.candidates[0].grounding_metadata.web_search_queries or []:\n", + " print(f\" - {query}\")\n", + " print(f\"Grounded sources: {len(sources)}\\n\")\n", + "\n", + " structuring = client.models.generate_content(\n", + " model=MODEL,\n", + " contents=extraction_prompt(record, evidence, sources),\n", + " config=types.GenerateContentConfig(\n", + " temperature=0.0,\n", + " response_mime_type=\"application/json\",\n", + " response_schema=contract,\n", + " thinking_config=types.ThinkingConfig(thinking_budget=0),\n", + " ),\n", + " )\n", + " enriched = structuring.parsed\n", + " verify_citations(enriched, record, sources)\n", + " return enriched\n", + "\n", + "\n", + "person_enrichment = enrich(person_row, PersonEnrichment, person_objective(person_row))\n", + "person_enrichment.model_dump()" ] }, { "cell_type": "markdown", - "id": "9e90e1ff", + "id": "a12ced9f", "metadata": {}, "source": [ - "### 3.2 Run the pipeline end to end\n", + "## 4. Product catalog enrichment\n", "\n", - "`enrich` packages the three steps exactly as sections 2.4–2.6 ran them: ground with unrestricted retrieval, structure against the contract's converted schema, verify against the grounded source list. This is the loop body you would run once per row to enrich a whole dataset — and because `GroundedGeminiClient` caches its OAuth token, the loop doesn't pay an auth round trip per record." + "A third record type, same pipeline. Product catalogs are a classic enrichment target: a merchandising or procurement dataset knows a product's name and manufacturer, but launch dates, category placement, and list prices go stale or arrive incomplete from upstream feeds.\n", + "\n", + "`ProductEnrichment` follows the same recipe as the previous contracts — identity fields copied from the input, format-precise fact fields (`release_date` pinned to `YYYY-MM-DD`, `list_price_usd` to a plain decimal number, so both stay machine-comparable), and the same `citations` bookkeeping. The objective steers retrieval toward the manufacturer's product pages and reputable technology press. Everything else — the policy, `extraction_prompt`, `verify_citations`, and `enrich` — is reused untouched." ] }, { "cell_type": "code", - "execution_count": 10, - "id": "f98df8ea", + "execution_count": 11, + "id": "e92f2dc9", "metadata": { "execution": { - "iopub.execute_input": "2026-07-07T02:08:04.923504Z", - "iopub.status.busy": "2026-07-07T02:08:04.923431Z", - "iopub.status.idle": "2026-07-07T02:08:13.017143Z", - "shell.execute_reply": "2026-07-07T02:08:13.016636Z" + "iopub.execute_input": "2026-07-14T21:30:26.919172Z", + "iopub.status.busy": "2026-07-14T21:30:26.919076Z", + "iopub.status.idle": "2026-07-14T21:30:37.291060Z", + "shell.execute_reply": "2026-07-14T21:30:37.290619Z" } }, "outputs": [ @@ -739,9 +812,9 @@ "output_type": "stream", "text": [ "Executed search queries:\n", - " - AMD headquarters location\n", - " - Lisa Su AMD current title 2026\n", - "Grounded sources: 5\n", + " - Google Pixel 10 Pro release date\n", + " - \"Pixel 10 Pro\" \"999\"\n", + "Grounded sources: 6\n", "\n" ] }, @@ -749,10 +822,10 @@ "name": "stdout", "output_type": "stream", "text": [ - "field verified against grounded sources url\n", - "current_title True https://www.amd.com/en/corporate/leadership/lisa-su.html\n", - "current_employer True https://www.amd.com/en/corporate/leadership/lisa-su.html\n", - "location True https://www.amd.com/en/corporate/leadership/lisa-su.html\n", + "field url\n", + "category https://en.wikipedia.org/wiki/Pixel_10_Pro\n", + "release_date https://en.wikipedia.org/wiki/Pixel_10_Pro\n", + "list_price_usd https://9to5google.com/2025/08/20/google-pixel-10-pro-xl-series-launch-price-specs-hands-on\n", "\n", "All citations verified.\n" ] @@ -760,58 +833,65 @@ { "data": { "text/plain": [ - "{'full_name': 'Lisa Su',\n", - " 'known_affiliation': 'AMD',\n", - " 'current_title': 'Chair and Chief Executive Officer',\n", - " 'current_employer': 'Advanced Micro Devices, Inc.',\n", - " 'location': 'Austin, Texas, United States',\n", - " 'citations': [{'field': 'current_title',\n", - " 'url': 'https://www.amd.com/en/corporate/leadership/lisa-su.html',\n", - " 'note': 'Her official title as stated by her employer is Chair and Chief Executive Officer'},\n", - " {'field': 'current_employer',\n", - " 'url': 'https://www.amd.com/en/corporate/leadership/lisa-su.html',\n", - " 'note': 'She works for Advanced Micro Devices, Inc.'},\n", - " {'field': 'location',\n", - " 'url': 'https://www.amd.com/en/corporate/leadership/lisa-su.html',\n", - " 'note': \"Dr. Lisa Su is professionally based out of AMD's major campus in Austin, Texas, United States\"}]}" + "{'product_name': 'Google Pixel 10 Pro',\n", + " 'manufacturer': 'Google',\n", + " 'category': 'smartphone',\n", + " 'release_date': '2025-08-28',\n", + " 'list_price_usd': '999.00',\n", + " 'citations': [{'field': 'category',\n", + " 'url': 'https://en.wikipedia.org/wiki/Pixel_10_Pro',\n", + " 'note': 'Smartphone'},\n", + " {'field': 'release_date',\n", + " 'url': 'https://en.wikipedia.org/wiki/Pixel_10_Pro',\n", + " 'note': 'August 28, 2025'},\n", + " {'field': 'list_price_usd',\n", + " 'url': 'https://9to5google.com/2025/08/20/google-pixel-10-pro-xl-series-launch-price-specs-hands-on',\n", + " 'note': '$999.00'}]}" ] }, - "execution_count": 10, + "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "def enrich(record: dict, contract: type[BaseModel], objective: str) -> BaseModel:\n", - " \"\"\"Ground -> structure -> verify one record against a pydantic contract.\"\"\"\n", - " grounding = client.generate(objective, model_id=MODEL, temperature=0.2)\n", - " evidence = grounding.text.strip()\n", - " sources = normalize_sources(grounding)\n", + "class ProductEnrichment(BaseModel):\n", + " product_name: str = Field(description=\"Product name, copied exactly from the input record.\")\n", + " manufacturer: str = Field(description=\"Manufacturer, copied exactly from the input record.\")\n", + " category: str = Field(\n", + " description=\"Product category as a short noun phrase, e.g. 'mixed-reality headset', or 'unknown'.\"\n", + " )\n", + " release_date: str = Field(\n", + " description=\"Date the product first went on sale, in YYYY-MM-DD format, or 'unknown'.\"\n", + " )\n", + " list_price_usd: str = Field(\n", + " description=(\n", + " \"Current US list price of the base model as a plain decimal number with no currency \"\n", + " \"symbol or thousands separators, e.g. '3499.00', or 'unknown'.\"\n", + " )\n", + " )\n", + " citations: list[Citation] = Field(description=\"Sources supporting every populated field.\")\n", "\n", - " print(\"Executed search queries:\")\n", - " for query in grounding.web_search_queries:\n", - " print(f\" - {query}\")\n", - " print(f\"Grounded sources: {len(sources)}\\n\")\n", "\n", - " structuring = client.generate(\n", - " extraction_prompt(record, evidence, sources),\n", - " model_id=MODEL,\n", - " grounded=False,\n", - " temperature=0.0,\n", - " max_output_tokens=8192,\n", - " generation_config={\n", - " \"responseMimeType\": \"application/json\",\n", - " \"responseSchema\": to_gemini_schema(contract),\n", - " \"thinkingConfig\": {\"thinkingBudget\": 0},\n", - " },\n", - " )\n", - " enriched = contract.model_validate_json(structuring.text)\n", - " verify_citations(enriched, record, sources)\n", - " return enriched\n", + "def product_objective(record: dict) -> str:\n", + " return f\"\"\"Research the product {record[\"product_name\"]} made by {record[\"manufacturer\"]}.\n", "\n", + "Find:\n", + "1. The product's category, as a short noun phrase.\n", + "2. The date the product first went on sale.\n", + "3. The current US list price of the base model.\n", "\n", - "person_enrichment = enrich(person_row, PersonEnrichment, person_objective(person_row))\n", - "person_enrichment.model_dump()" + "Prefer the manufacturer's official product and press pages for specifications and pricing,\n", + "and reputable technology press for launch details. Cite the source of every fact.\"\"\"\n", + "\n", + "\n", + "product_row = {\n", + " \"product_name\": \"Google Pixel 10 Pro\",\n", + " \"manufacturer\": \"Google\",\n", + "}\n", + "\n", + "product_enrichment = enrich(product_row, ProductEnrichment, product_objective(product_row))\n", + "product_enrichment.model_dump()" ] } ], @@ -831,7 +911,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.14.5" + "version": "3.12.11" } }, "nbformat": 4,