Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- **Web model and resolution controls locked**: the web app now always uses `gemini-3.5-flash` for research/planning and `gemini-3.1-flash-lite-image` for image generation/refinement, including stale localStorage migrations. Web resolution choices are limited to 0.5K, 1K, and 2K.
- **Portable skill quality model option**: the CLI skill keeps `gemini-3.1-flash-lite-image` as its default image model and adds a skill-only `--image-model gemini-3.1-flash-image` option for quality-focused runs.
- **Agent loop UX surfaced in Studio**: the generation and refinement flows now expose the bounded loop state (research, plan, render, review, refine), turn count, HITL review status, and stop rule instead of only showing a generic thinking/refining state.
- **Bumped free trial turn limit**: Increased the default trial turn limit from 3 to 5 to give users more free generations before requiring their own Gemini API key.
- **Portable skill rewritten to match the web pipeline (`infographic-agent` v3.0.0)**: the skill now generates infographics **directly with Gemini** using the same two agents as the web demo — a research orchestrator (`gemini-3.5-flash`) grounds the topic with Google Search and engineers a text-accurate prompt, then `gemini-3.1-flash-lite-image` renders the PNG. Replaces the previous HTML/CSS + headless-Chromium screenshot approach.
- **No browser dependencies**: removed Playwright/Chromium from the skill entirely. `npx infographic-agent --install` now just runs `pip install google-genai pillow` (seconds, not a browser download).
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Upload a PDF, paste a URL, or just describe a topic — a two-agent Gemini pipel

- **Multi-format input** — PDF, CSV/spreadsheets, images (PNG/JPEG/WebP/HEIC), plain text, or just a topic description
- **6 infographic modes** — Data Story, Executive Summary, Classroom Explainer, Technical Deep-Dive, Quick Slide, and fully Custom
- **Configurable output** — 6 aspect ratios (square, portrait, landscape, and more) and resolution from 0.5K up to 4K
- **Configurable output** — 6 aspect ratios (square, portrait, landscape, and more) and resolution from 0.5K up to 2K
- **Live "thought" stream** — watch the agent's reasoning render as streaming cards while it researches and designs
- **Multi-turn refinement chat** — keep talking to the agent to tweak colors, layout, or content after the first draft
- **Before/after slider** — compare each revision against the original at a glance
Expand Down Expand Up @@ -107,6 +107,7 @@ The CLI options and flags are shared between the `npx infographic-agent` command
| `--mode` | `-m` | Preset layout and style theme for the infographic. | `data-story`<br>Choices: `classroom`, `custom`, `data-story`, `executive-summary`, `quick-slide`, `technical-deep-dive` |
| `--aspect` | `-a` | Aspect ratio of the generated infographic image. | `9:16`<br>Choices: `1:1`, `1:4`, `3:4`, `4:3`, `9:16`, `16:9` |
| `--instructions` | `-i` | Custom layout, design, or style rules (e.g. brand hex colors, font preferences). | `""` |
| `--image-model` | *None* | Image model for the portable skill. The web app remains locked to `gemini-3.1-flash-lite-image`. | `gemini-3.1-flash-lite-image`<br>Choices: `gemini-3.1-flash-lite-image`, `gemini-3.1-flash-image` |
| `--no-research` | *None* | Skip the research agent and generate directly from your text (faster, doesn't use Google Search). | *Flag* |
| `--no-open` | *None* | Do not auto-open the generated infographic image in the default system viewer. | *Flag* |
| `--yes` | `-y` | Non-interactive execution. Generates once and exits immediately without entering the refine loop. | *Flag* (best for CI or autonomous agents) |
Expand Down Expand Up @@ -325,6 +326,5 @@ infographic-agent/

- **App keeps asking for an API key** — get a free one at [aistudio.google.com/apikey](https://aistudio.google.com/apikey) and either put it in `.env` as `VITE_GEMINI_API_KEY` (dev) or paste it into the settings panel (it's stored in your browser only).
- **"File exceeds maximum size" / files silently skipped** — individual files are capped at 20MB, 50MB total per generation, up to 14 files; split large PDFs or compress images.
- **Generation feels slow** — the analysis agent may search the web or read large files before the image agent starts rendering; watch the thought stream, it's usually still working, not stuck. Higher resolutions (3K/4K) also take longer.
- **Generation feels slow** — the analysis agent may search the web or read large files before the image agent starts rendering; watch the thought stream, it's usually still working, not stuck. Use 0.5K or 1K for faster web-app iteration, then upgrade to 2K when the layout is approved.
- **`npm run dev` prints `spawn xdg-open ENOENT`** — harmless in headless environments (SSH, containers, CI): the server is running fine, there's just no browser to auto-open. Visit `http://localhost:3456` yourself.

56 changes: 51 additions & 5 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,52 @@ The infographic generation process is structured in three primary stages to opti
└───────────────────────────────┘
```

The web app exposes this as a bounded agent loop rather than a plain loading state:

| Loop phase | Responsibility | Stop / handoff condition |
| --- | --- | --- |
| Intake | Collect files, pasted text, URLs, mode, aspect ratio, and quality settings. | User starts generation. |
| Research | Ground claims against uploaded content, Google Search, and URL context. | Enough attributable facts exist for a plan. |
| Plan | Finalize exact text strings, layout hierarchy, colors, and prompt. | Structured `PrepareResult` parses successfully. |
| Render | Send the engineered prompt and reference images to the image model. | A valid image response is returned. |
| Review | Hold the current artifact for human approval or feedback. | Stop unless the user requests a targeted edit. |
| Refine | Apply one focused edit using the current image and chat history. | Return to Review with a new revision. |

This contract is represented in the app as `AgentLoopState` (`src/types.ts`) and updated by `useInfographicFlow.ts`. It keeps a stable `sessionId`, a monotonic `turn`, a user-visible `goal`, a `stopRule`, phase statuses, and a state backend label. The browser build currently uses `stateBackend: "browser-local"` because the app has no server component and users bring an AI Studio Gemini API key. Chat history, current image state, and generated revisions are preserved in React state and IndexedDB.

### Enterprise Interactions API Adapter

For Gemini Enterprise Agent Platform deployments, the same loop can be backed by the stateful Gemini Interactions API instead of browser-local chat history. That adapter should persist the returned `interaction.id` and pass it as `previousInteractionId` on the next HITL turn:

```typescript
import { GoogleGenAI } from '@google/genai';

const ai = new GoogleGenAI({
enterprise: {
project: 'your-project-id',
location: 'global',
},
});

const turn1 = await ai.interactions.create({
model: 'gemini-3.5-flash',
input: 'Research and plan an infographic from the supplied source package.',
store: true,
tools: [{ googleSearch: {} }, { urlContext: {} }],
systemInstruction: prepareSystemInstruction,
});

const turn2 = await ai.interactions.create({
model: 'gemini-3.5-flash',
input: 'Apply the user review: make the title larger.',
previousInteractionId: turn1.id,
store: true,
systemInstruction: refineSystemInstruction,
});
```

Interactions require the unified Google Gen AI SDK (`@google/genai >= 2.0.0` or `google-genai >= 2.0.0`). Legacy packages such as `@google/generative-ai`, `google-generativeai`, `@google-cloud/vertexai`, and `google-cloud-aiplatform` are unsupported for Interactions. Legacy models such as `gemini-2.0-*` and `gemini-1.5-*` are deprecated and unsupported for Interactions. Parameters like `tools`, `systemInstruction`, and generation settings are turn-scoped, so the adapter must send them with every interaction request. Keep the repo's model contract intact: `gemini-3.5-flash` for research/planning and `gemini-3.1-flash-lite-image` for web-app image rendering.

---

## 2. Agent 1: Prepare (Content Analysis & Layout Planning)
Expand Down Expand Up @@ -189,7 +235,7 @@ const response = await client.models.generateContent('gemini-3.1-flash-lite-imag
},
imageConfig: {
aspectRatio: '16:9',
resolution: '1024x576', // Configurable: 1K-4K
resolution: '1024x576', // Web app choices: 0.5K, 1K, or 2K
},
thinkingConfig: {
thinkingLevel: 'HIGH', // For debugging
Expand All @@ -200,15 +246,15 @@ const response = await client.models.generateContent('gemini-3.1-flash-lite-imag

### Model Selection

- **Primary**: `gemini-3.1-flash-lite-image` (default, 3-5 seconds, good balance)
- **Alternative**: `gemini-3.1-flash-image` (higher quality, slower, 8-12 seconds)
- **Performance**: Flash Lite produces quality output 40% faster than Flash
- **Web app image model**: `gemini-3.1-flash-lite-image` only, for predictable latency and tool compatibility.
- **Portable skill option**: `gemini-3.1-flash-image` may be used by skill/CLI workflows that explicitly choose quality over latency.
- **Analysis model**: `gemini-3.5-flash` only, because search and URL-context tools belong in the research/planning phase.

### Key Capabilities

1. **Native Image Generation**
- Renders PNG images at specified aspect ratios and resolutions
- Supports 1K (1024x576), 2K (2048x1152), and 4K (4096x2304) resolutions
- Web app exposes 0.5K, 1K, and 2K only; 0.5K maps to the model's supported 1K request size

2. **Layout Enforcement**
- Follows step-by-step spatial instructions from Agent 1
Expand Down
4 changes: 0 additions & 4 deletions docs/assets/create-step.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
14 changes: 10 additions & 4 deletions docs/learnings.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ The SDK types do not always support newer parameters. Use the `as any` cast patt

### Model Capabilities & Thinking Levels
- **`gemini-3.5-flash`** (default orchestrator): Supports detailed reasoning. Configured with `thinkingConfig: { thinkingLevel: 'HIGH' }` for layout planning.
- **`gemini-3.1-flash-lite-image`** (default image generator/refiner): Optimized for faster image generation (3-5 seconds) with native image generation and refinement. This is the recommended model for most use cases due to its speed. Supports thinking config for streaming thoughts but **rejects `thinkingLevel: 'LOW'`** with a 400 error. It must be configured with `thinkingLevel: 'HIGH'` to return thought streams properly.
- **`gemini-3.1-flash-lite-image`** (required web-app image generator/refiner): Optimized for faster image generation (3-5 seconds) with native image generation and refinement. The web app keeps this model fixed for predictable latency and tool compatibility. Supports thinking config for streaming thoughts but **rejects `thinkingLevel: 'LOW'`** with a 400 error. It must be configured with `thinkingLevel: 'HIGH'` to return thought streams properly.
- **Behavior**: Consistently generates valid PNG images at any aspect ratio
- **Limitations**: Slightly lower fidelity on text rendering compared to Flash (but still acceptable for most infographics)
- **Recommendation**: Use for rapid iteration and user-facing workflows
- **`gemini-3.1-flash-image`** (alternative for quality-focused workflows): Provides enhanced quality for complex infographics with more accurate text rendering and visual fidelity. Use this when fast iteration is less important than output fidelity (8-12 seconds per generation).
- **Recommendation**: Use for rapid iteration and all web-app image workflows
- **`gemini-3.1-flash-image`** (portable skill quality option only): Provides enhanced quality for skill/CLI workflows where the caller explicitly chooses output fidelity over latency. Do not expose this model in the web app.
- **Thinking Part Filtering**: When thinking is enabled, the model returns thoughts alongside the regular text parts. Always filter out the thought parts to parse the output JSON or text correctly:
```typescript
const parts = response.candidates?.[0]?.content?.parts || [];
Expand Down Expand Up @@ -83,6 +83,12 @@ The SDK types do not always support newer parameters. Use the `as any` cast patt
- **Solution**: Adopt IndexedDB via `localforage` for persistent storage of infographic history. This bypasses the 5MB storage limit and supports large image blobs.
- **Migration Safeguard**: Ensure a fallback schema merge when loading settings, preventing stale configurations from breaking the application when default models or settings are updated.

### Browser State vs Enterprise Interactions State
- **Context:** The app is a static, browser-only SPA where users supply AI Studio Gemini API keys, but the portable/enterprise agent story also needs durable multi-turn state and HITL iteration.
- **Learning:** Keep the web app's loop state explicit and local (`AgentLoopState`, chat history, current image, IndexedDB revisions). Do not call the Gemini Enterprise Interactions API directly from the browser path unless the product adds an Enterprise auth/runtime layer.
- **Evidence:** `src/hooks/useInfographicFlow.ts` updates loop phases and turn state locally; `docs/architecture.md` documents the `previousInteractionId` adapter for Enterprise Agent Platform.
- **Use next time:** Preserve the same intake -> research -> plan -> render -> review -> refine contract across app, CLI skill, and Enterprise adapters, but choose the state backend that matches the runtime.

---

## 4. Design System & UX Decisions
Expand Down Expand Up @@ -304,7 +310,7 @@ This ordering can reduce prompt processing cost by 10-20% on repeated requests.
- **Thought streams**: Returns thoughts when `includeThoughts: true`, must filter in parsing
- **Image output**: Requires `responseModalities: ['TEXT', 'IMAGE']` in config
- **Aspect ratios**: Supports any aspect ratio (16:9, 1:1, 21:9, etc.) via `imageConfig`
- **Resolutions**: Supports 1K-4K; higher resolutions = longer generation (~2x slower at 4K)
- **Resolutions**: The web app exposes only 0.5K, 1K, and 2K. For `gemini-3.1-flash-lite-image`, 0.5K and higher web choices map to the model's supported 1K request size.
- **Text fidelity**: Good but not perfect; prefer verbatim quotes from Agent 1
- **Generation time**: 3-5 seconds typical, up to 15 seconds on overload

Expand Down
3 changes: 2 additions & 1 deletion skill/infographic-agent/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ npx infographic-agent "Top 5 programming languages in 2026"
The skill runs a fast, browser-less two-agent pipeline powered by Gemini:

1. **Research & Plan (`gemini-3.5-flash`)**: Grounds the topic with Google Search and engineers a precise, text-accurate prompt.
2. **Render (`gemini-3.1-flash-lite-image`)**: Renders the prompt directly into a high-quality, text-accurate infographic PNG.
2. **Render (`gemini-3.1-flash-lite-image` by default)**: Renders the prompt directly into a high-quality, text-accurate infographic PNG. The portable skill can opt into `gemini-3.1-flash-image` via `--image-model` for quality-focused runs.

## 🛠️ CLI Flags & Options

Expand All @@ -53,6 +53,7 @@ All flags are fully compatible between the Python script and the `npx` execution
| `--mode` | `-m` | Preset layout and style theme. | `data-story`<br>Choices: `classroom`, `custom`, `data-story`, `executive-summary`, `quick-slide`, `technical-deep-dive` |
| `--aspect` | `-a` | Aspect ratio of the generated infographic image. | `9:16`<br>Choices: `1:1`, `1:4`, `3:4`, `4:3`, `9:16`, `16:9` |
| `--instructions` | `-i` | Custom layout, design, or brand color rules. | `""` |
| `--image-model` | *None* | Image model for the portable skill. | `gemini-3.1-flash-lite-image`<br>Choices: `gemini-3.1-flash-lite-image`, `gemini-3.1-flash-image` |
| `--no-research` | *None* | Skip the research agent and generate directly (faster, no web search). | *Flag* |
| `--yes` | `-y` | Non-interactive execution (no refine loop, best for CI/agents). | *Flag* |
| `--setup` | *None* | Launch the interactive key onboarding walkthrough, then exit. | *Flag* |
Expand Down
Loading
Loading