MKE.dev

AI-Powered Civic Intelligence Platform for Milwaukee

Gemini 3 Hackathon Entry | Deadline: February 10, 2026

Overview

MKE.dev democratizes access to Milwaukee's civic development information by transforming complex zoning codes, financial incentives, and regulatory data into a single, intuitive AI-powered interface.

Key Features

Zoning Interpreter Agent - AI-powered zoning assistant using Gemini function calling with RAG (22 tools)
AI Site Visualizer - Transform photos into architectural renderings with Gemini 3 Pro Image
Generative UI Cards - Rich interactive cards for homes, parcels, zoning info, and properties
Map Measurement Tool - Draw polygons to measure area (sq ft/acres) and perimeter on any site
Interactive 3D Map - Mapbox GL JS with 2D/3D toggle and 8 Milwaukee ESRI data layers
File Search RAG - 42 documents across 5 stores (zoning codes, area plans, policies, incentives)
Conversation History - Persistent chat with search, starring, and PDF report generation
Document Analyzer - Upload civic documents (site plans, floor plans, narratives) for AI classification, zoning enrichment, and compliance analysis
Permit & Design Tools - Search Milwaukee permit forms, design guidelines, and get project-specific recommendations
Parcel Lookup - Look up lot size, owner, assessed value, and zoning from ESRI parcels layer
Homes MKE Integration - Search city-owned homes for sale with detailed property cards
Vacant Lots Layer - Browse city-owned vacant lots from Strong Neighborhoods program
Multi-Page PDF Reports - Export conversations with tables and compliance data via Hybiscus API
High-Performance Tiles - PMTiles (313,000+ features) for instant map rendering

Target Users

Homeowners exploring ADUs or renovations
Developers scouting opportunities and analyzing incentives
Architects verifying dimensional standards
City Planners reducing repetitive inquiries
Visually Impaired Residents seeking independent access to property information

Tech Stack

Layer	Technology
Frontend	Next.js 15 (App Router, React 19)
UI Components	RetroUI (neobrutalist design)
Styling	Tailwind CSS 3.4
Backend	Convex (real-time database)
Auth	Clerk (Google OAuth + email)
Maps	Mapbox GL JS + Milwaukee ESRI ArcGIS
Tiles	PMTiles on Cloudflare R2
AI/LLM	Google Gemini 3 (Flash + Pro)
Voice (Alpha)	Gemini Live API (bidirectional audio)
Vision	Gemini 3 Pro Image + 2.5 Flash (with fallback)
Generative UI	CopilotKit
Reports	Hybiscus PDF API
Monitoring	Sentry (error + performance)

Architecture

System Overview

graph TB
    subgraph User["User Interface"]
        Chat["Chat Panel"]
        Upload["Document Upload"]
        Map["Interactive 3D Map"]
        Viz["Site Visualizer"]
    end

    subgraph NextJS["Next.js 15 / React 19"]
        CK["CopilotKit<br/>24 Generative UI Cards"]
        MapGL["Mapbox GL JS<br/>8 ESRI Layers + PMTiles"]
        VizUI["Mask Painting<br/>Before/After Compare"]
    end

    subgraph Convex["Convex Backend"]
        Agent["Zoning Interpreter Agent<br/>22 Function-Calling Tools"]
        DocPipe["Document Pipeline<br/>Classify → Enrich → Analyze"]
        ImgGen["Image Generation<br/>+ Site Analysis"]
        Cache["Context Cache"]
    end

    subgraph Gemini3["Gemini 3 Models"]
        Flash["Gemini 3 Flash<br/>Function Calling + Thinking"]
        ProImg["Gemini 3 Pro Image<br/>Architectural Rendering"]
        FS["Gemini File Search<br/>42 Documents / 5 Stores"]
    end

    subgraph Data["Milwaukee Data Sources"]
        ESRI["ESRI ArcGIS REST<br/>Zoning, Parcels, TIF,<br/>Historic, Opportunity Zones"]
        Mapbox["Mapbox Geocoding"]
        PMT["PMTiles on R2<br/>313K Features"]
    end

    Chat --> CK
    Upload --> CK
    Map --> MapGL
    Viz --> VizUI

    CK --> Agent
    CK --> DocPipe
    VizUI --> ImgGen

    Agent -->|"Function Calling<br/>MINIMAL→HIGH Thinking"| Flash
    Agent -->|"RAG Retrieval"| FS
    DocPipe -->|"MINIMAL Thinking<br/>Classification"| Flash
    DocPipe -->|"HIGH Thinking<br/>Compliance Analysis"| Flash
    DocPipe -->|"Zoning Code Search"| FS
    ImgGen -->|"Image Generation"| ProImg
    ImgGen -->|"Vision Analysis"| Flash

    Agent --> ESRI
    Agent --> Mapbox
    DocPipe --> ESRI
    DocPipe --> Mapbox
    MapGL --> PMT
    MapGL --> ESRI

    style Gemini3 fill:#4285F4,color:#fff,stroke:#1a73e8
    style Flash fill:#34A853,color:#fff
    style ProImg fill:#EA4335,color:#fff
    style FS fill:#FBBC04,color:#000

Gemini 3 Feature Map

graph LR
    subgraph Capabilities["Gemini 3 Capabilities Used"]
        FC["Function Calling"]
        TH["Thinking Levels"]
        FSR["File Search RAG"]
        IG["Image Generation"]
        MM["Multimodal Input"]
        SJ["Structured JSON Output"]
    end

    subgraph Features["MKE.dev Features"]
        ZA["Zoning Agent<br/>22 Tools"]
        DC["Document Classification<br/>10 Types"]
        CA["Compliance Analysis<br/>Dimensional + Use"]
        AV["Architectural Visualization<br/>Zoning-Constrained"]
        SA["Site Photo Analysis<br/>Development Potential"]
        PR["Permit Recommendations<br/>Project-Specific"]
    end

    FC --> ZA
    FC --> PR
    TH -->|"MINIMAL"| DC
    TH -->|"HIGH"| CA
    TH -->|"MEDIUM"| SA
    FSR --> ZA
    FSR --> CA
    IG --> AV
    MM -->|"PDF + Image"| DC
    MM -->|"Photo"| SA
    SJ --> CA
    SJ --> DC

    style Capabilities fill:#4285F4,color:#fff,stroke:#1a73e8
    style FC fill:#34A853,color:#fff
    style TH fill:#34A853,color:#fff
    style FSR fill:#34A853,color:#fff
    style IG fill:#EA4335,color:#fff
    style MM fill:#FBBC04,color:#000
    style SJ fill:#FBBC04,color:#000

Document Analysis Pipeline

flowchart LR
    A["📄 Upload<br/>PDF or Image"] --> B["Gemini 3 Flash<br/>MINIMAL Thinking"]
    B -->|"Classify"| C["Document Type<br/>+ Extracted Data"]
    C --> D["Auto-Enrich"]
    D -->|"Geocode"| E["Mapbox API"]
    D -->|"Zoning"| F["ESRI GIS"]
    D -->|"Parcel"| G["ESRI Parcels"]

    E & F & G --> H["Gemini 3 Flash<br/>HIGH Thinking"]
    H -->|"+ File Search RAG<br/>Zoning Code"| I["Compliance Report"]

    I --> J["✅ Dimensional Compliance"]
    I --> K["✅ Use Compatibility"]
    I --> L["⚠️ Variance Needs"]
    I --> M["📊 Risk Assessment"]

    style B fill:#34A853,color:#fff
    style H fill:#34A853,color:#fff
    style I fill:#4285F4,color:#fff

Site Visualizer Pipeline

flowchart LR
    A["📸 Capture<br/>Map or Street View"] --> B["🎨 Paint Mask<br/>Select Area"]
    B --> C["💬 Describe<br/>What to Build"]
    C --> D["Gemini 3 Pro Image<br/>Image Generation"]

    E["Zoning Context"] -->|"Setbacks, Height,<br/>FAR, Lot Coverage"| D
    F["Parcel Data"] -->|"Lot Dimensions,<br/>Address"| D

    D --> G["🏗️ Photorealistic<br/>Rendering"]
    G --> H["Before / After<br/>Comparison View"]

    I["Gemini 3 Flash<br/>MEDIUM Thinking"] -->|"Site Photo<br/>Analysis"| J["📋 Development<br/>Potential Report"]

    style D fill:#EA4335,color:#fff
    style I fill:#34A853,color:#fff
    style G fill:#4285F4,color:#fff

Gemini Model Usage

Model	Feature	Gemini 3 Capability	Thinking Level
Gemini 3 Flash	Zoning Interpreter Agent	Function Calling (22 tools)	MINIMAL → HIGH
Gemini 3 Flash	Document Classification	Multimodal (PDF/Image)	MINIMAL
Gemini 3 Flash	Compliance Analysis	Structured JSON + File Search	HIGH
Gemini 3 Flash	Site Photo Analysis	Multimodal Vision	MEDIUM
Gemini 3 Pro Image	Site Visualizer	Image Generation + Inpainting	--
Gemini 3 Pro Image	Zoning-Aware Rendering	Constrained by setbacks, height, FAR	--
Gemini 3 Flash	File Search RAG	Retrieval over 42 docs	--
Gemini 3 Flash	Context Caching	1M context window	Configurable

Quick Start

Prerequisites

Node.js 20+
pnpm 9+
Mapbox account (free tier works)
Clerk account (free tier works)
Convex account (free tier works)

Installation

# Clone the repository
git clone https://github.com/tmoody1973/mkedev.git
cd mkedev

# Install dependencies
pnpm install

# Copy environment template
cp .env.local.example apps/web/.env.local

Configure Environment

Edit apps/web/.env.local with your API keys:

# Required
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_test_xxx
CLERK_SECRET_KEY=sk_test_xxx
NEXT_PUBLIC_MAPBOX_TOKEN=pk.xxx

# Optional (for full features)
CONVEX_DEPLOYMENT=dev:xxx
NEXT_PUBLIC_CONVEX_URL=https://xxx.convex.cloud
GEMINI_API_KEY=xxx
FIRECRAWL_API_KEY=xxx

Development

# Start Next.js dev server
pnpm dev

# In a separate terminal, start Convex (if using)
cd apps/web && npx convex dev

Visit http://localhost:3000

Project Structure

mkedev/
├── apps/
│   ├── web/                    # Next.js 15 application
│   │   ├── src/
│   │   │   ├── app/            # App Router pages
│   │   │   ├── components/     # React components
│   │   │   │   ├── chat/       # Chat panel components
│   │   │   │   ├── copilot/    # Generative UI cards
│   │   │   │   ├── map/        # Map and layer components
│   │   │   │   ├── shell/      # App shell and header
│   │   │   │   └── ui/         # RetroUI components
│   │   │   ├── contexts/       # React contexts (MapContext with 3D support)
│   │   │   ├── hooks/          # Custom hooks
│   │   │   └── lib/voice/      # Gemini Live voice integration
│   │   ├── convex/             # Convex schema & functions
│   │   │   ├── agents/         # Zoning Interpreter Agent
│   │   │   ├── documents/      # Document analyzer pipeline
│   │   │   └── ingestion/      # RAG & File Search Stores
│   │   └── scripts/            # Setup scripts
│   └── agents/                 # Standalone agent experiments
├── packages/
│   └── tile-builder/           # ESRI → PMTiles pipeline
├── agent-os/                   # Specs and documentation
│   ├── product/                # Mission, roadmap, tech stack
│   └── specs/                  # Feature specifications
└── data/                       # PDF documents for RAG
    ├── zoning-code-pdfs/       # Milwaukee Zoning Code (12 PDFs)
    └── plans/                  # City area plans

Voice Interface (Alpha)

MKE.dev includes an experimental voice interface powered by Gemini Live API with bidirectional audio and function calling. Voice and text conversations share the same chat.

Map Layers

MKE.dev integrates 8 Milwaukee GIS data layers:

Layer	Source	Description
Zoning Districts	ESRI Layer 11	Color-coded by category (residential, commercial, industrial, mixed-use)
Parcels	ESRI Layer 2	Clickable parcels with property info
TIF Districts	ESRI Layer 8	Tax Increment Financing zones
Opportunity Zones	ESRI Layer 9	Federal opportunity zone boundaries
Historic Districts	ESRI Layer 17	Historic preservation areas
ARB Areas	ESRI Layer 1	Architectural Review Board districts
City-Owned	ESRI MapServer	Municipal properties
Vacant Lots	Strong Neighborhoods	City-owned vacant lots with disposition status

Layers are served via PMTiles for optimal performance (313,000+ features).

Map Measurement Tool

Click the ruler button on the map to enter measurement mode:

Draw polygons by clicking to place vertices on any parcel or site
Double-click to finish the polygon and see results
Area displayed in square feet and acres
Perimeter displayed in feet
Press Escape or click the X button to exit

Uses a custom GeoJSON drawing system with Turf.js for geodesic-accurate calculations.

Zoning Interpreter Agent

The AI-powered Zoning Interpreter Agent helps users understand Milwaukee zoning requirements through natural conversation.

Agent Tools (22 Total)

Tool	Description
`geocode_address`	Convert street addresses to coordinates via Mapbox
`query_zoning_at_point`	Get zoning district + overlays from Milwaukee ESRI
`lookup_parcel`	Look up parcel details (lot size, owner, assessed value)
`calculate_parking`	Calculate required parking spaces by use type
`query_zoning_code`	RAG search against 12 zoning code PDFs
`query_area_plans`	Search neighborhood plans for development context
`query_incentives`	Search housing assistance programs (STRONG, ARCH, etc.)
`search_homes_for_sale`	Find city-owned homes with filters
`get_home_details`	Get full property info, images, listing URL
`search_commercial_properties`	Find commercial real estate
`get_commercial_property_details`	Get commercial property details
`search_development_sites`	Find development opportunities
`get_development_site_details`	Get development site details
`search_vacant_lots`	Find city-owned vacant lots with filters
`get_vacant_lot_details`	Get vacant lot info with parcel enrichment
`search_permit_forms`	Search Milwaukee permit forms and applications
`search_design_guidelines`	Search building design guidelines
`recommend_permits_for_project`	Get recommended permits for a project
`get_permit_form_details`	Get full details about a specific permit form
`get_guideline_details`	Get full details about a design guideline
`analyze_document`	Upload and analyze civic documents for zoning compliance
`analyze_site_photo`	Analyze site photos for development potential

Example Queries

"What zoning district is 500 N Water St in?"
→ C9F(A) - Downtown Office and Service

"How many parking spaces for a 5000 sq ft restaurant at that address?"
→ 0 required (downtown), 4 bicycle spaces required

"Show me 3-bedroom homes for sale"
→ Returns HomesListCard with matching properties

"What are the setback requirements for RS6 residential?"
→ Front: Average, Side: 3-6 ft, Rear: 20 ft (with code citations)

"Show me vacant lots in Harambee"
→ Returns VacantLotsListCard with available properties

"What's the lot size at 1021 S 21st Street?"
→ Looks up parcel data including owner, assessed value, zoning

"What permits do I need for a restaurant buildout?"
→ Returns PermitRecommendationsCard with required forms and fees

RAG Document Corpus

42 documents indexed across 5 Gemini File Search Stores:

Store	Documents	Content
mkedev-zoning-codes	12 PDFs	CH295 Subchapters 1-11 + Use Tables
mkedev-area-plans	13 PDFs	Neighborhood development plans
mkedev-policies	2 PDFs	City policies and guidelines
Milwaukee Planning Documents	7 docs	Comprehensive planning docs
mkedev-incentives	8 docs	Housing assistance programs (STRONG, ARCH, etc.)

Generative UI Cards

24 rich interactive card types render in chat for structured data:

Card Type	Use Case
`ZoneInfoCard`	Zoning district summary with category and overlays
`ParcelCard`	Full parcel info with address, zoning, permitted uses
`ParcelAnalysisCard`	Deep parcel analysis with development potential
`CodeCitationCard`	Zoning code excerpts with PDF viewer links
`IncentivesSummaryCard`	Financial incentives and assistance programs
`AreaPlanContextCard`	Neighborhood plan context and goals
`PermitProcessCard`	Permit process overview and requirements
`OpportunityListCard`	Development opportunities in an area
`HomeCard`	Detailed home listing with images and Street View
`HomesListCard`	List of homes with quick select
`CommercialPropertyCard`	Commercial property details
`CommercialPropertiesListCard`	List of commercial properties
`DevelopmentSiteCard`	Development opportunity details
`DevelopmentSitesListCard`	List of development sites
`VacantLotCard`	Vacant lot details with Street View and visualize
`VacantLotsListCard`	List of vacant lots with status badges
`PermitFormsListCard`	Search results for permit forms
`PermitRecommendationsCard`	Project-specific permit recommendations
`PermitFormDetailsCard`	Full permit form details with fields and fees
`DesignGuidelinesListCard`	Search results for design guidelines
`DesignGuidelineDetailsCard`	Full design guideline standards
`DocumentUploadCard`	Upload civic documents for AI analysis
`ComplianceReportCard`	Zoning compliance analysis with dimensional tables
`SiteAnalysisCard`	AI vision site assessment for development potential

Document Analyzer

Upload civic documents (PDFs, images) for AI-powered classification and zoning compliance analysis:

Pipeline

Upload - Drag & drop or click to upload documents via the paperclip button in chat
Classify - Gemini 3 Flash identifies document type (site plan, floor plan, elevation, narrative, variance application, etc.) and extracts structured data
Enrich - Auto-geocodes extracted addresses and fetches zoning district, overlays, and parcel data from Milwaukee ESRI
Analyze - Gemini 3 Pro performs deep compliance analysis against Milwaukee zoning code with dimensional requirements, setback checks, and condition assessment
Report - Results displayed as interactive ComplianceReportCard with status indicators

Supported Document Types

Type	Examples
Site Plan	Building footprint, setbacks, parking layout
Floor Plan	Interior layout with dimensions
Elevation	Building facade views, height, materials
Project Narrative	Written description of proposed development
Variance Application	Request to deviate from zoning standards
Conditional Use	Special use permit applications
Traffic Study	Traffic impact analysis
Environmental Assessment	Phase I/II environmental reports
Survey	ALTA, boundary, or topographic surveys

AI Site Visualizer

Transform photos into architectural renderings with Gemini 3 Pro Image:

Features

Map Screenshot Capture - Purple camera button captures current map view
Street View Integration - Capture and visualize from Street View modal
Mask Painting - Brush/eraser tools to mark areas for AI modification
Zoom & Pan - Scroll to zoom (0.5x-5x), space+drag to pan for precise masking
Screenshot Gallery - Persistent gallery of captured images (survives page refresh)
Zoning-Aware Prompts - Generation considers Milwaukee zoning constraints
Scale-Accurate Design - Uses lot dimensions for realistic proportions
Before/After Comparison - Side-by-side view with synced zoom
Model Fallback - Automatic fallback to Gemini 2.5 Flash Image if primary fails

Example Prompts

"Add a 4-story mixed-use building with retail on ground floor"
"Transform this into a community park with walking paths"
"Replace this parking lot with townhomes"
"Design a modern bungalow with nice landscaping"

Environment Variables

Variable	Description	Required
`NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY`	Clerk publishable key	Yes
`CLERK_SECRET_KEY`	Clerk secret key	Yes
`NEXT_PUBLIC_MAPBOX_TOKEN`	Mapbox access token	Yes
`CONVEX_DEPLOYMENT`	Convex deployment ID	No*
`NEXT_PUBLIC_CONVEX_URL`	Convex cloud URL	No*
`NEXT_PUBLIC_PMTILES_URL`	PMTiles R2 URL	No**
`GEMINI_API_KEY`	Google Gemini API key	Yes***
`NEXT_PUBLIC_GOOGLE_MAPS_API_KEY`	Google Maps API key (Street View)	No
`HYBISCUS_API_KEY`	Hybiscus PDF API key	No
`FIRECRAWL_API_KEY`	Firecrawl API key	No

* Required for database features ** Falls back to ESRI REST API if not set *** Required for AI, voice, and visualizer features

Development Roadmap

Week 1: Foundation (Complete)

Week 2: AI Agent & Features (Complete)

Week 3: Advanced Features (Complete)

Week 4: Document Analyzer & Polish (In Progress)

Scripts

# Development
pnpm dev                  # Start Next.js dev server
pnpm lint                 # Run ESLint
pnpm format               # Run Prettier
pnpm typecheck            # Run TypeScript check

# Convex (in apps/web directory)
npx convex dev            # Start Convex dev server
npx convex run agents/zoning:chat '{"message": "..."}'  # Test agent

# RAG Setup (one-time)
npx tsx scripts/setup-file-search-stores.ts  # Upload PDFs to Gemini
npx tsx scripts/upload-incentives.ts          # Upload incentives docs
npx convex run ingestion/fileSearchStores:syncStoresFromGemini  # Register stores

# Data Sync
npx convex run vacantLots:quickSync           # Sync vacant lots from ESRI

# Tile Building (requires tippecanoe)
pnpm --filter tile-builder export    # Export ESRI → GeoJSON
pnpm --filter tile-builder build     # Build PMTiles
pnpm --filter tile-builder upload    # Upload to R2

Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'feat: add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

License

MIT License - see LICENSE for details.

Acknowledgments

City of Milwaukee - Open GIS data via Milwaukee Maps
RetroUI - Neobrutalist component library
Mapbox - Map rendering and interaction
Anthropic - Claude AI assistance in development

Team

Built for the Gemini 3 Hackathon by Tarik Moody.

MKE.dev — Making Milwaukee's civic development accessible to everyone

Name		Name	Last commit message	Last commit date
Latest commit History 172 Commits
.claude		.claude
.github/workflows		.github/workflows
agent-os		agent-os
apps		apps
data		data
docs		docs
packages		packages
product-plan		product-plan
research		research
.env.local.example		.env.local.example
.eslintrc.js		.eslintrc.js
.gitignore		.gitignore
.mcp.json		.mcp.json
.npmrc		.npmrc
.prettierignore		.prettierignore
.prettierrc		.prettierrc
CLAUDE.md		CLAUDE.md
LICENSE		LICENSE
README.md		README.md
devlog.md		devlog.md
mkedev-logo.af		mkedev-logo.af
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.base.json		tsconfig.base.json
vercel.json		vercel.json

Folders and files

Latest commit

History

Repository files navigation

MKE.dev

Overview

Key Features

Target Users

Tech Stack

Architecture

System Overview

Gemini 3 Feature Map

Document Analysis Pipeline

Site Visualizer Pipeline

Gemini Model Usage

Quick Start

Prerequisites

Installation

Configure Environment

Development

Project Structure

Voice Interface (Alpha)

Map Layers

Map Measurement Tool

Zoning Interpreter Agent

Agent Tools (22 Total)

Example Queries

RAG Document Corpus

Generative UI Cards

Document Analyzer

Pipeline

Supported Document Types

AI Site Visualizer

Features

Example Prompts

Environment Variables

Development Roadmap

Week 1: Foundation (Complete)

Week 2: AI Agent & Features (Complete)

Week 3: Advanced Features (Complete)

Week 4: Document Analyzer & Polish (In Progress)

Scripts

Contributing

License

Acknowledgments

Team

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages