Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
55da742
Optimize LangGraph workflow: 68% token reduction + async execution
claude Nov 5, 2025
e451388
Apply workflow optimizations for local testing
claude Nov 5, 2025
c02c666
Add comprehensive optimization analysis for Connections Hub
bassem-elsodany Dec 25, 2025
115d753
Phase 1: Add optimized REST endpoints for connections list management
bassem-elsodany Dec 25, 2025
10109ab
Phase 1: Update frontend services to use lightweight connections endp…
bassem-elsodany Dec 25, 2025
a132daa
Phase 2: Add virtualization and memoization optimizations for connect…
bassem-elsodany Dec 25, 2025
b78d1da
Add comprehensive optimization progress report
bassem-elsodany Dec 25, 2025
8cad98f
Phase 3: Replace connections table with virtualized ConnectionsList c…
bassem-elsodany Dec 25, 2025
9b13d64
Phase 4: CSS cleanup and variable extraction
bassem-elsodany Dec 25, 2025
a7bd9b8
Add final optimization completion document
bassem-elsodany Dec 25, 2025
b6a56b7
Install react-window dependencies for virtualized connections list
bassem-elsodany Dec 25, 2025
ba06349
Add comprehensive testing guide for optimization work
bassem-elsodany Dec 25, 2025
2427440
Add memory cleanup for decrypted connection objects
bassem-elsodany Dec 25, 2025
a89469d
Fix: Include username and proper environment in lightweight connectio…
bassem-elsodany Dec 25, 2025
88d180c
CRITICAL FIX: Decrypt connection data to get actual username and envi…
bassem-elsodany Dec 25, 2025
75e6b72
Debug: Add logging and fix data structure issues in lightweight conne…
bassem-elsodany Dec 25, 2025
04547b3
Fix: Normalize environment field from domain URL to production/sandbo…
bassem-elsodany Dec 25, 2025
f9f522b
Revert complex environment normalization and add debug logging
bassem-elsodany Dec 25, 2025
766420a
Add INFO level logging to see actual decrypted connectionData structure
bassem-elsodany Dec 25, 2025
3246317
CRITICAL FIX: Set master key in connection_service before decryption …
bassem-elsodany Dec 25, 2025
640f3ba
Clean up debug logging from connections endpoint
bassem-elsodany Dec 25, 2025
e88ebec
Feature: Add connection disconnect confirmation when navigating from …
bassem-elsodany Dec 25, 2025
96cb2d8
Add debug logging to troubleshoot modal not showing
bassem-elsodany Dec 25, 2025
10545f1
Fix: Close settings menu before showing disconnect modal
bassem-elsodany Dec 25, 2025
6847c48
Add aggressive debugging to find why modal isn't showing
bassem-elsodany Dec 25, 2025
23e18ac
Fix: Use Mantine modals.openConfirmModal for reliable modal display
bassem-elsodany Dec 25, 2025
3646736
Fix TypeScript error: console.log in JSX returns void
bassem-elsodany Dec 25, 2025
e7a820a
Fix: Use native window.confirm() for disconnect confirmation
bassem-elsodany Dec 25, 2025
9f43c4a
Feature: Implement proper state-based disconnect confirmation modal
bassem-elsodany Dec 25, 2025
c8a4690
Clean up: Remove debug console.log statements
bassem-elsodany Dec 25, 2025
f66ac95
Phase 1-3: Apex Tab Complete Implementation - Code Executor, Library,…
bassem-elsodany Dec 25, 2025
3c28b58
Phase 4: Metadata Browser - Apex Classes and Triggers
bassem-elsodany Dec 25, 2025
e8d0611
Fix: Enable Create Apex button and remove disabled buttons
bassem-elsodany Dec 25, 2025
bb56723
Redesign Create Apex Modal with Professional Two-Column Layout
bassem-elsodany Dec 25, 2025
7a3d379
Fix Apex test execution endpoints and replace toolingexecute with res…
bassem-elsodany Dec 25, 2025
4241e96
Refactor: Rename saved_apex_uuid to uuid field in saved_apex collection
bassem-elsodany Dec 25, 2025
bea1bd8
Fix: Use Salesforce API v64.0 from settings instead of hardcoded vers…
bassem-elsodany Dec 25, 2025
eba54f9
Fix: Use simple_salesforce SDK methods instead of manual REST API calls
bassem-elsodany Dec 25, 2025
10cc5f1
Fix: Use correct simple_salesforce SDK methods (toolingexecute)
bassem-elsodany Dec 25, 2025
fe2d60d
Fix: Use restful() with full Tooling API paths instead of toolingexec…
bassem-elsodany Dec 25, 2025
8081f7b
Fix: Use toolingexecute() instead of restful() for Tooling API calls
bassem-elsodany Dec 25, 2025
aa2dd72
Fix: Use params kwarg for proper SOQL query URL encoding
bassem-elsodany Dec 25, 2025
3a8aacf
Fix: Use toolingexecute with proper Content-Type header for JSON POST…
bassem-elsodany Dec 25, 2025
3d05e86
Fix: Use toolingexecute with json= kwarg for proper JSON request enco…
bassem-elsodany Dec 25, 2025
cdab45c
Fix: Use toolingexecute with GET and query parameters for Tooling API
bassem-elsodany Dec 26, 2025
4da58cd
Fix: Use restful() with json parameter for Tooling API POST requests
bassem-elsodany Dec 26, 2025
c15c0cd
Fix: Replace all Tooling API restful() calls with pure toolingexecute…
bassem-elsodany Dec 26, 2025
afd1fd7
Fix: Proper Tooling API request handling using SDK session
bassem-elsodany Dec 26, 2025
09dafcc
Fix: Use POST for executeAnonymous via SDK session
bassem-elsodany Dec 26, 2025
479920a
Fix: Use simple_salesforce SDK's restful() method for all Tooling API…
bassem-elsodany Dec 26, 2025
5ce99b7
Fix: Use GET method with params for executeAnonymous
bassem-elsodany Dec 26, 2025
8811182
Fix: Use manual URL encoding for executeAnonymous query parameter
bassem-elsodany Dec 26, 2025
e4769ba
Fix: Improve Apex execution error handling and UI feedback
bassem-elsodany Dec 26, 2025
673acee
Add: Debug logging for Apex execution to diagnose issues
bassem-elsodany Dec 26, 2025
4590d9e
Fix Apex execution result modal z-index and visibility issues
bassem-elsodany Dec 26, 2025
bbe4431
Fix modal rendering and visibility by moving modals outside overflow:…
bassem-elsodany Dec 26, 2025
ff2632e
Add Portal rendering and explicit opacity/backgroundColor styles to m…
bassem-elsodany Dec 26, 2025
aa1c7d4
Remove all filter and backdrop-filter CSS to fix modal visibility
bassem-elsodany Dec 26, 2025
5670e8a
Add debug logging to trace Apex execution modal rendering
bassem-elsodany Dec 26, 2025
23f6636
Replace test results modal with right-side panel like edit panel
bassem-elsodany Dec 26, 2025
3341a68
Fix: Move test results panel inside apex-list container for proper la…
bassem-elsodany Dec 26, 2025
518421f
Fix: Use correct uuid field as React key in renderSavedApexItem
bassem-elsodany Dec 26, 2025
6e04fe6
Fix: Move test results panel to Test Runner tab with proper layout
bassem-elsodany Dec 26, 2025
a5a2c01
Remove old execution modal comment - complete cleanup of replaced modal.
bassem-elsodany Dec 26, 2025
4712d81
Fix: Remove blur effect and React prop warning from loading modal.
bassem-elsodany Dec 26, 2025
29ce636
Remove: Delete loading modal completely - results show directly in pa…
bassem-elsodany Dec 26, 2025
b96496a
Add: Loading indicator in execution results panel during Apex execution.
bassem-elsodany Dec 26, 2025
ad89678
Add: Test button in edit panel with results showing on right side.
bassem-elsodany Dec 26, 2025
c3a5ef8
Fix: Update SavedApex interface and handlers to use 'uuid' instead of…
bassem-elsodany Dec 26, 2025
fbc0561
Fix: Use correct 'totalTime' field from Salesforce Apex execution res…
bassem-elsodany Dec 26, 2025
7bf01a2
Fix: Use Salesforce execution time instead of local timing calculation
bassem-elsodany Dec 26, 2025
75b3890
Fix: Reload saved Apex list after execution to reflect updated counters
bassem-elsodany Dec 26, 2025
462a7c9
Fix: Reload data from database when switching between Apex tabs
bassem-elsodany Dec 26, 2025
dcddadd
Fix: Use correct Tooling API endpoint for querying Apex classes and t…
bassem-elsodany Dec 26, 2025
428a5c2
Fix: Add graceful error handling for Classes and Triggers tabs
bassem-elsodany Dec 26, 2025
e591a93
Fix: Execute saved apex with proper execution count tracking
bassem-elsodany Dec 26, 2025
e808381
Fix: Correct error handling scope in executeSavedApex method
bassem-elsodany Dec 26, 2025
8c1732b
Fix: Use correct endpoint key 'savedApex' in executeSavedApex method
bassem-elsodany Dec 26, 2025
6e55305
Fix: Close execution modal when opening test panel to prevent duplica…
bassem-elsodany Dec 26, 2025
13ebb97
Feature: Add formatted debug_log output to Apex execution response
bassem-elsodany Dec 26, 2025
8464970
Fix: Use correct 'logs' field from Salesforce Tooling API response
bassem-elsodany Dec 26, 2025
77e537f
Fix: Standardize API response field names to snake_case and add enhan…
bassem-elsodany Dec 26, 2025
9ac2aab
Fix: Resolve Salesforce debug log issue and import error
bassem-elsodany Dec 26, 2025
012172c
Fix: Make Apex code editor fill available vertical space
bassem-elsodany Dec 26, 2025
16b54b4
Fix: Redesign Apex edit panel layout for better code visibility
bassem-elsodany Dec 26, 2025
d3ec574
Fix: Make Apex code editor dynamically fill available vertical space
bassem-elsodany Dec 26, 2025
a4f2e0d
Fix: Prevent Monaco Editor from expanding beyond container bounds
bassem-elsodany Dec 26, 2025
18087ab
Fix: Make button group visible below code editor
bassem-elsodany Dec 26, 2025
f55f2f8
Fix: Button group now visible below code editor
bassem-elsodany Dec 26, 2025
f17d267
Fix: Buttons now properly visible below code editor
bassem-elsodany Dec 26, 2025
0c50274
Fix: Monaco Editor now has dynamic height with maximum constraint
bassem-elsodany Dec 26, 2025
f06121f
Fix: Monaco Editor now fills all available vertical space dynamically
bassem-elsodany Dec 26, 2025
488eff2
Fix: Add flex: 1 to .apex-edit-code-section to enable proper space di…
bassem-elsodany Dec 26, 2025
6f8fd20
Fix: Ensure buttons remain visible while editor fills available space
bassem-elsodany Dec 26, 2025
4e69779
Fix: Monaco Editor now properly displays Apex code and fills availabl…
bassem-elsodany Dec 26, 2025
453fe73
Fix: Ensure apex-main fills the entire vertical and horizontal section
bassem-elsodany Dec 26, 2025
85d4d3d
Fix: Ensure editor wrapper has explicit height: 100% constraint
bassem-elsodany Dec 26, 2025
376f9a3
Add connection editing UI, i18n support, and Apex tab layout fixes
bassem-elsodany Sep 16, 2026
1750bf4
Reset Langfuse defaults to non-LAN placeholder values
bassem-elsodany Sep 16, 2026
46154bc
Fix pre-existing TypeScript errors blocking npm run build
bassem-elsodany Sep 16, 2026
65466de
Add CI workflow for backend import smoke-test and dashboard build
bassem-elsodany Sep 16, 2026
6fd7f25
Fix CI: use uv sync instead of pip install for backend deps
bassem-elsodany Sep 16, 2026
9b194e8
README accuracy pass and CI badge
bassem-elsodany Sep 16, 2026
3542b86
Resolve backend API URL at runtime instead of baking in localhost
bassem-elsodany Sep 16, 2026
4fd6ff6
Add multi-arch Docker image publishing to GHCR
bassem-elsodany Sep 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: CI

on:
push:
branches: [master]
pull_request:
branches: [master]

jobs:
backend:
name: Backend (FastAPI)
runs-on: ubuntu-latest
defaults:
run:
working-directory: backend
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v3
with:
version: "0.7.12"

- name: Set up Python
run: uv python install 3.11

- name: Install dependencies
run: uv sync --frozen

- name: Import smoke test
run: uv run python -c "import app.main"

dashboard:
name: Dashboard (React)
runs-on: ubuntu-latest
defaults:
run:
working-directory: dashboard
steps:
- uses: actions/checkout@v4

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "20"
cache: "npm"
cache-dependency-path: dashboard/package-lock.json

- name: Install dependencies
run: npm ci

- name: Build
run: npm run build
59 changes: 59 additions & 0 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Publish Docker images

on:
push:
branches: [master]
tags: ["v*"]
workflow_dispatch: {}

env:
REGISTRY: ghcr.io

jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
strategy:
matrix:
include:
- image: datapilot-backend
dockerfile: docker/Dockerfile.backend
- image: datapilot-dashboard
dockerfile: docker/Dockerfile.dashboard
steps:
- uses: actions/checkout@v4

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ github.repository_owner }}/${{ matrix.image }}
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=semver,pattern={{version}}
type=sha

- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ${{ matrix.dockerfile }}
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -176,3 +176,6 @@ tramp
# Local development
.local/
.cache/

# Graphify knowledge graph output (local tooling artifact)
graphify-out/
44 changes: 37 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# DataPilot - Advanced Salesforce Data Platform
<div align="center">
<img src="imgs/logo/logo-large.png" alt="DataPilot Logo" width="550">

[![CI](https://github.com/bassem-elsodany/datapilot/actions/workflows/ci.yml/badge.svg)](https://github.com/bassem-elsodany/datapilot/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Custom%20Non--Commercial-blue)](LICENSE)

<img src="https://img.shields.io/badge/🤖-AI%20POWERED%20QUERIES-FF6B6B?style=for-the-badge&logo=robot&logoColor=white" alt="AI Powered Queries"> <img src="https://img.shields.io/badge/📊-VISUAL%20SCHEMA-4ECDC4?style=for-the-badge&logo=chart&logoColor=white" alt="VisualSchema"> <img src="https://img.shields.io/badge/⚡-SMART%20SOQL%20EDITOR-00D4AA?style=for-the-badge&logo=lightning&logoColor=white" alt="SmartSOQLEditor">

</div>
Expand Down Expand Up @@ -85,6 +89,21 @@
- **Bidirectional Integration**: Seamless schema canvas integration
- **Multi-Hierarchy Results**: Advanced result visualization with 4-level nesting
- **Real-time Validation**: Instant error detection and correction
- **Inline Record Editing**: Edit field values directly in the results grid and save changes back to Salesforce
---

### **4. [Apex Code Management](docs/05-apex-code-management.md)**
**APEX DEVELOPMENT ENVIRONMENT**

- **Anonymous Apex Execution**: Run Apex code against your connected org and view debug logs
- **Saved Apex Snippets**: Create, edit, and re-run saved Apex code
- **Apex Class & Trigger Browser**: Load and inspect existing classes/triggers, compile packages and triggers
- **Test Execution**: Run Apex tests and view results, including compile-and-test workflows

---

**Note on data writes**: DataPilot can already modify org data and metadata through inline record editing and Apex execution (both are direct, deliberate user actions). The AI Query Assistant itself is currently **read-only** — its tools only search, describe, and query Salesforce data; it does not yet perform inserts/updates/deletes on your behalf. See the [Roadmap](#roadmap) for planned AI-driven data updates.

---


Expand Down Expand Up @@ -123,7 +142,7 @@ graph LR
- **[AI-Powered Query Assistant](docs/01-ai-powered-query-assistant.md)** - Natural language to SOQL conversion
- **[Interactive Schema Explorer](docs/02-interactive-schema-explorer.md)** - Visual metadata exploration
- **[Advanced SOQL Query Editor](docs/03-advanced-soql-query-editor.md)** - Intelligent code editor
- **[Apex Code Management](docs/05-apex-code-management.md)** - Apex development tools (Under Development)
- **[Apex Code Management](docs/05-apex-code-management.md)** - Apex development environment

---

Expand All @@ -141,6 +160,21 @@ graph LR
- **Integration Setup**: API and external system connections
- **Monitoring**: Usage monitoring

### **Key Configuration & Seeded Credentials**

The Docker Compose setup seeds default MongoDB credentials for local development. **Change these before exposing DataPilot beyond your own machine:**

| Variable | Where | Default | Notes |
|---|---|---|---|
| `MONGO_USER` / `MONGO_PASS` | `docker/docker-compose.yml`, `docker/environment-configs/backend.env` | `datapilot` / `datapilot123` | Change immediately for any non-local deployment |
| `LLM_API_KEY` | `docker/environment-configs/backend.env` | placeholder | Required for the AI Query Assistant; set your own OpenAI/Groq/Ollama key |
| `LLM_PROVIDER` | `docker/environment-configs/backend.env` | `openai` | `openai`, `groq`, or `ollama` |
| `LANGFUSE_*` | `docker/environment-configs/backend.env` | placeholder / disabled | Optional; enables LLM call tracing |
| `VITE_API_BASE_URL` | `docker/environment-configs/dashboard.env` | `http://localhost:8001` | **Must be changed to your server's address for any deployment the browser doesn't run on the same machine as the backend** — `localhost` in a served frontend resolves to the viewer's machine, not your server |
| `CORS_ALLOW_ORIGINS` | `docker/environment-configs/backend.env` | `["*"]` | Restrict for production use |

See [Docker README](docker/README.md) for the full configuration walkthrough.

---

## **DEPLOYMENT & DEVELOPMENT DOCUMENTATION**
Expand All @@ -154,17 +188,13 @@ graph LR

### **🚀 Planned Features**

#### **Apex Development Tools**
- **Apex Code Management**: Basic Apex development environment
- **Apex Testing**: Simple test execution capabilities

#### **Enhanced Authentication**
- **Salesforce Web-based Authentication**: Native Salesforce OAuth integration
- **Multi-tenant Support**: Support for multiple Salesforce orgs

#### **Data Modification Capabilities**
- **Data Update Operations**: Basic insert, update, delete operations through UI
- **AI-Powered Data Updates**: Natural language data modification through AI agent
- **Record Insert & Delete**: The UI currently supports inline record *updates*; insert and delete operations through the UI are planned
- **AI-Powered Data Updates**: Natural language data modification through the AI agent (the AI agent is currently read-only)
- **Data Validation**: Simple data validation and error handling

---
Expand Down
4 changes: 2 additions & 2 deletions backend/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,12 @@ lock:
# Run the application (development mode)
run:
@echo "🚀 Starting DataPilot Backend..."
uv run python -m uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
uv run python run_server.py --dev --reload

# Run the application (production mode)
run-prod:
@echo "🚀 Starting DataPilot Backend (Production)..."
uv run python -m uvicorn app.main:app --host 0.0.0.0 --port 8000
uv run python run_server.py --prod --workers 4

# Run tests
test:
Expand Down
Loading
Loading