Enhance API clients, error handling, and model management - #1
Open
hari-2807 wants to merge 50 commits into
Open
Conversation
This file provides functions to interact with OpenAI's API for chat completions and audio transcriptions.
Updated flashcard generation to use new model and handle text length.
Updated model ID and added constants for max characters and history turns. Improved message handling and error reporting.
Added character limit for summaries and implemented error handling for API calls.
Implement a Socratic tutoring session with Claude AI.
Refactored the audio transcription process to utilize a temporary file for improved handling of audio data.
Added error handling to the image parsing function to manage exceptions during the description extraction process.
Added error handling for PDF parsing and included filename in metadata extraction.
Implemented a safety check for URLs to prevent access to local or private addresses, enhancing security. Added error handling for unsupported content types and low word count warnings.
Enhance error handling for YouTube transcript fetching and support additional YouTube URL formats.
Add functions to chunk text into overlapping segments and check if chunking is needed based on word count.
Implement embedding functions for text chunks and queries.
This module provides functionality for managing a vector index using Databricks Vector Search. It includes methods for creating or fetching an index, upserting embedded chunks, and performing similarity searches.
Implement Google authentication endpoint with user creation.
Implement payment processing with Stripe for subscriptions and webhooks.
Implement upload functionality for files and URLs with tiered limits.
Set up FastAPI with CORS middleware and health check endpoint.
Implement Google OAuth token verification and JWT creation.
Refactor Google login functionality to use external verification and token management services.
Refactor Stripe webhook handling by extracting signature verification and event handling into separate functions.
This file tracks user actions and manages upload limits based on user tiers.
Refactor upload checks and logging mechanisms to enhance functionality and maintainability.
This script reads newly uploaded documents from Blob Storage, chunks them, and writes the raw chunks into a Delta table for embedding.
This notebook reads newly ingested document chunks, embeds them using Azure OpenAI, and writes the resulting vectors to the embeddings table. It handles session IDs, checks for existing embeddings, and reports the number of successfully embedded chunks.
This script triggers a sync of the embeddings Delta table into the Databricks Vector Search index, allowing new chunks to become searchable. It includes error handling for the sync process and logs the session ID.
This script implements a retrieval-augmented Q&A system that processes user questions, retrieves relevant information from a vector store, and generates answers using the Claude model.
Added new environment variables for OpenAI, Databricks, Stripe, and JWT authentication.
Updated .gitignore to include additional files and directories for Expo, IDEs, Azure credentials, testing, and logs.
Updated project context with new features, pricing, and tech stack details.
Added teaching mode functionality with session management.
Updated package versions for various dependencies.
Updated LinkedIn badge URL to the correct profile.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Full pass across the Grasp codebase: fixed deprecated SDKs and retired
models throughout
agent/, built out the entire FastAPI backend(
api/), auth + payments layer (auth/), and the Databricksretrieval pipeline (
databricks/notebooks/). Also wired Teaching modeinto the Streamlit UI and brought config/docs current.
No changes to pricing, product scope, or core architecture — this is
consistency, completeness, and dependency fixes on top of the existing
plan.
Why
Several files were empty stubs, and several "working" files referenced
retired models (GPT-4o, Gemini 1.5) or deprecated SDKs
(
google-generativeai,python-jose) that would fail in production.Teaching mode existed in
agent/modes/teaching.pybut was never wiredinto
app.py, so it was unreachable.What changed
AI Engine (
agent/)llms/claude.py,llms/gemini.py,llms/openai.py— migrated offretired models and deprecated SDKs, added consistent error handling
parsers/*.py(pdf, image, youtube, web, audio) — consistency fixes,added
truncated/likely_scanned/warningfieldsmodes/*.py(summary, analysis, qa, teaching, flashcards) —standardized return shape across all five modes
chunker.py,embedder.py,vector_store.py— new retrievalpipeline (chunk → embed → Vector Search)
llm_router.py— cost-aware model routingBackend (
api/)main.py— FastAPI entry point, lifespan, CORS, global exceptionhandler
routers/upload.py,routers/agent.py,routers/auth.py,routers/payments.py— newservices/blob_service.py,services/sql_service.py,models/schemas.py— newauth/dependencies.py— JWTget_current_userdependencyAuth & Payments (
auth/)google_oauth.py— Google ID token verification + JWTaccess/refresh token issuance (migrated to PyJWT)
usage_tracker.py— Free tier (3 uploads/month) business logicstripe_webhook.py— signature-verified event handlingData Pipeline (
databricks/notebooks/)01_ingest.py,02_embed.py,03_vector_index.py,04_agent.py—new four-stage batch pipeline for scalable retrieval-augmented Q&A
UI & Config
app.py— added missing Teaching mode section, 25MB file sizeguard on audio upload
requirements.txt— addedpyjwt,google-auth,email-validator,databricks-vectorsearch; renamedgoogle-generativeai→google-genai.env.example— addedJWT_SECRET,STRIPE_PRO_PRICE_ID,FRONTEND_URL,DATABRICKS_VS_ENDPOINT.gitignore— added mobile build artifacts, IDE folders, CLIcredential caches
.cursorrules,README.md,GRASP_CONTEXT.md— updated to matchactual current model versions, modes, and file structure