Workshop toggle ext memory#3
Merged
Merged
Conversation
Broadly moved all content into the ws folder under /materials (where the toggle will occur) Removed : - progressive_agents_guide.md - CONTENT_COVERAGE_GAP_ANALYSIS.md - Deleted tests folder - Deleted docker_compose, readme, requirements.txt, and setup.md, UVlock, as they are not necessary in this setup Note to Nitin: Do we need the /arch folder in the notebooks directory?
Adding test changes to test toggle in ps portal
nkanu17
force-pushed
the
workshop-toggle-ext-memory
branch
2 times, most recently
from
32ef38f to
5e6eead
Compare
update gitignore for node_modules
fix: Update all paths after moving content into materials/ws/ folder Notebooks - folder check "workshop" → "notebooks": - notebooks/02_rag_essentials.ipynb - notebooks/03_data_engineering_theory.ipynb - notebooks/04_memory_systems.ipynb Notebooks - removed outdated Colab link: - notebooks/01_introduction_to_context_engineering.ipynb progressive_agents - parents[2] → parents[1] (19 files): - stage1_baseline_rag/cli.py - stage2_context_engineered/cli.py - stage3_full_agent_without_memory/cli.py, debug_search.py - stage4_hybrid_search/cli.py, test_hybrid_search.py, test_react_agent.py - stage5_working_memory/cli.py, debug_search.py, test_*.py (8 test files) - stage6_full_memory/cli.py, debug_search.py, test_*.py (5 test files) progressive_agents/agent - parent.parent → parent.parent.parent: - stage1_baseline_rag/agent/nodes.py - stage2_context_engineered/agent/nodes.py - stage3_full_agent_without_memory/agent/tools.py debug_search.py - removed erroneous /src/ from .env path: - stage3_full_agent_without_memory/debug_search.py - stage5_working_memory/debug_search.py - stage6_full_memory/debug_search.py pyproject.toml - demos → progressive_agents: - Updated 6 CLI entry points - Updated wheel packages list Edit to Litellm boot time in start.sh
Increasing timeout for start.sh
Changed connection defaults from localhost to Docker service names: - Redis: redis://localhost:6379 → redis://redis:6379 - Agent Memory Server: http://localhost:8088 → http://agent-memory-server:8000
…ical_manager - Replace text-embedding-ada-002 with text-embedding-3-small (3 places) - Update comment to reflect correct model name - Fixes embedding model not available error in LiteLLM proxy
- Update HybridQuery to use combination_method='RRF' (industry best practice) - Add documentation for RRF vs LINEAR combination methods - Replace deprecated alpha parameter with rrf_window/rrf_constant - Update code comments to reflect FT.HYBRID command (Redis 8.4+) - Improve explanation of hybrid search results
- Add HybridQuery import for FT.HYBRID command (Redis 8.4+) - Implement three search strategies: hybrid, exact_match, semantic_only - Use RRF combination method as default (industry best practice) - Add LINEAR combination support with configurable linear_alpha - Use course_manager.embeddings pattern (matches university code) - Remove separate _get_embeddings() helper function - Add department filtering support for hybrid queries
Add Part 3: Hybrid Search to stage_2_data_engineered_rag.ipynb: - Introduction explaining what hybrid search is and why it matters - Real-world scenarios showing when semantic/keyword search fail alone - Industry adoption context (Elasticsearch, Pinecone, Weaviate, Redis) - Detailed comparison of RRF vs LINEAR combination methods - RedisVL implementation examples with HybridQuery - Four code examples: basic RRF, LINEAR weighting, filters, comparison - Best practices and decision table for search type selection - Updated wrap-up section with hybrid search takeaways
- Add chat_notebooks directory with stage1_chat, stage4_chat, stage6_chat notebooks - Add chat_ui.py and chat_interface.py for Stage 1, 4, 6 agents - Update __init__.py exports to include start_chat functions - Fix Agent Memory Server port mapping in docker-compose.yml (8000:8000) - Use Docker service names (redis:6379, agent-memory-server:8000) by default - Match university materials configuration for seamless Docker execution
…upport" This reverts commit 23cb003.
Notebook fixes: - Add nest_asyncio.apply() to 01-1 and 01-2 for async support - Fix path stage2_data_engineered -> stage2_context_engineered in 01-2 - Fix module import data_engineering -> context_engineering in 01-2 - Add None check to count_tokens() in 02-1 - Fix token efficiency calculation in 02-1 Dependency updates (requirements.txt): - Add langchain-experimental>=0.3.0 (SemanticChunker) - Add langchain-text-splitters>=0.3.0 (RecursiveCharacterTextSplitter) - Add langchain-huggingface>=0.1.0 (HuggingFaceEmbeddings) - Add sentence-transformers>=2.0.0 (semantic chunking backend) - Add nest-asyncio>=1.5.0 (Jupyter async support) - Add jupyter>=1.0.0 and ipykernel>=6.0.0 - Add pypdf>=3.0.0 (PDF parsing) Dependency updates (pyproject.toml): - Add nest-asyncio>=1.5.0 - Add version constraint to pypdf>=3.0.0 All notebooks now execute successfully: - 01-1_baseline_rag.ipynb - 01-2_data_engineered_rag.ipynb - 02-1_data_engineering_theory.ipynb - 03-1_memory_systems.ipynb
Removed old notebooks (replaced with new naming convention): - 01_introduction_to_context_engineering.ipynb - 02_rag_essentials.ipynb - 03_data_engineering_theory.ipynb - 04_memory_systems.ipynb - stage6_chat.ipynb (duplicate, kept chat_notebooks/stage6_chat.ipynb) Updated chat notebooks with collapsible cells and fixes: - chat_notebooks/stage0_chat.ipynb - chat_notebooks/stage1_chat.ipynb - chat_notebooks/stage6_chat.ipynb Other updates: - arch/03_progressive_agents_demo.ipynb - stage1_baseline_rag/agent/nodes.py
Workshop notebooks (materials/ws/): - Remove LOCAL_DEV environment variable detection and conditional logic - Use Docker service names directly (redis:6379, litellm:4000, agent-memory-server:8000) - Clear stale notebook outputs containing localhost URLs University materials (materials/university/): - Update Agent Memory Server URL from localhost:8088 to agent-memory-server:8000 - Fix CLI files and notebooks to use correct Docker service names Files updated: - 11 workshop notebooks - 4 university files (stage5/stage6 CLI, query_long_term_memory.py, stage_5_working_memory.ipynb)
Change Path('') to Path('..') to correctly resolve project root from notebooks directory
…dency - Move notebooks to numbered subdirectories (00/, 01/, 02/, 03/) - Add university memory notebooks (stage_5_working_memory, stage_6_full_memory) - Copy missing test files to progressive_agents - Update all path references for new directory structure - Replace HuggingFaceEmbeddings with OpenAIEmbeddings in semantic chunking - Remove sentence-transformers and langchain-huggingface dependencies
nkanu17
force-pushed
the
workshop-toggle-ext-memory
branch
from
5e6eead to
19d0daa
Compare
…ming ## Standalone Notebook Setup - All notebooks now run locally without any manual environment wiring - Environment loading moved into agent package __init__.py files so importing the agent is sufficient to bootstrap all config - Added LOCAL=true flag to root .env as the single switch for local overrides; portal environments are completely unaffected ## Hidden Environment Configuration - stage1_baseline_rag/agent/__init__.py: added load_dotenv + LOCAL key mapping - stage2_context_engineered/agent/__init__.py: same pattern - stage3_full_agent_without_memory/agent/__init__.py: same pattern - stage4_hybrid_search/agent/__init__.py: same pattern - stage5_working_memory/agent/__init__.py: added load_dotenv + RAMS/Redis URL mapping - stage6_full_memory/agent/__init__.py: added load_dotenv + RAMS/Redis URL mapping - src/redis_context_course/__init__.py: load_dotenv runs before redis_config singleton is created, fixing import-order race condition ## Topic-Based Rebranding (Stage N -> Descriptive Names) Removed all 'Stage 0-6' labels from notebook titles, headings, and narrative. Replaced with consistent topic names throughout: Stage 0 -> System Prompts Stage 1 -> Baseline RAG Stage 2 -> Data-Engineered RAG Stage 3 -> Hierarchical Retrieval Stage 4 -> Hybrid Search Stage 5 -> Working Memory Stage 6 -> Long-Term Memory Cross-notebook transitions updated to reference topic names rather than stage numbers (e.g. 'In Stage 2 you will...' -> 'In the next notebook, Data-Engineered RAG, you will...'). ## Notebook Restructuring - Renamed rag_deepdive.ipynb -> workshop_rag_deepdive.ipynb - Replaced stage_5_working_memory.ipynb and stage_6_full_memory.ipynb with 03/01_working_memory.ipynb and 03/02_working_and_longterm_memory.ipynb - Removed old memory_systems.ipynb - chat_notebooks/stage0_chat and stage6_chat titles and print statements updated ## README - Added full local setup and testing instructions to materials/ws/README.md - Documents .env config, venv setup, and how to run/test each notebook
nkanu17
force-pushed
the
workshop-toggle-ext-memory
branch
from
19d0daa to
4c273d6
Compare
🛡️ Jit Security Scan Results
🚨 Summary• Two critical vulnerabilities found in Python dependencies 🔍 FindingsCRITICAL - 1 finding
1. LangChain Experimental Arbitrary Code Execution
HIGH - 1 finding
1. LangChain Text Splitters XXE Vulnerability
Remediation: - langchain-text-splitters>=0.3.0
+ langchain-text-splitters==0.3.9
|
❌ Jit Scanner failed - Our team is investigatingJit Scanner failed - Our team has been notified and is working to resolve the issue. Please contact support if you have any questions. 💡 Need to bypass this check? Comment |
PS Portal's Jupyter runs on ipykernel which starts its own event loop. asyncio.get_event_loop() returns that running loop, and then loop.run_until_complete() throws RuntimeError: This event loop is already running because you can't call run_until_complete on an already-running loop.
❌ Jit Scanner failed - Our team is investigatingJit Scanner failed - Our team has been notified and is working to resolve the issue. Please contact support if you have any questions. 💡 Need to bypass this check? Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.