Prompt-Viz: LLM Prompt Logic & Fallacy Analyzer

A tool for analyzing, visualizing, and improving LLM prompts by examining their logical structure and identifying potential fallacies.

Features

• Logical Structure Analysis: Breaks down prompts into sections and identifies logical operators
• Fallacy Detection: Uses pattern matching and AI analysis to detect common logical fallacies
• Symbolic Logic Visualization: Renders the logical structure as graphs and formulas
• Prompt Improvement: Suggests improvements based on detected issues
• AI-Powered Analysis: Uses OpenAI's API for deeper insight (optional)
• PDF Export: Generate comprehensive PDF reports of all analysis results and visualizations

Getting Started

Prerequisites

• Python 3.8+
• Streamlit
• NLTK
• NetworkX
• Matplotlib
• OpenAI API key (optional)
• ReportLab (for PDF generation)

Installation

1. Clone this repository
2. Install dependencies:

   pip install -r requirements.txt
   

3. Download NLTK data:

   import nltk
   nltk.download('punkt')
   nltk.download('punkt_tab')
   nltk.download('stopwords')

Usage

Run the application:

streamlit run app.py

Project Structure

prompt-viz/
├── app.py                  # Entry point
├── src/
│   ├── config/             # Configuration settings
│   │   ├── __init__.py
│   │   └── constants.py    # Application constants
│   ├── core/               # Core analysis functionality
│   │   ├── __init__.py
│   │   ├── analyzer.py     # Main analysis orchestration
│   │   ├── extraction.py   # Extract logical components
│   │   ├── fallacy_detection.py  # Detect logical fallacies
│   │   ├── openai_utils.py # OpenAI integration
│   │   ├── prompt_improvement.py # Generate improved prompts
│   │   └── visualization.py # Visualization functions
│   ├── ui/                 # UI components
│   │   ├── __init__.py
│   │   ├── input.py        # Input section
│   │   ├── overview_tab.py # Analysis overview
│   │   ├── structure_tab.py # Structure analysis
│   │   ├── symbolic_logic_tab.py # Logic visualization
│   │   ├── fallacies_tab.py # Fallacy detection results
│   │   └── suggestions_tab.py # Improvement suggestions
│   ├── utils/              # Utility modules
│   │   ├── __init__.py
│   │   └── pdf_generator.py # PDF report generation
│   └── main.py             # Main application logic
└── requirements.txt        # Dependencies

How It Works

1. Input: User enters an LLM prompt
2. Analysis:
   • Text is parsed into logical sections
   • Logical operators and entities are extracted
   • Pattern matching detects potential fallacies
   • OpenAI analysis provides additional insights (if enabled)
3. Visualization:
   • Logical structure is visualized as graphs
   • Operators are displayed in symbolic notation
4. Improvement:
   • Suggestions to address detected issues
   • AI-generated improved version of the prompt
5. Export:
   • Generate comprehensive PDF summary reports
   • Download analysis results and visualizations

PDF Export Features

The PDF export functionality provides a comprehensive report of your prompt analysis, including:

• Prompt Effectiveness Evaluation: Visual comparison of original vs. improved prompts with radar charts showing scores across multiple criteria
• Key Improvements: Highlighted list of the most significant improvements made to the prompt
• Detailed Analysis: In-depth explanation of how and why the improvements enhance the prompt's effectiveness
• Before/After Comparison: Side-by-side view of the original and improved prompts
• Fallacy Analysis: Summary of logical fallacies detected in the original prompt
• Logic Structure: Analysis of the prompt's logical structure and operators

To generate the PDF, navigate to the "Improvement Suggestions" tab after analyzing your prompt and click the "Generate PDF Summary" button.