Welcome to the official GitHub repository for the CHI Research Team's Personalized LLM Initiative, led by student researchers at City Tech. This project aims to design, evaluate, and deploy a lightweight, ethical, and scalable Large Language Model (LLM) tailored for real-time personalization across diverse domains—such as literature, STEM, and digital mediation within the Balanced Reality Platform System (BRPS). Our mission is to build an open, responsible, and interdisciplinary AI framework that is transparent, adaptable, and aligned with human values.
This project is framed as a practical research question: Can a lightweight, open-source, ethically governed personalized LLM pipeline produce useful and trustworthy adaptation across domain and user contexts while remaining reproducible in an academic setting?
Primary hypothesis (H1) A lightweight personalized LLM architecture, trained and evaluated with transparent workflows, can outperform non-personalized baselines on task relevance and user alignment without unacceptable increases in hallucination or latency.
Secondary hypotheses H2: Multimodal inputs (text plus image/signal features where applicable) improve personalization quality over text-only personalization for targeted use cases. H3: Structured feedback loops (human evaluation plus iterative model adjustment) improve alignment scores over static one-pass model tuning. H4: Privacy-first data governance and fairness audits can be integrated into the development cycle without materially degrading model utility.
Operational test signals
- Alignment: higher Alignment Score (AS) and rubric-based relevance across benchmark tasks.
- Reliability: reduced omission/error rates and controlled hallucination risk.
- Practicality: acceptable response latency for real-time or near-real-time use.
- Ethics and compliance: documented bias checks, consent-aware data handling, and GDPR/CCPA-aligned process controls.
Evaluate existing commercial and open-source LLMs (e.g., ChatGPT, Claude, DeepSeek). Design a custom AI system capable of interpreting multimodal inputs and adapting to individual users. Implement real-time personalization based on behavior, history, and preferences. Ensure data ethics and compliance with standards like GDPR and CCPA. Benchmark and continuously optimize the model using feedback loops and RL fine-tuning.
- Onboarding & Survey: Team setup, research review, initial LLM evaluations.
- Requirements Analysis: Define personalization levels, compliance boundaries, and data governance.
- Model Selection & Training: Develop modular, scalable architecture with API integrations.
- Model Implementation: Develop modular, scalable architecture with API integrations.
- Testing & Validation: A/B testing, fairness audits, and ethical validations.
- Deployment & Optimization: Real-world deployment, KPI monitoring, and user-driven iteration.
Leads: Kazi Tasin (Lead), Kazi Rahimu Islam Faculty Advisor: Prof. David Smith
We conducted a structured evaluation of three major LLMs—ChatGPT, Claude AI, and DeepSeek—by summarizing five articles from domains like economics, quantum computing, diplomacy, and digital realities.
Key Findings
Criteria
Summarization
Accuracy
Analysis
Hallucination
GPT
Summarization: Strong synthesis, slight oversimplification
Accuracy: Minor omissions
Analysis: Good, but shallow in complex topics
Hallucination: Moderate risk
Claude Summarization: Most accurate, structured summaries Accuracy: Highly factual Analysis: Deep and contextual Hallucination: Low
DeepSeek Summarization: Simplified, lacks political nuance Accuracy: Avoids controversial topics Analysis: Limited critical depth Hallucination: Low but context-dependent
Claude AI outperformed others with its balanced, detailed, and context-aware summaries. The evaluation was guided by criteria such as factual consistency, completeness, reasoning, and hallucination rate.
Tech Stack
Model Training: PyTorch, HuggingFace Transformers, LlamaCpp Evaluation Framework: Custom rubric based on Error Rate (ER), Omission Rate (OR), and Alignment Score (AS) Data Sources: Public articles (Forbes, Nvidia, USC, Webisoft, U.S. State Dept) Version Control: GitHub, Zotero (Research logs) Communication: Discord, BBS (Balanced Blended Space platform)
Ethical Commitment We are committed to developing a fair, explainable, and user-consent-based AI system. Our LLM incorporates:
Transparent training data and logic. Bias detection and fairness audits. Privacy-by-design principles (anonymization, encryption, access control). Alignment with legal standards (GDPR, CCPA).
Future Directions
Incorporate reinforcement learning with human feedback (RLHF) for adaptive responses. Expand multimodal capabilities (image, sound, and mathematical problem processing). Enable cross-platform AI agents for BRPS, education, and public services. Launch a web-based interaction demo showcasing our personalized LLM.
Contact & Contributions
Interested in collaborating? Join us!
Prof. David Smith Email: dsmith@citytech.cuny.edu
Student Researchers Kazi Tasin: kazi.tasin@mail.citytech.cuny.edu Kazi Islam: Kazi.Islam3@mail.citytech.cuny.edu