Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 24 additions & 28 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,31 +1,27 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.

# dependencies
/node_modules

# next.js
/.next/
/out/

# production
/build

# debug
# Node.js
node_modules/
dist/
npm-debug.log*
yarn-debug.log*
yarn-error.log*
.pnpm-debug.log*

# env files
.env*

# typescript
*.tsbuildinfo
next-env.d.ts

# Docker
.docker/

# Cache
.cache/
*.cache
.npm
.env
.env.local
.env.production
.DS_Store

# Next.js (Legacy)
.next/
out/

# Build outputs
build/
target/

# IDEs
.vscode/
.idea/

# MBTQ Data
data/
logs/
77 changes: 17 additions & 60 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,69 +1,26 @@
# MBTQ AI Platform - Multi-stage Docker Build
# Optimized for Generative AI and containerized deployment

# Stage 1: Dependencies
FROM node:18-alpine AS deps
RUN apk add --no-cache libc6-compat
# MBTQ Minimal Backend - Single Node Deployment
FROM node:20-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json* ./
RUN npm ci

# Install dependencies based on the preferred package manager
COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
RUN \
if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
elif [ -f package-lock.json ]; then npm ci; \
elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
else echo "Lockfile not found." && exit 1; \
fi

# Stage 2: Builder
FROM node:18-alpine AS builder
FROM node:20-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .

# Environment for build
ENV NEXT_TELEMETRY_DISABLED 1
ENV NODE_ENV production

# Build the application
RUN npm run build

# Stage 3: Production Runner
FROM node:18-alpine AS runner
FROM node:20-alpine AS runner
WORKDIR /app

ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED 1

# Create non-root user for security
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

# Copy public assets
COPY --from=builder /app/public ./public

# Set up Next.js output
RUN mkdir .next
RUN chown nextjs:nodejs .next

# Copy standalone output
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

# Copy services for runtime
COPY --from=builder --chown=nextjs:nodejs /app/services ./services
COPY --from=builder --chown=nextjs:nodejs /app/cli ./cli
COPY --from=builder --chown=nextjs:nodejs /app/frontend-inject ./frontend-inject

USER nextjs

EXPOSE 3000

ENV PORT 3000
ENV HOSTNAME "0.0.0.0"

# Health check
HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:3000/api/health || exit 1

CMD ["node", "server.js"]
RUN addgroup --system --gid 1001 nodejs && adduser --system --uid 1001 mbtq-user
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package.json ./package.json
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/components ./components
COPY --from=builder /app/templates ./templates
USER mbtq-user
EXPOSE 3001
ENV PORT 3001
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 CMD wget --no-verbose --tries=1 --spider http://localhost:3001/_health || true
CMD ["node", "dist/server/index.js"]
216 changes: 33 additions & 183 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,38 +1,46 @@
# AI Framework for Deaf Inclusion
# MBTQ Generative AI Platform - Backend Services

A comprehensive framework for building Deaf-inclusive web applications with AI-powered sign language recognition, WCAG-compliant components, and accessibility-first design.
A high-performance, Deaf-inclusive **Full Generative Developer Platform** focused on sign language recognition, translation, and accessibility. This is the **backend-only** version, providing a frontend-agnostic API layer for various clients (Unity, Mobile, Web) to generate accessible UI components and systems.

[![WCAG 2.1 AA](https://img.shields.io/badge/WCAG-2.1%20AA-green.svg)](https://www.w3.org/WAI/WCAG21/quickref/)
[![Deaf Inclusive](https://img.shields.io/badge/Deaf-Inclusive-blue.svg)](#deaf-inclusion-features)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
## 🌟 Vision

## 🌟 Key Features
MBTQ is not just a sign language tool; it is a full generative platform designed to create accessible digital experiences for the Deaf and hard-of-hearing community. Inspired by the long-term functionality of tools like v0 and Gemini, MBTQ provides a secure, agnostic architecture for generating React, Vite, and 3D assets.

### Accessible Components
- **AccessibleVideoPlayer**: WCAG 2.1 AA compliant video player with built-in captioning and sign language overlay
- **SignLanguageOverlay**: Real-time sign language video overlay supporting ASL, BSL, Auslan, NZSL, LSF, DGS, JSL
- **AccessibleCaptionDisplay**: Customizable caption display with high contrast options
## 🚀 Key Features

### Machine Learning Models
- **Vision Models**: Pre-trained models for sign language recognition with MediaPipe and TensorFlow.js
- **Language Models**: ASL/BSL syntax translation with grammar rule support
- **Accessibility Standards Generator**: Automated WCAG compliance checking
- **Generative Code Engine**: Generate React/Vite components and templates with built-in accessibility.
- **Sign Language AI**: Real-time sign recognition and text-to-sign translation services.
- **PinkSync Estimator**: Resource estimation for sync and generative tasks.
- **Agnostic API**: Secure REST API built with Express, TypeScript, and Zod.
- **Source Library**: Extensive collection of Deaf-inclusive components and templates used as a reference for generation.
- **Deaf-Centric RSS**: Aggregated feed of news and research relevant to the community.
- **Containerized**: Production-ready Docker and Docker Compose setup.

### Authentication
- **DeafAUTH**: Visual-first authentication system with video verification and visual CAPTCHA
## 📦 API Reference

### Infrastructure
- **MBTQ WCAG Action**: GitHub Action for CI/CD accessibility validation
- **Fibonrose DAO Nodes**: Decentralized deployment infrastructure
- **RSS Feed Aggregator**: Deaf-related content aggregation
See [docs/API_AGNOSTIC.md](docs/API_AGNOSTIC.md) for detailed documentation.

### Templates
- Deaf Organization SaaS starter
- Video Platform with ASL/BSL overlay
- Educational Platform template
## 🏗️ Architecture

- **Backend**: Node.js, Express, TypeScript
- **Database**: PostgreSQL (Neon Serverless)
- **AI**: Integration with LLMs (Groq) and custom ML pipelines
- **Security**: JWT, bcryptjs, Helmet, Rate Limiting
- **Agnostic Layer**: Pure API layer serving any frontend (Unity, React, Vue, etc.)

## 🎮 Visual Sign System Integration

The platform integrates with the **SignLanguageAssistant** (Unity/Blender/Python). See the `SignLanguageAssistant` directory for technical details.

## 🚀 Quick Start

### Prerequisites
- Node.js 20+
- Docker & Docker Compose
- PostgreSQL (or Neon.tech account)

### Installation

```bash
# Clone the repository
git clone https://github.com/pinkycollie/ai.mbtq.dev.git
Expand All @@ -42,167 +50,9 @@ cd ai.mbtq.dev
npm install

# Run development server
npm run dev
```

## 📦 Component Usage

### AccessibleVideoPlayer

```tsx
import { AccessibleVideoPlayer } from "@/components/accessibility"

<AccessibleVideoPlayer
src="/videos/presentation.mp4"
title="Welcome Message"
captions={[
{ src: "/captions/en.vtt", label: "English", language: "en", default: true }
]}
signLanguageOverlay={{
videoSrc: "/signs/asl/welcome.mp4",
language: "asl",
position: "bottom-right"
}}
/>
npm run dev &
```

### SignLanguageOverlay

```tsx
import { SignLanguageOverlay } from "@/components/accessibility"

<SignLanguageOverlay
text="Hello, welcome to our platform"
language="asl"
genAslEndpoint={process.env.AWS_GENASL_ENDPOINT}
size="medium"
position="bottom-right"
/>
```

### DeafAUTH Integration

```tsx
import { DeafAUTHService } from "@/lib/deaf-auth"

const deafAuth = new DeafAUTHService({
visualTwoFactor: true,
signLanguage: "asl",
visualCaptcha: true
})

// Create user with Deaf-friendly defaults
const user = await deafAuth.createUser(email, password, {
isDeaf: true,
preferredSignLanguage: "asl"
})
```

## 🔧 Machine Learning

### Sign Language Recognition

```typescript
import { createSignRecognitionModel } from "@/lib/ml"

const model = createSignRecognitionModel("asl", "balanced")
await model.load()

const result = await model.recognize(videoFrame)
console.log(result.sign, result.confidence)
```

### Text to Sign Translation

```typescript
import { SignLanguageTranslator } from "@/lib/ml"

const translator = new SignLanguageTranslator("asl")
const result = translator.translate("Hello, how are you?")
console.log(result.signSequence) // [{sign: "HELLO"}, {sign: "HOW"}, ...]
```

## 🔍 WCAG Compliance

This project includes a GitHub Action for automated WCAG compliance checking:

```yaml
# .github/workflows/mbtq-wcag-check.yml
# Runs on push/PR to validate accessibility
```

The action checks:
- Caption presence in video content
- Sign language overlay implementation
- Keyboard accessibility
- Color contrast ratios
- ARIA labels and roles

## 📂 Project Structure

```
├── app/ # Next.js application
│ ├── api/ # API routes
│ │ ├── rss/ # RSS feed aggregator
│ │ └── ...
│ └── ...
├── components/
│ ├── accessibility/ # Deaf-inclusive components
│ │ ├── AccessibleVideoPlayer.tsx
│ │ ├── SignLanguageOverlay.tsx
│ │ └── AccessibleCaptionDisplay.tsx
│ └── ui/ # Base UI components
├── lib/
│ ├── ml/ # Machine learning utilities
│ │ ├── vision-models.ts
│ │ └── language-models.ts
│ └── deaf-auth/ # DeafAUTH integration
├── templates/ # SaaS starter templates
└── .github/workflows/ # CI/CD workflows
```

## 🌐 Deployment

### Vercel (Recommended)
```bash
vercel deploy
```

### Fibonrose DAO Nodes
```bash
docker build -t fibonrose-registry/app:latest .
fibonrose deploy --config fibonrose.config.yml
```

See [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) for detailed instructions.

## 📖 Documentation

- [API Documentation](docs/API.md)
- [Deployment Guide](docs/DEPLOYMENT.md)
- [Contributing Guide](docs/CONTRIBUTING.md)
- [SignLanguageAssistant](SignLanguageAssistant/README.md)

## 🤝 Contributing

We welcome contributions! Please see [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md).

## 📄 License

MIT License - See [LICENSE](LICENSE)

---

## Legacy Documentation

### Repository Goals
This repository aims to improve understanding of system integration, AI workflow, and microservices, specifically focusing on `DeafAUTH`, `PinkSync`, and `Fibonrose`.

### Architectural Compatibility
To facilitate this understanding, [mbtq_architecture.html](mbtq_architecture.html) has been linked as a key component.

### Backend Transition: Flask to FastAPI
We are transitioning our backend from Flask to FastAPI for better performance and scalability. This transition aims to leverage FastAPI's asynchronous capabilities and automatic generation of API documentation.

### Blockchain Node Logging Strategy
Detailed logging strategies for blockchain nodes will be embedded to enhance traceability and performance monitoring. Strategies include transaction logging, error reporting, and performance benchmarks.
6 changes: 0 additions & 6 deletions app/api/auth/[...nextauth]/route.ts

This file was deleted.

Loading