Skip to content

vinay020606/Deploy-Me

BranchesTags

Repository files navigation

Deploy-Me

A self-hosted static deployment platform — paste a GitHub URL and get a live URL back in seconds.

Built with Node.js, React, MongoDB, AWS S3, and AWS ECS Fargate. Inspired by Vercel's core deployment pipeline.

How It Works

User pastes GitHub URL
        ↓
API Server creates a Project + Deployment record in MongoDB
        ↓
API Server triggers an AWS ECS Fargate task (containerized build runner)
        ↓
Build Container:
  · git clone <repo>
  · npm install && npm run build
  · uploads dist/ → S3 at __outputs/{project-slug}/
  · streams build logs back to API Server in real time
        ↓
Reverse Proxy maps {subdomain}.localhost:8000 → files served directly from S3

Users can watch build logs stream live in the dashboard via WebSocket (Socket.io).

Architecture

┌─────────────────────────────────────────────────────┐
│              React Frontend  :3000                  │
└──────────────────────┬──────────────────────────────┘
                       │  REST API + WebSocket
                       ▼
┌─────────────────────────────────────────────────────┐
│              API Server  :9000                      │
│  Express · Socket.io · MongoDB (Mongoose)           │
│  Auth (JWT + bcrypt) · Rate Limiting                │
└──────┬────────────────────────────┬─────────────────┘
       │ RunTask (ECS Fargate)      │ /deployment/log
       ▼                            │ /deployment/status
┌──────────────────────┐            │
│  Build Container     │────────────┘
│  (Docker on ECS)     │
│  node:20-alpine      │
│  · git clone         │──────── uploads ──────────────┐
│  · npm install/build │                               │
└──────────────────────┘                               ▼
                                        ┌──────────────────────┐
                                        │      AWS S3          │
                                        │  __outputs/{slug}/   │
                                        │  logs/{slug}/*.log   │
                                        └──────────┬───────────┘
                                                   │
                                        ┌──────────▼───────────┐
                                        │   Reverse Proxy :8000│
                                        │  {slug}.localhost    │
                                        │   → streams from S3  │
                                        └──────────────────────┘

Tech Stack

Layer Technology
Frontend React 18, Vite, React Router v7, Framer Motion, Lucide React
API Server Node.js, Express 5, Socket.io, JWT, bcrypt, express-rate-limit
Database MongoDB with Mongoose ODM
Build Runner Docker (node:20-alpine) deployed on AWS ECS Fargate
Object Storage AWS S3 (build artifacts + log archival)
Reverse Proxy Express + AWS SDK v3 (streams S3 files on subdomain requests)
Real-time Socket.io (live build log streaming to the browser)

Project Structure

deploy-me/
├── api-server/                   # REST API + WebSocket server
│   ├── controllers/
│   │   ├── auth.js               # Signup / login (bcrypt + JWT)
│   │   ├── project.js            # Create project, trigger ECS Fargate task
│   │   └── deployment.js         # Stream logs, update status, fetch deployments
│   ├── middleware/
│   │   └── auth.js               # JWT verification middleware
│   ├── models/
│   │   ├── User.js               # Mongoose: name, email, password, avatar
│   │   ├── Project.js            # Mongoose: name, gitURL, subDomain, deployments[]
│   │   └── Deployment.js         # Mongoose: status, logs[], url
│   ├── routes/
│   │   ├── auth.js               # POST /auth/signup|login
│   │   ├── project.js            # POST|GET /project
│   │   └── deployment.js         # GET|POST /deployment/*
│   └── index.js                  # Express + Socket.io server entry
│
├── build-server/                 # Containerized build runner
│   ├── Dockerfile                # node:20-alpine, installs git + bash
│   ├── main.sh                   # Entrypoint: git clone → node script.js
│   └── script.js                 # Build logic: install, build, upload to S3, send logs
│
├── frontend/                     # React SPA
│   └── src/
│       ├── pages/                # Dashboard, Deploy, Logs
│       ├── components/           # Shared UI components
│       ├── layouts/              # Layout.jsx (navbar + page wrapper)
│       └── context/              # Auth context
│
├── reverse-proxy/                # Subdomain → S3 file server
│   └── index.js                  # Resolves {slug}.localhost:8000 → S3 key
│
├── start-all.ps1                 # Windows one-click startup
└── deploy-example.yml            # Example GitHub Actions CI/CD workflow

Prerequisites

  • Node.js v18+
  • MongoDB — local instance or MongoDB Atlas
  • AWS Account with:
    • An S3 bucket (for build artifacts and logs)
    • An ECS Cluster with a Fargate task definition pointing to your build-server Docker image
    • An IAM user with ecs:RunTask, iam:PassRole, and s3:PutObject permissions
  • Docker — to build and push the build-server image to ECR

Environment Variables

api-server/.env

PORT=9000
MONGO_URI=mongodb://localhost:27017/deploy-me

# JWT
JWT_SECRET=your_jwt_secret_here

# AWS — ECS Fargate (build runner)
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
CLUSTER_ARN=arn:aws:ecs:us-east-1:123456789:cluster/your-cluster
TASK_DEFINITION=arn:aws:ecs:us-east-1:123456789:task-definition/build-server:1
BUILDER_IMAGE=build-server          # container name in the task definition
SUBNETS=subnet-aaa,subnet-bbb
SECURITY_GROUPS=sg-xxxxxxxx

# AWS — S3
S3_BUCKET=your-s3-bucket-name
API_SERVER_URL=http://localhost:9000

build-server/.env (local testing only — ECS injects these automatically)

AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
S3_BUCKET=your-s3-bucket-name
API_SERVER_URL=http://localhost:9000

# Injected per-task by ECS
PROJECT_ID=
DEPLOYMENT_ID=
GIT_REPOSITORY_URL=

reverse-proxy/.env

PORT=8000
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
S3_BUCKET=your-s3-bucket-name

Quick Start (Windows)

.\start-all.ps1

This starts both the API server (:9000) and frontend (:3000) automatically.

Then open http://localhost:3000.

The reverse proxy and build server are intended to run in the cloud (ECS + S3). For local testing, see Manual Setup.

Manual Setup

1. API Server

cd api-server
npm install
node index.js
# API Server Running on port 9000

2. Reverse Proxy

cd reverse-proxy
npm install
node index.js
# Reverse Proxy running on port 8000

3. Frontend

cd frontend
npm install
npm run dev
# http://localhost:3000

4. Build Server (Docker)

Build and push the image to AWS ECR, then reference it in your ECS task definition:

cd build-server

# Build
docker build -t build-server .

# Tag and push to ECR
aws ecr get-login-password --region us-east-1 | \
  docker login --username AWS --password-stdin <account>.dkr.ecr.us-east-1.amazonaws.com

docker tag build-server:latest <account>.dkr.ecr.us-east-1.amazonaws.com/build-server:latest
docker push <account>.dkr.ecr.us-east-1.amazonaws.com/build-server:latest

For local testing without ECS:

docker run \
  -e GIT_REPOSITORY_URL=https://github.com/user/repo \
  -e PROJECT_ID=my-project \
  -e DEPLOYMENT_ID=dep-001 \
  -e AWS_ACCESS_KEY_ID=... \
  -e AWS_SECRET_ACCESS_KEY=... \
  -e AWS_REGION=us-east-1 \
  -e S3_BUCKET=your-bucket \
  -e API_SERVER_URL=http://host.docker.internal:9000 \
  build-server

API Reference

Auth

Method Endpoint Body Description
POST /auth/signup { name, email, password } Register a new user, returns JWT
POST /auth/login { email, password } Login, returns JWT

Projects

Method Endpoint Auth Body Description
POST /project — { name, gitURL } Create project + trigger ECS build
GET /project — — List all projects
GET /project/:id — — Get project with deployments

Deployments

Method Endpoint Auth Description
GET /deployment/:id ✅ JWT Get a single deployment
GET /deployment/project/:projectId ✅ JWT List deployments for a project
POST /deployment/log — Receive log chunk from build container
POST /deployment/status — Update deployment status from build container

Health

GET /health  →  { "status": "ok" }

WebSocket

Connect to ws://localhost:9000 to receive real-time build logs:

import { io } from 'socket.io-client';

const socket = io('http://localhost:9000');
socket.emit('subscribe-logs', deploymentId);

socket.on('log', (line) => console.log(line));
socket.on('status', (status) => console.log('Status:', status));

Data Models (MongoDB / Mongoose)

User

Field Type Notes
name String Required
email String Required, unique
password String bcrypt hashed
avatar String Default avatar URL

Project

Field Type Notes
name String Required
gitURL String Required
subDomain String Auto-generated slug, unique
customDomain String Optional
deployments ObjectId[] Refs to Deployment

Deployment

Field Type Notes
project ObjectId Ref to Project
status String QUEUED → IN_PROGRESS → READY / FAIL
logs String[] Build log lines, appended in real time
url String Final deployment URL on success

Deployment Flow (Step by Step)

1.  User submits { name, gitURL } via frontend

2.  POST /project
    → Generates a unique random subdomain slug
    → Creates Project + Deployment (status: QUEUED) in MongoDB
    → Returns { project, deploymentId } immediately

3.  API Server fires RunTaskCommand to AWS ECS Fargate (async):
    · Passes GIT_REPOSITORY_URL, PROJECT_ID, DEPLOYMENT_ID,
      S3_BUCKET, API_SERVER_URL, AWS credentials as env vars

4.  ECS launches Build Container (node:20-alpine):
    a. main.sh: git clone <GIT_REPOSITORY_URL> /home/app/output
    b. script.js: npm install && npm run build
    c. Uploads all files from dist/ → S3 at __outputs/{PROJECT_ID}/
    d. POST /deployment/log  (batched every 1s, relayed via Socket.io)
    e. POST /deployment/status → READY + url, or FAIL
    f. Archives full log to S3 at logs/{PROJECT_ID}/{DEPLOYMENT_ID}.log

5.  Browser receives live logs via WebSocket during build

6.  On READY: deployment URL is http://{slug}.localhost:8000
    Reverse Proxy resolves {slug} → reads from S3 → streams to browser
    SPA fallback: non-asset paths serve index.html automatically

CI/CD Integration

Copy deploy-example.yml into your project's .github/workflows/ to auto-deploy on push:

name: Deploy to Deploy-Me

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Trigger Deployment
        run: |
          curl -X POST ${{ secrets.DEPLOY_ME_API }}/project \
            -H "Content-Type: application/json" \
            -d '{
              "gitURL": "https://github.com/${{ github.repository }}",
              "name": "${{ github.event.repository.name }}"
            }'

Rate Limiting

The API server applies 100 requests / 15 minutes per IP across all routes using express-rate-limit. The server trusts proxy headers (trust proxy: 1) for accurate IP resolution when running behind a load balancer.

Performance

Typical end-to-end deployment times for a standard React/Vite project:

Phase Time
ECS Fargate cold start (container spin-up) ~20 – 40 s
git clone (small repo, ~5 MB) ~3 – 8 s
npm install (no cache, ~300 packages) ~30 – 60 s
npm run build (Vite, typical SPA) ~10 – 20 s
S3 upload (dist/, ~50 files) ~5 – 10 s
Total (cold, no cache) ~70 – 140 s

Build times drop significantly for subsequent deployments of the same project once node_modules caching is enabled (the codebase has a cache slot already wired in script.js at cache/{PROJECT_ID}/node_modules.tar.gz).

Design Decisions

ECS Fargate over AWS Lambda

Build jobs are long-running, resource-intensive, and unpredictable in duration — a typical npm install && npm run build can easily exceed Lambda's 15-minute hard timeout. ECS Fargate has no execution time cap and gives each build a dedicated container with configurable CPU/memory. Lambda would also cold-start a new environment for every dependency install, making it slower and more expensive at scale. ECS lets us ship a single, version-pinned Docker image that includes git, bash, and node, removing environment inconsistency between builds entirely.

Direct HTTP Calls over a Message Queue (e.g. Kafka)

The build container communicates back to the API server via plain HTTP POST (/deployment/log, /deployment/status). A message queue like Kafka would add operational overhead (broker management, consumer groups, offset tracking) that isn't justified at this scale. The current design achieves the same result — decoupled, async log delivery — with far less infrastructure. Logs are batched in memory every 1 second before being flushed, which prevents the API server from being hammered with thousands of individual requests. The architecture is Kafka-ready if throughput demands it — you'd swap axios.post in script.js for a Kafka producer and add a consumer on the API server side.

S3 for Build Artifacts and Log Archival

S3 is the natural choice for storing build output because:

  • Files are served directly from S3 by the reverse proxy with no intermediate layer
  • S3's per-key ContentType headers mean MIME types are correct out of the box
  • Costs are near-zero for static file volumes typical of side projects
  • Logs are archived to S3 at logs/{PROJECT_ID}/{DEPLOYMENT_ID}.log so you can always replay or audit a past build without keeping them in MongoDB (which would bloat the deployments collection with potentially thousands of log lines per record)

Socket.io for Real-time Log Streaming

The frontend subscribes to a deploymentId room on the Socket.io server. The API server calls io.to(deploymentId).emit('log', line) each time a log chunk arrives from the build container. This is simpler and lower-latency than polling /deployment/:id on an interval, and Socket.io's room abstraction means multiple browser tabs (or future team members) can all watch the same deployment's logs simultaneously without any extra work.

MongoDB for All Data

All three collections — users, projects, deployments — live in MongoDB. The Deployment model embeds logs as a String[] directly on the document. This works well because logs are always accessed in full alongside their parent deployment, making a join-based relational model an unnecessary overhead. MongoDB's flexible schema also made iteration fast during development.

License

ISC

About

Production-style deployment platform built with Kafka, Redis, AWS ECS and Docker — inspired by Vercel's core deployment pipeline

Topics

nodejs docker redis distributed-systems kafka websocket aws-ecs deployment-platform

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages