Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 198 additions & 0 deletions docs/migrations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
# Database Migration Guide

This document covers how to run, roll back, and manage database migrations in the teachLink backend.

## Overview

Migrations are managed by the custom `MigrationModule` built on top of TypeORM and NestJS. Every migration implements a `MigrationConfig` interface with two methods:

- `up(connection)` — applies the schema change
- `down(connection)` — fully reverses the schema change

All migrations are registered in `src/migrations/migration.registry.ts` and tracked in the `migrations` database table.

---

## Migration Files

Migration files live in `src/migrations/samples/` and follow the naming convention:

```
NNN-description-of-change.migration.ts
```

Where `NNN` is a zero-padded sequence number (e.g. `001`, `002`). This ensures a deterministic execution order.

### Current Migrations

| # | Name | Description |
|---|------|-------------|
| 006 | `006-create-migrations-tracking-table` | Creates the `migrations` tracking table |
| 001 | `001-create-users-table` | Creates the `users` table with roles, status, and indexes |
| 002 | `002-create-courses-table` | Creates the `course` table with FK to users |
| 003 | `003-create-course-modules-table` | Creates the `course_module` table |
| 004 | `004-create-lessons-table` | Creates the `lesson` table |
| 005 | `005-create-enrollments-table` | Creates the `enrollment` table |

---

## Running Migrations

### Via npm scripts (requires the app to be running)

```bash
# Run all pending migrations
npm run migrate:run

# Check status of all migrations
npm run migrate:status
```

### Via HTTP API directly

```bash
# Run all pending migrations
curl -X POST http://localhost:3000/migrations/run

# List all migrations and their status
curl http://localhost:3000/migrations
```

### Automatic on startup

Set the environment variable to run migrations automatically when the app boots:

```bash
AUTO_RUN_MIGRATIONS=true
```

---

## Rolling Back Migrations

### Roll back the last migration

```bash
npm run migrate:rollback
# or
curl -X POST http://localhost:3000/migrations/rollback
```

### Roll back the last N migrations

```bash
# Roll back last 3 migrations
COUNT=3 npm run migrate:rollback:count
# or
curl -X POST http://localhost:3000/migrations/rollback/3
```

### Roll back a specific named migration

```bash
curl -X PUT http://localhost:3000/migrations/002-create-courses-table/rollback
```

> **Note:** This will fail if later migrations that depend on this one are still applied. Roll those back first.

### Roll back to a specific version

Rolls back all migrations applied *after* the named migration, leaving the named migration itself in place.

```bash
MIGRATION_NAME=002-create-courses-table npm run migrate:rollback:to
# or
curl -X POST http://localhost:3000/migrations/rollback/to/002-create-courses-table
```

---

## Resetting All Migrations (Development Only)

This rolls back every applied migration in reverse order and clears the tracking table.

```bash
npm run migrate:reset
# or
curl -X DELETE http://localhost:3000/migrations/reset
```

> ⚠️ **Never run this in production.** It will drop all managed tables.

---

## Creating a New Migration

1. Create a new file in `src/migrations/samples/` following the naming convention:

```typescript
// src/migrations/samples/007-add-bio-to-users.migration.ts
import { Injectable, Logger } from '@nestjs/common';
import { MigrationConfig } from '../migration.service';

@Injectable()
export class AddBioToUsersMigration implements MigrationConfig {
name = '007-add-bio-to-users';
version = '1.0.0';
dependencies = ['001-create-users-table'];

private readonly logger = new Logger(AddBioToUsersMigration.name);

async up(connection: any): Promise<void> {
await connection.query(`ALTER TABLE users ADD COLUMN bio TEXT;`);
}

async down(connection: any): Promise<void> {
await connection.query(`ALTER TABLE users DROP COLUMN IF EXISTS bio;`);
}
}
```

2. Register it in `src/migrations/migration.registry.ts`:

```typescript
import { AddBioToUsersMigration } from './samples/007-add-bio-to-users.migration';

export const MIGRATION_REGISTRY: MigrationConfig[] = [
// ... existing migrations ...
new AddBioToUsersMigration(),
];
```

---

## Migration Best Practices

- **Always implement `down()`** as the exact inverse of `up()` — same columns, same types, same constraints, in reverse order.
- **Declare dependencies** in the `dependencies` array. The runner validates them before executing.
- **Never modify an existing migration** that has already been applied to any environment. Create a new migration instead.
- **Test rollbacks locally** before merging. Run `up`, verify, then run `down` and verify the schema is restored.
- **Use `IF EXISTS` / `IF NOT EXISTS`** guards in SQL to make migrations idempotent where possible.
- **Back up your database** before running migrations in staging or production.

---

## Environment-Specific Considerations

| Environment | `AUTO_RUN_MIGRATIONS` | Notes |
|-------------|----------------------|-------|
| Development | `true` (recommended) | Migrations run on every app start |
| Test | `false` | Use `migrate:run` before test suites |
| Staging | `false` | Run manually after deployment |
| Production | `false` | Run manually with a backup in place |

---

## Troubleshooting

**Migration stuck in `pending` status**
The migration was registered but never executed. Run `npm run migrate:run`.

**Migration stuck in `failed` status**
Check the `error_message` column in the `migrations` table. Fix the underlying issue, then either re-run or roll back.

**`Dependency not met` error**
A migration's dependency hasn't been applied yet. Check the registry order and run the dependency first.

**`Cannot roll back` error**
Later migrations that depend on this one are still applied. Roll those back first, then retry.
8 changes: 7 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,13 @@
"test:e2e": "jest --config ./test/jest-e2e.json --forceExit --runInBand",
"test:ml-models": "jest src/ml-models --maxWorkers=1 --max-old-space-size=2048",
"test:ml-models-basic": "jest src/ml-models/ml-models.service.basic.spec.ts --maxWorkers=1 --max-old-space-size=2048",
"test:optimized": "node --max-old-space-size=2048 node_modules/.bin/jest --maxWorkers=1 --config jest.config.js"
"test:optimized": "node --max-old-space-size=2048 node_modules/.bin/jest --maxWorkers=1 --config jest.config.js",
"migrate:run": "curl -s -X POST http://localhost:3000/migrations/run | npx json",
"migrate:status": "curl -s http://localhost:3000/migrations | npx json",
"migrate:rollback": "curl -s -X POST http://localhost:3000/migrations/rollback | npx json",
"migrate:rollback:count": "curl -s -X POST http://localhost:3000/migrations/rollback/${COUNT:-1} | npx json",
"migrate:rollback:to": "curl -s -X POST http://localhost:3000/migrations/rollback/to/${MIGRATION_NAME} | npx json",
"migrate:reset": "curl -s -X DELETE http://localhost:3000/migrations/reset | npx json"
},
"dependencies": {
"@apollo/server": "^4.13.0",
Expand Down
43 changes: 37 additions & 6 deletions src/migrations/migration.controller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,13 +110,44 @@ export class MigrationController {
) {
this.logger.log(`Rolling back specific migration: ${migrationName}`);

// Note: In a real implementation, you'd need to map the migration name to the actual migration config
// For now, this is a placeholder
try {
await this.rollbackService.rollbackByName(migrationName);
return res.status(HttpStatus.OK).json({
success: true,
message: `Successfully rolled back migration: ${migrationName}`,
});
} catch (error) {
this.logger.error(`Error rolling back migration ${migrationName}`, error.stack);
return res.status(HttpStatus.INTERNAL_SERVER_ERROR).json({
success: false,
message: `Failed to rollback migration: ${migrationName}`,
error: error.message,
});
}
}

@Post('rollback/to/:migrationName')
@HttpCode(HttpStatus.OK)
async rollbackToVersion(
@Param('migrationName') migrationName: string,
@Res() res: Response,
) {
this.logger.log(`Rolling back to version: ${migrationName}`);

return res.status(HttpStatus.NOT_IMPLEMENTED).json({
success: false,
message: 'Specific migration rollback not implemented in this example',
});
try {
await this.rollbackService.rollbackToVersion(migrationName);
return res.status(HttpStatus.OK).json({
success: true,
message: `Successfully rolled back to version: ${migrationName}`,
});
} catch (error) {
this.logger.error(`Error rolling back to version ${migrationName}`, error.stack);
return res.status(HttpStatus.INTERNAL_SERVER_ERROR).json({
success: false,
message: `Failed to rollback to version: ${migrationName}`,
error: error.message,
});
}
}

@Get('conflicts')
Expand Down
25 changes: 25 additions & 0 deletions src/migrations/migration.registry.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
import { MigrationConfig } from './migration.service';
import { CreateUsersTableMigration } from './samples/001-create-users-table.migration';
import { CreateCoursesTableMigration } from './samples/002-create-courses-table.migration';
import { CreateCourseModulesTableMigration } from './samples/003-create-course-modules-table.migration';
import { CreateLessonsTableMigration } from './samples/004-create-lessons-table.migration';
import { CreateEnrollmentsTableMigration } from './samples/005-create-enrollments-table.migration';
import { CreateMigrationsTrackingTableMigration } from './samples/006-create-migrations-tracking-table.migration';

/**
* Central registry of all migrations in execution order.
*
* Rules:
* - Migrations are executed in the order they appear in this array.
* - Each migration's `dependencies` array is validated before execution.
* - To add a new migration: create the file in `samples/`, then append it here.
* - Never remove or reorder existing entries — only append new ones.
*/
export const MIGRATION_REGISTRY: MigrationConfig[] = [
new CreateMigrationsTrackingTableMigration(),
new CreateUsersTableMigration(),
new CreateCoursesTableMigration(),
new CreateCourseModulesTableMigration(),
new CreateLessonsTableMigration(),
new CreateEnrollmentsTableMigration(),
];
5 changes: 2 additions & 3 deletions src/migrations/migration.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { ConflictResolutionService } from './conflicts/conflict-resolution.servi
import { SchemaValidationService } from './validation/schema-validation.service';
import { RollbackService } from './rollback/rollback.service';
import { EnvironmentSyncService } from './environments/environment-sync.service';
import { MIGRATION_REGISTRY } from './migration.registry';

export interface MigrationConfig {
name: string;
Expand Down Expand Up @@ -109,9 +110,7 @@ export class MigrationService {
* Gets all registered migrations
*/
private getRegisteredMigrations(): MigrationConfig[] {
// In a real implementation, this would load migrations from files or configuration
// For now, returning an empty array - this would be populated based on your actual migrations
return [];
return MIGRATION_REGISTRY;
}

/**
Expand Down
67 changes: 63 additions & 4 deletions src/migrations/rollback/rollback.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { InjectRepository } from '@nestjs/typeorm';
import { Repository, Raw } from 'typeorm';
import { Migration, MigrationStatus } from '../entities/migration.entity';
import { MigrationConfig } from '../migration.service';
import { MIGRATION_REGISTRY } from '../migration.registry';

@Injectable()
export class RollbackService {
Expand Down Expand Up @@ -110,14 +111,72 @@ export class RollbackService {
* Gets all registered migrations
*/
private getRegisteredMigrations(): MigrationConfig[] {
// In a real implementation, this would load migrations from files or configuration
// For now, returning an empty array - this would be populated based on your actual migrations
return [];
return MIGRATION_REGISTRY;
}

/**
* Checks if a migration can be safely rolled back
* Rolls back all migrations down to (but not including) the target version.
* Migrations are rolled back in reverse-applied order.
*/
async rollbackToVersion(targetMigrationName: string): Promise<void> {
this.logger.log(`Rolling back to version: ${targetMigrationName}`);

const registeredMigrations = this.getRegisteredMigrations();

// Verify the target migration exists in the registry
const targetExists = registeredMigrations.some((m) => m.name === targetMigrationName);
if (!targetExists) {
throw new Error(`Target migration not found in registry: ${targetMigrationName}`);
}

// Get all applied migrations that were applied AFTER the target, in reverse order
const targetRecord = await this.migrationRepository.findOne({
where: { name: targetMigrationName },
});

const whereClause = targetRecord?.appliedAt
? { status: MigrationStatus.COMPLETED, appliedAt: Raw((alias) => `${alias} > :appliedAt`, { appliedAt: targetRecord.appliedAt }) }
: { status: MigrationStatus.COMPLETED };

const migrationsToRollback = await this.migrationRepository.find({
where: whereClause,
order: { appliedAt: 'DESC' },
});

for (const appliedMigration of migrationsToRollback) {
const migrationConfig = registeredMigrations.find((m) => m.name === appliedMigration.name);
if (migrationConfig) {
await this.rollbackMigration(migrationConfig);
} else {
this.logger.warn(`Could not find migration config for: ${appliedMigration.name}`);
}
}

this.logger.log(`Rollback to version ${targetMigrationName} complete.`);
}

/**
* Rolls back a specific named migration regardless of order.
*/
async rollbackByName(migrationName: string): Promise<void> {
this.logger.log(`Rolling back specific migration by name: ${migrationName}`);

const registeredMigrations = this.getRegisteredMigrations();
const migrationConfig = registeredMigrations.find((m) => m.name === migrationName);

if (!migrationConfig) {
throw new Error(`Migration not found in registry: ${migrationName}`);
}

const canRollback = await this.canRollbackMigration(migrationName);
if (!canRollback) {
throw new Error(
`Cannot roll back migration ${migrationName}: it either hasn't been applied or has dependent migrations applied after it.`,
);
}

await this.rollbackMigration(migrationConfig);
}
async canRollbackMigration(migrationName: string): Promise<boolean> {
// Check if the migration exists and is completed
const migration = await this.migrationRepository.findOne({
Expand Down
Loading
Loading