Skip to content

[Backend] Implement Image and Video Processing BullMQ Processors #503

@llinsss

Description

@llinsss

⚠️ This is a backend issue — work is done inside the backend/ folder

Description

backend/src/modules/processing/ has a ProcessingService that enqueues jobs into image-processing and video-processing BullMQ queues, and the processor files exist at processors/image.processor.ts and processors/video.processor.ts. However, both processor files are stubs — they contain no actual processing logic. Uploaded pet photos and documents are queued but never processed.

Current State

  • backend/src/modules/processing/processing.service.ts — enqueues jobs correctly
  • backend/src/modules/processing/processors/image.processor.ts — stub
  • backend/src/modules/processing/processors/video.processor.ts — stub
  • ProcessingOptions interface defines: generateThumbnail, compress, convertToWebP, addWatermark, stripMetadata

What Needs to Be Built

1. Image Processor (processors/image.processor.ts)

@Processor('image-processing')
export class ImageProcessor {
  @Process('process')
  async handle(job: Job<{ fileId: string; options: ProcessingOptions }>) { ... }
}

Use sharp (already a common NestJS dependency) to:

  • Generate thumbnail (300×300, WebP)
  • Compress original (quality 80)
  • Convert to WebP if convertToWebP is set
  • Strip EXIF metadata if stripMetadata is set
  • Upload processed variants to S3 via StorageModule
  • Update FileMetadata entity with thumbnail URL and processed status

2. Video Processor (processors/video.processor.ts)

  • Generate a preview thumbnail from frame at 1 second using ffmpeg
  • Update FileMetadata with preview URL and status: 'processed'

3. Error Handling

  • On failure: update ProcessingJob.status to FAILED with error message
  • Emit real-time status update via RealtimeGateway
  • Retry up to 2 times before marking failed

4. Unit Tests

  • Mock sharp and StorageModule
  • Test each processing option independently

Acceptance Criteria

  • Uploaded images are thumbnailed, compressed, and stripped of metadata
  • FileMetadata.thumbnailUrl is populated after processing
  • ProcessingJob.status transitions: PENDINGPROCESSINGCOMPLETED / FAILED
  • Real-time status updates are emitted via WebSocket
  • Failed jobs retry twice before marking FAILED
  • Unit tests cover all ProcessingOptions flags

Files to Implement

  • backend/src/modules/processing/processors/image.processor.ts
  • backend/src/modules/processing/processors/video.processor.ts
  • backend/src/modules/processing/processors/image.processor.spec.ts (new)

Priority

High — pet photos are uploaded but never processed or thumbnailed

Estimated Effort

2–3 days

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions