Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions apps/docs/content/docs/api/job-options.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ The `JobOptions` interface defines the options for creating a new job in the que
- `runAt?`: _Date | null_ — When to run the job (default: now).
- `timeoutMs?`: _number_ — Timeout for this job in milliseconds.
If not set, uses the processor default or unlimited.
- `forceKillOnTimeout?`: _boolean_ — If true, the job will be forcefully terminated (using Worker Threads) when timeout is reached. If false (default), the job will only receive an AbortSignal and must handle the abort gracefully.

**⚠️ Runtime Requirements**: This option requires **Node.js** and will **not work** in Bun or other runtimes without worker thread support. See [Force Kill on Timeout](/docs/usage/force-kill-timeout) for details.

- `tags?`: _string[]_ — Tags for this job. Used for grouping, searching, or batch operations.

## Example
Expand All @@ -28,6 +32,7 @@ const job = {
priority: 10,
runAt: new Date(Date.now() + 60000), // run in 1 minute
timeoutMs: 30000, // 30 seconds
forceKillOnTimeout: false, // Use graceful shutdown (default)
tags: ['welcome', 'user'], // tags for grouping/searching
};
```
139 changes: 139 additions & 0 deletions apps/docs/content/docs/usage/force-kill-timeout.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
---
title: Force Kill on Timeout
---

When you set `forceKillOnTimeout: true` on a job, the handler will be forcefully terminated (using Worker Threads) when the timeout is reached, rather than just receiving an AbortSignal.

## Runtime Requirements

**⚠️ IMPORTANT**: `forceKillOnTimeout` requires **Node.js** and uses the `worker_threads` module. It will **not work** in Bun or other runtimes that don't support Node.js worker threads.

- ✅ **Node.js**: Fully supported (Node.js v10.5.0+)
- ❌ **Bun**: Not supported - use `forceKillOnTimeout: false` (default) and ensure your handler checks `signal.aborted`

If you're using Bun or another runtime without worker thread support, use the default graceful shutdown approach (`forceKillOnTimeout: false`) and make sure your handlers check `signal.aborted` to exit gracefully when timed out.

## Handler Serialization Requirements

**IMPORTANT**: When using `forceKillOnTimeout`, your handler must be **serializable**. This means the handler function can be converted to a string and executed in a separate worker thread.

### ✅ Serializable Handlers

These handlers will work with `forceKillOnTimeout`:

```typescript
// Standalone function
const handler = async (payload, signal) => {
await doSomething(payload);
};

// Function that imports dependencies inside
const handler = async (payload, signal) => {
const { api } = await import('./api');
await api.call(payload);
};

// Function with local variables
const handler = async (payload, signal) => {
const localVar = 'value';
await process(payload, localVar);
};
```

### ❌ Non-Serializable Handlers

These handlers will **NOT** work with `forceKillOnTimeout`:

```typescript
// ❌ Closure over external variable
const db = getDatabase();
const handler = async (payload, signal) => {
await db.query(payload); // 'db' is captured from closure
};

// ❌ Uses 'this' context
class MyHandler {
async handle(payload, signal) {
await this.doSomething(payload); // 'this' won't work
}
}

// ❌ Closure over imported module
import { someService } from './services';
const handler = async (payload, signal) => {
await someService.process(payload); // 'someService' is from closure
};
```

## Validating Handler Serialization

You can validate that your handlers are serializable before using them:

```typescript
import {
validateHandlerSerializable,
testHandlerSerialization,
} from '@nicnocquee/dataqueue';

const handler = async (payload, signal) => {
await doSomething(payload);
};

// Quick validation (synchronous)
const result = validateHandlerSerializable(handler, 'myJob');
if (!result.isSerializable) {
console.error('Handler is not serializable:', result.error);
}

// Thorough test (asynchronous, actually tries to serialize)
const testResult = await testHandlerSerialization(handler, 'myJob');
if (!testResult.isSerializable) {
console.error('Handler failed serialization test:', testResult.error);
}
```

## Best Practices

1. **Use standalone functions**: Define handlers as standalone functions, not closures
2. **Import dependencies inside**: If you need external dependencies, import them inside the handler function
3. **Avoid 'this' context**: Don't use class methods as handlers unless they're bound
4. **Test early**: Use `validateHandlerSerializable` during development to catch issues early
5. **When in doubt, use graceful shutdown**: If your handler can't be serialized, use `forceKillOnTimeout: false` (default) and ensure your handler checks `signal.aborted`

## Example: Converting a Non-Serializable Handler

**Before** (not serializable):

```typescript
import { db } from './db';

export const jobHandlers = {
processData: async (payload, signal) => {
// ❌ 'db' is captured from closure
await db.query('SELECT * FROM data WHERE id = $1', [payload.id]);
},
};
```

**After** (serializable):

```typescript
export const jobHandlers = {
processData: async (payload, signal) => {
// ✅ Import inside the handler
const { db } = await import('./db');
await db.query('SELECT * FROM data WHERE id = $1', [payload.id]);
},
};
```

## Runtime Validation

The library automatically validates handlers when `forceKillOnTimeout` is enabled. If a handler cannot be serialized, you'll get a clear error message:

```
Handler for job type "myJob" uses 'this' context which cannot be serialized.
Use a regular function or avoid 'this' references when forceKillOnTimeout is enabled.
```

This validation happens when the job is processed, so you'll catch serialization issues early in development.
17 changes: 17 additions & 0 deletions apps/docs/content/docs/usage/job-timeout.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,20 @@ const handler = async (payload, signal) => {
```

If the job times out, the signal will be aborted and your handler should exit early. If your handler does not check for `signal.aborted`, it will keep running in the background even after the job is marked as failed due to timeout. For best results, always make your handlers abortable if they might run for a long time.

## Force Kill on Timeout

If you need to forcefully terminate jobs that don't respond to the abort signal, you can use `forceKillOnTimeout: true`. This will run the handler in a Worker Thread and forcefully terminate it when the timeout is reached.

**⚠️ Runtime Requirements**: `forceKillOnTimeout` requires **Node.js** and will **not work** in Bun or other runtimes without worker thread support. See [Force Kill on Timeout](/docs/usage/force-kill-timeout) for details.

**Important**: When using `forceKillOnTimeout`, your handler must be serializable. See [Force Kill on Timeout](/docs/usage/force-kill-timeout) for details.

```typescript
await queue.addJob({
jobType: 'longRunningTask',
payload: { data: '...' },
timeoutMs: 5000,
forceKillOnTimeout: true, // Forcefully terminate if timeout is reached
});
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
-- Up Migration: Add force_kill_on_timeout to job_queue
ALTER TABLE job_queue ADD COLUMN force_kill_on_timeout BOOLEAN DEFAULT FALSE;

-- Down Migration: Remove force_kill_on_timeout from job_queue
ALTER TABLE job_queue DROP COLUMN IF EXISTS force_kill_on_timeout;

Loading