Feature : Optional "Process Isolation and Safety"

[pomarec]

In one of our application, we would need our jobs to live in the same process/environment as our app (DI, shared memeory read only ressources, database pooling, etc.)

One way to achieve that would be to trade process isolation for this convenience. It could be done through config flag.

Some of the drawbacks might be adressed :

• Error Safety: set a default error handler
• Memory Protection: no workaround
• Resource Management: add a Sidequest.restart()
• Performance: no workaround

Performance seems the major drawback IMO as one of the purpose of a job queue is to offload the "main runloop". But AFAIK this is already indirectly the case as a child process take CPU time aswell, the risk is in non proper async code blocking the nodejs runloop.
Memory protection suffers the same problem as memory is ultimately shared between processes. The drawback here is that you would need to restart the whole app (not just sidequest) in case of memory leak.

I'm willing to work on this in case of acceptation.

[GiovaniGuizzo]

Can you elaborate a bit more on why we would need our jobs to live in the same process/environment as our app?

[pomarec]

Of course :

• We'd like to avoid running an additional service in our stack (lower maintenance, easier debug)
• Bootstraping our app "engine" (DI, caches, etc.) would be cumbersome in our specific case
• Sharing some readonly objects in memory is a optimization we'd like to have
• child process malfunction with our auto-reload library, this impacts our test & debug DX

I understand that those reason are subjectives. There is a "better" solution to each items (eg : external cache like redis for shared readonly object in memory). But this is the path we would have liked to explore.

FYI, we ended up using https://timgit.github.io/pg-boss/ as the implementation of thoses constraints were easier in the current situation. We are time constrained by one of our client. We thus "no longer have this need" for sidequest. Would it help others ?

Cheers