Process killed at text recognition step

[ololiuhqui]

I tried this two times with a big PDF (500 pages), every step goes on succesfully but at "text recognition" I get a "killed" message after a little while:

Processing: my book.pdf
Converting PDF to Markdown...
Recognizing Layout: 100%|██████████████████████████████████████████████████████████| 526/526 [3:19:16<00:00, 22.73s/it]
Running OCR Error Detection: 100%|███████████████████████████████████████████████████| 132/132 [00:06<00:00, 21.95it/s]
Detecting bboxes: 100%|██████████████████████████████████████████████████████████████| 132/132 [27:41<00:00, 12.59s/it]
Recognizing Text:   0%|                                                                      | 0/41327 [00:00<?, ?it/s]Killed

The PDF has a layout with columns and plenty of images, it's kinda a school book.

Is this common? Is there some log to check out which I can paste here?

I'm on Linux Debian 13, AMD Ryzen CPU

[overcuriousity]

Hi!
Does not like a familiar problem to mee, but feels like OOM. Did you monitor your RAM usage during operation?
If your problem persists, I will have a look.

[ololiuhqui]

Ehy! Thank you for answering, It does seem to work with other PDF.

Yes OOM gave me issues before the text-recognition step, then I stopped the low-memory-monitor.service with systemctl and it seemed to not have been killed by that anymore.

With that service active I would also get a "low ram" warning from GNOME, with that disabled the terminal just says "killed" without any further explanation, and that doesn't apply to a smaller PDF.

A systemctl list-unit-files | grep oom does not give me any result.

Linux kernel seem to have an OOM killer that exists independently from systemctl, I might give it another shot and see if the process gets killed.

Is it possible that's there's some kind of limit to the max character to be recognized and that the process panics when seeing such a big amounts of characters? (41327 in my case)

[ololiuhqui]

Ok I found this in journalctl and it seems to be the right time, I'm gonna see if disabling kernel oom is somehow possible:

kernel: oom_reaper: reaped process 141341 (python) total-vm:35721784kB, anon-rss:13214836kB, file-rss:516kB, shmem-rss:0kB, UID:0 pgtables:61348kB oom_score_adj:100

It seems like python was actually using a lot of memory so it's def this the right process (and I have 16Gbs of ram + swap)

nice to know I didn't know about the kernel level oom :)

[overcuriousity]

Interesting indeed!
Maybe if python or some library is the culprit, this might resolve itself one day anyways (if upsream has updates).
So I cant do much here, I guess.

[ololiuhqui]

I found out that starting the process while reducing oom_score_adj to 10 seems to do the trick, but of course it takes ages on weak hardware.

Thank you for this amazing piece of software anyway!