ingestion broken

[scottdraves]

i uploaded a bunch of dreams with the API and they seem to be failing in the worker. here are the logs:

2024-11-18T22:22:58.762501+00:00 app[worker.1]: Upload completed.
2024-11-18T22:22:58.769694+00:00 app[worker.1]: 2024-11-18 22:22:58,769 - rq.worker - ERROR - [Job 9a3f87d6-950e-45a0-8caa-7d6d7270c8ea]: exception raised while executing (utils.process_video.run_video_ingestion)
2024-11-18T22:22:58.769694+00:00 app[worker.1]: Traceback (most recent call last):
2024-11-18T22:22:58.769695+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.12/site-packages/rq/worker.py", line 1428, in perform_job
2024-11-18T22:22:58.769695+00:00 app[worker.1]: rv = job.perform()
2024-11-18T22:22:58.769696+00:00 app[worker.1]: ^^^^^^^^^^^^^
2024-11-18T22:22:58.769696+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.12/site-packages/rq/job.py", line 1278, in perform
2024-11-18T22:22:58.769696+00:00 app[worker.1]: self._result = self._execute()
2024-11-18T22:22:58.769696+00:00 app[worker.1]: ^^^^^^^^^^^^^^^
2024-11-18T22:22:58.769697+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.12/site-packages/rq/job.py", line 1315, in _execute
2024-11-18T22:22:58.769698+00:00 app[worker.1]: result = self.func(*self.args, **self.kwargs)
2024-11-18T22:22:58.769698+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-18T22:22:58.769698+00:00 app[worker.1]: File "/app/utils/process_video.py", line 117, in run_video_ingestion
2024-11-18T22:22:58.769699+00:00 app[worker.1]: process_filmstrip(
2024-11-18T22:22:58.769699+00:00 app[worker.1]: File "/app/utils/process_video.py", line 84, in process_filmstrip
2024-11-18T22:22:58.769699+00:00 app[worker.1]: edream_client.upload_file(
2024-11-18T22:22:58.769699+00:00 app[worker.1]: File "/app/.heroku/src/edream-sdk/src/edream_sdk/client/file_client.py", line 256, in upload_file
2024-11-18T22:22:58.769699+00:00 app[worker.1]: file_size = path.stat().st_size
2024-11-18T22:22:58.769699+00:00 app[worker.1]: ^^^^^^^^^^^
2024-11-18T22:22:58.769700+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.12/pathlib.py", line 840, in stat
2024-11-18T22:22:58.769700+00:00 app[worker.1]: return os.stat(self, follow_symlinks=follow_symlinks)
2024-11-18T22:22:58.769700+00:00 app[worker.1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-18T22:22:58.769700+00:00 app[worker.1]: FileNotFoundError: [Errno 2] No such file or directory: 'assets/cf6e3eff-39dc-4fcd-8b9e-1373d9db812b/filmstrip/frame-1501.jpg'
2024-11-18T22:22:58.769701+00:00 app[worker.1]:
2024-11-18T22:22:58.776619+00:00 app[worker.1]: 2024-11-18 22:22:58,776 - rq.worker - INFO - default: utils.process_video.run_video_ingestion({'dream_uuid': '61408c27-d7e4-45a6-b22a-a0e4686f5819', 'extension': 'mov'}) (7942f533-32d0-4963-bfe0-f2f1a6\
671a54)

you can see the 8 dreams at the end of this playlist are stuck "processing"
https://alpha.e-dream.ai/playlist/142f0b08-0288-45fe-a812-8a55b16f3f22

[scottdraves]

had the same bug when i just dragged a video into the create menu instead of using the api:
https://alpha.e-dream.ai/dream/8ce50298-e29a-4069-b2b7-e1807ac2f908

actually it looks like it's in an infinite loop ingesting the same dream.

[scottdraves]

actually the last one I did with the web create page eventually worked -- it just had to work through the queue of failures from the API. i'm guessing i need to update the python package in my client?

[dealparadize]

I sent changes for #37, take me some time but I had an error on frame extraction (using 1-based starting frames), there were some dreams where No such file or directory message appeared due last frame interval didn't exists (was out of the range) and video ingestion process got broken. I fixed it and sent the correction for the extracting array. The only weird thing for me is that never happened before, maybe always matched our current frame_step with an frame number but for these new videos the condition just was not fulfilled, hope being clear

def get_filmstrip_array(total_frames: int) -> list[int]:
    if total_frames < 2400:
        frame_step = 100
    else:
        frame_step = 300

[scottdraves]

sounds like we have a fragile step, let's look into it soon.