Production website hard restarting from export to csv

[jmadden173]

Seeing the following errors on the production website. Through a bit of debugging I believe that this is caused by the Export to CSV button. After clicking that the website becomes unresponsive. The deployed version of the website is 9c56bd9

Delete the branch hotfix/jmadden173/remove-csv-export when completed.

March 4, 2026, 11:45
	
[2026-03-04 19:45:52 +0000] [1] [INFO] Handling signal: term
	
dirtviz-api
March 4, 2026, 11:42
	
[2026-03-04 19:42:11 +0000] [12] [INFO] Booting worker with pid: 12
	
dirtviz-api
March 4, 2026, 11:42
	
[2026-03-04 19:42:11 +0000] [1] [ERROR] Worker (pid:10) was sent SIGKILL! Perhaps out of memory?
	
dirtviz-api
March 4, 2026, 11:42
	
[2026-03-04 19:42:10 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:10)

[jmadden173]

Source for formatting the csv from the database. Couple issues I see

• pandas??? why are we resampling in memory when we have an RDS to the resampling for us
• We only are getting voltage, current and phytos?? Shouldn't we have functions for all of the sensors. This can "kinda" be pulled from the ents python package.
• Again pandas merge when we do it in our RDS
• Sure chunking it to memory is fine, but not having a dynamic size so that the chunks can go indefinitely.
• Now we're going to numpy land (which to be fair is the underlying data types of pandas and exporting as csv
• Ontop of that it appears that our celery database is not doing anything.

There has to be a way to open a "stream" such that we either batch requests or directly stream data from a RDS call to avoid caching in-memory on the webserver. See the flask page on Streaming. You can encode an iterable as a response.

Tbh with this implementation I think that we can do away with the "worker" and "valkey" instance saving cost and complexity. Wouldn't need to maintain two separate docker containers...

ENTS-backend/backend/api/resources/cell_data.py

Lines 16 to 65 in 9c56bd9

	@shared_task(bind=True, ignore_result=False)
	def stream_csv(self, request_args):
	"""Creates a task that queries cell data and writes into buffer
	Arguments:
	request_args -- arguments of the request being made
	"""
	v_args = get_cell_data.load(request_args)
	cell_ids = v_args["cellIds"].split(",")
	for cell_id in cell_ids:
	teros_data = pd.DataFrame(
	TEROSData.get_teros_data_obj(
	cell_id,
	resample=v_args["resample"],
	start_time=v_args["startTime"],
	end_time=v_args["endTime"],
	)
	)
	power_data = pd.DataFrame(
	PowerData.get_power_data_obj(
	cell_id,
	resample=v_args["resample"],
	start_time=v_args["startTime"],
	end_time=v_args["endTime"],
	)
	)
	sensor_data = pd.DataFrame(
	Sensor.get_sensor_data_obj(
	name="phytos31",
	cell_id=cell_id,
	measurement="voltage",
	resample=v_args["resample"],
	start_time=v_args["startTime"],
	end_time=v_args["endTime"],
	)
	)
	data_frames = [teros_data, power_data, sensor_data]
	df_merged = reduce(
	lambda left, right: pd.merge(left, right, on=["timestamp"], how="outer"),
	data_frames,
	).fillna("void")
	csv_buffer = StringIO()
	# buffer writes to memory
	for chunk in np.array_split(df_merged, 10):
	chunk.to_csv(csv_buffer, index=False, header=(csv_buffer.tell() == 0))
	return csv_buffer.getvalue()

[CODEAbhinav-art]

@jmadden173,

Moving the resampling/merge logic down to PostgreSQL and streaming it back via a Python generator in Flask seems the right move to keep memory flat and bypass Celery. Let me know if it's ok for me to take this on—so that I can start working!

[jmadden173]

@Saurabhsing21 has asked to work on this first. I'll assign him and if I don't hear anything by Monday I'll switch it over to you

[Saurabhsing21]

Thanks @jmadden173 I will work on this and send you update if i face any issue

[Saurabhsing21]

I am fixing this issue, I have replaced the current CSV path (Pandas + in-memory StringIO + large Celery result) with a DB-driven streamed export that writes CSV in small batches directly to the client.
Use one export request for all selected cells (no per-cell task fanout), add range/size limits plus logging/metrics, and re-enable the button only after staging proves no timeout/OOM.

[jmadden173]

Okay sounds good, @Saurabhsing21. What is the current progress on it and is there anything that you are getting stuck on.

[Saurabhsing21]

Hey @jmadden173 .I was facing some issue with testing, i have fixed this i will create pr today