Skip to content

[local] Write files atomically to avoid 500s on concurrent reads#30

Merged
JornC merged 1 commit into
aerius:mainfrom
JornC:local-storage-atomic-write
Jun 30, 2026
Merged

[local] Write files atomically to avoid 500s on concurrent reads#30
JornC merged 1 commit into
aerius:mainfrom
JornC:local-storage-atomic-write

Conversation

@JornC

@JornC JornC commented Jun 30, 2026

Copy link
Copy Markdown
Member

The file-server intermittently returns HTTP 500 on GET of a stored file when the same file is overwritten while it is being read.

Root cause: LocalFileStorageSevice.putFile used Files.copy(in, file, REPLACE_EXISTING), which is not atomic. The JDK implementation deletes the target and then recreates it (CREATE_NEW) before streaming the bytes, so during an overwrite the file briefly does not exist. LocalFileController.getFile checks existence and returns a FileUrlResource, but the read and contentLength() happen afterwards, when Spring's ResourceHttpMessageConverter serializes the body, outside the controller's try/catch. If the writer's delete lands in that window, contentLength() throws FileNotFoundException, which escapes to the dispatcherServlet as a 500.

This surfaced in Register: on upload the API writes validation.json twice (a pending result, then the real result) while the frontend polls the validation-result endpoint in a tight loop, so overwrites overlap live reads. A single 500 there makes the upload fail for the user. Only the local storage profile is affected; the s3 profile redirects to an S3 URL and S3 overwrites are atomic.

Fix: stage the content in a temp file in the same directory and ATOMIC_MOVE it into place, so a concurrent reader always sees either the complete old file or the complete new one.

Reproduction, 8 concurrent readers and one writer overwriting for 3s:
non-atomic copy: 339751 reads failed with FileNotFoundException (each a 500)
atomic move: 0 failures

@JornC JornC merged commit b62d687 into aerius:main Jun 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants