Rename atomicity is not enough

[socketpair]

http://stackoverflow.com/questions/3764822/how-to-durably-rename-a-file-in-posix

So, you need to open directory where file exists (O_DIRECOTRY), and fsync() it's FD after rename. If not, application may think that rename is durable, but on accidental reboot, old file may appear.

[davidhq]

Using this lib on a project to write program state in .json file between process restarts...

Very occasionally the state is empty on restart of the process (mostly when the entire device reboots suddenly .. or (not sure) when there's a program error)..

Anyway my question is: can 100% reliable atomic file writes be achieved (with this library or in general) on Unix-like systems... and on SD cards ..

This is happening when testing the nodejs process on RaspberryPies ... (file system on SD card)

Could your claim be related to the problem I'm experiencing?

It is also possible that the issue is elsewhere but I'm starting to debug at this side... first I want to be really sure that this library / approach does not have any edge cases...

[socketpair]

@davidhq No, this is another issue. It's about possible revert of atomic rename on power loss.

F.Y.I. Atomic rename is NOT implemented on FAT32.

[davidhq]

Thank you... I use ext4 on all SD cards.

So I can be reasonably certain that the issue is elsewhere and no data loss is occuring because of failed atomic file renames?

[socketpair]

@davidhq I don't know. Renames in Linux (in normal FS) are always atomic. Possibly this library does not handle errors from write and fsync. Possibly your storage stack ignores flushes, or report completion when flush is still not complete. Or your SD card is a crap (i.e. it reports flushes as complete, when they are not).

[davidhq]

Thank you for explaining! In the meantime I have figured out the most likely culprit and it's something else... a very subtle bug before data is written, I actually passed empty data by coincidence at some rare cases and when this was the last write before process exited, then the file was empty... :/