Skip to content

Commit 4007be2

Browse files
committed
Abort when post_process is given a different grid than the restart files
post_process checks that the restart files exist but never that they hold the grid the case file asks for. Give it a case whose resolution no longer matches the run and it reads past the end of every file, post-processes the overrun, and exits 0 with NaN-filled output. Nothing reports a problem until someone plots it, or until a diagnostic built on that output starts returning NaN and gets blamed instead. That is what happened here: a 349 M cell run (m = 1644) post-processed with a case file that had since moved to m = 2056, so 681 M cells were read from files holding 349 M. x_cb.dat holds exactly m_glb + 2 cell boundaries, so its size states which grid wrote the restart. The check costs one inquire on a file already being opened, fires before any bulk read, and names both grids: Restart grid mismatch: this case has m = 2056 but .../restart_data/x_cb.dat was written with m = 1644. Post-processing must use the same grid as the run that wrote the restart files, or it reads past the end of every file and writes NaN. Claude-Session: https://claude.ai/code/session_01HMJ7cycfo7kTFSFq5yhHLG
1 parent b5715d4 commit 4007be2

1 file changed

Lines changed: 15 additions & 1 deletion

File tree

src/post_process/m_data_input.f90

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -276,6 +276,8 @@ impure subroutine s_read_parallel_data_files(t_step)
276276
integer(kind=MPI_OFFSET_KIND) :: offset
277277
character(LEN=path_len + 2*name_len) :: file_loc
278278
logical :: file_exist
279+
integer(kind=8) :: file_bytes
280+
character(len=10) :: case_m_str, file_m_str
279281
character(len=10) :: t_step_string
280282
integer :: i
281283

@@ -290,9 +292,21 @@ impure subroutine s_read_parallel_data_files(t_step)
290292
end if
291293

292294
file_loc = trim(case_dir) // '/restart_data' // trim(mpiiofs) // 'x_cb.dat'
293-
inquire (FILE=trim(file_loc), EXIST=file_exist)
295+
inquire (FILE=trim(file_loc), EXIST=file_exist, SIZE=file_bytes)
294296

297+
! The grid file holds exactly m_glb + 2 cell boundaries, so its size says which grid wrote the restart.
298+
! Without this check a case file whose resolution no longer matches the run reads past the end of every
299+
! restart file and post-processes silently, exiting 0 with NaN-filled output -- which is indistinguishable
300+
! from success until someone plots it.
295301
if (file_exist) then
302+
if (file_bytes /= int(m_glb + 2, 8)*8_8) then
303+
call s_int_to_str(m_glb, case_m_str)
304+
call s_int_to_str(int(file_bytes/8_8) - 2, file_m_str)
305+
call s_mpi_abort('Restart grid mismatch: this case has m = ' // trim(case_m_str) // &
306+
' but ' // trim(file_loc) // ' was written with m = ' // trim(file_m_str) // &
307+
'. Post-processing must use the same grid as the run that wrote the ' // &
308+
'restart files, or it reads past the end of every file and writes NaN.')
309+
end if
296310
data_size = m_glb + 2
297311
call MPI_FILE_OPEN(MPI_COMM_WORLD, file_loc, MPI_MODE_RDONLY, mpi_info_int, ifile, ierr)
298312

0 commit comments

Comments
 (0)