Operator playbooks (symptom → fix)¶
Short if-this-then-that pages for production-style failures. Deep background stays in troubleshooting.md and mpi_io_layout_checklist.md.
Hang or deadlock at first result write¶
Check |
Action |
|---|---|
All ranks call the writer path |
|
Same |
Mismatched |
Writable path from job cwd |
Relative paths in JSON are relative to the process cwd, not the input file location. |
Wrong or “empty” binary files¶
Check |
Action |
|---|---|
No header in file |
Raw layout only — you must track |
Mixed endianness |
Files are native endian; do not mix writers/readers across incompatible architectures without conversion. |
HeFFTe / FFT errors at plan or first transform¶
Check |
Action |
|---|---|
HeFFTe built with same MPI as OpenPFC |
Rebuild HeFFTe or fix |
GPU backend |
CUDA/HIP toolkit on PATH; GPU HeFFTe prefix matches |
NaNs or diverging step¶
Check |
Action |
|---|---|
|
Reduce |
Debug builds |
NaN checks may abort with a useful message — see |
See also¶
hpc_operator_guide.md— index of HPC docs