Not checkpointing non-geometric fields

[MakisH]

Our setupCheckpointing() currently looks like this:

openfoam-adapter/Adapter.C

Lines 890 to 924 in ac79b78

	void preciceAdapter::Adapter::setupCheckpointing()
	{
	SETUP_TIMER();
	// Add fields in the checkpointing list - sorted for parallel consistency
	DEBUG(adapterInfo("Adding in checkpointed fields..."));
	#undef doLocalCode
	#define doLocalCode(GeomField) \
	/* Checkpoint registered GeomField objects */ \
	for (const word& obj : mesh_.sortedNames<GeomField>()) \
	{ \
	addCheckpointField(mesh_.thisDb().getObjectPtr<GeomField>(obj)); \
	DEBUG(adapterInfo("Checkpoint " + obj + " : " #GeomField)); \
	}
	doLocalCode(volScalarField);
	doLocalCode(volVectorField);
	doLocalCode(volTensorField);
	doLocalCode(volSymmTensorField);
	doLocalCode(surfaceScalarField);
	doLocalCode(surfaceVectorField);
	doLocalCode(surfaceTensorField);
	doLocalCode(pointScalarField);
	doLocalCode(pointVectorField);
	doLocalCode(pointTensorField);
	// NOTE: Add here other object types to checkpoint, if needed.
	#undef doLocalCode
	ACCUMULATE_TIMER(timeInCheckpointingSetup_);
	}

This looks for fields of various types and adds them to a list of fields to checkpoint. It works nicely for geometric fields, but there are more that we need to track.

In particular, in private communication, users have reported issues with the residuals. We currently don't checkpoint the general solution state metrics, which can lead OpenFOAM having a wrong impression on the actual convergence, and exiting earlier, in particular when using only one outer PIMPLE iteration. The relative tolerances are absolute zero in that case, and a workaround seems to be (if I understand correctly) to enforce multiple PIMPLE iterations (nCorrectors).

Adding Info<< mesh_.thisDb().toc() << endl; prints the table of contents of the objects registry.

For the fluid-openfoam case (pimpleFoam) of the perpendicular flap, this prints:

30
(
solutionControl
Force
points
wallDist
neighbour
faceDiffusivity
MRFProperties
pointMesh
pointDisplacement
faces
U
turbulenceProperties
fvSchemes
preciceDict
fvOptions
faceZones
fvSolution
Uf
cumulativeContErr
nu
phi
owner
data
cellZones
boundary
dynamicMeshDict
p
cellDisplacement
pointZones
transportProperties
)

---[preciceAdapter] [DEBUG] Checkpoint nu : volScalarField
---[preciceAdapter] [DEBUG] Checkpoint p : volScalarField
---[preciceAdapter] [DEBUG] Checkpoint Force : volVectorField
---[preciceAdapter] [DEBUG] Checkpoint U : volVectorField
---[preciceAdapter] [DEBUG] Checkpoint cellDisplacement : volVectorField
---[preciceAdapter] [DEBUG] Checkpoint faceDiffusivity : surfaceScalarField
---[preciceAdapter] [DEBUG] Checkpoint phi : surfaceScalarField
---[preciceAdapter] [DEBUG] Checkpoint Uf : surfaceVectorField
---[preciceAdapter] [DEBUG] Checkpoint pointDisplacement : pointVectorField
---[preciceAdapter] [DEBUG] Writing a checkpoint...
---[preciceAdapter] [DEBUG] Stored time value t = 0.000000
---[preciceAdapter] [DEBUG] Storing mesh points...
---[preciceAdapter] [DEBUG] Stored mesh points.
---[preciceAdapter] [DEBUG] Checkpoint for time t = 0.000000 was stored.

which means that we are not checkpointing the following fields:

• solutionControl
• points
• wallDist
• neighbour
• MRFProperties
• pointMesh
• faces
• turbulenceProperties
• fvSchemes
• preciceDict
• fvOptions
• faceZones
• fvSolution
• cumulativeContErr
• owner
• data
• cellZones
• boundary
• dynamicMeshDict
• pointZones
• transportProperties

I am not saying we need to checkpoint everything, but probably some of these are problematic.

I already invested some time on this, but the main challenge I have is understanding how to request each field (in a generic way) and whether storing it is enough, or whether there are cross-dependencies.

I think that resolving this will probably need some refactoring of the checkpointing mechanism, which might also resolve #158.