Operation simulation and data evaluation

Palabos user documentation Chapter 10 and Chapter 12 of

(1) Running A Simulation

Original document

Time cycles of a Palabos program

The lattice Boltzmann method (or at least, the "classical" lattice Boltzmann method as it is implemented in Palabos) consists of an explicit solver. With a single iteration step, the state of the fluid evolves from a time t to the next time t+dt. The following discussion is led in the units of the lattice, with dt=1. One time iteration of this kind takes the following form in Palabos:

1. To start with, all fluid variables are defined at time t. The particle populations are in pre-collision state (also called "incoming populations").
2. The collision operator is applied to all cells. They are now in post-collision state (also called "outgoing variables").
3. The streaming operator is applied to the lattice.
4. All data processors are executed, to perform non-local operations or couplings between lattices.
5. The populations are now again in pre-collision state, but at time t+1.

The collision step (Step 2) is executed by invoking the method lattice.collide(), and the streaming step (Step 3) is called with the method lattice.stream(). These two methods can also be combined into a single call to lattice.collideAndStream(). On most hardware platforms, the collideAndStream() version is computationally more efficient, because it is executed by traveling through the memory of the lattice only once.
The data processors can be executed manually by calling the method lattice. executeInternalProcessors(), as explained in Section Executing, integrating, and wrapping up dataprocessing functionals. This is however rarely done, because the data processors are also executed automatically at the end of the function stream() and the function collideAndStream(). If you'd like to call stream() or collideAndStream without having the data processors executed, for example for debugging a program, you can use the domain-version of these functions, for example lattice.collideAndStream(lattice.getBoundingBox()).
In Palabos, data processors are always executed after the collision-streaming cycle. Consequently, non-local operations are always executed after the collision-streaming cycle, at a moment where the populations are in a pre-collision state (incoming populations). This bears no loss of generality, though, because executing an operation before the collision of time t is equivalent to executing it after the streaming of time t-1, right? The only problem arises with the initial condition, as it is most often desired to have the data processors executed exactly once at the very beginning, right after setting up the initial condition. This is achieved by calling the method lattice.initialize() right before starting the first iteration step. This method executes the data processors once, and performs an internal communication step inside the block to guarantee that its internal state is consistent.

At which point do you evaluate data?

To monitor the evolution of a program, it is useful evaluate some hydrodynamic quantities from time to time, such as the average energy:
pcout << computeAverageEnergy(lattice) << endl;
It is recommended to compute hydrodynamic variables always only when the system is in pre-collision state (incoming populations). While this distinction not really matters for the conserved variables density and velocity (they are equal in the pre- and post-collision state), it is important for the non-conserved variables such as the stress tensor. Non-conserved velocity moments can be related to hydrodynamic variables only when they are computed in the precollision state. Note that if you use the method collideAndStream(), there is no risk for doing things wrong, because you have no access to post-collision variables anyway.
Normally, the computation of hydrodynamic variables like the average energy in the example above is performed right after the call to the method collideAndStream(), because at this point all hydrodynamic variables are well defined, and correspond to the same moment in time (between collision and streaming the velocity is well defined, but the strain-rate is not). An exception is made for the internal statistics of lattice. Internal statistics are automatically computed without any impact on performance (at least not in serial program; for the parallel case, see the discussion in Section Controlling the efficiency), as a side-effect of executing the collision step. They are however evaluated for the fluid variables at time t, during the collision-streaming cycle which carries the system from time t to time t+1. It is therefore usual to access the internal statistics after the call to the method collideAndStream(). Computing the average energy as in the example above before collision-and-streaming produces the same result as accessing the average energy from internal statistics, as in the example below, after collision-and-streaming:
pcout << getStoredAverageEnergy(lattice) << endl;

Other important things to do

Numbers are often difficult to interpret. It is therefore useful to regularly produce images in your program, so that you can monitor the state of simulation, identify problems as soon as possible, and re-run the program when needed. Section Producing images in 2D and 3D simulations explains how to do this.
Finally, it is always good to save the state of a simulation from time to time, in order not to loose everything when the computer crashes, and in order to be able to recover the data if you forgot to produce a crucial output file for post-processing. This is explained in Section Checkpointing: saving and loading the state of a simulation.

Document translation

Time period of Palabos program

The gerzmann method (or at least the "classic" gerzmann method implemented in Palabos) contains an explicit solver. Through one iterative step, the state of fluid evolves from time t to the next time t+dt. According to the following discussion, dt=1 in lattice units. This time iteration takes the following form in Palabos:

1. First of all, all fluid variables are defined at time t, and particle swarm is in the state before collision (also known as incident particle swarm).
2. The collision action applies to all cells. They are now in the post collision state (also known as output variables).
3. The flow operation is applied to the lattice.
4. All data processors are executed to perform non local operations or inter lattice coupling.
5. The population is now back to its pre collision state, but the time is t+1.

Perform the collision step by calling the method lattice.collision() (step 2), and then use the method lattice.stream() to call the flow step (step 3). The two methods can also be combined to make a single call to lattice.collideandstream(). On most hardware platforms, the collideAndStream() version is more efficient because it runs only once in the memory of the lattice layer.
Data processors can be executed manually by calling the method lattice.executeInternalProcessors(), as described in the section executing, integrating, and wrapping up data processing functions (document section 16.3.4). But this rarely happens, because the data processor also executes automatically at the end of the functions stream() and collideAndStream(). If you want to call stream() or collideAndStream() without executing the data processor, for example, to debug a program, you can use the domain version of these functions, such as lattice.collideAndStream(lattice.getboundingbox()).
In Palabos, the data processor always executes after the collision flow evolution loop. Therefore, non local operations are always performed after the collision flow cycle, when the population is in the pre collision state (incident particle swarm). However, this does not lose generality, because performing an operation before t-Time collision is equivalent to performing it after t-1 time flow, right? The only problem is with the initial conditions, because the most common situation is that the data processor executes exactly once after the initial conditions are set. This is achieved by calling the method initialize() before starting the first iteration step. This method executes the data processor only once, and performs internal communication steps within the block to ensure that its internal state is consistent.

When is the data evaluated?

In order to monitor the development of the program, it is useful to evaluate some hydrodynamic quantities from time to time, such as average energy:
pcout << computeAverageEnergy(lattice) << endl;
It is recommended that hydrodynamic variables be calculated only when the system is in the pre collision state (incident particle swarm). Although this difference is not important for the density and velocity of conservation variables (they are equal before and after collision), it is important for non conservation variables, such as stress tensors. The nonconservative velocity moment can only be related to the hydrodynamic variables when it is calculated in the pre collision state. Note that if you use the collideAndStream() method, there is no risk of error, because you cannot access the variables after collision in any case.
In general, the hydrodynamic variables (such as the average energy in the above example) are calculated after the method collideAndStream() is called, because at this time, all the hydrodynamic variables corresponding to the same time are clearly defined (the velocity between collision and flow is also clearly defined, but the strain rate is not). The internal statistics of lattice is an exception. The internal statistics is an automatic calculation that will not have any impact on the performance (at least not in the serial program; for the parallel situation, please refer to the discussion in the original 15.2Controlling the efficiency section), which is a side effect of performing the collision step. However, in the collision flow cycle (from time t to time t+1), they are evaluated in terms of fluid variables at time t. As a result, internal statistics are usually accessed after a call to the collideAndStream() method. In the above example, calculate and calculate the average energy before collision and flow, and in the following example, obtain the average energy value in internal statistics after collision and flow, with the same result:
pcout << getStoredAverageEnergy(lattice) << endl;

Other important things

Numbers are often difficult to explain. Therefore, it is useful to generate images regularly in the program, so that you can monitor the status of the simulation, identify problems as soon as possible, and rerun the program if necessary. Generating images in 2D and 3D simulations explains how to do this.
Finally, it's always good to keep the state of the simulation regularly so that you don't lose everything when the computer crashes, and if you forget to generate critical output files for post-processing, you can also recover the data. This will be explained in checking: saving and loading the state of a simulation.

interpretative statement

This part mainly introduces the collision flow cycle in the program, which is very clear. The collideAndStream() method integrates the calculation and statistics steps of the data processor, which can be called directly without worrying about errors. The calculation steps of the data processor are used after the collision and flow, but in fact, after you set various internal and boundary conditions for your simulation, its internal particle swarm It is not set well, or the original state. Only after calling initialize(), the data processing step is called once, the particle swarm is formally initialized.
Sometimes I will use the version that does not include data processing steps in the original text. In the middle, I will write some correct operations in the code instead of the correct operations in the simulation calculation. Finally, I will call the corresponding function of data processing steps to realize the operation. The data I get is very good-looking but not necessarily right. Hee hee hee.