Peano

Searching...

No Matches

A simple particle database. More...

#include <TrajectoryDatabase.h>

Collaboration diagram for toolbox::particles::TrajectoryDatabase:

[legend]

Data Structures
struct	Entry

Public Member Functions
	TrajectoryDatabase (int growthBetweenTwoDatabaseFlushes, double positionDelta=1e-8, double dataDelta=1e-8, double timeDelta=0.0, bool clearDatabaseAfterFlush=true, bool deltasAreRelative=false)
	Trajectory database.

	~TrajectoryDatabase ()

void	clear (bool lockSemaphore=true)

void	clearHistory (bool lockSemaphore=true)
	This call does not throw away all particles, but it throws away all the history behind the particles.

void	dumpCSVFile ()
	Dump data into CSV file.

void	setOutputFileName (const std::string &filename)

void	setOutputPrecision (int precision)

void	setDataDeltaBetweenTwoSnapshots (double value, bool deltasAreRelative=false)

void	setPositionDeltaBetweenTwoSnapshots (double value, bool deltasAreRelative=false)

void	setTimeDeltaBetweenTwoSnapshots (double value)

void	clearDatabaseAfterFlush (bool value)

void	addParticleSnapshot (const std::pair< int, int > &number, double timestamp, const tarch::la::Vector< Dimensions, double > &x)
	Add particle snapshot.

void	addParticleSnapshot (int number0, int number1, double timestamp, const tarch::la::Vector< Dimensions, double > &x)

void	addParticleSnapshot (const std::pair< int, int > &number, double timestamp, const tarch::la::Vector< Dimensions, double > &x, int numberOfDataEntries, double *data)

void	addParticleSnapshot (int number0, int number1, double timestamp, const tarch::la::Vector< Dimensions, double > &x, int numberOfDataEntries, double *data)

Private Types
enum class	AddSnapshotAction { Ignore , Append , Replace }

Private Member Functions
AddSnapshotAction	getAction (const std::pair< int, int > &number, const tarch::la::Vector< Dimensions, double > &x, double timestamp)
	Determine what to do with new entry.

AddSnapshotAction	getAction (const std::pair< int, int > &number, const tarch::la::Vector< Dimensions, double > &x, double timestamp, int numberOfDataEntries, double *data)
	Determine what to do with new entry.

bool	dumpDatabaseSnapshot ()

Private Attributes
std::string	_fileName

double	_dataDelta

double	_positionDelta

double	_maxDataDelta

double	_maxPositionDelta

int	_numberOfDataPointsPerParticle

double	_timeDelta

const int	_deltaBetweenTwoDatabaseFlushes

int	_thresholdForNextDatabaseFlush

int	_precision

bool	_clearDatabaseAfterFlush
	Flag that indicates if we erase the database after a flush.

bool	_deltasAreRelative

int	_rank
	This is a hack: Usually, I'd just ask the rank what its number is.

double	_maxTimestamp

std::map< std::pair< int, int >, std::list< Entry > >	_data

tarch::multicore::BooleanSemaphore	_semaphore

Static Private Attributes
static tarch::logging::Log	_log

static constexpr double	_deltaCutOffThreshold = 1e-6

Detailed Description

A simple particle database.

The database can be configured with various thresholds such that we write out snapshots whenever a certain maximum size is reached.

The database is thread-safe.

Definition at line 29 of file TrajectoryDatabase.h.

Member Enumeration Documentation

◆ AddSnapshotAction

enum class toolbox::particles::TrajectoryDatabase::AddSnapshotAction

strongprivate

Enumerator
Ignore
Append
Replace

Definition at line 91 of file TrajectoryDatabase.h.

Constructor & Destructor Documentation

◆ TrajectoryDatabase()

toolbox::particles::TrajectoryDatabase::TrajectoryDatabase	(	int	growthBetweenTwoDatabaseFlushes,
		double	positionDelta = 1e-8,
		double	dataDelta = 1e-8,
		double	timeDelta = 0.0,
		bool	clearDatabaseAfterFlush = true,
		bool	deltasAreRelative = false )

Trajectory database.

The database dumps/stores data if and only if the delta of two particles is bigger than a threshold. We always work with the max norm. There's two different thresholds: one for the position, one for the data. So whenever a particle moves by more than positionDelta in any component of its position, we write a new snapshot of the particle. Whenever one of the particle's values changes by more than dataDelta in one of its components, we write a new snapshot of this particle (even though it might not have moved).

Flushing the database

Please read the documentation of clearDatabaseAfterFlush and growthBetweenTwoDatabaseFlushes below first. If you flush a database every now and then and if you clear the database after that, then the following situation can happen: One particle's data or position changes quite a lot. Another particle's data doesn't change at all. We trigger a flush and, after that, clear the database. Again, the one single particle is updated quite a lot. We flush again. The particle that doesn't change et al will not be contained in the second database snapshot.

Parameters

clearDatabaseAfterFlush	If this flag is set, each flush of the database will go into a separate file, and the code will clear the database after the flush. As a consequence, the database will never exceed the memory. However, you'll get a lot of files that you have to merge afterwards.
growthBetweenTwoDatabaseFlushes	Number of entries that we dump into the database before it is flushed the next time. Set it to max of an integer or zero to disable on-the-fly dumps. In this case, the database is only flushed when the simulation terminates. So, the thresholds for the data and position deltas determine how often entries and up in the database, and growthBetweenTwoDatabaseFlushes determines how often this database is written into a file.
positionDelta	The code dumps a new particle if and only if it is not in the database or if its position differs from the last position tracked by more than positionDelta. Therefore, the flag is similar to dataDelta but this time it is not the difference in the data that triggers a dump into the database, but a change of position. Obviously, these things can be totally independent (and also can be combined), as particles that move with the domain might not change their value, as they are tracers, while stationary seismograms usually do not change their position at all.

If you work with stationary particles, you can set this parameter to literally anything, as it will never become relevant. If you want the dump to write a snapshot only after fixed time intervals, set this value to a very high threshold, so it never kicks in and triggers a data dump. In this case, only the time stamp differences will issue data writes.

This flag has no impact whatsoever how often the data is dumped into a file. The frequency of datafile writes is solely controlled via growthBetweenTwoDatabaseFlushes.

Parameters

dataDelta Compare to positionDelta, but this time we analyse the data held by the particle. A particle is dumped into the database if it has switched the domain partition or its position has changed more then delta_between_two_snapshots. See toolbox::particles::TrajectoryDatabase::getAction() for a description how the C++ code interprets this threshold, but it is usually the max norm that is used here. If you set it to a very small number, you'll get a lot of entries in your database.

This flag has no impact whatsoever how often the data is dumped into a file. The frequency of datafile writes is solely controlled via growthBetweenTwoDatabaseFlushes.

Parameters

timeDelta this parameter ask the code to dump particle into database after certain time interval of time_delta_between_two_snapsots, even data and position do not change during this time interval. You can set the two parameter above to be extremely big to enforce code dump particle with (roughly) regular time interval.

This flag has no impact whatsoever how often the data is dumped into a file. The frequency of datafile writes is solely controlled via growthBetweenTwoDatabaseFlushes.

Parameters

deltasAreRelative By default (flag is false), we take the absolute deltas of the position or data to make a decision if to dump a particle or not. If this flag is set however, we track the maximum of the deltas, and we dump data if and only if it exceeds positionDelta times this maximum. So we use a relative quantity.

Definition at line 45 of file TrajectoryDatabase.cpp.

◆ ~TrajectoryDatabase()

toolbox::particles::TrajectoryDatabase::~TrajectoryDatabase ( )

Definition at line 62 of file TrajectoryDatabase.cpp.

Member Function Documentation

◆ addParticleSnapshot() [1/4]

void toolbox::particles::TrajectoryDatabase::addParticleSnapshot	(	const std::pair< int, int > &	number,
		double	timestamp,
		const tarch::la::Vector< Dimensions, double > &	x )

Add particle snapshot.

A particle is always uniquely identified by two integers (an integer pair). This way, we can initialise (hand out) particle numbers without any semaphore.

I do not really care how many attributes one tracks per particle. We can track none at all (like in this function) or an arbitrary number. This is the reason why this operation is heavily overloaded. It is the user's responsibility to use the addParticleSnapshot() routines in a consistent way.

Thread-safety

The routine is thread-safe. It actually locks the database before it invokes getAction() and thus will not mess up either the database analysis or any writes.

If the database is configured to write snapshots, the routine will also invoke the dump. However, it si important that we free the lock before, as I do not use recursive locks and as the dumping itself is thread-safe.

Definition at line 306 of file TrajectoryDatabase.cpp.

References tarch::multicore::Lock::free(), tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getRank(), and logInfo.

Here is the call graph for this function:

◆ addParticleSnapshot() [2/4]

void toolbox::particles::TrajectoryDatabase::addParticleSnapshot	(	const std::pair< int, int > &	number,
		double	timestamp,
		const tarch::la::Vector< Dimensions, double > &	x,
		int	numberOfDataEntries,
		double *	data )

Definition at line 356 of file TrajectoryDatabase.cpp.

References assertion, tarch::multicore::Lock::free(), tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getRank(), and logInfo.

Here is the call graph for this function:

◆ addParticleSnapshot() [3/4]

void toolbox::particles::TrajectoryDatabase::addParticleSnapshot	(	int	number0,
		int	number1,
		double	timestamp,
		const tarch::la::Vector< Dimensions, double > &	x )

Definition at line 342 of file TrajectoryDatabase.cpp.

◆ addParticleSnapshot() [4/4]

void toolbox::particles::TrajectoryDatabase::addParticleSnapshot	(	int	number0,
		int	number1,
		double	timestamp,
		const tarch::la::Vector< Dimensions, double > &	x,
		int	numberOfDataEntries,
		double *	data )

Definition at line 397 of file TrajectoryDatabase.cpp.

◆ clear()

void toolbox::particles::TrajectoryDatabase::clear ( bool lockSemaphore = true )

Thread-safety

The clear() operation is thread-safe if you set lockSemaphore. In this case, it first locks the sempahore and then it continues.

Definition at line 70 of file TrajectoryDatabase.cpp.

References tarch::multicore::Lock::lock().

Referenced by peano4.output.Makefile.Makefile::__init__().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ clearDatabaseAfterFlush()

void toolbox::particles::TrajectoryDatabase::clearDatabaseAfterFlush ( bool value )

Definition at line 183 of file TrajectoryDatabase.cpp.

◆ clearHistory()

void toolbox::particles::TrajectoryDatabase::clearHistory ( bool lockSemaphore = true )

This call does not throw away all particles, but it throws away all the history behind the particles.

Definition at line 81 of file TrajectoryDatabase.cpp.

References tarch::multicore::Lock::lock().

Here is the call graph for this function:

◆ dumpCSVFile()

void toolbox::particles::TrajectoryDatabase::dumpCSVFile ( )

Dump data into CSV file.

The operation is thread-safe, i.e. it first locks the sempahore and then it continues. If we are supposed to clear the database once we've dumped the CSV file, we will call clear(). In this case, it is important that I keep the lock up and call clear(). If I released the lock and then called clear, some other thread might squeeze its particle update in-between and we'd loose the information.

Definition at line 98 of file TrajectoryDatabase.cpp.

References tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getRank(), and logInfo.

Here is the call graph for this function:

◆ dumpDatabaseSnapshot()

bool toolbox::particles::TrajectoryDatabase::dumpDatabaseSnapshot ( )

private

Thread-safety

The routine is thread-safe, as it locks the database. As it uses the semaphore, it can't be const.

Returns: If the user should write the database into a file. This is the case if the total number of entries in the database exceeds _thresholdForNextDatabaseFlush.

Definition at line 289 of file TrajectoryDatabase.cpp.

References tarch::la::greater().

Here is the call graph for this function:

◆ getAction() [1/2]

toolbox::particles::TrajectoryDatabase::AddSnapshotAction toolbox::particles::TrajectoryDatabase::getAction	(	const std::pair< int, int > &	number,
		const tarch::la::Vector< Dimensions, double > &	x,
		double	timestamp )

private

Determine what to do with new entry.

Can't be const as it might augment the database with default entries.

The routine is not thread-safe, i.e. you have to protect it from outside.

Definition at line 207 of file TrajectoryDatabase.cpp.

References Append, tarch::la::equals(), Ignore, logWarning, tarch::la::norm2(), and Replace.

Here is the call graph for this function:

◆ getAction() [2/2]

toolbox::particles::TrajectoryDatabase::AddSnapshotAction toolbox::particles::TrajectoryDatabase::getAction	(	const std::pair< int, int > &	number,
		const tarch::la::Vector< Dimensions, double > &	x,
		double	timestamp,
		int	numberOfDataEntries,
		double *	data )

private

Determine what to do with new entry.

First of all, invoke other getAction() routine. This one will determine if the entry is new or if the positions have changed significantly or if the time stamp means that this entry is due to be plotted. If and only if the other getAction() yields Ignore, then study the data if the data deltas are big enough.

Definition at line 255 of file TrajectoryDatabase.cpp.

References std::abs(), Append, tarch::la::equals(), Ignore, and Replace.

Here is the call graph for this function:

◆ setDataDeltaBetweenTwoSnapshots()

void toolbox::particles::TrajectoryDatabase::setDataDeltaBetweenTwoSnapshots	(	double	value,
		bool	deltasAreRelative = false )

Definition at line 187 of file TrajectoryDatabase.cpp.

References assertion.

◆ setOutputFileName()

void toolbox::particles::TrajectoryDatabase::setOutputFileName ( const std::string & filename )

Definition at line 175 of file TrajectoryDatabase.cpp.

◆ setOutputPrecision()

void toolbox::particles::TrajectoryDatabase::setOutputPrecision ( int precision )

Definition at line 179 of file TrajectoryDatabase.cpp.

◆ setPositionDeltaBetweenTwoSnapshots()

void toolbox::particles::TrajectoryDatabase::setPositionDeltaBetweenTwoSnapshots	(	double	value,
		bool	deltasAreRelative = false )

Definition at line 195 of file TrajectoryDatabase.cpp.

References assertion.

◆ setTimeDeltaBetweenTwoSnapshots()

void toolbox::particles::TrajectoryDatabase::setTimeDeltaBetweenTwoSnapshots ( double value )

Definition at line 202 of file TrajectoryDatabase.cpp.

References assertion.

Field Documentation

◆ _clearDatabaseAfterFlush

bool toolbox::particles::TrajectoryDatabase::_clearDatabaseAfterFlush

private

Flag that indicates if we erase the database after a flush.

If not set, each dump consists all particle data ever recorded. If set, you will potentially get a huge set of data files, and you will have to concatenate them before you interpret the data.

See also: TrajectoryDatabase(); clearDatabaseAfterFlush()

Definition at line 56 of file TrajectoryDatabase.h.

◆ _data

std::map<std::pair<int, int>, std::list<Entry> > toolbox::particles::TrajectoryDatabase::_data

private

Definition at line 87 of file TrajectoryDatabase.h.

Referenced by peano4.datamodel.DaStGen2.DaStGen2Generator::construct_output(), peano4.toolbox.particles.ParticleSet.ParticleSetGenerator_ScatteredOnHeap_IndexByList::construct_output(), peano4.toolbox.particles.ParticleSet.ParticleSetGenerator_ScatteredOnHeap_IndexByVector::construct_output(), peano4.toolbox.particles.ParticleSet.ParticleSetGenerator_ContinuousPerVertex::construct_output(), peano4.toolbox.particles.ParticleSet.ParticleSetGenerator_GlobalContinuous::construct_output(), peano4.datamodel.DaStGen2.DaStGen2Generator::get_header_file_include(), peano4.toolbox.particles.ParticleSet.AbstractParticleSetGenerator::get_header_file_include(), peano4.datamodel.DaStGen2.DaStGen2Generator::get_stack_container(), peano4.datamodel.DynamicArrayOverPrimitivesToStdVector.DynamicArrayOverPrimitivesToStdVector::get_stack_container(), and peano4.toolbox.particles.ParticleSet.AbstractParticleSetGenerator::get_stack_container().

◆ _dataDelta

double toolbox::particles::TrajectoryDatabase::_dataDelta

private

Definition at line 34 of file TrajectoryDatabase.h.

◆ _deltaBetweenTwoDatabaseFlushes

const int toolbox::particles::TrajectoryDatabase::_deltaBetweenTwoDatabaseFlushes

private

Definition at line 42 of file TrajectoryDatabase.h.

◆ _deltaCutOffThreshold

constexpr double toolbox::particles::TrajectoryDatabase::_deltaCutOffThreshold = 1e-6

staticconstexprprivate

Definition at line 60 of file TrajectoryDatabase.h.

◆ _deltasAreRelative

bool toolbox::particles::TrajectoryDatabase::_deltasAreRelative

private

Definition at line 58 of file TrajectoryDatabase.h.

◆ _fileName

std::string toolbox::particles::TrajectoryDatabase::_fileName

private

Definition at line 33 of file TrajectoryDatabase.h.

◆ _log

tarch::logging::Log toolbox::particles::TrajectoryDatabase::_log

staticprivate

Definition at line 31 of file TrajectoryDatabase.h.

◆ _maxDataDelta

double toolbox::particles::TrajectoryDatabase::_maxDataDelta

private

Definition at line 36 of file TrajectoryDatabase.h.

◆ _maxPositionDelta

double toolbox::particles::TrajectoryDatabase::_maxPositionDelta

private

Definition at line 37 of file TrajectoryDatabase.h.

◆ _maxTimestamp

double toolbox::particles::TrajectoryDatabase::_maxTimestamp

private

Definition at line 70 of file TrajectoryDatabase.h.

◆ _numberOfDataPointsPerParticle

int toolbox::particles::TrajectoryDatabase::_numberOfDataPointsPerParticle

private

Definition at line 38 of file TrajectoryDatabase.h.

◆ _positionDelta

double toolbox::particles::TrajectoryDatabase::_positionDelta

private

Definition at line 35 of file TrajectoryDatabase.h.

◆ _precision

int toolbox::particles::TrajectoryDatabase::_precision

private

Definition at line 44 of file TrajectoryDatabase.h.

◆ _rank

int toolbox::particles::TrajectoryDatabase::_rank

private

This is a hack: Usually, I'd just ask the rank what its number is.

However, the database dump often is called in the very end, after the MPI rank is already down. So it will return -1. So what I do is that I store this piece of data whenever I insert a new entry.

Definition at line 68 of file TrajectoryDatabase.h.

◆ _semaphore

tarch::multicore::BooleanSemaphore toolbox::particles::TrajectoryDatabase::_semaphore

private

Definition at line 89 of file TrajectoryDatabase.h.

◆ _thresholdForNextDatabaseFlush

int toolbox::particles::TrajectoryDatabase::_thresholdForNextDatabaseFlush

private

Definition at line 43 of file TrajectoryDatabase.h.

◆ _timeDelta

double toolbox::particles::TrajectoryDatabase::_timeDelta

private

Definition at line 40 of file TrajectoryDatabase.h.

The documentation for this class was generated from the following files:

src/toolbox/particles/TrajectoryDatabase.h
src/toolbox/particles/TrajectoryDatabase.cpp

Data Structures

Public Member Functions

Private Types

Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

Member Enumeration Documentation

◆ AddSnapshotAction

Constructor & Destructor Documentation

◆ TrajectoryDatabase()

Flushing the database

◆ ~TrajectoryDatabase()

Member Function Documentation

◆ addParticleSnapshot() [1/4]

Thread-safety

◆ addParticleSnapshot() [2/4]

◆ addParticleSnapshot() [3/4]

◆ addParticleSnapshot() [4/4]

◆ clear()

Thread-safety

◆ clearDatabaseAfterFlush()

◆ clearHistory()

◆ dumpCSVFile()

◆ dumpDatabaseSnapshot()

Thread-safety

◆ getAction() [1/2]

◆ getAction() [2/2]

◆ setDataDeltaBetweenTwoSnapshots()

◆ setOutputFileName()

◆ setOutputPrecision()

◆ setPositionDeltaBetweenTwoSnapshots()

◆ setTimeDeltaBetweenTwoSnapshots()

Field Documentation

◆ _clearDatabaseAfterFlush

◆ _data

◆ _dataDelta

◆ _deltaBetweenTwoDatabaseFlushes

◆ _deltaCutOffThreshold

◆ _deltasAreRelative

◆ _fileName

◆ _log

◆ _maxDataDelta

◆ _maxPositionDelta

◆ _maxTimestamp

◆ _numberOfDataPointsPerParticle

◆ _positionDelta

◆ _precision

◆ _rank

◆ _semaphore

◆ _thresholdForNextDatabaseFlush

◆ _timeDelta