Peano

Searching...

No Matches

A simple global database. More...

#include <GlobalDatabase.h>

Collaboration diagram for toolbox::blockstructured::GlobalDatabase:

[legend]

Data Structures
struct	Entry

Public Member Functions
	GlobalDatabase (int growthBetweenTwoDatabaseFlushes=0, double dataDelta=1e-8, double timeDelta=0.0, bool clearDatabaseAfterFlush=true, bool deltasAreRelative=false)
	Global database.

	~GlobalDatabase ()

void	clear (bool lockSemaphore=true)

void	dumpCSVFile ()
	Dump data into CSV file.

void	setDeltaBetweenTwoDatabaseFlushes (const int deltaBetweenTwoDatabaseFlushes)

void	setOutputFileName (const std::string &filename)

void	setDataName (const std::string &dataname)

void	setOutputPrecision (int precision)

void	setDataDeltaBetweenTwoSnapshots (double value, bool deltasAreRelative=false)

void	setTimeDeltaBetweenTwoSnapshots (double value)

void	clearDatabaseAfterFlush (bool value)

void	addGlobalSnapshot (double timestamp)
	Add snapshot.

void	addGlobalSnapshot (double timestamp, int numberOfDataEntries, double *data)

Private Types
enum class	AddSnapshotAction { Ignore , Append , Replace }

Private Member Functions
AddSnapshotAction	getAction (double timestamp)
	Determine what to do with new entry.

AddSnapshotAction	getAction (double timestamp, int numberOfDataEntries, double *data)
	Determine what to do with new entry.

bool	dumpDatabaseSnapshot ()

Private Attributes
std::string	_fileName

std::string	_dataName

double	_dataDelta

double	_maxDataDelta

int	_numberOfGlobalDataPoints

double	_timeDelta

int	_deltaBetweenTwoDatabaseFlushes

int	_thresholdForNextDatabaseFlush

int	_precision

bool	_clearDatabaseAfterFlush
	Flag that indicates if we erase the database after a flush.

bool	_deltasAreRelative

int	_rank
	This is a hack: Usually, I'd just ask the rank what its number is.

double	_maxTimestamp

std::list< Entry >	_data

tarch::multicore::BooleanSemaphore	_semaphore

Static Private Attributes
static tarch::logging::Log	_log

static constexpr double	_deltaCutOffThreshold = 1e-6

Detailed Description

A simple global database.

The database can be configured with various thresholds such that we write out snapshots whenever a certain maximum size is reached.

The database is thread-safe. It was built after the model of the particle Database.

Definition at line 29 of file GlobalDatabase.h.

Member Enumeration Documentation

◆ AddSnapshotAction

enum class toolbox::blockstructured::GlobalDatabase::AddSnapshotAction

strongprivate

Enumerator
Ignore
Append
Replace

Definition at line 88 of file GlobalDatabase.h.

Constructor & Destructor Documentation

◆ GlobalDatabase()

toolbox::blockstructured::GlobalDatabase::GlobalDatabase	(	int	growthBetweenTwoDatabaseFlushes = 0,
		double	dataDelta = 1e-8,
		double	timeDelta = 0.0,
		bool	clearDatabaseAfterFlush = true,
		bool	deltasAreRelative = false )

Global database.

The database dumps/stores data if the data is either for a new timestamp which is a sufficient delta after the latest one. If the new data has the same timestamp as a previous one, it will replace it if the data is different by a sufficiently large margin. We always work with the max norm.

Flushing the database

Please read the documentation of clearDatabaseAfterFlush and growthBetweenTwoDatabaseFlushes below first.

Parameters

clearDatabaseAfterFlush	If this flag is set, each flush of the database will go into a separate file, and the code will clear the database after the flush. As a consequence, the database will never exceed the memory. However, you'll get a lot of files that you have to merge afterwards.
growthBetweenTwoDatabaseFlushes	Number of entries that we dump into the database before it is flushed the next time. Set it to max of an integer or zero to disable on-the-fly dumps. In this case, the database is only flushed when the simulation terminates. So, the thresholds for the data and position deltas determine how often entries and up in the database, and growthBetweenTwoDatabaseFlushes determines how often this database is written into a file.
dataDelta	If two pieces of data are defined for the same timestamp, this parameter is used to determine whether this is re-defining the data or whether it is just a duplicate. If the former is true, the existing data will be replaced with the new definition. In the later case, the data is not added to the database and is simply discarded. toolbox::blockstructured::GlobalDatabase::getAction() for a description how the C++ code interprets this threshold, but it is usually the max norm that is used here.

This flag has no impact whatsoever how often the data is dumped into a file. The frequency of datafile writes is solely controlled via growthBetweenTwoDatabaseFlushes.

Parameters

timeDelta this parameter ask the code to dump particle into database after certain time interval of time_delta_between_two_snapsots, even data and position do not change during this time interval. You can set the two parameter above to be extremely big to enforce code dump particle with (roughly) regular time interval.

This flag has no impact whatsoever how often the data is dumped into a file. The frequency of datafile writes is solely controlled via growthBetweenTwoDatabaseFlushes.

Parameters

deltasAreRelative By default (flag is false), we take the absolute deltas of the data to decide whether to replace data with duplicates or simply assume these to be identical. If this flag is set however, we track the maximum of the deltas, and we dump data if and only if it exceeds dataDelta times this maximum. So we use a relative quantity. This might make sense if one value in your database hovers around a size of 10^-3 and another is in the range of 10^10, in which case using the same absolute delta for both is not the ideal choice.

Definition at line 44 of file GlobalDatabase.cpp.

◆ ~GlobalDatabase()

toolbox::blockstructured::GlobalDatabase::~GlobalDatabase ( )

Definition at line 61 of file GlobalDatabase.cpp.

Member Function Documentation

◆ addGlobalSnapshot() [1/2]

void toolbox::blockstructured::GlobalDatabase::addGlobalSnapshot ( double timestamp )

Add snapshot.

The data is assumed to be global, and therefore uniquely identified by its timestamp. We can track any amount of data, such as none at all (like in this function) or an arbitrary number. Hence the overload. It is the user's responsibility to use the addGlobalSnapshot() routines in a consistent way. If you've added six pieces of data once and you try adding eight pieces in a next call, the code will do what we in the business like to call an "oopsie-woopsie", by which we mean an assertion will fail and your program will crash.

Thread-safety

The routine is thread-safe. It actually locks the database before it invokes getAction() and thus will not mess up either the database analysis or any writes.

If the database is configured to write snapshots, the routine will also invoke the dump. However, it is important that we free the lock before, as I do not use recursive locks and as the dumping itself is thread-safe.

Definition at line 228 of file GlobalDatabase.cpp.

References tarch::multicore::Lock::free(), tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getRank(), and logInfo.

Here is the call graph for this function:

◆ addGlobalSnapshot() [2/2]

void toolbox::blockstructured::GlobalDatabase::addGlobalSnapshot	(	double	timestamp,
		int	numberOfDataEntries,
		double *	data )

Definition at line 263 of file GlobalDatabase.cpp.

References assertion, tarch::multicore::Lock::free(), tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getRank(), and logInfo.

Here is the call graph for this function:

◆ clear()

void toolbox::blockstructured::GlobalDatabase::clear ( bool lockSemaphore = true )

Thread-safety

The clear() operation is thread-safe if you set lockSemaphore. In this case, it first locks the sempahore and then it continues.

Definition at line 69 of file GlobalDatabase.cpp.

References tarch::multicore::Lock::lock().

Referenced by peano4.output.Makefile.Makefile::__init__().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ clearDatabaseAfterFlush()

void toolbox::blockstructured::GlobalDatabase::clearDatabaseAfterFlush ( bool value )

Definition at line 147 of file GlobalDatabase.cpp.

◆ dumpCSVFile()

void toolbox::blockstructured::GlobalDatabase::dumpCSVFile ( )

Dump data into CSV file.

The operation is thread-safe, i.e. it first locks the sempahore and then it continues. If we are supposed to clear the database once we've dumped the CSV file, we will call clear(). In this case, it is important that I keep the lock up and call clear(). If I released the lock and then called clear, some other thread might squeeze its particle update in-between and we'd loose the information.

Definition at line 80 of file GlobalDatabase.cpp.

References tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getRank(), and logInfo.

Here is the call graph for this function:

◆ dumpDatabaseSnapshot()

bool toolbox::blockstructured::GlobalDatabase::dumpDatabaseSnapshot ( )

private

Thread-safety

The routine is thread-safe, as it locks the database. As it uses the semaphore, it can't be const.

Returns: If the user should write the database into a file. This is the case if the total number of entries in the database exceeds _thresholdForNextDatabaseFlush.

Definition at line 221 of file GlobalDatabase.cpp.

References tarch::la::greater().

Here is the call graph for this function:

◆ getAction() [1/2]

toolbox::blockstructured::GlobalDatabase::AddSnapshotAction toolbox::blockstructured::GlobalDatabase::getAction ( double timestamp )

private

Determine what to do with new entry.

Can't be const as it might augment the database with default entries.

The routine is not thread-safe, i.e. you have to protect it from outside.

Definition at line 163 of file GlobalDatabase.cpp.

References Append, tarch::la::equals(), Ignore, logWarning, and Replace.

Here is the call graph for this function:

◆ getAction() [2/2]

toolbox::blockstructured::GlobalDatabase::AddSnapshotAction toolbox::blockstructured::GlobalDatabase::getAction	(	double	timestamp,
		int	numberOfDataEntries,
		double *	data )

private

Determine what to do with new entry.

First of all, invoke other getAction() routine. This one will determine if the entry is the first entry, if it's timestamp is such that it should be added to the database or discarded, and if it is at the same timestamp as the last one (in which case it might need to replace it.) In the latest case, we only replace it if the data actually differs from the existing data.

Definition at line 184 of file GlobalDatabase.cpp.

References std::abs(), Append, and Replace.

Here is the call graph for this function:

◆ setDataDeltaBetweenTwoSnapshots()

void toolbox::blockstructured::GlobalDatabase::setDataDeltaBetweenTwoSnapshots	(	double	value,
		bool	deltasAreRelative = false )

Definition at line 151 of file GlobalDatabase.cpp.

References assertion.

◆ setDataName()

void toolbox::blockstructured::GlobalDatabase::setDataName ( const std::string & dataname )

Definition at line 139 of file GlobalDatabase.cpp.

◆ setDeltaBetweenTwoDatabaseFlushes()

void toolbox::blockstructured::GlobalDatabase::setDeltaBetweenTwoDatabaseFlushes ( const int deltaBetweenTwoDatabaseFlushes )

Definition at line 131 of file GlobalDatabase.cpp.

◆ setOutputFileName()

void toolbox::blockstructured::GlobalDatabase::setOutputFileName ( const std::string & filename )

Definition at line 135 of file GlobalDatabase.cpp.

◆ setOutputPrecision()

void toolbox::blockstructured::GlobalDatabase::setOutputPrecision ( int precision )

Definition at line 143 of file GlobalDatabase.cpp.

◆ setTimeDeltaBetweenTwoSnapshots()

void toolbox::blockstructured::GlobalDatabase::setTimeDeltaBetweenTwoSnapshots ( double value )

Definition at line 158 of file GlobalDatabase.cpp.

References assertion.

Field Documentation

◆ _clearDatabaseAfterFlush

bool toolbox::blockstructured::GlobalDatabase::_clearDatabaseAfterFlush

private

Flag that indicates if we erase the database after a flush.

If not set, each dump consists of all data ever recorded. If set, you will potentially get a huge set of data files, and you will have to concatenate them before you interpret the data.

See also: GlobalDatabase(); clearDatabaseAfterFlush()

Definition at line 56 of file GlobalDatabase.h.

◆ _data

std::list<Entry> toolbox::blockstructured::GlobalDatabase::_data

private

Definition at line 84 of file GlobalDatabase.h.

Referenced by peano4.datamodel.DaStGen2.DaStGen2Generator::construct_output(), peano4.toolbox.particles.ParticleSet.ParticleSetGenerator_ScatteredOnHeap_IndexByList::construct_output(), peano4.toolbox.particles.ParticleSet.ParticleSetGenerator_ScatteredOnHeap_IndexByVector::construct_output(), peano4.toolbox.particles.ParticleSet.ParticleSetGenerator_ContinuousPerVertex::construct_output(), peano4.toolbox.particles.ParticleSet.ParticleSetGenerator_GlobalContinuous::construct_output(), peano4.datamodel.DaStGen2.DaStGen2Generator::get_header_file_include(), peano4.toolbox.particles.ParticleSet.AbstractParticleSetGenerator::get_header_file_include(), peano4.datamodel.DaStGen2.DaStGen2Generator::get_stack_container(), peano4.datamodel.DynamicArrayOverPrimitivesToStdVector.DynamicArrayOverPrimitivesToStdVector::get_stack_container(), and peano4.toolbox.particles.ParticleSet.AbstractParticleSetGenerator::get_stack_container().

◆ _dataDelta

double toolbox::blockstructured::GlobalDatabase::_dataDelta

private

Definition at line 36 of file GlobalDatabase.h.

◆ _dataName

std::string toolbox::blockstructured::GlobalDatabase::_dataName

private

Definition at line 34 of file GlobalDatabase.h.

◆ _deltaBetweenTwoDatabaseFlushes

int toolbox::blockstructured::GlobalDatabase::_deltaBetweenTwoDatabaseFlushes

private

Definition at line 42 of file GlobalDatabase.h.

◆ _deltaCutOffThreshold

constexpr double toolbox::blockstructured::GlobalDatabase::_deltaCutOffThreshold = 1e-6

staticconstexprprivate

Definition at line 59 of file GlobalDatabase.h.

◆ _deltasAreRelative

bool toolbox::blockstructured::GlobalDatabase::_deltasAreRelative

private

Definition at line 57 of file GlobalDatabase.h.

◆ _fileName

std::string toolbox::blockstructured::GlobalDatabase::_fileName

private

Definition at line 33 of file GlobalDatabase.h.

◆ _log

tarch::logging::Log toolbox::blockstructured::GlobalDatabase::_log

staticprivate

Definition at line 31 of file GlobalDatabase.h.

◆ _maxDataDelta

double toolbox::blockstructured::GlobalDatabase::_maxDataDelta

private

Definition at line 37 of file GlobalDatabase.h.

◆ _maxTimestamp

double toolbox::blockstructured::GlobalDatabase::_maxTimestamp

private

Definition at line 68 of file GlobalDatabase.h.

◆ _numberOfGlobalDataPoints

int toolbox::blockstructured::GlobalDatabase::_numberOfGlobalDataPoints

private

Definition at line 38 of file GlobalDatabase.h.

◆ _precision

int toolbox::blockstructured::GlobalDatabase::_precision

private

Definition at line 44 of file GlobalDatabase.h.

◆ _rank

int toolbox::blockstructured::GlobalDatabase::_rank

private

This is a hack: Usually, I'd just ask the rank what its number is.

However, the database dump often is called in the very end, after the MPI rank is already down. So it will return -1. So what I do is that I store this piece of data whenever I insert a new entry.

Definition at line 67 of file GlobalDatabase.h.

◆ _semaphore

tarch::multicore::BooleanSemaphore toolbox::blockstructured::GlobalDatabase::_semaphore

private

Definition at line 86 of file GlobalDatabase.h.

◆ _thresholdForNextDatabaseFlush

int toolbox::blockstructured::GlobalDatabase::_thresholdForNextDatabaseFlush

private

Definition at line 43 of file GlobalDatabase.h.

◆ _timeDelta

double toolbox::blockstructured::GlobalDatabase::_timeDelta

private

Definition at line 40 of file GlobalDatabase.h.

The documentation for this class was generated from the following files:

src/toolbox/blockstructured/GlobalDatabase.h
src/toolbox/blockstructured/GlobalDatabase.cpp

Data Structures

Public Member Functions

Private Types

Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

Member Enumeration Documentation

◆ AddSnapshotAction

Constructor & Destructor Documentation

◆ GlobalDatabase()

Flushing the database

◆ ~GlobalDatabase()

Member Function Documentation

◆ addGlobalSnapshot() [1/2]

Thread-safety

◆ addGlobalSnapshot() [2/2]

◆ clear()

Thread-safety

◆ clearDatabaseAfterFlush()

◆ dumpCSVFile()

◆ dumpDatabaseSnapshot()

Thread-safety

◆ getAction() [1/2]

◆ getAction() [2/2]

◆ setDataDeltaBetweenTwoSnapshots()

◆ setDataName()

◆ setDeltaBetweenTwoDatabaseFlushes()

◆ setOutputFileName()

◆ setOutputPrecision()

◆ setTimeDeltaBetweenTwoSnapshots()

Field Documentation

◆ _clearDatabaseAfterFlush

◆ _data

◆ _dataDelta

◆ _dataName

◆ _deltaBetweenTwoDatabaseFlushes

◆ _deltaCutOffThreshold

◆ _deltasAreRelative

◆ _fileName

◆ _log

◆ _maxDataDelta

◆ _maxTimestamp

◆ _numberOfGlobalDataPoints

◆ _precision

◆ _rank

◆ _semaphore

◆ _thresholdForNextDatabaseFlush

◆ _timeDelta