Peano 4
Loading...
Searching...
No Matches
toolbox::blockstructured::GlobalDatabase Class Reference

A simple global database. More...

#include <GlobalDatabase.h>

Collaboration diagram for toolbox::blockstructured::GlobalDatabase:

Data Structures

struct  Entry
 

Public Member Functions

 GlobalDatabase (int growthBetweenTwoDatabaseFlushes, double dataDelta=1e-8, double timeDelta=0.0, bool clearDatabaseAfterFlush=true, bool deltasAreRelative=false)
 Global database.
 
 ~GlobalDatabase ()
 
void clear (bool lockSemaphore=true)
 
void dumpCSVFile ()
 Dump data into CSV file.
 
void setOutputFileName (const std::string &filename)
 
void setDataName (const std::string &dataname)
 
void setOutputPrecision (int precision)
 
void setDataDeltaBetweenTwoSnapshots (double value, bool deltasAreRelative=false)
 
void setTimeDeltaBetweenTwoSnapshots (double value)
 
void clearDatabaseAfterFlush (bool value)
 
void addGlobalSnapshot (double timestamp)
 Add snapshot.
 
void addGlobalSnapshot (double timestamp, int numberOfDataEntries, double *data)
 

Private Types

enum class  AddSnapshotAction { Ignore , Append , Replace }
 

Private Member Functions

AddSnapshotAction getAction (double timestamp)
 Determine what to do with new entry.
 
AddSnapshotAction getAction (double timestamp, int numberOfDataEntries, double *data)
 Determine what to do with new entry.
 
bool dumpDatabaseSnapshot ()
 

Private Attributes

std::string _fileName
 
std::string _dataName
 
double _dataDelta
 
double _maxDataDelta
 
int _numberOfGlobalDataPoints
 
double _timeDelta
 
const int _deltaBetweenTwoDatabaseFlushes
 
int _thresholdForNextDatabaseFlush
 
int _precision
 
bool _clearDatabaseAfterFlush
 Flag that indicates if we erase the database after a flush.
 
bool _deltasAreRelative
 
int _rank
 This is a hack: Usually, I'd just ask the rank what its number is.
 
double _maxTimestamp
 
std::list< Entry_data
 
tarch::multicore::BooleanSemaphore _semaphore
 

Static Private Attributes

static tarch::logging::Log _log
 
static constexpr double _deltaCutOffThreshold = 1e-6
 

Detailed Description

A simple global database.

The database can be configured with various thresholds such that we write out snapshots whenever a certain maximum size is reached.

The database is thread-safe. It was built after the model of the particle Database.

Definition at line 29 of file GlobalDatabase.h.

Member Enumeration Documentation

◆ AddSnapshotAction

Enumerator
Ignore 
Append 
Replace 

Definition at line 88 of file GlobalDatabase.h.

Constructor & Destructor Documentation

◆ GlobalDatabase()

toolbox::blockstructured::GlobalDatabase::GlobalDatabase ( int growthBetweenTwoDatabaseFlushes,
double dataDelta = 1e-8,
double timeDelta = 0.0,
bool clearDatabaseAfterFlush = true,
bool deltasAreRelative = false )

Global database.

The database dumps/stores data if the data is either for a new timestamp which is a sufficient delta after the latest one. If the new data has the same timestamp as a previous one, it will replace it if the data is different by a sufficiently large margin. We always work with the max norm.

Flushing the database

Please read the documentation of clearDatabaseAfterFlush and growthBetweenTwoDatabaseFlushes below first.

Parameters
clearDatabaseAfterFlushIf this flag is set, each flush of the database will go into a separate file, and the code will clear the database after the flush. As a consequence, the database will never exceed the memory. However, you'll get a lot of files that you have to merge afterwards.
growthBetweenTwoDatabaseFlushesNumber of entries that we dump into the database before it is flushed the next time. Set it to max of an integer or zero to disable on-the-fly dumps. In this case, the database is only flushed when the simulation terminates. So, the thresholds for the data and position deltas determine how often entries and up in the database, and growthBetweenTwoDatabaseFlushes determines how often this database is written into a file.
dataDeltaIf two pieces of data are defined for the same timestamp, this parameter is used to determine whether this is re-defining the data or whether it is just a duplicate. If the former is true, the existing data will be replaced with the new definition. In the later case, the data is not added to the database and is simply discarded. toolbox::blockstructured::GlobalDatabase::getAction() for a description how the C++ code interprets this threshold, but it is usually the max norm that is used here.

This flag has no impact whatsoever how often the data is dumped into a file. The frequency of datafile writes is solely controlled via growthBetweenTwoDatabaseFlushes.

Parameters
timeDeltathis parameter ask the code to dump particle into database after certain time interval of time_delta_between_two_snapsots, even data and position do not change during this time interval. You can set the two parameter above to be extremely big to enforce code dump particle with (roughly) regular time interval.

This flag has no impact whatsoever how often the data is dumped into a file. The frequency of datafile writes is solely controlled via growthBetweenTwoDatabaseFlushes.

Parameters
deltasAreRelativeBy default (flag is false), we take the absolute deltas of the data to decide whether to replace data with duplicates or simply assume these to be identical. If this flag is set however, we track the maximum of the deltas, and we dump data if and only if it exceeds dataDelta times this maximum. So we use a relative quantity. This might make sense if one value in your database hovers around a size of 10^-3 and another is in the range of 10^10, in which case using the same absolute delta for both is not the ideal choice.

Definition at line 43 of file GlobalDatabase.cpp.

◆ ~GlobalDatabase()

toolbox::blockstructured::GlobalDatabase::~GlobalDatabase ( )

Definition at line 60 of file GlobalDatabase.cpp.

Member Function Documentation

◆ addGlobalSnapshot() [1/2]

void toolbox::blockstructured::GlobalDatabase::addGlobalSnapshot ( double timestamp)

Add snapshot.

The data is assumed to be global, and therefore uniquely identified by its timestamp. We can track any amount of data, such as none at all (like in this function) or an arbitrary number. Hence the overload. It is the user's responsibility to use the addGlobalSnapshot() routines in a consistent way. If you've added six pieces of data once and you try adding eight pieces in a next call, the code will do what we in the business like to call an "oopsie-woopsie", by which we mean an assertion will fail and your program will crash.

Thread-safety

The routine is thread-safe. It actually locks the database before it invokes getAction() and thus will not mess up either the database analysis or any writes.

If the database is configured to write snapshots, the routine will also invoke the dump. However, it is important that we free the lock before, as I do not use recursive locks and as the dumping itself is thread-safe.

Definition at line 225 of file GlobalDatabase.cpp.

References tarch::multicore::Lock::free(), tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getRank(), and logInfo.

Here is the call graph for this function:

◆ addGlobalSnapshot() [2/2]

void toolbox::blockstructured::GlobalDatabase::addGlobalSnapshot ( double timestamp,
int numberOfDataEntries,
double * data )

Definition at line 260 of file GlobalDatabase.cpp.

References assertion, tarch::multicore::Lock::free(), tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getRank(), and logInfo.

Here is the call graph for this function:

◆ clear()

void toolbox::blockstructured::GlobalDatabase::clear ( bool lockSemaphore = true)

Thread-safety

The clear() operation is thread-safe if you set lockSemaphore. In this case, it first locks the sempahore and then it continues.

Definition at line 68 of file GlobalDatabase.cpp.

References tarch::multicore::Lock::lock().

Referenced by peano4.output.Makefile.Makefile::__init__().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ clearDatabaseAfterFlush()

void toolbox::blockstructured::GlobalDatabase::clearDatabaseAfterFlush ( bool value)

Definition at line 143 of file GlobalDatabase.cpp.

◆ dumpCSVFile()

void toolbox::blockstructured::GlobalDatabase::dumpCSVFile ( )

Dump data into CSV file.

The operation is thread-safe, i.e. it first locks the sempahore and then it continues. If we are supposed to clear the database once we've dumped the CSV file, we will call clear(). In this case, it is important that I keep the lock up and call clear(). If I released the lock and then called clear, some other thread might squeeze its particle update in-between and we'd loose the information.

Definition at line 79 of file GlobalDatabase.cpp.

References tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getRank(), and logInfo.

Here is the call graph for this function:

◆ dumpDatabaseSnapshot()

bool toolbox::blockstructured::GlobalDatabase::dumpDatabaseSnapshot ( )
private

Thread-safety

The routine is thread-safe, as it locks the database. As it uses the semaphore, it can't be const.

Returns
If the user should write the database into a file. This is the case if the total number of entries in the database exceeds _thresholdForNextDatabaseFlush.

Definition at line 218 of file GlobalDatabase.cpp.

References tarch::la::greater().

Here is the call graph for this function:

◆ getAction() [1/2]

toolbox::blockstructured::GlobalDatabase::AddSnapshotAction toolbox::blockstructured::GlobalDatabase::getAction ( double timestamp)
private

Determine what to do with new entry.

Can't be const as it might augment the database with default entries.

The routine is not thread-safe, i.e. you have to protect it from outside.

Definition at line 160 of file GlobalDatabase.cpp.

References Append, tarch::la::equals(), Ignore, logWarning, and Replace.

Here is the call graph for this function:

◆ getAction() [2/2]

toolbox::blockstructured::GlobalDatabase::AddSnapshotAction toolbox::blockstructured::GlobalDatabase::getAction ( double timestamp,
int numberOfDataEntries,
double * data )
private

Determine what to do with new entry.

First of all, invoke other getAction() routine. This one will determine if the entry is the first entry, if it's timestamp is such that it should be added to the database or discarded, and if it is at the same timestamp as the last one (in which case it might need to replace it.) In the latest case, we only replace it if the data actually differs from the existing data.

Definition at line 181 of file GlobalDatabase.cpp.

References Append, and Replace.

◆ setDataDeltaBetweenTwoSnapshots()

void toolbox::blockstructured::GlobalDatabase::setDataDeltaBetweenTwoSnapshots ( double value,
bool deltasAreRelative = false )

Definition at line 147 of file GlobalDatabase.cpp.

References assertion.

◆ setDataName()

void toolbox::blockstructured::GlobalDatabase::setDataName ( const std::string & dataname)

Definition at line 135 of file GlobalDatabase.cpp.

◆ setOutputFileName()

void toolbox::blockstructured::GlobalDatabase::setOutputFileName ( const std::string & filename)

Definition at line 131 of file GlobalDatabase.cpp.

◆ setOutputPrecision()

void toolbox::blockstructured::GlobalDatabase::setOutputPrecision ( int precision)

Definition at line 139 of file GlobalDatabase.cpp.

◆ setTimeDeltaBetweenTwoSnapshots()

void toolbox::blockstructured::GlobalDatabase::setTimeDeltaBetweenTwoSnapshots ( double value)

Definition at line 155 of file GlobalDatabase.cpp.

References assertion.

Field Documentation

◆ _clearDatabaseAfterFlush

bool toolbox::blockstructured::GlobalDatabase::_clearDatabaseAfterFlush
private

Flag that indicates if we erase the database after a flush.

If not set, each dump consists of all data ever recorded. If set, you will potentially get a huge set of data files, and you will have to concatenate them before you interpret the data.

See also
GlobalDatabase()
clearDatabaseAfterFlush()

Definition at line 56 of file GlobalDatabase.h.

◆ _data

◆ _dataDelta

double toolbox::blockstructured::GlobalDatabase::_dataDelta
private

Definition at line 36 of file GlobalDatabase.h.

◆ _dataName

std::string toolbox::blockstructured::GlobalDatabase::_dataName
private

Definition at line 34 of file GlobalDatabase.h.

◆ _deltaBetweenTwoDatabaseFlushes

const int toolbox::blockstructured::GlobalDatabase::_deltaBetweenTwoDatabaseFlushes
private

Definition at line 42 of file GlobalDatabase.h.

◆ _deltaCutOffThreshold

constexpr double toolbox::blockstructured::GlobalDatabase::_deltaCutOffThreshold = 1e-6
staticconstexprprivate

Definition at line 59 of file GlobalDatabase.h.

◆ _deltasAreRelative

bool toolbox::blockstructured::GlobalDatabase::_deltasAreRelative
private

Definition at line 57 of file GlobalDatabase.h.

◆ _fileName

std::string toolbox::blockstructured::GlobalDatabase::_fileName
private

Definition at line 33 of file GlobalDatabase.h.

◆ _log

tarch::logging::Log toolbox::blockstructured::GlobalDatabase::_log
staticprivate

Definition at line 31 of file GlobalDatabase.h.

◆ _maxDataDelta

double toolbox::blockstructured::GlobalDatabase::_maxDataDelta
private

Definition at line 37 of file GlobalDatabase.h.

◆ _maxTimestamp

double toolbox::blockstructured::GlobalDatabase::_maxTimestamp
private

Definition at line 68 of file GlobalDatabase.h.

◆ _numberOfGlobalDataPoints

int toolbox::blockstructured::GlobalDatabase::_numberOfGlobalDataPoints
private

Definition at line 38 of file GlobalDatabase.h.

◆ _precision

int toolbox::blockstructured::GlobalDatabase::_precision
private

Definition at line 44 of file GlobalDatabase.h.

◆ _rank

int toolbox::blockstructured::GlobalDatabase::_rank
private

This is a hack: Usually, I'd just ask the rank what its number is.

However, the database dump often is called in the very end, after the MPI rank is already down. So it will return -1. So what I do is that I store this piece of data whenever I insert a new entry.

Definition at line 67 of file GlobalDatabase.h.

◆ _semaphore

tarch::multicore::BooleanSemaphore toolbox::blockstructured::GlobalDatabase::_semaphore
private

Definition at line 86 of file GlobalDatabase.h.

◆ _thresholdForNextDatabaseFlush

int toolbox::blockstructured::GlobalDatabase::_thresholdForNextDatabaseFlush
private

Definition at line 43 of file GlobalDatabase.h.

◆ _timeDelta

double toolbox::blockstructured::GlobalDatabase::_timeDelta
private

Definition at line 40 of file GlobalDatabase.h.


The documentation for this class was generated from the following files: