Peano

Searching...

No Matches

Global statistics interface. More...

#include <Statistics.h>

Collaboration diagram for tarch::logging::Statistics:

[legend]

Data Structures
struct	DataSet
	One data set for one type (identifier) of statistics. More...

struct	LogMessage

Public Member Functions
void	log (const std::string &identifier, double value, bool disableSampling=false)

void	log (const std::string &identifier, const std::string &value)

void	inc (const std::string &identifier, double value=1.0, bool disableSampling=false, bool clearAfterDatabaseDump=false)

void	writeToCSV (std::string filename="statistics")
	Write data to csv file.

void	clear ()

Static Public Member Functions
static Statistics &	getInstance ()
	This is not the canonical realisation of singletons as I use it usually for stats in Peano.

Private Member Functions
	Statistics ()

bool	acceptNewData (const std::string &identifier, bool disableSampling)
	Not const, as it also updates the internal counters.

void	initData (const std::string &identifier)
	Ensures that dataset is there.

void	updateDataSnapshot (const std::string &identifier, double value)
	Updates snapshot.

Private Attributes
int	_maxCountInBetweenTwoMeasurements

double	_maxTimeInBetweenTwoMeasurements

tarch::timing::Watch	_globalWatch

std::map< std::string, DataSet >	_dataSetMap
	Mapping from identifier who wrote stats (key) onto DataSet.

std::map< std::string, LogMessage >	_logMessageMap

Static Private Attributes
static Statistics	_singleton

static Log	_log

static tarch::multicore::BooleanSemaphore	_semaphore

Detailed Description

Global statistics interface.

This interface only logs data if you have translated with

CXXFLAGS="... -DTrackStatistics"

To get the stats right, you might want to invoke the clear() operation explicitly when you start up your code. This way, you ensure that the internal timer (_globalWatch) is properly initialised.

You can control if you want to track data every time it is submitted or sample a quantity every now and then.

CSV format

The first column in the csv file will hold time stamps. Each further column presents one type of measurement. The name for these entries is determined by the source code logging the data. As we sample, not all columns will hold data for all entries, as the rows are ordered temporary and we only write entries if there are entries updated.

t   exahype2::EnclaveBookkeeping::memory-allocations  mpi wait times  tarch::multicore::bsp-concurrency-level   tarch::multicore::fuse-tasks  tarch::multicore::global-pending-tasks  tarch::multicore::thread-local-pending-tasks  grid-control-events
t, exahype2::EnclaveBookkeeping::memory-allocations, mpi wait times, tarch::multicore::bsp-concurrency-level, tarch::multicore::fuse-tasks, tarch::multicore::global-pending-tasks, tarch::multicore::thread-local-pending-tasks, grid-control-events
00130565, , (0.150548/0/0.157291/#10), (1/0/1/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00131057, , , (2/1/2/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00134099, , , (3/2/3/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00135324, , , (4/3/4/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00135806, , , (5/4/5/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00137025, , , (6/5/6/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,

Each entry follows a format of its own. The time stamps are given in seconds. Some columns (like the refinement strings) hold strings. Most comments will hold three-tuples:

The first entry is the actual value of the numeric value at the time when we make the snapshot.
The second entry is the minimum over the whole sample period, i.e. since the last data dump.
The third entry is the maximum.
The fourth entry holds the number of samples taken over the sampling period. If you sample every 1000 steps, it should be 1000 always. If you sample every x seconds, it will be a non-constant value.

There is a Python script peano4.postprocessing.plot-statistics.py which helps you to extract the 4-tuple data.

Definition at line 72 of file Statistics.h.

Constructor & Destructor Documentation

◆ Statistics()

tarch::logging::Statistics::Statistics ( )

private

Definition at line 23 of file Statistics.cpp.

Member Function Documentation

◆ acceptNewData()

bool tarch::logging::Statistics::acceptNewData	(	const std::string &	identifier,
		bool	disableSampling )

private

Not const, as it also updates the internal counters.

A new snapshot is written if the number of writes for this data set exceeds _maxCountInBetweenTwoMeasurements or if the time since the last write exceeds _maxTimeInBetweenTwoMeasurements. Furthermore, a new data set is written if the sampling is disabled. In this case, all the local counters are still reset, as you might want to combine manual disabled stats with sampled ones.

If the data has not changed since the last snapshot, I don't write anything.

Returns: Write a new snapshot.

Definition at line 73 of file Statistics.cpp.

References _dataSetMap, _maxCountInBetweenTwoMeasurements, and _maxTimeInBetweenTwoMeasurements.

◆ clear()

void tarch::logging::Statistics::clear ( )

Definition at line 68 of file Statistics.cpp.

References _dataSetMap.

Referenced by peano4.output.Makefile.Makefile::__init__(), and main().

Here is the caller graph for this function:

◆ getInstance()

tarch::logging::Statistics & tarch::logging::Statistics::getInstance ( )

static

This is not the canonical realisation of singletons as I use it usually for stats in Peano.

Definition at line 16 of file Statistics.cpp.

Referenced by exahype2::EnclaveBookkeeping::finishedTask(), main(), tarch::multicore::taskfusion::LogReadyTask::run(), tarch::multicore::taskfusion::ProcessReadyTask::run(), tarch::multicore::spawnAndWait(), tarch::multicore::spawnTask(), tarch::multicore::taskfusion::translateFusableTaskIntoTaskSequence(), peano4::parallel::SpacetreeSet::traverse(), and peano4::grid::Spacetree::traverse().

Here is the caller graph for this function:

◆ inc()

void tarch::logging::Statistics::inc	(	const std::string &	identifier,
		double	value = 1.0,
		bool	disableSampling = false,
		bool	clearAfterDatabaseDump = false )

Definition at line 107 of file Statistics.h.

Referenced by exahype2::EnclaveBookkeeping::finishedTask(), tarch::multicore::taskfusion::LogReadyTask::run(), tarch::multicore::spawnAndWait(), tarch::multicore::spawnTask(), and tarch::multicore::taskfusion::translateFusableTaskIntoTaskSequence().

Here is the caller graph for this function:

◆ initData()

void tarch::logging::Statistics::initData ( const std::string & identifier )

private

Ensures that dataset is there.

If not, it creates a new entry. If there's already an entry for identifier, it degenerates to nop.

Definition at line 108 of file Statistics.cpp.

References _dataSetMap, _globalWatch, tarch::timing::Watch::getCalendarTime(), and tarch::timing::Watch::stop().

Here is the call graph for this function:

◆ log() [1/2]

void tarch::logging::Statistics::log	(	const std::string &	identifier,
		const std::string &	value )

Definition at line 106 of file Statistics.h.

◆ log() [2/2]

void tarch::logging::Statistics::log	(	const std::string &	identifier,
		double	value,
		bool	disableSampling = false )

Definition at line 105 of file Statistics.h.

Referenced by tarch::multicore::taskfusion::ProcessReadyTask::run(), peano4::parallel::SpacetreeSet::traverse(), and peano4::grid::Spacetree::traverse().

Here is the caller graph for this function:

◆ updateDataSnapshot()

void tarch::logging::Statistics::updateDataSnapshot	(	const std::string &	identifier,
		double	value )

private

Updates snapshot.

Definition at line 118 of file Statistics.cpp.

References _dataSetMap, and assertion.

◆ writeToCSV()

void tarch::logging::Statistics::writeToCSV ( std::string filename = "statistics" )

Write data to csv file.

I do append the extension (csv) and a rank identifier myself, i.e. each rank will write a csv file of its own. The format of the csv file is discussed above. The name is not 100% correct, as the routine also erases snapshot data which is not required anymore, as we have dumped it on disk.

Realisation

Each data entry hosts a long list of snapshots which are chronologically ordered. That is, we have

  first-stats:  0.4xa, 0.7b, 0.8c, 1.2d
  second-stats: 0.3xe, 0.5f, 0.9g

In the map. The code runs through all stats iteratively, until no more stats are in the file, i.e. until we have dumped all data. In a first step per iteration, we find the minimum time stamp that has to be dumped. If there are no data left, this one is max and therefore we can terminate.

As long as there are entries, now have a valid cut off time stamp t. We run over all entries in our database next. In the example above, that would be two of them. We only ever have to study the first entry within a sequence of events, as they are chronologically ordered. If the time stamp of this first entry is smaller or equal to our cut-off, we dump it and then remove it from the data set.

Definition at line 170 of file Statistics.cpp.

References _dataSetMap, _logMessageMap, tarch::mpi::Rank::getInstance(), logDebug, logInfo, logWarning, and tarch::la::smallerEquals().

Referenced by main().

Here is the call graph for this function:

Here is the caller graph for this function:

Field Documentation

◆ _dataSetMap

std::map< std::string, DataSet > tarch::logging::Statistics::_dataSetMap

private

Mapping from identifier who wrote stats (key) onto DataSet.

Definition at line 194 of file Statistics.h.

Referenced by acceptNewData(), clear(), initData(), updateDataSnapshot(), and writeToCSV().

◆ _globalWatch

tarch::timing::Watch tarch::logging::Statistics::_globalWatch

private

Definition at line 156 of file Statistics.h.

Referenced by initData().

◆ _log

tarch::logging::Log tarch::logging::Statistics::_log

staticprivate

Definition at line 149 of file Statistics.h.

◆ _logMessageMap

std::map< std::string, LogMessage > tarch::logging::Statistics::_logMessageMap

private

Definition at line 195 of file Statistics.h.

Referenced by writeToCSV().

◆ _maxCountInBetweenTwoMeasurements

int tarch::logging::Statistics::_maxCountInBetweenTwoMeasurements

private

Definition at line 153 of file Statistics.h.

Referenced by acceptNewData().

◆ _maxTimeInBetweenTwoMeasurements

double tarch::logging::Statistics::_maxTimeInBetweenTwoMeasurements

private

Definition at line 154 of file Statistics.h.

Referenced by acceptNewData().

◆ _semaphore

tarch::multicore::BooleanSemaphore tarch::logging::Statistics::_semaphore

staticprivate

Definition at line 151 of file Statistics.h.

◆ _singleton

tarch::logging::Statistics tarch::logging::Statistics::_singleton

staticprivate

Definition at line 147 of file Statistics.h.

The documentation for this class was generated from the following files:

src/tarch/logging/Statistics.h
src/tarch/logging/Statistics.cpp

Data Structures

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

CSV format

Constructor & Destructor Documentation

◆ Statistics()

Member Function Documentation

◆ acceptNewData()

◆ clear()

◆ getInstance()

◆ inc()

◆ initData()

◆ log() [1/2]

◆ log() [2/2]

◆ updateDataSnapshot()

◆ writeToCSV()

Realisation

Field Documentation

◆ _dataSetMap

◆ _globalWatch

◆ _log

◆ _logMessageMap

◆ _maxCountInBetweenTwoMeasurements

◆ _maxTimeInBetweenTwoMeasurements

◆ _semaphore

◆ _singleton