Peano

Searching...

No Matches

Global statistics interface. More...

#include <Statistics.h>

Collaboration diagram for tarch::logging::Statistics:

[legend]

Data Structures
struct	DataSet
	One data set for one type (identifier) of statistics. More...

struct	LogMessage

Public Member Functions
void	log (const std::string &identifier, double value, bool disableSampling=false)

void	log (const std::string &identifier, const std::string &value)

void	inc (const std::string &identifier, double value=1.0, bool disableSampling=false, bool clearAfterDatabaseDump=false)

void	writeToCSV (std::string filename="statistics")
	Write data to csv file.

void	clear ()

Static Public Member Functions
static Statistics &	getInstance ()
	This is not the canonical realisation of singletons as I use it usually for stats in Peano.

Private Member Functions
	Statistics ()

bool	acceptNewData (const std::string &identifier, bool disableSampling, bool clearAfterDatabaseDump)
	Not const, as it also updates the internal counters.

void	initData (const std::string &identifier)
	Ensures that dataset is there.

void	updateDataSnapshot (const std::string &identifier, double value)
	Updates snapshot.

Private Attributes
int	_maxCountInBetweenTwoMeasurements

double	_maxTimeInBetweenTwoMeasurements

tarch::timing::Watch	_globalWatch

std::map< std::string, DataSet >	_dataSetMap
	Mapping from identifier who wrote stats (key) onto DataSet.

std::map< std::string, LogMessage >	_logMessageMap

Static Private Attributes
static Statistics	_singleton

static Log	_log

static tarch::multicore::BooleanSemaphore	_semaphore

Detailed Description

Global statistics interface.

This interface only logs data if you have translated with

CXXFLAGS="... -DTrackStatistics"

To get the stats right, you might want to invoke the clear() operation explicitly when you start up your code. This way, you ensure that the internal timer (_globalWatch) is properly initialised.

You can control if you want to track data every time it is submitted or sample a quantity every now and then.

CSV format

The first column in the csv file will hold time stamps. Each further column presents one type of measurement. The name for these entries is determined by the source code logging the data. As we sample, not all columns will hold data for all entries, as the rows are ordered temporary and we only write entries if there are entries updated.

t   exahype2::EnclaveBookkeeping::memory-allocations  mpi wait times  tarch::multicore::bsp-concurrency-level   tarch::multicore::fuse-tasks  tarch::multicore::global-pending-tasks  tarch::multicore::thread-local-pending-tasks  grid-control-events
t, exahype2::EnclaveBookkeeping::memory-allocations, mpi wait times, tarch::multicore::bsp-concurrency-level, tarch::multicore::fuse-tasks, tarch::multicore::global-pending-tasks, tarch::multicore::thread-local-pending-tasks, grid-control-events
00130565, , (0.150548/0/0.157291/#10), (1/0/1/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00131057, , , (2/1/2/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00134099, , , (3/2/3/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00135324, , , (4/3/4/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00135806, , , (5/4/5/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
00137025, , , (6/5/6/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,

Each entry follows a format of its own. The time stamps are given in seconds. Some columns (like the refinement strings) hold strings. Most comments will hold three-tuples:

The first entry is the actual value of the numeric value at the time when we make the snapshot.
The second entry is the minimum over the whole sample period, i.e. since the last data dump.
The third entry is the maximum.
The fourth entry holds the number of samples taken over the sampling period. If you sample every 1000 steps, it should be 1000 always. If you sample every x seconds, it will be a non-constant value.

There is a Python script peano4.postprocessing.plot-statistics.py which helps you to extract the 4-tuple data.

Definition at line 72 of file Statistics.h.

Constructor & Destructor Documentation

◆ Statistics()

tarch::logging::Statistics::Statistics ( )

private

Member Function Documentation

◆ acceptNewData()

bool tarch::logging::Statistics::acceptNewData	(	const std::string &	identifier,
		bool	disableSampling,
		bool	clearAfterDatabaseDump )

private

Not const, as it also updates the internal counters.

A new snapshot is written if the number of writes for this data set exceeds _maxCountInBetweenTwoMeasurements or if the time since the last write exceeds _maxTimeInBetweenTwoMeasurements. Furthermore, a new data set is written if the sampling is disabled. In this case, all the local counters are still reset, as you might want to combine manual disabled stats with sampled ones.

If the data has not changed since the last snapshot, I don't write anything.

If data has changed by more than 10%, I always write.

Returns: Write a new snapshot.

◆ clear()

void tarch::logging::Statistics::clear ( )

Referenced by peano4.output.Makefile.Makefile::__init__().

Here is the caller graph for this function:

◆ getInstance()

static Statistics & tarch::logging::Statistics::getInstance ( )

static

This is not the canonical realisation of singletons as I use it usually for stats in Peano.

◆ inc()

void tarch::logging::Statistics::inc	(	const std::string &	identifier,
		double	value = 1.0,
		bool	disableSampling = false,
		bool	clearAfterDatabaseDump = false )

Definition at line 107 of file Statistics.h.

◆ initData()

void tarch::logging::Statistics::initData ( const std::string & identifier )

private

Ensures that dataset is there.

If not, it creates a new entry. If there's already an entry for identifier, it degenerates to nop.

◆ log() [1/2]

void tarch::logging::Statistics::log	(	const std::string &	identifier,
		const std::string &	value )

Definition at line 106 of file Statistics.h.

◆ log() [2/2]

void tarch::logging::Statistics::log	(	const std::string &	identifier,
		double	value,
		bool	disableSampling = false )

Definition at line 105 of file Statistics.h.

◆ updateDataSnapshot()

void tarch::logging::Statistics::updateDataSnapshot	(	const std::string &	identifier,
		double	value )

private

Updates snapshot.

◆ writeToCSV()

void tarch::logging::Statistics::writeToCSV ( std::string filename = "statistics" )

Write data to csv file.

I do append the extension (csv) and a rank identifier myself, i.e. each rank will write a csv file of its own. The format of the csv file is discussed above. The name is not 100% correct, as the routine also erases snapshot data which is not required anymore, as we have dumped it on disk.

Realisation

Each data entry hosts a long list of snapshots which are chronologically ordered. That is, we have

  first-stats:  0.4xa, 0.7b, 0.8c, 1.2d
  second-stats: 0.3xe, 0.5f, 0.9g

In the map. The code runs through all stats iteratively, until no more stats are in the file, i.e. until we have dumped all data. In a first step per iteration, we find the minimum time stamp that has to be dumped. If there are no data left, this one is max and therefore we can terminate.

As long as there are entries, now have a valid cut off time stamp t. We run over all entries in our database next. In the example above, that would be two of them. We only ever have to study the first entry within a sequence of events, as they are chronologically ordered. If the time stamp of this first entry is smaller or equal to our cut-off, we dump it and then remove it from the data set.

Field Documentation

◆ _dataSetMap

std::map< std::string, DataSet > tarch::logging::Statistics::_dataSetMap

private

Mapping from identifier who wrote stats (key) onto DataSet.

Definition at line 194 of file Statistics.h.

◆ _globalWatch

tarch::timing::Watch tarch::logging::Statistics::_globalWatch

private

Definition at line 156 of file Statistics.h.

◆ _log

Log tarch::logging::Statistics::_log

staticprivate

Definition at line 149 of file Statistics.h.

◆ _logMessageMap

std::map< std::string, LogMessage > tarch::logging::Statistics::_logMessageMap

private

Definition at line 195 of file Statistics.h.

◆ _maxCountInBetweenTwoMeasurements

int tarch::logging::Statistics::_maxCountInBetweenTwoMeasurements

private

Definition at line 153 of file Statistics.h.

◆ _maxTimeInBetweenTwoMeasurements

double tarch::logging::Statistics::_maxTimeInBetweenTwoMeasurements

private

Definition at line 154 of file Statistics.h.

◆ _semaphore

tarch::multicore::BooleanSemaphore tarch::logging::Statistics::_semaphore

staticprivate

Definition at line 151 of file Statistics.h.

◆ _singleton

Statistics tarch::logging::Statistics::_singleton

staticprivate

Definition at line 147 of file Statistics.h.

The documentation for this class was generated from the following file:

src/tarch/logging/Statistics.h

Data Structures

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

CSV format

Constructor & Destructor Documentation

◆ Statistics()

Member Function Documentation

◆ acceptNewData()

◆ clear()

◆ getInstance()

◆ inc()

◆ initData()

◆ log() [1/2]

◆ log() [2/2]

◆ updateDataSnapshot()

◆ writeToCSV()

Realisation

Field Documentation

◆ _dataSetMap

◆ _globalWatch

◆ _log

◆ _logMessageMap

◆ _maxCountInBetweenTwoMeasurements

◆ _maxTimeInBetweenTwoMeasurements

◆ _semaphore

◆ _singleton