Peano 4
Loading...
Searching...
No Matches
tarch::logging::Statistics Class Reference

Global statistics interface. More...

#include <Statistics.h>

Collaboration diagram for tarch::logging::Statistics:

Data Structures

struct  DataSet
 One data set for one type (identifier) of statistics. More...
 
struct  LogMessage
 

Public Member Functions

void log (const std::string &identifier, double value, bool disableSampling=false)
 Log one particular value.
 
void log (const std::string &identifier, const std::string &value)
 
void inc (const std::string &identifier, double value=1.0, bool disableSampling=false)
 
void writeToCSV (std::string filename="statistics")
 Write data to csv file.
 
void clear ()
 

Static Public Member Functions

static StatisticsgetInstance ()
 This is not the canonical realisation of singletons as I use it usually for stats in Peano.
 

Private Member Functions

 Statistics ()
 
bool acceptNewData (const std::string &identifier, bool disableSampling)
 Not const, as it also updates the internal counters.
 
void initData (const std::string &identifier)
 Ensures that dataset is there.
 
void updateDataSnapshot (const std::string &identifier, double value)
 Updates snapshot.
 

Private Attributes

int _maxCountInBetweenTwoMeasurements
 
double _maxTimeInBetweenTwoMeasurements
 
tarch::timing::Watch _globalWatch
 
std::map< std::string, DataSet_dataSetMap
 Mapping from identifier who wrote stats (key) onto DataSet.
 
std::map< std::string, LogMessage_logMessageMap
 

Static Private Attributes

static Statistics _singleton
 
static Log _log
 
static tarch::multicore::BooleanSemaphore _semaphore
 

Detailed Description

Global statistics interface.

This interface only logs data if you have translated with

CXXFLAGS="... -DTrackStatistics"

To get the stats right, you might want to invoke the clear() operation explicitly when you start up your code. This way, you ensure that the internal timer (_globalWatch) is properly initialised.

You can control if you want to track data every time it is submitted or sample a quantity every now and then.

CSV format

The first column in the csv file will hold time stamps. Each further column presents one type of measurement. The name for these entries is determined by the source code logging the data. As we sample, not all columns will hold data for all entries, as the rows are ordered temporary and we only write entries if there are entries updated.

t exahype2::EnclaveBookkeeping::memory-allocations mpi wait times tarch::multicore::bsp-concurrency-level tarch::multicore::fuse-tasks tarch::multicore::global-pending-tasks tarch::multicore::thread-local-pending-tasks grid-control-events
t, exahype2::EnclaveBookkeeping::memory-allocations, mpi wait times, tarch::multicore::bsp-concurrency-level, tarch::multicore::fuse-tasks, tarch::multicore::global-pending-tasks, tarch::multicore::thread-local-pending-tasks, grid-control-events
0.00130565, , (0.150548/0/0.157291/#10), (1/0/1/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
0.00131057, , , (2/1/2/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
0.00134099, , , (3/2/3/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
0.00135324, , , (4/3/4/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
0.00135806, , , (5/4/5/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,
0.00137025, , , (6/5/6/#1), , , , "((refinementControl=Refine,offset=[-9.03,-9.03,-9.03],width=[18.06,18.06,18.06],h=[2.02,2.02,2.02]))" ,

Each entry follows a format of its own. The time stamps are given in seconds. Some columns (like the refinement strings) hold strings. Most comments will hold three-tuples:

  1. The first entry is the actual value of the numeric value at the time when we make the snapshot.
  2. The second entry is the minimum over the whole sample period, i.e. since the last data dump.
  3. The third entry is the maximum.
  4. The fourth entry holds the number of samples taken over the sampling period. If you sample every 1000 steps, it should be 1000 always. If you sample every x seconds, it will be a non-constant value.

There is a Python script peano4.postprocessing.plot-statistics.py which helps you to extract the 4-tuple data.

Definition at line 72 of file Statistics.h.

Constructor & Destructor Documentation

◆ Statistics()

tarch::logging::Statistics::Statistics ( )
private

Definition at line 23 of file Statistics.cpp.

Member Function Documentation

◆ acceptNewData()

bool tarch::logging::Statistics::acceptNewData ( const std::string & identifier,
bool disableSampling )
private

Not const, as it also updates the internal counters.

A new snapshot is written if the number of writes for this data set exceeds _maxCountInBetweenTwoMeasurements or if the time since the last write exceeds _maxTimeInBetweenTwoMeasurements. Furthermore, a new data set is written if the sampling is disabled. In this case, all the local counters are still reset, as you might want to combine manual disabled stats with sampled ones.

If the data has not changed since the last snapshot, I don't write anything.

Returns
Write a new snapshot.

Definition at line 61 of file Statistics.cpp.

References _dataSetMap, _maxCountInBetweenTwoMeasurements, and _maxTimeInBetweenTwoMeasurements.

◆ clear()

void tarch::logging::Statistics::clear ( )

Definition at line 56 of file Statistics.cpp.

References _dataSetMap.

Referenced by peano4.output.Makefile.Makefile::__init__(), and main().

Here is the caller graph for this function:

◆ getInstance()

◆ inc()

void tarch::logging::Statistics::inc ( const std::string & identifier,
double value = 1.0,
bool disableSampling = false )

◆ initData()

void tarch::logging::Statistics::initData ( const std::string & identifier)
private

Ensures that dataset is there.

If not, it creates a new entry. If there's already an entry for identifier, it degenerates to nop.

Definition at line 96 of file Statistics.cpp.

References _dataSetMap, _globalWatch, tarch::timing::Watch::getCalendarTime(), and tarch::timing::Watch::stop().

Here is the call graph for this function:

◆ log() [1/2]

void tarch::logging::Statistics::log ( const std::string & identifier,
const std::string & value )

Definition at line 91 of file Statistics.h.

◆ log() [2/2]

void tarch::logging::Statistics::log ( const std::string & identifier,
double value,
bool disableSampling = false )

Log one particular value.

Parameters
identifierUnique name (string) for this event

Definition at line 90 of file Statistics.h.

Referenced by tarch::multicore::internal::copyInternalTaskQueueOverIntoGlobalQueue(), tarch::multicore::internal::fusePendingTasks(), tarch::multicore::spawnTask(), peano4::parallel::SpacetreeSet::traverse(), and peano4::grid::Spacetree::traverse().

Here is the caller graph for this function:

◆ updateDataSnapshot()

void tarch::logging::Statistics::updateDataSnapshot ( const std::string & identifier,
double value )
private

Updates snapshot.

Definition at line 106 of file Statistics.cpp.

References _dataSetMap, and assertion.

◆ writeToCSV()

void tarch::logging::Statistics::writeToCSV ( std::string filename = "statistics")

Write data to csv file.

I do append the extension (csv) and a rank identifier myself, i.e. each rank will write a csv file of its own. The format of the csv file is discussed above. The name is not 100% correct, as the routine also erases snapshot data which is not required anymore, as we have dumped it on disk.

Realisation

Each data entry hosts a long list of snapshots which are chronologically ordered. That is, we have

  first-stats:  0.4xa, 0.7b, 0.8c, 1.2d
  second-stats: 0.3xe, 0.5f, 0.9g

In the map. The code runs through all stats iteratively, until no more stats are in the file, i.e. until we have dumped all data. In a first step per iteration, we find the minimum time stamp that has to be dumped. If there are no data left, this one is max and therefore we can terminate.

As long as there are entries, now have a valid cut off time stamp t. We run over all entries in our database next. In the example above, that would be two of them. We only ever have to study the first entry within a sequence of events, as they are chronologically ordered. If the time stamp of this first entry is smaller or equal to our cut-off, we dump it and then remove it from the data set.

Definition at line 158 of file Statistics.cpp.

References _dataSetMap, _logMessageMap, tarch::mpi::Rank::getInstance(), logDebug, logInfo, logWarning, and tarch::la::smallerEquals().

Referenced by main().

Here is the call graph for this function:
Here is the caller graph for this function:

Field Documentation

◆ _dataSetMap

std::map< std::string, DataSet > tarch::logging::Statistics::_dataSetMap
private

Mapping from identifier who wrote stats (key) onto DataSet.

Definition at line 177 of file Statistics.h.

Referenced by acceptNewData(), clear(), initData(), updateDataSnapshot(), and writeToCSV().

◆ _globalWatch

tarch::timing::Watch tarch::logging::Statistics::_globalWatch
private

Definition at line 141 of file Statistics.h.

Referenced by initData().

◆ _log

tarch::logging::Log tarch::logging::Statistics::_log
staticprivate

Definition at line 134 of file Statistics.h.

◆ _logMessageMap

std::map< std::string, LogMessage > tarch::logging::Statistics::_logMessageMap
private

Definition at line 178 of file Statistics.h.

Referenced by writeToCSV().

◆ _maxCountInBetweenTwoMeasurements

int tarch::logging::Statistics::_maxCountInBetweenTwoMeasurements
private

Definition at line 138 of file Statistics.h.

Referenced by acceptNewData().

◆ _maxTimeInBetweenTwoMeasurements

double tarch::logging::Statistics::_maxTimeInBetweenTwoMeasurements
private

Definition at line 139 of file Statistics.h.

Referenced by acceptNewData().

◆ _semaphore

tarch::multicore::BooleanSemaphore tarch::logging::Statistics::_semaphore
staticprivate

Definition at line 136 of file Statistics.h.

◆ _singleton

tarch::logging::Statistics tarch::logging::Statistics::_singleton
staticprivate

Definition at line 132 of file Statistics.h.


The documentation for this class was generated from the following files: