Peano

Searching...

No Matches

Node is Peano's abstraction for the hybrid of MPI/threads. More...

#include <Node.h>

Collaboration diagram for peano4::parallel::Node:

[legend]

Public Types
enum class	ExchangeMode { HorizontalData , VerticalData }

typedef std::pair< int, int >	PeriodicBoundaryStackIdentifier
	A periodic boundary stack is basically a stack (an integer), but I do augment it by a bitset which identifies which symmetry axis belong to this stack number.

typedef std::pair< int, MPI_Comm >	GridDataExchangeMetaInformation

Public Member Functions
int	getLocalTreeId (int treeId) const

int	getGlobalTreeId (int treeId) const

virtual	~Node ()
	The standard destructor calls MPI_Finalize().

int	getNumberOfRegisteredTrees () const

int	reserveId (int rank, int forTreeId)
	This operation is not const as it does some internal bookkeeping.

int	getRank (int treeId) const
	You hand in a tree number and the node tells you on which rank such a tree is hosted.

void	deregisterId (int id)
	Only the SpacetreeSet should call this operation.

bool	continueToRun ()
	You should call this operation only on the ranks >0 to find out whether you should do more iteration/sweeps.

void	setNextProgramStep (int number)
	The user tells the set which program step to use, i.e.

int	getCurrentProgramStep () const

void	init ()

void	shutdown ()
	The shutdown is invoked by peano4::shutdownSingletons()

GridDataExchangeMetaInformation	getGridDataExchangeMetaInformation (int sendingTreeId, int receivingTreeId, ExchangeMode exchange) const
	I use two tags per spacetree per rank: one for boundary data (horizontal) and one for up-down and synchronous (forks) data exchanges (vertical and fork/join).

Static Public Member Functions
static void	initMPIDatatypes ()
	I originally wanted to embed these guys into the singleton's constructor.

static void	shutdownMPIDatatypes ()

static Node &	getInstance ()
	This operation returns the singleton instance.

static bool	isGlobalMaster (int treeId)
	Is this the global master?

static int	getOutputStackNumberForHorizontalDataExchange (int id)
	Hand in a spacetree id and get back the number that we should use to send something to this tree.

static int	getInputStackNumberForVerticalDataExchange (int id)
	Get the input stack where a tree writes all of its vertical data to/from when it exchanges information with id.

static int	getOutputStackNumberForVerticalDataExchange (int id)

static int	getInputStackNumberForHorizontalDataExchange (int id)
	Counterpart of getOutputStackNumberOfBoundaryExchange(int)

static std::set< PeriodicBoundaryStackIdentifier >	getOutputStacksForPeriodicBoundaryExchange (const tarch::la::Vector< TwoPowerD, int > &flags)
	You hand in a the flags of a vertex and you get the boundary stack identifiers including their directions.

static std::string	toString (const std::set< PeriodicBoundaryStackIdentifier > &data)

static int	getOutputStackForPeriodicBoundaryExchange (int faceNumber)
	Identify output stack for periodic boundary data written by face.

static int	getPeriodicBoundaryExchangeInputStackNumberForOutputStack (int outputStackNumber)
	Hand in an output stack number of a face and you get back the input stack number.

static PeriodicBoundaryStackIdentifier	getPeriodicBoundaryExchangeInputStackNumberForOutputStack (PeriodicBoundaryStackIdentifier outputStackIdentifier)

static int	mapPeriodicBoundaryExchangeOutputStackOntoInputStack (int outputStack)

static bool	isHorizontalDataExchangeOutputStackNumber (int number)
	See getOutputStackNumberOfBoundaryExchange().

static bool	isHorizontalDataExchangeInputStackNumber (int number)

static bool	isVerticalDataExchangeOutputStackNumber (int number)

static bool	isVerticalDataExchangeInputStackNumber (int number)

static bool	isPeriodicBoundaryExchangeOutputStackNumber (int number)

static bool	isStorageStackNumber (int number)
	There are communication stacks and storage stacks.

static int	getTreeNumberTiedToExchangeStackNumber (int number)
	Gives you back the id of a communication partner, i.e.

static std::string	toString (ExchangeMode mode)

static std::string	getSemanticsForTag (int tag)
	I use this for debugging.

Static Public Attributes
static constexpr int	Terminate = -2

static constexpr int	MaxSpacetreesPerRank = 1024

static constexpr int	UndefProgramStep = -1
	Value for _currentProgramStep.

Private Member Functions
	Node ()
	The standard constructor assigns the attributes default values and checks whether the program is compiled using the -DParallel option.

	Node (const Node &node)=delete
	The copy constructor is private.

int	getId (int rank, int localTreeId) const
	Peano maps grid instance threads + mpi ranks onto global IDs through this routine.

void	registerId (int id, int masterId)
	The operation is not thread-safe as we call it only internally, i.e.

Static Private Member Functions
static std::bitset< 2 *Dimensions >	getPeriodicBoundaryNumber (const tarch::la::Vector< TwoPowerD, int > &flags)
	Analyse to which of the 2d faces it is adjacent.

Private Attributes
tarch::multicore::BooleanSemaphore	_semaphore

MPI_Comm	_dataExchangeCommunicators [MaxSpacetreesPerRank]

int	_currentProgramStep

int	_rankOrchestrationTag

std::map< int, TreeEntry >	_treeEntries

Static Private Attributes
static tarch::logging::Log	_log
	Logging device.

static Node	_singleton

static constexpr int	StacksPerCommunicationPartner = 4
	We need one stack for outgoing data, one for incoming data, and we distinguish horizontal and vertical data flow.

Friends
class	peano4::parallel::tests::NodeTest

Detailed Description

Node is Peano's abstraction for the hybrid of MPI/threads.

It represents one spacetree instance basically.

Author: Tobias Weinzierl

Definition at line 36 of file Node.h.

Member Typedef Documentation

◆ GridDataExchangeMetaInformation

typedef std::pair<int, MPI_Comm> peano4::parallel::Node::GridDataExchangeMetaInformation

Definition at line 382 of file Node.h.

◆ PeriodicBoundaryStackIdentifier

typedef std::pair<int, int > peano4::parallel::Node::PeriodicBoundaryStackIdentifier

A periodic boundary stack is basically a stack (an integer), but I do augment it by a bitset which identifies which symmetry axis belong to this stack number.

So if we return (23,010), then data goes to stack 23 and it is a stack that does realise periodic boundary conditions along the y axis.

The second argument (as sketched above) is of type std::bitset<Dimensions>. Unfortunately, bitsets do not work properly within the pair type as they seem not to define the comparison operator properly. So I use an int here and wrap it into a bitset whenever necessary.

See also: toString()

Definition at line 216 of file Node.h.

Member Enumeration Documentation

◆ ExchangeMode

enum class peano4::parallel::Node::ExchangeMode

strong

Enumerator

HorizontalData

Enumerator
HorizontalData
VerticalData	Vertical data is data running from the master to the worker and the other way round. Consequently, some tree management data are vertical data, too. See for example peano4::parallel::SpacetreeSet::addSpacetree().

VerticalData

Vertical data is data running from the master to the worker and the other way round.

Consequently, some tree management data are vertical data, too. See for example peano4::parallel::SpacetreeSet::addSpacetree().

Definition at line 370 of file Node.h.

Constructor & Destructor Documentation

◆ Node() [1/2]

peano4::parallel::Node::Node ( )

private

The standard constructor assigns the attributes default values and checks whether the program is compiled using the -DParallel option.

If this is not the case, a warning is logged.

Definition at line 61 of file Node.cpp.

◆ Node() [2/2]

peano4::parallel::Node::Node ( const Node & node )

privatedelete

The copy constructor is private.

◆ ~Node()

peano4::parallel::Node::~Node ( )

virtual

The standard destructor calls MPI_Finalize().

Definition at line 65 of file Node.cpp.

References assertionMsg, and tarch::mpi::Rank::getInstance().

Here is the call graph for this function:

Member Function Documentation

◆ continueToRun()

bool peano4::parallel::Node::continueToRun ( )

You should call this operation only on the ranks >0 to find out whether you should do more iteration/sweeps.

The spacetree set internally hijacks this operation also on rank 0 to trigger the send out of startup messages. Therefore, the operation is not const.

Definition at line 418 of file Node.cpp.

References tarch::mpi::Rank::getGlobalMasterRank(), tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getNumberOfRanks(), peano4::parallel::StartTraversalMessage::getStepIdentifier(), logDebug, logTraceIn, logTraceOutWith1Argument, peano4::parallel::StartTraversalMessage::receiveAndPollDanglingMessages(), peano4::parallel::StartTraversalMessage::sendAndPollDanglingMessages(), peano4::parallel::StartTraversalMessage::setStepIdentifier(), and peano4::parallel::StartTraversalMessage::toString().

Referenced by peano4::parallel::SpacetreeSet::traverse().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ deregisterId()

void peano4::parallel::Node::deregisterId ( int id )

Only the SpacetreeSet should call this operation.

See also: getId(int,int)

Definition at line 188 of file Node.cpp.

References assertion1, and logDebug.

Referenced by peano4::parallel::SpacetreeSet::cleanUpTrees().

Here is the caller graph for this function:

◆ getCurrentProgramStep()

int peano4::parallel::Node::getCurrentProgramStep ( ) const

Definition at line 447 of file Node.cpp.

Referenced by selectNextAlgorithmicStep(), and step().

Here is the caller graph for this function:

◆ getGlobalTreeId()

int peano4::parallel::Node::getGlobalTreeId ( int treeId ) const

Definition at line 131 of file Node.cpp.

References tarch::mpi::Rank::getInstance(), tarch::mpi::Rank::getNumberOfRanks(), and tarch::mpi::Rank::getRank().

Here is the call graph for this function:

◆ getGridDataExchangeMetaInformation()

peano4::parallel::Node::GridDataExchangeMetaInformation peano4::parallel::Node::getGridDataExchangeMetaInformation	(	int	sendingTreeId,
		int	receivingTreeId,
		ExchangeMode	exchange ) const

I use two tags per spacetree per rank: one for boundary data (horizontal) and one for up-down and synchronous (forks) data exchanges (vertical and fork/join).

The nice thing about Peano's new data management is that it is stack-only. Furthermore, all stack data exchange is triggered via the spacetree set, i.e. the order of the stacks is known externally, too. So we effectively do not need that many tags even though we need different tags per tree pair.

Definition at line 476 of file Node.cpp.

References assertion, logTraceInWith3Arguments, logTraceOutWith1Argument, and toString().

Referenced by peano4::parallel::SpacetreeSet::exchangeAllHorizontalDataExchangeStacks(), peano4::parallel::SpacetreeSet::streamDataFromSplittingTreeToNewTree(), and peano4::parallel::tests::NodeTest::testTagCalculation().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getId()

int peano4::parallel::Node::getId	(	int	rank,
		int	localTreeId ) const

private

Peano maps grid instance threads + mpi ranks onto global IDs through this routine.

To identify a global stack number, please use this function and add it to your local stack number.

Definition at line 113 of file Node.cpp.

References tarch::mpi::Rank::getInstance(), and tarch::mpi::Rank::getNumberOfRanks().

Here is the call graph for this function:

◆ getInputStackNumberForHorizontalDataExchange()

int peano4::parallel::Node::getInputStackNumberForHorizontalDataExchange ( int id )

static

Counterpart of getOutputStackNumberOfBoundaryExchange(int)

Definition at line 216 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

Referenced by peano4::parallel::SpacetreeSet::exchangeAllHorizontalDataExchangeStacks(), peano4::grid::Spacetree::receiveAndMergeGridVertexAtHorizontalBoundary(), and peano4::grid::Spacetree::receiveAndMergeUserData().

Here is the caller graph for this function:

◆ getInputStackNumberForVerticalDataExchange()

int peano4::parallel::Node::getInputStackNumberForVerticalDataExchange ( int id )

static

Get the input stack where a tree writes all of its vertical data to/from when it exchanges information with id.

Such vertical data is multiscale information or split/join data.

Definition at line 226 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

Referenced by peano4::grid::Spacetree::receiveAndMergeGridVertexAtVerticalBoundary().

Here is the caller graph for this function:

◆ getInstance()

peano4::parallel::Node & peano4::parallel::Node::getInstance ( )

static

This operation returns the singleton instance.

Before using this instance, one has to call the init() operation on the instance returned.

Returns: The singleton instance

Definition at line 108 of file Node.cpp.

Here is the caller graph for this function:

◆ getLocalTreeId()

int peano4::parallel::Node::getLocalTreeId ( int treeId ) const

See also: getId(int,int)

Definition at line 125 of file Node.cpp.

References tarch::mpi::Rank::getInstance(), and tarch::mpi::Rank::getNumberOfRanks().

Referenced by peano4::maps::HierarchicalStackMap< T >::clear(), peano4::maps::STDStackMap< T >::clear(), peano4::maps::HierarchicalStackMap< T >::empty(), peano4::maps::HierarchicalStackMap< T >::garbageCollection(), peano4::maps::STDStackMap< T >::garbageCollection(), peano4::parallel::SpacetreeSet::getAnswerTag(), peano4::maps::HierarchicalStackMap< T >::getForPop(), peano4::maps::HierarchicalStackMap< T >::getForPush(), peano4::maps::HierarchicalStackMap< T >::holdsStack(), and peano4::parallel::tests::NodeTest::testTagCalculation().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getNumberOfRegisteredTrees()

int peano4::parallel::Node::getNumberOfRegisteredTrees ( ) const

Returns: Number of registered trees on this rank

Definition at line 183 of file Node.cpp.

◆ getOutputStackForPeriodicBoundaryExchange()

int peano4::parallel::Node::getOutputStackForPeriodicBoundaryExchange ( int faceNumber )

static

Identify output stack for periodic boundary data written by face.

To make the mapping of face and vertex stacks consistent, we take the face number and determine what the adjacent, i.e. face-connected neighbour volume out of the \( 3^d \) adjacent, i.e. vertex-connected volumes is.

Take care that the result is not a mere linearisation of the index. You have to explicitly exclude the centre volume.

See also: getPeriodicBoundaryExchangeInputStackNumberForOutputStack() for a description how to compute an input stack index from the output index.; mapPeriodicBoundaryExchangeOutputStackOntoInputStack() for a mapping of periodic output stacks onto input stacks

Returns: The output stack for a periodic boundary data exchange for one face

Definition at line 250 of file Node.cpp.

References peano4::utils::dLinearised(), peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance, peano4::grid::PeanoCurve::NumberOfPeriodicBoundaryConditionStacks, and ThreePowerD.

Referenced by peano4::grid::Spacetree::receiveAndMergeUserData(), and peano4::grid::Spacetree::sendUserData().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getOutputStackNumberForHorizontalDataExchange()

int peano4::parallel::Node::getOutputStackNumberForHorizontalDataExchange ( int id )

static

Hand in a spacetree id and get back the number that we should use to send something to this tree.

Definition at line 211 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

Referenced by peano4::parallel::SpacetreeSet::exchangeAllHorizontalDataExchangeStacks(), peano4::grid::Spacetree::sendGridVertex(), and peano4::grid::Spacetree::sendUserData().

Here is the caller graph for this function:

◆ getOutputStackNumberForVerticalDataExchange()

int peano4::parallel::Node::getOutputStackNumberForVerticalDataExchange ( int id )

static

Definition at line 221 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

Referenced by peano4::grid::Spacetree::sendUserData(), peano4::parallel::SpacetreeSet::streamDataFromSplittingTreesToNewTrees(), peano4::parallel::SpacetreeSet::streamDataFromSplittingTreeToNewTree(), and peano4::grid::Spacetree::updateVertexAfterLoad().

Here is the caller graph for this function:

◆ getOutputStacksForPeriodicBoundaryExchange()

std::set< peano4::parallel::Node::PeriodicBoundaryStackIdentifier > peano4::parallel::Node::getOutputStacksForPeriodicBoundaryExchange ( const tarch::la::Vector< TwoPowerD, int > & flags )

static

You hand in a the flags of a vertex and you get the boundary stack identifiers including their directions.

Algorithm

We first analyse which faces with a periodic boundary flag are adjacent to the vertex. For this, we employ getPeriodicBoundaryNumber(). After that, we build the power set over the result: If a face is adjacent to the upper face and the left face, it also has to communicate along the diagonal, i.e. the logical combination of the two.

We now have to translate these powerset entries into stack indices. For this, we run over the \( 3^d \) neighbour cells or a cell, construct the index for this direction, and then look whether this index is contained within the power set.

Implementation

For the powerset, I'd like to use

 std::set< std::bitset<2*Dimensions> > powerSet;

but C++ defines no comparison operator on bitsets and therefore you can't use it in a set.

Definition at line 282 of file Node.cpp.

References assertion5, dfor3, enddforx, logTraceInWith2Arguments, logTraceOutWith2Arguments, peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance, and peano4::grid::PeanoCurve::NumberOfPeriodicBoundaryConditionStacks.

Referenced by peano4::grid::Spacetree::receiveAndMergeGridVertexAtHorizontalBoundary(), peano4::grid::Spacetree::receiveAndMergeUserData(), peano4::grid::Spacetree::sendGridVertex(), peano4::grid::Spacetree::sendUserData(), and peano4::parallel::tests::NodeTest::testGetOutputStacksForPeriodicBoundaryExchange().

Here is the caller graph for this function:

◆ getPeriodicBoundaryExchangeInputStackNumberForOutputStack() [1/2]

int peano4::parallel::Node::getPeriodicBoundaryExchangeInputStackNumberForOutputStack ( int outputStackNumber )

static

Hand in an output stack number of a face and you get back the input stack number.

If you want to use it for merging data, you have to be careful: Think about the right face of your domain. It is assigned a BC output stack. This routine tells you onto which input stack this output stack is mapped. In return, if you read in the periodic BC data for the right face, you will merge a data with its own! You however have to merge it with the input BC stack from the left face.

This routine is not made to orchestrate the stack exchange! It is meant that you hand in number to which a vertex or face would write to. And then you get the number back which it should read from. This is not how the stacks are permuted after each step! If you want to achieve this, you have to call mapPeriodicBoundaryExchangeOutputStackOntoInputStack().

See also: peano4::parallel::SpacetreeSet::exchangeAllPeriodicBoundaryDataStacks() for a routine which uses this one in return.; peano4::grid::PeanoCurve:MaxNumberOfCoreStacksPerSpacetreeInstance for a discussion of the ordering of the stacks.

Definition at line 355 of file Node.cpp.

References assertion, logTraceInWith4Arguments, logTraceOutWith1Argument, peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance, and peano4::grid::PeanoCurve::NumberOfPeriodicBoundaryConditionStacks.

Referenced by peano4::grid::Spacetree::receiveAndMergeGridVertexAtHorizontalBoundary(), peano4::grid::Spacetree::receiveAndMergeUserData(), and peano4::parallel::tests::NodeTest::testGetOutputStacksForPeriodicBoundaryExchange().

Here is the caller graph for this function:

◆ getPeriodicBoundaryExchangeInputStackNumberForOutputStack() [2/2]

peano4::parallel::Node::PeriodicBoundaryStackIdentifier peano4::parallel::Node::getPeriodicBoundaryExchangeInputStackNumberForOutputStack ( PeriodicBoundaryStackIdentifier outputStackIdentifier )

static

Definition at line 365 of file Node.cpp.

References assertion, logTraceInWith5Arguments, logTraceOutWith2Arguments, peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance, and peano4::grid::PeanoCurve::NumberOfPeriodicBoundaryConditionStacks.

◆ getPeriodicBoundaryNumber()

std::bitset< 2 *Dimensions > peano4::parallel::Node::getPeriodicBoundaryNumber ( const tarch::la::Vector< TwoPowerD, int > & flags )

staticprivate

Analyse to which of the 2d faces it is adjacent.

We enumerate again in the way we always enumerate, i.e. first the face with the x axis as normal, then the one with the y axis, ... And after these d faces, we enumerate all the opposite faces on the cube.

The routine is non-trivial: We store the \( 2^d \) adjacency flags per vertex, and now have to work backwards which periodic faces they do represent.

Definition at line 231 of file Node.cpp.

References dfor2, peano4::utils::dLinearised(), enddforx, and peano4::grid::Spacetree::RankOfPeriodicBoundaryCondition.

Referenced by peano4::parallel::tests::NodeTest::testGetPeriodicBoundaryNumber().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getRank()

int peano4::parallel::Node::getRank ( int treeId ) const

You hand in a tree number and the node tells you on which rank such a tree is hosted.

See also: getId(int,int)

Definition at line 119 of file Node.cpp.

References tarch::mpi::Rank::getInstance(), and tarch::mpi::Rank::getNumberOfRanks().

Referenced by peano4::parallel::SpacetreeSet::addSpacetree(), peano4::parallel::SpacetreeSet::cleanUpTrees(), peano4::parallel::SpacetreeSet::exchangeAllHorizontalDataExchangeStacks(), and peano4::parallel::SpacetreeSet::streamDataFromSplittingTreeToNewTree().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getSemanticsForTag()

std::string peano4::parallel::Node::getSemanticsForTag ( int tag )

static

I use this for debugging.

When I have a tag, I can ask the node whether the node is aware what this tag actually means.

Definition at line 497 of file Node.cpp.

References assertion.

◆ getTreeNumberTiedToExchangeStackNumber()

int peano4::parallel::Node::getTreeNumberTiedToExchangeStackNumber ( int number )

static

Gives you back the id of a communication partner, i.e.

a spacetree number, which corresponds to stack number. This works for all kinds of stacks.

Definition at line 413 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

Referenced by peano4::parallel::SpacetreeSet::exchangeAllHorizontalDataExchangeStacks().

Here is the caller graph for this function:

◆ init()

void peano4::parallel::Node::init ( )

Definition at line 77 of file Node.cpp.

References tarch::mpi::Rank::getInstance(), and tarch::mpi::Rank::reserveFreeTag().

Referenced by peano4::initSingletons().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ initMPIDatatypes()

void peano4::parallel::Node::initMPIDatatypes ( )

static

I originally wanted to embed these guys into the singleton's constructor.

However, the singleton might not yet be up when I run the (mpi) tests.

Definition at line 27 of file Node.cpp.

References peano4::datamanagement::CellMarker::initDatatype(), peano4::grid::AutomatonState::initDatatype(), peano4::grid::GridControlEvent::initDatatype(), peano4::grid::GridStatistics::initDatatype(), peano4::grid::GridVertex::initDatatype(), peano4::parallel::StartTraversalMessage::initDatatype(), peano4::parallel::TreeManagementMessage::initDatatype(), logTraceIn, and logTraceOut.

Referenced by peano4::initParallelEnvironment().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ isGlobalMaster()

bool peano4::parallel::Node::isGlobalMaster ( int treeId )

static

Is this the global master?

Definition at line 103 of file Node.cpp.

◆ isHorizontalDataExchangeInputStackNumber()

bool peano4::parallel::Node::isHorizontalDataExchangeInputStackNumber ( int number )

static

Definition at line 393 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

◆ isHorizontalDataExchangeOutputStackNumber()

bool peano4::parallel::Node::isHorizontalDataExchangeOutputStackNumber ( int number )

static

See getOutputStackNumberOfBoundaryExchange().

Definition at line 385 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

Referenced by peano4::parallel::SpacetreeSet::exchangeAllHorizontalDataExchangeStacks().

Here is the caller graph for this function:

◆ isPeriodicBoundaryExchangeOutputStackNumber()

bool peano4::parallel::Node::isPeriodicBoundaryExchangeOutputStackNumber ( int number )

static

Definition at line 377 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance, and peano4::grid::PeanoCurve::NumberOfPeriodicBoundaryConditionStacks.

Referenced by peano4::parallel::SpacetreeSet::exchangeAllPeriodicBoundaryDataStacks(), and peano4::parallel::tests::NodeTest::testGetOutputStacksForPeriodicBoundaryExchange().

Here is the caller graph for this function:

◆ isStorageStackNumber()

bool peano4::parallel::Node::isStorageStackNumber ( int number )

static

There are communication stacks and storage stacks.

This is a storage stack.

Definition at line 198 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

Referenced by peano4::maps::HierarchicalStackMap< T >::garbageCollection(), and peano4::maps::STDStackMap< T >::garbageCollection().

Here is the caller graph for this function:

◆ isVerticalDataExchangeInputStackNumber()

bool peano4::parallel::Node::isVerticalDataExchangeInputStackNumber ( int number )

static

Definition at line 407 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

◆ isVerticalDataExchangeOutputStackNumber()

bool peano4::parallel::Node::isVerticalDataExchangeOutputStackNumber ( int number )

static

Definition at line 401 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance.

◆ mapPeriodicBoundaryExchangeOutputStackOntoInputStack()

int peano4::parallel::Node::mapPeriodicBoundaryExchangeOutputStackOntoInputStack ( int outputStack )

static

See also: tests::NodeTest::testGetOutputStacksForPeriodicBoundaryExchange()

Definition at line 203 of file Node.cpp.

References peano4::grid::PeanoCurve::MaxNumberOfCoreStacksPerSpacetreeInstance, and peano4::grid::PeanoCurve::NumberOfPeriodicBoundaryConditionStacks.

Referenced by peano4::parallel::SpacetreeSet::exchangeAllPeriodicBoundaryDataStacks(), and peano4::parallel::tests::NodeTest::testGetOutputStacksForPeriodicBoundaryExchange().

Here is the caller graph for this function:

◆ registerId()

void peano4::parallel::Node::registerId	(	int	id,
		int	masterId )

private

The operation is not thread-safe as we call it only internally, i.e.

you are not supposed to call this function directly.

See also: getId(int,int)

Definition at line 164 of file Node.cpp.

References assertion, assertion2, logTraceInWith2Arguments, logTraceOut, peano4::parallel::TreeEntry::setId(), and peano4::parallel::TreeEntry::setMaster().

Here is the call graph for this function:

◆ reserveId()

int peano4::parallel::Node::reserveId	(	int	rank,
		int	forTreeId )

This operation is not const as it does some internal bookkeeping.

It internally invokes registerId() on the result.

Parameters

forTreeId If we run a split, this is the rank that wants to split. In general, this always is the parent/master of the new rank. So if you move a rank, you have to ensure that the master is preserved.

Returns: -1 If there are no ids left anymore. In principle, we do not limit the number of threads at all. However, we have to work with a limited number of tags in MPI, so we put a cap on the number of trees per node.

See also: getId(int,int)

Definition at line 137 of file Node.cpp.

References tarch::mpi::Rank::getInstance(), logDebug, and logWarning.

Referenced by peano4::parallel::SpacetreeSet::answerQuestions(), and peano4::parallel::SpacetreeSet::split().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ setNextProgramStep()

void peano4::parallel::Node::setNextProgramStep ( int number )

The user tells the set which program step to use, i.e.

which adapter to use for the next grid run or which step to run next. Should only be called on global master.

Definition at line 441 of file Node.cpp.

References assertion1.

Referenced by runParallel(), runParallel(), and selectNextAlgorithmicStep().

Here is the caller graph for this function:

◆ shutdown()

void peano4::parallel::Node::shutdown ( )

The shutdown is invoked by peano4::shutdownSingletons()

If we run it on the global master, we set the program state to terminate and distribute this message. For this, we hijack the routine continueToRun(). This is really a hack. We in particular have to be careful and may not call the continue operation on the other ranks. They typically are stuck in a continue while loop in their main routine and just wait for the global master to send out this terminate command.

It is important that this is the very first call when you want your application to go down. Invoked on the global master, it triggers the shutdown on all the other ranks. You will get deadlocks if you try to free any common memory (RDMA) or free datatypes if you don't call the node's shutdown first.

Other ranks might struggle to receive the shutdown command as they might still ask for further subtrees, e.g., or issue load balancing. Therefore, I insert a Peano 4 barrier right after we've issued the shutdown. It makes the rank that goes down handle all incoming data until we can be sure that everybody is in shutdown mode.

Definition at line 452 of file Node.cpp.

References tarch::mpi::Rank::barrier(), tarch::mpi::Rank::getInstance(), tarch::services::ServiceRepository::getInstance(), logTraceIn, logTraceOut, tarch::services::ServiceRepository::receiveDanglingMessages(), and Terminate.

Referenced by peano4::shutdownSingletons().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ shutdownMPIDatatypes()

void peano4::parallel::Node::shutdownMPIDatatypes ( )

static

Definition at line 44 of file Node.cpp.

References logTraceIn, logTraceOut, peano4::datamanagement::CellMarker::shutdownDatatype(), peano4::grid::AutomatonState::shutdownDatatype(), peano4::grid::GridControlEvent::shutdownDatatype(), peano4::grid::GridStatistics::shutdownDatatype(), peano4::grid::GridVertex::shutdownDatatype(), peano4::parallel::StartTraversalMessage::shutdownDatatype(), and peano4::parallel::TreeManagementMessage::shutdownDatatype().

Referenced by peano4::shutdownParallelEnvironment().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ toString() [1/2]

std::string peano4::parallel::Node::toString ( const std::set< PeriodicBoundaryStackIdentifier > & data )

static

Definition at line 262 of file Node.cpp.

Referenced by peano4::parallel::tests::NodeTest::testGetOutputStacksForPeriodicBoundaryExchange().

Here is the caller graph for this function:

◆ toString() [2/2]

std::string peano4::parallel::Node::toString ( ExchangeMode mode )

static

Definition at line 92 of file Node.cpp.

Friends And Related Symbol Documentation

◆ peano4::parallel::tests::NodeTest

friend class peano4::parallel::tests::NodeTest

friend

Definition at line 46 of file Node.h.

Field Documentation

◆ _currentProgramStep

int peano4::parallel::Node::_currentProgramStep

private

Definition at line 69 of file Node.h.

◆ _dataExchangeCommunicators

MPI_Comm peano4::parallel::Node::_dataExchangeCommunicators[MaxSpacetreesPerRank]

private

Definition at line 67 of file Node.h.

◆ _log

tarch::logging::Log peano4::parallel::Node::_log

staticprivate

Logging device.

Definition at line 51 of file Node.h.

◆ _rankOrchestrationTag

int peano4::parallel::Node::_rankOrchestrationTag

private

Definition at line 71 of file Node.h.

◆ _semaphore

tarch::multicore::BooleanSemaphore peano4::parallel::Node::_semaphore

private

Definition at line 55 of file Node.h.

◆ _singleton

peano4::parallel::Node peano4::parallel::Node::_singleton

staticprivate

Definition at line 53 of file Node.h.

◆ _treeEntries

std::map< int, TreeEntry > peano4::parallel::Node::_treeEntries

private

Definition at line 73 of file Node.h.

◆ MaxSpacetreesPerRank

constexpr int peano4::parallel::Node::MaxSpacetreesPerRank = 1024

staticconstexpr

Definition at line 39 of file Node.h.

Referenced by peano4::parallel::SpacetreeSet::init(), and peano4::parallel::tests::NodeTest::testTagCalculation().

◆ StacksPerCommunicationPartner

constexpr int peano4::parallel::Node::StacksPerCommunicationPartner = 4

staticconstexprprivate

We need one stack for outgoing data, one for incoming data, and we distinguish horizontal and vertical data flow.

Definition at line 62 of file Node.h.

Referenced by peano4::parallel::tests::NodeTest::testTagCalculation().

◆ Terminate

constexpr int peano4::parallel::Node::Terminate = -2

staticconstexpr

Definition at line 38 of file Node.h.

Referenced by shutdown().

◆ UndefProgramStep

constexpr int peano4::parallel::Node::UndefProgramStep = -1

staticconstexpr

Value for _currentProgramStep.

Definition at line 44 of file Node.h.

The documentation for this class was generated from the following files:

src/peano4/parallel/Node.h
src/peano4/parallel/Node.cpp

Public Types

Public Member Functions

Static Public Member Functions

Static Public Attributes

Private Member Functions

Static Private Member Functions

Private Attributes

Static Private Attributes

Friends

Detailed Description

Member Typedef Documentation

◆ GridDataExchangeMetaInformation

◆ PeriodicBoundaryStackIdentifier

Member Enumeration Documentation

◆ ExchangeMode

Constructor & Destructor Documentation

◆ Node() [1/2]

◆ Node() [2/2]

◆ ~Node()

Member Function Documentation

◆ continueToRun()

◆ deregisterId()

◆ getCurrentProgramStep()

◆ getGlobalTreeId()

◆ getGridDataExchangeMetaInformation()

◆ getId()

◆ getInputStackNumberForHorizontalDataExchange()

◆ getInputStackNumberForVerticalDataExchange()

◆ getInstance()

◆ getLocalTreeId()

◆ getNumberOfRegisteredTrees()

◆ getOutputStackForPeriodicBoundaryExchange()

◆ getOutputStackNumberForHorizontalDataExchange()

◆ getOutputStackNumberForVerticalDataExchange()

◆ getOutputStacksForPeriodicBoundaryExchange()

Algorithm

Implementation

◆ getPeriodicBoundaryExchangeInputStackNumberForOutputStack() [1/2]

◆ getPeriodicBoundaryExchangeInputStackNumberForOutputStack() [2/2]

◆ getPeriodicBoundaryNumber()

◆ getRank()

◆ getSemanticsForTag()

◆ getTreeNumberTiedToExchangeStackNumber()

◆ init()

◆ initMPIDatatypes()

◆ isGlobalMaster()

◆ isHorizontalDataExchangeInputStackNumber()

◆ isHorizontalDataExchangeOutputStackNumber()

◆ isPeriodicBoundaryExchangeOutputStackNumber()

◆ isStorageStackNumber()

◆ isVerticalDataExchangeInputStackNumber()

◆ isVerticalDataExchangeOutputStackNumber()

◆ mapPeriodicBoundaryExchangeOutputStackOntoInputStack()

◆ registerId()

◆ reserveId()

◆ setNextProgramStep()

◆ shutdown()

◆ shutdownMPIDatatypes()

◆ toString() [1/2]

◆ toString() [2/2]

Friends And Related Symbol Documentation

◆ peano4::parallel::tests::NodeTest

Field Documentation

◆ _currentProgramStep

◆ _dataExchangeCommunicators

◆ _log

◆ _rankOrchestrationTag

◆ _semaphore

◆ _singleton

◆ _treeEntries

◆ MaxSpacetreesPerRank

◆ StacksPerCommunicationPartner

◆ Terminate

◆ UndefProgramStep