![]() |
Peano
|
#include <BooleanSemaphore.h>
Data Structures | |
struct | SemaphoreMapEntry |
Public Member Functions | |
void | init () |
virtual void | shutdown () override |
~BooleanSemaphoreService ()=default | |
Destructor of the service. | |
virtual void | receiveDanglingMessages () override |
Receive any lock-related dangling messages. | |
void | acquireLock (int number) |
Acquire the lock. | |
void | releaseLock (int number) |
Release a lock. | |
int | getNumberOfLockedSemaphores () |
std::string | toString () const |
![]() | |
virtual | ~Service () |
Static Public Member Functions | |
static BooleanSemaphoreService & | getInstance () |
Don't use this routine. | |
Private Member Functions | |
BooleanSemaphoreService ()=default | |
BooleanSemaphoreService (const BooleanSemaphoreService &)=delete | |
void | serveLockRequests () |
Serve pending lock requests. | |
bool | tryLockSemaphoreOnGlobalMaster (int number, int forRank) |
Try to lock a semaphore. | |
void | lockSemaphoreOnGlobalMaster (int number, int forRank) |
void | unlockSemaphoreOnGlobalMaster (int number, int forRank) |
Unlock semaphore on global master. | |
void | addMapEntryLazily (int number) |
Add map entry. | |
Private Attributes | |
int | _semaphoreTag |
Tag used to exchange locking MPI messages. | |
tarch::multicore::BooleanSemaphore | _mapAccessSemaphore |
Semaphore for the global master's map of locks. | |
tarch::multicore::BooleanSemaphore | _reserverationRequestsSemaphore |
std::map< int, SemaphoreMapEntry > | _map |
Map of semaphores. | |
std::vector< std::pair< int, int > > | _pendingLockRequests |
List of pending lock requests. | |
Static Private Attributes | |
static BooleanSemaphoreService | _singleton |
Additional Inherited Members | |
![]() | |
tarch::multicore::RecursiveSemaphore | _receiveDanglingMessagesSemaphore |
Recursive semaphores. | |
Definition at line 73 of file BooleanSemaphore.h.
|
privatedefault |
|
privatedelete |
|
default |
Destructor of the service.
As the service is a singleton and a service (hence the name), it has to deregister itself. Otherwise, the overall service landscape still might try to call receiveDanglingMessages().
Acquire the lock.
If this routine is called on the global master, we redirect the call to lockSemaphoreOnGlobalMaster(). If it is called on another rank, we send a lock message to the global master.
Definition at line 214 of file BooleanSemaphore.cpp.
References _semaphoreTag, assertionMsg, tarch::mpi::Rank::getGlobalMasterRank(), tarch::mpi::Rank::getInstance(), tarch::services::ServiceRepository::getInstance(), lockSemaphoreOnGlobalMaster(), logDebug, tarch::services::ServiceRepository::receiveDanglingMessages(), tarch::mpi::Rank::setDeadlockTimeOutTimeStamp(), tarch::mpi::Rank::setDeadlockWarningTimeStamp(), tarch::mpi::Rank::triggerDeadlockTimeOut(), and tarch::mpi::Rank::writeTimeOutWarning().
Referenced by tarch::mpi::BooleanSemaphore::enterCriticalSection().
Add map entry.
Only defined/used on global master. We do not maintain a map of all existing semaphores, but build up a dictionary of these guys at runtime in a lazy fashion.
This routine is not thread-safe, i.e. I assume that somebody calling this routine has ensured that no data races occur and notably that noone uses _map meanwhile.
number | Every boolean semaphore has a globally unique positive number. The MPI messages we send around carry this number if we want to acquire a lock. They carry the negative number if we want to release a lock. To allow for these sign flips, number may not equal zero. |
Definition at line 206 of file BooleanSemaphore.cpp.
References _map, and assertion1.
Referenced by tryLockSemaphoreOnGlobalMaster().
|
static |
Don't use this routine.
It returns the global semaphore and is only used on the master/by the core.
Definition at line 201 of file BooleanSemaphore.cpp.
References _singleton.
Referenced by tarch::mpi::BooleanSemaphore::enterCriticalSection(), peano4::initSingletons(), and peano4::shutdownSingletons().
int tarch::mpi::BooleanSemaphore::BooleanSemaphoreService::getNumberOfLockedSemaphores | ( | ) |
Definition at line 191 of file BooleanSemaphore.cpp.
References _pendingLockRequests, _reserverationRequestsSemaphore, and tarch::mpi::Rank::getInstance().
void tarch::mpi::BooleanSemaphore::BooleanSemaphoreService::init | ( | ) |
Definition at line 50 of file BooleanSemaphore.cpp.
References _semaphoreTag, tarch::services::ServiceRepository::addService(), tarch::services::ServiceRepository::getInstance(), and tarch::mpi::Rank::reserveFreeTag().
Referenced by peano4::initSingletons().
|
private |
We always have to call receiveDanglingMessages() quite aggressively. On the one hand, we might try to lock a semaphore which is already locked by another rank. We have to give that other rank the opportunity to free the lock again. If we don't allow this to happen, we will never get the opportunity to lock this semaphore for our own rank, i.e. the global master. s * There is another reason for this polling, too: It is important that we don't prematurely lock the semaphore if we are called on the master. We should always try to serve other ranks first to avoid massive MPI rank divergence. Further to that, we take the flag _localRankLockRequestSemaphore into account. If it is set, we know that each rank will request the semaphore at least once. Consequently, we do acquire it if and only if another rank has grabbed it before. Basically, we try to prioritise all other ranks and let locks by rank 0 pass through last. Please consult the class documentation on potential deadlocks.
Definition at line 95 of file BooleanSemaphore.cpp.
References _map, assertion, tarch::mpi::Rank::getInstance(), logDebug, logWarning, receiveDanglingMessages(), tarch::mpi::Rank::setDeadlockTimeOutTimeStamp(), tarch::mpi::Rank::setDeadlockWarningTimeStamp(), and tryLockSemaphoreOnGlobalMaster().
Referenced by acquireLock().
|
overridevirtual |
Receive any lock-related dangling messages.
This routine polls and if there's a release or request message, it tries to answer it.
We rely on the fact that no multiple threads can access the receiveDangling thing at the same time if you go through the services. In return, please do never invoke this routine directly: If two threads jump into receiveDangingMessages, they might issue the Iprobe one after another and get a "yes" - even though there is only one message in the queue. Consequently, both threads bump into the receive operation. Now, only one message is there and, consequently, the second thread deadlocks.
The message polls for messages. Messages with a positive value mean the system tries to acquire an MPI lock. Messages with a negative value mean that the corresponding ranks wants to release the lock with the corresponding positive value.
Release requests are handled immediately by calling releaseLock(). Lock requests are not handled direclty. Instead, we enqueue them into _pendingLockRequests. Eventually, I serve them by invoking serveLockRequests(). This is a function I call if and only if no further messages are pending. Therefore, this code fragments prioritises frees over locks.
Implements tarch::services::Service.
Definition at line 159 of file BooleanSemaphore.cpp.
References _pendingLockRequests, _reserverationRequestsSemaphore, _semaphoreTag, assertion, tarch::mpi::Rank::getCommunicator(), tarch::mpi::Rank::getInstance(), logDebug, releaseLock(), serveLockRequests(), and toString().
Referenced by lockSemaphoreOnGlobalMaster().
Release a lock.
This routine delegates the call to unlockSemaphoreOnGlobalMaster() if we run it on the global master. To avoid rank divergence, we immediately serve follow-up lock requests after we've freed the lock.
If we are not on the global master, we send an unlock message to this master and return. There will be no confirmation or similar.
Definition at line 246 of file BooleanSemaphore.cpp.
References _semaphoreTag, assertion, assertionMsg, tarch::mpi::Rank::getGlobalMasterRank(), tarch::mpi::Rank::getInstance(), logDebug, serveLockRequests(), and unlockSemaphoreOnGlobalMaster().
Referenced by receiveDanglingMessages().
|
private |
Serve pending lock requests.
If a remote rank does request a lock, I don't reserve it straightway. Instead, I buffer it in _pendingLockRequests. This buffering is realised within receiveDanglingMessages().
From time to time, I now have to check whether I can serve the request. I do so in receiveDanglingMessages() whenever there are no more messages to be appended to the list of locks, and I also try to serve requests directly after someone has released a lock, hoping that this is the perfect timing.
The routine serves delegates the actual decision whether to lock or not to tryLockSemaphoreOnGlobalMaster(), but it can serve multiple lock requests. This makes sense, as the global master manages all sempahores and not only one, i.e. it might be able to serve multiple requests. Actually, to avoid deadlock scenarios, it is furthermore important that the routine runs through all requests.
Definition at line 132 of file BooleanSemaphore.cpp.
References _pendingLockRequests, _reserverationRequestsSemaphore, _semaphoreTag, assertion, tarch::mpi::Rank::getCommunicator(), tarch::mpi::Rank::getInstance(), logDebug, toString(), and tryLockSemaphoreOnGlobalMaster().
Referenced by receiveDanglingMessages(), and releaseLock().
|
overridevirtual |
Implements tarch::services::Service.
Definition at line 55 of file BooleanSemaphore.cpp.
References tarch::services::ServiceRepository::getInstance(), and tarch::services::ServiceRepository::removeService().
Referenced by peano4::shutdownSingletons().
std::string tarch::mpi::BooleanSemaphore::BooleanSemaphoreService::toString | ( | ) | const |
Definition at line 59 of file BooleanSemaphore.cpp.
References _map, and _pendingLockRequests.
Referenced by receiveDanglingMessages(), serveLockRequests(), and unlockSemaphoreOnGlobalMaster().
|
private |
Try to lock a semaphore.
This routine is only called on the global master. We try to lock the global semaphore number for forRank. If that works, we make a note for which rank we have locked it. Before we however check if we can serve a request, we invoke addMapEntryLazily(), i.e. we ensure that there is an entry in the database for forRank.
Definition at line 75 of file BooleanSemaphore.cpp.
References _map, _mapAccessSemaphore, addMapEntryLazily(), assertion, tarch::mpi::Rank::getInstance(), and logDebug.
Referenced by lockSemaphoreOnGlobalMaster(), and serveLockRequests().
|
private |
Unlock semaphore on global master.
Very simple implementation:
Definition at line 118 of file BooleanSemaphore.cpp.
References _map, _mapAccessSemaphore, assertion, assertion2, assertionEquals, tarch::mpi::Rank::getInstance(), logDebug, and toString().
Referenced by releaseLock().
|
private |
Map of semaphores.
This is the actual map which stores per semaphore number a bool that says whether it is currently taken. So false means noone's requested it, true means someone holds it.
Definition at line 124 of file BooleanSemaphore.h.
Referenced by addMapEntryLazily(), lockSemaphoreOnGlobalMaster(), toString(), tryLockSemaphoreOnGlobalMaster(), and unlockSemaphoreOnGlobalMaster().
|
private |
Semaphore for the global master's map of locks.
On the master, I have to be sure that no two threads do access the map of semaphores concurrently. Therefore, I need a shared memory semaphore.
Definition at line 89 of file BooleanSemaphore.h.
Referenced by tryLockSemaphoreOnGlobalMaster(), and unlockSemaphoreOnGlobalMaster().
|
private |
List of pending lock requests.
If another rank requests a semaphore, I store this request temporarily here. Each entry is a map from a rank that requests access to the number of the semaphore it is asking for.
Definition at line 134 of file BooleanSemaphore.h.
Referenced by getNumberOfLockedSemaphores(), receiveDanglingMessages(), serveLockRequests(), and toString().
|
private |
Definition at line 90 of file BooleanSemaphore.h.
Referenced by getNumberOfLockedSemaphores(), receiveDanglingMessages(), and serveLockRequests().
|
private |
Tag used to exchange locking MPI messages.
Definition at line 80 of file BooleanSemaphore.h.
Referenced by acquireLock(), init(), receiveDanglingMessages(), releaseLock(), and serveLockRequests().
|
staticprivate |
Definition at line 75 of file BooleanSemaphore.h.
Referenced by getInstance().