 |
Peano
|
|
Peano
|
#include <BooleanSemaphore.h>
Data Structures | |
| struct | SemaphoreMapEntry |
Public Member Functions | |
| void | init () |
| virtual void | shutdown () override |
| ~BooleanSemaphoreService ()=default | |
| Destructor of the service. | |
| virtual void | receiveDanglingMessages () override |
| Receive any lock-related dangling messages. | |
| void | acquireLock (int number) |
| Acquire the lock. | |
| void | releaseLock (int number) |
| Release a lock. | |
| int | getNumberOfLockedSemaphores () |
| std::string | toString () const |
 Public Member Functions inherited from tarch::services::Service | |
| virtual | ~Service () |
Static Public Member Functions | |
| static BooleanSemaphoreService & | getInstance () |
| Don't use this routine. | |
Private Member Functions | |
| BooleanSemaphoreService ()=default | |
| BooleanSemaphoreService (const BooleanSemaphoreService &)=delete | |
| void | serveLockRequests () |
| Serve pending lock requests. | |
| bool | tryLockSemaphoreOnGlobalMaster (int number, int forRank) |
| Try to lock a semaphore. | |
| void | lockSemaphoreOnGlobalMaster (int number, int forRank) |
| void | unlockSemaphoreOnGlobalMaster (int number, int forRank) |
| Unlock semaphore on global master. | |
| void | addMapEntryLazily (int number) |
| Add map entry. | |
Private Attributes | |
| int | _semaphoreTag |
| Tag used to exchange locking MPI messages. | |
| tarch::multicore::BooleanSemaphore | _mapAccessSemaphore |
| Semaphore for the global master's map of locks. | |
| tarch::multicore::BooleanSemaphore | _reserverationRequestsSemaphore |
| std::map< int, SemaphoreMapEntry > | _map |
| Map of semaphores. | |
| std::vector< std::pair< int, int > > | _pendingLockRequests |
| List of pending lock requests. | |
Static Private Attributes | |
| static BooleanSemaphoreService | _singleton |
Additional Inherited Members | |
| Protected Attributes inherited from tarch::services::Service | |
| tarch::multicore::RecursiveSemaphore | _receiveDanglingMessagesSemaphore |
| Recursive semaphores. | |
Definition at line 73 of file BooleanSemaphore.h.
|
privatedefault |
|
privatedelete |
|
default |
Destructor of the service.
As the service is a singleton and a service (hence the name), it has to deregister itself. Otherwise, the overall service landscape still might try to call receiveDanglingMessages().
| void tarch::mpi::BooleanSemaphore::BooleanSemaphoreService::acquireLock | ( | int | number | ) |
Acquire the lock.
If this routine is called on the global master, we redirect the call to lockSemaphoreOnGlobalMaster(). If it is called on another rank, we send a lock message to the global master.
|
private |
Add map entry.
Only defined/used on global master. We do not maintain a map of all existing semaphores, but build up a dictionary of these guys at runtime in a lazy fashion.
This routine is not thread-safe, i.e. I assume that somebody calling this routine has ensured that no data races occur and notably that noone uses _map meanwhile.
| number | Every boolean semaphore has a globally unique positive number. The MPI messages we send around carry this number if we want to acquire a lock. They carry the negative number if we want to release a lock. To allow for these sign flips, number may not equal zero. |
|
static |
Don't use this routine.
It returns the global semaphore and is only used on the master/by the core.
| int tarch::mpi::BooleanSemaphore::BooleanSemaphoreService::getNumberOfLockedSemaphores | ( | ) |
| void tarch::mpi::BooleanSemaphore::BooleanSemaphoreService::init | ( | ) |
|
private |
We always have to call receiveDanglingMessages() quite aggressively. On the one hand, we might try to lock a semaphore which is already locked by another rank. We have to give that other rank the opportunity to free the lock again. If we don't allow this to happen, we will never get the opportunity to lock this semaphore for our own rank, i.e. the global master. s * There is another reason for this polling, too: It is important that we don't prematurely lock the semaphore if we are called on the master. We should always try to serve other ranks first to avoid massive MPI rank divergence. Further to that, we take the flag _localRankLockRequestSemaphore into account. If it is set, we know that each rank will request the semaphore at least once. Consequently, we do acquire it if and only if another rank has grabbed it before. Basically, we try to prioritise all other ranks and let locks by rank 0 pass through last. Please consult the class documentation on potential deadlocks.
|
overridevirtual |
Receive any lock-related dangling messages.
This routine polls and if there's a release or request message, it tries to answer it.
We rely on the fact that no multiple threads can access the receiveDangling thing at the same time if you go through the services. In return, please do never invoke this routine directly: If two threads jump into receiveDangingMessages, they might issue the Iprobe one after another and get a "yes" - even though there is only one message in the queue. Consequently, both threads bump into the receive operation. Now, only one message is there and, consequently, the second thread deadlocks.
The message polls for messages. Messages with a positive value mean the system tries to acquire an MPI lock. Messages with a negative value mean that the corresponding ranks wants to release the lock with the corresponding positive value.
Release requests are handled immediately by calling releaseLock(). Lock requests are not handled direclty. Instead, we enqueue them into _pendingLockRequests. Eventually, I serve them by invoking serveLockRequests(). This is a function I call if and only if no further messages are pending. Therefore, this code fragments prioritises frees over locks.
Implements tarch::services::Service.
| void tarch::mpi::BooleanSemaphore::BooleanSemaphoreService::releaseLock | ( | int | number | ) |
Release a lock.
This routine delegates the call to unlockSemaphoreOnGlobalMaster() if we run it on the global master. To avoid rank divergence, we immediately serve follow-up lock requests after we've freed the lock.
If we are not on the global master, we send an unlock message to this master and return. There will be no confirmation or similar.
|
private |
Serve pending lock requests.
If a remote rank does request a lock, I don't reserve it straightway. Instead, I buffer it in _pendingLockRequests. This buffering is realised within receiveDanglingMessages().
From time to time, I now have to check whether I can serve the request. I do so in receiveDanglingMessages() whenever there are no more messages to be appended to the list of locks, and I also try to serve requests directly after someone has released a lock, hoping that this is the perfect timing.
The routine serves delegates the actual decision whether to lock or not to tryLockSemaphoreOnGlobalMaster(), but it can serve multiple lock requests. This makes sense, as the global master manages all sempahores and not only one, i.e. it might be able to serve multiple requests. Actually, to avoid deadlock scenarios, it is furthermore important that the routine runs through all requests.
|
overridevirtual |
Implements tarch::services::Service.
| std::string tarch::mpi::BooleanSemaphore::BooleanSemaphoreService::toString | ( | ) | const |
|
private |
Try to lock a semaphore.
This routine is only called on the global master. We try to lock the global semaphore number for forRank. If that works, we make a note for which rank we have locked it. Before we however check if we can serve a request, we invoke addMapEntryLazily(), i.e. we ensure that there is an entry in the database for forRank.
|
private |
Unlock semaphore on global master.
Very simple implementation:
|
private |
Map of semaphores.
This is the actual map which stores per semaphore number a bool that says whether it is currently taken. So false means noone's requested it, true means someone holds it.
Definition at line 124 of file BooleanSemaphore.h.
|
private |
Semaphore for the global master's map of locks.
On the master, I have to be sure that no two threads do access the map of semaphores concurrently. Therefore, I need a shared memory semaphore.
Definition at line 89 of file BooleanSemaphore.h.
|
private |
List of pending lock requests.
If another rank requests a semaphore, I store this request temporarily here. Each entry is a map from a rank that requests access to the number of the semaphore it is asking for.
Definition at line 134 of file BooleanSemaphore.h.
|
private |
Definition at line 90 of file BooleanSemaphore.h.
|
private |
Tag used to exchange locking MPI messages.
Definition at line 80 of file BooleanSemaphore.h.
|
staticprivate |
Definition at line 75 of file BooleanSemaphore.h.
1.10.0