@@ -37,10 +37,10 @@ management<cdo_management>` (and :ref:`memory management<mamba>` as well) is
then effectively delegated by Execution Frameworks and applications to Maestro
Core.
As an example of useful workflow components, staging and saving CDOs of choice
may be implemented by users via Librarian and Archiver components, the latter
relying especially on events to be notified of the availability of relevant
CDOs for archiving purposes.
As an example of useful workflow components, staging and archiving CDOs of
choice may be implemented by users via dedicated components, archiving relying
especially on events to be notified of the availability of relevant CDOs for
archiving purposes.
A keep-alive proxy may allow to retain CDOs from being withdrawn from the pool,
which would be before they have been properly handled by all the components
that might have needed them.
...
...
@@ -49,28 +49,18 @@ at finalize time, nonetheless Events allow to implement more specific
telemetry/logger components, with :ref:`selectors<events>` allowing
:ref:`cherry-picking<story_cherry>` of events and CDOs.
CDOs convey a great deal of data and memory semantics.
Data usage information such as CDO dependences or an eager transfer policy for
instance *(both yet to be implemented)* helps the scheduler and thus general
performance.
Data layout information lets Maestro make the transformations between a given
offered CDO in a given data layout L1 (say row-major) to the demanded CDO on
the consumer side that may have a layout L2 (say column-major). This allows
user to abstract away data layout, especially if tools / source-to-source
edition fills the data layout attribute (semi) automatically. Data layouts may
be distributed *(WIP)* and Maestro Core *will soon* handle the redistribution
complying with producer and consumer layout and dsitribution scheme
requirements.
.. image:: img/core_data_object.png
:alt: Anatomy of the Core Data Object
CDO location is expressed as a so-called Memory Object, which of course can mean a certain number of layers (DRAM, GDRAM, Parallel Filesystem, Object Store, etc) and associated access methods (pointer, path, object ID, etc.), which is then used by Maestro Core to handle the transport between layers and nodes.
Collections
- Group
- Selector (get/set, select, subscribe)
user metadata
CDOs may convey a considerable range of data and memory semantics.
In particular, data layout information lets Maestro make the transformations
between a given offered CDO in a given data layout L1 (say row-major) to the
demanded CDO on the consumer side that may have a layout L2 (say column-major).
This allows user to abstract away data layout, especially if tools or
source-to-source substitution fills the data layout attribute (semi)
automatically. Data layouts may be distributed and Maestro Core handle the
redistribution complying with producer and consumer layout and distribution
scheme requirements. CDO location is expressed as a so-called Memory Object,
which can mean a certain number of layers (DRAM, GDRAM, NVRAM, PFS, etc.) and
associated access methods (pointer, path, object ID, etc.), which is then used
by Maestro Core to handle the transport between layers and nodes.
