diff --git a/README.md b/README.md index 1d3ce928bc443455757e05f8e656bbdec6f8e289..7f7051ef8df9dca5af63d5c048c5278b1294a5d9 100644 --- a/README.md +++ b/README.md @@ -1,102 +1,63 @@ IO container: ============= -Description see D3.1 -https://bscw.zam.kfa-juelich.de/bscw/bscw.cgi/2995682?op=preview&back_url=2929708 - - -Components -1. Mero container expansion -2. Object service -3. I/O container -4. Manager - -Common requirements: -1. Messaging system - a. ZeroMQ interface - b. Protobuf for serialisation - -Changes from D3.1: ------------------- -The following lists main changes compared to description given in D3.1: -- Pivoted to using MIO instead of clovis directly - - -2. Object service: ------------------- -Sub components: -1. Object service process -2. API or interface: possibly intercepting Clovis calls -3. Messaging interface to enact process requests - -Open Questions: -- Based on what is the object service going to select specific containers? - - One option is prestart configs. These are set by the user before starting the object service. Ex. 10 object containers, round robin. - - Alternatively, set by direct calling of the object service functions directly. We will target this now and move to more complex alternatives based on user requests. - - - -4. Manager ----------- -Note: Manager needs to be able to remote execute (Start services) - - -5. I/O container ------------------ -An I/O container is made out of a set of applications to run and a set of object services. These are listed in a configuration file. Along with start and stop behaviour. -The I/O container is started by the Manager as well as paused and resumed. But if all applications within it are done, it will also stop operation. -When started the container will read the config file and update a container run file as necessary. - -An I/O container has a data structure for all executables in it's list. Each entry has the following items: -- type -- name or id -- options arguments -- pre-exec -- post-exec - -How to deal with object services on startup in container?: -- Only manager can start an object service: - - Advantage: Manager would always know about all running object services - - Disadvantage: Centralised and requires communication between starter task (ex. APP or I/O container) and Manager - -- Object service communicates with Manager to register it's ID on startup: - - Advantage: Startup of object service will just be a fork exec - - Disadvantage: How to recognise if object service successfully started. Example scenario, object service with same ID already running. - - -6. Object Container: --------------------- -- Pivoted away from Mero containers to using MIO KVS - - Object ids are now values (i.e. no need for applications to ask for generated obj_ids) - - The key schema is kept in case of need to port back to Mero container -- Problem: Missing NEXT operation - -6.1 Key strucutre and reserved keys -=================================== - -Following the object id and KVS id, the key length is 128bit. -In string hex format there are 32 digits: - "XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX" -Insertion layers | 1 || 2 || 3 || 4 ||5 ||6,7| - -We reserve the following list of keys: -| 1 || 2 || 3 || 4 ||5 ||6,7| -"XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX" : VALUE -|-----------------------------------------------------------------| - 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : Sort type - 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 : Read next key - 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 : Read previous key - 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 00 : Write next key - 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 : Write previous key - ... - 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 : First Object in container - ... - XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX : Object X in container (See insertion layers) - ... - ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff : Initial value (ERROR when inserted to KV) - ... - 00 XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX : Reverse list (Above key containing object) - +# Building +The IO container is built using CMake. + +Clone repository. +``` +$ git clone https://gitlab.version.fz-juelich.de/sage2/iocontainers.git +$ cd iocontainers +``` +## Requirments +With the exception of Motr all requirments are added as Git submodules. +- cortx-mio +- cppzmq +- libzmq +- spdlog + +Build all requirments with prefix `iocontainers/packages` + +In case of locally building or system installing the requirments, adjust the cmake configurations accordingly + +## Building the IO container components + +``` +$ mkdir iocontainers/build +$ cd iocontainers/build +$ cmake ../ +$ make +``` + +## Executables +- `key_test.x` key unittest +- `obj_container_test.x` object container unittest +- `obj_service_test.x` object service unittest +- `obj_service_daemon.x` object service daemon +- `io_container_daemon.x` IO container daemon + +# Usage +For usage check the unittests under `iocontainers/unittests`. + +``` +$ ./obj_service_daemon.x --help +./obj_service_daemon.x: invalid option -- '-' +Unknown option `--'. +Usage: ./obj_service_daemon.x -i link [-k kvs_id -s Schema] + -i full link to application ex. "tcp://localhost:5555" + -c MIO config file + -k kvs_id of the object service (If not available will be created, otherwise opened) + -s Schema of the object service (0: Round Robin, 1: SINGLE, ...) +``` + +``` +$ ./io_container_daemon.x --help +./io_container_daemon.x: invalid option -- '-' +Unknown option `--'. +Usage: ./io_container_daemon.x -c /path/to/config/file + -c /path/to/config/file + -h This message +``` # License The iocontainer software is distributed under Apache License (Version 2.0). See LICENSE for details.