Project deliverables
Below you can find links to the INTERTWinE project deliverables produced by the technical work packages.
The technical work packages are:
- WP3: Programming Model Interoperability
- WP4: Resource Management and Runtime Interoperability
-
WP5: Applications and Kernels
Deliverables for WP3: Programming Model Interoperability
This initial requirements analysis informed the direction of work at the project outset. It describes the status of the programming models addressed by the project, with particular attention to known interoperability issues already identified by the relevant standards bodies and/or developer teams. It then provides details of known pairwise interoperability issues between programming models, and identifies for each case any areas which INTERTWinE is capable of addressing.
This report contains the current state of API recommendations presented to the standard bodies. These recommendations, concerning interoperability, extension proposals and their implementations, have been collected from all of the technical work packages of the INTERTWinE project. The report describes the prioritised API combinations and co-design approach, reports on discussions held with the standard bodies, provides details on the effect on the APIs of the prioritised API combinations, and lists the recommendations presented to the standard bodies.
- D3.3 Final report on API recommendations, extension proposals and their implementation (Please note that this version of Deliverable D3.3 has been submitted to the EC, but has still to be approved by the project reviewers. The final version will be made available here at a later date).
D3.3 describes changes in standard and the implementations necessary to support the interoperability between programming models and task-based runtime systems. This description is formulated from the perspective of the respective standard bodies and implementation owners of the programming models and task-based runtimes. The main results are: Interoperability between GASPI and flat MPI code; Interoperability between GASPI and flat MPI code; Resource Manager; Communication Interoperability libraries between OmpSs and MPI/GASPI; Interoperability of MPI with threads and tasks; Interoperability between PaRSEC and OpenMP; Interface to the Directory/Cache service.
Deliverables for WP4: Resource Management and Runtime Interoperability
Within the context of INTERTWinE, the Resource Manager will play a key role in enabling interoperability between the APIs and libraries involved in the project and beyond, by ensuring coordinated usage of hardware resources. This document presents a survey of the current resource usage patterns of the project runtime systems and libraries. It presents some relevant use cases, details potential issues that may arise from such use cases, and lists requirements from the Resource Manager to address or mitigate these issues.
One of the tasks defined in the INTERTWinE project is the definition of a common, generic API for a Directory/Cache service for task-based runtime systems (such as OmpSs, StarPU and PaRSEC). The main goal is to achieve a programming model that hides most of the complexities associated with distributed systems, but also enables an efficient utilisation of remote compute and memory resources. The present document proposes an API for a Directory/Cache (a kind of virtual memory manager) that is sufficiently abstract and generic, promoting one-sided, asynchronous communication and providing automatic caching.
This deliverable defines the Resource Manager component being developed in INTERTWinE. The Resource Manager aims to coordinate resource allocation (also known as ownership), and to establish the means to dynamically share resources among multiple task-based runtime systems and additional support software which may execute within the same process. The main goal the Resource Manager pursues is to improve the composability of parallel libraries, the integration of task-based runtime systems and communication APIs, and to avoid negative performance effects of inappropriate use of the underlying resources by eliminating both oversubscription and undersubscription cases.
- D4.4 Report on runtime systems implementation of resource management API (Please note that this version of Deliverable D4.4 has been submitted to the EC, but has still to be approved by the project reviewers. The final version will be made available here at a later date.)
This deliverable describes the design, implementation and usage of the Resource Manager (RM) initially presented in D4.3 “Definition of resource manager API and report on reference implementation”. The RM is composed of four different APIs aimed to dynamically share CPU resources between different runtime systems running inside the same application, avoiding both over-subscription and under-subscription issues. This deliverable describes the overall RM and each of the four APIs. The RM can be used not only to coordinate the access to CPU cores from different runtimes inside the same application, but it can also be leveraged by library developers to build parallel libraries that can be concurrently used with other parallel libraries and applications efficiently.
- D4.5 Report on runtime system implementation on top of Directory/Cache API (Please note that this version of Deliverable D4.5 has been submitted to the EC, but has still to be approved by the project reviewers. The final version will be made available here at a later date.)
INTERTWinE aims to define a common, generic API for a Directory/Cache service for task-based runtime systems (such as OmpSs, GPI-Space, StarPU, and PaRSEC). The main goal is to achieve a programming model that hides most of the complexities associated with distributed systems, but also enables an efficient utilization of remote compute and memory resources. This deliverable reports on the experiments performed with a prototype implementation of the Directory/Cache and its integration with the above mentioned task-based runtime systems. We also report experiments with some example applications running on top of the runtime systems that are integrating the Directory/Cache.
- D4.6 Recommendations and roadmap for Exascale runtimes (Please note that this version of Deliverable D4.5 has been submitted to the EC, but has still to be approved by the project reviewers. The final version will be made available here at a later date.)
This deliverable summarises the work done with the Resource Manager (RM), the Directory/Cache (D/C) and the Task-Aware MPI (TAMPI) library. It provides a description and evaluation of the Resource Manager APIs, which have been implemented in the StarPU, OmpSs-2 and Intel OpenMP runtimes. It also explains and evaluates the Directory/Cache APIs and their integration with the different runtime systems.
Deliverables for WP5: Applications and Kernels
This document presents a first plan of the applications/kernels in combination with their initial evaluation and benchmark suite releases for use by WP3 and WP4. These initial releases of applications/kernels will be key to evaluating new or enhanced interoperability features that are made available in the respective runtime implementations. For each application/kernel, the document gives a brief introduction, describes the current state for the use case from the point of view of API combination, confirms the ambition plans with respect to API combination within the scope of INTERTWinE, and reviews the benchmarks and evaluation plans. There is also a summary of software (benchmark) release plans, focusing on the API combinations.
This document presents a list of API combinations prioritised by the project, provides highlights of the co-design aspects, and outlines the findings and lessons learnt from the API combination development on the partners’ applications/kernels: Ludwig, iPIC3D, TAU, BAR, Graph-BLAS and PLASMA. For each one of these, the document: 1) outlines a given application/kernel; 2) justifies the API combinations choice; 3) summarises the key implementation details, clearly stating the parts of applications being modified; 4) describes the preliminary performance results for the use case from the point of view of API combination; 5) provides analysis of the performance results with respect to the evaluation criteria; 6) discusses interoperability opportunities/issues, further improvements, and collaboration with the other technical work packages.
- D5.3 Performance evaluation report (Please note that this version of Deliverable D5.3 has been submitted to the EC, but has still to be approved by the project reviewers. The final version will be made available here at a later date.)
In this deliverable, we put special emphasis on performance evaluation, as performance is the critical criterion to determine readiness of APIs, runtimes, and applications for the upcoming Exascale hardware. However, performance is only the tip of the iceberg. Thus, we dive deeper, providing analysis of the obtained results using the following previously established evaluation criteria: performance, correctness of the codes, suitability for exascale, usability, and portability (see Deliverable D5.1 Initial application / kernel plans, evaluations and benchmark suite releases).
A version of this deliverable with higher-resolution images is available - please contact info [at] intertwine-project.eu.
- D5.4 Final report on application/ kernel plans, evaluations and benchmark suite releases (Please note that this version of Deliverable D5.4 has been submitted to the EC, but has still to be approved by the project reviewers. The final version will be made available here at a later date.)
This final deliverable in WP5 is focused on studying various API combinations in applications/ kernels and focuses on applying developments in WP3 on APIs like GASPI-SHAN and WP4 on the Resource Manager and the TAMPI library to tackle interoperability issues. Following the prioritised APIs combinations, we have grouped our efforts accordingly and collated the findings, lessons learnt, and reflections/ analysis into a set of INTERTWinE Resource Packs.
A version of this deliverable with higher-resolution images is available - please contact info [at] intertwine-project.eu.