Presentation
Kraken Mare – HPE’s Exascale Monitoring Framework Prototype
SessionProject Poster Session
Event Type
Project Poster
Pre-Recorded
TimeTuesday, June 23rd4:18pm - 4:20pm
LocationAnalog 2
DescriptionAn Exascale system will cost 100s of millions of Euros and consist of millions of components. It resembles a complex system which requires the use of advanced data analytic techniques for management and system optimization. These techniques require a much higher resolution and coverage of sensor data then current monitoring solutions can provide. This project poster describes the design and implementation of a monitoring framework, called Kraken Mare, that can collect all required data to enable the use of advanced data analytics (such as machine learning and artificial intelligence) for system and data center optimization.
Kraken Mare is being designed from the ground up to not only collect unprecedented volumes of data at scale but also to provide functionality that directly supports data usage by including data quality and correctness indicators embedded in the sensor meta data. The design fills the gap between a pure IoT model and a classical message bus. Based on Exascale system sizes, we have developed a component and sensor estimation leading to a design target of over 10 Million data messages per second.
Kraken Mare is designed to be a vertical and horizontally scalable framework that can be used for any type of sensor data collection at scale. Kraken Mare is composed of open source big data technologies implemented as micro-services. The uniqueness comes from the communication functionality, the data descriptions designed into the framework, and the framework manager.
Kraken Mare is being designed from the ground up to not only collect unprecedented volumes of data at scale but also to provide functionality that directly supports data usage by including data quality and correctness indicators embedded in the sensor meta data. The design fills the gap between a pure IoT model and a classical message bus. Based on Exascale system sizes, we have developed a component and sensor estimation leading to a design target of over 10 Million data messages per second.
Kraken Mare is designed to be a vertical and horizontally scalable framework that can be used for any type of sensor data collection at scale. Kraken Mare is composed of open source big data technologies implemented as micro-services. The uniqueness comes from the communication functionality, the data descriptions designed into the framework, and the framework manager.
Poster PDF
