1 of 1

RISE: Reducing I/O Contention in Staging-based Extreme-Scale In-situ Workflows

Scientific Achievement

Enables machine learning guided data offloading for extreme-scale in-situ workflows to significantly reduce data write access costs.

Significance and Impact

RISE quickly captures the data access interval and schedules background drain task, which reduces the write-response time by ∼44% as compared to Naive-Drain method and by ∼31% as compared to DataSpaces.

Research Details

In contrast to existing solutions that exclusively use memory either on simulation nodes or on the staging nodes to write data, RISE offers write latency like solutions that use direct data movement but can also use the staging nodes to offload data.
RISE models the interactions across multiple applications coupled via the staging framework, predicts periods when the server is likely to be idle and determines suitable data objects to drained to the staging node.
Although data draining happens between writers and server, all I/O requests coming to staging servers are analyzed to predict when the network is likely to be idle.

RISE leverages ML techniques to capture the data access patterns then uses this knowledge to efficiently drain the data from local memory to the staging memory

Subedi, P., Davis, P., & Parashar, M. In IEEE International Conference on Cluster Computing (CLUSTER'21). IEEE Press, Piscataway, NJ, USA

Data-staging frameworks have emerged as effective solutions for addressing data-related challenges at extreme scale and supporting in-situ/in-transit workflows. However, these DRAM based data-staging frameworks either store data purely on the client memory to provide immediate access to the data or on the staging server memory to utilize memory available on the staging servers. Efficient data movement and management in these frameworks remains a challenge. At Utah, we explored RISE, which bridges the gap between DataSpaces-like framework and DIMES-like frameworks to efficiently utilize the memory available on both client and staging server memory. Specifically, RISE makes the data immediately available to the reader/analysis applications by storing it locally and uses machine learning techniques to identify when the network is expected to be idle. During this expected idle period, RISE drains the locally stored data objects to the staging servers to minimize the interference between foreground read/write requests and background data drain. We implemented RISE within the DataSpaces staging framework and deployed it on Summit supercomputer at OLCF. Our experimental results using both synthetic and real-world large scale S3D application demonstrate that RISE can take advantage of staging nodes to offload the data without degrading the application's read and write requests being served by the staging server.