Abstract
The Coherent Accelerator Processor Interface (CAPI) is a general term for the infrastructure that provides high throughput and low latency path to the flash storage connected to the IBM POWER 8+ System. CAPI accelerator card is attached coherently as a peer to the Power8+ processor. This removes the overhead and complexity of the IO subsystem and allows the accelerator to operate as part of an application. In this paper, we present the results of experiments on IBM FlashSystem900 (FS900) with CAPI accelerator card using the “CAPIFlash - IBM Data Engine for NoSQL Software” Library. This library provides the application, a direct access to the underlying flash storage through user space APIs, to manage and access the data in flash. This offloads kernel IO driver functionality to dedicated CAPI FPGA accelerator hardware. We conducted experiments to analyze the performance of FS900 with CAPI accelerator card, using the Key Value Layer APIs, employing NASA’s MODIS Land Surface Reflectance dataset as a large dataset use case. We performed Read and Write operations on datasets of size ranging from 1MB to 3TB by varying the number of threads. We then compared this performance with other heterogeneous storage and memory devices such as NVM, SSD and RAM, without using the CAPI Accelerator in synchronous and asynchronous file IO modes of operations. The asynchronous mode had the best performance on all the memory devices that we used for this study. In particular, the results indicate that FS900 & CAPI, together with the metadata cache in RAM, delivers the highest IO/s and OP/s for read operations. This was higher than just using RAM, along with utilizing lesser CPU resources. Among FS900, SSD and NVM, FS900 had the highest write IO/s. Another important observation is that, when the size of the input dataset exceeds the capacity of RAM, and when the data access is non-uniform and sparse, FS900 with CAPI would be a cost-effective alternative.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Stuecheli, J., et al.: CAPI: a coherent accelerator processor interface. IBM J. Res. Dev. 59(1), 1–7 (2015)
Adrian, M.: Big data. Teradata Magazine. http://www.teradatamagazine.com/v11n01/Features/Big-Data/
Jun, H., et al.: HBM (high bandwidth memory) DRAM technology and architecture. In: 2017 IEEE International Memory Workshop (IMW), pp. 1–4 (2017)
IBM Data Engine for NoSQL Software Libraries Source Code. https://github.com/open-power/capiflash
Solution Reference Guide to IBM Data Engine for NoSQL - Integrated Flsh Edition. http://ibm.biz/capiflash
Starke, W.J., et al.: The cache and memory subsystems of the IBM POWER8 processor. IBM J. Res. Dev. 59(1), 1–3 (2015)
IBM Coherent Accelerator Processor Interface User’s Manual. http://www.nallatech.com/wp-content/uploads/IBM_CAPI_Users_Guide.pdf
IBM CAPI Flash. https://developer.ibm.com/linuxonpower/capi/
Alexandre, et al.: IBM Power Systems S822LC Technical Overview and Introduction. IBM Red Books
Linux Coherent Accelerator driver cxlflash.txt manual. https://www.kernel.org/doc/Documentation/powerpc/cxlflash.txt
Gilge, M.: Redpaper Flash or SSD: Why and When to Use IBM FlashSystem Overview (2013)
IBM Power Systems Managing PCIe adapter for the S822LC and I/O expansion drawer Manual. ftp://ftp.software.ibm.com/systems/power/docs/hw/p8/p8hcd_85x.pdf
MODIS Surface Reflectance User Guide. http://modis-sr.ltdri.org/guide/MOD09_UserGuide_v1.4.pdf
Sendir, B., et al.: Optimized durable commitlog for Apache Cassandra using CAPI-Flash. In: 2016 IEEE 9th International Conference on Cloud Computing (CLOUD). IEEE (2016)
Acknowledgement
We would like to thank Mike Vageline of IBM Cognitive Systems and Software Development for his support on the CAPIFlash - The IBM Data Engine for NoSQL library. We wish to acknowledge the NASA GSFC Distribution Active Archive Center (DAAC) for providing the MODIS Surface Reflectance (MOD09) data acquired from the Level-1 and Atmospheric Archive and Distribution System (LAADS) used for this study. We wish to thank Dale Pearson of IBM Yorktown Heights for providing this unique Power 8+ configuration with the FlashSystem 900. Finally, we wish to acknowledge the NSF Center for Accelerated Real-Time Analytics (NFS Award Number 1747724) and its industrial members for providing the resources to carry out this study.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Velusamy, K., Prathapan, S., Halem, M. (2019). Exploring the Behavior of Coherent Accelerator Processor Interface (CAPI) on IBM Power8+ Architecture and FlashSystem 900. In: Weiland, M., Juckeland, G., Alam, S., Jagode, H. (eds) High Performance Computing. ISC High Performance 2019. Lecture Notes in Computer Science(), vol 11887. Springer, Cham. https://doi.org/10.1007/978-3-030-34356-9_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-34356-9_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34355-2
Online ISBN: 978-3-030-34356-9
eBook Packages: Computer ScienceComputer Science (R0)