|
Article information
2019 , Volume 24, ¹ 6, p.5-12
Baranov A.V., Kiselev E.A.
HPC cloud services based on the Proxmox VE platform
Purpose. The purpose of this paper is to develop methods and technologies for building high-performance computing cloud services in scientific supercomputer centers. Methodology. To build a cloud environment for high-performance scientific calculations (HPC), the corresponding three-level model and the method of combining flows of supercomputer tasks of various types were applied. Results. A high-level HPC cloud services technology based on the free Proxmox VE software platform has been developed. The Proxmox VE platform has been integrated with the domestic supercomputer job management system called SUPPZ. Experimental estimates of the overheads introduced in the high-performance computing process by the Proxmox components are obtained. Findings. An approach to the integration a supercomputer job management system and a virtualization platform is proposed. The presented approach is based on the representation of the supercomputer jobs as virtual machines or containers. Using the Proxmox VE platform as an example, the influence of a virtual environment on the execution time of parallel programs is investigated experimentally. The possibility of applying the proposed approach to building cloud services of the PaaS and SaaS type in scientific supercomputing centers of collective use is substantiated for a class of applications for which the overhead costs introduced by the Proxmox components are acceptable.
[full text] [link to elibrary.ru]
Keywords: supercomputer, HPC, cloud service, job management system, scheduling supercomputer jobs, Proxmox
doi: 10.25743/ICT.2019.24.6.002.
Author(s):
Baranov Anton Viktorovich
PhD. , Associate Professor
Position: Leading research officer
Office: Joint Supercomputer Center of the Russian Academy of Sciences
Address: 119334, Russia, Moscow, 32a, Leninsky Prospect
Phone Office: (495) 954-02-21
E-mail: antbar@mail.ru
SPIN-code: 3595-7482Kiselev Evgeny Andreevich
PhD.
Position: Research Scientist
Office: Joint Supercomputer Center of the Russian Academy of Sciences
Address: 119334, Russia, Moscow, 32a, Leninsky Prospect
Phone Office: (495) 954-02-21
E-mail: kiselev@jscc.ru
SPIN-code: 5674-0020 References:
[1] Tadviser. Govenrment. Business. IT. Cloud computing. Available at: http://tadviser.com/index.php/Cloud_Computing (acessed 21.11.2019). (In Russ.)
[2] Tadviser. Government. Business. IT. Cloud services in the Russian market, 2019. Available at: http://www.tadviser.ru/index.php/Ñòàòüÿ:Îáëà÷íûå_ñåðâèñû_%28ðûíîê_Ðîññèè%29 (accessed 21.11.2019). (In Russ.)
[3] Reuther, A., Byun, C., Arcand, W., Bestor, D., Bergeron, B., Hubbell, M., Jones, M., Michaleas, P., Prout, A., Rosa, A., Kepner, J. Scalable system scheduling for HPC and big data. Journal of Parallel and Distributed Computing. 2018; (111):76–92. DOI: 10.1016/j.jpdc.2017.06.009
[4] Kudryavtsev, A.O., Koshelev, V.K., Izbyshev, A.O., Avetisyan, A.I. HPC as a Cloud Îáëà÷íûå HPC-ñåðâèñû íà áàçå Proxmox 7. Service: the Key Problems. Proc. of the Intern. Conf. “Parallel computational technologies (PCT)2013”. Chelyabinsk; 2013:432–438. (In Russ.)
[5] Aladyshev, O.S., Baranov, A.V., Ovsyannikov, A.P., Balayan, G.A., Sinitsin, V.S. Methods and tools for combining job streams from cloud platforms and managing of supercomputer resources. Software Journal: Theory and Applications. 2018. No. 4. P. 54–63. DOI: 10.15827/2311-6749.29.337. (In Russ.)
[6] Proxmox Main Page. Available at: https://pve.proxmox.com/wiki/Main_Page (accessed 21.11.2019)
[7] Parallel jobs management system (SUPPZ). Programmer (user) guide. Available at: http://www.jscc.ru/wp-content/uploads/2017/06/SUPPZ-user-guide-2016.pdf (accessed 03.10.2019). (In Russ.)
[8] Slurm Containers Guide, 2019. Available at: https://slurm.schedmd.com/containers.html (accessed 03.10.2019)
[9] Baranov, A.V., Nikolaev, D.S. The use of container virtualization in the organization of high-performance computing. Program Systems: Theory and Applications. 2016; 1(28):117–134. DOI: 10.25209/2079-3316-2016-7-1-117-134. (In Russ.)
[10] Baranov, A.V., Savin, G.I., Shabanov, B.M., Shitik, A.S., Svadkovskiy, I.A., Telegin, P.N. Methods of Jobs Containerization for Supercomputer Workload Managers. Lobachevskii Journal of Mathematics. 2019; 40(5):52–534. DOI: 10.1134/S1995080219050020
[11] MVS-100K supercomputer. Available at: http://www.jscc.ru/resources/hpc/#item88 (accessed 03.10.2019). (In Russ)
[12] Shabanov, B.M., Ovsyannikov, A.P., Baranov, A.V., Kiselev, E.A., Leshchev, S.A., Dolgov, B.V., Gumennyy, D.G., Shurov, D.L. Solving problems for photorealistic computer graphics based on the protected infrastructure of a supercomputer collective sharing center. Proc. of the Intern. Conf. “Russian Supercomputing Days”. Moscow: MGU; 2017:733–741. (In Russ.)
[13] Anikeev, F.A., Baranov, A.V., Zaytsev F.S., Kiselev E.A., Leshchev S.A. The virtual
educational laboratory based on the Proxmox VE virtualization platform. Trudy nauchno- issledovatel'skogo instituta sistemnykh issledovaniy Rossiyskoy akademii nauk. 2018; 8(6):103–111. DOI: 10.25682/NIISI.2018.6.0014. (In Russ.)
Bibliography link:
Baranov A.V., Kiselev E.A. HPC cloud services based on the Proxmox VE platform // Computational technologies. 2019. V. 24. ¹ 6. P. 5-12
|
