The Wiki's way of Deployment, Access and Customization of Computationally Demanding Applications in Clouds

From Bioinformatics

Jump to: navigation, search

[edit] Related Publications

Adam Wong and Andrzej Goscinski, A VMD Plugin for NAMD Simulations on Amazon EC2, accepted for presentation and publication at the International Conference on Computational Science, ICCS 2012.


[edit] Aim

The aim of this project is to carry out the study into the development of a technology for simplifying the deployment, access and customization of computationally demanding applications (which would be exposed as high level user services) in cloud environments. This technology forms a basis of research environments enabling specialists of disciplines such as biology, medicine, physics and engineering to use High Performance Computing facilities easily, on-demand and at reasonable costs for the discovery of new and significant knowledge from massive data sets.

[edit] Abstract

High Performance Computing (HPC) has helped to discover new knowledge and add to existing knowledge in many scientific and engineering disciplines by providing computer facilities that perform large and complex simulations, database searches, and system modelling within reasonable time frames. However, HPC requires powerful and expensive computational and data storage hardware, advanced middleware, and sophisticated discipline oriented applications. A response to the above problem faced by discipline specialists lies in cloud computing Recent years, some public cloud vendors including the Amazon’s Elastic Compute Cloud (EC2) have provided solutions specifically designed for running HPC applications. However, while cloud computing alleviates the costs of procuring required IT resources, the cost and time overheads in learning how to prepare a HPC cloud and then the applications to be run in it remains a problem.

Currently, most HPC clouds are based on the IaaS/PaaS clouds enhanced by additional hardware and middleware to support HPC. In general, HPC cloud users are presented with a set of virtual servers (with a stock install of an operating system) and are then required to put the servers together to form the HPC facilities they need to run their software applications. The software applications must be properly installed and configured in the underlying HPC facilities. Thus, if discipline specialists want to use HPC clouds for scientific discovery, they must also become the system administrators and computer specialists performing time consuming resource management and software configuration activities that slow down the process of discovery.

The three major problems discipline specialists have to deal with when using HPC clouds are:

1. Complex Application Deployment: Deploying an application to an IaaS or PaaS cloud requires installing and then configuring within a (virtual) server. This is usually a non-trivial task even for a computer systems specialist as he/she must be knowledgeable in handling many complicated issues such as hardware compatibility, library dependence, optimization and error handling.

2. Non-trivial Access: IaaS and PaaS clouds are not user friendly environments. For accessing every software application, the user is responsible for launching and terminating his/her required HPC resources. Even worst, the software application and its execution process is presented as primitive command-line sequences that are difficult to use, especially by those discipline scientists who have little computing skills.

3. Lack of Support for Customization: For each unique scientific application, its deployment and accessing in a HPC cloud involve a great amount of human effort and knowledge as explained in Item 1 and Item 2 above. Successful outcomes should be preserved as these valuable users’ experiences could be shared and then be enhanced by other researchers if they wish to work on the same scientific application. Currently, the issues of enhancement and customization of already developed HPC applications for clouds have been ignored.

To our best knowledge, currently there is very limited work related to HPC on SaaS cloud available out there on the market or even in the research domain for discipline specialists to use. We are convinced there is a need for HPC on SaaS clouds. The destination outcome of the proposed research is threefold: (i) the approach to automatically deploying computationally demanding applications in Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) clouds; (ii) the approach to exposing HPC applications as services; (iii) the approach to creating Software as a Service (SaaS) Library of discipline-oriented services, which can be invoked through user friendly discipline specific interfaces. Thus the proposed cloud technology will support the discipline specialists executing their computationally demanding applications easily and on demand.

Personal tools