Amazon Research Proposal - SaaS Clouds Supporting Non Computing Specialists Executing HPC Applications

From Bioinformatics

Jump to: navigation, search

The following application was submitted to Amazon on June 17th, 2014 for a cloud research grant.

[edit] SaaS Clouds Supporting Non Computing Specialists Executing HPC Applications

Philip C. Church and Andrzej Goscinski

From the viewpoint of biology specialists, the difficulty of using clouds for HPC is a major contributor to the slow adoption of cloud technologies in genomics research. To utilize the cloud for HPC, a user must be aware of the procedures to setup and administrate a cluster. As IaaS clouds resources are provided at the server level, they must be modified to support distributed applications; middleware installed on each virtual server and distributed software deployed. These administrative tasks are time consuming and out of the scope of most researchers. Popular frameworks such as Galaxy (via CloudMan) can simplify genomic analysis on the cloud, however this methodology is based on mirroring Galaxy servers and is thus inefficient. Our goal is to apply the Galaxy approach directly to Amazon EC2 to support the exposure of individual tools as Amazon cloud services. We have carried out initial projects to demonstrate the feasibility of this project including: a dynamic molecular simulation (VMD) software plugin which provides an integrated framework for NAMD to be executed on Amazon EC2 [1]; a method to expose mpiBLAST as a service within a private cloud, HPCynergy [2]; and new approaches and frameworks for deploying and exposing HPC applications in clouds [3]. We wish to extend our work developing technology to support non-computing researchers access the cloud by addressing the following significant research challenges:

  • Task 1: The lack of automation of HPC Application Deployment

It is necessary to determine how to utilize and build higher layer abstraction and graphical interfaces to best provide access to deployed application services, and what data structures should be used to store and host these applications for future use. We propose development of a HPC Cloud Service Stack to support research which will make the configuration and utilization of HPC cloud resources transparent to researchers.

  • Task-2: Inadequate Resource Selection and Workflow Generation approaches

Current work has explored the use of limited attributes (such as CPU, RAM, and Reliability) to select resources that best fit the user’s service. Further research should extend available metrics and widen the pool of resources available for selection to include different cloud providers and non-cloud resources (clusters, servers etc). This task will be achieved by the development of novel resource selection algorithms supporting clouds.

  • Task-3: The lack of automation of HPC Application Service and Web-form Generation

There is a need to devise methods which enable researchers to take the role of cloud developer. Current work has explored the automatic generation of web interfaces using XML based languages. While this has reduced the work required to expose software it doesn’t support users deploying software. We instead propose the use of natural language parsing to automate the creation of services. To accomplish our proposed goals we will need to allocate heterogeneous virtual clusters on Amazon EC2 resources and automate the deployment of software as services on these provisioned resources. As such, in order to develop and validate our solutions we will require, $49141.14 dollars of AWS services credit over two years which will cover compute and storage resources.

[edit] References

[1] A. Wong and A. Goscinski, "A VMD Plugin for NAMD Simulations on Amazon EC2," Procedia Computer Science, vol. 9, pp. 136-145, 2012.

[2] M. Brock and A. Goscinski, "Execution of Compute Intensive Applications on Hybrid Clouds (Case Study with mpiBLAST)," Proceedings of the 2012 Sixth International Conference on Complex, Intelligent, and Software Intensive Systems (CISIS), 2012.

[3] A. Goscinski, P. Church, A. Wong and Z. Tari, "SaaS Clouds Supporting Biology and Medicine," in Cloud Computing with E-science Applications, L. M. a. O. Terzo, Ed., ed: CRC Press/Taylor & Francis,2014.

[edit] Projects

  • Uncinus
Personal tools