A Seminar on Implementation Of

Exploiting Dynamic Resource Allocation for Efficient Parallel Data Processing in the Cloud
Presented By: Hussain khan Md M.Tech(S.E) IIyear 10311D2506

Under The Guidance of

Mrs. Aruna Varanasi, H.O.D of C.S.E Dept, Snist, Ghatkesar

Description:
In recent years ad-hoc parallel data processing has emerged to be one of the killer applications for Infrastructure-as-a-Service (IaaS) clouds. Major Cloud computing companies have started to integrate frameworks for parallel data processing in their product portfolio, making it easy for customers to access these services and to deploy their programs.

However, the processing frameworks which are currently used have been designed for static, homogeneous cluster setups and disregard the particular nature of a cloud. Consequently, the allocated compute resources may be inadequate for big parts of the submitted job and unnecessarily increase processing time and cost.

 In this paper we discuss the opportunities and challenges for efficient parallel data processing in clouds and present our research project named NEPHELE. It is the first data processing framework to explicitly exploit the dynamic resource allocation offered by todays IaaS clouds for both, task scheduling and execution.

Particular tasks of a processing job can be assigned to different types of virtual machines which are automatically instantiated and terminated during the job execution.
Based on this new framework, we perform extended evaluations of Map Reduce-inspired processing jobs on an IaaS cloud system and compare the results to the popular data processing framework Hadoop.

TECHNIQUE USED
Nephele framework:
The actual execution of tasks which a Nephele job consists of is carried out by a set of instances. Each instance runs a so-called Task Manager (TM). A Task Manager receives one or more tasks from the Job Manager at a time, executes them. After that informs the Job Manager about their completion or possible errors.

Unless a job is submitted to the Job Manager, we expect the set of instances (and hence the set of Task Managers) to be empty.
Upon job reception the Job Manager then decides, depending on the jobs particular tasks, how many and what type of instances the job should be executed on, and when the respective instances must be allocated/ deallocated to particular instance It ensures a continuous but cost-efficient processing. Our current strategies for these decisions are highlighted at the end of this section.

SYSTEM ARCHITECTURE
Description: Here the system architecture depicts that Through client page we can give HTTP request to the servlet and retrieve resources from persistent storage by data access object. From servlet job manager will get the HTTP response and handover the task to the task manager.

Task manager will process the task and again forward the response to the client page

DATA FLOW DIAGRAM

Description:
From the data flow diagram it clearly shows that: User can connect with the cloud after checking the credentials through login. Through the main page client can give request through main page then this job will redirect to job manager for scheduling and allocating. Then proper task manager will perform particular task by getting the necessary resources from the persistent storages

Finally the result is being sent to the client

Design Work
List of Modules in the project Login Client request Job manager Resource storage Task manager Process the task

Implementation Work Home Page

1)Login Page

Registration Page
A new user registers details and gets authentication

Upon Successful Registration a message would be displayed

Then the User Logins with so created mail Id and Password

Module: 2)Client Request

The client provides his request to job Manager by entering the data in the empty space provided

Sample Request created by the client

After creating the Request, by clicking upon the Submit button, the request would be sent directly to the Job Manager

Code for creating Client Request

protected void btnSubmit_Click(object sender, EventArgs e) {

string Name = Session["username"].ToString(); //ObjClientBO.Request = txtClient.Text; try { SqlConnection con = new SqlConnection(constring); con.Open(); SqlCommand cmd = new SqlCommand("Insert into tblClientRequest(Name,Request)values('" + Name + "','" + txtClient.Text + "')", con); SqlDataAdapter da = new SqlDataAdapter(cmd); cmd.ExecuteNonQuery(); ScriptManager.RegisterStartupScript(Panel1,Panel1.GetType(), "MyScript", "alert('Your Request has been sent successfully');", true); txtClient.Text = ""; } catch (Exception ex) { ex.Message.ToString(); } clear();

Module : 3) Job Manager

Job Manager logins to Cloud controller to view the client requests and process the requests.

A window opens displaying the View Request Tab, to view the requests sent by the client

Then the request sent by different would be displayed.

Then the job manager checks for the resources needed to process the tasks

A window opens displaying the services available by the job manager

If the resources are available, then the Job Manager selects the particular resources depending upon the request sent by the clients.

 Code for Job manager

public partial class JobManager : System.Web.UI.Page {
string constring = @"Data Source=HOME-PC;Initial Catalog=CloudComputing;Persist Security Info=True;User ID=sa;Password=srinivas"; CloudBO.JobManager ObjJobBO = new CloudBO.JobManager(); int id = 0; protected void Page_Load(object sender, EventArgs e) { if (!IsPostBack) { data(); plat(); } } protected void btnViewRequest_Click(object sender, EventArgs e) { Panel4.Visible = true; Panel1.Visible = false;

protected void btnAvailableResourse_Click(object sender, EventArgs e) { Panel1.Visible = true; Panel4.Visible = false; lblmessage.Visible = true; lblmessage.Text = "IF RESOURCE IS UNAVAILABLE PLEASE REGISTER IN TO CLOUD REGISTRATION"; lblmess.Visible = true; lblmess.Text = "IF RESOURCE IS UNAVAILABLE PLEASE REGISTER IN TO CLOUD REGISTRATION"; } protected void btnsoftware_Click(object sender, EventArgs e) { ddlSW.Visible = true; btnCloudRegistration.Visible = true; }

protected void btnPlatform_Click(object sender, EventArgs e) { ddlPF.Visible = true; btnplatCloudReg.Visible = true; } protected void btnSubmit_Click(object sender, EventArgs e) { // ObjJobBO.DdlSoftware = ddlSW.SelectedItem.Text; // ObjJobBO.DdlPlatform = ddlPF.SelectedItem.Text; // ObjJObBAL.InsertJobManager(ObjJobBO); try { id = Convert.ToInt32(lblrequestid.Text); SqlConnection con = new SqlConnection(constring); con.Open(); SqlCommand cmd = new SqlCommand("Insert into tblJobManager(ddlSoftware,ddlPlatform,RequestID,Request,Name) values('" + ddlSW.SelectedItem.Text + "','" + ddlPF.SelectedItem.Text + "'," + id + ",'" + lblrequestmesg.Text + "','" + lblrequestname.Text + "')", con); cmd.ExecuteNonQuery(); }

Module : 4) Resource storage

If the resources are unavailable, then the Job Manager gets additional resources from the Cloud Persistent storage by providing the authenticated details for accessing the storage

The job Manager provides his authenticated details for accessing the storage of the Cloud

Upon successful registration a message would be displayed

A window would be displayed for searching the needed resources

Then the Job Manager enters his needed resources and searches for availability

Job manager selects all the resources depending upon the necessity

Upon the successful selection the resource would be added to the existence resources of the job manager

Then the assigned resources for particular request would be displayed

Module : 5) Task Managers

Depending upon the number of request sent by a particular client the Job Manager selects the tasks and assigns the tasks to task managers.

Code for Task Managers

public partial class AssignedTask : System.Web.UI.Page { string constring = ConfigurationManager.ConnectionStrings["PharmaConnection"].ToStrin g(); //string constring = @"Data Source=HOME-PC;Initial Catalog=CloudComputing;Persist Security Info=True;User ID=sa;Password=srinivas"; protected void Page_Load(object sender, EventArgs e) { } protected void ddljob_SelectedIndexChanged(object sender, EventArgs e) { if (ddljob.SelectedItem.Text == "--select--") { btnTask1.Visible = false; btnTask2.Visible = false; Button1.Visible = false; }

Module: 6) Processing the Tasks By clicking upon the individual task managers three windows would be opened parallel in the browser. Task 1: To create a Student Registration page for client as first request .

Task2 : To create a Student Profile form for client as second request. Task3: To display the so entered details by using database

Data Base Tables

Sample Data base For Job Manager

ADVANTAGES Reduce the workload on cloud. Fast processing. Easy to maintain and troubleshoot. Easy to design as the infrastructure we need.

Thank You