Distributed System Examples

Introduction to key topics in Distributed Computing

NJIT
1
7/16/2013

Warning

This is a difficult course, because it requires you to learn how to configure heterogeneous languages and environments. You will work in teams to solve problems. The instructor cannot provide individual technical support. All technical support will be on Moodle, with students answering each others questions. You should be familiar with Java, and have some knowledge of C or C++.
7/16/2013

Distributed Systems Technology

7/16/2013

The Fourth Wave

Starting about 2000, there has been a move toward parallel computing using inexpensive blade servers, often using Linux. The next slide give an example from a company named Cassat formed by specialists from the Cray supercomputer company. Note that the example gives 7 times the performance for one tenth the cost!
7/16/2013

The Fourth Wave

Cassatts program, Collage sits atop a pool of servers and shifts processing power on the fly Pharmaceutical giant Pfizer became a Cassat customer after a demo in which Collage enabled a $300,000 cluster of IBM blade servers, running 24 ordinary Intel microprocessors, to crunch through an Oracle data warehousing job in 45 minutes. Previously Pfizer had run the job on a $3 million Sun machine. It took five and a half hours. 1 1Forbes magazine, September 18, 2006, page 106.
7/16/2013

Subsequent Developments

Key developments since 1998 include the Further development of the Internet, including portable Internet technologies such as WiFi and Bluetooth, as well as the growth of Peer-to-Peer systems starting with the great popularity of Napster.

7/16/2013

Not about technology

The Distributed Systems marketplace was driven by technology until 2000. That is no longer true. Now it is driven by business value. Knowing the latest and greatest technology will no longer get you a job. Now you have to deliver increased profit and/or lower cost to a company.
7/16/2013

Distributed Computing Strategic Factors

End-User Application Developer System Administrator
Personalized Environment Predictable Response Location Independence Platform Independence Flexibility Increased Code Complexity Reusability Real-Time Access Lack of Interoperability to information Scalability Management Portability Faster Development Tools Reduced And deployment of Changing Complexity Business Solutions Technology

Organization 8
7/16/2013

Interview with Carly Fiorina, CEO of Hewlett Packard

Information technology has become so central to business that the tradeoffs are bigger than they used to be, so technology has to yield to business discipline. Its about being able to deliver multiple things: lower total cost of ownership, improved quality, risk mitigation and agility.2
August 11, 2003, page 78
7/16/2013

2Forbes,

Strategic Use of IT

On the morning of September 12, 2001, Wal-Mart Stores computers picked up an increase in Flag Sales after the terror attack and immediately bought most of the nations available inventory of flags, leaving competitors in the dust.3
ASAP, October 7, 2002, page 15
7/16/2013

3Forbes

10

Distributed Computing Reference Model

Management And Support Interoperability Portability Integration
Enterprise Systems: Perform enterprise activities Application Systems: Support enterprise systems Distributed Computing platform Application Support Services (OS, DB support, Directories, RPC) Communication Network Services (Network protocols, Physical devices) Hardware

11

Network Management

7/16/2013

Expanded Model
Management & Support
Enterprise Systems: .Engineering Systems .Manufacturing .Business Systems .Office Systems
Application Systems: User Processing Interfaces Programs

Data files & Databases

Network Management

Distributed Computing Platform Application Support Services Dist. Data Distributed C/S Support OS Trans. Mgmt. Common Network Services Network protocols & interconnectivity OSI TCP/IP SNA Protocols
7/16/2013

12

Interoperability Portability Integration

The Internet

The Internet is a vast interconnected collection of computer networks of many different types. [Coulouris et al, p.3] It is the dominant distributed system at the current time, although intranets, the public switched telephone network (PSTN) and other networks are still important.

13

7/16/2013

Effect of the Web

Some industries have already been changed beyond all recognition by the web. Nearly all new car buyers have shopped the web first. That has dramatically changed the balance of power between dealer and customer. The stock brokerage and travel businesses have also been impacted dramatically.
7/16/2013

14

Intranets

An intranet is a private section of the Internet that is separately controlled, usually by a company, government or similar organization. It is used to provide information to employees, and sometimes suppliers and customers. It is usually protected from public access with authorization and authentication mechanisms and firewalls.
7/16/2013

15

Mobile and Ubiquitous Computing

Small portable devices have been integrated into information systems with wireless networking. These devices include laptop computers, handheld devices such as personal digital assistants, cell phones, wearable devices like smart watches, and devices embedded in appliances and automobiles.
7/16/2013

16

Resource Sharing and the Web

Distributed Systems began with the sharing of printers and files. Today, we take for granted the ready access to vast quantities of information. While the Internet may be little more than the ability to share information from a vast collection of files, the speed, search capabilities, and tools have changed much of our society.
7/16/2013

17

Web Portable Protocols

KISS Platform Independent Read-only; first generation Internet as backbone URL Global naming HTTP RPC-Like protocol HTML documents Web browsers as universal clients
7/16/2013

18

A Web Client/Server Interaction

Simple Web Client/Server Interaction 1. Select target URL 2. Browser send HTTP request to server 3. Server processes request a. Well-known port 80 for HTTP b. Send back requested HTML file; close connection 4. Browser interprets HTML commands If HTML type, displays page. Otherwise, launches helper
7/16/2013

19

The URL Structure

URL: Uniform Resource Locatorlocates
a resource on the net If you have the URL and an appropriate protocol you can retrieve the resource URI: Uniform Resource Identifier Identifies a resource

20

7/16/2013

URL (Uniform Resource Locator)

Naming scheme + how to get to resource Protocol scheme

21

HTTP, Gopher, News, FTP, Mailto, nntp, Telnet HTTP is native Web protocol Server name Internet host domain name or raw IP address Port number (or default) HTTP = 80; Gopher = 70; FTP = 21 Path to resource
7/16/2013

HTML: Hypertext Markup Language

ASCII file

text + tags Tags are commands Tag-pairs--command/inverse-command Document structure Header/Body Rooted in ISO SGML (Standard Generalized Markup Language) DTD (Document Type Definition) Extensions for hyper-linking
7/16/2013

22

HTTP:
Web's RPC on top of TCP/IP Stateless protocol Separate TCP connection to download each BLOB

document with 5 inline images = 6 connections Typed data using RPC Negotiate representation for each connection MIME-like content minus Internet Mail MIME's 7 types: Plain text, audio, video, still images, message, multipart message, applicationspecific data
7/16/2013

23

CGI: Common Gateway Interface

Supported by most Web servers Appropriate for small amounts of dynamic content
E.g. mostly static web pages plus phone number lookup Unacceptable for highly interactive web sites Each CGI request starts a processstartup delay Process-per-request requirement: overloads the server if many requests Now mostly obsolete, but important as an illustration of emerging need for efficiency.
7/16/2013

24

The Document is the Object

l XML (eXtensible Markup Language) Describes the structure of a document Defines new tags Specifies metadata that lets programs discover document structure l DOM (Document Object Model) Allows programmatic access to XML structure and content of XML documents l XSL (eXtensible Style Language) The XML version of Style sheets
25
7/16/2013

XML: Structured Data in a Text File

l Spreadsheets, address books, configuration parameters, financial transactions, product catalogs l XML defines a set of rules and conventions for designing text formats for such data l Easy to generate and read by computer l Extensible
26
7/16/2013

Role of XML

Applications built on different technologies can communicate via XML. New integration tools and integration servers capitalize on emergence of XML as an integration technology.

27

7/16/2013

Challenges

The concerns introduced at the end of chapter one in Coulouris et al will be major topics for discussion in future lectures in this course.

Heterogeneity Openness Security Scalability Failure Handling Concurrency Transparency

28

7/16/2013

A Broad View of Challenges in Distributed Systems

Your instructor comes from a business background. Business applications are less challenging than scientific applications: Simple models are adequate Client/Server applications are dominant Industry standard preferred over state of the art Most theoretical research in computer science ignores client/server and often focuses on parallel computing.

29

7/16/2013

Distributed System Challenges from a Systems Perspective

Communication Processes Naming Synchronization Data Storage/Access Consistency/Replication Fault Tolerance

Security API and Transparency Scalability/Modularity

30

7/16/2013

Distributed System Challenges from an Algorithmic Perspective

Execution models and Frameworks Distributed Graph and Routing Algorithms Time and Global States Synchronization and Coordination Group communication and multicasting

Monitoring distributed events Design and verification tools Distributed debugging Replication and consistency Web design cache, search, scheduling

31

7/16/2013

More Challenges from an Algorithmic Perspective

Distributed shared memory Reliable and fault tolerant systems Load balancing Real time scheduling Performance

These are broad topics, and each of them may have several subtopics. For example, Distributed shared memory is concerned with wait-free algorithms, mutual exclusion, and consistency models.

32

7/16/2013

References

George Coularis, Jean Dollimore and Tim Kindberg, Distributed Systems, Concepts and Design, Addison Wesley, Fourth Edition, 2005
Figures from the Coulouris text are from the instructors guide and are copyrighted by Pearson Education 2005

Andrew Tannenbaum and Maarten Steen, Distributed Systems, Principles and Paradigms, Prentice Hall, 2002 Umar, A., Distributed Computing and Client/Server Systems, Prentice Hall, 1993 Forbes magazine, various issues.
7/16/2013

33