WEB SERVICES

Durlabh Hawa durlabhh@techmahindra.com

1. Introduction......................................................................................................................3 2. What are web services?....................................................................................................3 3. Web Services indepth....................................................................................................4 3.1 WSDL........................................................................................................................5 4. What are the challenges in testing web services? ...........................................................6 4.1 Different types of testing...........................................................................................7 4.1.1 Component level testing.....................................................................................7 4.1.2 Proof of concept testing......................................................................................8 4.1.3 Functional testing................................................................................................8 4.1.4 Regression testing...............................................................................................8 4.1.5 Load / stress testing.............................................................................................8 4.1.6 Monitoring..........................................................................................................8 5. Load / stress testing..........................................................................................................9

1. Introduction
This document aims to give an introduction to the challenges in testing web services. This document puts particular emphasis on load testing of web services.

2. What are web services?

In the past, if you wanted to book a holiday over the Internet you would browse to a travel agent's web site, select a holiday and book it. If you were lucky, you'd be able to book car hire and possibly even check on the weather. Behind the scenes, the web server (the computer and software that the web site lives on) would be accessing data in a proprietary format in some sort of database, or would be communicating, again in a proprietary fashion, to back-office systems to get information about your holiday. This is all set to change with the concept of web services. A web service is a system which sits somewhere on a network and all it ever does is service specific requests from clients. These clients aren't real people browsing the Internet from their desks; they're other computers. When you book a holiday, you'll still browse to your travel agent's web site but behind the scenes something different will happen. To find flight details, rather than access proprietary data in a database, the web server will talk to a web service somewhere else on the Internet. All this web service will do, day in, day out, is respond to requests for flight information from travel agents' web sites. What is more, these web services will be able to publicize their existence and communicate with web sites, and other web services, in an open, public, format. The travel agent's web site will access different web services for flight, accommodation and weather information. Since the web service is on the Internet, and since it communicates in standard, published, protocols, that means that anybody can communicate with it. If you're running a conference and want people to visit, you can have a 'fly here' page on your web page. With 5 minutes' coding, you'll be able to let people view flight availability to your conference and book flights online, without leaving your site.

Figure 1 - A travel agents web service

3. Web Services indepth

A web service is a piece of software functionality that is accessible over a network and built on technologies that are independent of platform, programming language and component model. Web services are based on a Service-Oriented Architecture (SOA) where software functionality is distributed as a set of services. In order for a service to exist within the SOA realm, there needs to be a common mechanism to describe, discover, and invoke services. Figure 2, illustrates the different roles and the interactions between the roles in a ServiceOriented Architecture.

Figure 2 An SOA describes three basic roles: Service Provider: The Service Provider is responsible for the implementation of the web service. One of the first tasks that a service provider needs to do is determine the functionality they want to expose as a web service. Often this functionality is already present in their existing systems. Once they have done that, they need to describe the interface to this functionality in a standard fashion. Finally, they need to publish this interface to a registry for service consumers to find (discover) their web service Service Registry: The Service Registry functions as a repository of web services. Service providers typically publish their web service definitions to this registry. So, service registries have facilities not only to allow service providers to publish their web services but also to act as a repository for potential service consumers to find (discover) these web services and provide them with the necessary information to help them bind and invoke the web service. Service Consumer: The Service Consumer makes use of the web service created by the service provider. It retrieves all the necessary information for binding to the web service from the service registry, including the interface to the web service that the service provider has published. The interface gives it the details of the methods, parameters, and transport protocol necessary to make use of the service. The SOA defined above is an abstract model but it does have several concrete manifestations. One of these is the interoperable web services stack, as defined by several companies like IBM, Microsoft, etc. This describes the technologies used to achieve the functions that we saw above: find, bind, and publish. A common web services stack, which employs platform- and vendor-neutral standards, is essential in order to achieve interoperability.

Figure 3 shows the web services stack and its related technologies.

Figure 3 At the lowest level is the transport layer that is responsible for the communication between endpoints - communication ports that can send and receive messages. Web services technologies protect the investment in existing network technologies by utilizing some of the commonly available transport mechanisms that are already in widespread use, especially HTTP. It is important to note that we could use any transport protocol and are in no way bound to just HTTP. One level up, the messaging layer is responsible for the invocation of a web service by the service consumer. Here, the service consumer uses SOAP messages to call methods exposed by the web service and the service provider describes their web service interface in a standard fashion with WSDL (Web Services Description Language). Finally, at the top level is the discovery layer, implemented in UDDI (Universal Description and Discovery Interface). Before we get to AXIS itself then, we'll quickly recap the three technologies at work in the web services stack: SOAP - a standard mechanism for invoking web services WSDL - a standard mechanism for describing web services UDDI - a standard mechanism for publishing and discovering web services With an understanding of each of these, we'll be able to appreciate the advances that AXIS makes over previous versions of Apache SOAP

3.1 WSDL
Web Services Description Language (WSDL) is a grammar for describing web services. With it, we can describe the purpose of the web service: what it does, the methods that can be invoked, the parameters that need to be passed to the methods, the parameter types, the binding protocol to use, and so on. In short, given a WSDL document about a web service, you should be in a position to invoke the functionality of the web service. A WSDL document is an XML document, structured as shown in Figure 4:

Figure 4 It's easiest to view a WSDL document as consisting of two parts: first, an abstract definition of the web service the data types, methods, etc. and second, details of the binding to the transport protocol that that particular service and SOAP server implementation are using. Let us briefly describe the different elements: Type A <type> element represents the data types used: float, integer, string, etc. WSDL uses the XML Schema specification for data type encoding. Message Each <message> element represents a parameter of a service invocation. An operation in WSDL terminology is the method signature the relationship between the input and output messages. PortType A <portType> is a collection of operations (i.e. methods) exposed by the service. A <portType> has a collection of messages it is similar in concept to a class in Java. Just as a class has methods, a <portType> contains messages (methods). Binding Binding is used to specify two things. The transport protocol which will be used (SOAP can be used over HTTP, SMTP, or possibly any other transport) and the style of request (rpc and document are the two styles). Service A <service> element is the actual implementation-specific detail. It will reference a <Binding> element and associate it with a <port> element, which contains the address of the endpoint: a HTTP URL, etc.

4. What are the challenges in testing web services?

Like all software, web services need to be tested. Like web sites, they are often highly visible, so they need to be very robust. There are two broad types of web services - web services used in an intranet and web services used on the internet. Intranet web services are web services used internally by organizations but not exposed to the general public. For example, an intranet web service might be responsible for handling vacations requests from employees. The company's intranet web site would then access this web service and employees could request vacations. Managers could authorize the vacations and colleagues could check when other employees were on holiday. Human resources could then write a simple application in Visual Basic to make sure that helpdesk staff don't take all their vacation at the same time without having detailed knowledge of how or where this information is kept.

An example of an Internet web service is the holiday web service in the previous paragraph. Testing intranet and internet web services provides subtly different problems. With an intranet web service you, as an organization, are likely to have control over who access your web service. Since it is on an internal network, only internal users can have access to it, so you have a theoretical maximum. Similarly, you can make certain assumptions about security. With an internet web service, anybody can access it. This means that there are additional scalability and security considerations. Another challenge in testing web services is that they are completely UI-less; they do not display a user interface that can be tested. This means that they hard to test manually but are an ideal candidate for automated testing. A consequence of this is that some programming skills are almost certainly needed for testers who need to test web services. A web service is not the sort of application you can test by key bashing.

4.1 Different types of testing

As with any other application, there are different types of testing you can carry out on web services:

4.1.1 Component level testing

You can test Web services by calling Web methods from unit tests. Testing Web services is much like testing other code by using unit tests in that you can use Assert statements, and the tests produce the same range of results. Two ways to test Web services with unit tests: The Web service runs on an active Web server . There are no special requirements for testing a Web service that runs on a local or a remote Web server, such as IIS. To do this, add a Web reference and then call the Web methods of the Web service from your unit tests just as they would call the methods of a program that is not a Web service. The Web service is not hosted in an active Web server . You can test a Web service that runs on your local computer and not in a Web server, such as IIS. Testing a Web Service Locally The process for testing Web service that runs on your local computer but not in IIS includes: 1. 2. 3. Create the Web service on the local file system. Generate unit tests against the Web service in the standard way for generating unit tests. Add the attribute to the unit test. The arguments for this attribute class point to the site of the Web service and name the server. 4. Within the unit test, add a call to the TryURLRedirection method to point the Web service object to the correct server. Verify that it returns true, and use and Assert statement to fail the test if the redirection fails.

5.

Call the Web service or exercise it in any other way that you feel is necessary to test it thoroughly.

4.1.2 Proof of concept testing

Web services are a new type of software. Because of this, you will need to understand if the architecture you have chosen for your web service is the correct one. There are many different choices to be made - which tool vendor to use, which programming language and which database backend, for example. If you can clarify and resolve these issues before you start developing, or early on in the development lifecycle, then you will save a lot of time and money further on down the line. Because most of the questions you need to resolve will concern scalability issues (will the architecture we're using really cope with 1000 simultaneous users), a proof of concept test is normally a cut-down load test (see below). There is no need to run it on powerful hardware, or get exact answers; your aim is to find answer the question "am I going in the right direction?

4.1.3 Functional testing

This is to ensure that the functionality of the web service is as expected. If you have a web service that divides two numbers does it gives the expected result? If you pass in a 0 as the denominator, does it handle this correctly? Does your web service implement security / authentication as it is meant to? Does your web service support all the communications protocols it is meant to? Because your web service can be accessed by clients that you can't control, what happens if they make requests you aren't expecting? Bounds testing and error checking is especially important.

4.1.4 Regression testing

A regression test is normally a cut-down version of a functional test. Its aim is to ensure that the web service is still working between builds or releases. It assumes that some area of functionality was working in the past, and its job is to check that it still does. If your development team has changed the method that divides two numbers, does that method still work? Does the method that multiplies two numbers still work? Is the performance still acceptable? Since regression testing is, by its nature, a repetitive task then it must be automated. This is true for traditional applications and web sites; it is even truer for UI-less web services.

4.1.5 Load / stress testing

The aim of load / stress testing is to find how your web service scales as the number of clients accessing it increases. You have carried out functional and regression testing, so you know that your web service will cope with a single user. What you need to know now is if it will cope with 10, 100 or 1000 users, or how many users it will cope with. If you double the number of users, do response times stay the same? If you double the number of servers running your web service, does its capacity double? The following section in this document will go into more details about these issues.

4.1.6 Monitoring
Once your web service live and being used by real clients, you will need to keep an eye on it. Is it still working? Are response times adequate? At what times of day is it busiest? It is essential to monitor the web service.

5. Load / stress testing

This document concentrates on load and stress testing a web service. It is important to note that this, almost by definition, must be an automated task. You cannot feasibly employ 1000 people to simulate 1000 clients accessing your web service. To produce objective results, it is important to carry out the testing in a controlled environment. If you want to know how your web service responds as you simulate more and more users, then this means that you must keep all other factors (hardware and networking, for example) constant. This means that you should not carry out a load test on a live system over the Internet. Apart from the problem that bandwidth would pose; this is not a controlled environment. You might decide that you simply cannot carry out the test in-house. If you are writing a largescale web service that needs to cope with 10,000 requests a second then the odds are you do not have the necessary hardware in-house to do this. You will probably need to consider hiring a third party consultancy, or using a third party scalability lab to carry out this test. The ultimate aim of load testing is to reassure you, and confirm "My web service will respond acceptably for up to x clients making y requests a second". Before you can start, you need to decide who the clients will be. Will they be other web services, other web sites or other sorts of applications? Do you have any idea of how many potential clients your web service will have? Once you have answered that, you need to ascertain what these clients will be doing, and how often. You may have different sorts of clients doing different things; 10% of your clients might be booking flights while 90% might be simply checking flight availability. If you are expecting to have 100,000 clients, but these clients will only ever make one request a day then that is not too bad (about 1 request a second). If, however, 50% of these requests will happen between the 9 and 10am then that's a different story (about 15 requests a second). Another possibility is that you will only have 100 clients, but that these clients will be making 10 requests a second, each and every second of the day. In summary, it is important to know your clients. You might expect, on average, to be able to service 100 requests a second, but what happens in the event of a massive peak? Will your web service cope, slow down, or crash? Or will it simply refuse to service the 101st request? These are all things you need to know, and test, before you release your web service. You must also now how the clients will be accessing the web services. The odds are they will be making SOAP requests. In this case, you should make sure that your web-testing tool supports this protocol. Some testing tools on the market record scripts by browsing to the web page representing the web service and then record the HTTP GET and POST requests that the browser makes. Although similar, this is not the same as using the SOAP requests that your clients will almost certainly be making. The next question to answer is "what is acceptable? Your service level agreements might specify that you need to respond to 95% of requests within 1 second; in this case you know what is acceptable. In any case, it is important to realize that a web service is not the same as a web site, so acceptable response times will be different. While it may be acceptable to keep response times for a web site under 10 seconds, a web site may be making 10 calls to different web services, so the web site will expect each web service to return information within 1 second. While you are carrying out your test, the tool you are using should be able to identify how the following values change as you increase the number of clients. These are all measurements of how the client is experiencing your web service: Time to connect: This is the time it takes to make a connection from the client to the web service. This should be as low as possible. Time to first byte: This is the time it takes for the client to start receiving data back from the web service. If the web service needs to do a lot of thinking for each request, then this time could be significant. Time to last byte: This is the time it takes for the client to receive the last byte of information back from the service. If the service needs to return a large amount of data (if it is returning maps, or images, for example), then this could be significant.

Although the absolute values of these metrics are important (you want to keep these numbers as low as possible), the way they change as you increase the load on the web service is more important. Ideally you want these metrics to remain constant. If they increase linearly, then you're in for a shock. Suppose that with 1 user the time to last byte is blazingly fast 10 milliseconds, for 10 users is 1/10th second and for 100 users it's 1 second. That's still reasonable. For 1000 users it's going to be 10 seconds, which is probably unacceptable. If your web service needs to cope with 10,000 users then requests are going to take more than a minute. That's clearly not acceptable. It's more likely that you'll find that your web service scales (i.e. the response time for requests remains constant) until a certain number of virtual users, and then it stops scaling. To troubleshoot this, you need to keep an eye on the performance on the server the service is running on. You need to know what is causing this change in behaviors; is the CPU saturated, is the disk thrashing or is the network card causing the bottleneck? These are all possible causes of performance problems. This detailed analysis is outside the scope of this document. Once your system has hit a bottleneck, you need to know if there is a way round this problem. If you add another processor to the server, will it double its capacity? If you add another server, will that double its capacity? If you have a web service that will scale in this way, then you know that you will be able to cope with extra demand by adding more and more hardware at the problem. If your web service doesn't scale in this way, then your web service won't perform no matter how much money you spend on expensive hardware.

Figure 5 - Web services with different scalability properties In the above illustration, Web Service 1 scales well until about 120 requests / second (the time to last byte is constant), but then stops scaling. Web service 2 scales poorly - as the number of requests / second increases, the responsiveness decreases linearly.

10