Jyu Thesis

A UI-driven Approach to Facilitating
Effective Development of Rich and

Composite Web Applications
Jin Yu
A dissertation submitted in fulfillment of the requirements for the degree of
Doctor of Philosophy
School of Computer Science and Engineering
University of New South Wales
Sydney, NSW 2052, Australia
Supervisor: Prof. Boualem Benatallah
October 31, 2008

Originality Statement
‘I hereby declare that this submission is my own work and to the best of my knowledge
it contains no materials previously published or written by another person, or substantial
proportions of material which have been accepted for the award of any other degree or
diploma at UNSW or any other educational institution, except where due
acknowledgement is made in the thesis. Any contribution made to the research by
others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in
the thesis. I also declare that the intellectual content of this thesis is the product of my
own work, except to the extent that assistance from others in the project's design and
conception or in style, presentation and linguistic expression is acknowledged.’
Jin Yu
October 31, 2008

v
Acknowledgements
It has been a great experience to work as an external PhD student in the School of
Computer Science and Engineering, the University of New South Wales. As an off-
campus student, I faced numerous challenges in the last three years, due to the physical
distance (I reside in California) and time zone differences. I am most grateful to my
supervisor, Prof. Boualem Benatallah, who made all this possible after all.
Foremost, I would like to thank Prof. Benatallah for accepting and accommodating me
as an external research student, for he had to spend countless hours on email
communications with me. As a dedicated researcher and inspiring mentor, he has taught
me many valuable lessons, including key research disciplines and methodologies.
I would like to give special thanks to my co-supervisor, Prof. Fabio Casati, who I
interacted frequently (when he was in HP Labs in Palo Alto) and had many interesting
and fruitful discussions. Prof. Casati was also instrumental in many of my research
papers, as he not only gave insightful comments but also directly edited some the
papers.
In addition, I would like to thank my research collaborators Regis Saint-Paul, Florian

Daniel, Maristella Matera, and many students who have participated in the research
projects. I enjoyed the collaboration very much and gained invaluable skills by working
with them. Among the research students, I wish to express my sincere appreciation to
Evi Syukur, who spent many hours in helping me with proof reading and the printing
and binding of thesis hardcopies.
Finally, I am grateful to my parents and my wife, for their endless support and
encouragement throughout my PhD study. I express special thanks to my wife, who took
on the dreadful task of packing and unpacking when we moved to a bigger house,
allowing me to concentrate on the thesis and finish it on time.
vii
Abstract
It is well-recognized that the development of user interfaces is one of the most time-
consuming tasks in the overall application development process. At the same time, there
is an increasing demand for rich and fluid user interfaces from web users. As a result,
developers are facing increasing challenges in delivering web applications, especially
those with rich UI requirements.
In this thesis we present two solutions to facilitate the execution and rapid development
of web applications with rich user interfaces. The first solution is a rich internet
application (RIA) framework aimed at providing high usability and productivity to web
applications, while the second solution is a UI integration framework that simplifies
web application development by facilitating the composition of reusable UI
components.
The foundation of our RIA framework is an XML-based high-level protocol for

communicating asynchronous events and incremental UI updates on the web. The
protocol facilitates rich and highly interactive UI, while at the same time eliminates
frequent and slow page refreshes and provides a more responsive user experience.
Built on top of the protocol, a server-side runtime allows UI logic code to be executed
on the server side, while a set of server-side event-driven API enables developers to
implement sophisticated application-specific UI behavior. On the client side, a thin
client renders UI and processes native events, but leaves application-specific logic to the
server side. The thin client thus allows end users to enjoy a rich UI experience in a safe
client environment, without executing any downloaded code.
The proposed UI integration framework includes an abstract UI component model

which allows UI components to be programmatically manipulated via events,
operations, and properties, essentially exposing UI as services. To facilitate component
interactions, the framework offers an event-based composition model, which allows
integration logic to be specified in the form of event listeners.
viii
Composite applications are executed via a lightweight runtime middleware, which

provides component adapters that allow the middleware to communicate with native UI
components implemented in a variety of languages and platforms. Finally, a graphical
development environment allows composite applications to be built in a drag-and-drop
fashion.
ix
Table of Contents
1. Introduction ...............................................................................................................1
1.1 Requirements and Challenges ...........................................................................2
1.1.1 Developing Rich Internet Application ......................................................3
1.1.2 UI Integration ............................................................................................4
1.2 Contributions Overview....................................................................................6
1.2.1 Developing Rich Internet Application ......................................................7
1.2.2 UI Integration ............................................................................................9
1.3 Thesis Organization ........................................................................................12
Part 1: OpenXUP – an RIA Framework .........................................................................15
2. Background and State of Art in RIA.......................................................................17
2.1 Dimensions for Characterizing RIA Technologies .........................................17
2.1.1 Usability and Productivity.......................................................................17
2.1.2 UI Code Location ....................................................................................19
2.1.3 Asynchronous UI.....................................................................................20
2.1.4 Development Environment .....................................................................20
2.2 RIA Technologies ...........................................................................................21
2.2.1 Classic HTML.........................................................................................21
2.2.2 Rich Client Technologies........................................................................22
2.2.3 Server-side Approaches ..........................................................................28
2.2.4 AJAX ......................................................................................................31

x
2.2.5 Remote Display Technologies ................................................................33
2.2.6 Rich UI Languages..................................................................................36
2.2.7 Comparisons ...........................................................................................36
2.3 Summary .........................................................................................................38
3. UI Transport Protocol .............................................................................................39
3.1 User Interface Model.......................................................................................40
3.2 Protocol Design...............................................................................................42
3.2.1 Definitions...............................................................................................42
3.2.2 Protocol Operations.................................................................................43
3.2.3 Network Event Delivery .........................................................................46
3.2.4 Server-side Notification ..........................................................................48
3.3 Examples.........................................................................................................49
3.3.1 Example 1 ...............................................................................................49
3.3.2 Example 2 ...............................................................................................51
3.4 Summary .........................................................................................................54
4. Developing and Executing RIAs.............................................................................57
4.1 Example Scenario ...........................................................................................58
4.2 Runtime Framework .......................................................................................60
4.2.1 OpenXUP Architecture ...........................................................................61
4.2.2 MVC Pattern in OpenXUP .....................................................................63
4.2.3 Location of UI and Application Code .....................................................64

xi
4.2.4 Runtime Behavior ...................................................................................65
4.2.5 Server-side Asynchronous Notifications.................................................67
4.3 Development Environment .............................................................................67
4.3.1 Application Development APIs ..............................................................67
4.3.2 UI Templates...........................................................................................73
4.3.3 Multiple UI Languages............................................................................76
4.3.4 Asynchronous Event Handling ...............................................................77
4.3.5 Multi-threading .......................................................................................78
4.3.6 Application Development .......................................................................80
4.4 Implementation ...............................................................................................84
4.4.1 Protocol Module......................................................................................85
4.4.2 OpenXUP Server.....................................................................................86
4.4.3 UI Templates...........................................................................................87
4.4.4 SUL .........................................................................................................88
4.4.5 Tracking UI Updates ...............................................................................89
4.4.6 Performance Analysis .............................................................................89
4.4.7 Developer Productivity ...........................................................................93
4.5 Summary .........................................................................................................95
5. Delivering RIAs via Thin Client .............................................................................97
5.1 Thin Client Design ..........................................................................................98
5.1.1 Architecture...........................................................................................100
xii
5.1.2 UI State Management............................................................................102
5.1.3 Network Event Delivery .......................................................................103
5.1.4 Synchronous and Asynchronous Events ...............................................104
5.1.5 Thin Client vs. Fat Client......................................................................105
5.1.6 XUPClient vs. Web Browsers...............................................................109
5.2 Running XUPClient ......................................................................................109
5.2.1 Accessing Web Applications ................................................................110
5.2.2 User Interactions ...................................................................................110
5.2.3 Desktop-like User Experience ..............................................................113
5.3 Implementation .............................................................................................113
5.3.1 Protocol Module....................................................................................115
5.3.2 Tracking UI Updates .............................................................................116
5.3.3 Performance Analysis ...........................................................................116
5.4 Summary .......................................................................................................117
Part 2: Mixup – a UI Integration Framework................................................................119
6. Background and State of Art in UI Integration .....................................................121
6.1 Integration Layers..........................................................................................122
6.2 Dimensions for Characterizing UI Integration Technologies .......................126
6.2.1 Component Model ................................................................................127
6.2.2 Composition Model ..............................................................................128
6.2.3 Runtime Infrastructure ..........................................................................129

xiii
6.2.4 Development Environment ...................................................................131
6.3 UI Integration / Component Technologies....................................................131
6.3.1 Desktop UI Components .......................................................................131
6.3.2 Browser Plug-in Components ...............................................................133
6.3.3 Web Portals and Portlets .......................................................................133
6.3.4 Web Mashups........................................................................................135
6.3.5 Comparisons .........................................................................................143
6.4 Summary .......................................................................................................145
7. UI Component Model ...........................................................................................147
7.1 Reference Scenarios ......................................................................................148
7.2 Lessons Learned from EAI / Service Composition.......................................150
7.3 Design Principles ..........................................................................................151
7.4 Abstract Component Model ..........................................................................155
7.5 UI as Services................................................................................................160
7.6 Summary .......................................................................................................163
8. UI Composition Model .........................................................................................165
8.1 Lessons Learned from EAI / Service Composition.......................................167
8.2 Guiding Principles for UI Composition ........................................................168
8.3 Composition Model ......................................................................................170
8.3.1 Event Subscriptions ..............................................................................171
8.3.2 XPIL......................................................................................................172
xiv
8.3.3 Additional Integration Logic.................................................................173
8.3.4 Layout Information ...............................................................................176
8.4 UI Content Consolidation .............................................................................177
8.4.1 Value-Changed Events..........................................................................179
8.4.2 Hiding Content Items ............................................................................181
8.4.3 Discussions ...........................................................................................182
8.5 Component Embedding.................................................................................183
8.6 Composite Component..................................................................................188
8.7 Summary .......................................................................................................192
9. UI Integration Framework.....................................................................................193
9.1 Example Scenario .........................................................................................194
9.2 Runtime Middleware.....................................................................................196
9.2.1 Event Automation .................................................................................197
9.2.2 Component Adapters.............................................................................199
9.2.3 Runtime Behavior .................................................................................203
9.3 Development Environment ...........................................................................204
9.3.1 Creating Component Descriptors..........................................................205
9.3.2 Specifying Composition Logic..............................................................207
9.3.3 Defining Layout ....................................................................................209
9.3.4 Deployment ...........................................................................................210
9.4 Framework Implementation ..........................................................................210

xv
9.4.1 Runtime Middleware.............................................................................210
9.4.2 Development Tools ...............................................................................220
9.5 Summary .......................................................................................................224
10. Conclusions.......................................................................................................225
10.1 Summary .......................................................................................................225
10.1.1 OpenXUP ..............................................................................................225
10.1.2 Mixup....................................................................................................227
10.2 Future Work ..................................................................................................230
Publications...................................................................................................................233
Project Web Site .......................................................................................................234
Bibliography..................................................................................................................235
xvi
List of Figures
Figure 2.1 Standard dialog ..............................................................................................18
Figure 2.2 Architecture of AJAX applications ...............................................................31
Figure 2.3 X Window System.........................................................................................34
Figure 3.1 Protocol message exchange ...........................................................................43
Figure 3.2 Before button click ........................................................................................49
Figure 3.3 After button click...........................................................................................50
Figure 3.4 Before clicking "Add to list" button ..............................................................51
Figure 3.5 After clicking "Add to list" button.................................................................52
Figure 3.6 Selecting list item "Chocolate Chip" .............................................................53
Figure 4.1 Example application: XCat............................................................................59
Figure 4.2 OpenXUP framework architecture ................................................................61
Figure 4.3 XComponent class .........................................................................................68
Figure 4.4 Classes related to events ................................................................................69
Figure 4.5 XApplication class.........................................................................................70
Figure 4.6 XUserSession class........................................................................................71
Figure 4.7 SUL classes....................................................................................................71
Figure 4.8 SUL events.....................................................................................................72
Figure 4.9 SUL event masks ...........................................................................................72
Figure 4.10 XHTML classes ...........................................................................................77
Figure 4.11 XML parsing performance...........................................................................90

xvii
Figure 4.12 Request processing performance .................................................................92
Figure 5.1 XUPClient architecture................................................................................100
Figure 5.2 Open URL dialog.........................................................................................110
Figure 5.3 Initial window..............................................................................................112
Figure 5.4 After button click.........................................................................................112
Figure 5.5 XUPClient's system menu ...........................................................................113
Figure 6.1 Component integration at different layers ...................................................123
Figure 6.2 HousingMaps...............................................................................................137
Figure 7.1 The National Park Guide .............................................................................150
Figure 7.2 Component-defined event vs. native UI event.............................................156
Figure 7.3 Relationship between UI components and application logic / data services157
Figure 8.1 National Park Guide (event-based model)...................................................168
Figure 9.1 Architecture of the UI integration framework, Mixup.................................193
Figure 9.2 New York Times example ...........................................................................196
Figure 9.3 Event automation .........................................................................................198
Figure 9.4 Component adapters ....................................................................................200
Figure 9.5 Architecture of Mixup's development environment ....................................205
Figure 9.6 Eclipse-based Mixup Editor ........................................................................209
Figure 9.7 AJAX-based Mixup Editor..........................................................................221

xviii
List of Tables
Table 2.1 Comparison of web-based UI technologies ....................................................37
Table 3.1 XUP protocol elements for incremental UI updates .......................................45
Table 3.2 Managing XUP status requests .......................................................................48
Table 3.3 Protocol comparison .......................................................................................55
Table 4.1 Server code size ..............................................................................................85
Table 4.2 XML parsing performance..............................................................................90
Table 4.3 Request processing performance ....................................................................91
Table 4.4 Code size comparison (classic HTML vs. OpenXUP) ...................................94
Table 5.1 Client code size .............................................................................................114
Table 6.1 Comparison of UI integration approaches ....................................................143
Table 9.1 Application startup performance...................................................................218
Table 9.2 User study: composition development time..................................................223

xix
Code Listings
Listing 3.1 UI model example (in SUL) .........................................................................41
Listing 3.2 XUP request for Example 1 ..........................................................................50
Listing 3.3 XUP response for Example 1........................................................................50
Listing 3.4 XUP request for Example 2 (button click) ...................................................52
Listing 3.5 XUP response for Example 2 (after button click).........................................53
Listing 3.6 XUP request for Example 2 (selecting list item) ..........................................54
Listing 4.1 UI template example.....................................................................................74
Listing 4.2 Multiple UI languages example ....................................................................76
Listing 4.3 C# source code fragment for XCat ...............................................................81
Listing 4.4 UI template "catalog.xml".............................................................................82
Listing 4.5 "Selection-changed" event request................................................................84
Listing 5.1 Event selector example ...............................................................................104
Listing 5.2 Startup XUP request ...................................................................................110
Listing 5.3 Startup XUP response.................................................................................111
Listing 5.4 Event request for button click.....................................................................112
Listing 5.5 Event response for button click ..................................................................112
Listing 7.1 UISDL descriptors (National Park Guide)..................................................159
Listing 7.2 Binding for the park listing component ......................................................160
Listing 7.3 UISDL descriptor with multiple bindings...................................................162
Listing 8.1 Composition model description (National Park Guide) .............................172

xx
Listing 8.2 Process-based integration logic...................................................................175
Listing 8.3 Component descriptors for credit application and vehicle registration ......178
Listing 8.4 XPIL fragment for content consolidation ...................................................179
Listing 8.5 Component descriptors with value-changed events....................................180
Listing 8.6 Component descriptors for YouTube and the news report .........................184
Listing 8.7 Composition model for the multimedia news report ..................................185
Listing 8.8 Component descriptors for Google Maps and PImage ...............................187
Listing 8.9 Composition model for the real estate application .....................................187
Listing 8.10 Composite component for the original park guide ...................................189
Listing 8.11 Composition model for the multimedia park guide ..................................191
Listing 9.1 UISDL descriptors for Yahoo pipe news feed and YouTube video ...........195
Listing 9.2 XML response from the "getWeatherInfo" operation.................................201
Listing 9.3 Generated JavaScript object for the XML response ...................................202
Listing 9.4 Composition model for the New York Times example..............................208
1
Chapter 1. Introduction
1. Introduction
Web-based applications and services are now the dominant form of software
development and deployment. It is well-recognized that a significant amount of time in
developing an application is spent on its user interface (UI) [Mye92]. Therefore, UI
development has become a key element in the web application development process
[Bri02][Gar02].
Traditionally web applications use HTML to render their user interface. UI is delivered
as HTML pages coupled with some simple JavaScript for data validation. This form of
UI is very simple, but lacks true interactivity, since HTML was originally designed to
render text with embedded images and therefore does not offer rich UI controls [Jel04].
In addition, web pages are always reloaded after each user actions (e.g., mouse click).
This is very annoying to end users, since it takes time to reload a web page even if only
a very small portion of the user interface needs to be updated.
In response to that, developing rich internet applications (RIA) has become an essential
part of the recent Web 2.0 [ORe05] trend. In RIA, interactions are no longer page-based;
that is, user actions do not always result in page refreshes – only the changed UI
elements will be re-rendered. Thus, web UI becomes as fluid and as rich as traditional
desktop UI. RIA not only benefits end users, it also benefits the overall network
infrastructure as it requires less roundtrip to the servers and the data involved are
typically small (i.e., not entire pages of data). At the same time, it presents new
challenges to developers because the shift from traditional page-based programming
model to RIA may require additional skills and development effort.
With the added rich UI requirement, the already time-consuming task of web UI
development becomes even more daunting. To save development time and effort, the
reusability of UI is of critical importance. There are many UI toolkits, such as Java
Swing1 and .NET Windows Forms2, which provide pre-packaged classes modeling fine-
grained UI controls (e.g., buttons and menus). However, the reusability of the UI
1
http://java.sun.com/products/jfc/
2
http://msdn.microsoft.com/en-us/netframework/aa497342.aspx
2
remains low since the unit of reuse is low-level UI controls such as buttons and panels,
not high-level UI components encapsulating real application functionalities.
High-level UI components are essentially the presentation front-ends of web

applications or services. Examples of such UI components are: a stock quote portlet that
retrieves stock prices from Yahoo, a weather gadget that displays weather information
from WeatherBug's weather database3, and street maps such as Google Maps4 and
Yahoo Maps5. The availability of these high-level UI components will dramatically
shorten the UI development time of web applications and services.
Once the required UI components are in place, they need to be put together to form an
integrated web application. This process is what we called UI integration or
composition, where UI components from various sources are assembled together to
behave in a synchronized fashion in the composite web application [Yu07b][Yu07c].
This is similar to traditional application integration and web service composition, where
components (i.e., services) are integrated together to form composite applications or
services. With the availability of high-level UI components and appropriate UI
integration middleware, web applications can be quickly put together, without having to
redevelop UI from scratch every time.
In this chapter, we will first discuss the requirements and challenges in developing UI
for web applications. We then outline our major contributions. Finally, we describe the
structure of this dissertation.
1.1 Requirements and Challenges

In this section we outline the requirements and challenges in facilitating web UI
development. We separate the challenges in two areas: rich internet applications and UI
integration.
3
http://www.weatherbug.com
4
http://maps.google.com/
5
http://maps.yahoo.com/
3
1.1.1 Developing Rich Internet Application

To provide end users with a rich and responsive user interface, RIA applications will
face the following challenges:
High usability and productivity. Traditional HTML-based web user interface lacks
true interactivity, due to the fact that HTML was originally designed to render hypertext,
not rich UI. This results in a reduction of both the web application's usability and the
end user's productivity. What's needed is a set of rich UI controls equivalent to those
found in desktop applications, so that the web UI can be as fluid and as rich as its
desktop counterpart.
For developers, this calls for a rich UI toolset, with a familiar programming model
similar to those found in desktop GUI toolkits. Essentially, developers should be able to
develop web applications the same way they develop desktop applications.
Asynchronous and incremental UI updates. HTML's page-based model requires the

entire page to be refreshed for each user action, even though typically only a small
portion of the UI needs to be updated. However, users expect fast response time from
their applications. This implies the use of asynchronous mechanism to perform UI
operations while computing in the background. Additionally, UI updates should be
applied incrementally, eliminating the latency and annoyance associated with slow but
frequent page refreshes. Furthermore, both asynchronous and incremental UI
functionalities should be provided by the framework, without the need of complex
programming techniques such as multi-threading and callback functions.
Secure client environment. While many rich web client technologies tried to address
the limitation of HTML, their common approach is to download code to be executed on
the client side (i.e., the browser). The downloaded code could be binary (e.g., applet
byte code) or text (e.g., JavaScript), both impose security risks. Therefore, with
increasing number of internet-based security risks (e.g., identity theft6), the client-side
environment should only execute safe UI code (i.e., markups), without compromising
the security of the end user's computer.
4
Development complexity. Many rich web client technologies leverage JavaScript to

manage and extend the client-side UI. However, comparing to traditional programming
languages, large amount of JavaScript code remains much harder to develop, test, and
maintain. Although browser incompatibilities have been mitigated due to the availability
of sophisticated JavaScript toolkits, the performance of the JavaScript interpreter still
varies greatly among browsers [Wei08]. In addition, developers now need to be
concerned about intellectual property protection issues, as downloaded code can be
easily viewed and copied on the client side.
With traditional HTML applications, developers do not need to worry about network
issues since both UI logic and business logic code reside on the server side. However,
with rich web client technologies, UI logic is now at the client side, and therefore the
developers are responsible for the network communication between the client-side UI
logic and server-side business logic (e.g., to catch network exceptions). Ideally, the
development environment should allow developers to be oblivious of the deployment
method. The burden of managing client/server communications should be handled by
the runtime framework, not by the developers.
1.1.2 UI Integration
To facilitate the rapid development of web-based user interfaces, UI integration
frameworks will face the following challenges:
Reusable high-level UI components. Application componentization has long been a

common practice in web application development; that is, application functionalities are
packaged as modules or components to be reused. When developing a web application,
the developer first selects application components containing desired functionality and
then glues them together appropriately. After that, she builds a user interface for the
integrated web application. Therefore, user interfaces are typically re-developed from
scratch every time. To simply the development of web-based user interfaces, it is
important to make them componentized so that they could be reused, just like
application functionalities.
6
http://en.wikipedia.org/wiki/Identity_theft
5
The granularity of the reuse should be high-level UI components encapsulating real

application functionalities, not low-level UI controls such as buttons and panels. A high-
level UI component can be regarded as the presentation tier of a web application
component or module. Examples of such UI components are: a news ticker that displays
headlines from CNN, a stock quote portlet that retrieves stock prices from Yahoo, and a
weather gadget that displays weather information from WeatherBug. A high-level UI
component may in fact consist of many low-level controls. The weather gadget, for
example, may contain a panel which houses a graph control for displaying the
barometer, two text labels for displaying high and low temperatures, a text field for
inputting zip code, and a few buttons to cycle through weather forecasts for different
days or ranges of days. Obviously, the desired unit of reuse is the entire weather gadget,
not the low-level UI controls within it.
Component heterogeneity. There are a variety of component technologies that can be

used to develop UI components. Those technologies (and their underlying languages and
platforms) are typically incompatible with one another. In addition, components are
typically built by different developers before the development of the composite
application. As a result, the composite application developer must be able to cope with
existing heterogeneous UI components from a variety of sources. For example, when
putting together a PIM (Personal Information Management) application, the developer
may need to integrate a .NET calendar component, a Java applet task list, and an AJAX-
based address book. All three components were built with incompatible component
technologies7. This is an overwhelming task as the composition developer must possess
the intimate knowledge of multiple component technologies in order to reuse
components built by them.
Therefore, what's needed is a facility that allows components built with different
technologies to be reused in the same composite application, while at the same time
hides the platform and language differences from the composition developer. Hence, the
7
The three components may in fact be available in the same component technology. For illustration
purpose, we assume they were inaccessible to the developer for reasons such as license and
distribution restrictions.
6
key is the ability to facilitate the communication among components from different
technologies so that they could seamlessly work in the same composite application.
UI composition. While the composition techniques in application integration (e.g.,

services composition) have been well-researched, there has been little study in the UI
area. As we have already discussed the importance of reusing user interfaces, proper UI
composition techniques are needed to assemble UI components into new, value-added
composite applications.
Currently, web-based UI composition exists mostly in form of combining page clips,

where several HTML fragments enclosed in <div> or <iframe>8 are combined to
produce a new page. This form of UI composition is very limited in functionality. For
example, the page clips cannot effectively communicate or interact with one another;
they just sit side by side in the final page.
Therefore, what's needed is a composition model that facilitates the communications and
interactions among UI components, while at the same time provides a consistent visual
layout. This ensures that the UI components will behave in a synchronized fashion in the
composite application. In addition, the composition model needs to be simple yet
effective. Simplicity will facilitate quick adoption by developers and tech-savvy
business users, and may eventually enable end-user compositions.
1.2 Contributions Overview

Our goal is to facilitate and simply the development of web-based, rich user interfaces.
To achieve this goal, we propose an RIA framework to develop highly interactive web
user interfaces, and a UI integration framework to facilitate the development of
composite applications by reusing existing, heterogeneous UI components.
8
These are HTML container elements that allow any web content to be embedded.
7
1.2.1 Developing Rich Internet Application

To provide a rich web UI experience to end users and to simply the development of
highly interactive user interfaces, we propose an RIA framework (called OpenXUP),
consisting of the following ingredients.
UI transport protocol. The foundation of our RIA framework is the Extensible User
Interface Protocol (XUP) [YC02], an XML-based high-level protocol for
communicating events and incremental user interface changes on the web. We chose
SOAP/HTTP [Gud07a][Gud07b] as the default binding protocol because it is well-
understood, and its implementations in different platforms are widely available.
Alternative bindings such as REST9 [Fie00] are also possible.
In XUP, user actions result in UI events, which are sent as requests to the server side for
processing. Event requests can be delivered from the client to the server asynchronously,
so end users will find applications to be much more responsive. After processing the
events, the server sends back a response containing necessary UI updates. Since the UI
updates are incremental, not one full page at a time, end users will no longer experience
slow page refreshes.
Server-side runtime and development environment. OpenXUP offers a server-side

runtime environment similar to traditional web applications. That is, UI logic and
behavior are programmed on the server side. This allows the applications to be centrally
managed, without the hassles of client-side software maintenance. Unlike traditional
runtimes which return full HTML page in each response, our runtime keeps track of UI
changes, and only returns UI deltas in each response. This is achieved by maintaining a
UI model that corresponds to the exact user interface rendered to the end user.
OpenXUP's development environment includes a set of event-driven APIs, which enable

developers to implement sophisticated application-specific UI behavior on the server
side. OpenXUP APIs are designed to be familiar, closely resembling the APIs from
desktop GUI toolkits. This allows developers to quickly migrate their existing desktop-
based applications. In addition, since all application code resides on the server side, it
9
http://en.wikipedia.org/wiki/Representational_State_Transfer
8
makes web applications easier to debug and maintain, without the need to worry about
issues from distributed computing.
OpenXUP is very extensible in that it does not dictate a particular UI model. It places no
restriction on the UI control set, the properties or events associated with each control, or
the style or appearance of the UI. That is, OpenXUP can practically work with any UI
model that has an XML-based representation.
Finally, web applications built with OpenXUP are fully compatible with existing
backend technologies (EJB10, CORBA [OMG08], COM11, etc.), since OpenXUP's server
side is designed to run within existing, established web application servers. This allows
OpenXUP-based applications to leverage all existing backend data and business logic
components.
Thin client. OpenXUP employees a thin client design while at the same time providing
rich UI to end users. Similar to browsers, OpenXUP's client side remains thin in terms
of application logic; that is, no application code is executed on the client side. The client
renders UI and processes native events, but leaves application-specific logic to the
server side. However, the client takes advantages of the desktop computing power to
enable a rich and interactive user experience for end users. It fully leverages the rich UI
capability offered by native desktop GUI toolkits such as Windows Forms and Java
Swing, while at the same time maintains a small footprint.
As the client is thin, the security risks associated with executing downloaded code,
whether binary or script, are avoided all together. This ensures that end users will enjoy
a rich UI experience in a safe client environment.
Implementation. To validate our approach, we provide an implementation of the

proposed RIA framework, which includes a full implementation of the XUP protocol, a
.NET-based server-side runtime environment, a set of event-driven APIs for application
development, and a thin client that can be executed either as a standalone application or
10
http://java.sun.com/products/ejb/
11
http://www.microsoft.com/com
9
as an Internet Explorer plugin. The implementation prototype fully leverages industry

standards such as SOAP and XML.
We also developed a lightweight UI modeling language called Simple User Interface

Language12 (SUL) in order to build some sample applications. However, since
OpenXUP is completely independent of the actual UI model, any UI model with an
XML representation can be used (e.g., XUL13 [Goo01], XAML [MS07b], and UIML14
[Abr99]).
1.2.2 UI Integration
To further simply the development of web applications with sophisticated user
interfaces, we propose a UI integration framework (called Mixup) aiming at the
development of composite applications by reusing heterogeneous UI components.
UI component model. Aiming at combining simplicity with effectiveness, we propose a

UI component model to represent the presentation front-ends of existing web
applications or modules. The key observations are that UI components require 1) a
conceptual, application-specific notion of state (e.g., the location and the zoom level for
street maps), 2) operations to request state changes, 3) events to notify state changes that
are primarily caused by user interactions, and 4) layout and appearance characteristics to
give a consistent look and feel to the composite application.
The proposed model is abstract, meaning that it is not tied to specific implementation
technologies. As a result, it can be used to describe existing UI components developed
with heterogeneous component technologies. This allows us to model UI components as
services, where an abstract UI component may have multiple bindings to native
component implementations.
To describe UI components, we propose the UI Service Description Language (UISDL),

which models a UI component as a service by describing both the abstract component
12
http://openxup.org/TR/sul.pdf
13
http://www.mozilla.org/projects/xul
14
http://www.uiml.org
10
model and its bindings to concrete implementations. That is, a UISDL document
contains the events, operations, and properties of a UI component, as well as their
bindings to native implementations. The design of UISDL follows closely to WSDL
[Chr01][Chi07a].
UI composition model. Aiming at integration in the presentation layer, we propose an

event-based composition model, as we believe that UI integration is mostly event-driven
(i.e., driven by user interactions).
The composition model includes event subscription information to facilitate the

communication among UI components, in the form of event listeners, where each
listener maps an event from one component to an operation of another component. For
cases where event-based one-on-one mapping is insufficient, additional integration logic
(e.g., sequencing and flow control) may also be specified in the form of simple scripts or
references to external code. This allows script-based process logic to complement with
event-based, declarative integration logic.
In addition, when direct mappings between event parameters and operation inputs are
infeasible, additional data mappings and transformations can be specified in XSLT
[Cla99b] within event listeners. Finally, the composition model also includes layout and
positioning information so that the UI components can be positioned properly in the
composite application.
To describe the UI composition model, we propose the eXtensible Presentation

Integration Language (XPIL), which is an XML-based language for modeling composite
applications. XPIL is declarative, since UI integration logic is primarily event-based;
this in turn makes it easy to author and interpret.
UI consolidation and embedding. Since composite applications are built with

components from various sources, very often two or more UI components in the same
composite application may overlap in content or functionality. Therefore, to ensure a
coherent presentation of the composite application, we have devised a content
consolidation mechanism to merge, propagate, or hide semantically identical UI content
from different UI components.
11
In a composite application, it is often necessary to place a component inside another. For

example, in a real estate application, it is desirable to place a small thumbnail image of
the selected property on a map showing the property location. Here, the thumbnail
image is provided by a real estate listing component and the map is provided by map
services such as Google Maps. Hence, we propose a component embedding mechanism
that allows one component to be embedded into another, where the two components
may not be aware of each other a priori.
Runtime middleware. In conjunction with the UI composition model, we provide a

lightweight middleware for the execution of composite web applications. The runtime
middleware interprets the XPIL document containing the composition logic and the
UISDL documents that represent the involved UI components. At runtime, the
middleware facilitates the interactions among the components by capturing events fired
by one component and dispatching them to operations of subscribing components.
Component adapters and inspectors. In order to support heterogeneous components,

the runtime middleware supports the notion of component adapters, which allow the
middleware to communicate with components from different component technologies.
Using these adapters, the middleware will permit the integration of UI components
developed using a wide variety of technologies, as long as the corresponding component
adapters are available.
This reinforces the notion of UI as a service, where components adapters facilitate the
bindings from the abstract UI component model to concrete native component
implementations. That is, the abstract model of a UI component is bound to one of its
bindings through an appropriate component adapter at the runtime.
On the development side, we introduce the notion of component inspectors, which allow
the automatic generation of component descriptors (i.e., UISDL documents) from native
UI components. As long as the appropriate meta-language facility (e.g., reflection) from
a component technology is available (e.g., Java, .NET), a component inspector can be
used to find out a legacy component's native events, operations and properties and then
generate a component descriptor with appropriate bindings to the native component
implementation.
12
Implementation. To validate our approach, we provide an implementation of the

proposed UI integration framework, Mixup, which includes a runtime middleware for
the execution of composite applications and a development environment to facilitate the
application design and development process.
The runtime middleware is a lightweight JavaScript library that can be executed in any
standard browser. It instantiates UI components defined in UISDL documents and
coordinates their interactions according to the composition logic specified in the XPIL
document.
We provide two development tools to assist the building of composite applications. The
first tool is an Eclipse GEF15 based visual editor. It allows developers to efficiently
create component descriptors (i.e., UISDL documents) and composition model (i.e.,
XPIL document) in a drag and drop fashion. The second tool is web-based (AJAX) and
serves the same purpose. Since it can be executed in the browser, no software
installation is necessary. As a result, the AJAX version is more suitable for tech-savvy
end users who demand quick and easy access, whereas the Eclipse version may be ideal
for developers who require maximum features with efficiency.
1.3 Thesis Organization

The remainder of this thesis is structured into two parts. In Part 1, we discuss our
proposed RIA framework, OpenXUP, and in Part 2, we present our UI integration
framework, Mixup.
In Part 1, we start with a discussion of the current state of art of rich internet
applications in Chapter 2. We identify several dimensions which can be used to
characterize different RIA approaches, followed by a survey of related work using the
dimensions as a guideline. Next, we present the details of the proposed RIA framework,
OpenXUP. In particular, we discuss the UI protocol, XUP, in Chapter 3, the server-side
runtime / development environment and its implementation in Chapter 4, and the design
and implementation of the thin client in Chapter 5.
15
http://www.eclipse.org/gef/
13
Part 2 focuses on the proposed UI integration framework, Mixup. Chapter 6 discusses

the current state of art of UI integration, in both research and commercial development.
We identify several dimensions which can be used to characterize different UI
integration techniques, followed by a survey of several representative approaches in the
field, using the dimensions as a guideline. Next, we present the details of the proposed
UI integration framework, Mixup. First, we describe the abstract UI component model
together with the concept of UI as services in Chapter 7, followed a thorough discussion
of the event-based UI composition model in Chapter 8. The UI composition model also
includes content consolidation and component embedding mechanisms to enable a
seamless integrated UI. After that we describe the overall UI integration framework in
Chapter 9, which includes a lightweight runtime middleware to execute composite
applications, as well as a graphical development environment to facilitate rapid UI
composition. In the same chapter, we also present the notion of component adapters to
support legacy, heterogeneous components, and the notion of component inspectors to
support the automatic creation of component descriptors from native component
implementations. We then conclude the chapter by a discussion of the framework
implementation.
Finally, in Chapter 10, we give concluding remarks of this thesis and discuss possible
directions for future work.
Part 1: OpenXUP – an RIA Framework
Part 1: OpenXUP 17
Chapter 2. Background and State of Art in RIA
2. Background and State of Art in RIA

RIA has become a key component of Web 2.0, as more and more users demand rich web
user interfaces which may greatly improve web applications' usability and productivity.
As a result, numerous RIA technologies with varying architectures have emerged in both
research and industry, resulting different features and functionalities.
In this chapter we present the current state of art in rich internet application
development, by illustrating the features and differences, strengths and weaknesses of
several leading RIA approaches. First, we give an overview of the field, followed by a
set of dimensions that can be used to compare and characterize related researches and
technologies in this area. Finally, we discuss related work in this area using the
dimensions as guidelines.
2.1 Dimensions for Characterizing RIA Technologies

To discuss related technologies in the RIA field, we need a set of dimensions to
compare them. First, RIA technologies have varying degrees of usability and
productivity. Second, the location of UI code dictates the architecture of the solutions,
with impact from security, communications, to intellectual property protections. In
addition, the support of asynchronous UI allows user interfaces to be more interactive,
without having users "wait" for the UI. Finally, the development environment dictates
how easy RIA applications can be developed; this includes, for example, choices of
programming languages and APIs.
2.1.1 Usability and Productivity

The primary purpose of rich internet applications is to improve the usability and
productivity of web applications. By usability, we mean whether a user interface is easy
to learn and operate. A highly usable interface is thus intuitive and allows users to start
using the application without extensive initial training [BV03]. Similarly, productivity
implies whether a user interface allows users to work with the web application
efficiently [Bai88]; i.e., to achieve a task with the minimum number of mouse clicks
and/or keystrokes.
Part 1: OpenXUP 18
UI conventions and metaphors improve a user interface's usability and productivity and
reduce its learning curve. There are many established UI conventions in desktop UI
[Mye00]. For example, if a dialog box in Windows contains a "Yes" and a "No" button,
users can either click on the buttons to invoke the desired functionality, or they can use
keyboard shortcut, the "Enter" key for "Yes" and the "Escape" key for "No" (see Figure
2.1). Similarly, many keyboard shortcuts help users navigate menus and edit text (e.g.,
"control-c" and "control-v" to copy / paste text).
Figure 2.1 Standard dialog
This type of conventions is much less established in web UI. Very few web applications
support keyboard shortcuts. Some newer AJAX-based [Gar05] applications do support
them, but have very different conventions. For example, GMail16 and Yahoo Mail17
support keyboard shortcuts, but each with very different conventions. As a result, the
keyboard shortcuts offered by these applications are only used by a limited number of
power users.
Desktop UI is very mature as a result of many years of research and development.

Desktop-based UI toolkits offer many rich, powerful, and standardized UI controls
which users are very familiar with. Examples are tree, combo box (editable list), slider,
and context menu. Therefore, to provide a high level of usability and productivity in
web applications, users should be able to interact with a set of rich UI controls
equivalent to those found in desktop applications.
16
http://gmail.com
17
http://mail.yahoo.com
Part 1: OpenXUP 19
2.1.2 UI Code Location

We use the term UI code to refer to the code that renders UI (i.e., UI definition) as well
as the code that handles UI events caused by user actions (i.e., UI logic). For web
applications, the code that renders UI is usually in the form of markups (e.g., HTML),
whereas UI logic code must be programmed in script or traditional programming
languages.
UI code can be either located on the client side or on the server side. UI definition is
processes at the client side since the rendering of UI markups is usually performed by
the web browser. It is the location of UI logic code that determines the architecture of a
RIA solution.
When UI logic is located on the client side, the code needs to be downloaded to be
executed by the client. The execution of downloaded code may impose security risks to
the end user's computer. Therefore, both .NET and Java have sophisticated security
sandbox to restrict the execution of downloaded code. In addition, for trusted execution,
downloaded code must be certified by public CAs (certification authority). On the other
hand, if UI logic code is located on the server side, client-side security is no longer a
concern as no downloaded code needs to be executed by the client.
UI logic code on the client side needs to handle issues arisen from client/server
communications. Typically, the client-side UI logic code employs some form of RPC
mechanism to communicate with the server-side application logic code. During this
process, the UI code must catch any exceptions caused by network issues. On the other
hand, if UI logic code is located on the server side, the communication between UI logic
and application logic is much more reliable, so developers do not need to worry about
network issues in the UI logic code.
Finally, if UI logic code is downloaded to the client side, the intellectual property (IP)
associated with that code needs to be carefully evaluated. If the downloaded code is
script-based, IP cannot be protected since anyone can view the source code in the
browser. Binary-based byte code (e.g., Java classes and .NET assemblies) could be
Part 1: OpenXUP 20
decompiled, so the associated IP could also be revealed. However, IP protection

becomes a non-issue if UI logic code is on the server side, as the code is never exposed.
2.1.3 Asynchronous UI
Users expect their applications to be very responsive. However, many applications block
users from further interactions while performing computations (e.g., by presenting an
hour glass cursor to the users). To make applications more responsive and interactive,
some form of asynchronous mechanism must be employed to perform UI operations
while computing in the background.
Asynchronous UI makes applications to appear more responsive, but it adds complexity

in application development. Applications must support background computation via
multi-threading or callback functions, both of which require additional effort on the
developer's part.
2.1.4 Development Environment

An application development environment offers languages and APIs to create rich
internet applications. Both traditional languages such as Java or C# and scripting
languages such as JavaScript can be used to program rich UI behavior.
Traditional languages may be more appropriate for large scale development, since they
are more structured and have mature tool support. In addition, it is likely that developers
can use the same programming language to code both UI logic and application logic,
allowing an easier integration. Scripting languages are less verbose and easy to get
started. However, large of amount of scripts is hard to maintain, and therefore may not
be suitable for applications with complex UI logic.
APIs play an important role in creating rich UI behavior. Since UI logic mainly consists
of the handling of UI events caused by user actions, the API should be event-driven,
reflecting the nature of rich user interfaces. In order to support rich and highly
interactive UI, the API should support the manipulation of rich UI controls and the
handling of their associated events, similar to those found in desktop-based GUI
toolkits. This also brings familiarity to developers who have experiences in desktop GUI
Part 1: OpenXUP 21
programming. Finally, the API should be extendable, so that developers can create their
own custom UI controls.
2.2 RIA Technologies

A large number of RIA technologies exist to aid the development and deployment of
web applications with rich and interactive UI. In this section, we attempt to group them
into several categories and compare their strengths and weaknesses along the
dimensions introduced earlier. Although classic HTML-based web applications cannot
be regarded as rich internet applications, they serve as a baseboard upon where other
RIA technologies can be compared. After all, the primary goal of RIA technologies is to
overcome the limitations of classic HTML-based applications.
2.2.1 Classic HTML

Traditionally, web applications are built with HTML pages. We call this the classic
HTML approach, where web pages are generated on the server side, with some
manually inserted JavaScript code for data validation. Each user interaction (e.g.,
clicking on a link or a button) in the browser results in an HTTP request, which in turn
causes a new page to be generated and returned.
This page-based model lacks true user interactivity, since HTML was designed
explicitly for presenting passive documents and therefore lacks many useful user
interface controls. In addition, frequently page refreshes are very annoying to end users,
since it takes time to reload a web page even if only a very small portion of the user
interface needs to be updated. Also, this wastes network resources since a full web page
is always downloaded after each user interaction. Overall, user interfaces developed
using the classic HTML approach lack both usability and productivity.
Any programming languages can be used in this approach. However, the API is page-
based. That is, the API revolves around how to efficiently generate HTML pages, mostly
based on a combination of templating mechanism with embedded code. Examples are
JavaServer Pages18 (JSP), Active Server Pages19 (ASP), and PHP20. The page-based
18
http://java.sun.com/products/jsp
Part 1: OpenXUP 22
programming model does not capture the notion of UI events, therefore cannot provide a
high level of interactivity to end users.
Since UI logic is programmed at the server side, no code needs to be downloaded and
executed by the client browser. Therefore, client-side security is not a concern. In
addition, since both UI logic code and application logic code reside on the server side,
developers do not need to worry about communication issues between the two. Finally,
IP protection is not an issue because UI logic code is kept on the server side and never
exposed to the users.
The classic HTML approach does not support asynchronous UI. Users must not act on
the UI before the browser finishes the ongoing request / response cycle. If the user
attempts to interact with the application (e.g., clicking on a link or pressing a button),
the current request / response cycle will be interrupted, resulting adverse effect on the
application. While terminating a search may not cause much damage, interrupting a
credit card transaction may result the user's credit card been charged without booking
the order into the seller's inventory system.
2.2.2 Rich Client Technologies

To overcome the limitations in classics HTML-based applications, a slew of client-side
technologies have emerged. They are typically deployed as browser plug-ins and require
downloaded UI code to be executed on the client side.
ActiveX 21
is one of the first technologies that allow downloadable code to be executed
inside browser. An ActiveX control is typically packaged in a CAB [MS05] file that is
downloaded on demand, while the browser renders the HTML file containing reference
to the CAB file. Once downloaded and instantiated, an ActiveX control may execute
any native Windows code. Therefore, an Active control may contain any UI controls
provided by Windows, resulting a rich and interactive UI that's identical to Windows'
desktop UI experience.
19
http://www.asp.net/
20
http://www.php.net/
Part 1: OpenXUP 23
Since an ActiveX control allows any native code to be executed on the user's computer,
it can cause many harmful effects. Therefore, Internet Explorer requires downloaded
binaries to be signed by a trusted public certificate authority22 (CA). At runtime, the
certificate issued to the ActiveX control will be presented to the user, asking her
permission to install / execute this control. However, the user usually has no idea of
what's inside the control, so she can either reject it or blindly grant the permission. Once
granted, the downloaded code can practically do anything to the user's computer.
ActiveX mixes the UI definition code that renders the UI and the UI logic code that
provides UI behavior in response to user actions. Both are programmed in whatever
programming language used to implement the ActiveX control. This makes UI
appearance customization less flexible, as it requires UI source code to be changed and
recompiled.
Since ActiveX controls are on the client side, developers must explicitly handle client /
server communications. Although ActiveX controls are downloaded by the browser and
exposed to the end users, IP protection is not a concern since they are compiled machine
binaries.
On the development side, ActiveX controls can be created using many traditional
programming languages, such as C++ and VB. ActiveX controls can access any APIs
provided by the desktop operating system, including Windows' graphical libraries. The
ActiveX API itself is extendable, allowing developers to create custom controls. Finally,
ActiveX does not offer any explicit mechanism for asynchronous UI, so developers must
use callback functions or multiple threads to support that.
Java Applet 23
is Java's alternative to ActiveX controls. It allows Java code to be
executed inside the browser. Instead of native machine code, a Java applet consists of
Java classes made of high-level, platform-neutral byte code. Java applets must be
21
http://msdn.microsoft.com/en-us/library/aa268985.aspx
22
http://en.wikipedia.org/wiki/Certificate_authority
23
http://java.sun.com/applets/
Part 1: OpenXUP 24
executed by the Java browser plug-in, which is available as part of the Java Runtime
Environment (JRE) installation.
Since Java applets have access to most of Java's desktop GUI APIs, they may provide
the same level of usability and productivity as desktop Java applications. Specifically,
the APIs are event-driven, and include a comprehensive set of UI controls for building
rich and interactive user interfaces. The APIs are also extendable – developers can build
custom UI controls based on one or more built-in Java UI controls. Java has good
support for callback methods and multi-threading, so developers can take advantage of
those facilities to implemented asynchronous UI.
On the security side, Java Applet provides a sandbox mechanism in addition to signature
from a trusted public CA. The security sandbox restricts what APIs can be called by the
downloaded code, so that applets cannot perform harmful operations (e.g., removing
local files) on the user's computer.
Similar to ActiveX controls, Java applets mix UI definition code and UI logic code, both
of which must be programmed in Java. As a result, applet source code must be changed
and recompiled in order to alter or customize the UI appearance of an applet.
Since applets reside on the client side, developers must manage client / server
communications through RMI24 or other messaging facilities. Although applets are in
binary form, they can be easily decompiled. Therefore, developers must ensure that
valuable IP is not exposed through downloadable applets.
.NET Smart Client [Hil04] offers several improvements to ActiveX. It allows .NET
code to be executed inside the browser. Similar to Java applets, a .NET smart client
contains high-level byte code rather than native machine code. The byte code is stored in
one or more .NET assemblies (DLL files), which are executed by the .NET Common
Language Runtime (CLR). However, .NET CLR is only pre-installed on newer versions
of Windows (i.e., Vista), so users with older Windows must manually install CLR.
24
http://java.sun.com/javase/technologies/core/basic/rmi/
Part 1: OpenXUP 25
Smart clients have access to the .NET class libraries, including .NET Windows Forms.
As the APIs provided by Windows Forms are event-driven and include a comprehensive
set of UI controls, smart clients can offer a rich and interactive web UI on par with
desktop UI. In addition, developers can create custom UI controls by extending from the
built-in UI controls provided by Windows Forms. Since .NET has comprehensive
support for callback methods and multi-threading, developers can leverage those
facilities to implemented asynchronous UI, providing a more responsive user
experience.
To mitigate client-side security risks, .NET provides Code Access Security25 (CAS)
through a complex set of security APIs to restrict what can be done by the downloaded
code. However, since the security APIs are very complex, they add additional challenges
to the already challenging task of rich web UI development.
Similar to ActiveX controls and Java applets, smart clients mix UI definition code and
UI logic code, both of which must be programmed in languages supported by .NET
(e.g., C#, VB.NET, C++). As a result, the source code of a smart client must be changed
and recompiled in order to alter or customize the UI appearance of the smart client.
Since smart clients reside on the client side, developers must manage client / server
communications through messaging facilities such as .NET Remoting [OH01] or
ASP.NET Web Services26 [How01]. Similar to Java applets, although in binary form,
.NET assemblies can be easily decompiled. Therefore, developers must ensure that
valuable IP is not exposed through downloadable .NET assemblies.
Flash 27
is a popular browser plug-in that was originally designed to render animations
(i.e., vector graphics) in browsers. More functionality was added later to provide rich UI
functionalities. Due to its ubiquitous status among browsers, quite a few RIA
frameworks have been built on top of Flash; for example, Flex28 and OpenLaszlo29. All
25
http://msdn.microsoft.com/en-us/library/930b76w0(VS.80).aspx
26
http://msdn.microsoft.com/en-us/library/t745kdsh.aspx
27
http://www.adobe.com/products/flashplayer/
28
http://www.adobe.com/products/flex/
29
http://www.openlaszlo.org/
Part 1: OpenXUP 26
these frameworks leverage the Flash browser plug-in to render rich UI and provide user
interactions.
Although Flash-based frameworks provide many rich UI controls, the look and feel is
very different from standard desktop UI controls such as those from Windows Forms
and Java Swing. A new look and feel may appear attractive, but the disadvantage is that
users would have to learn a set of new UI conventions and metaphors. In addition,
keyboard shortcuts are not adequately supported, which negatively impacts the user's
productivity.
Developing Flash-based UI involves creating two types of files, MXML [Coe03] (or
LZX30 for OpenLaszlo), an XML-based declarative language for modeling UI, and
ActionScript31, a JavaScript-based language provided by Adobe (previously
Macromedia). MXML files contain the UI definition and ActionScript files contain the
UI logic. Since the Flash browser plug-in cannot direct interpret MXML or
ActionScript, they must be compiled into an SWF32 file, which can then be executed by
the Flash plug-in. Separating UI definition from UI logic allows an application's UI
appearance to be changed without affecting its UI logic. However, since MXML and
ActionScript are typically compiled into a single SWF file, changing either one will
require a recompilation of both.
With Flash-based applications, both UI definition and UI logic code (in the compiled
form) reside on the client side. As the UI logic code may contain any ActionScript,
client-side security becomes an issue just like Java applets and ActiveX controls. In
addition, developers must manage client / server communication via messaging facilities
such as BlazeDS33. Although the SWF file format is binary, it can be easily decompiled
to reveal ActionScript source code. Therefore, developers must ensure that valuable IP
is not exposed through compiled ActionScript in downloadable SWF files.
30
http://www.openlaszlo.org/lps/docs/reference/
31
http://www.adobe.com/devnet/actionscript/
32
http://www.adobe.com/devnet/swf/
33
http://opensource.adobe.com/wiki/display/blazeds/
Part 1: OpenXUP 27
Flex allows UI definition to be specified in MXML and UI logic to be programmed,

whereas OpenLaszlo uses a combination of LZX and JavaScript. The APIs are
extendable so that custom UI controls can be added by developers. Since Flash runtime
does not support multi-threading, developers must manually divide lengthy computation
tasks and use callbacks to achieve asynchronous UI.
Silverlight 34
is Microsoft's alternative to Flash. It is deployed as a browser plug-in and
provides rich user interactions with sophisticated graphical and multimedia capabilities.
Comparing to Flash, Silverlight has a relatively small deployment base, since it is still in
early product lifecycle and has limited platform / browser support.
Developing Silverlight applications involves coding in XAML for UI definition and

JavaScript for UI logic. Unlike Flash, Silverlight can directly interpret XAML and
JavaScript. JavaScript code can be either embedded in XAML document or can be
referenced externally. Separating UI definition from UI logic allows an application's UI
appearance to be changed without affecting its UI logic.
XAML provides a rich set of UI controls. In addition, starting from version 2.0,
Silverlight can execute any downloaded .NET assemblies (subject to the same security
restrictions as .NET smart clients). This enables developers to use rich UI controls from
both XAML and .NET Windows Forms libraries, resulting a high level of usability and
productivity similar to desktop applications.
With Silverlight-based applications, both UI definition and UI logic code reside on the
client side. As the UI code may consist of JavaScript and downloaded .NET assemblies,
client-side security becomes an issue just like Flash and .NET Smart Client. In addition,
developers must manage client / server communication via messaging facilities such as
.NET Remoting or ASP.NET Web Services. Since JavaScript is in source code form
and .NET assemblies can be easily decompiled, developers must ensure that valuable IP
is not exposed through downloaded JavaScript and .NET assemblies.
On the development side, Silverlight supports JavaScript for coding the application's UI
logic. With the ability to execute .NET assemblies in Silverlight 2.0, any .NET
Part 1: OpenXUP 28
languages can be used to develop rich UI in Silverlight. As for API, applications can use
JavaScript to access and manipulate XAML's Document Object Model (DOM) [Hor00],
as well as the HTML DOM [Ste03] in the containing browser. In addition, Silverlight
2.0 applications may call any .NET libraries, including those providing rich user
interactions. Both the JavaScript API and the .NET API are event-driven and can be
extended to create custom UI controls. For Silverlight 2.0 applications, .NET's
comprehensive support for callback methods and multi-threading can be leveraged to
implemented asynchronous UI, providing a more responsive user experience.
2.2.3 Server-side Approaches

Since the client-side technologies mentioned above all require users to install browser
plug-ins to execute downloaded code, several server-side approaches have emerged that
allow UI logic be programmed at the server side, just like classic HTML applications.
Improving upon the page-based programming model, they offer abstractions that
represent rich UI controls and a set of APIs resembling their desktop counterpart.
ASP.NET Web Forms [She01] provides an event-driven programming model to page-

based applications. In addition to standard UI controls offered by HTML, it provides a
set of extended UI controls resembling the ones found in the desktop environment. The
goal is to provide developers with an API similar to desktop UI toolkits.
Since it ASP.NET runs on top of the .NET framework, any .NET programming
languages can be used to develop ASP.NET Web Forms applications. Its API is quite
extendable, so developers can easily create custom UI controls. In ASP.NET Web
Forms, UI definition is provided in ASP pages while UI logic primarily consists of event
handlers that can be programmed in any .NET languages. The separation of UI logic
from UI definition allows the UI appearance to be changed or customized without
altering the UI logic code (and vise versa).
Client-side security is no longer an issue since the UI logic code is on the server side
and consequently no downloaded code needs to be executed on the client side. In
34
http://silverlight.net/
Part 1: OpenXUP 29
addition, developers do not need to manage client / server communications since both
UI logic and application logic reside on the server side. IP protection is also not an issue
because no source code is exposed to the users.
Although ASP.NET Web Forms provides many desktop-like UI controls and an

associated event-driven programming model, it is still limited by the basic controls
offered by HTML browsers. This is because the rich UI controls are made of basic
HTML controls. For example, the calendar control is made of an HTML table
containing links to each day of the month. In addition, most user actions result in a
request / response cycle that returns a full HTML page, causing an annoying page
refresh. Furthermore, asynchronous UI is hard to implement as now the UI logic runs on
the server side, so the developer has no control of the client side, unless using advanced
JavaScript techniques (e.g., AJAX). As a result, although it improves the web UI
development process by offering an event-driven API with desktop-like UI controls, the
end users' usability and productivity remain unchanged when comparing to classic
HTML-based applications.
JavaServer Faces 35
(JSF) provides a Java-based, extendable event-driven API for
developing web applications on the server side. A unique feature of JSF is that it offers
a set of abstract UI controls that can be bound to multiple concrete UI controls. For
example, the "UISelectBoolean" control represents a single boolean (true or false) value.
It can be rendered as a check box for one application and a toggle button or switch for
another application. Similarly, the "UISelectOne" control allows users to choose one
item from a collection of items, and it can be rendered as a dropdown list, a single
selection box, or even a menu. This means that JSF UI controls are abstract and can be
rendered in a variety of forms depending on the binding implementations.
With JSF, UI logic code resides on the server side, so client-side security is not an issue
since the client does not execute downloaded code. In addition, developers do not need
to manage client / server communications as both UI logic and application logic reside
on the server side. Since source code is never exposed to users, IP protection is no
longer an issue. In JSF, UI definition is provided in JSP pages while UI logic primarily
35
http://java.sun.com/javaee/javaserverfaces/
Part 1: OpenXUP 30
consists of event handling code in Java. The separation of UI logic from UI definition
allows the UI appearance to be changed or customized without altering the UI logic
code (and vise versa).
JSF's default HTML / JSP based implementation is very similar to ASP.NET Web
Forms, and therefore it suffers from the same HTML-based limitations, resulting low
usability and productivity. Similarly, asynchronous UI is hard to implement as the UI
logic runs on the server side, so the developer has no control of the client side without
using advanced JavaScript techniques (e.g., AJAX). Since JSF only standardizes on the
API (the abstract UI controls), it could be used with other rich UI languages (e.g., XUL)
and rich client technologies (e.g., AJAX). However, similar to ASP.NET Web Forms,
JSF still maintains the "page" concept, which is not ideal for highly interactive user
interfaces that are fluid and without page boundaries.
XForms [Boy07] improves upon traditional HTML Forms36 by providing better client-
side events and data validations. XForms applications can be developed in a variety of
programming languages at the server side. However, the APIs remain page-based,
similar to page-based APIs in classic HTML applications.
As XForms provides comprehensive client-side events and data validations, there is

very little JavaScript required on the client side. Consequently UI logic code resides on
the server side, so developers do not need to worry about client-side security, client /
server communication, or IP protection issues. With XForms, UI definition is provided
in XML, and UI logic is programmed in whatever language supported by the server
runtime. So it is easy to change or customize an application's UI appearance without
affecting its UI logic.
XForms inherits HTML's page-based model; i.e., the response of a form submission
either replaces the current form or the entire containing page. In addition, it only offers a
limited set of UI controls and asynchronous UI is not supported. Therefore, for end
users, the usability and productivity remain low, while for developers, it is still hard to
build rich web user interfaces with XForms.
36
http://www.w3.org/TR/html401/interact/forms.html
Part 1: OpenXUP 31
2.2.4 AJAX
AJAX (asynchronous JavaScript and XML) [Gar05] is a development technique that
heavily utilizes client-side JavaScript to enhance the richness and interactivity of
HTML-based applications. Its core is the XMLHttpRequest37 object (XHR), which
allows web applications to retrieve data from the server asynchronously in the
background without blocking users from interacting with the existing page. During
runtime, the base HTML page is first rendered by the browser. Subsequent user
interactions trigger client-side JavaScript code to be executed to update the UI by
manipulating the HTML DOM. At the same time, data is retrieved using the XHR
object behind the scene (i.e., asynchronously) in the form of XML or JSON38.
Figure 2.2 Architecture of AJAX applications [Gar05]
37
http://www.w3.org/TR/XMLHttpRequest/
38
http://www.json.org/
Part 1: OpenXUP 32
Another advantage of AJAX is that now UI can be updated incrementally, without

annoying page refreshes. In fact, most user interactions only result a small portion of the
page to be updated – this is achieved through DOM manipulation via client-side
JavaScript. Using XHR, data communication with the server happens behind the scene,
so users are not blocked from further interacting with the UI. As a result, AJAX-based
applications offer improved usability and productivity over classic HTML applications.
There are a large number of AJAX frameworks and development toolkits, offering many
different APIs for building rich web user interfaces, some remain page-based, while
other are more event-driven. The dominant approach is to provide client-side JavaScript
libraries that offer rich UI controls and simplify some common tasks (e.g., hiding
browser incompatibilities and facilitating communications with the server). Developers
then write client-side JavaScript code that calls methods or functions from these
libraries when needed. Since JavaScript is hard to maintain, several AJAX toolkits
allow applications to be developed in traditionally programming languages but executed
on the client side. For example, Google Web Toolkit39 (GWT) allows developers to
create AJAX applications in Java. When the applications are deployed, GWT employs a
cross-compiler to translate the Java code into optimized JavaScript code that can then be
executed on the client side.
With AJAX, UI logic is programmed in JavaScript and resides on the client side, while
UI definition is provided in HTML and CSS [Bos98]. Although toolkits such as GWT
allow UI logic to be programmed in Java, it is still executed as JavaScript on the client
side at runtime. As a result, AJAX applications are subject to the same security risks
faced by other rich client technologies due to client-side code execution and JavaScript
vulnerabilities. Since the UI logic resides on the client side while the application logic is
on the server side, developers must manage client / server communications either
directly using XHR or through wrapper functions provided by AJAX toolkit libraries. In
addition, developers must take IP protection into consideration as JavaScript code can
be viewed by any users.
39
http://code.google.com/webtoolkit/
Part 1: OpenXUP 33
Although AJAX offers improved usability and productivity over classic HTML
applications, it still cannot compare with desktop-based UI. First, AJAX has no
consistent user interface guidelines. Applications built with one AJAX toolkit may
differ significantly from ones built with another toolkit. As a result, users are faced with
the challenge in learning a variety of new UI conventions and metaphors. In addition,
keyboard shortcut support is quite limited due to the potential conflicts with different
browser's built-in keyboard shortcuts. Although many AJAX applications provide
limited keyboard shortcuts to improve productivity, they are often ignored by the users
due to inconsistencies among applications and browser focus issues. Finally, lack of
consistent UI guideline also hampers the application's accessibility.
While AJAX promises to improve interactivity through asynchronous UI, JavaScript's

performance remains to be a major obstacle to this [Wei08]. For example, using
JavaScript to perform drawing tasks (e.g., vector graphics) or process large amount of
XML data will likely make the application less responsive to user input. In addition,
browser incompatibility remains to be an issue due to the lack of standardization of
JavaScript support in different web browsers and even in different versions of the same
browser. As an AJAX toolkit may not cover all browsers (or all versions of a particular
browser) required by the target application, the developer inevitably needs to perform
manual tweaking, resulting in high development cost.
2.2.5 Remote Display Technologies

Remote display technologies can be used to display remote UI and graphics on the
local desktop. Mature technologies such X1140, NeWS [Gos90], and Display PostScript
(DPS) [Ado93] offer low level UI protocols to transport bitmaps and low-level
primitives. In X11, everything is based on bitmaps and windows, so the protocol does
not offer higher level abstractions such as buttons or scrollbars. Rich UI controls are
instead provided by toolkit libraries (e.g., Motif41) that are built on top of low level
primitives. Usability and productivity are both high because these are essentially
desktop UI technologies. From the network's point of view, the local desktop is actually
40
http://www.x.org/
41
http://www.opengroup.org/motif/
Part 1: OpenXUP 34
the "server", since it must open a port to receive graphical instructions from remote
programs.
Applications can be developed in traditionally programming languages like C and C++,

and for NeWS and DPS, PostScript42 can be directly used. APIs are event-driven and
extendable. UI logic and UI definition are programmed together; there is no easy way to
separate the two. Asynchronous UI must be manually supported (e.g., by dividing up
long running tasks and using callback functions).
Figure 2.3 X Window System [GE06]
Since UI logic code resides remotely, there are no client-side code execution risks or IP
protection issues. In addition, since both UI logic and application logic reside remotely,
developers do not need to manage client / server communications themselves.
42
http://en.wikipedia.org/wiki/PostScript
Part 1: OpenXUP 35
However, since the local desktop must open a server port to receive remote graphical
instructions, it is subject to all kinds of network security attacks. In addition, the port
may be blocked by network firewalls, making it unsuitable for wide area networks such
as the Web. Furthermore, these protocols have very high network bandwidth
requirement, as typically every keystroke and mouse movement is transported over the
network. Therefore, these protocols only work well in a local area network environment,
where network resource is plenty.
Remote desktop technologies allow the entire remote desktop to be displayed on the
local desktop. Examples are Virtual Network Computing43 (VNC) and Remote Desktop
Protocol (RDP) [MS07a]. Their protocols have similar high bandwidth requirement.
However, unlike the remote display technologies mentioned above, they are not
designed for UI development, so they do not offer any development toolkits or UI
libraries. Their primary goal is desktop sharing and thin client computing. Desktop
sharing facilitates remote administration (e.g., server monitoring and technical support),
whereas thin client computing allows multiple thin clients to connect to the same server
to share its computing resources.
RemoteJFC 44
[Lok02] is a distributed user interface toolkit based on the JFC (Java
Swing) API. It facilitates the development of server-side applications by leveraging a
JFC-like remote API that displays its UI output on the local desktop. RemoteJFC has
two components: the client is a viewer with Java Swing look and feel; the server
contains application code, which uses an API that mimics JFC. Through the server-side
API, the application code processes UI events from the client and sends UI updates to
the client. Following the JFC model, UI logic and UI definition are programmed
together on the server side, so there is no easy way to separate the two.
RemoteJFC offers a high level of usability and productivity since the client renders the
UI using Swing, so its look and feel are identical to desktop-based Swing applications.
However, as applications have no direct control of the client, asynchronous UI cannot be
easily supported.
43
http://www.realvnc.com/
Part 1: OpenXUP 36
Since all UI code resides on the server side, RemoteJFC is not subject to client-side
code execution risks and IP protection issues. However, it uses RMI for network
communication, which is firewall unfriendly. In addition, it requires two-way RMI
servers. When the client sends UI events to the server, the server behaves as an RMI
server; when the server sends UI updates to the client, the client also becomes an RMI
server. This architecture poses serious security problems. Finally, RemoteJFC has very
high bandwidth requirement as the client may send a large amount of UI events to the
server, including high frequency ones such as mouse movement. Therefore, it is
unsuitable for deployment on wide area networks.
2.2.6 Rich UI Languages

There is a number of XML-based UI markup languages that aimed at providing desktop-
like UI controls not found in HTML. Examples are XUL, UIML, XIML45, XAML,
MXML, and LZX. They typically provide a set of rich UI controls with comprehensive
event support.
These UI languages allow UI definition to be specified declaratively in XML. To specify

UI logic, most of them allow scripts to be embedded in the markups to provide the
necessary UI behavior (i.e., handling events). For more extensive UI logic, the rich
client technologies mentioned earlier also allow external code (script or binary) to be
downloaded and executed.
These UI languages alone are not sufficient to implement rich UI experiences. They
must be used in conjunction with some client-side or server-side software (or protocol),
such as the various RIA technologies mentioned earlier in this section.
2.2.7 Comparisons
In this section we summarize and compare different web-based UI development
approaches. The following table provides a comparison of the aforementioned
approaches along the characteristic dimensions introduced earlier.
44
http://graphics.cs.columbia.edu/proj/monet/rjfc/html/
45
http://ximl.com/
Part 1: OpenXUP 37
Table 2.1 Comparison of web-based UI technologies
Usability and UI logic code location Asynchronous UI Development

productivity environment
Classic Low Server; separate UI No Page-based API
HTML definition and UI logic
Rich clients High Client; mixed UI Using callbacks or Event-driven API
(ActiveX, (desktop-like) definition and UI logic multiple threads with rich UI controls;
Java applet) traditional languages
Rich clients High Client; separate UI Using callbacks or Event-driven API
(Flash, (desktop-like) definition and UI logic multiple threads with rich UI controls
Silverlight)
Servers-side Low Server; separate UI No Event-driven API
approaches definition and UI logic
AJAX Medium to Client; separate UI Using XHR object Ad-hoc scripting API
high definition and UI logic
Remote High Remote; mixed UI Using callbacks or Event-driven API
display (desktop-like) definition and UI logic multiple threads with rich UI controls;
technologies traditional languages
The main advantage of the rich client technologies is their improved usability and
productivity on the user interfaces. However, since they must download and execute
code on the client side (whether binary or script), they all suffer from client-side code
execution risks and face IP protection challenges. As a result, many end users choose
not to install download code (e.g., applets) when prompted, even though they are
certified by public CAs. In addition, due to the same security concerns, many
organizations do not allow the installation of downloaded code, with a few exceptions
for well-known applications such as Acrobat PDF Reader.
The recommended practice in these rich client technologies is to have pure UI logic
code on the client side and application logic code on the server side. However, without
hard limit, there is a tendency for developers to mix UI and application logic code on the
client side, making the client side code (and the overall application) hard to maintain.
As complex UI implies a large UI logic code size (not to mention the possible client-
side application logic code), the downloaded code can quickly become quite bulky as
the application evolves. Slow download is annoying to end users for both first time
download and any subsequently code updates.
Part 1: OpenXUP 38
With the server-side approaches, there are no client-side security or IP protection issues
as no code will be downloaded and executed by the client. In addition, comparing to
classic HTML, they offer better development support since event-driven APIs are more
natural than page-based APIs. However, since they rely on basic controls in HTML, the
usability and productivity remain low.
Comparing with the rich client technologies, AJAX's major advantage is that no plug-in
needs to be installed. However, it is subject to similar client-side code execution and IP
protection issues, since UI logic code (i.e., JavaScript) must be downloaded and
executed on the client side. Similar to the rich client technologies, there is also a
tendency for developers to mix both UI logic and application logic code on the client
side, thus making the application hard to maintain. Although there are numerous AJAX
toolkits available, large amount of JavaScript remains to be a development challenge.
Finally, AJAX's usability and productivity cannot compare with those rich client
technologies or traditional desktop UI, due to the lack of consistent UI guidelines and
JavaScript's performance limitations.
The remote display technologies are very mature and have been widely deployed. They
offer desktop-grade UI with high usability. However, their high bandwidth requirements
and security restrictions call for well-controlled local network environment (e.g.,
enterprise LAN), making them unsuitable for wide area networks such as the Web.
2.3 Summary
In this chapter we have introduced some basic concepts in rich internet applications, and
presented the state of art in current research and industrial efforts. In particular, we have
provided an overview of the representative approaches, by illustrating their strengths
and weaknesses along a set of characteristic dimensions.
Our findings indicate that, although tremendous efforts and results have been made and
obtained in this area, RIA technologies are far from mature and face open research
challenges. In the next few chapters, we will discuss and present our solution to RIA.
Part 1: OpenXUP 39
Chapter 3. UI Transport Protocol
3. UI Transport Protocol
Traditional web applications are built with HTML pages. This page-based programming
model lacks true user interactivity, since HTML was designed explicitly for presenting
passive documents and therefore lacks many useful user interface controls. In addition,
the de-facto web UI protocol, a form submission protocol46 ("application/x-www-form-
urlencoded"), is very primitive. The request consists of a list of URL-encoded name /
value string pairs, and the response is always a full page of markup. As a result, in
HTML-based server-side applications, web pages are always reloaded after each user
click. This is very annoying to end users, since it takes time to reload a web page even if
only a very small portion of the user interface needs to be updated. Also, this wastes
network resources since a full web page is always downloaded after each user
interaction.
Many RIA technologies have emerged to improve user interactivities. The client-side
approaches do not need any standardize protocol for delivering UI, as UI is now
programmed at the client side. Typically some form of RPC is used by the client-side UI
logic code to communicate with the server-side application logic; however, this
communication has to be handled by the developer manually. Since these rich client
technologies (including AJAX) all resort to executing large amount of UI logic code in
the browser, client-side code execution security and IP protection become a major issue.
On the other hand, the server-side RIA approaches rely on the same form submission
protocol as traditional HTML applications, therefore suffering from the same set of
limitations.
Hence, in this chapter we propose an alternative UI transport protocol, the Extensible

User Interface Protocol (XUP), an XML-based protocol for communicating events and
user interface changes on the web. It lays a foundation for a framework that would
enable the development and consumption of highly interactive web applications.
With XUP, rich UI events are delivered from client to server as XML messages, rather
than URL-encoded name / value strings. Programmers implement event handlers on the
46
http://www.w3.org/TR/html401/interact/forms.html
Part 1: OpenXUP 40
server side. They no longer need to process form data as URL-encoded values. User
interface changes are delivered from server to client as incremental updates, so end users
will no longer experience slow page refreshes, and network bandwidth is conserved.
The remainder of the chapter is organized as follows. The next section describes the
concept of user interface model. After that we discuss the detailed design of the
protocol. Finally, we illustrate two simple examples to further clarify the concept.
3.1 User Interface Model

A user interface model (UI model) is a representation of the user interface which the end
user perceives and interacts with. Common UI models include .NET Windows Forms,
Java Swing, XUL, XAML, UIML, etc.
Some UI models have XML-based representations, such as XUL and UIML. We call
them declarative UI modeling languages (or just UI languages). That is, the UI model is
declared, not programmed. A UI model typically consists of a tree of UI controls (e.g.,
buttons, panels), a set of events (e.g., mouse click), and a list of resources (e.g., button
images, files to be downloaded or uploaded). Therefore, a declarative UI model can be
described by a tree of XML elements, with UI controls mapping to XML elements and
the properties (e.g., color, size) of the controls mapping to XML attributes47. One major
advantage of XML-based UI languages is that they can be edited with existing XML
authoring tools. It is also easy to develop UI design tools to further simplify XML-based
UI model creation.
Listing 3.1 shows an example UI model in SUL48. The root of the UI model is a window;
within it there is a panel which contains two push buttons.
<window id='w1' position="width:200;height:100"

xmlns="http://www.openxup.org/2004/02/sul">
<panel id='p1' border='single'...>

<button id='b1' ... >Refresh</button>
47
Within the context of XML, we will use the terms UI elements and UI attributes to refer to UI
controls and their properties.
48
http://www.openxup.org/TR/sul.pdf
Part 1: OpenXUP 41
<button id='b2'... >Update</button>

......
</panel>
......
</window>
Listing 3.1 UI model example (in SUL)
XHTML and WML [WAP01] can be also regarded as UI modeling languages, although
they are more suitable for presenting textual information rather than rich user interfaces.
XUP is a protocol for delivering events and user interface updates. Specifically, its
payload includes rich UI events caused by user actions and instructions to manipulate
the UI model. With extensibility in mind, XUP has been designed to work with any UI
models with XML-based representations. In fact, it is independent of the actual UI
model, and it places no restriction on the UI control set, or the properties or events
associated with each control.
An event model is a representation of events triggered by user interactions with the UI

model. The event model defines how events are fired, what types of events are fired, and
from which UI controls the events are fired in the UI model. The event model is often
part of or closely associated with the UI model.
XUP supports both delegation (e.g., Java Swing) [Sun97] and capturing/bubbling (e.g.,
DOM) [Pix00] event models, as long as they have XML-based representations. In a
delegation-based event model, an event can be sent to and processed by any interested
parties. To receive the event, one simply registers with the UI control that will be firing
the event. This type of event model is also called event subscription/notification. In a
capturing/bubbling based event model, an event will propagate up and down the UI tree,
starting from the source UI control that fired the event. Any UI controls along the
propagation path may receive and process the event. Delegation-based event model is
often used in desktop UI frameworks, such as Java Swing and .NET Windows Forms,
whereas capturing/bubbling based event model is typically used in document-based
applications, such as HTML/XML DOM.
To accommodate a wide range of application scenarios, XUP has been designed to work
with both types of event models. In addition, it does not dictate any event details, such
Part 1: OpenXUP 42
as what types of events may occur on which UI controls, or the syntax and semantics of
any particular event.
3.2 Protocol Design

To avoid client-side code execution security risks and IP protection issues, the protocol
must allow all application-specific code (i.e., UI logic code) to reside on the server side.
The client-side software should only render declarative UI definition code (i.e.,
markups). This leads to the following design goals:
• Facilitating server-side event processing: protocol requests should contain rich

UI events encoded in XML. In addition, to minimize network event traffic, the
protocol needs to support an event selection/filtering mechanism. To provide a
responsive UI experience, it must support the asynchronous delivery of UI
events.
• Delivering UI updates: protocol responses should contain instructions to update

the client-side UI model. In addition, to enable a fluid user interface, the protocol
needs to support the incremental delivery of UI updates.
In the following we provide an overview of the various elements of XUP and the
interactions among them.
3.2.1 Definitions
Before describing protocol operations, we need to make the following definitions:
An XUP client is a program that sends event requests and processes server responses on
behalf of the end user. The client renders user interfaces and interacts with the end user.
An XUP server is a program that processes event requests from the client and sends
back UI update in responses. It processes UI events and generates UI updates by
executing application-supplied code.
Part 1: OpenXUP 43
An XUP application is an application-supplied code that resides in the server and

provides desired functionalities to the end user through the client. The application uses
an API provided by the server to process events and manipulate UI.
An event handler is part of the XUP application. The server invokes the application by
calling registered event handlers upon receiving an event request from the client.
An event selector defines event selection criteria in the client. To minimize network
traffic, not all client-side UI events are delivered to the server – only those matched by
event selectors are sent.
3.2.2 Protocol Operations

An XUP client communicates with an XUP server by sending events encapsulated in
XUP requests. The server then handles the events and returns UI updates (and event
selectors) in XUP responses.
Figure 3.1 Protocol message exchange
An end user interacts with an XUP application by operating a client which sends events
to the XUP server hosting the XUP application. XUP applications are developed by
programmers to provide desired application functionalities for end users. An XUP
server may host one or more XUP applications which may be running concurrently.
Part 1: OpenXUP 44
Similarly, an end user may use a client to interact with more than one application from
one or more servers at the same time.
In this section, we describe protocol message exchanges and the interactions among
different components as a result of the message exchanges (Figure 3.1). First, the end
user starts by establishing a session with an XUP application. Second, the end user
interacts with the application which causes events and UI updates to be exchanged
between the client and the server. Finally, the end user terminates the session with the
application.
3.2.2.1 Startup
The client starts an application by sending a request to the URL identifying the server.
This request includes the name of the application and a list of UI model namespaces
supported by the client. The server then responds by sending a session ID used for
identifying the client in subsequent request, the initial UI model, and an initial list of
event selectors. The server maintains a user session for each client, identified by a
session ID49. The namespace of the initial UI model must be in the list of UI model
namespaces supported by the client.
The purpose of the UI model namespace negotiation is to allow the client and the server
to agree upon a common UI language. Both the client and server may support more than
one UI languages (e.g., the client may support SUL and XAML, while the server may
support UIML and SUL), but they must agree on one of them in order to execute the
application.
The client interprets this startup UI model and renders its initial UI accordingly. The
event selectors will be used by the client to deliver events in subsequent requests.
3.2.2.2 User Interactions

After the startup phase, the end user interacts with the application by manipulating the
UI model, which triggers events to be sent to the server. An event is sent to the server
49
The protocol does not specify how the server or the application should maintain user sessions, nor
the contents of user sessions.
Part 1: OpenXUP 45
only when it matches with an event selector in the client. Together with the event, a list
of UI data is also sent. Those UI data reflect changes made by the end user to the UI
model, such as text entered in a text box and a check box been unchecked.
Upon receiving an event request from the client, the server locates the XUP application
and invokes the event handlers registered by the application for the event. The UI data
sent with the event are also available to the event handlers. Event handlers execute the
necessary UI logic code and generate UI model updates (and selector updates, if any) to
be returned to the client. If some errors occurred, a SOAP50 fault is returned in place of
the UI model updates.
The client processes event response by rendering UI changes and updating its list of
event selectors. The UI model changes include both UI element51 level and attribute52
level changes, and the UI elements could be simple UI controls or complex UI
containers. This approach allows applications to perform both macro and micro UI
updates, eliminating end users' visual discomfort with frequent page refreshes as in
HTML-based applications. As shown in Table 3.1, XUP supports fine-grained updates
to the UI model. For example, <addUIElement> may be used to add a list item to a list
box, and <updateUIAttr> may be used to update the background color of a button.
Table 3.1 XUP protocol elements for incremental UI updates
<xup:addUIElement> Add a UI control under a specified parent control

<xup:updateUIElement> Update the content of the specified UI control
<xup:removeUIElement> Remove the specified UI control
<xup:moveUIElement> Reposition the specified UI control under its parent
<xup:updateUIAttr> Update the specified UI property of the UI control
50
We chose SOAP/HTTP as the default transport binding for XUP, since it's well-understood, and
its implementations in different platforms are widely available. Another good transport binding is
REST.
51
We use the term "UI element" and "UI control" interchangeably in this context, since a UI control
is represented by an XML element.
52
Similarly, we also use the term "UI attribute" and "UI property" interchangeably.
Part 1: OpenXUP 46
3.2.2.3 Session Termination

There are multiple ways for an end user to exit an XUP application. First, XUP defines
an optional shutdown message for the client to terminate the session with an XUP
application. Alternatively, the client may simply close the transport connection and
expect the server or application to purge the user session after some time interval. In
addition, a particular event on a UI control may be interpreted by the application as a
termination notice. For example, a click event on a button labeled "quit" may be used by
the application to terminate the user session.
3.2.3 Network Event Delivery

In XUP, events are triggered by user interactions on the UI model in the client. Once an
event is fired, the client sends a request encapsulating the full event detail and its
associated UI data to the server for further processing. Server sends back UI changes
and selector updates after handling the event.
The client maintains a list of event selectors which are updated by server through XUP
responses. An event is sent to the server when a selector matches with the event in the
following manner:
• In delegation event models
o The selector's event type matches with the event's type, and
o The element to which the selector is attached is the same as the element at
which the event is targeted. In this model, the element is the source of the
event.
• In capturing/bubbling event models
o The selector's event type matches with the event's type, and
o The element to which the selector is attached is the same as the element at
which the event is targeted, and
Part 1: OpenXUP 47
o If the selector specifies a source element, it must match the source element of
the event, and
o The selector's phase attribute (i.e., capturing or bubbling phase) must match
with that of the event, or both attributes must be unspecified.
Together with the event, a list of UI data is sent. These data reflect UI changes made by
the user, such as text entered in a text box and a check box been unchecked. The client
will send the event without any UI data if the user did not change the UI model. Note
that an event will not be sent if there are no selectors matching the event. If multiple
event selectors match an event, the client will send the event to the server only once.
After the server receives the event, it calls one or more event handlers registered by the
XUP application. The event handlers process the event and perform any necessary
computation for the application. For capturing/bubbling event models, the application
may also indicate in the response whether to stop the propagation of the event in the
client. By default, events propagate in the client according to their types (e.g., some
types of events bubble, others do not), as defined in the event model.
3.2.3.1 Asynchronous Events

To support a more responsive user interface, the client should not always block user
interactions while sending events (and waiting for server responses). Therefore, XUP
allows events to be delivered either synchronously or asynchronously.
During a synchronous network event delivery, the client will prevent the end user from
interacting with the UI model until the event response is received and processed. On the
other hand, during an asynchronous event delivery, the user may continue interacting
with the UI model while the client is sending the previous event (and waiting for the
server response).
To denote the mode of event delivery, XUP provides the "async" attribute on the
<selector> element. When set to true, it instructs the client to send the matching events
in an asynchronous fashion.
Part 1: OpenXUP 48
3.2.3.2 Minimize Event Traffic

It may appear that XUP is a fairly verbose protocol, since UI events are sent over the
network and there are many types of UI events. However, XUP has been designed to
minimize the amount of network event traffic.
Because events are dispatched over the network, to conserve network bandwidth, the
client does not send certain types of high-frequency events, such as mouse movement. In
addition, XUP's event selector mechanism allows an application to select interested
client-side UI events based on their event types. For example, an application may want
to receive mouse events but not keyboard events from a button. The client will only send
the events selected by the application. Finally, to filter events of the same type, XUP
supports the notion of event mask, which further refines the event selection criteria. For
example, for keyboard events on a tree node, an event mask may specify that an event
should be sent only if the "delete" key or the "return" key is pressed; so no other
keystrokes will cause the client to send the event to the server.
3.2.4 Server-side Notification

Due to the nature of HTTP/SOAP request/response model, it is not easy for the server to
send asynchronous notification to the client. This facility would be needed when
reporting server-side events and the status of long time server-side operations.
Table 3.2 Managing XUP status requests
<xup:startStatusMonitor> Instruct the client to start sending periodical status

requests according to the specified time interval
<xup:updateStatusMonitor> Instruct the client to change the frequency of the
status requests
<xup:stopStatusMonitor> Instruct the client to stop sending periodical status
requests
To simulate server-side notification, XUP provides status messages between the client
and server. The client may send periodic status requests (i.e., polling messages) to the
server, and the server may then send asynchronous notifications (in the form of UI
model updates) in status responses. For example, the server may send back a dialog box
Part 1: OpenXUP 49
alerting the user of a disk failure. Of course, the frequency of the status messages (and
whether to use status messages at all) is up to the XUP application. Table 3.2 shows the
XUP protocol elements that allow applications to manage the way the client sends status
requests.
3.3 Examples
This section illustrates how XUP works by showing two examples. Both of them are
very simple, but much richer user interfaces can be built with the same techniques. The
examples use XUL as the UI modeling language, with a simple delegation-based event
model (namespace URI: "http://www.openxup.org/event/delegation"). The UI controls
in the examples all have XML IDs, which are used by XUP to uniquely indentify the UI
elements and their attributes. Note that the examples can be replicated in other UI
models and event models in a similar fashion.
3.3.1 Example 1
Figure 3.2 Before button click
In Example 1, the user clicks on "Button1" (Figure 3.2). The XUP request in Listing 3.2
describes the button click event. The XUP <event> element specifies a client-side UI
event. Note that this event matches the selector53 "btn1Click", which is specified in the
"selector" attribute of <event>.
<soap:Envelope
xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
xmlns:xup="http://www.openxup.org/2004/02/xup">
<soap:Header>
53
The selector "btn1Click" selects the mouse click event from the button. The content of the selector
would have been sent in an earlier XUP response, which is not shown in this example.
Part 1: OpenXUP 50
<xup:session soap:mustUnderstand="1"
application="sample1" sid="4124"/>
</soap:Header>
<soap:Body>
<xup:event type="ev:click" selector="btn1Click"
xmlns:ev="http://www.openxup.org/event/delegation">
<detail>
<ev:mouseClick button="0" clientX="20" clientY="7"/>
</detail>
</xup:event>
</soap:Body>
</soap:Envelope>
Listing 3.2 XUP request for Example 1
In the server response, the XUP application instructs the client to update the text of the
label element with ID "lbl1". This is achieved by using the XUP protocol element
<updateUIAttr>, as in Listing 3.3. The content of the <updateUIAttr> element specifies
the value of the attribute "xul:value" of the label element "lbl1". Figure 3.3 shows the
result after the client processes the response and updates the text of "lbl1" to "Button1
clicked."
Figure 3.3 After button click
<soap:Envelope
xmlns:xup="http://www.openxup.org/2004/02/xup"
xmlns:xul="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
<soap:Body>
<xup:eventResponse>
<updateUIAttr element="lbl1"
attr="xul:value">Button1 clicked.</updateUIAttr>
</xup:eventResponse>
</soap:Body>
</soap:Envelope>
Listing 3.3 XUP response for Example 1
Part 1: OpenXUP 51
3.3.2 Example 2
Figure 3.4 Before clicking "Add to list" button
In Example 2, the user types "Chocolate Chip" into the text box with ID "text1" and
then clicks the button "Add to list" (Figure 3.4). The XUP request in Listing 3.4
describes the button click event. Note that this request includes a piece of UI data: the
content of the text box "text1", which is the string "Chocolate Chip". This is achieved by
using the XUP protocol element <updateUIAttr>, similar to the protocol response in the
previous example54. The content of <updateUIAttr> specifies the value of the text box
"text1".
<soap:Envelope
<soap:Header>
</soap:Header>
<soap:Body>
<xup:event type="ev:click" selector="addBtnClick"
<detail>
<ev:mouseClick button="0" clientX="18" clientY="6"/>
</detail>
54
The <updateUIAttr> element can be used in both XUP request and response. In request, it
specifies the UI data entered by the end user; whereas in response, it specified the incremental UI
update resulting from programmatic manipulation of the UI model on the server side.
Part 1: OpenXUP 52
<updateUIAttr element="text1"
attr="xul:value">Chocolate Chip</updateUIAttr>
</xup:event>
</soap:Body>
</soap:Envelope>
Listing 3.4 XUP request for Example 2 (button click)
The XUP application processes the event and creates a list item based on the string
"Chocolate Chip", which is the value of the text box "text1". It then inserts the list item
before the second item in the list box, which has an XML ID "list1". This is achieved by
the XUP element <addUIElement>, as shown in Listing 3.5. The XUP <addUIElement>
element describes a single UI element to be added to the client UI. In our example, the
UI element is a simple list item, but it may contain nested children in more complex
scenarios. In addition, the application also clears the text box "text1", so that the user
may input further text. This is accomplished via an empty <updateUIAttr>, which
essentially clears out the value of the "xul:value" attribute of the UI element "text1". The
resulting UI is shown in Figure 3.5.
Figure 3.5 After clicking "Add to list" button
<soap:Envelope
<soap:Body>
<xup:eventResponse>

Part 1: OpenXUP 53
<addUIElement parent="list1" position="1">

<xul:listitem label="Chocolate Chip"/>
</addUIElement>

<updateUIAttr element="text1" attr="xul:value"/>
<xup:addEventSelectors>
<selector id="listSelector1"
event="ev:select" element="list1"
</selector>
</xup:addEventSelectors>
</soap:Body>
</soap:Envelope>
Listing 3.5 XUP response for Example 2 (after button click)
In the XUP response in Listing 3.5, the application also adds an event selector
"listSelector1" to the list box "list1". That means all list item selection event will be sent
to the server (By default, no list selection event is sent to the server, since an XUP client
will not send any event unless there is a matching selector).
Figure 3.6 Selecting list item "Chocolate Chip"
In Figure 3.6, the user clicks the list item "Chocolate Chip", which triggers the list
selection event to be sent to the server. Listing 3.6 describes the XUP event request. The
XUP <updateUIAttr> element specifies the value of the "xul:selectionIndex" attribute of
the list box "list1". This value ("1") is the index of selected list item ("Chocolate Chip").
Part 1: OpenXUP 54
<soap:Envelope
<soap:Header>
</soap:Header>
<soap:Body>
<xup:event type="ev:select" selector="listSelector1"

<updateUIAttr element="list1"
attr="xul:selectionIndex">1</updateUIAttr>
</xup:event>
</soap:Body>
</soap:Envelope>
Listing 3.6 XUP request for Example 2 (selecting list item)
The above examples are intentionally simple, since code listings for more complicated
UI interaction scenarios will be rather lengthy. However, from these simple examples, it
should be obvious to see the advantage of XUP over the traditional form submission
protocol used by classic HTML-based web applications.
3.4 Summary
In this chapter we have presented an alternative web UI protocol, aimed at the
development of web applications with highly interactive user interfaces. The proposed
protocol, XUP, has the following features:
• To facilitate server-side UI programming, it delivers rich UI events from the

client to the server in XML format.
• To minimize network traffic, it supports the notion of event selector for event
selection and filtering.
• It transports UI update instructions from the server to the client, thus allowing
the server-side application to directly control the UI perceived by the end user.
Part 1: OpenXUP 55
• To provide end users with a fluid UI, it supports incremental UI updates, thereby
avoiding frequent whole page/screen refreshes.
• It supports asynchronous event delivery to enable a responsive UI experience.
• It provides status messages to support server-side notifications.
As a final note, we provide a comparison between XUP and the traditional form / page-
based protocol in classic HTML-based applications:
Table 3.3 Protocol comparison
form-urlencoded XUP
(on top of HTTP) (on top of SOAP/HTTP)
Request Form submission UI event
Response Whole page Incremental UI updates
UI sophistication Form / page-based UI Fluid, desktop-like UI
Table 3.3 shows the major differences. The traditional form-based protocol only carries
application-defined name / value string pairs; it has no notion of UI event, which is
essential for highly interactive user interfaces. In addition, each protocol response
includes a full HTML page, which causes frequent and annoying page refreshes. XUP,
on the other hand, carries rich UI events and delivers incremental UI updates. Therefore,
XUP can enable fluid, desktop-like user interfaces.
Further details about the protocol can be found in the protocol specifications [YC02]
[Yu06a].
Part 1: OpenXUP 57
Chapter 4. Developing and Executing RIAs
4. Developing and Executing RIAs

The web infrastructure offers unprecedented opportunities for quickly deploying
applications on a large scale, while maintaining them on the server in a centralized way.
The trend in enterprise application development is the use of web browsers to provide
access to complex and feature-rich applications (e.g. CRM, ERP) used in production
environment. Ideally, web applications should have the same level of productivity and
usability as traditional desktop applications. By productivity, we mean how efficiently
end users can perform their job functionalities. High productivity requires the
applications to provide highly usable user interfaces.
As previously mentioned, the traditional desktop-based model offers rich and powerful
user interface controls, but it requires client-side software administration. The web page
based model has no client software administration cost, as it has a universal client – the
web browser, but it lacks sophisticated user interface controls required by highly
interactive web applications. While many client-side RIA technologies provide richer
UI, they all suffer from client-side code execution security risks and IP protection issues.
Therefore, our goal is to provide a user interface programming model that combines the
advantages of both the desktop and web page based models; that is, to offer the benefit
of zero client administration cost as well as better, richer, and more powerful user
interactivities.
To satisfy the requirements set out in Chapter 1 and overcome the limitations in existing
approaches, we propose an alternative framework for web UI development, called
OpenXUP. The framework is built on top of XUP and provides a UI programming
model that combines advantages in both web page-based applications and desktop
applications.
For developers, the framework provides a programming environment that closely

resembles desktop applications. That is, rich user interface controls, such as those found
in Windows Forms and Java Swing, can now be used in building web applications. At
the same time, OpenXUP maintains one of the most important benefits of web
Part 1: OpenXUP 58
applications – zero client-side administration, since all application code resides on the
server side. Consequently, OpenXUP has no IP protection issues.
For end users, the framework aims at bringing web applications closer to their desktop
counterparts. That is, web applications built with OpenXUP will appear identical to
desktop applications. In addition, OpenXUP's asynchronous event delivery and
incremental UI updates mechanism enables fast application responses without annoying
page refreshes. Finally, the framework provides a safe client environment, without the
common security risks arisen from client-side code execution.
OpenXUP offers a set of server-side event-driven APIs, which enables developers to

implement sophisticated application-specific UI behaviors. The OpenXUP APIs are
designed to be familiar, closely resembling the APIs from desktop GUI toolkits. In
addition, since all application code resides on the server side, it makes web applications
easier to debug and maintain, without the need to worry about issues from distributed
computing. As for the backend infrastructure, web applications built with OpenXUP can
leverage all existing backend components (EJB, CORBA, COM, etc.), since OpenXUP's
server side is designed to run within existing web application servers.
The outline of this chapter is as follows. In the next section we describe an example
scenario to illustrate the requirements of rich web user interfaces. We then present the
detailed design of the runtime framework, focusing on the server side (the client side
will be discussed in the next chapter). After that we describe the development
environment and show how the example application can be developed. Finally, we
discuss the implementation of the framework, including the server-side runtime
environment and the associated development tools.
4.1 Example Scenario

To illustrate the requirements of a rich web UI environment, we show an example
application, XCat, which is a hosted multi-buyer, multi-seller e-commerce solution.
XCat is typically hosted by an ASP (application service provider) or a marketplace
operator and used by consumers (buyers) and retailers (sellers). XCat has two interfaces
Part 1: OpenXUP 59
that deal with consumers and retailers respectively. The retailer part of XCat includes
functionalities such as electronic catalog editing, order processing, and logistics.
In this example we focus on the electronic catalog editing functionality, which allows
multiple users within a retailer's organization to concurrently and efficiently edit their
product information stored in server-side databases. It also has a workflow-based
preview and approval process, which allows product managers to perform quality
assurance on the accuracy of the product information. Since each retailer may have
thousands of products, the editing, preview, and approval user interfaces must be very
efficient. Without high usability, the data entry specialists and product managers'
productivity may suffer, and many of the retailer's products may not get into the
electronic catalog on time. Hence, XCat should provide a rich and efficient UI
experience analogous to that of desktop applications.
Figure 4.1 Example application: XCat
Figure 4.1 shows the desired user interface for XCat. In this screen, the left hand side is
a tree interface, containing a hierarchy of product categories; this allows efficient
catalog navigation and selection. The right hand side is a tabbed panel, with two tabs
showing the list of attributes and the list of products under the currently selected
category. The products tab (not shown) also has scrolling and pagination control,
allowing thousands of products to be displayed in an incremental fashion.
Part 1: OpenXUP 60
To allow maximum efficiency and usability, the end user should be able to select a
category by clicking on the tree node, using arrow keys, or typing the tree node's label
while the tree control has the focus. Similarly, to expand a category, the end user may
click on the "+" icon, double-click on the category name, or press the right arrow key. In
addition, the product information editing interface should support full keyboard
navigation. The end user should be able to go to a particular input field with keyboard
shortcut (without using mouse), and traverse input fields via the tab key. This is the
behavior that we would expect from a well-designed desktop application.
4.2 Runtime Framework

In this section we illustrate the design of the runtime framework by providing an
overview of the various elements of the framework and the interactions among them.
Before discussing the details of our framework, we need to provide some additional
background information. We begin by reiterating the concept of user interface model
(UI model), user interface language (UI language), UI definition, and UI logic.
As mentioned in the last chapter, a UI model is a representation of the user interface

which the end user perceives and interacts with. UI models may have XML-based
representations. We call these representations UI languages. Common UI languages
include XUL, XAML, UIML, and SUL.
A UI model typically consists of a tree of UI controls (e.g., buttons, panels), a set of

events (e.g., mouse click), and a list of resources (e.g., button images, files to be
downloaded or uploaded). To handle user interactions, developers process events
emitted from the UI tree and manipulate the UI model to update the user interface. The
code than handles events and manipulate the UI model is called UI logic, whereas the
code that renders the UI model to end users is called UI definition.
UI definition is a description of the user interface, specifically, the UI controls and their
properties. UI definition is typically coded in declarative languages such as XUL and
XHTML. In this case, UI controls can be described by a tree of XML elements, with the
UI controls mapping to XML elements and their properties (e.g., color, size) mapping to
XML attributes. On the other hand, UI logic contains code that handles UI events and
Part 1: OpenXUP 61
communicates with application logic code. UI logic is typically coded in a programming

language.
Consistent with the protocol, OpenXUP is also independent of the actual UI language. It
places no restriction on the UI control set, or the properties or events associated with
each control. That is, OpenXUP is designed to work with any UI model that has an
XML-based representation.
4.2.1 OpenXUP Architecture

In order to illustrate how OpenXUP works, we first need to describe the different
components that are part of the framework. Since OpenXUP is built on top of XUP, the
framework components expand from the basic protocol components discussed in the last
chapter. The framework's components and the overall architecture are depicted in Figure
4.2.
Figure 4.2 OpenXUP framework architecture
4.2.1.1 OpenXUP Client

The OpenXUP client has two main tasks: one is to manage the interaction with the end
user (display UI controls and capture UI events) and the other is to communicate with
the server (transform UI events to XML messages and then send them to the server;
receive UI updates in XML messages from the server and then render them). These tasks
are performed by the view manager and the XML serializer, respectively.
Part 1: OpenXUP 62
The view manager is responsible for managing the native desktop user interface. First, it
renders UI controls using the appropriate native desktop GUI toolkit. Second, it captures
native UI events caused by end user actions, and then delivers them to the server
through the XML serializer.
The client-side XML serializer is responsible for XML serialization. First, it parses and
generates XUP protocol messages. Second, it turns a UI control to and from its XML
representation. For example, let us assume that a push button UI control has an object
representation SButton and an XML representation <button>. The XML serializer will
parse the XML element <button> into an SButton object, and generate the SButton
object into a <button> element. In addition, the client-side XML serializer is responsible
for serializing a UI event into XML and sending it to the server.
Finally, the OpenXUP client is a thin client, just like web browsers; developers do not
write any code on the client side. Further details about the client will be discussed in the
next chapter.
4.2.1.2 OpenXUP Server

The OpenXUP server processes event requests from the client and sends back UI and
resource updates in the responses. The server side includes application-specific code
provided by application developers (i.e., XUP applications) as well as OpenXUP
middleware components: the event dispatcher, the application manager, and the XML
serializer.
An XUP application provides desired functionalities to the end user through the client.
Application developers use the set of event-driven APIs provided by the server to
process events and update the UI.
An event handler is a piece of code (e.g., instance method) in the XUP application. The
server executes the application by invoking registered event handlers upon receiving an
event request from the client. Event handlers make up the bulk of the UI logic code in
an XUP application.
Part 1: OpenXUP 63
A user session (or simply session) represents the end user's UI state stored on the server
side. Specifically, this includes the UI model which the end user perceives and interacts
with. The UI model consists of a tree of UI controls (called the UI tree). A session
corresponds to a single end user that's interacting with the application. Therefore, an
XUP application may contain many sessions concurrently, each representing an
individual user.
The event dispatcher is a middleware component that receives UI event through the
XML serializer and invokes registered event handlers for the event.
The application manager is a middleware component that manages XUP applications. It

is responsible for finding, loading, and destroying XUP applications.
Finally, the server-side XML serializer is responsible for receiving the XML event
request and parsing it into a UI event object. In addition, when the XML serializer
parses protocol messages, it will parse the XML representations of the UI controls into
their object representations. Similarly, when the XML serializer generates protocol
messages, it will generate the XML representations of the UI controls from their object
representations.
The OpenXUP server only handles user interface processing, leaving backend
components (e.g., CORBA, EJB) intact. Therefore, all backend components in existing
web applications can be preserved or reused.
The OpenXUP server may host one or more XUP applications which may be running
concurrently. Similarly, an end user may use the client to interact with more than one
application from one or more servers at the same time.
4.2.2 MVC Pattern in OpenXUP

The OpenXUP framework reflects a slightly modified version of the MVC pattern
[Ree79][Bur87], popular with desktop-based GUI programming.
As shown in Figure 4.2, the client maintains the UI model as well as the view, while the
server side only maintains the UI model. On the client side, the end user manipulates the
view, which in turn causes the UI model to be updated. On the server side, the
Part 1: OpenXUP 64
application programmatically manipulates the UI model through event handlers (i.e., the
controller in MVC). Through XUP protocol message exchanges, the framework
automatically synchronizes the client-side UI model and the server-side UI model.
4.2.3 Location of UI and Application Code

Here we illustrate the differences between OpenXUP, various rich web client
approaches (e.g., Java applet, ActiveX), and AJAX, in terms of the location of the UI
definition, UI logic, and application logic.
In the rich web client approaches, both UI definition and UI logic are programmed,
rather than declared. In case of Java Applet, it is in Java; and in case of ActiveX, it is in
C++. In addition, both UI definition and UI logic are on the client side, while most of
application logic is on the server side. However, the client-side code also contains some
application logic, in order to render the application data and communicate with the
server side.
In the AJAX approach, the UI definition is provided by HTML, and the UI logic is
programmed in JavaScript. Similar to the rich web client approaches, both UI definition
and UI logic are on the client side, while most of application logic is on the server side.
However, the client-side JavaScript also contains some application logic, in order to
render the application data and communicate with the server side.
In OpenXUP, the UI definition is provided by declarative UI languages such as XUL or

SUL, and the UI logic is programmed. In our approach, the UI definition is interpreted
on the client side, but the UI logic is executed on the server side. In addition, application
logic also resides on the server side. Essentially, this approach separates UI definition
from UI logic and application logic. The client side renders UI definition and processes
native UI events by leveraging the desktop machine's rich UI capability and computing
power. In addition, the end user's desktop remains very secure since no application code
is downloaded to be executed on the client side.
Part 1: OpenXUP 65
4.2.4 Runtime Behavior

The runtime behavior of the framework is based on XUP protocol message exchanges.
In this section, we expand the discussion about protocol operations in the last chapter to
cover the framework's runtime behavior.
Just like protocol message exchanges, the runtime behavior of the framework can be
described in three phases (Figure 4.2). First, the end user starts by establishing a session
with an XUP application. Second, the end user interacts with the application which
causes events and UI updates to be exchanged between the client and the server. Finally,
the end user terminates the session with the application.
4.2.4.1 Startup
The client begins by sending a startup request, which includes the name of the
application and a list of UI model namespaces supported by the client. The server
locates the application through the application manager and creates a user session for
the client. The server then transfers the control to the application which creates an initial
UI model, including UI controls, resources, and a list of event selectors, whose
namespaces must be in the list of UI model namespaces supported by the client. Then
the server sends a response through its XML serializer, including the initial UI model
and the session ID used for identifying the client in subsequent requests. The client
receives this UI model from its XML serializer and renders its initial UI through the
view manager.
The initial UI model also includes a list of event selectors, which defines the event
selection criteria in the client. To minimize network traffic, not all client-side UI events
are delivered to the server in subsequent requests; only those matched by event selectors
are sent. In addition, event selectors allow applications to selectively handle events. For
example, an application may want to receive mouse events but not keyboard events from
the client (i.e., the application is interested in the user's mouse clicks, but not
keystrokes).
Part 1: OpenXUP 66
4.2.4.2 User Interactions

After the startup phase, the end user interacts with the application by manipulating the
UI model, which triggers native UI events. The view manager will capture those native
events and send them to the server through the XML serializer. The client maintains a
list of event selectors which are updated by the server through XUP responses. An event
is sent to the server only if the event satisfies the event selection criteria defined by an
event selector. Together with the event, a list of UI data is also sent. Those UI data
reflect changes made by the end user to the UI controls, such as text entered in a text
box and a check box been unchecked.
Upon receiving an event request through the XML serializer, the server locates the XUP
application and user session, and passes the event to the event dispatcher, which in turn
invokes all the registered event handlers for that event. Note that there is only one event
in each request, but multiple event handlers within an application may be invoked to
handle that event. Event handlers execute the necessary UI and application logic (e.g.,
by calling some backend components) and update the UI model accordingly. The server
then returns the UI model updates to the client via its XML serializer. If some errors
occurred in the event handlers, a SOAP fault is returned instead.
The client receives the XUP response from its XML serializer. It then updates its UI
controls and the list of event selectors. After that the client asks the view manager to
render the UI updates, which include changes to UI controls, properties, and resources.
Because events are delivered over the network, to conserve network bandwidth, the
client does not send certain types of high-frequency events, such as mouse movement.
For medium-frequency events, such as key press, XUP applications should created event
selectors matching a particular key press (e.g., F5) to avoid all keystrokes being sent
over the network.
4.2.4.3 Session Termination

An end user may terminate the session with an XUP application in one of the following
ways. First, a particular event on a UI control may be interpreted by the application as a
termination notice. For example, a click event on a button labeled "Quit" may be used
Part 1: OpenXUP 67
by the application to terminate the user session. Second, the end user may explicitly exit
the application by instructing the client to send a "shutdown" request to the server; the
server will close the user session after receiving the request. Finally, the server may
close and purge the user session due to inactivity.
4.2.5 Server-side Asynchronous Notifications

In many situations, it is desirable for the application to send asynchronous notifications
to the user. For example, if a disk failed, the application should notify the user by
prompting a dialog box. In addition, this facility would also be needed to report the
status of long time server-side operations. For instance, assuming the application is
performing a long search operation, rather than having the client kept the connection
open to wait for the search result, it is more desirable for the application to popup a
window displaying the result when the search is completed.
However, due to the nature of HTTP/SOAP's request/response model, it is not easy for
the server to send asynchronous notifications to the client. Therefore, the framework
leverages XUP's status messages to simulate server-initiated notifications. The XUP
application can instruct the client to send periodic status requests (i.e., polling messages)
to the server, and the server may then send asynchronous notifications (in the form of UI
model updates) in status responses. In addition, the application may ask the client to
start / stop status requests or change the frequency of the status requests at any time.
4.3 Development Environment

To facilitate rapid web UI development, OpenXUP offers a set of server-side event-
driven APIs and a UI templating mechanism. The APIs allow developers to program
rich UI behaviors while the UI templating mechanism allows developers to declaratively
specify the UI definition.
4.3.1 Application Development APIs

The current OpenXUP server implementation is in the form of a server toolkit running
inside ASP.NET. The server toolkit provides several class libraries which contain a set
of high-level .NET APIs. The APIs are event-driven, similar to desktop GUI toolkits
Part 1: OpenXUP 68
such as Windows Forms and Java Swing. Developers use these APIs to develop highly
interactive web applications.
The APIs consist of two layers of classes: UI language independent classes and UI
language specific classes. We now discuss each of these layers and the motivation
behind separating the APIs into two layers.
4.3.1.1 UI Language-Independent Classes

The framework offers a set of UI language-independent classes that provide a common
abstraction for all UI controls, events, and resources. These classes serve two purposes.
First, they allow the framework to support additional UI languages via plug-in modules.
Second, they allow application developers to minimize the amount of UI language
specific code.
At the base of any UI model is the notion of component, which represents UI controls
such as buttons and labels. For this purpose, OpenXUP includes a class called
XComponent, which is abstract, and serves as the base class for all UI controls.
Figure 4.3 XComponent class
UI controls in a UI model form a tree structure, therefore methods and properties in

XComponent allow developers to access and manage the tree (retrieve the parent or the
Part 1: OpenXUP 69
children, add / remove children, etc.). In addition, this class has a namespaceUri
property which supports XML namespace mechanism. This allows the framework to be
extremely extensible. Support for new UI languages can be added by defining new XML
namespaces and then implementing new UI controls by inheriting from XComponent.
Finally, this class includes methods to allow selection of UI events to be handled
(methods addEventSelector and removeEventSelector).
Note that this class does not contain any UI properties such as color and size. This is
because XComponent is the root abstract class for all UI controls from any UI model,
and it does not depend on any specific UI languages.
Figure 4.4 Classes related to events
The framework also includes classes for managing UI events. What is needed here is the
ability to selectively handle client-side UI events by specifying an event type, so that the
client only sends events selected by the application. This strategy allows for very
efficient network event delivery. The class XEventSelector is defined for this purpose
(Figure 4.4).
In addition, we need to denote whether events are synchronous or not; in fact, the client
should be able to send the event to server in the background, without blocking the end
user from further interacting with the application. Typically, asynchronous events
should be used when the corresponding handling code in the application takes long time
to execute.
Moreover, we need the ability to further qualify the event of interest. For example, for a
mouse click event, the application may specify that the event should be sent only if the
Part 1: OpenXUP 70
right mouse button is pressed while the control key is held down. Similarly, for a
keyboard event, the application may indicate that only events for the "return" or "delete"
keys should be sent. Therefore, XEventSelector allows applications to further refine
event selection criteria by defining event masks.
The addHandler and removeHandler methods of XEventSelector manage event handlers

in the selector. An event handler is a delegate, which is a pointer to an instance method
in a class defined by the application. Multiple event handlers may be attached to the
same selector. When the server receives an event from the network, it will first identify
the event selector, and then sequentially invoke all event handlers attached to the
selector.
Event handlers take a single parameter of type XEvent, which can be used by the
application code to examine the details of the UI event. Note that just like XComponent,
XEvent does not contain any event details, because it is the root abstract class for all UI
events from any UI model.
Finally, the abstract class XApplication allows XUP applications to be hooked into the
server. All XUP applications must derive from this class and provide implementations
for the abstract methods. For example, the two abstract methods, startSession and
endSession, allow the application to initialize and terminate a user session, respectively.
Figure 4.5 XApplication class
The user session is represented by the XUserSession class, which contains the end user's
UI state (i.e., the UI model which the end user perceives and interacts with).
Specifically, the session includes a tree of UI controls, with the root UI control
represented by the componentRoot property. To update the UI, the application traverses
the UI tree starting from the root UI control and modifies UI controls and their
Part 1: OpenXUP 71
properties as appropriate. Since a session corresponds to a single end user who is

interacting with the application, an instance of XApplication may contain many
instances of XUserSession, each representing an individual user.
Figure 4.6 XUserSession class
The above APIs can be called from any languages that target the .NET Common
Language Runtime, allowing programmers to use any .NET languages to develop XUP
applications.
4.3.1.2 UI Language Specific Classes

We have discussed the UI language independent APIs in the previous section. For an
XUP application to be developed, a concrete XML-based UI language is needed.
Figure 4.7 SUL classes
For the purpose of prototype implementation, we have created SUL (Simple User
Interface Language). SUL is implemented by a set of classes (Figure 4.7) extending
from the UI language-independent classes outlined in the last section. The SUL classes
Part 1: OpenXUP 72
include a set of concrete UI controls for application development, such as SButton and
SMenuBar. All concrete UI controls have XML representations. For example, in our UI
model example in the beginning of the last chapter, the XML elements <button> and
<panel> correspond to the SButton and SPanel classes, respectively.
For UI events, the base class in SUL is SEvent, which directly extends from XEvent. All
other concrete UI events in SUL derive from SEvent.
Figure 4.8 SUL events
SUL also defines a set of event mask classes, which allow for fine-grained event
filtering. For example, an application may use SMouseClickMask to specify that a
mouse click event should be sent only if the right mouse button is pressed while the
control key is held down. Similarly, the application may use SKeyDownMask to instruct
the client to only send "return" or "delete" key events.
Figure 4.9 SUL event masks

Part 1: OpenXUP 73
Note that the framework is independent of any UI languages, including SUL. Support
for SUL is achieved via a plug-in module. Similarly, supporting additional UI languages
such as XUL and XAML only involves the creation of appropriate plug-in modules, and
there is no need to change the source code of the base framework.
4.3.2 UI Templates
Applications use the APIs defined in the last section to create and modify UI controls
and to handle events. However, creating a large number of UI controls in source code
may be quite tedious. Therefore, the framework also offers a templating mechanism
which allows UI controls to be defined in XML-based templates.
The concept of UI template is similar to that of JSP and ASP, which consist of UI
markups with "hooks" to executable code. We call it "template", rather than "page",
because applications developed in OpenXUP are very fluid and no longer page-based.
To separate UI definition from UI logic, our UI templates do not allow embedded
executable code; that is, executable code must be referenced externally. This is unlike
JSP or ASP, where the page can contain arbitrary executable code mixed with HTML
markups. The following listing shows an example UI template.
<t:template xmlns:t="http://www.openxup.org/2004/04/template">
<window id='w1' position="width:200;height:100"

xmlns="http://www.openxup.org/2004/02/sul">
<panel id='p1' ...>

<button id='b1' ... />
<button id='b2'... />
......
</panel>
......
</window>
<t:objects>
<t:object id='o1' class='testapp1.TemplateObj1'
scope='session'>
<t:property name='prop1'>20000</t:property>
</t:object>
</t:objects>
Part 1: OpenXUP 74
<t:selectors>
<selector id="s1"
event="sev:window-closing" element='w1'
xmlns="http://www.openxup.org/2004/02/xup"
xmlns:sev="http://www.openxup.org/2004/04/sev">
<t:handler object="o1" method="onWin1ClosingA" />
<t:handler object="o1" method="onWin1ClosingB" />
</selector>
<selector id="s2"
event="sev:mouse-click" element='w1'
xmlns="http://www.openxup.org/2004/02/xup"
xmlns:sev="http://www.openxup.org/2004/04/sev">
<t:handler object="o1" method="onWin1Click" />
</selector>
</t:selectors>
</t:template>
Listing 4.1 UI template example
A UI template contains a tree of UI controls, ranging from a single button to a window

with nested panels. In the above example, the top level control is a window with ID
"w1". Within the window, there is a panel that in turn contains several buttons. More
sophisticated user interface consisting of multiple windows and many nested controls
can be defined in a similar fashion. An application may use as many templates as
needed, for different parts of the application's user interface.
In addition to UI controls, an application may also define event selectors in the template.
This further reduces application coding. In the above example template, there are two
selectors "s1" and "s2", which instruct the client to deliver "window-closing" and
"mouse-click" events, respectively. The application may then attach event handlers to
the selectors via the API call XEventSelector.addHandler. Alternatively, the application
can define the event handlers in the template as well.
Event handlers are delegates, which are instance methods in objects defined by the
application. To define handlers in the template, the application must first define the
objects containing those methods. In the above example, one such object, "o1" has been
defined in the template. The object is defined by the qualified class name,
"testapp1.TemplateObj1", and has a scope of "session". The qualified class name allows
the object to be dynamically created; the scope value limits the object's visibility.
Part 1: OpenXUP 75
Possible values for scope are "application" (throughout the application), "session"
(within the current user session), and "template" (within the current template). The
object definition may also contain a set of properties, which are used to initialize the
internal state of the object. In the above example, the property "prop1" of the object "o1"
has been initialized to "20000".
After defining objects, the application can define event handlers by referring to the IDs
and method names of those objects. The selector "s1" in above example contains two
event handlers, one referring to the "onWin1ClosingA" method of the object "o1", and
the other referring to the "onWin1ClosingB" method of "o1".
During runtime, when a UI template is accessed, the framework will create the UI tree
and then instantiate the objects, event selectors, and event handlers defined in the
template. After that, the application code in the event handlers can freely access and
manipulate the UI tree. As an application may use as many templates as needed (for
different parts of the application's user interface), the UI tree in the template must be
attached to the appropriate branch of the main UI tree in the user session.
Unlike JSP or ASP, we do not use special control syntaxes such as "<%". As a result,
our templates are in pure XML (i.e., well-formed [Bra06]). Template control elements
are in a dedicated XML namespace (i.e., "http://www.openxup.org/2004/04/template"),
which contains XML elements that define objects, event selectors, and event handlers.
The UI controls in the above example are from SUL, so they are under the SUL UI
control namespace ("http://www.openxup.org/2004/02/sul"). Similarly, the SUL event
namespace ("http://www.openxup.org/2004/04/sev") defines event types. This design
allows the UI templating mechanism to be independent of the actual UI controls and
events from the UI languages.
Finally, the framework provides a UI template viewer which helps developers to author
and test UI templates. Alternatively, since the template syntax is quite simple,
developers can simply use their favorite XML editors to create UI templates.
Part 1: OpenXUP 76
4.3.3 Multiple UI Languages

As mentioned previously, the framework is independent of the actual UI language being
used in an XUP application. This leads to the possibility to using multiple UI languages
in the same application. Multiple languages may be needed when a single UI language
cannot satisfy the requirements of a particular application. For example, the application
may require rich text (text with varying font types, colors, and sizes) display
functionality not found in SUL. However, XHTML, while lacking rich UI controls, is
fully capable of displaying rich text with embedded images. So what's needed here is the
ability to mix XHTML and SUL in the same application.
<window id='w-main' title='XHTML example' width='500' height='500'

xmlns='http://www.openxup.org/2004/02/sul'>
<flow-panel dir='top-bottom' border='single'>
<html xmlns='http://www.w3.org/1999/xhtml'
width='450' height='350'>
<body bgcolor='gray'>
<h1>XYZ</h1>
</body>
</html>
<flow-panel border='single' top-margin='10'>

<button id='dom' left-margin='10' access-key='D'>DOM</button>
<button id='text' left-margin='10' access-key='T'>Text</button>
<button id='url' left-margin='10' access-key='U'>URL</button>
</flow-panel>
</flow-panel>
</window>
Listing 4.2 Multiple UI languages example
The above listing shows an example of mixing SUL and XHTML. The top level SUL
window contains an XHTML control on the top and three SUL buttons in the bottom.
The framework leverages XML's namespace mechanism to mix UI controls from
multiple languages. As shown in Listing 4.2, the XHTML UI control (i.e., the <html>
element) is from the XHTML namespace ("http://www.w3.org/1999/xhtml"), while the
rest of SUL controls are from the SUL namespace.
The support for XHTML is achieved via a UI language plug-in module, which provides
classes that map to XHTML UI controls. As shown in Figure 4.10, the
Part 1: OpenXUP 77
XHTMLComponent class extends from XComponent and serves as a base class for all
XHTML UI controls. All concrete UI controls have corresponding supporting classes
with XML representations. For example, the XHTMLImg class supports the HTML
<img> element, and the XHTMLTable class supports the HTML <table> element.
Figure 4.10 XHTML classes
4.3.4 Asynchronous Event Handling

To improve the responsiveness of web UI, applications must be able to perform
computations in the background, while at the same time, handle user inputs in the
foreground. Classic HTML applications do not support asynchronous UI. When the user
performs another action (e.g., clicking on a link or button) while the server hasn't
finished processing the previous request, the browser will interrupt the previous request.
This will often lead to undesirable consequences if not properly handled by the server-
side application. Traditional desktop applications (as well as the rich client technologies
discussed earlier) are capable of supporting asynchronous UI. However, developers
must use advanced programming techniques, such as multi-threading or callback
functions to provide asynchronous behavior.
In OpenXUP, asynchronous UI behavior is provided by the client via asynchronous

event delivery. By default, event delivery is synchronous. To instruct the client to send a
UI event asynchronously, the application simply specifies that while creating the event
selector (i.e., setting XEventSelector.asynchronous to true). When an event that matches
the selector is fired, the client will deliver it to the server asynchronously. That is, the
Part 1: OpenXUP 78
user may continue interacting with the application while the client is sending the event
and waiting for its response from the server. Essentially, the client sends the UI event,
waits for the server response, and finally processes the response, all in the background.
When processing asynchronous events, the server-side application can treat them just
like regular synchronous events, except when accessing the session object. To access or
update the UI model, the application must access the UI tree in the session object. Since
there may be multiple threads accessing the session object at the same time (further
details will be explained in the next section), the application should obtain the lock on
the session before manipulating the UI tree. However, locking on the session object is
trivial, since in C# one can simply use the "lock" keyword to synchronize access to the
session object (Listing 4.3). This programming model is much easier since developers
need to manage neither multiple threads nor convoluted callback functions.
4.3.5 Multi-threading
The OpenXUP server has a multi-threaded architecture that allows multiple threads to
be executed on the same user session. Multi-threaded access is needed when the
application is processing asynchronous event requests and periodical status requests.
This is because while these types of requests are in progress, the end user may further
interact with the UI, thereby generating additional event requests which must be
processed by the application at the same time.
Specifically, at any point of time, the following threads may be concurrently executing
in a user session:
• Zero or one thread processing a synchronous event request. There could only be
up to one synchronous event request because when the request is in progress, the
user is blocked from further interacting with the UI.
• Zero or more threads processing asynchronous event requests. There may be

more than one asynchronous event requests because while one is in progress, the
user may continue interacting with the UI, triggering additional events to be sent
to the server.
Part 1: OpenXUP 79
• Zero or one thread processing a status request. There could only be up to one
status request because the client sends status requests periodically, and the next
one won't be sent until the response to the previous request has been received by
the client.
To simplify the handling of multi-threaded access to the session object, the server
automatically locks on the session after receiving a synchronous event request. The
application code can freely modify the UI tree. For asynchronous events and status
requests, the application code must manually lock on the session object when accessing
the UI tree.
The rationale behind this design is that processing asynchronous events and status
requests typically takes a long time (e.g., retrieving product information from the
backend database in the XCat example), so placing a lock on the session may prevent
other requests to be processed. Therefore, to enhance the application performance, the
application code in asynchronous event handlers and status request handlers should only
lock on the session when necessary. That is, during an asynchronous event or status
request, the application code should lock on the session when it needs to update the UI,
and release the lock when it's performing other computation (e.g., communicating with
backend components or performing large database queries).
In traditional desktop applications and rich client based applications (including AJAX),
developers must manually manage multiple threads or callback functions to support
asynchronous UI behavior. In server-based approaches such as ASP.NET Web Forms,
the server serializes all requests when the session is marked read and write, forbidding
any concurrent requests on the session.
Our model maximizes the UI's responsiveness and application's concurrency without
requiring developers to manually manage multiple threads or callback functions. All
they need to remember is to lock on the session object when they need to update the UI
tree while handling asynchronous event and status requests.
Part 1: OpenXUP 80
4.3.6 Application Development

In this section we show how XUP applications are developed. To illustrate the concepts,
we will use our example application, XCat.
Developers use the classes and APIs illustrated in the previous sections to build
applications such as XCat. They typically use an integrated development environment
such as Visual Studio.NET. Of course, they may also use text editors and command line
compilers, since the APIs are packaged in libraries. For building UI templates, they may
use our UI template viewer or their favorite XML editors.
To implement a new XUP application, the developer performs the following steps.
4.3.6.1 Creating Main Application Class

The first step is to create a main application class that extends from XApplication, which
is an abstract class provided by the framework. The main application class must
implement the startSession method, which serves the same purpose as the "main()"
function in desktop applications. Within this method, the developer creates the initial UI
model. She may use APIs to create the initial UI controls; alternatively, she may load
them from a UI template. After that, she adds event handlers for the appropriate events.
For example, when an end user connects to XCat for the first time, the application
creates a login prompt to ask the user to input login information. In addition, an event
handler is added to handle the "dialog-closed" event from the login prompt.
// application main class

public class XCatSrv: XApplication {
public override bool startSession(XUserSession session, ......) {
......
// create a new catalog editing session
XCatSess catSess = new XCatSess();
catSess.start(session);
return true;
}
......
}
public class XCatSess {

private XUserSession mSess = null;
public void start(XUserSession session) {

mSess = session;
Part 1: OpenXUP 81
// display the login prompt
SInputPrompt loginPrompt = new SInputPrompt();
......
// add an event handler to authenticate user and display the main window
loginPrompt.addEventHandler(new XEventHandler(onLoginPromptClosed),
SDialogClosedEvent.QUALIFIED_TYPE);
}
// event handler for dialog-closed event

private void onLoginPromptClosed(XEvent ev) {
// authenticate the user
......
// load main window from a UI template
UITemplate temp = templateLoader.load("/catalog.xml");
// attach components in the template to the UI tree

mSess.componentRoot.addChild(temp.component);
// get a handle of the category tree component

STree catTree =(STree) mSess.component("tr-category");
// fill the tree with categories from backend database

......
// add an asynchronous event handler for category selection event
catTree.addEventHandler(new XEventHandler(onCatTreeSelectionChanged),
SSelectionChangedEvent.QUALIFIED_TYPE, true);
......
}
// event handler for category selection-changed event

private void onCatTreeSelectionChanged(XEvent ev) {
STree catTree = (STree) ev.component;
STreeNode selectedNode = null;
lock (mSess) { selectedNode = catTree.selected; }
// retrieve products and attributes of the selected category from backend database
......
// update UI
lock (mSess) {
// fill the Attributes and Products tabs of the selected category
......
}
}
......
}
Listing 4.3 C# source code fragment for XCat
4.3.6.2 Creating UI Templates

UI templates allow developers to declare UI controls, rather than creating them in source
code. Developers may use multiple UI templates for different parts of the UI model. UI
templates may simply contain UI controls, or they may declare application objects and
event selectors.
The listing below contains the UI template "catalog.xml", which is used to render the
XCat's main window after the user logins in.
Part 1: OpenXUP 82
<t:template xmlns:t="http://www.openxup.org/2004/04/template">
<t:resources>

<image xmlns="http://www.openxup.org/2004/07/sres" local-path="/cat/new.gif" id="new"/>
<image xmlns="http://www.openxup.org/2004/07/sres" local-path="/cat/open.gif" id="open"/>
......
</t:resources>


<window xmlns='http://www.openxup.org/2004/02/sul' id='w-main' ...>
<menu-bar id="m-main">
<menu-item id="mi1" access-key="f" text="File"/>
<menu-item id="mi2" access-key="e" text="Edit"/>
......
</menu-bar>
<panel id="p-toolbar" ...>

<button id='b1' image='new' .../>
<button id='b2' image='open' .../>
......
</panel>
<split-panel id="sp-main" ...>


<tree id="tr-category" .../>
<tabs id="tabs-content">

<tab-panel id="tp-attr" title="Attributes" ...>
<label ...>Name</label>
<label ...>Display Name</label>
......
</tab-panel>


<tab-panel id="tp-prod" title="Products" .../>
</tabs>
</split-panel>
</window>
</t:template>
Listing 4.4 UI template "catalog.xml"
4.3.6.3 Implementing Event Handlers

Event handlers are instance methods containing an application's event handling code.
They typically invoke some application logic classes and backend components, and then
update the UI model appropriately. Event handlers may also load additional UI
templates to update the UI model.
The bulk of the UI logic code in an XUP application is in event handlers, which are
invoked by the server after receiving event requests from the client. This event-driven
programming style allows easy separation of business logic from presentation logic.
Part 1: OpenXUP 83
In addition, the application logic classes and backend components invoked by the event
handlers may come from existing web applications. This provides an easy migration
path from traditional web page-based applications to OpenXUP.
In our XCat example, we demonstrate two event handlers (Listing 4.3), corresponding to
two user interactions in the application. The first event handler processes user login and
displays the main application window; and the second event handler processes category
selection changes.
i) User Logging-in
After the end user fills in the account information and hits the "ok" button, the
framework will execute the registered event handler to handle the "dialog-closed" event.
This event handler authenticates the end user, and then proceeds to display the main
application window by loading the UI template "catalog.xml".
After loading the UI template, the event handler fills the tree control on the left hand
side of the split panel by retrieving product categories from a backend database. Finally,
it adds another event handler to handle tree node selection events. Note that the newly
added event handler is asynchronous, so the client will send node selection events
asynchronously.
ii) Changing Category Selection
To view the attributes and products of the "Operating Systems" category, the end user
selects the tree node "Operating Systems" (Figure 4.1). This action causes the client to
send the following event:
<soap:Envelope xmlns:soap="http://...">
<soap:Header>
<session soap:mustUnderstand="1" sid="X12619A33"
application="xcat/bin/xcat:xcat.XCatSrv"
xmlns="http://www.openxup.org/2004/02/xup"/>
</soap:Header>
<soap:Body>
<event type="sev:selection-changed" selector="_S5615D246"
xmlns:sev="http://www.openxup.org/2004/04/sev"
xmlns="http://www.openxup.org/2004/02/xup">
Part 1: OpenXUP 84
<detail sev:descendant="_ED7559A5"/>
......
</event>
</soap:Body>
</soap:Envelope>
Listing 4.5 "Selection-changed" event request
Here, the event is "selection-changed", under SUL's event namespace. The event detail
contains an SUL event attribute descendant, which specifies the ID of the newly
selected tree node, "_ED7559A5" ("Operating Systems").
The server receives this event and executes the registered event handler which retrieves
the attributes and products of the selected category from the backend database and fills
them in the "Attributes" and "Products" tabs on the right hand side of the main window.
Note that this event is asynchronous, so the client will not block end users from further
interacting with the application, giving them a more pleasant experience. We have made
this event asynchronous because the code in the event handler (to retrieve product
information from the backend database) may take a long time to execute.
4.3.6.4 Application Deployment

Deploying applications in OpenXUP is relatively straight forward. First, the developer
compiles the application source code into one or more .NET assemblies (DLLs).
Second, if a previous version of the application is still running, it needs to be shutdown.
Finally, the developer copies the assemblies and UI templates into the appropriate
directories specified in the server configuration. Note that during the entire deployment
process, the OpenXUP server does not need to be restarted.
4.4 Implementation
As a proof of concept, we have implemented the proposed server-side RIA framework
in .NET. The prototype consists of the following components:
• Protocol module. It fully implements XUP.

Part 1: OpenXUP 85
• OpenXUP server. It contains a server-side runtime for application execution as

well as a set of event-driven APIs for application development.
• UI template viewer. It facilitates rapid UI development by allowing previewing

and modification of UI templates.
• UI language plug-in modules. Two language modules are included in the

current prototype: SUL and XHTML.
The following table shows the code size of the above components.
Table 4.1 Server code size
Lines of code
Protocol module 1000
OpenXUP server 7000
UI template viewer 2500
UI language modules 15000
(SUL and XHTML)
Total 25500
In the remainder of this section, we describe the above components in details. In

addition, we will also discuss the performance of the server-side runtime framework as
well as the effectiveness of the development environment. The client side of the
framework will be described separately in the next chapter.
4.4.1 Protocol Module

The protocol module implements XUP messages. It leverages .NET's SOAP and XML
serialization [Oba03] support, so only a small amount of XML is manually parsed
through DOM.
The protocol module has two primary responsibilities. When processing a client request,
the module is responsible for parsing the UI event from XML. Similarly, when
Part 1: OpenXUP 86
generating a server response, the module is responsible for serializing the UI updates
into XML.
4.4.2 OpenXUP Server

The OpenXUP server is in deployed as a toolkit running inside the ASP.NET web
application server. As a result, it offers a set of event-driven .NET APIs for application
development.
The server toolkit is structured into two layers: a base framework layer, which is
independent of actual UI languages, and a language dependant layer, consisting of UI
language plug-in modules. This design makes the framework very extensible, since new
UI languages can be supported by plugging in additional UI language modules, and the
source code of the based framework does not need to be changed or recompiled. In
addition, the server leverages XML namespaces to support the mixture of multiple UI
languages in the same user session within an application. Currently, the framework
provides UI languages modules for SUL and XHTML, so an application can mix UI
controls from them to provide the desired UI appearance and behavior.
During runtime, the server maintains a UI model (a tree of UI controls) in the memory
for each user session, and XUP messages are passed between the client and server to
synchronize this UI tree and the client's UI view (which is what visually perceived by
the end user). An application developer uses the server-side APIs to handle events from
the client and make modifications to the UI tree; in turn, those UI modifications will be
passed back to the client for rendering. Similarly, when the end user interacts with the
UI view, the corresponding changes in the client-side UI model will be propagated back
to the server via XUP messages.
Since the server runs inside ASP.NET, XUP applications can leverage existing
middleware and backend infrastructure, such as ADO.NET55, COM, Web Services, and
business logic packaged in .NET class libraries. Furthermore, the server toolkit only
handles user interface processing, leaving backend components intact. Consequently,
55
http://msdn.microsoft.com/en-us/library/h43ks021(VS.71).aspx
Part 1: OpenXUP 87
web applications developed using OpenXUP APIs should be able to preserve and reuse
all existing application logic code and backend components.
Finally, the OpenXUP server can run on any systems with ASP.NET 2.0, including
Linux-based systems with Mono56 installed. We also plan to implement a Java-based
server toolkit running inside servlet containers; this will provide a set of event-driven
Java APIs for developing highly interactive web applications. In addition, this would
allow XUP applications to reuse Java-based backend components, such as EJB.
4.4.3 UI Templates
The framework supports a UI templating mechanism to allow the declarative
specifications of UI definition. UI templates are stored in .xut files (XUP UI Template)
and can be made globally available or application specific. Global UI templates are
located in a central server directory and can be accessed by all XUP applications;
whereas application-specific templates are located in a directory allocated for that
particular application and therefore can be accessed by that application only.
The UI tree specified in a template can be denoted as read-only. For a read-only UI tree,
there is only one memory instance shared among all user sessions within the application.
Furthermore, if the template is a global template, the memory instance will be shared
among all user sessions in all XUP applications. This design allows the OpenXUP
server to conserve memory when running many XUP applications with large amount of
user sessions.
The implication of a read-only UI tree is that it cannot be changed by application code

during runtime (e.g., cannot add controls to the UI tree or update the foreground color of
a control). However, there are many situations where a UI control does not need to be
changed; for example, a label control with static text which never changes. Similarly,
while the retailer side of XCat allows retailer's employees to edit the electronic catalog,
the same category tree in the buyer side of XCat is not editable. That is, buyers can only
browse through categories and search for products. In this case, the UI control that
represents the category tree can be made read-only. Since the category tree may be quite
Part 1: OpenXUP 88
large (e.g., thousands of tree nodes each representing a product category), this will save
a significant amount of memory in the OpenXUP server.
Finally, to assist the authoring of UI templates, the framework also provides a template
viewer, which allows templates to be edited and previewed. The template viewer is
structured into two layers: a base layer, the viewer core, which is independent of actual
UI languages, and a language dependant layer, consisting of UI language plug-in
modules. This design makes the template viewer very extensible, since new UI
languages can be supported via additional UI language plug-in modules, and the source
code of the viewer core does not need to be changed. In addition, the template viewer
leverages XML namespaces to support the mixture of multiple UI languages (e.g., SUL
and XHTML) in the same template. As a result, the template viewer allows developers
to mix UI controls from them to achieve the desired UI appearance and behavior during
design time.
For development efficiency, the template viewer allows multiple UI templates to be

loaded and previewed instantly. We also support minor editing functionalities such as
adding and modifying event selectors. In the future, we plan to support Visual Studio
integration to enable full featured drag-and-drop style UI editing.
4.4.4 SUL
For the implementation prototype, we first considered XUL and XAML. XUL was
originally designed to model Mozilla's menus and windows. Its major drawback is the
lack of standardization; there is no single specification that clearly describes the
language. XAML is Microsoft's XML language for their next generation Windows
desktop and web UI. However, XAML is unfortunately tightly-coupled with Windows
user interface.
Therefore, for prototyping purpose, we have created SUL (Simple User Interface
Language). Comparing to XUL and XAML, SUL is more compact and less verbose, so
it is better suited for network transmission. In addition, its simplicity allows it to be
56
http://www.mono-project.com/
Part 1: OpenXUP 89
easily implemented in multiple platforms. SUL is very open and its specification is
publicly available [Yu07a].
Although simple, SUL provides many rich UI controls not found in HTML; for
example, combo box, tabbed panel, progress bar, property sheet, and context menu. In
addition, it offers a file upload control, which is capable of uploading multiple files with
progress indication. SUL also supports common desktop-style interactions such as
double clicking, drag-and-drop, and tooltips. Finally, to facilitate the proper positioning
of UI controls, SUL provides a variety of layout managers, including form layout (with
absolute coordinates and attachment), box layout (in a horizontal or vertical flow), and
grid layout (in a table with cells).
4.4.5 Tracking UI Updates

One of the key features of OpenXUP is the ability to deliver incremental UI updates.
This requires the framework to track UI changes.
Consequently, whenever the application code makes any changes to the UI model, the
server needs to record that in the user session. To accomplish this, the server intercepts
and records all calls from the application that attempt to make changes to the UI model.
4.4.6 Performance Analysis

In this section, we analyze the performance of OpenXUP's server-side runtime, in three
aspects: processing speed, memory requirement, and network bandwidth consumption.
All tests are performed on PC running Windows Vista Business 32-bit SP1, with Intel
Core 2 Duo (T7200) 2.0 GHz processor and 2GB RAM.
Since the protocol is XML-based, our first concern is the performance of XML parsing.
Therefore, we measured XML parsing speed inside the server, with two sets of sample
data. The first sample data is the UI tree of the initial XCat user interface in SUL,
consisting of a main window with a menu bar, a tool bar, a tree on the left, and a tabbed
panel on the right (see Figure 4.1). This sample contains 5 Kbytes of data, and
represents the size of a medium to complex UI. For the second sample, we simply
Part 1: OpenXUP 90
duplicate the first sample 6 times, resulting a data size of 30 Kbytes. We believe 30
Kbytes is more than enough to describe a large and extremely complex UI.
Table 4.2 XML parsing performance
10 100 1000 10000

iterations iterations iterations iterations
5 Kbytes (108 lines) 5ms 44ms 427ms 4276ms
30 Kbytes (648 lines) 38ms 189ms 1682ms 16458ms
100,000
16,458
10,000
4,276
1,682
1,000
Milliseconds
427 30 Kbytes (648 lines)

189 5 Kbytes (108 lines)
100
38 44
10
5
1
10 100 1000 10000
Iterations
Figure 4.11 XML parsing performance
For each sample data, we iterate the parsing task a number of times, as each iteration can
be regarded as the parsing of a single event request. The result is shown in Table 4.2. As
illustrated in the accompanying Figure 4.11, the performance slightly increases as the
number of iterations increases, mainly due to the initial overhead of parser initialization.
Overall, the XML parsing performance is more than adequate, and the parsing time is
Part 1: OpenXUP 91
almost negligible when comparing to network transmission and application execution

time.
Next, we move onto measuring the server's request processing performance. Our goal is
to measure the pure request processing time inside server, excluding application code
(UI logic) execution time and network transport time. Specifically, this includes parsing
the event request (average less than 1Kbytes), identifying the user session and searching
for event handlers, parsing the UI template (5Kbytes for XCat), keeping track of UI
updates from application code, and generating response (1 to 5Kbytes).
For this measurement, we have developed a client-side command-line program that

issues concurrent XUP event requests to the server, emulating multiple users using the
XCat application at the same time.
Table 4.3 Request processing performance
1 request 10 requests 50 requests

10 users 12ms 42ms 160ms
100 users 65ms 357ms 2229ms
500 users 269ms 2712ms 24135ms
1000 users 504ms 5938ms 55534ms
Multiple tests are performed for 10 to 1000 concurrent users. And for each user, the test
program issues a number of event requests in sequence, ranging from 1 to 50 (i.e.,
corresponding to the number of interactions from a single user). The measurement is
taken in the server (in order to exclude network time) and recorded in Table 4.3. As
illustrated in the accompanying Figure 4.12, the server performs quite well. There is a
slight degradation when reaching 500 concurrent users with 10 requests or more. This is
most likely due to OS and processor limitations, since our test PC is not a server-grade
machine.
Part 1: OpenXUP 92
100,000
55,534
24,135
10,000
5,938
2,229 2,712
Milliseconds
1,000
504 50 requests
357
269 10 requests
160
1 request
100
42 65
10 12
1
10 100 500 1000
Concurrent users
Figure 4.12 Request processing performance
As for memory requirement, the main focal point is the data structure representing the
UI model tree, since each user session in an application has an instance of the UI tree.
Due to the fact that the server only keeps the data model of the UI tree, not the physical
UI view, the data structure is actually quite compact.
We have taken a measurement of the UI tree for XCat, with 24 product categories
loaded, and an average of 12 attributes in each category. This medium to complex UI
model consumes only 30,436 bytes of memory. With 1000 concurrent users, only 30MB
of memory is needed – a fairly small number in today's PC and server configurations.
As mentioned before, our templating mechanism supports the notion of read-only UI

tree, for which there is only one memory instance shared among all user sessions within
the application. Furthermore, for a global template, the memory instance will be shared
among all user sessions in all XUP applications. This design significantly reduces
server's memory requirement when running many XUP applications with large amount
of user sessions. As a result, we have not seen any memory-related scalability issues in
our server runtime.
Part 1: OpenXUP 93
Overall, the performance and scalability of OpenXUP-based applications is on-par with

traditional page-based web applications; e.g., applications based on ASP.NET Web
Forms or JSP. In addition, our programming model requires less lock / synchronization
and therefore offers more concurrency than traditional web application models. For
example, in applications based on ASP.NET Web Forms, the server serializes all
requests when the session is marked read and write, forbidding any concurrent requests
on the session57.
To analyze network bandwidth requirement, we distinguish between two types of web-

based UI: form-filling and incremental. A form-filling application consists of a series of
forms, collecting information from the end user. In this type of application, each form is
distinct from the other, so the same amount of network traffic will be generated whether
the application is built with OpenXUP or as a traditional HTML application. However,
for applications with incremental UI, where each user action only results a small portion
of the UI to be changed, OpenXUP has significant advantage.
Consider OpenXUP-based XCat versus HTML-based IntuiCat58, both having the

identical catalog editing functionalities. In IntuiCat, the HTML-based UI is
approximately 8Kbytes, and each user interaction results the entire UI to be refreshed. In
XCat, the initial SUL-based UI is 5Kbytes, and each user interaction results an
incremental UI update of less than 1Kbytes. So for 10 user actions, IntuiCat requires
80Kbytes of network transmission, whereas only 14Kbytes is required for XCat (i.e.,
5Kbytes for initial UI and 1Kbytes each for next 9 user interactions). For 1000 users, the
combined difference between the two approaches is about 66000Kbytes.
4.4.7 Developer Productivity

Our prototype packages the event-driven APIs in a set of .NET class libraries. As a
result, developers can use the rich development environment of Visual Studio59 to create
57
Although our prototype is based on ASP.NET, it only leverages ASP.NET's SOAP and web
services facilities, not its session management.
58
IntuiCat is a multi-buyer / multi-seller e-commerce marketplace solution, developed by the author's
employer, Martsoft Corporation. It includes an HTML-based e-catalog editor. XCat is our attempt
to improve the UI usability and productivity of the e-catalog editor.
59
http://msdn.microsoft.com/en-us/vstudio/default.aspx
Part 1: OpenXUP 94
XUP applications using any of the .NET languages. Building an XUP application simply
involves the compilation of the source code into a .NET assembly, which can then be
deployed by copying it to the appropriate server directory.
Developing an XUP application is very similar to developing desktop-based UI, due to

the fact that the proposed event-driven APIs closely resemble their desktop counterparts.
As a result, for applications with complex UI requirements, OpenXUP offers clear
productivity gains over traditional HTML-based UI development.
We will again compare XCat to IntuiCat to show the productivity gains. The following
table illustrates different components of the two applications and their respective code
sizes. Note that we are only concerned with UI code, so the numbers shown do not
include, for example, code to authenticate users or to access backend databases.
Table 4.4 Code size comparison (classic HTML vs. OpenXUP)
IntuiCat XCat
Data structure for product 350 lines 300 lines
categories, attributes, etc.
UI definition 300 lines (HTML & 100 lines (SUL template)
JavaScript)
UI logic (manipulating UI) 3650 lines 1200 lines
Total 4300 lines 1600 lines
As illustrated in Table 4.4, the data structure sizes are about the same. For UI definition,
IntuiCat is 3 times larger than XCat; this is due to the fact that HTML was not originally
designed for UI, and consequently achieving the desired appearance requires a large
amount of HTML and JavaScript to tweak the layout. Similarly, the UI logic code in
IntuiCat is also significantly larger than XCat. Although IntuiCat is developed in Java
whereas XCat is developed in C#, we believe the language differences do not contribute
to the huge code size differences. A simple code review shows that most of IntuiCat's UI
logic code is in outputting further HTML and JavaScript code to fine tune the UI and to
tweak the layout. On the contrary, everything in XCat is achieved through simple
manipulation of the UI tree, and the framework takes care of generating the incremental
Part 1: OpenXUP 95
UI updates that contain the updated SUL markups. There is never a need for developers
to manually output any SUL markups.
In general, we believe for simple UI, OpenXUP and the classic HTML approach may
not differ much; however for applications with complex UI, OpenXUP may potentially
offer developers significant productivity gains over the classic HTML approach.
For rich client based approaches, such as Java Applet, ActiveX, and .NET Smart Client,
the resulting UI should look very similar to OpenXUP, as they all offer desktop-grade
UI experiences. However, OpenXUP has a major development advantage – the UI
definition is declared (e.g., in SUL), not programmed. In the rich client based
approaches (as well as traditional desktop UI toolkits), the UI definition is typically
programmed (e.g., in Java, C#) and mixed together with UI logic code. Obviously,
separating UI definition from UI logic allows them to be independently updated. In
addition, having the UI definition in a declarative format simplifies graphical GUI
design; that is, it is much easier for a GUI design tool to generate and maintain
declarative XML markups than to maintain (i.e., output and parse back) Java or C#
code.
4.5 Summary
In this chapter, we have presented OpenXUP, a server-side rich UI framework that
provides a model bridging the gap between the traditional desktop (client / server) based
and web page (browser) based user interface paradigms. It allows centralized application
management with zero client-side administration, and it has no downloaded code
(binary or script) to execute on the client side, so there are no security concerns or IP
protection issues.
To bring web applications closer to desktop applications, the framework provides a

programming environment that resembles desktop applications. That is, rich user
interface controls, such as those found in Windows Forms and Java Swing, can now be
used in building web applications. Through OpenXUP's event-driven APIs,
programmers implement event handlers on the server side. They no longer need to
process form data as URL-encoded strings. In addition, user interface changes are
Part 1: OpenXUP 96
delivered from the server to the client as incremental updates, not one full page at a
time, so end users will no longer experience slow page refreshes.
Finally, the OpenXUP server toolkit does not affect the business logic part of the
application. Developers can use any existing technologies (e.g., .NET classes, Web
Services) to program business logic. In addition, since the OpenXUP server is designed
to run within existing web application servers, web applications built with OpenXUP
can leverage all existing backend infrastructure, such as Web Services and backend
databases. This enables an easy migration path from classic HTML-based applications
to rich internet applications based on OpenXUP, since all existing business logic and
backend components can be preserved and reused.
Part 1: OpenXUP 97
Chapter 5. Delivering RIAs via Thin Client
5. Delivering RIAs via Thin Client

With the help of rich web client technologies, developers are creating rich internet
applications in response to end users' growing demand in richer web experiences.
However, most of existing rich web client technologies share one thing in common –
they are all fat client based and thus require application code to be downloaded and
executed on the client side in order to enable rich user interfaces. The application code
could be in either binary (e.g., Java Applet, ActiveX) or script (e.g., Flash, AJAX), and
must be downloaded and executed for each web application the end user interacts with.
As previously mentioned, it is well-known that the client-side execution of downloaded

code may impose security risks to the end user's computer. Although both .NET and
Java have sophisticated security sandbox to restrict the execution of downloaded code,
security remains to be a major concern. As a result, many end users choose not to install
downloaded code when prompted, even though they are certified by public CAs. In
addition, due to the same security concerns, many organizations do not allow the
installation of downloaded code, with a few exceptions for well-known applications
such as Acrobat PDF Reader. Finally, executing downloaded scripts (as in AJAX) has
similar security problems; for example, cross site scripting60 and script injection [Sil08]
are common techniques to compromise browser's security.
To overcome limitations in the fat client based approaches, in this chapter we present a
thin client, XUPClient, for delivering rich internet applications to end users. XUPClient
is the client side of the OpenXUP rich UI development framework. It aims at closing the
gap between web and desktop user interfaces by allowing end users to interact with rich
UI controls only found in desktop environment, while remaining thin in terms of
application logic; i.e. all application code resides on the server side, and the client only
renders declarative UI definitions. Being a thin client, XUPClient has the following
advantages:
• There are no client-side security issues since the client does not execute
downloaded code.
Part 1: OpenXUP 98
• It provides a better security model for sensitive business data, since only UI
definitions, not application data, are transported over the network and
manipulated on the client side. This is especially important for applications with
high security requirements, such as government and financial applications.
• It facilitates centralized application management. As a result, there is no client-

side administration cost.
Since XUPClient is based on XUP, user interfaces are updated incrementally, not one
page at a time, so users will no longer perceive slow and annoying page refreshes.
Furthermore, XUPClient employs an asynchronous communication mechanism to avoid
blocking user interactions, thereby giving end users a more pleasant and responsive user
interface. Finally, XUPClient's open architecture facilitates the support of rich UI
languages such as SUL, XAML, and XUL. This allows XUPClient to bring web
applications closer to their desktop counterparts. As a result, when using XUPClient,
end users will find little user interface differences between web and desktop
applications.
The rest of the chapter is organized as follows. In the next section we describe the
design of XUPClient. After that we illustrate its runtime behavior using an example.
Finally, we conclude this chapter with a discussion of its implementation.
5.1 Thin Client Design

The term fat client refers to client software that executes application-specific code. On
the other hand, if a client only renders UI definitions, we call it a thin client. Most
existing rich web clients are fat clients, since they all download and execute application-
specific code. In the classic HTML approach, the web browser is considered to be a thin
client since it only renders declarative markups (with perhaps a small amount of
JavaScript for data validation). To enhance the user interactivity, AJAX improves upon
the classic HTML approach by using advanced JavaScript to handle events and update
HTML DOM on the client side. However, the web browser is no longer a thin client in
60
http://en.wikipedia.org/wiki/Cross-site_scripting
Part 1: OpenXUP 99
this case, since a large amount of application-provided JavaScript code is downloaded

and executed in the browser, on a per-application basis.
Our primary goal in designing XUPClient is to provide a rich web user interface to end
users while letting developers keep their application code on the server side. To avoid
executing application-specific code, XUPClient interprets XML-based declarative UI
markups and then let end users interact with the resulting user interface model.
As mentioned in previous chapters, a UI model is a representation of the user interface

which the end user perceives and interacts with. Given an XML markup fragment,
XUPClient builds a UI model by mapping XML elements to UI controls, such as
buttons and panels, and by mapping XML attributes to UI properties, such as color and
size. Based on this abstraction, we can provide support to any XML-based declarative
languages, such as SUL, XUL, XHTML, and WML. However, it is more advantageous
to use rich UI languages such as SUL and XUL which focus on fluid, desktop-style user
interfaces. These languages provide a rich set of UI controls suitable for building highly
interactive applications; whereas XHTML and WML are more appropriate for page-
based hypertext applications.
In addition to UI controls and their properties, a UI model also includes events, which
are fired when the end user interacts with the UI controls. Some UI events are common
to most UI controls, whereas others are only fired by specific UI controls. For examples,
events like "mouse click" and "key press" are available to most controls, while the
"window closed" event is specific to the UI control "window". A rich UI language such
as SUL or XUL includes a large set of UI events, which allow for rich user interactions;
whereas a page-based language such as XHTML only provides a limited number of UI
events.
Having a set of rich UI controls and events alone is insufficient to provide a highly
interactive user interface. What is needed is the UI behavior, which defines how to
handle user actions and update the UI model. In fat client based approaches, such as
AJAX and Java Applet, the UI behavior is provided by the downloaded UI logic code
which handles UI events and manipulates the UI model. However, a thin client does not
Part 1: OpenXUP 100
download or execute application code, so some other mechanism is needed to provide

the UI behavior.
To provide the necessary UI behavior, XUPClient leverages XUP, which is a SOAP-

based protocol that delivers UI events and incremental UI updates. Since it delivers UI
events from the client to the server, application code on the server side now has the
opportunity to handle the events caused by user actions and behave appropriately. In
addition, since XUP delivers incremental UI updates from the server to the client,
XUPClient can render the changes to its UI view without executing application-specific
code.
For example, the <addUIElement> element may be used to add a list item to a list box,
and the <updateUIAttr> element may be used to update the background color of a
button. Therefore, with the protocol level support of the delivery of UI events and
incremental UI updates, XUPClient can provide rich and dynamic UI behaviors without
downloading or executing application code.
5.1.1 Architecture
XUPClient's architecture is depicted in Figure 5.1. We take a layered approach to ensure
its modularity and extensibility.
Figure 5.1 XUPClient architecture
First, XUPClient makes a clear separation between the model and view of the user
interface. The model is a data representation of the user interface in an XML-based UI
language such as SUL or XUL; whereas the view corresponds to the rendering of the
Part 1: OpenXUP 101
model using the underlying native user interfaces toolkit offered by the platform. The
decoupling of the model and view of a user interface allows XUPClient to effectively
support XML-based UI languages.
In addition, XUPClient separates the UI model into two layers: the abstract UI layer and
the concrete UI model. The abstract UI layer models UI controls as XML elements and
UI properties as XML attributes; it interfaces with the XML serializer to generate XML
from UI controls and to create UI controls from XML. The concrete UI model contains a
tree of UI controls from a rich UI language such as SUL or XUL. For example, the root
of the UI tree is a window, with two panels as its children; and one of the panels in turn
contains two children, a button and a label. The abstract UI layer allows XUPClient to
support multiple UI languages.
The event processor is responsible for translating native UI events to XML-based UI

events defined in the appropriate UI languages. It listens to native UI events from the UI
view, and sends the translated XML UI events to the server via the XML serializer.
Finally, the XML serializer is responsible for the parsing and serialization of the UI
controls and events into XML. It communicates with the protocol layer to deliver UI
events to and receive UI updates from the server.
At runtime, when an end user manipulates the UI view (e.g., click on a button, input
some text), a native UI event is fired from the view. The event processor captures that
event and translates it to an XML UI event in the UI language used by the current UI
model. It then passes that to the XML serializer which delivers the UI event to the server
through the protocol layer.
Similarly, when the web application on the server side modifies the user interface, the
server will send the corresponding UI updates to XUPClient, which will in turn update
its UI model appropriately. After that XUPClient pushes the updates to the UI view by
rendering the user interface changes.
Part 1: OpenXUP 102
5.1.2 UI State Management

Unlike web browsers, XUPClient maintains the UI state (both model and view) of the
web application it interacts with. This is the key to displaying incremental UI updates,
since without UI state, a thin client would have to redisplay the entire screen / page of
the application for each UI update. In addition, since the client now maintains the UI
state, the server-side application code no longer needs to regenerate the entire screen /
page every time.
Through XML protocol elements, XUPClient allows application code on the server side
to manipulate its client-side UI state. In this case, UI updates are marshaled through
XUP protocol responses, and each response may contain multiple additions
(<addUIElement>), removals (<removeUIElement>), and updates (<updateUIElement>
and <updateUIAttr>) of UI controls and properties. For example, the UI updates in a
response may instruct XUPClient to add a button to a panel and change the text of the
window title.
To allow application code on the server side to process user inputs, XUPClient also
sends UI updates to the server. The UI updates represent the changes to the UI controls
and properties caused by user actions; for example, text entered in a text box and a
check box been unchecked. Unlike the UI updates originated from the server side, the
UI updates caused by end user actions typically include updates but not additions or
removals of UI controls and properties. Note that XUPClient does not send UI updates
immediately; they are queued and sent to the server within an event request. This
accommodates for data entry scenarios where the user enters text in a number of fields
and then clicks a button to submit the information when done.
In summary, changes to the UI model are communicated bi-directionally. The UI values

sent by the client (in event requests) are originated from end user actions; whereas the
UI updates received by the client (in server responses) are originated from application
code.
Part 1: OpenXUP 103
5.1.3 Network Event Delivery

It is very important for a thin client to minimize network traffic caused by UI events,
while at the same time provide a rich and interactive user interface. First, the thin client
must not send every UI event to the server; it should filter out high frequency native
events such as mouse movements. Second, it should only send events of interest. That is,
it should only deliver events that the server-side application wants to receive. For
example, a mouse click on a menu bar will display a submenu. This behavior can be
handled locally at the client side, so the application has no interest in receiving this
mouse click event. Another example, typing in a text field generates a key press event
for each key stroke. But the application may not care about each key stoke; it only wants
to receive the entire text when the user clicks on a button labeled "submit". In this case,
the application is interested in the mouse click event on the "submit" button, but not
interested in the key press events in the text field.
To support the event selection concept described above, XUPClient supports the notion
of event selector, which allows an application to selectively receive events of interest.
As mentioned in previous chapters, event selectors are defined by applications and sent
to the client in XUP responses. Specifically, an event selector defines the type of event
and the UI control that fires the event; e.g. mouse click on a button. On the client side,
whenever the event processor captures a native UI event, it will first identify the event
type and the specific UI control that fired the event, and then search for a matching
event selector. If a match is found, the event will be sent to the server; otherwise, the
event will be discarded.
To further reduce unwanted event traffic, a thin client should be able to filter events
based on their content. For example, an application may want to receive mouse click
event on an "OK" button, but it only cares when the left mouse button is used, not the
middle or right mouse buttons. Another example, the application may be only interested
in specific key press events, such as delete or enter keys, from a tree control; so other
unwanted key press events should be filtered out by the client. Therefore, to support the
event filtering concept, the client supports the notion of event mask (as alluded earlier in
the last chapter), which may be attached to an event selector to define fine-grained event
filtering criteria specific to a particular event type.
Part 1: OpenXUP 104
<xup:selector id="s1" event="sul:key-down"

element="tree1" async="false"
xmlns:sul="http://www.openxup.org/2004/02/sul">
<xup:mask sul:keys="del enter"/>

</xup:selector>
Listing 5.1 Event selector example
Listing 5.1 shows an example event selector. Here the application is interested in the
"sul:key-down" event from a tree control with the ID "tree1". To ensure extensibility,
event types are XML namespace qualified, so "sul:key-down" means the "key-down"
event from the SUL language. In addition, the event selector also contains an event
mask, which specifies that only "del" and "enter" key events should be sent to the
application on the server side. The "sul:keys" attribute in the event mask is qualified by
the SUL namespace because the keys are defined in the UI language (SUL), not in the
protocol (XUP).
The combination of event selection and filtering mechanisms allows for dynamic and
versatile UI behavior. Applications can decide what, where, and when to receive UI
events. For example, assuming a user is trying to fill in an account creation form, with
text fields such as username, password, credit card number, and a "create" button. The
application may simply add a single event selector to select the mouse click event on the
"create" button. In this case, no events will be generated while the user is entering text;
the values of the text fields will be sent together with the mouse click event on the
"create" button. Alternatively, the application may wish to add two additional event
selectors, for selecting the "tab" key event on the username and credit card number text
fields. This allows the application to immediately check duplications of user names and
validate the credit card number when the user tabs away from the respective text fields.
Essentially, with the two additional event selectors, the application has enabled a more
fluid user interface by providing immediate feedback to the end user.
5.1.4 Synchronous and Asynchronous Events

Normally UI events are sent to the server synchronously. This has two consequences.
First, events are serialized; i.e., events are sent one after another, not at the same time.
Part 1: OpenXUP 105
Second, the end user is blocked from further interactions when the network event
delivery is in progress; that is, the client will respond to further user input only after it
has received and processed the event response from the server. This can be characterized
as a typical "click and wait" interaction pattern.
To make the user interface more responsive, XUPClient supports asynchronous event
delivery. That is, the client sends the UI event to the server in the background, while at
the same time continues to accept user inputs. As a result, the user spends less time in
waiting; she can continue to work with the application while the client communicates
with the server in the background.
As also mentioned in the last chapter, asynchronous event delivery should be used when
it takes the application a long time to process the event on the server side. For example,
a mouse click event on a "search" button might take the application a long time to
process since it needs to search through millions of database records. In this case, the
event should be delivered asynchronously, so that the user can continue interacting with
rest of the application functionality while the search is in progress.
In addition, an application should use asynchronous event delivery if it expects network

delays; e.g., if the application was to be deployed on the internet with global access
rather than in a corporate intranet. In our account creation example in the last section,
the "tab" key events from the username and credit card fields should be sent
asynchronously; otherwise, after pressing the "tab" key, the user might not be able to
instantly enter text in the next text field.
Finally, an application must designate asynchronous event delivery using event

selectors. As shown in Listing 5.1, the "async" attribute indicates whether the event
should be delivered synchronously (if false) or asynchronously (if true).
5.1.5 Thin Client vs. Fat Client

In this section we discuss some of the differences between thin client and fat client from
the architecture's point of view.
Part 1: OpenXUP 106
5.1.5.1 Code Location

In general, the user interface of an application can be separated into definition and
behavior. By UI definition, we mean the description of the UI controls and their
properties. The UI definition of an application can be programmed via an imperative
language such as Java or C#, or declared in an XML-based language such as XUL or
XHTML. By UI behavior, we mean how the application changes its user interface in
responses to user actions. Since UI behavior is dynamic in nature, it is usually
programmed, not declared. Hence we use the term UI logic to refer to the programmatic
code that provides the dynamic UI behavior.
In fat client based approaches, UI behavior is always executed at the client side via
downloaded application-specific UI logic code. For instance, with Java Applet, both UI
behavior and UI definition are programmed in Java and executed on the client side; and
for AJAX, the UI definition is provided in HTML, but the UI behavior is programmed
in JavaScript and executed on the client side. On the other hand, in our thin client based
approach, the client only renders XML-based rich UI definition, leaving the execution of
the UI behavior to the UI logic code on the server side.
In addition, in fat client based approaches, the client must understand some business
logic, in order to render business data and communicate with the server-side business
process. On the contrary, our thin client only knows about UI, not business logic or data.
Furthermore, in fat client based approaches, application data is transported over the
network and manipulated at the client side, potentially exposing sensitive business data
(both at the client side and over the network). As an example, in a shopping session, a
purchase order must be serialized into binary or XML to be transported to the client side
for display. For better security, our approach does not transport application data over the
network; only UI markups are sent over the network and sensitive business data never
leaves the server side.
To avoid communication overhead, UI logic code should stay close to application logic
code and application data. Therefore, in our thin client approach, UI logic resides on the
server side and developers do not need to worry about the communications between UI
logic and application logic code. However, in fat client based approaches, UI logic is on
Part 1: OpenXUP 107
the client side, whereas application logic is on the server side, so developers must
manually manage remote communications (e.g., via some form of RPC).
Finally, since fat clients require UI logic code to be downloaded and executed, a desktop
PC may end up with many downloaded applets or plug-ins installed (and perhaps
multiple versions of them). The number could become overwhelming if all web sites
featuring rich content were to install their own applets / plug-ins. In addition, once
installed, the downloaded code may be hard to remove. This is clearly a management
headache for both end users and IT staff.
In summary, in our thin client approach, the declarative portion of the application (i.e.,
UI definition) is processed by the client, whereas the programmed portion of the
application (i.e., UI behavior and business logic) is executed on the server side. This
separation allows XUPClient to provide a rich UI experience to the end user while at the
same time ensuring a secure desktop environment.
5.1.5.2 Leveraging Desktop Computing Power

As desktop computers are very powerful these days, it is desirable to leverage that
computing power. For example, desktop machines can provide distributed computing
resources in a scientific computing grid. Obviously, fat clients are more suitable for this
type of data crunching applications, since application code can be executed on the client
side.
On the other hand, for management and security reasons, most business applications
keep data processing on the server side, and the desktop is only used for inputting data
and displaying results. Hence XUPClient is ideal for these applications. Furthermore,
although XUPClient does not take advantage of the desktop's data processing capability,
it can fully leverage the desktop's graphical computing power to render rich UI and
graphics.
5.1.5.3 Network Bandwidth Considerations

The event selection and filtering mechanisms introduced earlier eliminate most of
unwanted event traffic. In addition, since XUPClient maintains UI state, many user
Part 1: OpenXUP 108
actions are handled locally by the UI controls themselves. In our XCat example in the
last chapter, the tree control that represents the product categories remembers its own
content – the hierarchy of tree nodes. Therefore, once the tree is populated, expanding a
tree node (clicking on the '+' icon) is entirely a local operation, without a roundtrip to the
server61.
Furthermore, simple data validations can be performed locally on the client side,
without roundtrips to the server. For instance, a text field control may support
declarative input patterns such as date format, number range, text length, and even
regular expressions; all these can be supported by the UI control locally, without
involving application code on the server side. More complex validations need to be
handled at the server side. However, the same applies to the fat client approaches. For
example, in XCat, adding a product item into the catalog involves checking for
duplicates in the backend product database; obviously this has to be done on the server
side.
Although a fat client does not send UI events to the server, it must send and receive
application data to and from the server. The application data are usually serialized
application objects, in XML or binary form. Therefore, the amount of traffic generated
by a fat client is no less than the amount of event traffic from XUPClient.
To minimize network traffic, fat clients may cache application data on the client side. In
XCat, a fat client may cache the product items for a visited category, so when that
category is revisited, it does not need to retrieve those products from the server again.
However, this complicates the client-side logic, as the fat client needs to manage data
caching. Furthermore, the data cache may become stale as XCat is a multi-user
application, so the product items must be retrieved from the server again anyway.
Finally, fat clients require not only the initial download of application binaries, but also
frequent re-download of updates and new releases. Since the binaries could be quite
61
Of course, the end user has the option to force a remote operation in order to refresh the client-side
product category tree. In this example, she may simply hold down the shift key while clicking on
the '+' icon. This will cause the client to send a "tree-node-expanding" event to the server, and the
application on the server side in turn will update the content of the tree control.
Part 1: OpenXUP 109
large, frequent re-download may take a long time and therefore become an annoyance
for end users. As a result, this is not ideal for Web 2.0 and open source applications that
require "release early, release often" [Ray99].
5.1.6 XUPClient vs. Web Browsers

XUPClient provides a rich set of UI controls that dramatically improves the usability
and productivity of web applications. It also eliminates the annoying page-refreshes that
are common in classics HTML-based web applications.
Most web applications require some type of data entry / form filling. However, with
browsers, the tab key traverses hyperlinks as well as input fields (often in an
unpredictable fashion, depending on how the HTML is structured). In addition, there is
no keyboard shortcut to jump to a particular input field. XUPClient, on the other hand,
offers a desktop-like data entry user interface, with proper tab key traversals and
keyboard shortcuts.
In addition, browser-based user interfaces are not very accessible. Due to poor keyboard
shortcut support, users must click on links or buttons to perform actions. XUPClient, on
the other hand, offers UI controls that conform to the native desktop's UI guidelines. As
a result, it provides the same level of keyboard shortcut support as native desktop UI,
allowing users to perform many operations without the mouse. Furthermore, just like the
native desktop UI, user interfaces rendered by XUPClient can be controlled by voice
recognition software, so users can use voice to interact with web applications (e.g., by
speaking menu / button labels), without relying on keyboard and mouse.
5.2 Running XUPClient

In this section we illustrate the runtime behavior of XUPClient. To simplify the
illustration, we use the classic "Hello World" example to demonstrate the user
interactions. However, the same interaction patterns apply to rich internet applications in
the real world.
Part 1: OpenXUP 110
5.2.1 Accessing Web Applications

The first step is to access the web application. XUPClient uses XUP URLs to identify
and access applications. XUP URLs are very similar to HTTP URLs, except that the
URL scheme is either "xup" or "xups" (for SSL connection). Figure 5.2 shows an XUP
URL inside XUPClient's open URL dialog box. Note that the part of URL after the
hostname depends on how and where the application is deployed.
Figure 5.2 Open URL dialog
Alternatively, since XUPClient supports browser integration, the user can access an
XUP-based web application by either typing the appropriately XUP URL in the web
browser's address bar or clicking a hyperlink containing the XUP URL. Finally, since an
XUP URL can be stored in a Windows shortcut file, the user can just double click on the
file to launch the corresponding web application.
5.2.2 User Interactions

After the user enters the URL in XUPClient's open URL dialog box (or the web
browser's address bar), XUPClient sends the following startup request to the server:
<soap:Envelope xmlns:xup="http://www.openxup.org/2004/02/xup" ...>

<soap:Header> ...... </soap:Header>
<soap:Body>
<xup:startup application="HelloWorld"/>
</soap:Body>
</soap:Envelope>
Listing 5.2 Startup XUP request
The <xup:startup> element in the SOAP body is used to identify the application name,
"HelloWorld". After processing the above request, the server returns the following
startup response:
Part 1: OpenXUP 111

<soap:Header>
<xup:session application="HelloWorld" sid="X35759759"/>
</soap:Header>
<soap:Body>
<xup:startupResponse
xmlns:sul="http://www.openxup.org/2004/02/sul">
<xup:addResources>
<sul:image id="globe">R0lGODlhGAAYAMQAAP......</sul:image>
</xup:addResources>
<xup:addUIElement position="-1">
<sul:window id="w-main" title="Hello World">
<sul:button id="b1" position="..." image="globe"/>
<sul:label id="l1" position="...">. . . . . .</sul:label>
</sul:window>
</xup:addUIElement>
<xup:addEventSelectors>
<xup:selector id="sel1" element="b1" event="sul:action"/>
</xup:addEventSelectors>
</xup:startupResponse>
</soap:Body>
</soap:Envelope>
Listing 5.3 Startup XUP response
In the above SOAP header, the server returns a session ID to be used for identifying the
client in subsequent requests. The <xup:addResources> element in the body specifies an
image (globe icon) to be made available to the client. Note that since the image is very
small, its content is Base6462 encoded inline. If the image is large, it can be separately
downloaded by the client via HTTP GET. The <xup:addUIElement> element adds a tree
of UI controls to the client's UI model. Here, the root of the UI tree is the main window
of the "HelloWorld" application, which contains a button and a label. Finally, the
<xup:addEventSelectors> element includes an event selector ("sel1") which selects the
action event (i.e., mouse click) from the button.
XUPClient processes the startup response and displays the corresponding UI view. As
shown in Figure 5.3, the window contains a button with the globe icon and a label with
"……" as its text.
Part 1: OpenXUP 112
Figure 5.3 Initial window Figure 5.4 After button click
Now the end user clicks on the globe button, which causes XUPClient to send the
following event request:

<soap:Header>
<xup:session application="HelloWorld" sid="X35759759" .../>
</soap:Header>
<soap:Body>
<xup:event xmlns:sul="http://www.openxup.org/2004/02/sul"
type="sul:action" selector="sel1"/>
</soap:Body>
</soap:Envelope>
Listing 5.4 Event request for button click
Here, the request contains the SUL "action" event, which corresponds to the mouse click
action. For a button, the action event fires when the button is activated (by mouse click,
keyboard shortcut, etc.). Note that this event matches the event selector "sel1", which
instructs XUPClient to listen to action events from the button.
After the server-side application processes the event, the following response is returned:

<soap:Body>
<xup:eventResponse>
<xup:updateUIElement element="l1">
Hello World!!!
</xup:updateUIElement>
</soap:Body>
</soap:Envelope>
Listing 5.5 Event response for button click
62
http://en.wikipedia.org/wiki/Base64
Part 1: OpenXUP 113
Here, the event response contains an <xup:updateUIElement> element, which specifies

the new text for the label "l1". As a result, XUPClient will display the new label text
"Hello World!!!", as shown in Figure 5.4.
5.2.3 Desktop-like User Experience

Unlike web browsers, XUPClient does not have a main window. It leaves all the screen
real estate to display web applications, and its control functionalities are housed in a
menu in the system tray.
Figure 5.5 XUPClient's system menu
As a result, web applications are no longer confined by browser windows; they can have
floating toolbars, modal dialogs, etc. Essentially, with XUPClient, users will perceive
little difference between using a web application and using a desktop application.
5.3 Implementation
As a proof of concept, we have implemented the thin client in .NET. The prototype can
be deployed as a standalone Windows program or as a browser plug-in. Since the client
is a thin client, it needs to be deployed only once, not on a per-application basis. In
addition, installation should be fast as the client has a small footprint, currently about
500KB.
The standalone version of the client has full desktop support – its control functionalities
are housed in a menu in the system tray, leaving all the screen real estate to display web
applications. For browser integration, the client can be deployed as an Internet Explorer
plug-in, allowing users to interact with rich internet applications without leaving the
browser.
Part 1: OpenXUP 114
Similar to the OpenXUP server, XUPClient is structured into two layers: a base layer,
the client core, which is independent of actual UI languages, and a language dependant
layer, consisting of UI language plug-in modules. This design makes the client very
extensible, since new UI languages can be supported via additional UI language plug-in
modules, and the source code of the client core does not need to be changed or
recompiled. In addition, the client leverages XML namespaces to support the mixture of
multiple UI languages in the same application. Currently, the client provides UI
languages modules for SUL and XHTML, so XUP applications can mix UI controls
from them to achieve the desired UI appearance and behavior. Since both XUPClient
and the UI template viewer (discussed in the last chapter) provide the visual rendition of
UI controls, the UI languages plug-in modules are actually shared among the two.
XUPClient also supports embedded rich media controls, which can be used to render
PDF documents, display Flash animations, and play audio / video clips, etc. These
multimedia UI controls can be embedded in both SUL and XHTML.
In addition, XUPClient should be fully interoperable with all OpenXUP server

implementations, including the planned Java servlet-based server we mentioned in the
last chapter. This is due to the fact that they all speak the common protocol, XUP. For
example, an end user can use XUPClient to interact with two XUP applications at the
same time: one written in Java, running in an OpenXUP server toolkit inside a J2EE
servlet container, and the other written in C#, running in an OpenXUP server toolkit
inside ASP.NET.
Table 5.1 Client code size
Lines of code
Protocol module 1000
XUPClient 9000
UI language modules 24500
(SUL and XHTML)
Total 34500
Part 1: OpenXUP 115
Finally, Table 5.1 summarizes the code size of the different components in the client
implementation. Note that the client-side UI language modules are larger than the
server-side ones, due to the fact that the client-side modules need to actually render the
UI. The combined footprint of the RIA framework, including the thin client, the server-
side runtime environment, and the development tools, is therefore about 60,000 lines of
code.
5.3.1 Protocol Module

Similar to the OpenXUP server, the client side also has a protocol module, which has
two primary responsibilities. When sending an event request, it is responsible for
serializing the UI event into XML, and when processing server responses, it is
responsible for parsing UI update instructions from XML.
In addition, the protocol module introduces two URI schemes. The "xup" scheme can be
used for accessing XUP applications through HTTP, and the "xups" scheme should be
used for accessing XUP applications through SSL-enabled HTTP connections (i.e.,
HTTPS). The syntax of XUP URIs is:
• xup://hostname:port/path-to-openxup-server?app=app-name&param1=v1&param2=v2
• xups://hostname:port/path-to-openxup-server?app=app-name&param1=v1&param2=v2
In the above URIs, "path-to-openxup-server" identifies the OpenXUP server executable,

since the OpenXUP's server side is designed to be deployed as a server toolkit running
inside existing web application servers (e.g., ASP.NET, Java servlet container). In
addition, "app-name" points to the specific XUP application running within the
OpenXUP server, and "param1" and "param2" are optional parameters that can be
passed to the application when initializing new user sessions.
XUP URIs are typically used by the client to access XUP applications. In addition, users
can type them into browser's address bar or click on hyperlinks that point to XUP URIs,
and XUPClient will be automatically launched to access the respective XUP
applications. Similarly, XUP URIs can also be typed directly into Windows Explorer's
address bar (i.e., Start menu -> Run) to access XUP applications.
Part 1: OpenXUP 116
5.3.2 Tracking UI Updates

To synchronize the UI state between the client and the server, XUPClient must notify
the server of any UI changes made by the end user. Therefore, the client also needs to be
able to track UI changes.
On the client side, XUPClient keeps track of all changes the end user makes on the UI
view; for example, checking a check box, entering text in a text box. Unlike the server
side, the client cannot directly intercept API calls as the changes are resulted from user
actions. Therefore, XUPClient use the following alternatives to track UI updates:
• If a user action triggers an immediate native event, the client will capture that
event to record the changes to the UI view. For example, when a user checks a
check box, the client can capture the corresponding native UI event and record
the UI change.
• If a user action does not trigger an immediate native event, or if the native event
is too frequent, resulting in performance hit, the client will record the initial
value of the UI control and compare it against the current value before sending
any UI updates to the server. For example, when the user is typing text into a text
field, a native UI event will be fired for each keystroke, potentially impacting the
client's performance. Therefore, in this case, the client will record the initial
value of the text field, and just before sending the next UI event to the server, it
will compare the initial value with the current value of the text field to determine
if the end user has modified the UI.
5.3.3 Performance Analysis

The performance of XUPClient is similar to standalone desktop applications. There is
no scalability issues since only a single user is operating the client and the user can only
interact with a limited number of XUP applications at the same time. In addition, due to
its thin client nature, XUPClient is only responsible for rendering UI and handling user
interactions, without intensive data processing.
Part 1: OpenXUP 117
On the network side, we already showed in the last chapter that our approach saved
considerable amount of network bandwidth for applications with incremental UI. In
addition, through the proposed event selection (event selectors) and event filtering
(event masks) mechanisms, the number of UI events sent over the network is greatly
reduced. In general, for the same type of application, the number of network events
generated by XUPClient is comparable to the number of HTTP requests (e.g., resulting
from button or link clicks) in the classic HTML approach.
For applications with rich UI, such as XCat, XUPClient may actually require less
roundtrips, due to the fact that rich UI controls are often capable of maintaining UI state,
so many user actions are handled locally by the UI controls themselves. For example, in
XCat once the category tree is populated, expanding a tree node (clicking on the '+' icon)
is entirely a local operation, without a roundtrip to the server63. Similarly, navigating
from the "Products" tab to the "Attributes" tab on the right hand side is also a local
operation. Comparing XCat to IntuiCat's HTML-based e-catalog editor, we found that
XCat requires 30% less server roundtrips than IntuiCat, in a typical user session that
includes a combination of browsing and editing tasks. Finally, this not only reduces
network traffic, but also the server load, since less network UI events means the server
has less requests to process, thereby improving the server's scalability.
5.4 Summary
Our thin client based design enables a rich web experience for end users while at the
same time ensures a safe desktop environment. With XUPClient, end users will no
longer experience frequent "page refreshes"; instead, they shall perceive much faster
response time since UI updates are delivered to them incrementally, rather than one full
page at a time.
Specifically, XUPClient takes advantages of the desktop computing power to enable a

rich and interactive UI experience for end users. That is, the client can fully leverage the
rich UI capability offered by native desktop GUI toolkits such as Windows Forms and
63
It is possible for classic HTML applications to store UI state in the browser, using a combination
of JavaScript and DOM. However, this further complicates the UI logic and hence the overall
design of the application.
Part 1: OpenXUP 118
Java Swing. However, the client remains thin in terms of application logic; i.e., no
application code is executed on the client side. The client renders UI and processes
native events, but leaves application-specific UI logic to the server side.
Finally, XUPClient complements, rather than replaces, the traditional desktop and web
page based models. The desktop model is more appropriate for graphics-intensive
applications, such as video games, and the web page based model is more suitable for
information browsing, such as browsing news web sites.
Part 2: Mixup – a UI Integration Framework
Part 2: Mixup 121
Chapter 6. Background and State of Art in UI Integration
6. Background and State of Art in UI Integration

The problem of facilitating the creation of applications from components has been one
of the biggest areas of investigation in software engineering and data management over
the past twenty years. It has led to a large body of research and development in areas
such as component-based systems, enterprise information integration, enterprise
applications integration, and service composition. While results from these efforts
simplify integration at the data or application level, little work has been done to
facilitate integration at the presentation level. Development of user interfaces is one of
the most time-consuming parts of application development, testing, and maintenance
[Mye92]. Clearly, the reuse of UI components is as important as the reuse of application
logic.
In this chapter we investigate the problem of UI integration; that is, integration of

components by combining their presentation front-ends, rather than their application
logic or data schemas. The granularity of components, as intended in this thesis, is that
of stand-alone modules or applications, and the goal is to build composite applications
that leverage the components' individual UIs to produce applications with rich and
composite UI. The need for such integration is manifest, and examples are numerous:
applications overlaying real estate information over Google Maps, aggregated
dashboards showing consoles monitoring different aspects of a computer's performance,
or "web" operating systems that allow coordinated interactions with multiple
applications on the same web page. All these examples require coordination among
application's user interfaces (e.g., zooming out on a map means that the overlaid
information on houses for sale must change as well).
The objective of this chapter is to identify the basic characteristics of UI integration as a

research discipline and present the main approaches that can be taken to address them.
Specifically, we describe and exemplify the characteristics, challenges and opportunities
of UI integration in comparison with data and application integration. This is important
not only to understand why UI integration differs from other integration problems – and,
hence, requires unique solutions – but also to understand the similarities, as many
lessons can be learned from these other types of integration. Then, we characterize the
Part 2: Mixup 122
main dimensions of the UI integration problem, which give us a framework to analyze

existing approaches to UI integration. Although our discussion focuses on graphical UI
technologies, the identified dimensions and the concepts behind the expressed ideas
have a general applicability to other presentation paradigms (e.g., speech-based
interfaces). Finally, we discuss what is missing in current approaches and how we
believe that the field should or will evolve to facilitate UI integration.
6.1 Integration Layers

To describe the different types of integration we use a simple but concrete scenario,
based on actual applications developed within Hewlett-Packard. Consider a set of
applications that monitor the performance and quality of systems, networks, services,
and business processes. For example, a system monitoring tool logs metrics (such as
CPU utilization) for a set of machines and sends alerts in case they go above certain
thresholds, while a process monitoring application looks at business process executions
and reports on key performance indicators such as process duration or process
instantiation rate. Each of these applications, like most modern applications, is
structured into three layers: presentation (UI), application logic64 (AL), and data access
(DA).
While the monitoring applications have been developed independently, there is an

increasing need to look at them in an integrated fashion. This is useful to perform root
cause analysis (understand what system problem is the cause of a delay at the process
level) as well as business impact analysis (understand what is the "damage", at the
process level, of a failure or performance degradation in a backend system), and in
general to have an end-to-end view of the managed IT systems. As a simple integration
example, assume that our integrated monitoring application is formed by two
components: a business process monitoring tool and a system monitoring tool. A desired
behavior of the integrated application is that, when the user chooses in the process
monitoring tool to visualize a specific process, the health and availability status of its
supporting systems should be displayed in the system monitoring application. We next
64
The AL layer is also often called the business logic layer, so we use the two terms interchangeably
in this thesis.
Part 2: Mixup 123
examine the different types of integration that can be used to build this integrated
application.
In data integration [Len02] approaches, composite applications have their own UI and
AL layers, while the DA layer is an integration of data sources independently
maintained by the component applications (Figure 6.1a). In the example scenario, the
different monitoring applications collect data in their local repositories, unaware of
being the object of integration. The DA layer brings data sources together and exposes a
unified, homogeneous view to the composite application. This integration layer can be
materialized or it can remain virtual.
Application Application Application

Presentation Presentation Integration
Business Logic Integration
Presentation 1 Presentation 2
Integration
Business Logic 1 Business Logic 2 Business Logic 1 Business Logic 2
Data Source 1 Data Source 2 Data Source 1 Data Source 2 Data Source 1 Data Source 2
(a) Integration of different data sources (b) Integration of distributed business (c) Integration of two autonomous
logic elements applications
Figure 6.1 Component integration at different layers
There are many established data integration technologies. ETL (Extract, Transform,
Load) is a common technique used by data warehouses to integrate data. It starts by
extracting data from outside sources, transforming them to fit business requirements
(both syntactic and semantic), and finally loading it into the target data store (e.g., the
data warehouse or simply a relational database). During the extraction process, the
source data (e.g., flat files, database tables) are retrieved and parsed. The transformation
phase applies a series of functions to the extracted data to derive the data to be loaded
into the end target. This includes, for example, selecting only certain columns of a
database table, translating code values, validating and cleansing data, etc. Finally, the
load phase loads the transformed data into the target data store. Since the dataset is
usually quite large, the actual loading is typically performed on a preset schedule (e.g.,
weekly, hourly) rather than in real time.
Another data integration approach is EII (Enterprise Information Integration), which

uses data abstraction techniques to address the data access challenges associated with
Part 2: Mixup 124
data heterogeneity and data contextualization65. EII offers a uniform access method and
data representation for heterogeneous data sources, such as SQL tables, flat files, XML,
and other URI-based resources. The virtualized data is then accessible via a single
standard interface, such as JDBC66 or ADO.NET67. For example, an EII tool may offer a
single JDBC interface for XML documents returned from disparate web services as well
as relational database tables used by enterprise applications from different software
vendors. As a result, EII greatly simplifies enterprise data access and management by
offering developers a single, consistent data representation and access method.
Data integration presents a number of issues, ranging from the resolution of mismatches
between component data models (e.g., same terms may have different meanings) to the
construction and maintenance of virtual schemas and query mappings between global
and local schemas. Data integration is often used because it requires little "cooperation"
from component applications. One can always tap into the applications' databases by
means of SQL queries, or via EII techniques. The drawback is that this requires a
significant effort to understand the data models, to analyze semantic heterogeneities, and
to maintain the composite schema in the wake of changes to the component data
schemas [Hal05].
Application integration has been thoroughly studied over the last twenty years, giving
rise to technologies such as remote procedure calls (RPCs), object brokers (such as
CORBA [OMG08]), and Web Services [Alo04]. In application integration, a composite
application has its own UI, but its business logic layer is developed by integrating
functions exposed by the component applications (Figure 6.1b).
In the example scenario, the monitoring applications could expose APIs that allow
clients to retrieve performance data for a certain system, or to subscribe to alerts for
performance degradations. The composite application would use these APIs to retrieve
information, correlate it across the different monitoring applications, and display a
consolidated monitoring on its GUI.
65
http://en.wikipedia.org/wiki/Enterprise_Information_Integration
66
http://java.sun.com/products/jdbc/overview.html
67
http://msdn.microsoft.com/en-us/library/h43ks021(VS.80).aspx
Part 2: Mixup 125
When such APIs are available, this integration model has many benefits: (1) the
granularity of functions provided by the component applications is generally well-suited
for high level integration (for example, we can tell a monitoring application to begin
monitoring machine XYZ, without considering the detail of how this affects the data in
the integrated application's database), and (2) it is more stable, as the component
application is aware of the integration (it is exposing the API), and hence will drive
towards making the interface more stable across versions.
Currently, the dominant application integration paradigm is SOA (Service-oriented

Architecture68), where distributed application components are packaged as services that
can be combined to perform new functionalities. In SOA, application components are
exposed as services (e.g., via SOAP). Each service comprises of a set of operations
which can be invoked to perform desired tasks. Services are described by WSDL
[Chr01][Chi07a] documents, which specify the APIs (i.e., operations) that expose the
services' functionalities. Through web services, SOA allows applications logic from
multiple services to be combined and integrated to form new applications to achieve
desire functionalities. Although SOA is designed to work across organization
boundaries, most deployments are within enterprises. Therefore, SOA is often deployed
for enterprise application integrations69 (EAI).
An Enterprise Service Bus70 (ESB) is typically part of an SOA deployment that acts as a
message broker to facilitate the communications among applications or services. The
bus replaces direct point-to-point communications among applications and therefore
allows them to be independently developed. As legacy applications are typically
unaware of ESB and SOA, the bus needs to transform the messages into a format that is
understandable by the applications, via so called adapters. An ESB may also provide
additional services such as routing, QoS, monitoring and auditing.
BPEL (Business Process Execution Language) [And03] is an XML-based service

composition language that is often used in an SOA deployment to build business
68
http://en.wikipedia.org/wiki/Service-oriented_architecture
69
http://en.wikipedia.org/wiki/Enterprise_application_integration
70
http://en.wikipedia.org/wiki/Enterprise_service_bus
Part 2: Mixup 126
applications by composing services from various sources. The language offers

workflow-based constructs that can be used to specify business processes; each consists
of one or more services. The services are composed in an orchestrated fashion; that is,
there is a central process engine that coordinates the executions of the business
processes by invoking the specified operations from remote services. Although BPEL
engines usually come with graphical design tools, the composition process remains
complex, and is usually performed by developers, not by business users. Therefore, end
user composition remains to be a challenging task.
UI integration (cf. Figure 6.1c) aims at composing applications by addressing the

presentation layer only, leaving the responsibility of data and business logic
management to each component. Such integration would facilitate the construction of
complex and rich applications. UI integration is particularly applicable in cases where
application or data integration is not feasible (e.g., applications do not expose business
level APIs), or where the development of a new UI from scratch is too costly (e.g., the
component application often changes or its UI is complex).
Since UI integration is the focus of this thesis, we will spend the rest of this chapter
discussing related works in this area. We will start with deriving a set of characterizing
dimensions in order to illustrate the features and differences and compare the strengths
and weaknesses of different UI integration approaches.
6.2 Dimensions for Characterizing UI Integration

Technologies
To discuss related technologies in UI integration, we need a set of dimensions to
compare them. Much like in data and application integration, we characterize the UI
integration approaches by looking at the objects of integration (i.e., the components) and
at how such objects are glued together (i.e., the composition logic) and the
communications among them.
In addition, we believe it is important to separately look at the runtime support (the

runtime environment) and the design-time support (the development environment)
Part 2: Mixup 127
provided by the different UI integration technologies. We next examine these

dimensions in detail.
6.2.1 Component Model

This dimension deals with the characteristics of the UI component as the unit of
integration in a composite application. Ideally, models and specifications of UI
components have to be simple enough to be understood and easily adopted by users,
formal enough to be parsed by applications or tools and to provide an added value (e.g.,
search for components, analysis of compatibility between components, etc), and
expressive enough to model a wide range of functionalities.
In application integration, components are essentially characterized by an API and a

component model (e.g., the CORBA component model). In data integration,
components are described by data source schemas (e.g., relational or XML schemas). UI
integration has hitherto mostly been intended as reusing class libraries to facilitate
development (e.g., Java Swing class libraries). However, for high level UI components,
performing integration at the presentation layer also requires a component model able to
support complex interactions and coordination. The challenge here is to describe, for
each single component, both the software interface of the component to be used for
integration purposes and the user interface of the component to be used for interacting
with the end users.
The component model determines the nature of components and influences how
components can be glued together, i.e., how they can be composed. A well-defined
component interface, for instance, facilitates reusability, while, at the same time, a
flexible component interface ensures extensibility. Hence we characterize a component
model by the type of interface it exposes and its extensibility.
Some components expose a GUI interface only. This is analogous to a traditional

monolithic desktop application. All interactions with the component are performed via
its GUI. The only way to integrate a component application is to have an intimate
knowledge of its UI, to be able to track the mouse position and/or to intercept the text
entered by the user, and in this way understand what is shown by the component's UI
Part 2: Mixup 128
and possibly even execute actions that cause UI modifications (e.g., by having the
composite application simulate mouse clicks or keystrokes). Most web pages can be also
treated as GUI-only components. To integrate them, one must use page-scrapping
techniques to extract useful information or just embed them inside <div> or <iframe> in
a side-by-side fashion in the final page. Integration in this case is a daunting task.
More advanced component provides a description of its UI and an API to manipulate the
UI at runtime. The API can be described at different levels of abstraction. A low level
API may allow control of individual UI elements such as buttons or text fields. A high
level API would instead expose a set of entities, i.e., observable and controllable
objects, such as "system" or "network" in our example scenario, as well as operations to
change the status of entities, such as "show status of system xyz".
Finally, the extensibility of a component model determines whether an existing

component can be extended to accommodate specific application requirements (e.g., by
adding new operations), and whether new components can be easily created.
6.2.2 Composition Model

The composition model determines how components are integrated to form the
composite application, assuming components are readily available. The challenge here is
how to specify the orchestration of events from the UI components in order to define the
UI of the composite application.
In data integration, composition is often done via SQL views that allow a global schema
to be expressed as a set of views over local schemas [Len02]. In application integration,
composition is done either via general-purpose programming languages such as Java, or
via dedicated application integration languages, such as workflow or service
composition languages (e.g., BPEL). Both data and application integration thus
recognize the need for composition or integration languages. Therefore, we believe that
similar approaches can be adopted in UI integration.
UI composition could be performed by means of generation purpose languages like

Java or scripting languages like JavaScript. Such languages are very flexible but lack
abstractions for the composition of coarse-grained components (e.g., facilities for
Part 2: Mixup 129
component binding, or high-level primitives for synchronizing what is displayed by UI

components). Therefore, specialized composition languages that are high-level and
XML-based (preferably declarative), can be tailored to the composition of UI
components at the level of abstract/external descriptions. This idea is similar to service
composition languages. The main benefit of such languages is that of a higher level
programming of the composition, which leverages the characteristics of the component
model.
The composition model may offer different orchestration styles. Orchestrating

components implies specifying how the execution order of components is defined and
how they are synchronized. There are two main approaches: flow-based styles define
orchestration as sequencing or partial order among tasks/components and are expressed
by means of flow-chart like formalisms, whereas event-based approaches are based on
publish/subscribe models, particularly powerful for maintaining a synchronized
behavior among UI components.
Finally, the composition model may or may not support exception and transaction
handling. If supported, exception handling can follow the throw and catch approach
(Java style) or can be rule-based, by means of ECA71 (event-condition-action) rules
coupled to the composition. Transactions, if supported, always follow some variation of
the SAGA model [GMS87].
6.2.3 Runtime Infrastructure

A proper runtime infrastructure enables the efficient execution of composite
applications, and determines how they are delivered to the end users.
First, the runtime environment must facilitate effective inter-component

communications. This includes the interaction models among components and the
composite application. In the example scenario, the issue is how the monitoring
components exchange UI events to notify user actions significant to the composite
71
http://en.wikipedia.org/wiki/Event_Condition_Action
Part 2: Mixup 130
application (e.g., a user selects a different system and therefore all UI components need
to show updated information) or to receive instructions on what to display.
In data integration, components typically are passive participants that do not initiate
communications with the integrating application. This perspective is also common in
application integration, where we have a centralized entity (the composite application)
that invokes components as needed, although fully distributed interactions are becoming
more common (e.g., where a seller, a buyer, and a shipper interact without a central
coordinator).
In UI integration we can also distinguish between centrally-mediated communication,

where the composite application has a central coordinator that receives events from
components and issues instructions (e.g., via API calls) to manipulate the components'
UIs, and direct component to component communication, where the composite
application is a cooperation of the individual components, and there is no first-class
application to orchestrate their activities.
An additional distinction, orthogonal to the above one, is between RPC-style interaction,

where information is exchanged via method calls and returned data, and publish-
subscribe interaction [Eug03], where components communicate in a loosely-coupled
way via messages exchanged through message brokers. In the latter case, messages are
distributed based on their content (or topic, as it is often called in the literature).
In addition, the location of the runtime environment determines where the

composition is executed (i.e., where the UI components are composed together to form
the composite application). In general, UI components can be assembled at the server
side, the client side (e.g., inside web browser via JavaScript), or both. If the integration
is performed at the server side, the browser is merely displaying the resulting composite
application.
Finally, the scalability of the runtime environment is always an important consideration.

In general, client-side approaches do not suffer from any scalability problems, since the
composition is executed at the client side and the user can only interact with a limited
number of UI components at a time. For server-side approaches, the scalability problems
Part 2: Mixup 131
are related to the number of instances and, hence, to the number of users and to the
complexity of the compositions. In all cases, scalability techniques are the same as for
traditional forms of integrations or traditional web applications.
6.2.4 Development Environment

The characteristics of the development environment affect the efficiency of composition
development and determine the success of the UI integration approaches. Development
tools vary greatly in the level of support they provide to their users. Some tools are
strictly for developers; others are more oriented toward tech-savvy end users. We
characterize the development environments by means of the following properties:
Interface paradigm and target users: Design support may be provided via different
interface/modeling paradigms, such as visual drag-and-drop features, textual editors, or
a combination of the two. The interface can be targeted at the average web user,
advanced (tech-savvy) business users, or programmers. The ease of use of the interface
is the key to facilitate end user composition.
System requirements: A development tool may be a standalone application that needs

to be installed prior to use, or it may be browser-based with or without the need of
additional plug-in modules.
6.3 UI Integration / Component Technologies

In this section we review some of the mainstream approaches to UI integration. We
attempt to group them into several categories and compare their strengths and
weaknesses in light of the dimensions presented above.
6.3.1 Desktop UI Components

Historically, UI composition has first been considered for desktop applications. The
introduction of component technologies eventually provided an environment where
applications developed using heterogeneous languages could interoperate. A typical
example is given by ActiveX, which leverages Microsoft's COM technology for
embedding complete application UI into host applications. Other examples are Apple
Part 2: Mixup 132
OpenDoc [App95] or GNOME Bonobo72. These technologies rely heavily on the

underlying operating system or component middleware for component interoperability.
By contrast, the Composite UI Application Block (CAB [MS05]) is a framework for UI

composition in .NET. Its container service allows applications to be built upon loadable
modules or plug-ins. Developers concentrate on reusable components that can be
dynamically plugged into the container at runtime. CAB separates UI components from
the business logic. CAB components can be used with any .NET language to build
composite containers and to perform component-container communications. CAB
further provides an event broker for many-to-many, loosely coupled inter-component
communication based on a publish/subscribe runtime event model.
Eclipse's Rich Client Platform73 (RCP) provides a similar framework. RCP includes an
application shell (i.e., a container) with UI facilities such as menus and toolbars. RCP
offers a module-based API, enabling developers to build applications on top of this
shell. In addition, Eclipse allows UI components (i.e., Eclipse plug-ins) to be
customized and extended via so-called "extension points", a combination of Java
interfaces and XML markups defining component interfaces and facilitating their loose
coupling.
Desktop UI components typically use general-purpose programming languages to

integrate components (i.e., C# for CAB and Java for RCP), since the component
interfaces are language-specific programming APIs. Flexible communication styles are
supported; i.e., centrally-mediated and direct component to component. Exceptions are
typically supported by the language or runtime, but transactions are usually handled
manually. Runtime scalability is generally not an issue because the number of
components running on the desktop is limited. Since desktop UI is very mature, there
are a variety of development tools, mostly aimed at programmers.
Finally, many of the technologies in this category are OS-dependent. Although CAB and
RCP do not depend on the OS directly, they rely on their respective runtime
72
http://developer.gnome.org/arch/component/bonobo.html
73
http://wiki.eclipse.org/index.php/Rich_Client_Platform
Part 2: Mixup 133
environments (i.e., .NET for CAB and JVM for RCP). The lack of technology-agnostic,
declarative interfaces makes interoperation among components implemented using
different technologies difficult to achieve.
6.3.2 Browser Plug-in Components

Rich UI features in markup-based interfaces are often achieved, besides through
JavaScript and dynamic HTML, by means of embedded UI components such as Java
applets, ActiveX controls, and Flash.
The external interface of such components is very simple and usually only requires to
set proper configuration parameters when embedding the components into the markup
code, which represents the composition language. Plug-in components provide for their
own rendering, and usually there is little further communication between components
and the containing web page, or among components. Component bindings are specified
at page-authoring time; during runtime, the browser downloads and instantiates the
components.
Embedded UI components are easy to use, but the lack of a systematic communication
framework is a limitation. Communication with components can be achieved through
ad-hoc JavaScript, but this is far from a uniform approach to deal with components.
This limitation derives more from the browser’s sandbox mechanism for the execution
of plug-ins than from the component model itself. Similar to desktop UI components,
exceptions are typically supported by the language or runtime, but transactions are
usually handled manually.
Since all components are executed by the client-side browser, scalability is only subject
to the browser's performance on the desktop computer. On the development side, there
are a variety of development tools, many of which are shared with desktop UI
components.
6.3.3 Web Portals and Portlets

This approach explicitly distinguishes between UI components (the portlets) and
composite applications (the portals) and is probably the most advanced approach to UI
Part 2: Mixup 134
composition as of today. We use the term "portlets" from Java Portlets [Abd03][Hep08],
but our considerations also hold for ASP.NET Web Parts74 [Wal05].
Portlets are full-fledged, pluggable web application components. Portlets generate

document markup fragments (e.g., (X)HTML, or WML) that adhere to certain rules
(e.g., common CSS styles) to facilitate content aggregation in portal servers to form
composite pages. Portal servers typically allow users to customize composite pages
(e.g., to rearrange or show/hide portlets) and provide single sign-on and role-based
personalization. Typical portlets include weather reports, news tickers, and stock quotes.
Analogous to Java servlets75, portlets implement a specific Java interface (i.e., JSR-168
[Abd03], the portlet API), intended to enable developers to create portlets that can be
plugged into any conforming portal server. JSR-168 defines a runtime environment for
portlets (i.e., a portlet container) and the Java API between container and portlet. Portal
applications, in the case of Java portlets, are based on the Java programming language,
while in the case of Web Parts, applications are programmed in .NET.
With portal and portlets, integration is performed at the server side (within the portal
server). The portal server aggregates the markup outputs of its portlets and manages the
communications in a centrally mediated fashion. All interactions need to be processed
by the server, since the portlets cannot interact with one another on the client side.
Exceptions are typically supported by the language or runtime, while transaction support
depends on the portal server's implementation. Presentation-wise, individual portlets are
usually displayed in their own sub-windows within the portal page, without any
overlapping. Therefore, it is difficult to mix portlet content (e.g., embedding a weather
report inside a news article, in order to show the weather at the news location).
JSR-168 does not provide inter-portlet communication mechanisms, but work on this is
underway (JSR-286 [Hep08]). ASP.NET Web Parts support inter-part communication
by means of shared data structures. However, shared data structures make Web Parts
tightly-coupled. A publish/subscribe based event mechanism might be more desirable.
74
http://msdn.microsoft.com/en-us/library/e0s9t4ck(vs.80).aspx
75
http://java.sun.com/products/servlet/
Part 2: Mixup 135
Although Java Portlets and Web Parts have similar goals and architectures, they are not
interoperable. Web Services for Remote Portlets (WSRP) [Kro03] addresses this issue
at the protocol level: WSRP exposes remote portlets as web services, and
communications between the portal server (WSRP consumer) and portlets (WSRP
producer) occur via SOAP. Portal and portlets can thus be built with different languages
and runtimes.
Specifically, WSRP portlet interface has two primary operations: the "getMarkup"
operation allows the portlet to render itself by producing markups (e.g., in HTML); and
the "performBlockingInteraction" operation allows the portlet to process user's input /
interactions. A major limitation of WSRP 1.0 is the lack of support for inter-portlet
communications. As a result, WSRP 2.0 [Tho08] added an event distribution
mechanism via the "handleEvents" operation, which allows the portlet to handle events
from other portlets or the portal application.
Finally, since portal and portlets is a server-side approach, runtime scalability is a major
concern. The most important factor is the number of concurrent users that can be
supported, with a suitable fail-over strategy. On the development side, many tools exist
to assist developers to create portlets. Some of them have a drag-and-drop user interface,
which allows end users to customize their portal page view by selecting the portlets
(from a registry) to be shown on the page and arranging their layout.
6.3.4 Web Mashups

Web mashups [Mer06][Yu08] are web applications generated by combining content,
presentation, or application functionality from disparate web sources. The aim is to
combine them in a value-adding manner in order to create useful new applications or
services. Content and presentation elements typically come in the form of RSS76 or
Atom77 feeds, various XML formats, or as HTML, SWF, or other graphical elements;
application functionality, instead, is typically provided via publicly accessible APIs
(e.g., in JavaScript). Content, functionality, and presentation are then glued together in
76
http://en.wikipedia.org/wiki/RSS
77
http://en.wikipedia.org/wiki/ATOM
Part 2: Mixup 136
disparate ways: via JavaScript in the browser, via server-side scripting languages like
PHP or Ruby, or via traditional languages like Java or C#.
The early mashups could not rely on APIs, as the actual content providers did not even
know that their web sites were wrapped into other applications. The first mashups with
Google Maps, for example, predate the official release of Google Maps API78. The API
is Google's answer to the growing number of hacked map integrations, where developers
had to go through the whole AJAX code of the map application in order to derive the
needed functionalities.
Publicly available APIs for mashups are still rare but growing. Integration is performed
in an ad-hoc fashion by leveraging whatever programming language supported by the
content source, on either client side (e.g., AJAX) or server side (e.g., PHP, Java,
ASP.NET). Contents are typically provided as markup code and integrated in a centrally
mediated way. The lack of infrastructure makes inter-component communication
difficult.
As a result, recently a number of mashup tools have emerged to improve the mashup
development process. In this section, we provide an overview of some of the popular
mashups tools and show how they can be used to facilitate mashup development.
As an example of a successful mashup, consider the HousingMaps79 application shown

in Figure 6.2. It combines property listings from Craigslist with map data from Google
Maps in order to assist people moving from one city to another and searching for
apartment rentals. Typically, when browsing through a list of properties, the address of
the properties is not expressive enough to people who are not yet familiar with the new
city. HousingMaps hence aims to provide users with a list of properties and to plot the
respective location and property information on the map upon user's selection (by means
of the popup cloud visible in Figure 6.2).
78
http://www.google.com/apis/maps/
79
http://www.housingmaps.com/
Part 2: Mixup 137
Figure 6.2 HousingMaps
Using this example mashup, next we discuss some of the popular mashup tools by
showing their strengths and weaknesses along the dimensions we set out earlier in the
chapter. For presentation purposes, we have selected what we think are the most popular
or representative approaches of mashup tools, namely Yahoo Pipes80, Google Mashup
Editor81, Microsoft Popfly82, Intel Mash Maker83, and IBM's QEDWiki84. Let's intuitively
introduce the ideas underlying the selected tools and show how in practice they can
support the development of the HousingMaps application.
80
http://pipes.yahoo.com
81
http://code.google.com/gme/
82
http://www.popfly.ms
83
http://mashmaker.intel.com
84
http://services.alphaworks.ibm.com/qedwiki/
Part 2: Mixup 138
Yahoo Pipes allows the mixing of popular data feeds to create data mashups via a
visual editor. A pipe is a data processing pipeline (hence the name "pipe"), which
consists of one or more data sources (e.g., RSS/Atom feeds or XML sources) and a set
of interconnecting operators, each of which performing a specific task. There are
operators for manipulating data feeds (e.g., sorting or filtering) and operators for
features like looping, regular expressions, or counting. More advanced features such as
location extraction (i.e., geo coordinates identified and converted from location
information found in text fragments) or term extraction (i.e., keywords) are also
supported. The goal of Yahoo Pipes is to enable users to design data processing
pipelines that filter, transform, enrich, and combine data feeds.
Let's consider the HousingMaps example and try to understand how Pipes could aid its
development: Since Pipes does not provide user interfaces, i.e., its output is in the form
of an RSS feed, it is not possible to implement the user interface shown in Figure 6.2.
Instead, we could use Pipes to process the Craigslist feed and identify location
information (geo codes) by leveraging Pipes' location extractor. The identified location
information could be used to augment the Craigslist feed with a link that allows the user
to display the property's address on the map by passing the geo codes to Google Maps.
Yahoo Pipes supports data and application logic components through operators that
provide access to RSS/Atom feeds and external web services: DA components have a
read-only interface, and external web services have a RESTful interface based on JSON
or RSS. The component models are fixed, so new components cannot be easily created.
In addition, UI components are not supported, so it cannot be used to integrate
components at the presentation layer. Pipes offers a flow-based composition model,
where data among components are passed in the execution flow. Its composition model
does not provide exception handling or transaction support.
Pipes are hosted by Yahoo, so compositions are computed and assembled at the server
side. Hence, executing a pipe does not pose any particular system requirements on the
client; however, the server-side engine that executes the pipes might suffer from
scalability issues if many pipes are executed at the same time (i.e., accessed by a large
number of users) or comprised of hundreds of sources. On the development side, Yahoo
Part 2: Mixup 139
Pipes provides a pure visual drag-and-drop AJAX editor targeted at advanced users with
basic programming skills. The editor can be executed in any standard web browsers with
support for the XMLHttpRequest JavaScript object.
Google Mashup Editor (GME) provides a template-based environment for mashup

development. It offers a set of standard modules that allow the encapsulation and layout
of external data. As an example, the List module represents an RSS/Atom feed as a list,
whereas the Item module represents a single item in a feed. Modules may fire pre-
defined events, which can be captured by other modules to act accordingly. Creating
mashups involves developing UI templates that contain a mixture of XML control tags
and HTML/CSS layout elements with embedded JavaScript code. During runtime, the
UI templates are filled by GME and presented as web pages.
For the HousingMaps application, the Craigslist feed could be integrated by means of a
List module, and the Item module could be used to show the details of a particular
property. Google Maps is natively supported by GME's map module. When the user
clicks on a property in the Craigslist module, the module emits a "select" event, which
can be captured by the map module to popup the cloud window on top of the marker and
display the information about the selected property. The Craigslist module and Google
Maps need to be embedded into the UI template that specifies the layout of the actual
mashup application.
GME supports components at all three integration layers. DA components are typically
interfaced via markup, AL components via JavaScript, and UI components via both
markup and JavaScript. The component model is quite flexible. The composition model
is event-based, and communications among components are primarily in the form of
event parameters. GME does not support exceptions or transactions in its composition
model.
GME mashups are hosted by Google, so mashups are executed at the server side. Hence
it is subject to the same scalability issues as Yahoo Pipes. On the development side,
GME supports a browser-based textual AJAX editor with syntax highlighting and
automatic tag completion, which is targeted at experienced programmers.
Part 2: Mixup 140
Although GME supports all three types of components, integration is still performed at
the data layer, and the UI is merely added on top of the already integrated mashup
application. Specifically, through its templating mechanism, GME supports UI
programming, but not UI integration.
Microsoft Popfly offers a component-based, visual environment for developing

mashups. In Popfly, reusable components are called "blocks". A block can act as a
middleman between externally provisioned services, such as Web Services, or it may
implement a useful function (in JavaScript), e.g. a function that calculates the area of a
circle given a radius. Blocks have operations with inputs and outputs, which are
specified in a dedicated XML descriptor. A block may also act as a display surface, i.e.,
a piece of UI that takes data from other blocks and displays them, allowing the user to
interact with it and enabling the layout of the mashup application.
To build the HousingMaps application, we basically need three blocks: an RSS feed
block for the Craigslist feed, a map block, and a table block. If instead of Google Maps
we use Virtual Earth85, the three blocks are readily available. Hence we need to drag the
blocks onto the mashup design surface and then connect the output of the getItems
operation of the RSS block to the two display blocks (i.e., table and Virtual Earth). The
placement of the initial set of markers may require the extension of the RSS block with
a suitable JavaScript operation.
Popfly supports AL and UI components. Components have a JavaScript interface, with

an extensible component model. Popfly uses a semi flow-based orchestration style, and
its composition model does not support exceptions or transactions.
Popfly mashups are hosted by Microsoft, but the execution is however performed at the
client side. That is, the mashup (and the blocks it uses) are hosted by Microsoft, but they
are downloaded and executed in the browser at runtime. The client-side execution
facilitates scalability, as the integration of multiple sources is mostly done at the client
side. Popfly offers a graphical and textual editor with drag-and-drop support, which can
be used by developers as well as tech-savvy web users. The development environment,
85
http://www.microsoft.com/VIRTUALEARTH/
Part 2: Mixup 141
though browser-based, requires Microsoft's Silverlight technology, a mandatory browser

plug-in.
Intel Mash Maker provides an environment for the integration of data from annotated
source web pages based on a dedicated browser plug-in. Rather than taking input from
structured data sources such as RSS or Atom, Mash Maker allows developers and users
to annotate the structure of web pages while browsing and to use such annotations to
scrap contents from annotated pages. Advanced users may leverage the integrated
Structure Editor to input XPath [Cla99a] expressions with the help from FireBug's
DOM Inspector (a plug-in for the Firefox web browser). Composing mashups with
Mash Maker occurs via a copy/paste paradigm, based on two modes of merging
contents: whole page merging, where the content of one page is inserted as a header into
another page; and item-wise merging, where contents from two pages are combined at
row level, based on additional user annotations. Both techniques can be used to merge
more than two pages.
For the HousingMaps example, the developer first needs to annotate the structure of the
appropriate Craigslist page, since Mash Maker operates on regular HTML content rather
than on RSS. Next, the developer merges the Craigslist page with the Google Maps page
using the copy/paste mechanism. Specifically, the developer adopts the item-wise
merging, since each item from the Craigslist page will be plotted as an individual
marker on the map.
Mash Maker supports DA components extracted from annotated web pages (e.g., table
and map). Their interface can be interpreted as XML / HTML markup, and the
component model is fixed. With Mash Maker, contents are glued together either in a
whole page merging style or in an item-wise merging style (flow-like). Exceptions and
transactions are not supported.
Although hosted by Intel, mashups are downloaded and executed inside web browsers at
runtime. Due to the client-side execution, Mash Maker should be able to scale
adequately. On the development side, Mash Maker supports a point-and-click UI that
allows developers and advanced users to annotate pages and then extract and merge data
Part 2: Mixup 142
via copy-and-paste. However, it requires the installation of a dedicated browser plug-in

for both runtime execution and mashup development.
QEDWiki (Quick and Easily Done Wiki) is IBM's proposal of a wiki-based "mashup
maker", fully running inside the client browser and allowing access to IBM's Mashup
Hub86, which provides support for the creation of data feeds and UI "widgets" and also
incorporates DAMIA [Alt07] for data assembly and manipulation. As a wiki
environment, it allows both the editing and immediate viewing as well as the easy
sharing of mashups. Mashups are assembled from PHP-based widgets, whose wiring
determines the behavior of the mashup. Widgets represent application components and
may or may not have their own UI. The assembly of a mashup requires the selection of a
page layout (an HTML template), followed by the dragging and dropping of widgets
onto the page grid, and finally the configuration of their interactions (e.g., the
specification of the data passing logic).
The development of the HousingMaps application with QEDWiki could be done as

follows. We start with creating a new wiki page and by selecting a grid layout for the
page; in our case, we opt for a layout that allows us to place Google Maps and the
properties list side by side (e.g., two columns). Then we search for the GoogleMap and
ShowData widgets in the widget palette, drag them over the grid layout, and drop
GoogleMap into the left column and ShowData into the right column. Then we use the
LoadFeed widget to access the Craigslist RSS feed and to populate the ShowData
widget with the property information (by telling the ShowData widget that it should
source data from the LoadFeed widget). At runtime, to place properties on the map, one
can simply drag addresses from the ShowData widget and drop them onto the
GoogleMap widget.
QEDWiki supports components at all three layers. Its component model is equipped
with an extendable PHP interface, with events and properties. QEDWiki uses a page-
based composition style, where components are assembled together on a wiki canvas.
Exceptions and transactions are not supported by the composition model.
86
http://services.alphaworks.ibm.com/mashuphub/
Part 2: Mixup 143
QEDWiki pages are hosted by IBM, and mashups are executed mostly on the server side
(in PHP). Some of the widgets have AJAX support, so a subset of user interactions can
be handled within the client-side browser. Due to server-side execution, the wiki engine
might encounter scalability issues when there are a large number of users and / or the
mashups are trying to integrate a large amount of widgets or data sources. On the
development side, QEDWiki comes with an easy-to-use drag-and-drop interface for
developers and advanced users, where components can be immediately visualized. It
runs in standard web browsers and does not require any plug-ins.
6.3.5 Comparisons
In this section we summarize and compare different web-based UI integration
approaches. The following table provides a comparison of the aforementioned
approaches along the characteristic dimensions.
Table 6.1 Comparison of UI integration approaches
Component Composition Runtime Development

model model infrastructure environment
Desktop UI Extendable General-purpose Flexible For programmers;
components API languages; manual communication; desktop-based
exception handling desktop runtime;
scalable
Browser Extendable General-purpose Communication For programmers;
plug-ins API languages; manual through JavaScript; desktop-based
exception handling client-side runtime;
scalable
Portlets Fixed portlet General-purpose Limited For programmers;
API languages; manual communication; desktop-based
exception handling server-side runtime;
web server scalability
Mashup – REST / RSS; Flow-based; no Communication For programmers and
Pipes fixed model exception handling through data flows; tech-savvy users;
server-side runtime; AJAX-based; drag-n-
web server scalability drop UI
Mashup – Flexible Event-based; no Event-based For programmers;
GME JavaScript exception handling communication; browser-based; textual
API server-side runtime; editor
web server scalability
Part 2: Mixup 144
Mashup – Extendable Semi flow-based; Communication For programmers and

Popfly JavaScript no exception through JavaScript; tech-savvy users;
API handling client-side runtime; browser / Silverlight
scalable based; drag-n-drop UI
Mashup – GUI Page merging Limited For programmers and
Mash interface model; no communication; tech-savvy users;
Maker only (HTML exception handling client-side runtime; browser plug-in based;
markups) scalable point-n-click UI
Mashup – Extendable Page-based model; Communication For programmers and
QEDWiki PHP API no exception through PHP; server- tech-savvy users;
handling side runtime; web AJAX-based; drag-n-
server scalability drop UI
Desktop UI components are very mature, with sophisticated support for events and
inter-component communications. However, they are operating system or platform
dependant. There is no systematic support to combine components from different
component technologies into the same application.
Browser plug-ins allow quick client-side composition by embedding plug-ins in the web
page. However, inter-component communications are very limited, especially if the
plug-ins are from different component technologies (e.g., an ActiveX control and a Java
applet). Composition developers may use JavaScript to call into plug-ins (if supported
by the plug-ins); however, there is no systematic support for call-outs from plug-ins
(i.e., firing events to inform the web page and other plug-ins of its state changes).
Portal and portlets are the de-facto server-side UI component technology. However,
inter-portlet communications remain challenging. In addition, portlets are only
concerned with handling web requests and generating markup responses. The notion of
semantically rich, application-specific UI events is missing. Furthermore, portal servers
require portlets to adhere to a strict programming interface. This makes portal / portlets
a UI component and development technology, rather than a UI integration framework
that can accommodate a variety of components built with heterogeneous component
technologies.
With web mashups, most effort in the development process goes into manual testing
since component interfaces may not be stable. In addition, due to the lack of framework
Part 2: Mixup 145
support, code isolation is not guaranteed (i.e., code collision from two pieces of
JavaScript code), and conflicts among components may occur.
To improve the development process, a slew of mashup tools have emerged. However,
just like early days in traditional integration, most of these tools are focusing on the data
layer, leaving UI as an afterthought. In addition, the mashup tools today focus on the
development aspect only. That is, each mashup tool imposes its own programming
interface and requires mashups to be developed with components created in that
particular tool. For example, Popfly mashups must be built from Popfly blocks, and
cannot easily mix with QEDWiki widgets. Note that it is certainly possible for a Popfly
mashup to use a block that takes a Yahoo pipe's RSS output as its input, but this is
merely integrating / reusing a data source, not a component built with a different
mashup tool.
Overall, we believe most existing approaches are either UI development frameworks or

UI component technologies. They facilitate the development of composite web-based UI
by assembling components built with their own dedicated programming interface. Web
mashup can be regarded as a form of ad-hoc lightweight integration; however, current
tools are still focusing on the development aspect, each imposing its own programming
interface. What's needed is a UI integration framework similar to what has been done in
traditional integrations such as EAI and Web Services, where components and services
from heterogeneous component technologies and platforms can be integrated together to
form value-added composite applications.
6.4 Summary
In this chapter we have introduced the concept of UI integration in relation to traditional
integration techniques at the application and data layers, and presented the state of art in
current research and industrial efforts in the field. In particular, we provided an
overview of the representative approaches by illustrating their strengths and weaknesses
along a set of characteristic dimensions.
Although tremendous efforts and results have been made and obtained in application
and data integration, little has been done in UI integration. Most of existing UI
Part 2: Mixup 146
frameworks focuses on the development aspect (i.e., creating new components using
their own dedicated programming interfaces), rather than the integration of existing,
heterogeneous components. As a result, UI integration technologies are far from mature
and face open research challenges. In the next few chapters, we will discuss and present
our solution to UI integration.
Part 2: Mixup 147
Chapter 7. UI Component Model
7. UI Component Model
Creating composite applications from reusable components or modules is an important
technique in software engineering and data management. A large body of research and
development exists in integration-related areas such as enterprise application integration
(EAI), enterprise information integration (EII), and service composition. However, most
of these efforts focus on simplifying integration at the data or application level, while
little work has been done to facilitate integration at the presentation level. It is well-
recognized that the development of user interfaces is one of the most time-consuming
parts of application development [Mye92], so this indicates that reuse is also critical at
the presentation level. However, UI development today is mostly facilitated by toolkits
(e.g. Java Swing and .NET Windows Forms) providing pre-packaged classes modeling
fine-grained UI controls such as buttons and menus; the integration of high-level UI
components encapsulating reusable application functionalities has received little
attention.
The need for integrating coarse-grained components at the presentation level is manifest
and examples are numerous, both in the enterprise and the consumer space. Indeed,
hundreds of examples of UI integration exist today, in the form of web mashups87.
However, since there is very little support in terms of model and tools for UI integration,
the presentation aspect of most mashups today is developed manually. That is, a
developer needs to glue the UI of the desired components together using scripts or
general purpose programming languages, in an ad-hoc fashion. Most of the developer's
time is spent in trying to figure out the programming interfaces of the components, and
then using the appropriate runtime and languages to integrate them.
This situation is similar to that witnessed at the dawn of data and application integration,
where the need for integration was present but methodologies and tools were not. People
resorted to hacking components and information together by writing all the integration
logic from scratch, using conventional programming languages such as C or SQL.
Eventually, the importance of reuse and of structured approaches to integration
87
See programmableweb.com for a list of popular mashups.
Part 2: Mixup 148
supported by tools was recognized, and entire multi-billion dollar industries came to life
in the space of EII and EAI. We argue that a similar path needs to be followed by UI
integration.
In this thesis we introduce a framework for integration at the presentation level; that is,
integration of components by combining their presentation front-ends, rather than their
application logic or data. The framework is built upon the notion of UI component, a
loosely-coupled, coarse-grained module or application which includes a presentation
layer (i.e., UI and logic to manage user interactions), that offers both a graphical
interface and a programmatic interface to facilitate its integration with other UI
components into an overall coherent user interface. The granularity of components is
that of standalone modules or applications encapsulating reusable functionalities; the
goal is to build composite applications that leverage the components' individual user
interfaces to produce composite applications possibly with rich and highly interactive
UI. These UI components should also be easily reusable in various composite
applications and, conversely, a composite application would ideally be able to swap
between components providing similar UI functionality (e.g., different map providers or
different image feed providers).
The remainder of this chapter is organized as follows. We start with a reference scenario
to be used to illustrate the concepts and designs introduced in this chapter. We then
discuss the lessons learned from traditional integration techniques, such as EAI and
service composition. After that, we introduce the notion of abstract UI component
model with bindings to concrete component implementations, leading to the concept of
UI as services.
7.1 Reference Scenarios

To illustrate the concept of UI components and UI integration, consider the development
of a US national park interactive guide (see Figure 7.1). There are three UI components
in this example: a national park listing which contains a list of US national parks, an
image displayer which shows images given a point of interest, and a map which displays
the location of a given address or point of interest. When the user selects a national park
Part 2: Mixup 149
from the park listing component, the image displayer will show an image of the selected
park while the map will display its location.
Instead of building the above three UI components from scratch, we choose to reuse
existing components. For the national park listing component, we can leverage the "Find
a Park" service from US National Park Service88. For the image displayer, we can use
the Flickr.NET component, which displays images retrieved from flickr.com given some
keyword tags. And for the map service, we can use Google Maps, which displays the
location map given a point of interest.
For the above example, one can manually build a composite application using client-
side JavaScript to maintain the coordination among the components, so that the
selection of a park name causes the map and the image to change. Most of web-based UI
integrations are done this way, which has several important drawbacks: the developer
needs to be intimately familiar with the details of each component, the integration code
is not reusable, and components become tightly coupled. In fact, if developers want to
switch components (e.g., use MapQuest instead of Google Maps) or "reuse" Google
Maps and Flickr in other applications, the development effort is significant.
Another very common example is the integration of user interfaces within enterprise
applications. For example, the HP monitoring tools mentioned in the last chapter
includes consoles for IT management, service management, and process management,
separately developed over time or through acquisition. Ideally, users want a single
enterprise console that integrates these more specific consoles to have an overall view of
the business process, of the services supporting this process, and of the IT infrastructure
supporting the services. Note that integration does not just mean to put the three GUIs
side by side: interactions need to be coordinated so that for example user interactions
with one component UI (e.g., visualization of a process) affect what is displayed by the
other UIs (e.g., displaying information on services and the IT infrastructure used by that
process).
88
http://www.nps.gov/
Part 2: Mixup 150
Figure 7.1 The National Park Guide
7.2 Lessons Learned from EAI / Service Composition

A large of amount research exists in the fields of integration and component-based
software development. Although problems and solutions differ based on the application
scenarios, certain issues appear to be common and certain approaches seem to be more
successful and applicable than others. A key learning from research in EAI is the need
for a homogeneous way to describe the different components to be integrated. This
description should be simple, formal, human readable, and modular.
Simplicity is paramount: it has been proven over and over that complex models and
languages do not succeed. In application integration, only simple languages made it into
the mainstream use, such as IDL and WSDL. Formalization is needed as the tool
support is essential. Tools relevant for integration include both development
environment as well as runtime middleware that handle bindings and interactions.
Readability is important as, although tools often act as mediation between a language
representation and the user, developers often need to read the specifications directly
(e.g., to overcome inflexibility of the tools).
Part 2: Mixup 151
Modularization is essential to disseminate a new integration model. Approaches that

tried to push a single specification to cover all aspects in a big bang approach had very
limited success. The problem here is that, first, the learning curve should be small and
developers only want to learn what is needed for the case they are handling; second, and
most importantly, the requirements become clear only after a technology is being used.
Hence, the best approach is to start simple, understand requirements, and then add
additional functionalities later if needed. This is for example the path adopted by Web
Services, which started with a very simple model, language, and protocol (SOAP and
WSDL) and then added additional features over time (coordination, transaction,
reliability, etc.), and is contrary to the path followed by ebXML89, which had a much
lesser success.
7.3 Design Principles

We now discuss the approach we take towards the notion of UI component. In the next
section we present a specific component model and a language that follows the
principles outlined here.
Services to be integrated at the data and application layers are all based on languages for
defining components, which are in turn based on a few fundamental abstractions. For
instance, in application-level integration, WSDL is often used to describe components
based on the notions of operations, messages, bindings, etc. In data integration, besides
"traditional" integration based on views and SQL, there are REST-based approaches that
support a CRUD (create, read, update, delete) based model for exposing data services on
the web. For example, recently there is a push towards data services by several major
software vendors (e.g., IBM's Service Data Object90, Microsoft's ADO.NET Data
Services [Fla08]).
We believe that providing appropriate component models for exposing web application's
UI is the key to bringing UI integration the benefits that service-oriented architectures
and middleware principles brought to information and application integration. The
89
http://www.ebxml.org/
90
http://www.ibm.com/developerworks/library/specification/ws-sdo/
Part 2: Mixup 152
challenge here is how to make the UI become a component, just like data or application
components in more traditional forms of integration.
On the web, most applications expose their user interfaces in HTML and CSS, which
can be only consumed by humans. What is needed is a UI component model to expose
UI as services, just like the way REST and RSS allow data to be exposed as services.
With a proper UI component model, developers can create reusable components and
integrate them in various composite applications. We are in particular interested in UI
components that encapsulate rich application functionalities (i.e., entire applications
including the UI layer), not low-level UI controls such as buttons, labels, and panels. In
fact, a typical UI component could consist of multitude of buttons, panels, and even top
level windows. Examples of UI components are Google Gadgets91, Vista Sidebar
Gadgets [Wes07], Java applets, ActiveX controls, and various AJAX components. For
instance, Live Weather92 is a Google Gadget that provides a UI front-end to
WeatherBug's weather database93.
In our national park example, the image displayer component serves as a UI front-end to
the Flickr image service that allows images to be retrieved based on keyword tags. The
map component leverages Google Maps to show the location of the selected national
park. While Google Maps is a complex web application involving UI, application logic,
as well as data services, the map component here primarily focuses on the UI aspect by
allowing developers to adjust the map's zoom level and change its font and background
color.
To further understand the interactions among UI components, application logic

components, and data services, let us consider the classical Model-View-Controller
(MVC) [Bur87] pattern. In MVC, the model contains the data model of the application;
it may also include any necessary application logic that manipulates the data. The view
renders the application's user interface to the user; it also handles user interactions (e.g.,
mouse click, text entry) and dispatches them to the controller. The controller contains
91
http://code.google.com/apis/gadgets/
92
http://www.labpixies.com/gadget_page.php?ID=49
93
http://www.weatherbug.com/
Part 2: Mixup 153
the UI logic that processes the user interactions by 1) invoking the necessary application
logic to retrieve / update the data in the model; 2) updating the view to render the new
UI to the user.
In the context of application integration, we can consider the model to include

application logic components and data services, whereas the view and the controller
both map to UI components. In this mapping, the view corresponds to the part of the UI
component rendering the UI while the controller corresponds to the part of the
component processing user interactions and communicating with application logic and
data services.
In the data layer, data access objects94 (DAO) expose data services and allow them to be
reused. Similarly, web services allow application logic to be exposed and reused.
However, a web service can achieve a higher level of reuse because it may reuse one or
more data services internally. In the same token, a UI component may internally reuse
one or more data services and application logic components. Therefore, UI components
allow the highest level of reuse since the application logic and data access logic behind
the UI are automatically reused.
Unlike regular application logic or data components, UI components are primarily user
driven. That is, user interactions ultimately drive the application's behavior. When the
user acts on the UI (e.g., press a button, type some text), the UI component may perform
the following tasks, according to its UI logic:
• Fire events to signal state changes
• Invoke some business logic components (e.g., invoking remote web services)
and retrieve / update some data
• Render UI update (e.g., results from service invocations)
In our example, when the user clicks on a specific national park displayed in the park
listing, the component will fire an event to signal this park selection. This event, in turn,
94
http://en.wikipedia.org/wiki/Data_Access_Object
Part 2: Mixup 154
will trigger the image displayer to show a photo of the selected park, and at the same
time, the map component will display a map of the park's location.
Conceptually, a UI component has two distinct facets. First, it has a graphical user
interface, which can be viewed and manipulated by the end user. Second, to perform
integration at the UI layer, UI components need to be manipulated programmatically
(similar to application logic components and data services). In fact, this is the service
aspect of UI components. Therefore, UI components need to support a programmable
interface, i.e., an API. This allows UI components to be exposed as services, which can
be easily integrated into composite applications. In addition, the API also allows UI
components to communicate with one another (directly or indirectly). Both the graphical
user interface and the API will affect the state of the component, which defines what can
be seen and controlled in terms of changes to the UI.
In a composite application, we need to ensure that all UI components play together in an

orchestrated fashion. When one component's UI changes, all other components need to
be aware of the change, and possibly react to the change as appropriate. As already
anticipated earlier, this means that the fundamental ingredients of a component model
are 1) a notion of state that describes what is currently displayed in the UI, 2) events
through which a component notifies state changes (including changes that have occurred
due to user interactions), and 3) operations which allow the state to be manipulated (i.e.,
by other components or the composite application). The state can be complex and
consist of multiple attributes (e.g., map location and zoom level). Note that this is a
significant departure from service technologies such as web services that do not include
a notion of state and where implementations are often in the form of stateless objects.
To specify the component's initial state (e.g., initial map location and initial zoom level),
we also need the notion of properties, which allow developers to specify the
component's initial state at design time and to query and inspect the component' state at
runtime.
In addition, UI components may require configuration parameters, such as Google

Maps' API key. Configuration values must be specified before accessing the
corresponding services. Therefore, configuration values need to be specified at design
Part 2: Mixup 155
time (or at component creation time). However, unlike properties, configuration

parameters are not part of the component's state, since they do not change at runtime
once specified.
Furthermore, UI components have UI appearance characteristics such as font face and

background color. It is up to the component developer to define what characteristics are
part of the state (and hence firing events to signal changes) and what characteristics are
configuration parameters. For example, a UI component monitoring a server's health
may change its background color to signal warning or error conditions, while its font
face remains constant. In this case, the background color becomes part of the
component's state, whereas the font face is a configuration parameter. In general, the
state should be kept as simple as possible to facilitate integration and reuse. This is
because state changes are what cause events to be exchanged among components and
therefore need to be handled in the composite application.
Finally, for the composite application to properly manage the UI components, it needs to
know if the UI is visible or hidden, minimized or maximized, etc. Therefore, the
composite application should be able to monitor, query, and update the presentation
modes of the components.
7.4 Abstract Component Model

Following the above discussion, we propose an abstract model for representing UI
components, where abstract means that it is not tied to specific implementation
technologies, and that it should be able to describe existing UI components from
heterogeneous component technologies.
The interface of a UI component consists of a set of events, operations, and properties

that expose its state at runtime, and a set of configuration parameters for design time
configuration:
Events: Events allow the component to notify the runtime environment of its internal
state changes. In our reference example, the selection of a national park in the park
listing component represents a typical abstract event.
Part 2: Mixup 156
Figure 7.2 Component-defined event vs. native UI event
It is fully up to the component's internal logic to decide which user interaction yields the
notification of an abstract event. That is, the user interacts with an application via mouse
and keyboard and thereby generates native events that are interpreted by the component
by means of its internal logic. For example (see Figure 7.2), when the user clicks on a
specific park in the list, we are interested in knowing what the actual meaning of that
click is (i.e., the selection of a national park). The interpretation of the low-level mouse
click event should be handled and translated into an abstract event by the component's
internal logic. The abstract event (i.e., "ParkSelectionChanged") represents the actual
interesting knowledge to be communicated to other components in the composition.
Essentially, UI components can be considered as standalone application modules, which

manage low-level native UI events internally and only expose high-level, abstract,
application-specific UI events.
Operations: Operations allow the internal state of the component to be changed

(typically to react to events fired by another component). For example, the map
component may provide an operation that allows the map location to be changed.
Again, it is fully up to the component developer to decide which operations to expose.
An operation typically supports a list of input parameters which allow the caller to pass
in values. The support of multiple input values enables an operation to set a property of
the component state with various options, or even to set multiple properties of the state
at the same time (e.g., setting map location and zoom level within a single operation).
The operations exposed by a UI component could be purely UI in nature, or more

commonly, they could also indirectly affect the business logic or data layers of the
Part 2: Mixup 157
application. Figure 7.3 depicts the relationship between UI components and business
logic / data components.
The email form is a UI component that allows an end user to send inquiries to the web
site that's hosting the component. Among the APIs exposed by the email form, "setTo"
and "setSubject" merely fill in values in the appropriate text fields, and therefore are
purely UI operations that do not involve any remote business logic or data services. On
the other hand, "sendMail" will invoke the appropriate APIs of GMail, Yahoo Mail, or
an SMTP server found in the local network95, depending on the user's email address.
Clearly, this operation involves remote application logic. Similarly, setting the zoom
level and the background color of the map component only involves UI, while showing
a new street address on the map requires invoking the appropriate API of Google's
mapping service in order to query and retrieve the new map data. Essentially, the UI
components on the left hand side of Figure 7.3 can be regarded as the presentation front-
end of the respective business / data services on the right hand side.
Figure 7.3 Relationship between UI components and application logic / data services
Properties: Properties can be used to expose various aspects of the component's state,
typically of UI appearance in nature. For instance, the map component may expose its
zoom level and background color as properties. At design time or component creation
95
Local SMTP servers can be usually identified through DHCP, DNS record, or directory services.
Part 2: Mixup 158
time, properties are used to represent the initial state of the component. At runtime,
properties can be also used to expose component's state. However, unlike an operation, a
property is usually expressed as a pair of getter and setter, supporting a single value.
This means that properties are simpler and easier to manage than operations, and
therefore more suitable for visual composition tools at design time or deployment time.
Configuration parameters: UI components may need to be properly configured before

execution, during either design time or instantiation time. Unlike properties which
represent the component's state, configuration parameters do not change during runtime.
For example, the API key96 that is required to invoke the map service from Google
represents such a configuration parameter. Unlike the zoom level, Google Maps' API
key only needs to be set at startup, and it will not change during runtime.
Presentation modes: In addition to events, operations, and properties, there are

characteristics common to all components which allow the runtime environment to
properly manage the component's execution. Collectively, we call them presentation
modes, which include:
• Component's visual appearance characteristics, such as its visibility (visible or

hidden) and window state (normal, minimized, or maximized);
• Component's lifecycle information. A component can be in one of the following

lifecycle states: instantiated (downloaded and instance created), ready (finished
initial configuration and ready to handle tasks), busy (busy processing tasks), and
destroyed (instance destroyed).
Presentation modes are different from component properties: their semantics must be
understood by the runtime environment for the components to be properly managed. As
a result, the runtime should be able to monitor, query, and update the presentation
modes of a component.
To properly describe UI components, we propose the UI Service Description Language

(UISDL), an XML-based component description language inspired by WSDL. For
96
http://code.google.com/apis/maps/signup.html
Part 2: Mixup 159
example, Listing 7.1 shows the UISDL descriptors for the three components in our
national park example.
<component id="parkListing">
<title>Park Listing</title>
<event name="ParkSelectionChanged">
<param name="parkName" type="xsd:string"/>
</event>
</component>
<component id="flickr">
<title>Flickr Image Displayer</title>
<operation name="displayPhoto">
<param name="tags" type="xsd:string"/>
</operation>
<config name="appKey" type="xsd:string"/>

</component>
<component id="gmap">
<title>Google Maps</title>
<operation name="showAddress">
<param name="address" type="xsd:string"/>
</operation>
<operation name="displayMap">
<param name="title" type="xsd:string"/>
<param name="geoLat" type="xsd:double"/>
<param name="geoLong" type="xsd:double"/>
</operation>
<property name="currentLocation" type="xsd:string"/>

<property name="zoomLevel" type="xsd:int">12</property>
<config name="apiKey" type="xsd:string"/>

</component>
Listing 7.1 UISDL descriptors (National Park Guide)
The <component> element describes the abstract component model. As shown above,
the park listing component will fire the event "ParkSelectionChanged" to signal park
selection changes (i.e., when the user clicks on a different park in the list). The Flickr
image displayer supports the operation "displayPhoto", which searches for images with
the given keyword tags in Flickr, and then displays the first image in the search result.
Part 2: Mixup 160
The map component supports two operations: the "showAddress" operation shows the
location of the given address or point of interest; the "displayMap" operation displays
the area map around the specified GPS coordinates. The map component also has two
properties: "currentLocation" represents the address of the location currently shown on
the map; "zoomLevel" controls the zoom level of the map display, which has the default
value "12". Finally, since both Flickr and Google Maps require API keys, the
configuration parameters "appKey" and "apiKey" supply that information for Flickr and
Google Maps, respectively.
7.5 UI as Services
The proposed abstract component model is aimed at exposing UI as services. That is,
the events, operations, and properties exposed by UI components allow them to be
modeled as services, just like application logic and data components.
The component model described so far (e.g., components in Listing 7.1) is abstract and
independent of any technologies used for the implementation of the native components.
However, in order to invoke a UI service, the abstract component model needs to be
bound to a concrete native implementation (i.e., a binding) for the component to be
executed at runtime.
<uisdl xmlns="http://www.openxup.org/2006/08/mixup/component"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<component id="parkListing">
... abstract component model ...
</component>
<binding name="ajaxBinding"
type="http://www.openxup.org/2007/08/bindings/Sack"
address="http://www.nps.gov/findapark/index.htm">
<event name="ParkSelectionChanged" address="selectPark"/>

</binding>
</uisdl>
Listing 7.2 Binding for the park listing component
As a result, the UISDL descriptor of a UI component is actually structured in two

sections, one describing the abstract component model (i.e., Listing 7.1), and the other
Part 2: Mixup 161
describing its bindings. Listing 7.2 contains the abstract component model (abbreviated
to save space) for the park listing component and its AJAX binding.
The <binding> element defines the component's binding. The type attribute specifies an
URI that indicates the component technology in which the component is implemented,
and the address attribute contains an URL where the component's implementation can
be downloaded. In the case of the park listing component, the binding refers to the
Simple AJAX Code-Kit97 (SACK), implying that the UI component has a SACK-based
AJAX implementation. As a result, the address of the binding points to an HTML page
containing the component's implementation – a mixture of HTML and JavaScript code.
In addition, the <binding> element also contains binding information for events,
operations, and properties. For example, the address of the "ParkSelectionChanged"
event in its SACK binding is "selectPark", which is the name of a JavaScript method.
Furthermore, it is advantageous for a UI service to have multiple bindings, just like Web
Services. Multiple bindings provide more flexibility when deploying composite
applications by allowing dynamic bindings at runtime (e.g., replacing component
implementations at runtime depending on platform compatibility or user preferences).
As shown in Listing 7.3, the Flickr image displayer has two bindings, a Java applet
binding and a .NET binding. The Java binding points to an applet class in a JAR file,
and the .NET binding points to a .NET class in a .NET assembly, respectively. In
addition, the bindings also map the "displayPhoto" operation to the appropriate native
methods in the component implementations: the Java method "searchAndDisplay" in the
Java binding and the .NET method "DisplayFirstPhoto" in the .NET binding.
<component id="imageDisplayer">
... abstract component model ...
</component>
<binding name="flickrJavaBinding"
type="http://www.openxup.org/2007/08/bindings/jApplet"
97
http://www.twilightuniverse.com/projects/sack/
Part 2: Mixup 162
address="http://.../flickr.jar#FlickrApplet">
<operation name="displayPhoto" address="searchAndDisplay"/>

</binding>
<binding name="flickrNETBinding"
type="http://www.openxup.org/2007/08/bindings/dotNET"
address="http://.../FlickrNet.dll#FlickrUI">
<operation name="displayPhoto" address="DisplayFirstPhoto"/>

</binding>
</uisdl>
Listing 7.3 UISDL descriptor with multiple bindings
Modeling UI as services provides a solid foundation for UI integration. Specifically, our

notion of UI services is protocol agnostic. That is, the abstract component model can be
accessed via a variety of protocols. For example, assume a UI component has three
bindings: the first binding may allow it to be accessed via SOAP, the second binding
may expose it via REST, and the third one may allow it to be invoked through RMI.
Furthermore, the proposed abstract component model does not enforce any specific API.
All it requires is the notion of events and operations, which are common in many
languages and runtimes. For example, operations can be mapped to direct method or
RPC calls, whereas events can be mapped to callback functions. As a result, the generic
nature of the proposed component model allows a wide range of existing, heterogeneous
components to be integrated and reused. Through bindings to multiple native
implementations, the abstract component model facilitates effective UI integration with
maximum runtime flexibility.
Finally, the notion of UI as services and the proposed abstract component model
distinguish our UI integration framework from other UI development frameworks. That
is, UI development frameworks all require components to adhere to strict APIs for them
to be integrated into applications. On the other hand, through our abstract component
model and the concept of multiple component bindings, our UI integration framework
allows existing, heterogeneous native UI components to be integrated without enforcing
specific APIs, as long as they exhibit the notion of operations (functions or methods)
and events (callback functions).
Part 2: Mixup 163
7.6 Summary
In this chapter we have presented the notion of UI components, leveraging lessons
learned from traditional integration techniques such as EAI and Web Services.
Specifically, our approach includes an abstract UI component model, where abstract
means the model description is not tied to any particular languages or platforms. This
design allows the component model to adapt to a wide range of native component
technologies. The key elements of the abstract component model are the notion of
events that signal state changes (typically caused by user actions), operations that
manipulate the component's state, and properties that allow the state to be queried.
To facilitate effective UI integration, we have presented the concept of UI as services,

which allows UI components to be invoked as services without requiring specific
protocols. Through multiple component bindings, our model makes it possible for native
UI components running on heterogeneous platforms and communicating in different
network protocols to be incorporated into the same composite application.
Finally, the proposed UI Service Description Language, modeled after WSDL, allows
the declarative specification of the abstract component model together with its native
component bindings. The language is designed to be simple and human readable, and
therefore can be edited by graphical tools and text editors alike.
Part 2: Mixup 165
Chapter 8. UI Composition Model
8. UI Composition Model
Research in data and application integration has made tremendous progress over the last
twenty years, enabling faster development and higher-quality software by supporting the
reuse of existing application components as opposed to redeveloping the application
from scratch. Work in this area has produced methodologies and tools supporting
component-based software development, data integration, application integration,
service composition, and middleware in general [Alo04]. Capabilities arising from
advances in integration technologies are now coming to the Web. However, research on
integration on the Web, especially when user interface aspects are included in the scope
of the integration problem, is still in the early stages.
UI composition is needed in a variety of situations. First, when a component does not

expose any API and its data cannot be directly accessed, integration can be only
performed at the UI layer. This is very common as many web applications and services
were designed for a specific business goal, not for reusing by other applications. Second,
UI composition may be the only form of integration that can be targeted at end users.
This is because end users perceive "UI" in their everyday application usage, so
composition in the UI layer is natural for them. On the other hand, learning sophisticated
application APIs and data schema presents a significant challenge for end users. As a
result, UI composition may eventually facilitate so called end user composition. Finally,
UI composition allows a higher level of reuse than composition at the application logic
or data layers. This is due to the fact that when a UI component is reused, the
application logic and data access logic behind the UI are automatically reused. As a
result, UI composition allows the component application's functionality to be maximally
reused.
However, existing composition techniques are not appropriate for UI composition,

where the objects of composition are high-level, coarse-grained UI components, such as
maps, photos, videos, and news sites. In particular, a major limitation is the inability to
easily create a composite application that reacts to user interactions in a fashion that is
coordinated among its constituent components, which is essential if we want to achieve
integration on the web. Therefore, we argue that in UI integration a key requirement is
Part 2: Mixup 166
that of synchronization, as all UI components need to display related content in a

coordinated fashion. For example, when the user clicks on a park name in the park
listing (in our national park guide), the image displayer should show a photo of the park
and the map should display its location automatically.
Consequently, we propose a UI composition model with a simple, event-based

approach, leveraging the UI component model we presented in the last chapter.
Specifically, the integration occurs in the form of a publish/subscribe model with the
help of event listeners, where each listener links a UI event from one component to an
operation of another component. This event-based model is particularly well-suited for
building component-based rich internet applications (RIA) and web mashups. Internally,
UI components may invoke (remote) application logic such as web services behind the
scene, but UI integration does not need to know (and does not care) about such internal
communications, as it only focuses on inter-component communications at the UI level.
Since composite applications are built with components from diverse sources, very often
two UI components in the same application may overlap in content. For example, two
components in a composite application may both need a person's name as input, so they
would both display an input field for users to enter name. However, it is desirable to
hide one of the input fields, as users should not be asked to input the same information
twice. Therefore, we need a UI content consolidation mechanism to merge semantically
identical UI content from different UI components.
In a composite application, it is often necessary to place a UI component inside another.

For example, many online news articles have embedded ads and video clips, where the
news report and the embed content are provided by different UI components. In this
case, the ads and video UI components need to be placed inside the news report
component. Hence, what is needed is a component embedding functionality that allows
one component to be embedded into another, where the two components may not be
aware of each other a priori.
With a flat component hierarchy, a composite application must be built with a flat list of
components. To achieve a higher level of reuse, it is desirable to package a composite
application itself as a reusable composite component. In that way, the composite
Part 2: Mixup 167
component can be further integrated into other composite applications in the same way
as regular simple components. The result is that a composite application will contain a
list of simple or composite components (i.e., forming of a tree of components).
In designing the composition model, our goals and challenges are 1) to derive
abstractions, models, and tools that are tailored for UI integration; and 2) to ensure that
derived models and tools are simple but effective, to avoid the multiplication of
formalisms and abstractions even if the scope of the integration problem is expanded.
This chapter is organized as follows. We first discuss the lessons learned from
traditional integration techniques, such as EAI and service composition. We then
describe our design principles and the relevance to traditional integration techniques.
After that we present the proposed event-based UI composition model in detail, using
the national park guide as an illustrative example. Finally, we illustrate several
additional features of our UI composition model, namely, UI content consolidation,
component embedding, and composite component.
8.1 Lessons Learned from EAI / Service Composition

A plethora of research is available in the fields of integration, such as EAI and service
composition. Although integration problems and solutions differ based on application
requirements, certain issues appear to be common and certain approaches seem to be
more successful than others.
In application integration, queue-based, publish/subscribe, and bus-mediated approaches

to interoperability [Alo04] have been quite popular. This has been proven by the success
of EAI and message broker platforms, and by the fact that even in Web Services,
originally born for fully decentralized interaction with no assumption on a common
middleware, the notion of enterprise service bus quickly emerged and now it is the
common approach to implement SOAs, at least within the enterprise.
Another interesting lesson, borrowed from application integration, is that there is no

easy solution to syntactical and semantic heterogeneity in application integration. In the
end, the solutions adopted amount to the specification of mapping and transformation so
Part 2: Mixup 168
that data can be exchanged among components, possibly with the aid of tools that
facilitate data matching and mapping definitions [Alo04].
8.2 Guiding Principles for UI Composition

As discussed in the last section, there are lessons to be learned from application and data
integration. However, there are also important differences that we need to keep in mind
when developing a composition model at the UI layer.
A major difference with respect to integration at other layers is that UI composition is

typically event-driven, and specifically driven by users' actions. When the user interacts
with the UI of a component, it will react according to its own UI behavior which may
result in certain state changes. At this point, other components in the same composite
application need to be aware of the new UI state of the first component, so that they can
update their UI accordingly.
In our national park example, this means that when the user selects a different park from
the park listing component, this component should fire a "ParkSelectionChanged" event
(Figure 8.1). This event notifies Flickr and Google Maps to update their UI accordingly
(i.e., displaying the image and the map of the newly selected park). Loose coupling here
advices the use of an intermediation as opposed to implementing point to point links
among components. As we will see this loose coupling is achieved via an event broker.
Park Listing
ParkSelectionChanged
(“Yellowstone”)
Composition
middleware
displayPhoto showAddress
(“Yellowstone”) (“Yellowstone”)
Flickr.NET Google Maps
Figure 8.1 National Park Guide (event-based model)

Part 2: Mixup 169
Hence, communication among components mainly consists of notifications of (and

requests for) state changes. In a composite application, what is important for the purpose
of UI coordination is being able to manipulate a component's state as well as to detect its
state changes (i.e., via events).
This is unlike EAI where a component offers an arbitrary set of methods consisting of
invocation and reply data, possibly complex and/or with large attachments. Furthermore,
in EAI, the integration is mainly procedural, achieved via the specification of fairly
complex control logic (e.g., in BPEL [And03] or other workflow-like language) that
causes the invocation of services, typically in some predefined sequence. The interaction
with the individual component is fairly complex as well and possibly regulated by a
business protocol. EAI components also typically do not have a first class, application-
specific notion of state.
Another difference is that UI components require layout customization, not just the
positioning of individual components, but a coherent layout of all components as a
whole. As a result, the location, size, shape, transparency, z-order, and window state
(i.e., minimized or maximized) of the components need to be taken into consideration
when building composite UI.
Furthermore, there are situations where the desired application components to be

integrated do not expose any invocable API. In this case, the only way to integrate the
desired functionality is through their user interface. For example, a particular web
application may not have been designed to expose any APIs or data sources, so it must
be integrated by manipulating its HTML output directly. This UI-level composition has
been brought to attention through the concept of mashups, where web applications were
repurposed and integrated while not having been designed with such intent.
While traditional integration has been the precinct of highly-skilled software engineers
building business applications, UI composition may in fact be suitable for tech-savvy
end users who wish to build applications that are not mission critical (a trend
popularized in Web 2.0). This is natural because UI composition allows end users to
directly model the way they usually interact with each component. Ideally, end users
would use a graphical tool to select desired UI components, place it on a canvas, and
Part 2: Mixup 170
then specify the interactions logic among the components along with their appearance
and layout properties in a drag-and-drop fashion – all based on an easy to understand
and almost self-explanatory model.
Simplicity in the composition phase does not come for free. Since the integration
problems are complex, complexity needs to reside somewhere. In the abstractions and
models presented in this thesis, the underlying idea is that complexity resides inside the
(reusable) components or in the wrappers/adapters developed for them, while the
composition model is kept at minimal complexity – unlike what happens in many other
contexts such as Web Services, where the composition model and language is rather
complex. This is due to many reasons: first, components are the reusable entities. They
are built once and reused many times, so it makes sense to embed the complexity there.
Furthermore, web applications today have rapidly evolving requirements, so the
composition logic may need to be updated frequently. As a result, the ease of
composition is crucial here.
Indeed, in a way the key challenge lies in how to remove features that one is tempted to
add to the composition model. That is, favoring simplicity over completeness, the goal
is to meet the requirements of a wide range of applications with a model that remains
simple yet usable.
Finally, EAI is characterized by hard requirements in terms of reliability,

transactionality, and security. In typical applications of UI integration this level of
reliability and security is not expected to be of crucial importance. Hence, the extra
complexity generated by reliability and security requirements may not be justified.
Therefore, to keep simplicity and ease of use as the major design goal, we will not put
emphasis on reliability and security in the proposed UI composition model, until if and
when such requirements materialize and are fully understood.
8.3 Composition Model

The proposed UI composition model is event-based. As such, it primarily includes event
subscription information to facilitate the communication among UI components. In
addition, the composition model may contain XSLT-based [Cla99b] data mappings and
Part 2: Mixup 171
additional integration logic in the form scripts or references to external code. Finally, the
composition model also includes layout information so that the UI components can be
positioned properly. To better explain the concept, we will again use our national park
example.
8.3.1 Event Subscriptions

Components exchange events through an event broker that facilitates loose coupling.
The composition model supports a one to many publisher / subscriber relationship
among UI components. That is, one component publishes an event (i.e., declares that it
will fire an event), and other components subscribe to it (i.e., declare that they will listen
to and handle this event). In our national park example, the image displayer and the map
component (subscribers) listen to the park selection changed event from the park listing
component (publisher).
The publisher/subscriber relationship is specified via event listeners. Each listener

specifies an event publisher, event type, event subscriber, and an operation of the
subscribing component. In addition, multiple event listeners can be used to support
multiple event subscribers for a single event from the event publisher. Note that to
facilitate loose coupling, event listeners are specified in the composition model, not in
the component model descriptors (i.e., UISDL) of the subscribing components.
Our national park example contains two event listeners: the first one links the
"ParkSelectionChanged" event from the park listing component to the "displayPhoto"
operation of Flickr, and the second one links the same event to the "showAddress"
operation of Google Maps.
Our event-based composition model works well with applications that are primarily
event-driven, where each interaction is a transition from an event in one component to
an operation in another component. Using multiple event listeners, multiple operations
from one or more components can be invoked concurrently in response to the event.
Finally, when direct mappings between event parameters and operation parameters are
impossible, additional mappings and transformations can be defined inside event
listeners. Specifically, inline or external XSLT style sheets may be specified in the event
Part 2: Mixup 172
listeners to define data transformation logic that maps the event parameters to operation
parameters.
8.3.2 XPIL
To properly describe the UI composition model, we propose the eXtensible Presentation
Integration Language (XPIL), an XML-based declarative composition language. For
example, Listing 8.1 describes the composition model of our national park example.
<xpil xmlns="http://www.openxup.org/2006/08/mixup/integration">
<component id="parkListing" ref=".../parklist.uisdl"
binding="ajaxBinding"/>
<component id="flickr" ref=".../flickr.uisdl"

binding="flickrNETBinding">
<config name="appKey">2cb8dd0691cee5b9817793467cae6427</config>
</component>
<component id="gmap" ref=".../gmap.uisdl" binding="...">

<config name="apiKey">P7TNqWBRsU8r5Us63k0HZSj0pLy9uo0jZrQ</config>
</component>
<listener id="parkChangedImgListener"
publisher="parkListing" event="ParkSelectionChanged"
subscriber="flickr" operation="displayPhoto"/>
<listener id="parkChangedMapListener"
subscriber="gmap" operation="showAddress"/>
<layout manager="http://www.openxup.org/2006/08/mixup/layout/css2">
......
</layout>
</xpil>
Listing 8.1 Composition model description (National Park Guide)
The XPIL document contains the integration logic among the three UI components in
our example. First, the three UI components must be properly referenced by pointing to
their respective UISDL documents. For each component, the developer must also
specify a concrete binding to be used. In our example, the developer chooses the SACK
/ AJAX binding of the park listing component and the .NET binding of Flickr. In
addition, component properties and configuration parameters may be initialized with
Part 2: Mixup 173
appropriate values within the component references. Here, the appropriate API keys for
accessing Flickr and Google Maps are specified.
The composition logic among components is defined in event listeners, which are
represented by XPIL's <listener> elements. For each <listener> element, the developer
specifies the publisher (the component that fires the event), the event, the subscriber (the
component that listens to the event), and the operation to be invoked on the subscriber
when the event is actually fired at runtime.
Our national park example contains two event listeners, which map the
"ParkSelectionChanged" event from the park listing component to the "displayPhoto"
operation of Flickr and the "showAddress" operation of Google Maps, respectively.
When the user selects a park from the list, the park listing component fires the
"ParkSelectionChanged" event. The first listener captures the event and then executes
the "displayPhoto" operation of Flickr by passing the park name as the operation's input
parameter, thereby causing a photo of the selected park to be displayed. For the same
event, the second listener invokes the "showAddress" operation of Google Maps,
resulting a new map of the selected park to be shown.
8.3.3 Additional Integration Logic

The primary goal of the composition model is to facilitate the declarative composition of
UI components. However, additional integration logic may be needed when simple one-
on-one event-based mapping is insufficient.
For example, assuming we want to extend our National Park Guide so that when the
user selects a park, the current weather information of the park will be displayed. The
weather information can be obtained from a weather service such as WeatherBug.
However, in order to retrieve the weather information, we need to first find the zip code
of the park via an address analyzer service. Therefore, we need to invoke the two
services sequentially, with the second one depending on the result of the first one.
Similarly, flow control logic (i.e., if-then-else) may be needed in certain applications.
Essentially, this calls for process-based integration logic in addition to event-based
logic.
Part 2: Mixup 174
We could extend our event model with additional XML constructs inside event listeners,
to specify sequencing and flow control logic. However, this will greatly complicate our
simple event-based model, making end user composition practically impossible.
Therefore, we believe complex process-based composition logic is better left to the
developers.
For developers, there are several alternatives in specifying process-based integration

logic. We could directly borrow XML elements from BPEL or use a simple scripting
language such as JavaScript. BPEL certainly provides comprehensive support for
process-based compositions. However, as one of our main goals in this research is
simplicity, we have opted for JavaScript due to the fact most web developers are already
familiar with JavaScript, whereas BPEL is only used in enterprise applications.
Another benefit of our scripting approach is that it gives developers a finer control of
the integration process, through the direct invocations of operations and properties of the
UI components. That is, a developer can further enhance a composite application by
writing code on top of the declarative composition framework that directly calls the
operations and properties of individual UI components. This allows the developer to
directly manipulate the state of the UI components and pass data among them.
In an ideal situation, building a composite application may be a joint effort between a

tech-savvy business user and a seasoned web developer. Since the business user is
familiar with the detailed business requirement, she should be responsible for the overall
design of the application, including the specification of event-based composition logic
in the form of event listeners. The web developer may further enhance the integration
logic if needed, by adding additional scripts inside those event listeners.
Listing 8.2 adds process-based integration logic to the event-based composition model
illustrated in Listing 8.1. We added an event listener98 to specify JavaScript code that
models process-based logic. In our national park example, after the user selects a new
98
The scripting logic could also be added to one of the existing event listeners; in that case, the script
will be executed before invoking the operation from the subscribing component. However, since
the weather information is not related to either Flickr or Google Maps, we choose to add a new
listener instead.
Part 2: Mixup 175
park, the "ParkSelectionChanged" event will be fired, resulting the script in the
"parkChangedWeatherListener" listener to be executed. Here, the address analyzer
service is invoked to find the zip code of the selected park, and then the weather service
is invoked asynchronously99 to retrieve the weather information based on the zip code.
Once the weather information for the given zip code is returned, it will be displayed in a
dialog window100.
......
<listener id="parkChangedWeatherListener"
publisher="parkListing" event="ParkSelectionChanged">
<script type="application/ecmascript"><![CDATA[
var zip = addressAnalyzer.findZip(event.parkName);
weatherBug.getWeatherInfoAsync(zip, function (weatherInfo) {
alert("Weather condition for " + event.parkName + "\n\n"
+ "Current: " + weatherInfo.current + "\n"
+ "Low: " + weatherInfo.low + "\n"
+ "High: " + weatherInfo.high);
});
]]></script>
</listener>
</xpil>
Listing 8.2 Process-based integration logic
Instead of inline scripts, event listeners may also refer to external code that contains
process-based scripting logic. This allows process-based logic to be separately defined
from the main event-based logic. That is, while a business user is specifying the main
event-based composition logic, a web developer can program process-based logic in an
external script file at the same time.
Finally, event listeners may contain any standard JavaScript statements, including code
that handles faults and exceptions from operation invocations and property accesses. In
addition, as an alternative to XSLT, data transformation logic that maps between event
parameters and operation inputs could also be specified in JavaScript.
99
The service is invoked asynchronously because retrieving weather information may take some
time. The result of the invocation is obtained via a JavaScript function closure, which allows the
callback function to be defined inline, thus avoiding explicitly defining a callback function.
100
We use JavaScript's alert dialog to simplify the illustration here. A better way is to display the
weather information as a marker on Google Maps, using the component embedding mechanism to
be discussed later in this chapter.
Part 2: Mixup 176
8.3.4 Layout Information

The composition model itself does not define any layout mechanism, but supports the
notion of external layout managers. This design facilitates maximum reuse of existing
layout technologies while at the same time providing a flexible and extensible layout
service for UI integration. The layout information is not interpreted by the composition
middleware; instead, it is simply passed to the appropriate external layout manager at
runtime.
For web-based deployment, the layout information is typically specified in CSS. To

properly position UI components, the CSS expressions in the composition model must
referred to the IDs of the UI components. In addition to the CSS expressions, an external
HTML layout template must be created to provide the necessary coupling between the
CSS layout specification and the UI components. The HTML template contains a layout
placeholder element for each UI component. Examples of layout elements are <div>,
<span>, and <iframe>. The coupling of a UI component and a specific HTML layout
element is achieved by assigning the placeholder element the same ID as the one
assigned to the UI component. As a result, the CSS expressions can be applied to
position the UI components through the corresponding HTML layout elements.
In addition, this design also allows the HTML layout elements to be positioned within
regular HTML constructs (e.g., tables and list). Essentially, the layout of a composite
application can be specified with a combination of HTML constructs (in the HTML
layout template) and CSS expressions (in the composition model). The web designer
may also apply any additional visual styles to the overall HTML document. In this way,
the composite layout can be easily designed with various professional web design tools
available on the market.
In our national park example, the <layout> element in the XPIL document (Listing 8.1)
contains the layout information for the composite application. As mentioned above, the
composition model does not define a specific layout mechanism; as a result, XPIL
allows any layout specifications inside the <layout> element, as long as they are well-
formed. The "manager" attribute defines the external layout manager to be used; in this
case, it's CSS2 [Bos98].
Part 2: Mixup 177
8.4 UI Content Consolidation

Composite applications are built with components from various sources (i.e., by
different developers and with different application requirements in mind). As a result,
very often two or more UI components in the same composite application may overlap
in content or functionality. Therefore, we propose in this section a content consolidation
mechanism to merge and propagate semantically identical UI content from different UI
components.
To illustrate content merging and propagation, we use online car purchasing as an

example. Car purchasing is a rather complicated process, but in this example we only
consider two components of this process: a credit application form and a vehicle
registration form. The credit application requests the user's name, social security
number, date of birth, driver's license number, and so on. And the vehicle registration
requires name, driver's license number, vehicle VIN number, etc. As we can see, the
information requested has some overlap (i.e., name and driver's license number).
In this example, we assume the two components were developed by different vendors.
As a result, the components are not aware of each other and therefore the overlapped
information needs to be filled in twice. That is, both components would display an input
field for the user to enter her name. Obviously, to give the end user a seamless UI
experience, it is desirable to hide one of the input fields, as the user should not be asked
to input the same information twice. The following listing shows UISDL descriptors for
the two components.
<component id="credApp">
<title>Credit Application</title>
<property name="firstName" type="xsd:string"/>

<property name="MI" type="xsd:string"/>
<property name="lastName" type="xsd:string"/>
<property name="SSN" type="xsd:string"/>
<property name="DOB" type="xsd:date"/>
<property name="DL" type="xsd:string"/>
......
</component>
......
Part 2: Mixup 178
<component id="vehReg">
<title>Vehicle Registration</title>
<property name="fullName" type="xsd:string"/>

<property name="driverLic" type="xsd:string"/>
<property name="VIN" type="xsd:string"/>
......
</component>
Listing 8.3 Component descriptors for credit application and vehicle registration
We use the term content item to refer to the part of component's UI content that can be
merged with other components in the same application. Content items are part of the
component's overall state and therefore they can be exposed by properties and modified
by operations, and their changes are notified via events.
In the above example, the credit application component uses several properties to
expose its content items. For instance, the property "DL" would be displayed as an input
field that requests the driver's license number at runtime, and in a similar fashion the
"DOB" property represents the user's date of birth. For the vehicle registration
component, the content items that represent the user's full name, driver's license number,
and vehicle VIN number are exposed as properties.
Our goal is that once the user enters her driver's license number in the vehicle
registration component, the same information will be automatically duplicated in the
credit application components. And similarly, once the user fills in her first name,
middle initial, and last name in the credit application component, her full name will be
computed and automatically propagated to the vehicle registration component. The
XPIL fragment in Listing 8.4 illustrates the proposed content merging / propagation
mechanism.
As shown below, content consolidation is specified in the composition model, via the
<content-merging> elements, which describe how content items from different
components are merged. For the driver's license number, all we need to do is to map the
credit application component's "DL" property to the vehicle registration component's
"driverLic" property, so that the value "DL" will be automatically copied from the value
of "driverLic".
Part 2: Mixup 179
<xpil ...>
......
<content-merging>
<propagate component="credApp" property="DL">
<depend component="vehReg" property="driverLic"/>
</propagate>
<propagate component="vehReg" property="fullName">

<depend component="credApp" property="firstName"/>
<depend component="credApp" property="MI"/>
<depend component="credApp" property="lastName"/>
<value language="application/ecmascript"> <![CDATA[

credApp.firstName + ' ' + credApp.MI + ' ' + credApp.lastName
]]> </value>
</propagate>
<hide component="credApp" name="DL"/>

<hide component="vehReg" name="fullName"/>
</content-merging>
</xpil>
Listing 8.4 XPIL fragment for content consolidation
For the user's name, it is slightly complicated, since the credit application requires the
name to be separated into three parts, whereas the vehicle registration requires the full
name as a single string. As a result, we need to make the "fullName" property of the
vehicle registration component depending on the "firstName", "MI", and "lastName"
properties of the credit application component. This is shown in the second <propagate>
element in Listing 8.4. In addition, the <value> element allows us to specify an
expression that combines the three-part name into a single full name. In fact, the
<value> element supports both JavaScript expression and XSLT template for deriving
the value of the target content item from the values of other content items that it depends
on.
8.4.1 Value-Changed Events

The content consolidation specification in the composition model alone is not sufficient
to propagate the value of one content item to another. This is because the value of a
content item may be dynamically changed (i.e., by direct end user actions or some
operations resulted from user actions), and when that happens the values of other
Part 2: Mixup 180
content items that depend on it would become obsolete. Therefore, what's needed is a
mechanism to monitor the value changes in content items.
To monitor content items' values, we introduce the notion of value-changed events. The
following listing shows the modified version of the two component descriptors, with
value-changed events added.
<component id="credApp">
<title>Credit Application</title>
......
<property name="firstName" type="xsd:string"/>
<property name="MI" type="xsd:string"/>
<property name="lastName" type="xsd:string"/>
......
<event name="firstNameChanged">
<param name="firstName" type="xsd:string"/>
</event>
<event name="MIChanged">
<param name="MI" type="xsd:string"/>
</event>
<event name="lastNameChanged">
<param name="lastName" type="xsd:string"/>
</event>
</component>
<component id="vehReg">
<title>Vehicle Registration</title>
......
<property name="driverLic" type="xsd:string"/>
......
<event name="driverLicChanged">
<param name="driverLic" type="xsd:string"/>
</event>
</component>
Listing 8.5 Component descriptors with value-changed events
As shown above, the credit application component exposes three events that notify the
value changes in the content items that correspond to the three parts of the name.
Similarly, the vehicle registration component exposes an event to signal the value
changes in the content item that represents the driver's license number.
Part 2: Mixup 181
With the addition of value-changed events, the runtime middleware can now capture
value changes from one content item and automatically propagate it to other dependant
content items.
In general, for each content item, the component should expose a corresponding value-
changed event, with a parameter that represents the new value of the content item. In
addition, the event name should be derived from the corresponding property name, with
the string "Changed" appended. As an example, the property name for the driver's
license number is "driverLic"; therefore the corresponding value-changed event should
be named "driverLicChanged". This design allows the runtime middleware to
automatically capture content item changes and then compute the values of the
dependant content items based on those changes.
Therefore, the basic requirement for content merging and consolidation is that the
component exposes a property and a value-changed event for each content item. For
example, suppose the vehicle registration component is implemented as an HTML
Form, then the input field corresponds to the driver's license number can be exposed as a
property and the form validation event can be used as the value-changed event.
8.4.2 Hiding Content Items

The framework is now able to propagate values among semantically identical content
items. However, the user will perceive duplicated display of the values. For example,
once the driver's license number is entered into vehicle registration, the same
information will appear in the credit application. To provide a seamless integrated user
experience and to save screen real estate, it is desirable to hide the derived content item.
In this case, we would like to hide the input field of the driver's license number in the
credit application component.
The <hide> element in Listing 8.4 serves this purpose. It hides the content item with the
specified property name. Although typically used to hide derived content items, it can be
actually used to hide any content items, for example to disable certain functionality of a
component.
Part 2: Mixup 182
Supporting content item hiding depends on whether the UI component is markup-based

(e.g., HTML, XUL) or non-markup based (e.g., ActiveX controls, Java applets). For
most markup-based components, it can be done via XPath [Cla99a] and CSS. First, in
the component's binding, the address of the property that represents the content item
needs to be specified in XPath. In our car purchasing example, the XPath expression
"//form/input/[@name='DL']" would identify the driver's license number input field in
the credit application component. At runtime, the composition middleware can simply
retrieve the input field identified by the XPath expression and apply CSS property
"display:none" to hide the input field. This approach works for UI components based on
most markup languages, as long as they support CSS or other style sheet mechanisms.
However, for non-markup based components, the composition middleware may not be
able to manipulate the internal UI of the components, so there is no systematic way in
hiding content items in these components. In that case, the component needs to expose
an operation to allow the framework to turn off the display of its content items. The
operation name must be derived from the property name of the content item, with the
string "hide" prepended. As an example, the property name for the driver's license
number in the credit application component is "DL"; therefore the corresponding hide
operation should be named "hideDL".
In summary, at runtime when the user enters her driver's license number into the vehicle
registration component, the corresponding value-changed event ("driverLicChanged")
will be fired. The runtime middleware automatically captures the event and propagates
the updated property value to the credit application component. Furthermore, the <hide>
element instructs the middleware to hide the corresponding input field for the driver's
license number in the credit application component. To do so, the middleware may
invoke an operation named "hideDL", if provided by the component; otherwise, the
middleware will attempt to identify the content item through XPath and then apply the
appropriate CSS style to hide it.
8.4.3 Discussions
As shown in the above example, content consolidation may occur between more than
just two items. That is, the value of one content item may be derived from two or more
Part 2: Mixup 183
other items. In addition, content consolidation may occur among more than two
components. In fact, the value of a content item may be derived from two or more
content items in two or more components. The content consolidation mechanism
proposed here is flexible enough to support the above requirements.
In addition, content items may represent output displays rather than end user inputs. For
example, the content item that corresponds to driver's license number in the credit
application component may be rendered as a display label. When the user enters her
driver's license number in the vehicle registration component, the display label in the
credit application component can be automatically updated with the value.
Furthermore, content items can be hidden, not showing any input or output. For
example, the credit application component may not display any UI for the driver's
license number at all; that is, the content item could be hidden to begin with. The
component would expect the information to be provided through API rather than user
input. In our case, the value for this hidden content item would be automatically
propagated once the user entered her driver's license number into the vehicle registration
component.
Finally, although most of the content items in our example have simple string types, the
same content consolidation technique applies to content items of complex types as well.
In fact, the content items to be merged or propagated may have mismatching types. All
it takes is for the composition developer to provide the appropriate XSLT template or
JavaScript expression that performs the necessary data transformation.
8.5 Component Embedding

In a composite application, side-by-side and overlapping positioning of components can
be easily achieved (e.g., via an HTML / CSS layout template). However, very often one
component may need to embed into another component. For example, many online news
articles have embedded ads, images, and video clips, etc. And in a real estate
application, it is desirable to place a small thumbnail image of the selected property on a
map showing the property location.
Part 2: Mixup 184
To illustrate the component embedding concept, assume there is a UI component that

displays a new report, and we wish to embed the YouTube video component somewhere
in the report to display a video clip associated with the report. The following listing
contains the component descriptors for the two UI components.
<uisdl xmlns="http://www.openxup.org/2006/08/mixup/component">
<component id="youtube">
<title>YouTube Video</title>
......
</component>
<binding name="ytDotNETBinding">......</binding>
</uisdl>
<uisdl xmlns="http://www.openxup.org/2006/08/mixup/component">
<component id="news">
<title>News Report</title>
......
<slot id="videoClip"/>
<slot id="ads"/>
</component>
<binding name="newsHTMLBinding"
type="http://www.openxup.org/2007/08/bindings/XHTML"
address="http://.../news-report.jsp?article=...">
......
<slot ref="videoClip" address="//div/[@id='videoDiv']"/>
<slot ref="ads" address="//div/[@id='adsDiv']"/>
</binding>
</uisdl>
Listing 8.6 Component descriptors for YouTube and the news report
As shown above, the news report component exposes two slots, one for embedding a
video clip, and the other for embedding an advertisement. Slots are abstract, just like
events and operations. The component's binding specifies the implementation aspect of
the slots. In our example, since the component has an XHTML binding, the slots can
simply be implemented as HTML <div> elements. Therefore, the addresses of the slots
are XPath expressions pointing to the corresponding <div> elements. Of course, the
slots can be also implemented by other HTML elements, such as <span> and <iframe>.
And the component may apply any layout or visual styles to the slots.
Part 2: Mixup 185
To embed the YouTube video component into the news report, we need a new construct
in the composition model, as shown in the following listing.
<xpil ...>
......
<embedding>
<embed component="youtube" host="news" slot="videoClip"/>
</embedding>
</xpil>
Listing 8.7 Composition model for the multimedia news report
The <embed> element specifies that the YouTube video should be embedded into the
"videoClip" slot of the news report component. At runtime, the YouTube video will be
inserted into the HTML <div> element with the ID "videoDiv".
One might argue that it is relatively straight forward for the composition developer to
manually embed a YouTube video into an HTML-based news report. However, the
news report component is likely to be provided by some third party; therefore the
composition developer may not have access to its internal implementation. In addition,
by using the proposed slot mechanism, any video players can be easily embedded, such
as Windows Media Player or RealPlayer. Assuming the UISDL descriptors for the
above mentioned video players are already available, the composition developer only
needs to create the appropriate <embed> elements to complete the embedding
specification. That means the composition developer is no longer required to learn the
specific HTML embedding details for those video players, no matter how simple or
complex the syntaxes are.
We will now illustrate a more complicated example with a real estate application. Here,
we have two components, a property image component that displays a thumbnail of a
given property and Google Maps which shows the location of the property. The goal is
to display the property's thumbnail image at the property location on the map. Therefore,
we need to embed the property image component into Google Maps, at a location
specified at runtime.
Part 2: Mixup 186
One could perform this embedding manually via Google Maps API, using markers and
the info window101 (e.g., GMarker.openInfoWindowHtml). However, this would require
the composition developer to be familiar with Google Maps API. Our goal is to make
the embedding process as easy as possible, without intimate knowledge of the native
components involved.
The major difference between this example and the news report example is that the slot
location is now dynamic. In the news report example, the slot points to an HTML <div>
element that is statically specified at design time; whereas the slot in this example
corresponds to the selected property address, which dynamically changes at runtime. To
satisfy this requirement, slots need to be parameterized, just like events and operation.
The following listing shows component descriptors for Google Maps and the property
image component, PImage, respectively:
<component id="gmap">
<title>Google Maps</title>
......
<slot id="streetAddr">
<param name="address" type="xsd:string"/>
</slot>
<slot id="geoCode">
<param name="latitude" type="xsd:double"/>
<param name="longitude" type="xsd:double"/>
</slot>
</component>
<binding name="gWrapperBinding"
type="http://www.openxup.org/2007/08/bindings/wrapper"
address="http://.../gmap-wrapper.js">
......
<slot ref="streetAddr" address="embedByAddress"/>
<slot ref="geoCode" address="embedByGeocode"/>
</binding>
</uisdl>
101
http://code.google.com/apis/maps/documentation/reference.html
Part 2: Mixup 187
<component id="pImg">
<title>PImage</title>
......
<property name="addr" type="xsd:string"/>
</component>
<binding name="...">......</binding>
</uisdl>
Listing 8.8 Component descriptors for Google Maps and PImage
As shown above, our map component includes two slots, one based on street address
and the other based on geo-code. The locations of the slots are dynamic during runtime
because they are now parameterized. In addition, the bindings of the two slots point to
JavaScript functions in the component's native implementation. For example, the
"embedByAddress" function takes an HTML fragment and displays it in an info window
on top of a marker set at the property address on the map. Finally, the following XPIL
fragment completes this example:
<xpil ...>
......
<embedding>
<embed component="pImg" host="gmap" slot="streetAddr">
<param name="address" value="pImg.addr"/>
</embed>
</embedding>
</xpil>
Listing 8.9 Composition model for the real estate application
In the composition model, the <embed> element now includes a parameter value that
matches the parameter of the corresponding slot in the hosting component. In addition,
the parameter value points to the "addr" property of PImage, which specifies the real
estate property's street address at runtime. As a result, at runtime PImage (i.e., the
property's thumbnail image) will be embedded dynamically at the property's location on
the map.
In summary, the slot-based component embedding mechanism offers a flexible yet

powerful way for runtime component embedding. Slots are now part of the abstract
component model, just like events and operations. When parameterized, slots can be
used to specify any desired embedding locations, whether static or dynamic.
Part 2: Mixup 188
Component embedding works well for embedding both markup-based and non-markup
based components. For markup-based components, the markup output of one
component can be simply embedded into another. For non-markup based components,
the corresponding HTML element (e.g., <object> for ActiveX controls and Java applets)
can be inserted into the hosting component when deployed in a web-based environment.
Finally, component embedding changes the positioning relativity of embedded

components. When embedded, the component's layout and positioning is now relative to
its hosting component, instead of the composite application.
8.6 Composite Component

Creating composite components has been an important technique in component-based
software development. We would like to apply the same concept in UI integration. The
goal is to reuse the integration logic among components.
In Mixup, with a flat component hierarchy, a composite application must be built with a
flat list of components from scratch. To achieve a higher level of reuse, it is desirable to
package a composite application itself as a reusable composite UI component.
Therefore, the integration logic specified in the composite application can be reused as
well. In addition, the composite component can be further integrated into other
composite applications in the same way as regular simple components. Consequently, a
composite application may contain a list of simple or composite components, essentially
forming of a tree of components.
To illustrate the composite component concept and its applicability in UI integration, we

will again use the US national park guide example, which consists of three UI
components: a park listing which contains a list of US national parks, an image
displayer (Flickr) which shows images given some keyword tags, and a map (Google
Maps) which displays the location of a given address or point of interest. When the user
selects a park from the park listing component, Flickr will show an image of the selected
park while Google Maps will display its location.
Now consider enhancing the park guide application by adding a video component,
therefore resulting in a multimedia park guide. There are two ways to achieve this. First,
Part 2: Mixup 189
we could add the YouTube component to the existing composite application and then
create an event listener between the park listing component and YouTube. The second
approach is to package the existing composite application as a reusable composite
component, and then build a new composite application which contains this composite
component and YouTube. The two approaches require similar amount of effort.
However, with the second approach, the composite component containing the original
park guide could be reused in many other applications in addition to our enhanced
multimedia park guide. We shall now proceed with the second approach by packaging
the original park guide as a composite component.
xmlns:xpil="http://www.openxup.org/2006/08/mixup/integration"
<component id="origNPG">
<title>US National Park Guide UI Component</title>
<component ref="http://.../parklist.uisdl" id="parkListing"/>

<component ref="http://.../gmap.uisdl" id="gmap"/>
<component ref="http://.../flickr.uisdl" id="flickr"/>
<xpil:listener id="parkChangedMapListener"
subscriber="gmap" operation="showAddress"/>
<xpil:listener id="parkChangedImgListener"


<event name="ParkSelected" sub="parkListing.ParkSelectionChanged">
</event>


<operation name="displayPhotoAndMap"
sub="flickr.displayPhoto|gmap.showPOI">
</operation>
......
</component>
</uisdl>
Listing 8.10 Composite component for the original park guide
Part 2: Mixup 190
The UISDL document above describes the composite component for the national park
guide. In fact it is quite similar to the XPIL document that describes the original
composite application, as shown in Listing 8.1. They have the same set of component
references and event listeners. The major difference is that the component references
here only point to the referenced component descriptors, but not to any specific
bindings. This is because the component model for a composite component should be
abstract; as a result, the bindings to the nested components should be specified at the
composition model when building composite applications. In addition, the composite
component itself does not have any bindings, since it is made up of other nested
components and therefore not directly implemented by a native component. Another
minor difference is that the listener elements here are under XPIL's namespace, since
they are not part of UISDL.
Just like a regular component, a composite component should also expose events and
operations in order to interact with other components in the containing application.
However, to hide the internal details of the composite component, the events and
operations of its sub-components are not automatically exposed to the outside (i.e.,
events and operations have a local scope). Therefore, a composite component should
define its own set of events and operations, with appropriate mappings to the events and
operations in the nested components.
In Listing 8.10, the composite component exposes a composite event "ParkSelected",

which maps to the "ParkSelectionChanged" event of the park listing sub-component. As
a result, when the user selects a different park, the composite event "ParkSelected" will
be fired by the composite component. Similarly, invoking the composite operation
"displayPhotoAndMap" will in turn execute the "displayPhoto" operation of Flickr and
the "showAddress" operation of Google Maps. Note that the "|" operator between the
two sub-operations implies they will be executed concurrently. If replaced by the ","
operator, the two sub-operations will be executed sequentially instead. In addition, the
composite operation may contain XSLT expression or JavaScript code that performs
data transformation when needed; for example when the sub-operations' input
parameters do not match the composite operation's input parameters.
Part 2: Mixup 191
Now that we have the composite component descriptor for the original park guide, we
can proceed to build the composite application for our enhanced multimedia park guide.
The following listing shows the composition model of the application:
<xpil ...>
<title>Multimedia National Park Guide</title>
<component ref=".../npg-composite.uisdl" id="origNPG">

<bind component="parkListing">ajaxBinding</bind>
<bind component="gmap">gWrapperBinding</bind>
<bind component="flickr">flickrNETBinding</bind>
......
<property name="gmap.zoomLevel">13</property>
<config name="flickr.appKey">cb8d0691cee5b98</config>
</component>
<component ref="http://.../youtube.uisdl" id="youtube"

binding="ytDotNETBinding">
<config name="devId">N0m3tA9LJfr</config>
</component>
<listener id="parkSelectedListener"
publisher="origNPG" event="ParkSelected"
subscriber="youtube" operation="displayVideo"/>
</listener>
</xpil>
Listing 8.11 Composition model for the multimedia park guide
The composition model above references two components: the YouTube UI component
and the composite component for the original park guide (Listing 8.10). The component
reference to the original park guide specifies the bindings to its sub-components. In
addition, the initial property and configuration values (i.e., Google Maps' zoom level
and Flickr's application key) of the sub-components are also defined there.
In addition, the event listener defined in Listing 8.11 specifies the interaction between
YouTube and the composite component for the original park guide. Specifically, when
the composite event "ParkSelected" is fired, YouTube's "displayVideo" operation will
be invoked.
At runtime, when the user selects a different park, Flickr will display an image of the
park and Google Maps will show a map of the park's location. And at the same time,
Part 2: Mixup 192
YouTube will play a video related to the park. The first two interactions happen among
the sub-components within the composite component, while the last interaction is
between YouTube and the composite component, at the composite application level.
8.7 Summary
In this chapter we have presented an event-based UI composition model, leveraging
lessons learned from traditional integration techniques such as EAI and Web Services.
We believe the event-based model is ideal for UI composition due to the fact that user
interfaces are event-driven by nature.
Specifically, our approach allows UI composition logic to be specified in so called event

listeners, each of which defines a publisher / subscriber relationship between the
component that fires the event and the component that reacts to the event. Specifying
event listeners in the composition model allows UI components to be loosely-coupled.
That is, UI components working in the same composite application are not required to
have any prior knowledge of each other.
The proposed UI composition language, XPIL, allows the declarative specification of

event-based interaction logic among loosely-couple UI components. Just like UISDL,
the language is designed to be simple and human readable, and therefore can be edited
by graphical tools and text editors alike.
In addition, the UI content consolidation mechanism allows semantically identical UI

content to be merged or propagated among two or more components, giving users a
seamless UI experience. Furthermore, the component embedding functionality allows a
component to be embedded into another, a feature often needed in web mashups.
Finally, we extended our component model to allow composite components, made from
existing composite applications. This facilitates the reuse of an entire composite
application (including its composition logic) as well as hiding the details of its internal
nested components.
Part 2: Mixup 193
Chapter 9. UI Integration Framework
9. UI Integration Framework
Based on the proposed abstract UI component model and event-based composition
model, we present in this chapter a framework to facilitate UI integration. Figure 9.1
illustrates the high-level architecture of the proposed framework (called Mixup) for the
execution of composite applications.
Components
C1 C2 C3
Operations
Events
Event
Listeners
L L L
Event Broker
UI Integration Middleware
Registry
C3
C2
C1 L
Composer
XPIL HTML
Composition template
Component Descriptors
Figure 9.1 Architecture of the UI integration framework, Mixup
A composite application consists of one or more components, a specification of the

composition model (i.e., integration logics that coordinate the components at runtime)
and a middleware for the execution of the composition. The middleware includes an
event broker that manages a set of event listeners defined in the composition model. The
event listeners map state change events, generated by one component, onto operations
(i.e., state change requests) of other components.
The specification of the composition is performed by the application composer (i.e.,

composition developer) at design time, who may also consult a proper component
registry to identify UI components that suit his/her application requirements by
inspecting the respective UISDL component descriptors.
Part 2: Mixup 194
The rest of the chapter is organized as follows. We first show an example application to
be used for illustrating the design and features of the framework. We then discuss the
middleware that is responsible for the runtime execution of composite applications,
including a component adapter infrastructure that allows the runtime to communicate
with existing native UI components in heterogeneous platform technologies. After that
we present a development environment and its associated tools to facilitate rapid
creation of composite applications. Finally, we discuss the implementation of the
framework, including both the runtime middleware and development tools.
9.1 Example Scenario

To illustrate the design and features of the proposed UI integration framework, consider
the development of an interactive news reader (see Figure 9.2). There are four UI
components in this example: a news feed containing a list of news items from New
York Times, a map which displays the location of the news story, an image displayer
that shows images relevant to the selected news article, and a video player which plays
videos related to the news articles. When the user selects a news item from the list, the
image displayer and video player will show an image and video of the selected news
article respectively, while the map will display its location.
Instead of building the above four UI components from scratch, we choose to reuse
existing components. For the news listing, we leverage a Yahoo pipe102 that processes
New York Times' RSS feed. For the image displayer, we can use Flickr, which displays
images given some keyword tags. Similarly, for the video player, we will use YouTube,
which plays a video given some keyword tags. Finally, for the map service, we can use
Google Maps, which shows the location map given a geo-code (latitude and longitude).
We have already covered Flickr and Google Maps in the previous chapters. Listing 9.1
shows the UISDL descriptors for the Yahoo pipe news listing and YouTube video103.
102
We use the Yahoo pipe to add keyword (i.e., tags) and geo-code (i.e., latitude and longitude)
information to the New York Times news feed.
103
The news feed UI component renders a list of news articles from the New York Times pipe
hosted by Yahoo Pipes; whereas the YouTube UI component serves as a UI front-end to the
YouTube video service at www.youtube.com.
Part 2: Mixup 195
<component id="rss">
<title>News feed (Yahoo Pipes)</title>
<event name="ItemTagsActivated">
</event>
<event name="ItemGeoActivated">
<param name="title" type="xsd:string"/>
<param name="geoLat" type="xsd:double"/>
<param name="geoLong" type="xsd:double"/>
</event>
<config name="pipeUrl" type="xsd:anyURI" required="true"/>

</component>
<binding name="...">......</binding>
</uisdl>
<component id="youtube">
<title>YouTube</title>
<operation name="displayVideo">
</operation>
<operation name="prevVideo"/>
<operation name="nextVideo"/>
<event name="EndOfStream"/>
<config name="devId" type="xsd:string" required="true"/>

</component>
<binding name="ytDotNETBinding"
type="http://www.openxup.org/2007/08/bindings/dotNET"
address="http://.../YouTubeUI.dll#...YouTubeUI">
<operation ref="displayVideo" address="PlayVideo"/>

<operation name="prevVideo" address="PlayPrevVideo"/>
<operation name="nextVideo" address="PlayNextVideo"/>
<event name="EndOfStream" address="EndOfVideoEvent"/>

</binding>
<binding name="ytDojoBinding"
type="http://www.openxup.org/2007/08/bindings/dojo"
address="http://.../scripts/youtube.js">
......
</binding>
</uisdl>
Listing 9.1 UISDL descriptors for Yahoo pipe news feed and YouTube video
Part 2: Mixup 196
Figure 9.2 New York Times example
9.2 Runtime Middleware

The framework provides a lightweight runtime middleware for executing composite
applications. The runtime middleware integrates UI components by leveraging
information in the composition model. The middleware is in charge of loading and
instantiating the components, placing their UIs into the respective HTML layout
elements, setting up the necessary listeners as specified in the XPIL definition of the
composition, and dispatching events and invoking operations in response to user
actions.
There are two key ingredients in the middleware. First, the middleware offers an event
automation mechanism which allows the invocation of designated component
operations in response to events; second, it provides a component adapter framework
for connecting to components from heterogeneous component technologies.
Part 2: Mixup 197
In addition, though not discussed in this chapter, the middleware also supports common
services such as data transformation, component naming, location, and lifecycle
management. Our middleware currently does not provide advanced features found in
EAI, such as transactions and queues. As stated earlier, we want to start simple and
hence, will not emphasize on non-functional aspects such as security or reliability.
Following examples in service composition, those features can be added later if and
when needed (e.g., WS-* specifications for Web Services).
9.2.1 Event Automation

To facilitate the declarative specification of UI integration, the middleware supports the
notion of event automation. Via event automation, the middleware captures an event
from a source component and automatically dispatches it to the designated operations of
the subscribing components, based on the event listener specifications in the
composition model.
Conceptually, this is similar to how message brokers and event buses behave, with the
difference that there is no explicit subscription done by the components (i.e., in the
component model). Instead, the event subscriptions are specified via event listeners in
the composition model.
In traditional publish/subscribe or observer models, subscribers and/or publishers must

be aware of the event dispatching logic. Therefore, there is a tight coupling either with
the event (often called topic) being published (publish/subscribe model) or with the
subscriber (observer pattern). To avoid this tight coupling, the definition of which event
triggers which operation invocation, as well as the definition of related data mappings,
must reside in the composition model, not the component model. As a result, our
middleware can automatically perform transformations from events raised by one
component onto operations of the subscribing components. Thanks to this design
choice, our runtime middleware allows for rich interactions among loosely coupled, pre-
built UI components.
Figure 9.3 provides a simple illustration of what happens at the runtime, using our New
York Times example:
Part 2: Mixup 198
1. Capturing event from the publishing component
a. After the user clicks on a news item's title, the news feed component fires
two events: "ItemTagsActivated" and "ItemGeoActivated".
b. The middleware captures these events.
2. Automatically invoking operations of the subscribing components
a. For each event, the middleware searches for a list of event listeners
matching the event.
b. For each listener, the middleware executes the data transformation logic
(if any) that maps event parameters to operation inputs, and then invokes
the specified operation on the subscribing component. In our example,
for the "ItemTagsActivated" event, Flickr's "displayPhoto" operation and
YouTube's "displayVideo" operation will be invoked, causing the two
components to display images and videos related to the news story. For
the "ItemGeoActivated" event, the "displayMap" operation of Google
Maps will be invoked, resulting a map showing the news story's location.
Figure 9.3 Event automation
In summary, the event automation mechanism goes one step further than the traditional
event publishing and subscription mechanism: it facilitates the automatic invocation of
component operations in response to events. In addition, event subscriptions are
Part 2: Mixup 199
specified in the composition model, not the component model. That is, a component
publishes the events it fires via <event> elements in the component model; and the
<listener> elements in the composition model define event subscriptions by linking the
events to the designated operations in the subscribing components. This lays a solid
foundation for the declarative composition of loosely-coupled UI components.
9.2.2 Component Adapters

A key requirement in an integration framework is the ability to support native
components implemented in existing component technologies, such as ActiveX, .NET,
Java Applet, and AJAX. Therefore, the framework provides the notion of component
adapters to integrate and reuse existing heterogeneous components.
9.2.2.1 Adapters for UI Component Technologies

Component adapters allow the bindings from the abstract UI component model to its
concrete (native) component implementations. Adapters enable the communication
between the runtime middleware and the native component implementations in a
particular component technology. Specifically, a component adapter performs the
following functionalities:
• Component location and instantiation: locating the component implementation

through URI, local class name, etc., and then creating an instance of the
component.
• Marshalling events: capturing native component events and exposing them to the
runtime middleware as the corresponding abstract events defined in the
component model.
• Marshalling operation and property calls: allowing the middleware to invoke

abstract operations and properties by executing their corresponding native
counterparts in the component implementation.
• Data type mapping: mapping the component's native data types to and from the
platform-independent data types used in the abstract component model (i.e.,
XML Schema types).
Part 2: Mixup 200
Referring to the YouTube component in Listing 9.1, the "type" attribute of the
<binding> element refers to an URI that allows the runtime middleware to find a
component adapter that can be used to communicate with that particular component
technology, and the "address" attribute specifies the location of the component
implementation which allows the adapter to download and instantiate the component. In
addition, the <event>, <operation>, and <property> elements within a binding also
contain an "address" attribute, which allows the adapter to identify the native event,
operation, or property in the component's implementation (e.g., a JavaScript function, a
Java method or event).
en n
ev atio
ev ratio
op
er
t
en
op
e
t
n
op ven
er n t
er
n
op ve
io
at
e
at
e
io
n
t
Figure 9.4 Component adapters
In general, we expect the majority of existing UI components to be built using

established component technologies (e.g., ActiveX, Java Applet) or AJAX toolkits (e.g.,
Yahoo UI104, Dojo105). Within a given component technology or toolkit, components
expose a uniform notion of events, operations, and properties; as a result, the mapping
from abstract UI events, operations, and properties to their native counterparts can be
usually achieved through meta-language facilities such as reflection. Therefore, once a
104
http://developer.yahoo.com/yui/
105
http://dojotoolkit.org/
Part 2: Mixup 201
component adapter for a specific component technology or toolkit has been built, all
components using the same technology can be integrated into our runtime middleware.
Through appropriate component adapters, the runtime middleware can practically
interface with any component technologies, and therefore has the potential to compose a
wide range of components from a variety of sources (see Figure 9.4).
9.2.2.2 Adapters for Application and Data Services

Many reusable components do not have user interfaces. They contain application or data
logics that are only exposed via appropriate protocols or APIs. Application logic (AL)
components provide desired application functionalities by performing tasks upon
request, whereas data access (DA) components allow data to be accessed and
manipulated. Examples of AL components are web services which expose their
functionalities via protocols such as SOAP and REST. Similarly, RSS and Atom based
feeds can be regarded as DA components that can be accessed via REST.
In order to integrate AL and DA components, we have developed generic component

adapters for web services based on SOAP and REST as well as REST-based RSS feeds.
For web services, the SOAP and REST adapters are responsible for managing
communications with the remote services by generating appropriate client-side
JavaScript proxies from their service descriptions in WSDL. Specifically, each client
proxy includes a dedicated JavaScript stub object with methods corresponding to
respective operations of the web service. As a result, scripts embedded in the event
listeners can directly call these methods, which will in turn invoke the appropriate
operations of the remote service.
These adapters are also responsible for data mapping from XML elements to JavaScript
objects. For example, assuming the "getWeatherInfo" operation of the weather service
mentioned in the last chapter (section 8.3.3) generates the following response:
<WeatherInfo>
<current xsi:type="xsd:int">70</current>
<low xsi:type="xsd:int">50</low>
<high xsi:type="xsd:int">79</high>
</WeatherInfo>
Listing 9.2 XML response from the "getWeatherInfo" operation
Part 2: Mixup 202
And then the SOAP (or REST) adapter will convert the XML message to the following
JavaScript object:
function WeatherInfo() {
this.current = 0;
this.low = 0;
this.high = 0;
}
var xWeatherInfo = new WeatherInfo();
xWeatherInfo.current = 70;
xWeatherInfo.low = 50;
xWeatherInfo.high = 79;
Listing 9.3 Generated JavaScript object for the XML response
This is a rather simple JavaScript object, but in a similar fashion the adapters can create
JavaScript objects with arrays and nested structures from more complex XML messages.
In addition, the adapters will also try to leverage E4X [Sch05] when available, and
thereby entirely avoiding XML parsing.
For DA components, we provide a generic RSS adapter that can be used by UI

components as well as script-based process logic to access and manipulate any RSS
feeds. Specifically, the adapter provides CRUD106 operations that allow items in an RSS
feed to be created, read, updated, and deleted. In addition, advanced operations such as
sorting and simple keyword search are also supported.
The news listing UI component in our New York Times example is in fact built on top
of the RSS adapter. The UI component leverages the RSS adapter to process the news
feed from Yahoo Pipes, renders the feed items in HTML, and finally attaches the
appropriate UI events (i.e., "ItemTagsActivated" and "ItemGeoActivated") to the news
items.
9.2.2.3 Component Wrappers

Adapters for UI components bridge the communication gap between various native
component technologies and our runtime middleware. However, very often, a legacy
component may not directly satisfy the functional requirements of the composite
106
http://en.wikipedia.org/wiki/Create,_read,_update_and_delete
Part 2: Mixup 203
application; it needs to be "wrapped" to be usable as a UI component (i.e., to expose

useful events and properties). Therefore, the middleware supports the notion of
component wrappers, which are hand written on a per-component basis to bridge the
functional requirement gap between legacy components and the composite application.
9.2.3 Runtime Behavior

Figure 9.2 shows the execution of the New York Times composite application at
runtime. The left hand side includes a list of news items from Yahoo Pipes, while the
right hand side contains Google Maps, Flickr, and YouTube. To conserve space, we
only illustrate a single interaction in this example: after the user selects a news item (by
clicking on its title), Flickr and YouTube will show a photo and video related to the
news story (according to its keywords), respectively, and Google Maps will display the
location of the story.
When the user selects the first news item, "U.N. Confirms N. Korea Reactor Shutdown",
the following happens:
1. The news feed component captures this user action, and fires two events: the
"ItemTagsActivated" event, with a parameter containing the keywords, "nuclear
reactor"; and the "ItemGeoActivated" event, with a parameter containing the
geo-code (40,127).
2. The middleware tries to locate listeners matching the events. In this case it finds
two listeners for the event "ItemTagsActivated" and one listener for the event
"ItemGeoActivated".
3. For the listener "keywordToVideo" (Listing 9.4) that matched the event
"ItemTagsActivated"
a. The middleware locates the component (YouTube) and the operation

("displayVideo") referred to by the listener. It then locates an appropriate
component adapter that matches the selected binding. In this case, it finds
a .NET adapter.
Part 2: Mixup 204
b. The middleware dispatches the event to the component by passing the

name of the operation, "displayVideo", and the event parameter "tags"
with the value "nuclear reactor" to the .NET component adapter.
c. The adapter translates the event parameter from XML Schema type (an
xsd:string) to the appropriate native type supported by the component
implementation (a .NET string), and locates the operation's native
address as specified in the binding (the .NET method "PlayVideo").
d. The adapter executes the .NET method "PlayVideo", passing in value for
the "tags" input parameter (i.e., "nuclear reactor").
e. YouTube updates its display to show a video related to "nuclear reactor".
4. The middleware performs similar steps to execute the "keywordToPhoto"

listener (Listing 9.4) that matched the "ItemTagsActivated" event. Flickr will in
turn show an image related to the keywords.
5. The middleware performs similar steps to execute the "geocodeToMap" listener

(Listing 9.4) that matched the "ItemGeoActivated" event. Google Maps will in
turn display the location related to the news item.
The above steps illustrate the integrated UI at work – when the user interacts with one
component, the other related components will change in a synchronized fashion
according to the specifications in the composition model.
9.3 Development Environment

The framework would not be complete without effective development tools. In this
section, we describe the framework's development environment by illustrating how the
New York Times example application can be built.
Specifically, there are four steps in building a composite application:
1. If the desired component descriptors (UISDL documents) are not yet

available, the component developer finds the native UI components that
Part 2: Mixup 205
satisfy the application requirement and creates component descriptors for

them. She then saves the UISDL documents in a component registry.
2. The composer (i.e., composition developer) selects the desired component

descriptors from the registry and specifies component interactions (i.e.,
integration logic) via event listeners.
3. The composer (or web designer) authors an HTML template, specifying

layout for the composite application, including positioning information for
each individual UI components.
4. The composer generates the XPIL document and deploys it together with the
HTML layout template to the runtime environment.
To simplify the development process, we provide a development tool, called Mixup

Editor, which facilitates the creation of both component descriptors and composition
logic. The architecture of the development environment is depicted in Figure 9.5.
s
p tor
cri
es
ntD
e
on
mp
Co
Figure 9.5 Architecture of Mixup's development environment
9.3.1 Creating Component Descriptors

The component developer needs to provide abstract component model descriptions as
well as their bindings to existing native component implementations. However, this step
usually does not require the involvement of the developers who implemented the
original native UI components. As a matter of fact, anyone who is familiar with the
components' native APIs can author the corresponding abstract component models in
Part 2: Mixup 206
UISDL. That means any existing, legacy UI components could be integrated by simply
providing component descriptors in UISDL, as long as the appropriate component
adapters are available.
The XML fragment in Listing 9.1 shows two of the component descriptors for our New
York Times example. The component developer can manually create a UISDL
document for each component in the example, using simple text editors. However, to
simplify the process, Mixup Editor includes a built-in component editor that allows a
component descriptor to be created by specifying its events, operations, properties,
configurations, and bindings to native implementations. This however requires that the
component developer be familiar with the interfaces or APIs of the native UI
components.
Therefore, to further simply the component descriptor creation process, the component
editor includes a component inspector framework, that allows the automatic generation
of component descriptors from native UI components.
If a particular component technology supports abstract descriptions (e.g., WSRP) or

meta-language facilities such as refection (e.g., Java Applet), the component editor may
invoke a suitable component inspector to find out the component's native events,
operations and properties and then generate the component descriptor with the
appropriate bindings to the native implementation (this is similar to automatically
generating a WSDL document from a Java class). Specifically, a component inspector
has the following responsibilities:
• Component inspection: identifying the native addresses of events, operations,

and properties within component implementation, through means such as
reflection. This implies, for example, being able to map an event member in a
.NET class to an abstract UI event and map a method in a Java class to an
abstract UI operation.
• Generating component descriptor based on the inspection.
The component developer may take the automatically generated component descriptor
and directly deposit it into the registry. However, typically not all events, operations and
Part 2: Mixup 207
properties are needed for the integration. The developer may choose to filter them and to
keep only the relevant ones, possibly renamed for better readability.
In cases where component inspectors are unavailable (e.g., due to the lack of meta-
language facilities), the developer can always resort to manually create component
descriptors. For example, she can create an abstract operation and specify its binding by
pointing to the appropriate JavaScript function in an AJAX component.
With the help of the graphical editing tool and component inspectors, the programming
skill required on the component developer is greatly reduced. She does not need to know
the internal details of the native UI components. She just needs to have certain
understanding of the interface / API of the relevant native components in order to
translate them into our abstract component model.
9.3.2 Specifying Composition Logic

Once the desired UI components (and their descriptors) are available, the developer
must specify the composition logic that describes the runtime interactions among the UI
components. The XML fragment in Listing 9.4 contains the composition logic for our
New York Times example.
<component ref="http://.../youtube.uisdl" id="youtube"
binding="ytDotNETBinding">
<config name="devId">N0m3tA9LJfr</config>
</component>
<component ref=".../gmap.uisdl" id="gmap" binding="...">

<property name="zoomLevel">13</property>
......
</component>
<component ref=".../flickr.uisdl" id="flickr" binding="...">

......
</component>
<component ref=".../pipe.uisdl" id="rss" binding="...">

......
</component>
<listener id="keywordToPhoto"
publisher="rss" event="ItemTagsActivated"
Part 2: Mixup 208
<listener id="keywordToVideo"
publisher="rss" event="ItemTagsActivated"
subscriber="youtube" operation="displayVideo"/>
<listener id="geocodeToMap"
publisher="rss" event="ItemGeoActivated"
subscriber="gmap" operation="displayMap"/>
<layout manager="http://www.openxup.org/2006/08/mixup/layout/css2">
......
</layout>
</xpil>
Listing 9.4 Composition model for the New York Times example
The composer can manually create an XPIL document for the composite application,
using simple text editors. However, to simplify the process, Mixup Editor includes a
built-in composition editor that allows the composite application to be created by
specifying its components and their interactions.
First, the composer must select the appropriate components from the registry and places
them in the editing panel (Figure 9.6). She then defines event listeners by drawing
arrows that link events of one component to operations of the subscribing components.
As shown in Figure 9.6, the arrow from Yahoo Pipes (with ID "rss") to Google Maps
(with ID "gmap") links the "ItemGeoActivated" event of Yahoo Pipes to Google Maps'
"displayMap" operation. Similarly, the arrow to Flickr links "ItemTagsActivated" event
to Flickr's "displayPhoto" operation.
If the event parameters and operation inputs do not match (e.g., number of parameters,
data types), the editor will request additional input from the composer, such as XSLT
expressions that can transform the event parameters to operation inputs. If event
automation is insufficient, the composer may also specify additional integration and/or
transformation logics in the form of scripts or references to external code.
Note that the composer is not required to possess programming skills unless additional
transformation or integration logic needs to be specified. In addition, with our graphical
development tools, she does not even need to understand XPIL or UISDL's XML
syntax.
Part 2: Mixup 209
Figure 9.6 Eclipse-based Mixup Editor
Although not shown in this example, Mixup Editor also allows developers to easily
create composite components from existing compositions. When building a composite
component, one could manually translate an XPIL document to a composite UISDL
document. However, Mixup Editor provides this functionality automatically. Given any
XPIL document, the tool can automatically generate a UISDL document containing a
composite component with the same set of component references and event listeners.
9.3.3 Defining Layout

Specifying layout information is quite straight forward for web-based composite
applications. The composer (or a web designer) simply provides an HTML document
that contains a placeholder element (e.g., <div>, <iframe>) for each UI component. This
HTML template works in conjunction with the CSS expressions inside XPIL's <layout>
element to achieve a coherent composite layout. Inside the HTML template, the web
designer may use a combination of CSS and HTML constructs to apply additional style
and layout refinement to the placeholder elements, as well as the overall HTML
template.
Part 2: Mixup 210
This design allows layout specification to be decoupled from composition logic. It also
makes defining layout a simple and familiar task, as the composer is very likely to be
already versed in designing web pages with HTML and CSS. Finally, the HTML
template can be authored with any professional web design tools, which may further
simplify the layout process.
9.3.4 Deployment
Once the composition logic is fully specified, an XPIL document can be generated and
deployed together with the HTML template containing layout information. The
deployment procedure is very straight forward: simply copy the XPIL and HTML
documents into the appropriate directory specified by the runtime middleware (the
referenced UISDL documents can be kept in their original locations). Our development
environment includes a testing runtime, which consists of a browser and a web server
running in the local development machine (Figure 9.5).
9.4 Framework Implementation

As a proof of concept, we have provided an implementation of the proposed framework,
consisting of a browser-based lightweight runtime middleware and two development
tools: one desktop-based, and the other web-based. The prototype allows developers to
create UISDL descriptors from existing native UI components, specify integration logic
among them in a drag and drop fashion, and quickly deploy and execute the resulting
composite application.
9.4.1 Runtime Middleware

Our prototype consists of a lightweight runtime middleware to execute composite
applications and a set of component adapters to communicate with existing UI
components developed by heterogeneous component technologies. There are plenty of
implementation alternatives. We chose the web-based model for our prototype and the
web browser as the integration platform, since this will make resulting composite
applications accessible to the widest audience. More importantly, web browsers provide
Part 2: Mixup 211
built-in support for many component technologies; this allows many existing native UI
components developed in heterogeneous technologies to be easily integrated and reused.
In our current prototype, the runtime middleware is implemented in pure JavaScript that
can be completely executed in the client browser. Specifically, the middleware is
provided as a JavaScript library that can be easily embedded in the HTML layout
template (i.e., referenced in HTML's <script> tag). This allows the framework to
instantiate components, to place them into the placeholders in the HTML layout, and to
set up the necessary listeners when the page is loaded into the client browser.
To execute a composite application, the user simply points her browser to the URL
identifying its HTML layout document, and the browser will automatically load and
execute the JavaScript library that contains the runtime middleware. Once the HTML
template is loaded by the browser, the runtime middleware will perform the following
tasks to initialize the composite application:
1. Using the XMLHttpRequest [Kes08] object, the middleware loads the XPIL
document as referenced in the HTML template document.
2. The middleware retrieves the UISDL descriptor for each UI component

referenced in the XPIL document, again via the XMLHttpRequest object.
3. For each UI component, the middleware locates the appropriate component

adapter as specified in the binding of the component.
4. For each UI component, the middleware instruct the corresponding component

adapter to locate and then download the component's native implementation.
5. Once the native component implementation has been located and downloaded,
the middleware instructs the component adapter to instantiate the component.
6. The middleware proceeds with interpreting the event listener specifications.
7. Finally, the middleware sets up callback functions to capture runtime UI events.

Part 2: Mixup 212
Specifically, for the middleware to support event brokering, it must register a callback
function for each event fired by the component. That is, each event needs a
corresponding callback function. This event registration process requires the
component's adapter to support function registration; that is, the middleware needs to be
able to pass a function pointer or object reference to the native component
implementation. Most component technologies support this; however, for native
components that do not support event registration, the middleware supplies an event
notification JavaScript function which can be called by the components to fire events at
runtime.
After the initial download of the composition (the XPIL document and HTML layout
template) and components (the referenced component descriptors and their native
implementations), all user actions and component interactions happen in the same web
page. Although the components may communicate with some remote business logic
behind the scene, the runtime middleware itself makes no further page navigations or
roundtrips to the server side. This model works very well for rich internet applications
(RIA), where frequent page refreshes are undesirable.
Once the composite application is properly initialized, it's ready for user interactions.
When the end user acts on one of the UI components, the following happens:
1. The native component captures this user action and then fires appropriate
high-level UI event(s) through its component adapter. The actual firing of the
event(s) is achieved by calling the callback function(s) registered by the
middleware.
2. The middleware tries to locate event listeners matching the event(s).
3. For each matching listener, the middleware identifies the component that
subscribes to this event and the operation to be invoked, as specified in the
listener.
4. The middleware then locates the subscribing component's adapter, as

specified in its binding.
Part 2: Mixup 213
5. The middleware dispatches the event to the component by passing the name
of the abstract operation and any event parameters to the component adapter.
6. The adapter translates the event parameters from XML Schema types to the
appropriate native types supported by the component implementation, and
locates the operation's native address as specified in the binding.
7. The middleware instructs the adapter to execute the native method

corresponding to the operation, passing in any converted parameter values.
8. The subscribing component updates its UI appropriately.
9.4.1.1 Scripting Support

Since the final composite application is executed in the browser, any additional
integration logic in the composition model (i.e., inside <listener>) could be specified in
JavaScript, which will be executed by the browser as is during runtime. To allow
JavaScript code to access and manipulate UI components, the middleware creates a
JavaScript wrapper object for each UI component, with member functions and
properties corresponding to the component's abstract operations and properties107. To
facilitate the scripting of process-based logic, we need to formalize the linkage between
JavaScript wrapper objects and UI component instances:
• JavaScript wrapper object. For each UI component, the runtime middleware

creates a JavaScript wrapper object, with the variable name identical to the
component's ID. For example, the JavaScript object "youtube" corresponds to the
YouTube UI component, with ID "youtube".
• JavaScript method. For each abstract component operation, the middleware

creates a JavaScript method inside the wrapper object. For example, the method
"youtube.displayVideo" corresponds to the "displayVideo" operation of the
YouTube component.
107
The JavaScript wrapper methods and properties refer to the component's abstract operations and
properties, not their native counterparts. This ensures that the script-based composition logic is
independent of the native component implementations.
Part 2: Mixup 214
• JavaScript property. For each abstract component property, the middleware

creates a JavaScript property (with getter / setter support) inside the wrapper
object. For example, the JavaScript property "gmap.zoomLevel" corresponds to
the "zoomLevel" property of Google Maps.
• Event and event parameters. The middleware creates a JavaScript object named
"event" for the current event, and attaches a JavaScript property for each event
parameter. For example, "event.title" represents the "title" parameter of the
"ItemGeoActivated" event from the news listing component (Yahoo Pipes).
As a result, developers may write JavaScript code in the composition model (externally
referenced or embedded in <listener>) to access and manipulate UI components, by
referring to their component IDs. Specifically, the JavaScript code may directly invoke
component operations and retrieve property values through the JavaScript wrapper
object.
To provide the user with a responsive UI experience, it is important that the user is not
"blocked" after performing some actions. Therefore, for the event listeners that provide
one-on-one mapping between events and operations, the runtime middleware will try to
invoke the operations asynchronously by default.
Similarly, it is also desirable to invoke operations in scripts in an asynchronous fashion.

Therefore, for each operation, the middleware will attempt to create two JavaScript
wrapper methods: a synchronous method corresponding to the name of the operation,
and an asynchronous method, whose name is the name of the operations appended with
the string "Async".
For example, the "findZip" operation of the address analyzer service mentioned in the
last chapter (section 8.3.3) can be called synchronously as it returns the resulting zip
code right away. On the other hand, since retrieving the weather information (see section
8.3.3) may take some time, the corresponding operation should be invoked
asynchronously, via the JavaScript method "getWeatherInfoAsync".
Part 2: Mixup 215
For asynchronous operation invocations, we need to provide callbacks to receive the

results. To avoid explicitly defining callback functions, we leverage JavaScript's
function closure mechanism, which allows callback functions to be defined inline.
In summary, the JavaScript-based scripting approach integrates nicely with the

declarative composition model. JavaScript is a full featured scripting language with
built-in support for function closure and fault handling, and it is by far the most popular
language for client-side scripting. Since we employ standard ECMAScript108 syntax
which is browser independent, developers do not need to worry about browser
compatibility issues when specifying process-based integration logic.
9.4.1.2 Layout
Since our delivery platform is the web browser, the implementation leverages the
browser's CSS engine as the external layout manager. Composition developers may
specify any valid CSS fragment inside the <layout> element in the composition model,
which will be inserted into the HTML template document as is during runtime. The CSS
fragment may refer to the component IDs as defined in the XPIL document, since the
HTML layout elements corresponding to those components have the same ID values. Of
course, the composition developers may also directly apply CSS styles to the layout
elements in the HTML template document.
9.4.1.3 Component Adapters

Component adapters are the key to the proposed UI integration framework, as they allow
existing native UI components developed in heterogeneous languages and technologies
to be integrated and reused in the same composite application. With browsers' built-in
support for most popular components technologies (e.g., ActiveX, Java Applet, Flash),
component adapters are relatively easy to implement. As a result, we have implemented
component adapters for several major component technologies as well as a few popular
AJAX toolkits.
108
http://en.wikipedia.org/wiki/ECMAScript
Part 2: Mixup 216
For example, we have implemented a .NET component adapter, which can be used to
integrate any .NET components, including the Flickr image displayer and the YouTube
video front-end in our New York Times example. Similarly, for the park listing
component in the national park example, we implemented a SACK adapter, which will
work with any AJAX components built with the SACK toolkit.
As mentioned in the last section, the middleware relies on component adapters to

instantiate component instances. During runtime, the component adapters generate
appropriate HTML markups and insert them into the HTML layout placeholder
elements. For non-markup based components (e.g., ActiveX controls, Java applets), an
HTML <object> element is generated for each component, which will be in turned
interpreted by the browser to create the appropriate native objects. For markup-based
components, their HTML markups will be inserted into the HTML layout elements as is.
To communicate with native UI components and to map abstract events, operations, and
properties to their native counterparts, component adapters typically rely on meta-
language facilities such as reflection. Without meta-language facilities, a component
adapter can still leverage standard conventions for event registration and callbacks in a
particular component technology or toolkit. Essentially, component adapters can be
created as long as the native UI components exhibit a uniform notion of operations (i.e.,
direct function or method calls) and events (i.e., callback functions).
Finally, communication with AL components such as web services is achieved through

two AL component adapters: a SOAP adapter and a REST adapter. Implementing the
adapters is actually quite straight forward, since reference implementations for
JavaScript-based WSDL parsers and proxy generators are readily available. To provide
users with a responsive UI experience, the adapters allow remote operations to be
invoked asynchronously. Specifically, the adapters leverage the asynchronous facility in
browser's XMLHttpRequest [Kes08] object to download and parse WSDL documents
and to invoke operations from remote web services.
Part 2: Mixup 217
9.4.1.4 Performance and Scalability

In our prototype, each composite application has its own instance of the runtime
middleware, due to the fact that each application is executed in a single instance of the
browser. This implies that the middleware is only executing a single composite
application at a time. In addition, a composite application typically consists of a limited
number of UI components, as too many visual components would in fact overwhelm the
user. As a result, the middleware only needs to manage a limited number of UI
components. Therefore, we believe runtime scalability of the middleware is not a major
concern in our browser-based approach.
On the performance side, our goal is to minimize the time for the composite application
to startup and the time to handle each user interaction. The startup time is the time
difference between when the user first visited the URL of the composite application and
when the middleware finishes loading components and becomes ready to handle user
interactions. The startup time is therefore affected by three tasks: the downloading of the
XPIL / UISDL documents and native component implementations, the parsing and
interpretation of the downloaded XML documents, and the instantiation of the native
components.
When the XPIL / UISDL documents and native component binaries are located in
remote servers with slow connections, the perceived startup time of the composite
application could be long. In this case, we suggest the composition developer make local
copies of the documents and binaries (if copyright allows) and deploy them on a server
with faster connection. The instantiation time of the native components is determined by
the OS and browser. However, in our experience, such as in the New York Times
example, components are usually instantiated without perceivable delays by the end
users.
Our main concern is with the performance of XML parsing, as it is well-known that
parsing large amount of XML data in JavaScript is one of the bottlenecks in browser
performance. Fortunately, our XML data size is relatively small. First, as we said earlier,
the number of UI components in a composite application is usually small (e.g., less than
ten), as too many visual components would in fact overwhelm the end user. Second, the
Part 2: Mixup 218
file sizes of UISDL documents are typically small, as they only contain descriptions of
operations, events, etc. For example, the component descriptors in our New York Times
example range from 1Kbytes to 2Kbytes, containing 20 to 70 lines of XML. Finally, the
XPIL document is also fairly small. For instance, the XPIL file that describes the New
York Times application is approximately 5Kbytes, containing about 100 lines of XML.
We now proceed with measuring the startup time of the New York Times application
and the Nation Park Guide application we discussed in previous chapters. Table 9.1
shows the start time for the two applications. The numbers are purely middleware
execution time, including parsing and interpreting XPIL / UISDL documents and setting
up event listeners and callback functions, but excluding documents and components
download time and native component instantiation time. The total application startup
time varies greatly, depending on network condition and the file sizes of native
component implementations. For our example applications deployed in local servers, the
total startup time is never more than one second.
Table 9.1 Application startup performance
# of Total size of Size of XPIL Middleware

components UISDL documents document execution time
New York 4 5.44Kbytes 5.15Kbytes 78ms
Times
National Park 3 4.17Kbytes 2.78Kbytes 47ms
Guide
The prototype leverages XML-for-<SCRIPT>109 for XML parsing. To maximize parsing

performance, we use its SAX parser rather than DOM parser. As shown in the Table 9.1,
the middleware performs quite well. Note that XML-for-<SCRIPT> uses standard
JavaScript to interpret XML, and therefore runs in all major browsers. To further
improve performance, we could leverage native XML parsers such MSXML110 in IE.
However, this will limit our implementation to a specific browser. We decided not to
pursue this route as the current performance level is already more than adequate.
109
http://xmljs.sourceforge.net/
Part 2: Mixup 219
The second factor in the performance of the runtime middleware is the time it takes to
handle user interactions. Therefore, to improve the responsiveness of composite
applications, the middleware will try to invoke component operations asynchronously
whenever possible. This allows the user to continue interacting with the application
while a particular UI component is performing lengthy computations.
Since the middleware is implemented in JavaScript which has no built-in threading

support, the middleware must rely on component adapters to support asynchronous
invocations. Fortunately, most component technologies (e.g., ActiveX, Java Applet)
have built-in asynchronous functionality, and AJAX-based components naturally
support asynchronous invocations. Even for native components that only support
synchronous operations, end users should rarely experience perceivable delays since
most UI operations are executed quickly.
During a user interaction, the middleware's is responsible for identifying the event
listeners matching the event, executing data transformations (if specified), and then
instructs the appropriate component adapters to asynchronously invoke the operations
specified in the listeners. In our example applications, this process never takes more
than 20 milliseconds (and less than 10ms without data transformations). As a result, the
middleware execution time spent in handling user interactions does not affect the end
user experience. In fact, it is practically negligible comparing to the execution time of
native component operations.
Finally, we observed that consuming web services directly in browsers (via the SOAP
and REST adapters) had marginal impact on the overall performance. This is mainly due
to the fact that XML messages resulting from operation invocations are typically small
and low frequency (i.e., triggered by user's actions), and that they are processed
asynchronously (via XMLHttpRequest). Comparing to network transmission time, the
XML processing time is practically negligible during an operation invocation. As a
result, delays are rarely perceived when interacting with composite applications that
contain UI components mixed with web services.
110
http://msdn.microsoft.com/en-us/library/ms763742.aspx
Part 2: Mixup 220
9.4.2 Development Tools

To facilitate the rapid development of composite applications, our prototype provides
two complementary development tools, one desktop-based aimed at developers, and the
other web-based aimed at tech-savvy end users, both of which allow the creation of
component descriptors and the specification of composition logic in a drag and drop
fashion.
We have already illustrated the desktop version in previous sections (see Figure 9.6). It
is implemented as an Eclipse plug-in, based on Eclipse's Graphical Editing
Framework111 (GEF). We chose Eclipse GEF because it allows easy rendering,
modeling, and manipulation of graphical objects, which is ideal for our drag and drop
style of graphical composition. Comparing to the web version, the desktop-based
implementation is richer in feature and provides a more efficient user interface;
however, unlike the web version, it must be installed prior to use.
As a result, the prototype also includes a web-based version of the development tool to
facilitate end-user composition. The web version leverages AJAX techniques to provide
a similar drag and drop experience. As shown in Figure 9.7, the AJAX-based editor
allows developers to graphically create XPIL compositions by dragging and dropping
components (on the left) onto the grid (on the right). The XPIL tab in the lower part of
the editor allows the designer to check the XML equivalent of the graphical composition
model. Two other tabs allow the editing of the HTML template with the placeholder
elements (HTML) and the drag-and-drop placement of the UI components into the
respective placeholders (Layout). Finally, the Preview tab provides the developers with
an instant preview function which allows composite applications to be tested without
first storing the UISDL and XPIL files on the server. Completed compositions can be
stored on the web server for later retrieval and editing. A composite application can thus
be executed by simply accessing the unique URL identifying it.
111
http://www.eclipse.org/gef/
Part 2: Mixup 221
Figure 9.7 AJAX-based Mixup Editor
The AJAX-based editor is based on the OpenjACOB112 AJAX library and communicates
with a backend Java servlet for file management. Opening an existing XPIL
composition requires accessing the respective XPIL file and the referenced UISDL
descriptors on the web server via the XMLHttpRequest object. Storing a new or edited
composition is performed via the dedicated file storage servlet on the web server. The
servlet acts as a gateway for the web server's file system and allows the on-the-fly
editing of composite applications; that is, modified compositions are immediately
available online once they have been stored.
9.4.2.1 Component Inspectors

The prototype provides component inspectors for several major component technologies
as well as a few AJAX toolkits. The inspectors can automatically generate UISDL
112
http://www.draw2d.org/draw2d/
Part 2: Mixup 222
descriptors by inspecting native component implementations. Rather than mapping

abstract events and operations to the native implementation (as in the case of component
adapters), inspectors leverage meta-language mechanisms and standard programming
conventions to map native events and methods to abstract events and operations. Due to
their symmetric nature, component inspectors and adapters are usually implemented in
pairs, since they share the same meta-language mechanism for a particular component
technology.
9.4.2.2 Developer Productivity

To improve developer's productivity and to reach a wide range of users, we have
provided two development tools, one for programmers, and the other for tech-savvy web
users. Our tools offer a drag and drop user interface that's on par with popular mashup
tools such as Yahoo Pipes.
Developing a composite application involves creating a UISDL descriptor for each

native UI component and an XPIL document to describe the composition logic. As we
have shown in the examples, UISDL documents are typically quite small and consist of
less than 100 lines of XML, whereas the XPIL document is only slightly larger, with a
small amount of JavaScript for additional integration logic (if needed). These documents
can be created in the development tools and fine-tuned by hand if necessary. Since they
are XML files, the tools can easily parse back hand edited documents.
To measure the effectiveness of the development tools, the author invited seven of his
colleagues113 to participate in a short user study. Four of them are security software
engineers, with some exposure to web-based software development; but none of them
are familiar with UI development. The remaining three consist of marketing and sales
staff, who can be categorized as tech-savvy web users.
The seven users were given a one hour training and tutorial ahead of time. The four
engineers were then asked to develop the National Park Guide and New York Times
applications using the Eclipse-based tool; whereas the three business users were asked
113
All of them work for the author's employer, Martsoft Corporation.
Part 2: Mixup 223
to create the two applications using the AJAX-based development tool114. In addition,
the engineers were instructed to create UISDL documents from native UI components115
using the appropriate component inspectors and then manually fine-tune them (e.g.,
removing unwanted operations); whereas the business users were provided with the
finished UISDL documents, as we do not expect business users to understand
component details.
Table 9.2 User study: composition development time
Nation Park National Park New York New York

Guide Guide Times Times
(engineers) (business users) (engineers) (business users)
Create UISDL 12 N/A 5 N/A
via inspectors
Fine-tune 18 N/A 8 N/A
UISDL
Define 55 95 28 52
composition
logic
Testing 30 42 20 33
Total 115 137 61 85
Table 9.2 records the average time (in minutes) the engineers and business users needed
to finish each step of the development process. In general, the engineers are slightly
faster than the business users due to their technical skills. The development time on the
second application (New York Times) is shorter than the first application (National Park
Guide), despite that the second application is actually more complex than the first one.
This is due to the fact that users became more familiar with the tools after finishing the
first application, and that the two applications share some of the UI components.
114
The development tools were already installed before the test.
115
Note that for this user study the native component implementations are readily available.
Otherwise, implementing native UI components could take a variable amount of time, depending
on their complexity. However, as an integration framework, we are not concerned about detailed
component implementations. Our focus is at the composition level and our goal is to integrate and
reuse existing UI components implemented in heterogeneous technologies.
Part 2: Mixup 224
Through this experiment, we believe that the tools are quite effective in building
composite applications from reusable UI components. Comparing to manually gluing
components together using ad-hoc JavaScript, our models and tools may save
developers a considerable amount of time.
9.5 Summary
In this chapter we have presented our UI integration framework, Mixup, which builds
upon the proposed abstract UI component model and event-based composition model.
The framework consists of two complementary parts, a lightweight runtime middleware
and a graphical development environment to facilitate the rapid creation and deployment
of UI-rich composite web applications.
The runtime middleware provides an event broker that coordinates the interactions
among UI components at runtime. In addition, it leverages the web browser as the
integration platform, since browsers have a broad support for many popular component
technologies. Consequently, component adapters can effectively communicate with
existing native UI components developed in heterogeneous technologies, thus allowing
those native components to be seamlessly integrated into the same composite
application.
To facilitate rapid development of composite web applications, the framework provides

a graphical development environment, consists of a web-based online editor and a
desktop-based offline editor. The web-based editor is based on AJAX, and allows tech-
savvy end users to create composite applications in a drag and drop fashion, without
installing any software. The desktop version is based on Eclipse GEF; it is more suitable
for developers, since it is richer in feature and provides a more efficient user interface.
Finally, the development environment also provides a set of component inspectors,
which allow UISDL descriptors to be automatically generated by inspecting
corresponding native UI components.
225
Chapter 10. Conclusions
10. Conclusions
In this chapter, we summarize the contributions of this dissertation and discuss some
future research directions for rich internet applications and UI integration.
10.1 Summary
The development of user interfaces has always been a crucial yet time-consuming part
of the overall application development process. For developers, this task has become
even more challenging due to web users' increasing demand for rich web UI.
Hence the goal of this thesis is to facilitate and simply the development of web
application with sophisticated user interfaces, while at the same time offering end users
a rich and highly interactive user experience. To achieve this goal, we have presented an
RIA framework to assist the execution and development of highly interactive web
applications, and a UI integration framework to facilitate the development of composite
web applications by reusing existing native UI components developed in heterogeneous
languages or platforms.
10.1.1 OpenXUP
To simply the development of highly interactive user interfaces and to provide a rich
web UI experience to end users, we have proposed an RIA framework, OpenXUP
[YB05][Yu06b][Yu06c], which consists of the following components.
UI transport protocol. The foundation of the framework is the Extensible User

Interface Protocol (XUP), an XML-based high-level protocol for communicating
asynchronous events and incremental user interface updates on the web.
With XUP, user actions result in UI events, which are sent as requests to the server side
for processing. To provide end users with a more responsive UI, XUP allows event
requests to be delivered asynchronously, so end users may continue interacting with the
application while the framework is performing computation behind the scene. After
processing the event requests, the server sends back a response containing necessary UI
226
updates. Since the UI updates are incremental, not one full page at a time, end users will
no longer experience frequent slow page refreshes.
Server-side runtime and development environment. Similar to traditional web

applications, OpenXUP offers a server-side runtime environment, which allows UI logic
and behavior to be programmed at the server side. This facilitates centralized application
management, without the hassles of client-side software maintenance. However, unlike
traditional runtimes (e.g., JSP, ASP) which return a full HTML page in each response,
our runtime keeps track of UI changes, and only returns UI deltas in each response. This
is achieved by maintaining a server-side UI model that corresponds to the exact user
interface perceived by the end user.
To facilitate rapid application development, OpenXUP provides a set of server-side

event-driven APIs, which enables developers to implement sophisticated application-
specific UI behavior. OpenXUP APIs are designed to be familiar, closely resembling the
APIs from desktop GUI toolkits. This minimizes the learning curves and allows
developers to quickly migrate their existing desktop-based applications to OpenXUP.
Thin client. To mitigate the security risks associated with executing downloaded code,
OpenXUP employees a thin client design. Similar to browsers, the thin client renders UI
and processes native events, but leaves application-specific logic to the server side. Like
other rich client technologies, our thin client takes advantage of the desktop computing
power to enable a rich and interactive user experience for end users. It fully leverages
the rich UI capability offered by native desktop GUI toolkits such as Windows Forms,
while at the same time maintains a small footprint.
Since no application code is executed on the client side, the security risks associated
with executing downloaded code, whether binary or script, are avoided all together. This
ensures that end users will enjoy a rich UI experience in a safe client environment.
Implementation. To validate our approach, we have implemented a prototype of the

proposed RIA framework, consisting of a full implementation of the XUP protocol, a
.NET-based server-side runtime environment, a set of .NET-based server-side event-
227
driven APIs, and a thin client that can be deployed either as a standalone application or
as an Internet Explorer plugin.
We have also created a lightweight UI modeling language, SUL, in order to build some
sample applications to test the concept. However, since OpenXUP is completely
independent of the actual UI model, any XML-based UI languages can be used in the
framework with appropriate language plugin modules. Finally, our prototype fully
leverages existing industry standards, such as SOAP and XML.
10.1.2 Mixup
To further simply the development of web applications with complex user interfaces, we
have proposed a UI integration framework, Mixup [Yu07b][Yu07c], which aims at the
development of composite applications by reusing native UI components developed in
heterogeneous component technologies. The framework consists of the following
ingredients.
UI component model. We have presented an abstract UI component model to represent

the presentation front-ends of existing web applications or modules. The model allows
UI components to be programmatically manipulated by exposing their events,
operations, and properties. Adding a programmable interface to UI components
essentially allows us to model UI as services.
The component model is abstract and therefore not tied to any specific implementation
technologies. As a result, it can be used to describe existing native UI components
developed with heterogeneous component technologies. This reinforces the concept of
UI as services, where an abstract UI component may have multiple bindings to native
To describe UI components, we have created the UI Service Description Language

(UISDL), whose design follows closely to WSDL. UISDL allows a UI component to be
exposed as a service by describing both the abstract component model and its bindings
to concrete implementations. Specifically, a UISDL document contains the abstract
events, operations, and properties of a UI component, as well as their bindings in the
native component implementations.
228
UI composition model. Since an application's user interface is driven by events

resulting from user actions, UI integration is naturally event-driven. Therefore, we have
proposed an event-based composition model for integrating components at the
presentation layer. To facilitate inter-component communications, the composition
model allows event subscription information to be defined in the form of event listeners,
where each listener maps an event from one component to an operation of another
component.
To describe the UI composition model, we have proposed the eXtensible Presentation

Integration Language (XPIL), which is a lightweight XML-based language for modeling
composite applications. Since UI integration logic is primarily event-based, XPIL has a
simple declarative syntax. This makes it easy to author and interpret.
Due to the diverse nature of UI components, frequently several UI components in the

same composite application may overlap in content or functionality. Therefore, to
ensure a coherent presentation of the composite application, the UI composition model
includes a content consolidation mechanism to merge, propagate, or hide semantically
identical UI content from different UI components. In addition, it is often necessary to
embed one component inside another component; for example, a real estate application
may require the placement of a small thumbnail image of the selected property on a map
showing the property location. As a result, the composition model also supports a
component embedding mechanism that allows one component to be embedded into
another, where the two components may not be aware of each other a priori.
Runtime middleware and development tools. We have provided a lightweight

runtime middleware to execute composite web applications. For each composite
application, the middleware interprets the XPIL document containing the composition
logic and the UISDL documents representing the involved UI components. At runtime,
the middleware facilitates the interactions among the components by capturing events
fired by one component and dispatch them to operations of the subscribing components.
In order to support native UI components developed in heterogeneous technologies, the

runtime middleware supports the notion of component adapters, which allow the
middleware to communicate with components from a variety of sources. Essentially, the
229
component adapters facilitate the communication between the middleware and native UI
components and allow the binding from the abstract component model to its native
To facilitate rapid development of composite web applications, we have provided a

comprehensive development environment to assist the creation of component
descriptors (i.e., UISDL documents) and composition logic (i.e., XPIL document) in a
drag-and-drop style. The development environment also includes the notion of
component inspectors, which allow the automatic generation of component descriptors
from native UI components. Specifically, a component inspector leverages the
appropriate meta-language facilities to find out a legacy component's native events,
operations and properties and then generates a component descriptor with the
appropriate bindings to the native component implementation.
Implementation. To validate our approach, we have provided an implementation of the

proposed framework, consisting of a browser-based lightweight runtime middleware
and a graphical development environment. The prototype allows developers to create
UISDL descriptors from existing native UI components, specify integration logic among
them in a drag and drop fashion, and quickly deploy and execute the resulting composite
applications.
The runtime middleware is packaged as a lightweight JavaScript library that can be

executed in standard web browsers. It is responsible for instantiating UI components
defined in UISDL documents and coordinating their interactions according to the
composition logic specified in the XPIL document.
The development environment includes a desktop-based visual editor and a web-based

online editor. The desktop version is based on Eclipse GEF and allows developers to
efficiently create component descriptors and composition logic in a drag and drop
fashion. The web version is AJAX-based. It serves the same purpose but caters to a
different audience. Since the AJAX-based editor can be executed in the browser, it is
more suitable for tech-savvy end users who demand quick and easy access, whereas the
Eclipse version is more ideal for developers who require maximum features and
efficiency.
230
10.2 Future Work

Both OpenXUP and Mixup can be improved and extended in numerous ways. In this
section we discuss several possible future extensions to our work.
For OpenXUP, we are considering adding client-side support for mobile devices such as
cell phones and PDAs, since they have become quite powerful and have a large screen
real estate these days. Similarly, embedded devices such as shopping kiosks may also
benefit from rich UI. The goal is to allow web applications to provide a rich UI
experience to regular PC users as well as mobile and embedded device users.
Our initial target will be Windows Mobile116 based PDAs and cell phones, and Windows
XP Embedded117 based embedded devices. Since both platforms support .NET, porting
the thin client should not be a significant effort. The challenge here is in supporting
multi-modal interactions; that is, instead of keyboard and mouse, the thin client needs to
deal with touch screen and hand writing. Although the multiple implementations of the
thin client should be fully interoperable with the .NET-based the server (due to the
common protocol – XUP), the server and the server-side applications need to handle
client-side capability differences. For example, a single XUP application may need to
deal with clients that support different UI languages, or different features within the
same UI language.
Another interesting idea is to facilitate the configuration of embedded devices such as

routers and firewalls. Currently, these headless devices either have proprietary desktop
configuration tools or simple HTML-based configuration interfaces. Desktop
configuration tools require extra installation, so most users prefer HTML-based
configuration interfaces. As a result, these devices usually have a mini web server with
some CGI118 code that produces HTML markups. Due to their limited resources, these
devices often cannot generate HTML and JavaScript code that is compatible across
browsers, and the generated UI is usually low in usability. As a result, we are exploring
116
http://en.wikipedia.org/wiki/Windows_Mobile
117
http://www.microsoft.com/windowsembedded/en-us/products/wexpe/default.mspx
118
http://en.wikipedia.org/wiki/Common_Gateway_Interface
231
the possibility of running a lightweight OpenXUP server inside these devices,

supporting a simple UI language such as SUL. This would allow users to perform device
configurations by simply pointing XUPClient to the device's configuration IP address.
For Mixup, our future work will focus on the refinement of the proposed component and
composition models. For example, since a component may contain different visual parts
(e.g., a text listing and an image), we are investigating how to embed just part of the
component, rather than the entire component, into another component. The result is that
a component may embed itself in one or more components, offering a more flexible UI
experience.
To allow a wider range of components to be integrated, we plan to provide support for

generic data services in addition to REST and RSS based components (e.g., leveraging
data access technologies such as XQuery [Boa07] and SQL). In addition, more
component adapters will be added for AJAX-based toolkits, so as to enable the easy
creation of web mashups from a variety of existing native UI components built with
different component technologies and AJAX toolkits.
Finally, we intend to study the relationship between our event-based composition model
and the hyperlink-based navigation model of traditional web applications, and the
consequence of mixing the two models together in the same composite application. In
addition, we will investigate how existing model-driven web development methods
(such as WebML [Cer00]) may assist the design and implementation of reusable
components. These efforts will eventually lead to a proper platform for the development
and execution of user-specified compositions.
233
Publications
Publications
1. Jin Yu, Boualem Benatallah, Fabio Casati, Florian Daniel, and Regis Saint-Paul.
Hybrid Integration for Web Applications. Submitted to ICSE 2009.
2. Jin Yu, Boualem Benatallah, Fabio Casati, and Florian Daniel. Mashups, SaaS, and
Cloud Computing: Evolutions and Revolutions in the Integration Landscape.
Seminar proposal. In Proceedings of the 25th International Conference on Data
Engineering (ICDE 2009). Shanghai, China. March 2009.
3. Jin Yu, Boualem Benatallah, Fabio Casati, and Florian Daniel. Understanding
Mashup Development. IEEE Internet Computing. September/October 2008.
4. Jin Yu, Boualem Benatallah, Fabio Casati, Florian Daniel, Maristella Matera, and
Regis Saint-Paul. Mixup: a Development and Runtime Environment for Integration
at the Presentation Layer. In Proceedings of the 7th International Conference on
Web Engineering (ICWE 2007). Como, Italy. July 2007.
5. Jin Yu, Boualem Benatallah, Fabio Casati, Florian Daniel, Maristella Matera, and
Regis Saint-Paul. A Framework for Rapid Integration of Presentation Components.
In Proceedings of the 16th International World Wide Web Conference (WWW
2007). Banff, Canada. May 2007.
6. Florian Daniel, Jin Yu, Boualem Benatallah, Fabio Casati, Maristella Matera, and
Regis Saint-Paul. Understanding UI Integration: A Survey of Problems,
Technologies, and Opportunities. IEEE Internet Computing. May/June 2007.
7. Jin Yu, Boualem Benatallah, Fabio Casati, and Regis Saint-Paul. XUPClient – a
Thin Client for Rich Internet Applications. In Proceedings of the 7th International
Conference on Web Information Systems Engineering (WISE 2006). Wuhan, China.
October 2006.
8. Jin Yu, Boualem Benatallah, Fabio Casati, and Regis Saint-Paul. OpenXUP – an
Alternative Approach to Developing Highly Interactive Web Applications. In
234
Publications
Proceedings of the 6th International Conference on Web Engineering (ICWE 2006).

Palo Alto, California. July 2006.
9. Jin Yu and Boualem Benatallah. XUP – a SOAP-based Simple User Interface

Protocol. In Proceedings of the 11th Australasian World Wide Web Conference
(AusWeb 2005). Gold Coast, Australia. July 2005.
10. Jin Yu and Jun Chen. XUP – Extensible User Interface Protocol. W3C Note. May
2002. <http://www.w3.org/TR/xup/>
Project Web Site

http://www.openxup.org/
235
Bibliography
Bibliography
[Abd03] Alejandro Abdelnur, et al. Java Portlet 1.0 Specification (JSR 168). October
2003. <http://jcp.org/en/jsr/detail?id=168>
[Abr99] Marc Abrams, et al. UIML: An Appliance-Independent XML User Interface

Language. In Proc. of the 8th International World Wide Web Conference. Toronto,
Canada. May 1999.
[Ado93] Adobe Systems Incorporated. Display PostScript System – Introduction:

Perspective for Software Developers. April 1993. <http://partners.adobe.com
/public/developer/en/ps/sdk/DPS.refmanuals.INTRO.pdf>
[Alo04] Gustavo Alonso, et al. Web Services: Concepts, Architectures, and

Applications. Springer Verlag. 2004.
[Alt07] Mehmet Altinel, et al. Damia – A Data Mashup Fabric for Intranet Applications.
In Proc. of the 33rd International Conference on Very Large Data Bases (VLDB
2007). Vienna, Austria. September 2007.
[And03] Tony Andrews, et al. Business Process Execution Language for Web Services
Version 1.1 (BPEL4WS). May 2003.
<http://www.ibm.com/developerworks/library/specification/ws-bpel/>
[App95] Apple Computer Inc. OpenDoc Programmer's Guide for the MAC OS.
Addison-Wesley. 1995.
[Bai88] Wayne A. Bailey, et al. Effects of Interface Design upon User Productivity. In
Proc. of the 1988 SIGCHI Conference on Human Factors in Computing Systems.
Washington, D.C., USA. 1988.
[BV03] Betsy Beier and Misha W. Vaughan. The Bull’s-Eye: A Framework for Web
Application User Interface Design Guidelines. In Proc. of the 2003 SIGCHI
Conference on Human Factors in Computing Systems. Ft. Lauderdale, Florida,
USA. April 2003.
236
Bibliography
[Bis06] Judith Bishop. Multi-platform User Interface Construction – a Challenge for

Software Engineering-in-the-Small. In Proc. of the 28th International Conference on
Software Engineering (ICSE 2006). Shanghai, China. May 2006.
[Boa07] Scott Boag, et al. XQuery 1.0: An XML Query Language. W3C
Recommendation. January 2007. <http://www.w3.org/TR/xquery/>
[Boo06] Matthias Book. An Instant Message-Driven User Interface Framework for Thin
Client Applications. In Proc. of 21st IEEE/ACM International Conference on
Automated Software Engineering (ASE 2006). Tokyo, Japan. September 2006.
[Bos98] Bert Bos, et al. Cascading Style Sheets Level 2 Specification. W3C
Recommendation. May 1998. <http://www.w3.org/TR/CSS2/>
[Boy07] John M. Boyer, et al. XForms 1.0 (Third Edition). W3C Recommendation.
October 2007. <http://www.w3.org/TR/xforms/>
[Bra08] Daniele Braga, et al. Mashing Up Search Services. IEEE Internet Computing,
Vol. 12(5), pp. 16–23. September/October, 2008.
[Bra06] Tim Bray, et al. Extensible Markup Language (XML) 1.0 (Fourth Edition).
W3C Recommendation. August 2006. <http://www.w3.org/TR/REC-xml>
[Bri02] Gregory Brill. Codenotes for Web Based UI. Random House. January 2002.
[Bur87] Steve Burbeck. Applications Programming in Smalltalk-80: How to use Model-

View-Controller. 1987. <http://st-www.cs.uiuc.edu/users/smarch/st-docs/mvc.html>
[Car05] Richard Cardone, et al. Using XForms to Simplify Web Programming. In Proc.
of the 14th International World Wide Web Conference (WWW 2005). Chiba, Japan.
May 2005.
[Cas06] Paul Castro, et al. Before-Commit Client State Management Services for AJAX
Applications. In Proc. of the 1st IEEE Workshop on Hot Topics in Web Systems and
Technologies (HotWeb 2006). Boston, Massachusetts, USA. November 2006.
237
Bibliography
[Cer00] Stefano Ceri, et al. Web Modeling Language (WebML): a modeling language
for designing Web sites. In Proc. of the 9th International World Wide Web
Conference (WWW 2000). Amsterdam, the Netherlands. May 2000.
[Cer02] Stefano Ceri, et al. Designing Data-Intensive Web Applications. Morgan

Kauffmann. December 2002.
[Cet07a] Semih Cetin, et al. A Mashup-Based Strategy for Migration to Service-

Oriented Computing. In Proc. of the 2007 IEEE International Conference on
Pervasive Services (ICPS 2007). Istanbul, Turkey. July 2007.
[Cet07b] Semih Cetin, et al. Legacy Migration to Service-Oriented Computing with

Mashups. In Proc. of the 2nd International Conference on Software Engineering
Advances (ICSEA 2007). Cap Esterel, French Riviera, France. August 2007.
[Cla99a] James Clark and Steve DeRose. XML Path Language (XPath) Version 1.0.
W3C Recommendation. November 1999. <http://www.w3.org/TR/xpath>
[Chi07a] Roberto Chinnici, et al. Web Services Description Language (WSDL) Version
2.0 Part 1: Core Language. W3C Recommendation. June 2007.
<http://www.w3.org/TR/wsdl20/>
[Chi07b] Roberto Chinnici, et al. Web Services Description Language (WSDL) Version
2.0 Part 2: Adjuncts. W3C Recommendation. June 2007.
<http://www.w3.org/TR/wsdl20-adjuncts/>
[Chr01] Erik Christensen, et al. Web Services Description Language (WSDL) 1.1. W3C
Note. March 2001. <http://www.w3.org/TR/wsdl>
[Cla99b] James Clark. XSL Transformations (XSLT) Version 1.0. W3C

Recommendation. November 1999. <www.w3.org/TR/xslt>
[Coe03] Christophe Coenraets. An overview of MXML: The Flex markup language.

November 2003. <http://www.adobe.com/devnet/flex/articles/paradigm.html>
238
Bibliography
[Cow93] D. D. Cowan, et al. CIRL/PIWI: A GUI Toolkit Supporting Retargetability.

Software Practice and Experience, Vol. 23(5), pp. 511–527. May 1993.
[Cru07] Ariadne A. Cruz, et al. Development of a Robust and Flexible Weblab

Framework based on AJAX and Design Patterns. In Proc. of the 7th IEEE
International Symposium on Cluster Computing and the Grid (CCGrid 2007). Rio
de Janeiro, Brazil. May 2007.
[Cur07] Kevin Curran. Geographical Mashups, review of Google Maps Hacks by Rich
Gibson and Schuyler Erle. IEEE Distributed Systems Online, Vol. 8(4). 2007.
[Dan07] Florian Daniel, et al. Understanding UI Integration: A Survey of Problems,

Technologies, and Opportunities. IEEE Internet Computing, Vol. 11(3), pp. 59–66.
May/June, 2007.
[DP05] Oscar Díaz and Iñaki Paz. Turning Web Applications into Portlets: Raising the
Issues. In Proc. of the 2005 Symposium on Applications and the Internet (SAINT
2005). Trento, Italy. January 2005.
[Eis07] Marc Eisenstadt. Does Elearning Have To Be So Awful? (Time to Mashup or

Shutup). In Proc. of the 7th IEEE International Conference on Advanced Learning
Technologies (ICALT 2007). Niigata, Japan. July 2007.
[Eis01] Jacob Eisenstein, et al. Applying Model-Based Techniques to the Development

of UIs for Mobile Computers. In Proc. of the 2001 International Conference on
Intelligent User Interfaces (IUI 2001). Santa Fe, New Mexico, USA. January 2001.
[Eug03] Patrick Th. Eugster, et al. The many faces of publish/subscribe. ACM
Computing Surveys, Vol. 35(2), pp. 114– 31. June 2003.
[Fie00] Roy Thomas Fielding. Architectural Styles and the Design of Network-based
Software Architectures. PhD dissertation. University of California, Irvine. 2000.
<http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm>
239
Bibliography
[Fje07] Tore Fjellheim, et al. A process-based methodology for designing event-based

mobile composite applications. Data and Knowledge Engineering, Vol. 61(1), pp.
6–22. April 2007.
[Fla08] Mike Flasko. Using Microsoft ADO.NET Data Services. August 2008.
<http://msdn.microsoft.com/en-us/library/cc907912.aspx>
[FV04] Murielle Florins and Jean Vanderdonckt. Graceful Degradation of User

Interfaces as a Design Method for Multiplatform Systems. In Proc. of the 2004
International Conference on Intelligent User Interfaces (IUI 2004). Madeira,
Funchal, Portugal. January 2004.
[Flo07] Ingbert R. Floyd, et al. Web Mash-ups and Patchwork Prototyping: User-driven
technological innovation with Web 2.0 and Open Source Software. In Proc. of the
40th Hawaii International Conference on System Sciences (HICSS 2007). Waikoloa,
Big Island, Hawaii, USA. January 2007.
[Fuj04] Jun Fujima, et al. Clip, Connect, Clone: Combining Application Elements to
Build Custom Interfaces for Information Access. In Proc. of the 17th Annual ACM
Symposium on User Interface Software and Technology (UIST 2004). Santa Fe, New
Mexico, USA. October 2004.
[GW04] Krzysztof Gajos and Daniel S. Weld. SUPPLE: Automatically Generating User
Interfaces. In Proc. of the 2004 International Conference on Intelligent User
Interfaces (IUI 2004). Madeira, Funchal, Portugal. January 2004.
[GMS87] Hector Garcia-Molina, Kenneth Salem. Sagas. ACM SIGMOD Record, Vol.
16(3), pp. 249-259. December 1987.
[Gar02] Jesse James Garrett. The Elements of User Experience: User-Centered Design
for the Web. New Riders Press. October 2002.
[Gar05] Jesse James Garrett. Ajax: A New Approach to Web Applications. February
2005. <http://www.adaptivepath.com/publications/essays/archives/000385.php>
240
Bibliography
[GE06] David Gerard and Efitu. Image:X client sever example.svg. November 2006.
<http://en.wikipedia.org/wiki/Image:X_client_sever_example.svg>
[Goo01] Ben Goodger, et al. XML User Interface Language (XUL) 1.0. 2001.
<http://www.mozilla.org/projects/xul/xul.html>
[Gos90] James Gosling, et al. NeWS Book: An Introduction to the Network/Extensible

Window System. Springer Verlag. March 1990.
[GH01] John Grundy and John Hosking. Developing Adaptable User Interfaces for
Component-based Systems. Interacting with Computers, Vol. 14(3), pp. 175–194.
2001.
[GY03] John Grundy and Biao Yang. An environment for developing adaptive, multi-
device user interfaces. In Proc. of the 4th Australasian User Interface Conference
(AUIC 2003). Adelaide, Australia. February 2003.
[GZ02] John Grundy and Wenjing Zou. Building multi-device, adaptive thin-client web
user interfaces with Extended Java Server Pages. Cross-Platform and Multi-device
User Interfaces. Wiley, 2002.
[Gud07a] Marin Gudgin, et al. SOAP Version 1.2 Part 1: Messaging Framework
(Second Edition). W3C Recommendation. April 2007.
<http://www.w3.org/TR/soap12-part1/>
[Gud07b] Marin Gudgin, et al. SOAP Version 1.2 Part 2: Adjuncts (Second Edition).
W3C Recommendation. April 2007. <http://www.w3.org/TR/soap12-part2/>
[Guo08] Rui Guo, et al. CompoWeb: A Component-Oriented Web Architecture. In

Proc. of the 17th international conference on World Wide Web (WWW 2008).
Beijing, China. April 2008.
[Hal05] Alon Y. Halevy, et al. Enterprise information integration: successes, challenges

and controversies. In Proc. of the 2005 ACM SIGMOD International Conference on
Management of Data (SIGMOD 2005). Baltimore, Maryland, USA. June 2005.
241
Bibliography
[HI06a] Noriko Hanakawa and Nao Ikemiya. A new web browser including a
transferable function to Ajax codes. In Proc. of 21st IEEE/ACM International
Conference on Automated Software Engineering (ASE 2006). Tokyo, Japan.
September 2006.
[HI06b] Noriko Hanakawa and Nao Ikemiya. A web browser for Ajax approach with
asynchronous communication model. In Proc. of the 2006 IEEE/WIC/ACM
International Conference on Web Intelligence (WI 2006). Hong Kong, China.
December 2006.
[Hep08] Stefan Hepper. Java Portlet 2.0 Specification (JSR 286). June 2008.
<http://jcp.org/en/jsr/detail?id=286>
[Hil04] David Hill, et al. Smart Client Architecture and Design Guide. June 2004.
<http://msdn.microsoft.com/en-us/library/ms998506.aspx>
[Hor00] Arnaud Le Hors, et al. Document Object Model (DOM) Level 2 Core
Specification. W3C Recommendation. November 2000.
<http://www.w3.org/TR/DOM-Level-2-Core/>
[How01] Rob Howard. Web Services with ASP.NET. February 2001.

[IETF06] IETF. Widex (Widget Description Exchange Service). 2006.

<http://www.softarmor.com/widex/>
[IT03] Kimihito Ito and Yuzuru Tanaka. A Visual Environment for Dynamic Web
Application Composition. In Proc. of the 14th Conference on Hypertext and
Hypermedia (HT 2003). Nottingham, U.K. August 2003.
[Jel04] Rick Jelliffe. Knobs and Trees. July 2004.

<http://blogs.oreilly.com/digitalmedia/2004/07/knobs-and-trees.html>
[Jhi06] Anant Jhingran. Enterprise Information Mashups: Integrating Information,

Simply. In Proc. of the 32nd International Conference on Very Large Data Bases
(VLDB 2006). Seoul, Korea. September 2006.
242
Bibliography
[Kes08] Anne van Kesteren. The XMLHttpRequest Object. W3C Working Draft. April
2008. <http://www.w3.org/TR/XMLHttpRequest/>
[Kha07] Nawaz Khan, et al. From Local Laboratory Data to Public Domain Database in
Search of Indirect Association of Diseases: AJAX Based Gene Data Search Engine.
In Proc. of the 20th IEEE International Symposium on Computer-Based Medical
Systems (CBMS 2007). Maribor, Slovenia. June 2007.
[Kon07] Fumikazu Konishi, et al. RABC: A Conceptual Design of Pervasive

Infrastructure for Browser Computing based on Ajax technologies. In Proc. of the
7th IEEE International Symposium on Cluster Computing and the Grid (CCGrid
2007). Rio de Janeiro, Brazil. May 2007.
[Kro03] Alan Kropp, et al. Web Services for Remote Portlets Specification 1.0. OASIS
Standard. August 2003. <http://www.oasis-
open.org/committees/download.php/3343/oasis-200304-wsrp-specification-1.0.pdf>
[LD07] Lihui Lei and Zhenhua Duan. Integrating AJAX and Web Services for
Cooperative Image Editing. IT Professional, Vol. 9(3), pp. 25–29. May/June 2007.
[Len02] Maurizio Lenzerini. Data integration: a theoretical perspective. In Proc. of the

21st ACM Symposium on Principles of Database Systems (PODS 2002). Madison,
Wisconsin, USA. June 2002.
[Lev96] C. H. Levy, et al. IUP/LED: A Portable User Interface Development Tool.

Software Practice and Experience, Vol. 26(7), pp. 737–762. July 1996.
[Liu07] Xuanzhe Liu, et al. Towards Service Composition Based on Mashup. In Proc.
of the 2007 IEEE Congress on Services (SERVICES 2007). Salt Lake City, Utah,
USA. July 2007.
[Lok02] Simon Lok, et al. A Graphical User Interface Toolkit Approach to Thin-Client
Computing. In Proc. of the 11th International World Wide Web Conference (WWW
2002). Honolulu, Hawaii, USA. May 2002.
243
Bibliography
[Lu05] Yi-Hsuan Lu, et al. Pollock: Automatic Generation of Virtual Web Services
from Web Sites. In Proc. of the 2005 ACM Symposium on Applied Computing (SAC
2005). Santa Fe, New Mexico, USA. March 2005.
[Man05] Ioana Manolescu, et al. Model-Driven Design and Deployment of Service-

Enabled Web Applications. ACM Transactions on Internet Technology, Vol. 5(3),
pp. 439–479. August 2005.
[Max07] E. Michael Maximilien, et al. A Domain-Specific Language for Web APIs and
Services Mashups. In Proc. of the 5th International Conference on Service-Oriented
Computing (ICSOC 2007). Vienna, Austria. September 2007.
[Max08] E. Michael Maximilien, et al. An Online Platform for Web APIs and Service
Mashups. IEEE Internet Computing, Vol. 12(5), pp. 32–43. September/October,
2008.
[Mer06] Duane Merrill. Mashups: The new breed of Web app. August 2006.
<http://www.ibm.com/developerworks/library/x-mashups.html>
[MD07a] Ali Mesbah and Arie van Deursen. An Architectural Style for Ajax. In Proc.
of the 6th Working IEEE/IFIP Conference on Software Architecture (WICSA 2007).
Mumbai, India. January 2007.
[MD07b] Ali Mesbah and Arie van Deursen. Migrating Multi-page Web Applications to
Single-page AJAX Interfaces. In Proc. of the 11th European Conference on
Software Maintenance and Reengineering (CSMR 2007). Amsterdam, the
Netherlands. March 2007.
[MS05] Microsoft Corporation. Smart Client - Composite UI Application Block.

December 2005. <http://msdn.microsoft.com/en-us/library/aa480450.aspx>
[MS07a] Microsoft Corporation. Understanding the Remote Desktop Protocol (RDP).

March 2007. <http://support.microsoft.com/kb/186607>
[MS07b] Microsoft Corporation. XAML Overview. November 2007.

244
Bibliography
[Mor06] Maria Angeles Moraga, et al. Ontology Driven Definition of a Portlet

Functionality Model. In Proc. of the 10th IEEE International Enterprise Distributed
Object Computing Conference (EDOC 2006). Hong Kong, China. October 2006.
[MS04] Sabrina Silva de Moura and Daniel Schwabe. Interface Development for
Hypermedia Applications in the Semantic Web. In Proc. of the 2nd Latin American
Web Congress (LA-WEB 2004). Ribeirao Preto-SP, Brazil. October 2004.
[Mye92] Brad Myers and Mary Beth Rosson. Survey on user interface programming. In
Proc. of the 1992 SIGCHI Conference on Human Factors in Computing Systems
(SIGCHI 1992). Monterey, California, USA. May 1992.
[Mye00] Brad Myers, et al. Past, Present, and Future of User Interface Software Tools.
ACM Transactions on Computer-Human Interaction, Vol. 7(1), pp. 3–28. March
2000.
[Nad06] Anthony Nadalin, et al. Web Services Security: SOAP Message Security 1.1
(WS-Security 2004). OASIS Standard Specification. February 2006.
<http://www.oasis-open.org/committees/download.php/16790/wss-v1.1-spec-os-
SOAPMessageSecurity.pdf>
[Nic04] Jeffrey Nichols, et al. Improving Automatic Interface Generation with Smart
Templates. In Proc. of the 2004 International Conference on Intelligent User
Interfaces (IUI 2004). Madeira, Funchal, Portugal. January 2004.
[Oba03] Dare Obasanjo. XML Serialization in the .NET Framework. January 2003.
[OH01] Piet Obermeyer and Jonathan Hawkins. Microsoft .NET Remoting: A Technical
Overview. July 2001. <http://msdn.microsoft.com/en-us/library/ms973857.aspx>
[OMG08] Object Management Group. Common Object Request Broker Architecture

(CORBA) 3.1. January 2008. <http://www.omg.org/spec/CORBA/3.1/>
245
Bibliography
[Oka01] Wataru Okada, et al. Applying CC/PP to User’s Environmental Information for
Web Service Customization. In Proc. of the 10th International World Wide Web
Conference (WWW 2001). Hong Kong, China. May 2001.
[Ols00] Dan R. Olsen Jr. Cross-modal Interaction using XWeb. In Proc. of the 13th
Annual ACM Symposium on User Interface Software and Technology (UIST 2000).
San Diego, California, USA. November 2000.
[ORe05] Tim O'Reilly. What Is Web 2.0 – Design Patterns and Business Models for the
Next Generation of Software. September 2005.
<http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-
20.html>
[Pad07] Srinivas Padmanabhuni, et al. Coupling RDF/RSS, WSRP and AJAX for
Dynamic Reusable Portlets: An Approach and a Use Case. In Proc. of the 2007
IEEE International Conference on Services Computing (SCC 2007). Salt Lake City,
Utah, USA. July 2007.
[Pau05] Linda Dailey Paulson. Building Rich Web Applications with Ajax. IEEE
Computer, Vol. 38(10), pp. 14–17. October 2005.
[Pem00] Steven Pemberton, et al. XHTML 1.0: The Extensible HyperText Markup
Language – A Reformulation of HTML 4 in XML 1.0 (Second Edition). W3C
Recommendation. January 2000. <http://www.w3.org/TR/xhtml1/>
[Pix00] Tom Pixley. Document Object Model (DOM) Level 2 Events Specification.
W3C Recommendation. November 2000. <http://www.w3.org/TR/DOM-Level-2-
Events/>
[Pon01] Shankar R. Ponnekanti, et al. ICrafter: A Service Framework for Ubiquitous

Computing Environments. In Proc. of Ubicomp 2001. Atlanta, Georgia, USA.
September 2001.
[Ray99] Eric S. Raymond. The Cathedral and the Bazaar. O'Reilly Media. 1999.
<http://www.catb.org/esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s04.html>
246
Bibliography
[Ree79] Trygve M. H. Reenskaug. Models-Views-Controllers. Xerox PARC Technical

Note. December 1979. <http://heim.ifi.uio.no/~trygver/themes/mvc/mvc-
index.html>
[Ria08] Anton V. Riabov, et al. Wishful Search: Interactive Composition of Data

Mashups. In Proc. of the 17th international conference on World Wide Web (WWW
2008). Beijing, China. April 2008.
[Ros08] Florian Rosenberg, et al. Composing RESTful Services and Collaborative

Workflows: A Lightweight Approach. IEEE Internet Computing, Vol. 12(5), pp.
24–31. September/October, 2008.
[RR06] James F. Ryan and Blair L. Reid. Usable Encryption Enabled by AJAX. In
Proc. of the International Conference on Networking and Services (ICNS 2006).
Santa Clara, California, USA. July 2006.
[Sab07] Marwan Sabbouh, et al. Web Mashup Scripting Language. In Proc. of the 16th
International World Wide Web Conference (WWW 2007). Banff, Canada. May 2007.
[Sam07] Hani Samir, et al. Swing2Script: Migration of Java-Swing applications to Ajax

Web applications. In Proc. of the 14th Working Conference on Reverse Engineering
(WCRE 2007). Vancouver, British Columbia, Canada. October 2007.
[Say06] Ahmet Sayar, et al. Integrating AJAX Approach into GIS Visualization Web
Services. In Proc. of AICT/ICIW 2006. Guadeloupe, French Caribbean. February
2006.
[SA07] Nicolás Serrano and Juan Pablo Aroztegi. Ajax Frameworks for Interactive Web
Apps. IEEE Software, Vol. 24(5), pp. 12–14. September/October 2007.
[SV07] Douglas C. Schmidt and Steve Vinoski. Object Interconnections: Dynamic

CORBA, Part 1: The Dynamic Invocation Interface. 2007.
<http://microsites.cmp.com/documents/s=9063/cujcexp2007vinoski/>
247
Bibliography
[Sch05] John Schneider, et al. ECMAScript for XML (E4X) Specification. December
2005. <http://www.ecma-international.org/publications/files/ ECMA-ST/ECMA-
357.pdf>
[She01] Paul D. Sheriff. Introduction to ASP.NET and Web Forms. November 2001.
[She07] Amit P. Sheth, et al. SA-REST: Semantically Interoperable and Easier-to-Use

Services and Mashups. IEEE Internet Computing, Vol. 11(6), pp. 91–94.
November/December 2007.
[Sil00] Paulo Pinheiro da Silva. User Interface Declarative Models and Development
Environments: A Survey. In Proc. of the 7th International Workshop on Design,
Specification and Verification of Interactive Systems (DSV-IS 2000). Limerick,
Ireland. June 2000.
[Sil08] Sanjit Sil. Understanding Script Injection Attacks. May 2008.

<http://aspalliance.com/1648_Understanding_Script_Injection_Attacks.all>
[Smi06] Keith Smith. Simplifying Ajax-Style Web Development. IEEE Computer, Vol.
39(5), pp. 98–101. May 2006.
[Son06] Michael Sonntag. Ajax Security in Groupware. In Proc. of the 32nd

EUROMICRO Conference on Software Engineering and Advanced Applications
(EUROMICRO-SEAA 2006). Cavtat/Dubrovnik, Croatia. August 2006.
[Spo07] Anselm Spoerri. Visual Mashup of Text and Media Search Results. In Proc. of
the 11th International Conference Information Visualization (IV 2007). Zurich,
Switzerland. July 2007.
[Ste03] Johnny Stenback, et al. Document Object Model (DOM) Level 2 HTML
Specification. W3C Recommendation. January 2003. <http://www.w3.org/TR/DOM-
Level-2-HTML/>
[Sun97] Sun Microsystems. Java AWT: Delegation Event Model. February 1997.
<http://java.sun.com/j2se/1.3/docs/guide/awt/designspec/events.html>
248
Bibliography
[Tho08] Rich Thompson. Web Services for Remote Portlets Specification 2.0. OASIS
Standard. April 2008. <http://docs.oasis-open.org/wsrp/v2/wsrp-2.0-spec.html>
[Tre08] Jonathan Trevor. Doing the Mobile Mash. IEEE Computer, Vol. 41(2), pp.
104–106. February 2008.
[Tru07] Salvador Trujillo, et al. Feature Oriented Model Driven Development: A Case
Study for Portlets. In Proc. of the 29th International Conference on Software
Engineering (ICSE 2007). Minneapolis, Minnesota, USA. May 2007.
[Tur02] Volker Turau. A framework for automatic generation of web-based data entry
applications based on XML. In Proc. of the 2002 ACM Symposium on Applied
Computing (SAC 2002). Madrid, Spain. March 2002.
[VC04] Chris Vandervelpen and Karin Coninx. Towards Model-Based Design Support
for Distributed User Interfaces. In Proc. of NordiCHI 2004. Tampere, Finland.
October 2004.
[W3C06] W3C. Rich Web Clients Activity. 2006. <http://www.w3.org/2006/rwc/>
[Wal05] Stephen Walther. Introducing the ASP.NET 2.0 Web Parts Framework. March
2005. <http://msdn.microsoft.com/en-us/library/ms379628.aspx>
[WP05] Roy Want and Trevor Pering. System Challenges for Ubiquitous & Pervasive
Computing. In Proc. of the 27th International Conference on Software Engineering
(ICSE 2005). St. Louis, Missouri, USA. May 2005.
[WAP01] WAP Forum. Wireless Markup Language Version 2.0. September 2001.
[Wat07] Stephen Watt. Mashups – The evolution of the SOA, Part 2: Situational
applications and the mashup ecosystem. November 2007.
<http://www.ibm.com/developerworks/webservices/library/ws-soa-mashups2/>
[Wei08] Coach Wei. A Study of Ajax Performance Issues. January 2008.

<http://www.coachwei.com/blog/_archives/2008/1/22/3480119.html>
249
Bibliography
[Wes07] Donavon West. Build Your Own Windows Vista Sidebar Gadget. August
2007. <http://msdn.microsoft.com/en-us/magazine/cc163370.aspx>
[Woo07] Jo Wood, et al. Interactive Visual Exploration of a Large Spatio-Temporal

Dataset: Reflections on a Geovisualization Mashup. IEEE Transactions on
Visualization and Computer Graphics, Vol. 13(6), pp. 1176–1183.
November/December 2007.
[WW07] Quincy Wu and Yan-Hsiang Wang. Building Web-base SIP Analyzer with
Ajax Approach. In Proc. of the 21st International Conference on Advanced
Information Networking and Applications Workshops (AINAW 2007). Niagara Falls,
Canada. May 2007.
[Xie06] Qing Xie. Developing Cost-Effective Model-Based Techniques for GUI

Testing. In Proc. of the 28th International Conference on Software Engineering
(ICSE 2006). Shanghai, China. May 2006.
[Yin06] Hao Yin, et al. Providing Portlet-Based Client Access to CIMA-Enabled

Crystallographic Instruments, Sensors, and Data. In Proc. of the 7th IEEE/ACM
International Conference on Grid Computing (Grid 2006). Barcelona, Spain.
September 2006.
[YC02] Jin Yu and Jun Chen. XUP – Extensible User Interface Protocol. W3C Note.
May 2002. <http://www.w3.org/TR/xup/>
[YB05] Jin Yu and Boualem Benatallah. XUP - a SOAP-based Simple User Interface
Protocol. In Proc. of the 11th Australasian World Wide Web Conference (AusWeb
2005). Gold Coast, Australia. July 2005.
[Yu06a] Jin Yu. XUP – Extensible User Interface Protocol 2.0. January 2006.
<http://www.openxup.org/TR/xup/>
[Yu06b] Jin Yu, et al. OpenXUP – an Alternative Approach to Developing Highly

Interactive Web Applications. In Proc. of the 6th International Conference on Web
Engineering (ICWE 2006). Palo Alto, California. July 2006.
250
Bibliography
[Yu06c] Jin Yu, et al. XUPClient – a Thin Client for Rich Internet Applications. In
Proc. of the 7th International Conference on Web Information Systems Engineering
(WISE 2006). Wuhan, China. October 2006.
[Yu07a] Jin Yu. SUL – Simple User Interface Language 1.0. May 2007.
<http://www.openxup.org/TR/sul.pdf>
[Yu07b] Jin Yu, et al. A Framework for Rapid Integration of Presentation Components.
In Proc. of the 16th International World Wide Web Conference (WWW 2007). Banff,
Canada. May 2007.
[Yu07c] Jin Yu, et al. Mixup: a Development and Runtime Environment for Integration
at the Presentation Layer. In Proc. of the 7th International Conference on Web
Engineering (ICWE 2007). Como, Italy. July 2007.
[Yu08] Jin Yu, et al. Understanding Mashup Development. IEEE Internet Computing,
Vol. 12(5), pp. 44–52. September/October, 2008.
[Zha07] Hao Zhang, et al. Asynchronously Rendering Interoperable Remote Portlets. In

Proc. of the 2007 IEEE International Conference on Web Services (ICWS 2007).
Salt Lake City, Utah, USA. July 2007.
[Zho07] Luo Zhong, et al. A Design for an Online RSS Reader Based On AJAX. In
Proc. of the 8th ACIS International Conference on Software Engineering, Artificial
Intelligence, Networking, and Parallel/Distributed Computing (SNPD2007).
Qingdao, China. July 2007.
[Zyp08] Kristopher William Zyp. Ajax performance analysis. April 2008.

<http://www.ibm.com/developerworks/web/library/wa-aj-perform/>

Jyu Thesis

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Jyu Thesis

Diunggah oleh

Hak Cipta:

Format Tersedia

A UI-driven Approach to Facilitating

Effective Development of Rich and

A dissertation submitted in fulfillment of the requirements for the degree of

School of Computer Science and Engineering

University of New South Wales

Sydney, NSW 2052, Australia

Supervisor: Prof. Boualem Benatallah

October 31, 2008

October 31, 2008

In addition, I would like to thank my research collaborators Regis Saint-Paul, Florian

The foundation of our RIA framework is an XML-based high-level protocol for

The proposed UI integration framework includes an abstract UI component model

Composite applications are executed via a lightweight runtime middleware, which

1.1 Requirements and Challenges ...........................................................................2

1.1.1 Developing Rich Internet Application ......................................................3

1.1.2 UI Integration ............................................................................................4

1.2 Contributions Overview....................................................................................6

1.2.1 Developing Rich Internet Application ......................................................7

1.2.2 UI Integration ............................................................................................9

1.3 Thesis Organization ........................................................................................12

Part 1: OpenXUP – an RIA Framework .........................................................................15

2. Background and State of Art in RIA.......................................................................17

2.1 Dimensions for Characterizing RIA Technologies .........................................17

2.1.1 Usability and Productivity.......................................................................17

2.1.2 UI Code Location ....................................................................................19

2.1.3 Asynchronous UI.....................................................................................20

2.1.4 Development Environment .....................................................................20

2.2 RIA Technologies ...........................................................................................21

2.2.1 Classic HTML.........................................................................................21

2.2.2 Rich Client Technologies........................................................................22

2.2.3 Server-side Approaches ..........................................................................28

2.2.4 AJAX ......................................................................................................31

2.2.5 Remote Display Technologies ................................................................33

2.2.6 Rich UI Languages..................................................................................36

2.2.7 Comparisons ...........................................................................................36

2.3 Summary .........................................................................................................38

3. UI Transport Protocol .............................................................................................39

3.1 User Interface Model.......................................................................................40

3.2 Protocol Design...............................................................................................42

3.2.2 Protocol Operations.................................................................................43

3.2.3 Network Event Delivery .........................................................................46

3.2.4 Server-side Notification ..........................................................................48

3.3.1 Example 1 ...............................................................................................49

3.3.2 Example 2 ...............................................................................................51

3.4 Summary .........................................................................................................54

4. Developing and Executing RIAs.............................................................................57

4.1 Example Scenario ...........................................................................................58

4.2 Runtime Framework .......................................................................................60

4.2.1 OpenXUP Architecture ...........................................................................61

4.2.2 MVC Pattern in OpenXUP .....................................................................63

4.2.3 Location of UI and Application Code .....................................................64

4.2.4 Runtime Behavior ...................................................................................65

4.2.5 Server-side Asynchronous Notifications.................................................67

4.3 Development Environment .............................................................................67

4.3.1 Application Development APIs ..............................................................67

4.3.3 Multiple UI Languages............................................................................76

4.3.4 Asynchronous Event Handling ...............................................................77

4.3.5 Multi-threading .......................................................................................78

4.3.6 Application Development .......................................................................80

4.4 Implementation ...............................................................................................84

4.4.1 Protocol Module......................................................................................85

4.4.2 OpenXUP Server.....................................................................................86

4.4.4 SUL .........................................................................................................88