EPAM SYSTEMS

UI Automation/White/CodedUI.
common problems & tricks

Dmitry Hes, Tsimafei Avilin, EPAM Sys.

4/6/2015

Summary: The page describes some of the problems in QA Automation that occur
when using UIAutomation (CodedUI, White) to automated desktop applications
and methods for their solution on the example of the project developed with
DevExpress library.
 At the beginning was UI Automation
Using Automation UI
Using White framework.
Using Visual Studio 2010 Coded UI.
o Working with White
Some tricks to automate unsupported controls.
o UI Automation properties
o Using patterns, on SelectionPatternIdentifiers example
o DevExpress.Xpf.Core.ClearAutomationEventsHelper.IsEnabled
property
o When simple way ends
Use LegacyIAccessible
Using COM-wrapper
Use Win32 API
Call Invoke()
Use BoundingRectangle
Data caching

Summary: The page describes some of the problems in QA Automation that occur
when using UIAutomation (CodedUI, White) to automated desktop applications
and methods for their solution on the example of the project developed with
DevExpress library.

At the beginning was UI Automation

Using Automation UI

This framework provides access to user interface elements Win32, Windows

Forms or WPF applications with the possibility to get or set the element property,
emulate user inputs (the official documentation is available here). Note that it is
base for some popular frameworks such as CodedUI and White.

In the UI Automation UI elements are represented as a tree structure of

AutomationElement objects with desktop as a root. The searching of element are
carried out through the tree by conditions specify in PropertyCondition. Some
controls implement patterns (e.g., SelectionItemPattern, SelectionPatternIdentifiers
the pattern for ListBox, CheckBox, GroupBox).

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 1

Control patterns provide a way to categorize and expose a control's functionality
independent of the control type or the appearance of the control.

UI Automation uses control patterns to represent common control behaviors. For

example, the Invoke control pattern uses for controls that can be invoked (such as
buttons) and the Scroll control pattern for controls that have scroll bars (such as list
boxes, list views, or combo boxes). Because each control pattern represents a
separate functionality, their combination describes the full set of functionality
supported by a particular control.

Some advantages of UI Automation:

- free;

- support Win32, WinForms, WPF;

- included in .Net framework.

Drawback:

- the framework works well standard UI controls, but to support custom and
complex controls its necessary to make additional implementations;

- a lot of code for controls searching and processing.

Using White framework.

Based on the UI Automation this framework has convenient access to the controls
and their properties. He has the same features as UI Automation. The framework
can be used to test the Win32, WPF, WinForm, SWT and Silverlight applications.
White framework allows you to write tests simpler and more readable.

You can use a ready BDDfy framework (read here), which allows to easily
implement BDD testing with White.

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 2

Using Visual Studio 2010 Coded UI.

Coded UI Microsofts solution based of the same UI Automation, appeared in

Visual Studio 2010 (you can read here). Under this link you can find useful article
how to use AutomationPeer to automate WPF/Silverlight custom controls.

Profits:

- recorder is available, recording user actions for autogenerating tests;

- Visual Studio in-box;

- support from Microsoft, integration in TFS;

- frequently recurring test steps can easily be re-used. Its up to you how fine-
grained you want to record your test steps. Calling a step (consisting of one or
more actions) is just a matter of a single function call;

- generates C# and XML code by default;

- features fuzzy matching of UI elements. This seems to make the tests more
robust when updating the user interface;

- you can test many different kinds of user interfaces, not just the web.

Shortcomings:

- the same as that UI Automation;

- proprietary software: Coded UI available in more expensive Visual studio

editions: Ultimate, Premium;

- test steps are stored in an XML file (called UIMap) which in turn compiles to
C# code. The files is big and clunky and there currently is no editor or
documentation for it. So if you want to make some changes, things can get
complicated unless you want to re-record an entire test step.

- creating very simple assertions (such as look for the string foo on this web
page) is a bit clumsy and requires too many mouse clicks. An IE accelerator
would be great!

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 3

- while fuzzy matching works great in most cases, it can get in the way in
others.

Based on the above choice was made in favor of the White framework, since it is
free and makes writing tests easier and makes them look intuitive.

Working with White

It is easy to start working with the White for people who has not did. Even it does
not have such a good guide to use as MSDN, but for beginning White has enough
resources here http://teststack.azurewebsites.net/white/index.html.

To find the required control by AutomatedId property its enough to execute Get
method parameterized of control type, and as a function parameter to specify the
desired AutomatedId. For example, for Edit field with AutomatedId = "txtCity":

TextBox button = window.Get< TextBox>("txtCity");

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 4

To set other search criteria, such as text, you can use the SearchCriteria class.
They can be combined with the methods And(). So to find the button by the text
"OK":

Button button = window.Get<Button>(SearchCriteria.ByText("OK"));

To find the controls in the application, as well as getting its properties are useful
Inspectors like as Inspect (download - Inspect.exe) supplied with MS win8 SDK,
or ACorns.Hawkeye (https://hawkeye.codeplex.com/). Moreover, Hawkeye shows
the actual type of the object, as well as an object implementation library.

Some tricks to automate unsupported controls.

It was noted that White is a wrapper of UI Automation, and thus it implements the
same basic controls. However, the application under test uses set of custom
controls implemented by e.g. DevExpress library. Some troubles in processing
such controls and their solutions were found out.

UI Automation properties

The first trouble is that White does not process custom DateEdit field. The
framework does not treat it neither like any DateTimePicker, nor Edit. As the
DataGrid contains MaskBox field, to get the control values you can get the value
of control through UI Automation properties:

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 5

 //Get value from whiteElement.AutomationElement
private string GetDate( AutomationElement element)

AutomationElement dateEdit = element.FindFirst(

TreeScope.Children, new PropertyCondition(
AutomationElement.ControlTypeProperty, ControlType.Edit));

object valuePattern = null;

dateEdit.TryGetCurrentPattern( ValuePattern.Pattern, out valuePattern);

return ((ValuePattern)valuePattern).Current.Value;

To set values use typing emulation:

//Type text @dateString into @element

private void SetDataToField( AutomationElement element, string dateString)

AutomationElement dateEdit
= element.FindFirst(TreeScope.Children, new
PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Edit));

dateEdit.SetFocus();

System.Windows.Forms.SendKeys.SendWait(dateString);

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 6

Using patterns, on SelectionPatternIdentifiers example

The next problem is the processing RadioGroup - getting the value of the selected
item. Applying White isSelected property for element RadioButton, always return
False value.

So UI Automation is useful. The implementation of custom function helps us. This

function refers to the parental control (RadioGroup) and takes the value of the
selected item, using the pattern SelectionPatternIdentifiers.

public bool isSelected()

string selectedElement =
GetSelectedElementName( _element.AutomationElement);

return selectedElement.ToLower().Equals(_element.Name.ToLower());

//Gets selected element name

private string GetSelectedElementName(AutomationElement element)

AutomationElement parentElement = Utils.Util.GetParentElement(element);

var selection = parentElement.GetCurrentPropertyValue(

SelectionPatternIdentifiers.SelectionProperty);

AutomationElement[] selectedElements = selection as AutomationElement[];

if (selectedElements.Length == 0) { return ""; }

return selectedElements[0].Current.Name;

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 7

DevExpress.Xpf.Core.ClearAutomationEventsHelper.IsEnabled property

ComboBox controls refer to DevExpress XtraEditors.LookUpEdit class library and

represent as an Edit field combined with a Button, clicking on which opens a drop-
down list. However, the list is a UI Automation Window element, is a direct
descendant (child) of the main form.

In addition, when working with a library DevExpress a problem with dynamically

created controls or dynamically fills tables was spotted. UI Automation does not
see and does not find such controls and child elements of the table even searching

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 8

methods in TreeScope. The solution to this problem is setting to disable static
cleaning event queue property in the application under test:

DevExpress.Xpf.Core.ClearAutomationEventsHelper.IsEnabled = false

However, this may reduce the performance of the application (more info here).

When simple way ends

White is not so quite good for processing Tables. Especially when tables using a
complex structure as a set of data in the collapsing combined group, nested tables,
any controls instead of text in cells.

After reading variety of websites and forums, solutions for the processing of
custom controls were found out.

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 9

Use LegacyIAccessible

To get cell values use UI Automation. To identify the type of line (collapsing
group element / data line) read the value LegacyIAccessible.State. To do this, you
should use IUIAutomationLegacyIAccessiblePattern, which provides the interface
ILegacyIAccessibleProvider. However, LegacyIAccessible pattern is not available
for control based on UI Automation, because it is not implemented neither UI
Automation nor White

if ((
bool) child.GetCurrentPropertyValue(AutomationElementIdentifiers.IsLegacyIAccessiblePatternAvailableProperty))
{ var pattern = ((LegacyIAccessiblePattern) child.GetCurrentPattern(LegacyIAccessiblePattern.Pattern));
var state = pattern.GetIAccessible().accState;

// Do something with state...

Using COM-wrapper

Using COM-wrapper library UIAComWrapper.dll distributed with Windows Kits

is one of the way to solution. It have implementation for Inspect.exe of
ILegacyIAccessibleProvider for getting LegacyIAccessible properties.

Use Win32 API

In addition, you can use Win32 API to send window messages and thus get
control properties or invoke events.

Here is an example to get DevExpress panel Caption.

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 10

using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Automation;
using System.Windows.Automation.Text;

[System.Runtime.InteropServices.DllImport("user32.dll", EntryPoint =
"SendMessage", CharSet = System.Runtime.InteropServices.CharSet.Auto)]

private static extern bool SendMessage(IntPtr hWnd, uint Msg, int wParam, StringBuilder
lParam);

[System.Runtime.InteropServices.DllImport("user32.dll", SetLastError = true)]

private static extern IntPtr SendMessage(int hWnd, int Msg, int wparam, int lparam);

//Get Text property of UI element using Win32 API

public static string GetText(AutomationElement element)

const int WM_GETTEXT = 0x000D;

const int WM_GETTEXTLENGTH = 0x000E;
//Get handle property of the control
int wndHandle =
(int)element.GetCurrentPropertyValue(AutomationElement.NativeWindowHandleProperty);
StringBuilder title = new StringBuilder();
// Get the size of the string required to hold the window title.
Int32 size = SendMessage(wndHandle, WM_GETTEXTLENGTH, 0, 0).ToInt32();
// If the return is 0, there is no title.
if (size > 0)
{
title = new StringBuilder(size + 1);
SendMessage( new IntPtr(wndHandle), ( int)WM_GETTEXT, title.Capacity, title);
}
return title.ToString();

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 11

Call Invoke()

Furthermore you can call Invoke() method to get control properties. This method
implements InvokePattern, which provides support operation and invoking
unambiguous action of controls that do not maintain state by activation.

Use BoundingRectangle

Worst case to use BoundingRectangle property to get control layout. Then use
mouse clicks via SendInput expanding/collapsing/ the table.

Data caching

For processing controls which consist volume data (lists, tables, trees), as well as
getting properties and control patterns in one operation it is necessary to use data
caching. It allows improving performance and, in some cases, avoiding prolonged
searching or exceeding time-out interval through the frequent accessing to
elements. It may lead to test failures.

In UI Automation, caching means pre-fetching of data. Caching occurs when the

application activates a CacheRequest object and then uses any method or property
that returns an AutomationElement, for example, FindFirst, FindAll. The methods
of the TreeWalker class are an exception; caching is only done if a CacheRequest
is specified as a parameter (for example,
TreeWalker.GetFirstChild(AutomationElement, CacheRequest)). Caching data
helps to avoid direct access to date. Any changes made to the CacheRequest after
you subscribe to the event have no effect. More information see here.

Dmitry Hes, Tsimafei Avilin, EPAM Sys. 12