PC Products
User’s Guide
TestBench-PC
User’s Guide
September 2007
Copyright 2007 The Original Software Group Limited.
Contact Details
Origsoft Inc.
Suite 170, 601 Oakmont Lane, Westmont,
IL 60559, USA
1 TestPlan
Introduction
TestPlan is a unique application that enables users to set up, manage and
monitor all activity relating to testing plans. TestPlan is far more than
just a test planning and defect tracking tool. It offers the ability to control
major elements of the testing process from within a test plan, quickly
making it the centre of testing functionality and creating large
improvements in test process efficiency.
Getting Started
When TestPlan is launched, the following window is displayed.
When TestPlan is first installed, Admin is the only pre-existing User ID.
Enter Admin into the User ID field. Leave the password blank when
entering TestPlan for the first time. Upon clicking ‘OK’ on the login
window the following window will appear.
The next step is to set the administrator’s password. This is the first and
only time this screen will appear. From this point forward the
administrator (or a user with administrator authority) must create any new
TestPlan users along with their initial passwords.
TestPlan Versions
There are two versions of the TestPlan application. The first, TestPlan
Administrator gives access to all the functionality and is described in
more detail throughout the rest of this guide. The second is TestPlan
User, a reduced functionality version, which enables the user to update
tasks and add/maintain defects but does not allow any changes to the
structure of the plan. Please note that the functionality available can be
further restricted by setting authority levels for the user, see later chapter
for more details.
TestPlan User
The following functionality is available in TestPlan User:-
• Creation of defects.
• Editing defects (including the maintenance of associated elements
such as PC Files, Test Results and Components).
• Editing tasks (including the maintenance of associated elements
such as PC Files, Test Results and Components).
• Generation of reports.
• Generation of charts/statistics.
• Search facility for tasks/defects.
• Viewing of milestones.
• Filtering of viewable task/defect information.
TestPlan Administrator
Functionality which is only available in TestPlan Administrator includes:-
File
New Add a new Plan - see the following section for more
information. The New option is also available by
clicking the New icon on the toolbar.
Properties Create Task and Defect User Fields for this Plan only.
These are fields which will appear on additional tabs on
the Task and Defect forms to enable extra information
to be recorded. They can also be defined at the system
level using the Admin form. See later section for more
information.
Help
Help Topics Access the online TestPlan help information.
The Help option is also available by clicking the
Help icon or by pressing F1.
Check for updates Open the Service Packs area of the Original
Software customer support site. Please contact
your point of supply for a user id and password.
Toolbar Options
Most toolbar options also exist on the above menus and these have been
described in the previous section. The exception is the Search function
which is explained later in this user guide.
Admin Options
When the Admin option is selected from the File menu the following
window is displayed. If the user does not have Admin authority only the
Plans tab is available.
When errors are found within a task they are assigned as defects. The
following two options relate to these defects.
Task User Fields Define additional fields that will be available via
an extra tab on the Task form. These can be
used to specify information specific to your
business needs. Fields defined here will be
available to all Plans, but User Fields can also
be defined at the specific Plan level.
Defect User Fields Define additional fields that will be available via
an extra tab on the Defect form. These can be
used to specify information specific to your
business needs. Fields defined here will be
available to all Plans, but User Fields can also
be defined at the specific Plan level.
Menu Options
The following options are available via the screen icons or right click
menu when an item has been highlighted and are only enabled when they
can perform a useful function.
Edit Resource
The following screen is displayed when either the New or Edit options
are selected from the Resources panel.
Email Address Enter a valid email address for this user. When
a defect is raised or edited and the ‘Send e-mail
notification’ box is checked, this field is
accessed and copied into an outgoing email.
Edit Plan
The following window is displayed when either the New or Edit options
are selected from the Plans panel.
new Plan.
The following window always appears as part of the Plan copy and will
follow the Welcome screen if tasks are not being copied.
Do not copy resources The only resource that will be copied to the new
Plan is the user performing the copy and they
will be given Administrator authority.
Copy source resources All resources with access to the original Plan
will be copied to the new Plan, their authorities
remain unchanged.
Copy used resources All resources that have been assigned to a task
or defect in the Original Plan or who have raised
a defect will be added to the new Plan, their
authorities remain unchanged. Resources with
access to the original Plan but who have not
been allocated within the Plan are not copied.
Copy selected resources The user can select which resources should be
copied to the new Plan and can optionally re-
allocate tasks and defects to another resource in
the new Plan. To do this click the New
Resource name in the table and select from the
drop down list.
The following window is displayed once all copy options have been
specified. Click Finish to perform the copy, Back to revise your
selections or Cancel to end the copy.
Defect Priority
A default list of Low, Medium and High priorities is shipped as standard
with TestPlan. This option can be completely customized at the global
level. The sequence of the priorities in this list determines the sequence
in which they will appear on the defect chart and can be changed with the
Move Up/Move Down options.
Defect Status
A default list of To Do, Fixed and Closed statuses are shipped as standard
with TestPlan. This option can be completely customized at the global
level. Every status is associated with a Defect Stage which is an internal
code defining the defect as open or closed.
Task Status
A default list of Not Started, In Progress, Testing Finished, and
Completed statuses are shipped as standard with TestPlan. This option
can be completely customized at the global level. Every status is
associated with a Task Stage which is an internal code defining the task
as open or closed.
Data Sources
This option can be completely customized at the global level. Every Data
Source can then be utilized by the Test Components and Test Results
functionality. Once configured here this will allow full maintenance of
components stored on different servers. Choosing one of the Data
Sources from the drop down box will point the application to search on
the specified database for the Test Component/Result which you wish to
attach. TestPlan fully supports the following Data Sources – iSeries,
SQL, Oracle, and Access.
New You can enter any alpha numeric name for the
Data Source, however duplicate names are not
permissible.
These fields can also be defined at the Plan level by selecting the
‘Properties’ option from the ‘File’ menu.
List Values For list and Multi-List field types, define the valid list
of values that the user will be able to select from.
Default Value Optionally specify a value that will appear in this field
by default.
Required Use this option to specify whether or not this field will
be mandatory for the Task or Defect.
Show in Filter This option is only available for List or Multi-List field
types. It controls whether or not the field will be
available to be used by the Task or Defect Filter.
Main Toolbar
Search
Access to the following window is obtained by clicking either the Search
button on the TestPlan Main Panel, selecting Search from the Edit menu
or pressing Ctrl+F.
Include History By checking this box the user can include any
textual elements which have appeared in the
historical logs of defects and tasks. Note the
string of text entered must exactly match that in
the historical task or defect log.
There are also two more features to the Search facility. You can Search
Items in the ‘Current’ state based on different parameters and also using
user configurable search parameters it is possible to search on a
‘Transition’ state – see below.
Edit Parent Task (Option only available for defects) Right click
on the result line corresponding to the defect and
choose this option to view or change the details
of the task to which this defect is assigned. The
resource must have edit authority or higher in
order to use this option
Open/New Plan
Access to the following window is obtained by clicking either the Open
button or the New button on the TestPlan Main Panel, selecting the
relevant option from the File menu or pressing Ctrl+N or Ctrl+O. It is
also the default view when TestPlan is launched. When New is selected
the New Plan tab is given focus and the New Plan options are displayed.
When Open is chosen the Open Plan tab receives the focus and the Open
Plan options are made available.
Open last plan The last Plan is based on the most recent plan
accessed on the same PC. It will appear in
double quotation marks next to the Open last
plan radio button. Select this option and then
click on the Open button.
Open other plan Click on an existing Plan in the lower part of the
screen to highlight it. These are the Plans
available to the current user based on Plan
authority. Click on the Open button with this
option selected to access the highlighted Plan.
Only Show Favourites Checking this option means that only those
Plans that have been selected by ticking the
‘Favourite’ column are listed.
Rename Plan Right click a Plan in the list and select this
option to rename it.
Delete Plan Right click a Plan in the list and select this
option to remove it. A confirmation screen will
be displayed.
Create a new plan To add a new empty Plan select the ‘Create a
new plan’ option and then click on the Open
button. A prompt will appear requesting a name
for the new Plan. This name is alphanumeric
and can be between 1 and 35 characters in
length. If this name is completely numeric, the
Plan name will be right justified. If the name
contains a letter the plan name will be left
justified in the field. Users must have Create
Plan authority to utilise this option. The user
that creates the Plan is automatically given
authority to edit the Plan.
Only Show Favorites Checking this option means that only those
Plans that have been selected by ticking the
‘Favorite’ column are listed.
Rename Plan Right click a Plan in the list and select this
option to rename it.
Delete Plan Right click a Plan in the list and select this
option to remove it. A confirmation screen will
be displayed.
The borders of each pane can be clicked and dragged in order to re-size
them. The pane configuration can also be changed by clicking on the title
bar and dragging to a new location which is indicated by a grey box. If
the grey box has a tab at the bottom the two panes will overlay each
other, as with the Folders and User Tasks on the picture below. Any of
the panes can be added as tool bar options if you do not wish them to be
permanently in view as with the View Options and Filter on the picture
below. Click on the icon to temporarily display the pane. Click on the
pin icon if you then wish the pane to become a permanent part of the
display.
It is also possible for the user to define and save their preferred layout.
This is achieved by right-clicking on any of the main panel banners.
There are three options available, although the ‘Restore preferred layout’
option is only available once one has been ‘saved’.
Folders
Folders are used as a framework to organize testing and planning at the
highest level. A folder is an envelope used to hold other folders along
with various tasks that need to be accomplished within a certain area or
application. Each task may have one or more defects. In order to utilize
the options available within a folder, left click the folder to bring it into
focus and then right click it. All folders are indented from the top folder
which shares its title with the Plan name.
There are three options which control the folder view. These can be
changed using the View Options.
Task Count Two numbers in square brackets before the folder name
show the number of tasks with a stage of ‘Open’ out of
the total tasks that have been added to the folder. These
numbers are affected by any filters that have been
applied to the Plan. The folder icon will contain a green
square if all of the tasks have a stage of Closed or a blue
square if any tasks are Open. Where the ‘Show all
items in sub-folders’ view option has been selected, if
all items within the selected folder and its sub-folders
are Closed then the coloured square will be green.
However where this is not the case and some items
within the subfolders are open the square will be split
diagonally with the top half being shaded green and the
bottom half blue. Conversely if all the items in the sub-
folders are Closed but some are Open in the parent
folder then the square will be displayed as a diagonal
split with the top half blue and the bottom half green.
Defect Count Two numbers in square brackets before the folder name
show the number of defects with a stage of ‘Open’ out
of the total that have been added to the folder. They are
the aggregate number of defects for all tasks within the
folder. These numbers are affected by any filters that
have been applied to the Plan. The folder icon will
contain a green square if all of the defects have a stage
of ‘Closed’. If any defects are still ‘Open’ this square
will appear in red. If there are no defects the colored
The following are the right click menu options. The resource’s authority
within each plan will dictate which of these options are available. For
example, to move or copy a folder the resource must have Edit Plan
authority or higher.
Drag and Drop This option has the same effect as the two step
process of cutting and pasting. Unlike those
options, it is not accessed by right clicking the
mouse. Instead, determine the highest level
folder to be moved, left click the folder and hold
down the mouse button. Drag the folder, by
moving the mouse until the new parent folder is
highlighted. At this point release the left mouse
button. After completing these steps, the folder
that was selected will be removed from the
folder it was attached to along with all of its
subfolders, tasks, defects and attachments and
added to the new location. (See Cut and Paste
above)
NOTE: The top level folder cannot be manipulated the way other folders
can. For all intents and purposes the top folder is the Plan. Deleting the
top level folder is equivalent to deleting the Plan which must be done
from the Open Plan panel. The Plan can also be renamed here. The only
available options are New Folder, Chart and Report. The chart works
differently for the primary folder than any other folder, the chart and
statistics are displayed based on all subfolders of the primary folder.
Tasks
A task is a detailed item to be accomplished and resides within a folder.
Tasks can be allocated to a specific owner or Resource who will then be
responsible for seeing the work through to completion. By default, Tasks
are displayed in the upper right hand pane of the TestPlan main panel, but
this can be changed via the drop down list to view Defects, TestBench
Components, PC Files or Test Runs.
The following options are displayed after right clicking within the task
pane. The resource’s authority within each plan will dictate which of
these options are available. For example, to cut or copy a task one must
have Delete Task authority or higher.
Move Down Move the Task down one item in the list. This
option is not available if the ‘Show all items in
sub-folders’ option is checked.
Add/Edit Task
Task Details
The following panel is displayed after the New or Edit option is selected
from the Task pane.
Set as Milestone Check this box if the date for this task should be
considered a very significant event. Then
choose the appropriate radio button (Start date
as Milestone or End date as Milestone) to
indicate which date is most important. If
selected this task will appear in the Milestones
table.
User Fields
This tab is only visible if any User Fields have been defined either at the
System level (using the Admin form) or the Plan level (Properties option
from the File menu). User Fields enable additional information to be
recorded about the Task. This information can then be searched upon or
included in the Filter.
History
The task history enables anyone involved with a certain task to list their
comments in a central location. This allows for a dialogue between the
manager and the resource responsible for testing the task. It also gives
anyone else involved the opportunity to review what has transpired to that
point and even adds comments of their own. Proper use of task history
will result in tasks being tested and closed rapidly and accurately. To add
a log history record, key some text into the Add Log History field and
click on the Save button. Once an entry has been made it can not be
edited, this provides an accurate account of the events surrounding your
task. To view the detail simply hover your mouse over the entry under the
column named Log. If you require a more detailed view you can also
double click the entry to show the task history panel.
lack of space. If this occurs, move the cursor in between the header titles.
A vertical line with arrows pointing outward will appear. Click and hold
the left mouse button and drag the header bar to give the appropriate
description below more viewable space. The entire window can also be
expanded by clicking and dragging the window edges.
Status The status of the task at the time the log history
was added.
Log The text that was keyed into the Add Log
History field. Can be used to respond to another
resource or just to enter additional notes about
the task.
Steps
The tab labeled Steps can be used to store detailed information on the
execution of a task. This detail being broken down into the different steps
required for the successful task completion. This is an ideal method for
reminding staff of all the correct steps required to complete a task and can
also act as a checklist. Each step can have multiple columns, these can be
used for instructions, data input and expected results. When a task is
executed and a manual test initiated, the steps are converted to Test Items
which can then be passed or failed and stored in the results database for
the task.
Each step can be one of five different types as listed below, examples of
the application of each step type are shown on the screen print above.
• Instruction
• Screen
• Keyboard
• Mouse
• Check
Change the step type by clicking on the step and then on the relevant icon
on the toolbar. Change the sequence of the steps by clicking and
dragging the grey box to the left of each row.
Toolbar Options
Insert Add a new step to the grid after the current step.
Test Items from the Task Steps. Due to the fact that
TestDrive-Assist appears as a narrow sidebar, only the
default step column will be displayed. Double click the
column header from within TestDrive-Assist to show an
extra window containing all of the columns. At the end
of the test the results can be saved back to the Task.
This option is only available for Tasks with Steps. See
the Manual Testing chapter for more information.
Results
The following panel is displayed when Results is selected from the
Add/Edit Task or Defect screens.
Any test Result which is stored on a known Data Source can be attached
to a task or defect using this tab. This feature is intended to allow testers
to capture the test results associated with a particular test, for example to
assist with the explanation of a particular defect.
Attach
By entering parameters for the Data Source, Project and Test Case the
required Test Result can be attached to the Task or Defect. Summary
details on each are displayed in the window above once the Result has
been attached.
View Details
Delete
Remove the link to any attached Test Result. Note the result itself is not
deleted from the server.
Paste
Use this option to add in Results copied or cut from other tasks or defects.
Copy
Cut
Use this option to cut results from any given task or defect.
PC Files
The following panel is displayed when the PC Files tab is selected from
the Add/Edit Task or Defect screens.
the left mouse button and drag the header bar to give the appropriate
description below more viewable space. The entire window can also be
expanded by clicking and dragging the window edges.
Modified Date Date the file was created or last changed. This
is not the date that the file was attached to the
task.
TestBench Components
The following panel is displayed when the Components tab is selected
from the Add/Edit Task or Defect screens.
functions, developing Data Cases, Test Cases and more. They will then
have the ability to attach any of those components directly to the task or
defect for which they are responsible. This will allow the developer to
launch these modules themselves and see precisely what the tester found,
bringing communication and shared knowledge to a whole new level. In
addition the results of any test cases or scripts can be attached and
viewed. The modules are not saved within TestPlan. Rather, TestPlan
saves the database location of the modules. In this way, a module can be
accessed and launched from any pc possessing the appropriate software.
For all components other than Test Cases, double click the component or
right click and choose Select to attach it. Alternatively highlight a
component and then click the Add button. Continue to add as many
components as required. For Test Cases, right click and choose Select to
attach, or double click to reveal the list of Scripts which are stored against
the Test Case which can in turn also be selected.
Change the view Click on the button or the down arrow to toggle
the list of components between the Detail view
shown above, the List view which includes the
component name but not its description and the
Icon view.
Defect Pane
A defect is attached to a task and should be raised when an issue is found
with the task. Defects can be allocated to a specific owner or ‘Resource’
who will then be responsible for correcting the error that is occurring.
Defects for a selected task are viewed by default in the lower right hand
pane of the TestPlan main panel. However, all defects for the selected
folder can also be viewed in the upper right hand pane by selecting
Defects from the drop down list.
The following menu options are available on right click. The resource’s
authority within each plan will dictate which of these options are
available. For example, to cut a defect the user must have Delete Defect
authority or higher.
Add/Edit Defect
The following panel is displayed after the New Defect or Edit option is
selected from the defect pane.
The User Fields tab is only visible if any User Fields have been defined
either at the System level (using the Admin form) or the Plan level
(Properties option from the File menu). User Fields enable additional
information to be recorded about the Defect. This information can then
be searched upon or included in the Filter.
The defect history enables anyone involved with a certain defect to list
their comments in a central location. This allows for a dialogue between
the tester and the resource responsible for fixing the defect. It also gives
a manager, or anyone else involved, the opportunity, to review what has
transpired up to that point and even add comments of their own. Proper
use of defect history will result in defects being closed rapidly and
accurately. To add a log history record, key some text into the Add Log
History field and click on the Save button. Again, for integrity, the entry
details cannot be adjusted once they are saved.
some cases the complete contents or description will not be viewable for
lack of space. If this occurs, move the cursor in between the header titles.
A vertical line with arrows pointing outward will appear. Click and hold
the left mouse button and drag the header bar to give the appropriate
description below more viewable space. The entire window can also be
expanded by clicking and dragging the window edges.
Log The text that was keyed into the Add Log
History field. Can be used to respond to another
resource or just to enter additional notes about
the defect.
Reporting
Reporting Window
The TestPlan Reporting window is displayed by clicking the button
containing the Report icon at the top of the TestPlan Main Panel or by
selecting Report from the list under File on the TestPlan menu bar. As is
illustrated below, creating a report is as simple as selecting two
parameters and clicking the refresh button. The footer on every report
will show the date the report was created, the name of the resource that
created it and the page number.
Reporting Options
Folder Choose a folder from the Folder list box by clicking the
drop down arrow at the right edge of the list box. The
selected folder will appear on the report along with all
subfolders that exist beneath it. The option to select
‘All Folders’ will run the report based on every folder
in the Plan.
View Select an option from the View list box by clicking the
drop down arrow at the right edge of the list box.
Choose between Folders, Tasks, Defects, Audit Report,
Plan Statistics and Defect Graph.
Refresh Button Before clicking Refresh, any changes to the Folder list
box, the View list box and/or items that reside within
the options button should be made. Clicking the
Refresh button then creates a new report based on the
report items selected at that time.
Print Button To create a hardcopy of the report click the Print button.
TestPlan integrates with Microsoft Printer Options.
Any printer currently defined on the user’s pc may be
used to print the report.
Save Button Once generated the report can be saved in Rich Text,
HTML, Acrobat pdf or Excel format.
Defect Chart
The Defect Chart and statistics are displayed by clicking the button
containing the Chart icon and name at the top of the TestPlan Main Panel
or by selecting Chart from the list under File on the TestPlan menu bar.
The Chart can also be viewed for defects residing in individual folders by
right clicking the folder and selecting chart. The chart is a three
dimensional image of defects by status and priority. Also, a tabular view
of the related statistics can be displayed by accessing the statistics tab.
Statistics
The statistics tab displays the chart details in tabular form. This may be
useful if a view with a more precise number of defects per status and
priority is needed.
Milestones
Using Milestones
Milestones are a set of important dates or events that can either be task
related or independent of task. Use this option to view existing
milestones, add or delete milestones. The Milestones option is available
by clicking the Milestones icon on the toolbar or from the File menu
option. This option is greyed out when a Plan is not open.
The tick to the left of the milestone indicates that this milestone is related
to a task and must be updated from the Edit Task panel. The following
menu options are available on right click.
Add New Create a new milestone that will not be directly related
to a task. See below for more information.
Edit Change the description or the date for non task related
milestones.
Delete Remove the milestone. A confirmation screen is
displayed.
Edit Task Access the Edit Task panel for task related milestones.
Amend Milestone
The following screen is displayed when either the Add or Edit milestone
options are selected.
Filters
Using Filters
The filter feature enables the user to limit the Plan view individually for
each PC by various task and/or defect criteria. The Filter selection option
is available by clicking the Filter icon Tab on the left side of the TestPlan
panel. The filter can be fixed to the pane layout by clicking on the pin
icon at the top right side of the filter pane. Use the check boxes that drop
down when you click on a folder icon with the filter description to select
which filters to implement. Once selections are made the options stay on
view beneath their parent folder. Any Task or Defect User Fields appear
at the bottom of the panel.
Activate
To enable the filter simply click the ‘Click here to enable the filter’ blue
text. This will change the status of the filter which will toggle between
the two icon values and with a clear statement to describe the
current status.
By activating the filter you will only see the tasks and/or defects you
described within the selection criteria. When you de-activate the filter you
will return to a full view of all the tasks and defects on the plan.
Edit
When editing the filter to include new selections or remove unwanted,
you will be given the option to ‘Apply’ or ‘Cancel’ the changes. This
provides the chance to revoke the changes at the last minute. By clicking
on ‘Apply’ or by selecting the ‘Click here to enable the filter’ the changes
will be taken and applied to the view over all the panes. If you select
‘Cancel’ the changes will be ignored and you will be returned to an
unchanged view.
View Options
PC File View This option allows you to change the view in the Details
Pane when PC file has been selected to show full details
(including modified date, Path and computer name that
attached the item) or simply a large/Small Icon.
User Tasks
To re-allocate a task you can drag and drop the task from the Main pane
to the target resource listed in the user tasks pane.
2 TestBench-PC
Introduction
More and more companies are recognising the need for effective software
testing and realise that to achieve it requires tools such as Original
Software’s ‘TestBench’ – a unique software testing tool.
And, with the database in better shape, it also means that the final user
acceptance testing phase isn’t going to be hijacked by fundamental code
quality issues. So, overall IT service standards are enhanced and user
testing stays correctly focused upon ensuring that business needs are
satisfied.
System Requirements
Minimum Spec Recommended Spec
Pentium III 600 Mhz Pentium III 1.0 Ghz
64MB Memory 256MB Memory
Getting Started
When TestBench-PC is launched, the following panel is displayed. The
User ID and Password are optional fields and should be entered if your
database requires it. For Oracle and SQL Server, if you are using
Windows authentication then the User ID and Password should be left
blank.
Product Registration
Without any product registration, TestBench-PC simply provides a view
of the product structure which includes Projects and Test Cases, and
allows these placeholders to be defined and modified. However, in order
to create scripts, run tests and manage the database, one or all of the
following products should be registered.
The following steps can be used to register any of the above products.
1. Click on the ‘Help’ menu option at the top left of the TestBench-PC
panel.
TestBench-PC Panel
The following panel is displayed after you have made your selections on
the Data Source Connection panel. The panel can be re-sized as required.
Menu
File Select the ‘Exit’ option to close the TestBench-PC
application, in which case a confirmation screen is
displayed.
Toolbar
Show Types Filter the list of TestBench components to only
those of a specific type. If either TestDrive-
Green or TestDrive-Gold scripts are selected
while viewing Test Cases and other iSeries
components, the list will be filtered to show
only Test Cases, with those which do not
contain any of the requested script type greyed
out.
Change the view Click on the button or the down arrow to toggle
the list of components between the Detail view
shown above, the List view which includes the
component name but not its description and the
Icon view.
Search
The following panel is displayed in an additional tab when the Search
button is clicked on the main Project Explorer form. It is possible to have
multiple search forms open in a single TestBench-PC session.
Text Search Key the full or partial text string that you wish
to search for within the selected component
names.
Right click an item in the Search form to obtain the following options.
Home Tab
Click on the far left tab to display the Announcements Area of the
Original Software web site. All of the latest product news can be viewed
here and it is also a point of contact for support and sales staff at Original
Software. If this page has changed since you last opened TestBench-PC
you will be defaulted to this tab on entry, thereafter it can be viewed at
any time by clicking on the tab. Select your country of residence at the
top of the page in order to display any location specific news.
Main Panel
The main panel consists of four key areas under the toolbar and toolbar
icons. The options and information displayed in the three left hand panels
is dependant upon the component currently in focus. There are three tabs,
Home, Project Explorer and Test Environments, but the Test
Environments tab is only visible when TestBench-PC is installed on an
Oracle server.
• The main component panel is split into two lists. The list of available
Projects or Test Environments is displayed in the left hand panel, each
of these can be expanded to show the Test Cases or Checkpoints
within. Click on either a Project, Test Case or Test Environment to
populate the right hand panel with the components stored inside it.
• The ‘Actions’ area contains all of the options which relate to the
component in focus. For example, if a Project is highlighted then the
Actions panel can be used to Edit, Delete or maintain Notes for the
Project. These options are also available via a right click menu.
Commands
Project Actions
To gain access to the following functions either click the options in the
Actions panel or right click the Project name on the display list.
Notes Edit the notes associated with the Project - see later
section for more information on using notes.
Commands
New Test Case Add a new Test Case, see later section for more
information.
New Action Map Launch Action Map and create a new Map. See
separate Action Map chapter for more
information.
New Variable Data Create a new variable data set, see separate
section in this chapter for more information.
New Data Rule Create a new Project level Data Rule, see
separate section in this chapter for more
information. (TestBench for Oracle only)
New Data Case Create a new Data Case to extract some test
data, see separate section in this chapter for
more information. (TestBench for Oracle only)
To gain access to the following functions either click the options in the
Actions panel or right click the Test Case name on the display list.
Edit Change the Description for the selected Test Case. See
later section for more information.
Copy Copy the selected Test Case to a new Test Case in the
same or any other existing Project.
Notes Edit the notes associated with the Test Case - see later
section for more information on using notes.
Commands
New Data Rule Create a new Test Case level Data Rule, see
separate section in this chapter for more
information. (TestBench for Oracle only)
View Results Display the results list in a separate tab for the
selected Test Case. Includes an indication of
how many results are available to view within
brackets. See later section for more
information.
To gain access to the following functions simply select the option from
the Actions panel or right click the Action Map from the display list.
Notes Edit the notes associated with the Action Map - see later
section for more information on using notes.
Delete Remove the Action Map from the Project. This will not
delete component objects which exist in their own right.
Commands
View Results View results for all previous runs of this Action Map.
See later section for more information.
To gain access to the following functions simply select the option from
the Actions panel or right click the Variable Data Set from the display
list.
To gain access to the following functions simply select the option from
the Actions panel or right click the Script from the display list.
Script Actions
Notes Edit the notes associated with the Script - see later
section for more information on using notes.
Commands
View Results View all Results for this Script - see later section for
more information on using Results.
Right click on the single grey box in the top left of the Edit Variable Data
window to begin creating variable data fields and transactions for new
Variable Data Sets. The following options are available.
Column Options
Copy Column Add the contents of the selected column to the
Clipboard.
Row Options
When at least one row has been added some additional options are
available on right click on the transaction numbers. This screen is cursor
sensitive; the options listed apply to the row on which the right click
occurred.
In addition to the right click options, rows can be moved simply by left
clicking the transaction number and dragging to the desired new position.
Multiple rows are selected by using left click in combination with
Shift/Ctrl.
Paste Row Insert the data in the previously copied row into
the row where the right click occurred.
Data Options
This screen is cursor sensitive; the options listed apply to the cell on
which the right click occurred. Multiple cells are selected by using left
click and dragging the cursor.
=date (dd/mm/yy)
A - is valid in addition to a +.
The valid operators are day, days, month,
months, year, years. The expression is not case
sensitive.
Play empty values When selected, this option will play empty
values into the fields on your script.
Data Rules
Data Rules represent a key component of TestBench as they allow the
database effects of a test run to be proactively checked without requiring
further action by the user. Data Rules are available both on the iSeries
and on Oracle if a valid TestBench for Oracle license exists. However, if
the iSeries is the server then Data Rules must be defined and their results
viewed on the server (see the Testing chapter of the TestBench User
Guide for more information), only TestBench for Oracle Data Rules can
be managed using TestBench-PC.
In TestBench for Oracle, Data Rules can be defined at the Project level or
for an individual Test Case. If defined at a Project level then they will be
automatically applied to all Test Cases, unless the option on the Test Case
is set to exclude them. Data Rules can be defined for any table and there
is no limit to the number of Data Rules that can be defined for a single
table.
Data Rules can be used in all aspects of testing, but they are often
constructed by Systems Analysts, Database Designers and Technical
staff. However, their power to validate can be maximised if they are also
applied during later phases of testing such as regression or UAT. With
TestBench this does not mean the persons testing at this stage have to
understand or be familiar with the rules, as the tests they perform can still
benefit from the knowledge invested in the rules by others. For this
reason, it is recommended to define rules at the highest appropriate level
(Project) whenever possible and have only very specific rules defined at
the Test Case level.
A data rule is only applied to tables which have been updated by the test.
The Test Case within which the script is stored must be using
At the end of each test, Data Rules are applied and checked against the
database effects captured from the test and any discrepancies are reported
as Warnings. The full details of the discrepancy can be viewed side by
side with the Data Rule in TestBench-PC results.
Click the Next button in order to display the following screen where the
table is selected.
Click on the table over which the Data Rule will be built and then click
on Next to define the Data Rule details.
The top half of the screen defines the optional When clause, which
determines under which conditions this rule will be applied. The second
half of the screen is where the Test clause, the detail of the Data Rule
checks are defined.
Clause The first row in the two tables has a defaulted value of
When and Test respectively. For all of the other rows
the valid values are ‘And’ and ‘Or’.
Value 1 This can be either a field on the table being check or the
special value of ‘Operation’, both of which can be
selected from the drop down list that is displayed by
clicking in the cell on the grid. ‘Operation’ will check
whether the table activity in question was a Write,
Update or Delete, depending on the Value 2 selected.
Condition Choose from a drop down list containing =, >, <=, <,
>=, and <>. This is used to compare Value 1 to Value
2.
The Save button is only enabled when the Rule has been entered
correctly, any fields in error are highlighted in red. Press the Back button
at any time to modify entries on a previous screen, click on Save to store
the new Data Rule or Cancel to discard any changes made.
Test Environments
When a valid TestBench for Oracle license exists, an additional Test
Environments tab is available in TestBench-PC. Test Environments
consist of a list of Schemas which contain the test data for an application.
If a Test Environment is defined for a Test Case, when any scripts stored
within that Test Case are played back with the ‘Activate Test_IT’ option
checked, results in the form of database effects and data rules are
reported.
Test Environments also enable data that has been changed by a test to be
rolled back to a previous state, thus allowing you to run a program many
times knowing that the starting point of all the tables will be consistent.
This capability is essential if you are using a ‘record and playback’ tool
such as TestDrive-Gold to exercise your interactive programs, as without
a data starting point identical to that when the script was created, it is
inevitable that many differences will be found during the execution of the
script. This can be done even if TestBench is not being used to initiate
tests via Test Cases and Scripts, but when testing is being performed in
your application directly.
Database View
A view similar to the following is displayed when the TestBench database
name is highlighted.
Commands
To gain access to the following functions either click the options in the
Actions panel or right click the Test Environment name on the display
list.
Commands
Checkpoints
The following panel is displayed when the ‘New Checkpoint’ option is
selected. A checkpoint consists simply of a description which can be
changed in edit mode at any time without affecting the operation of the
Checkpoint.
When a checkpoint is first added it is not active and the rollback option is
not available, as shown for Checkpoint number 3 below. Only when the
‘Activate’ option is selected can any changes made to the database since
the activation be rolled back. It is however possible to roll back to an
earlier active checkpoint.
Checkpoint Actions
Commands
Activate Access the logs and store the current position of the
database so that at any point in the future the rollback
option can be selected to reset the data to its current
state. When a Checkpoint is activated, the date and
time is logged and displayed on screen. Checkpoints
must be activated in ID sequence, for example you
cannot activate a Checkpoint if earlier inactive
Checkpoints exist. NOTE: If you re-set an active
Checkpoint, any later active Checkpoints will be
rendered inactive. This action is prevented if any earlier
active Checkpoints exist to avoid the loss of any
intermediate database updates.
Launch Files
Launch files are used for two purposes. Firstly they support integration
with Mercury Quality Center and enable you to launch a particular Script,
Map or set of Results without going through TestBench-PC. To use the
file in Quality Center simply add it as an attachment as you would any
other PC file. To launch outside of Quality Center simply right click the
file and select ‘Launch’. Secondly they enable complete information to
be specified which facilitates the fully automated execution of a Script or
Action Map in unattended mode. This means that a scheduler can be
used to launch a series of tests at a specified date and time.
Maintenance
Launch files of the first type can be created for an entire Project or Test
Case by right clicking the Project or Test Case in TestBench-PC and
selecting the ‘Create Launch File’ option. The launch files are created in
a folder called My Original Software\Launch Files in the MyDocuments
folder on the PC and have the extension .TBC. Files of both types can be
created for individual components by either right clicking a Script or
Action Map in TestBench-PC, or by selecting the option from the File
menu when the Script or Action Map is open. The following wizard is
then used to specify the required parameters.
Map contains If the launch file being created is for an Action Map,
this option will automatically be populated with the
types of scripts contained within the Map. This is
important because it controls which options will be
displayed on the following wizard screens. It will only
need to be modified if you are editing an existing
launch file and the components of the Action Map have
changed.
If the Play option is checked, click Next to access the next screen of the
wizard, otherwise click Finish.
The first three ‘Target’ options are only displayed for TestDrive-Gold
scripts or Action Maps which contain them. The last two ‘Target’ options
are only shown for TestDrive-Green scripts or Action Maps which
contain them.
performed.
iSeries session Use named session means that the session specified in
the ‘Session name’ below will be the one over which
any TestDrive-Green scripts will play back.
Use first free session does not require an entry in the
following field and means that the first available 5250
session will be used.
Session name Specify the short name of the 5250 session over which
any TestDrive-Green scripts will play back if ‘Use
named session’ is specified above.
Click Next to display the final screen of the wizard where the parameters
required to connect to the TestBench database at playback time are
specified.
Click Finish to store the changes made in the wizard and create or update
the TBC file. You will be prompted for a location in which to save the
file.
Once the TBC file has been created, either double click it or right click
and select Edit to change any of the above parameters.
Execution
Once a launch file has been created it can be launched in one of two
ways.
• Right click the file directly and select ‘Play’. This will initiate an
immediate playback of the Script or Map.
• Launch the file via a command interface. The command can be run
from a scheduler to enable Scripts or Maps to be played back at a
specific date and time. Do this by calling the Play.exe program
which can be found in the location C:\Program Files\Original
Software\Programs and pass the name of the file as a parameter.
For example:
Results
Every playback which successfully initiated will have its results stored in
the TestBench database. Information about all unattended runs is also
written to the Windows Event Log which can be accessed from Control
Panel / Administrative Tools / Event View and selecting the Application
item in the panel. A log entry is created every time a playback begins and
ends and details of any errors and result Run Numbers are also logged.
There are three types of event; Information, Warning and Error. If a
Script or Map starts and ends without errors, this is logged as an
Information event. If playback completes but errors such as screen
differences are reported, this is a Warning event. If a Script or Map is
unable to start or fails to complete this is logged as an Error and the
reason for the error is added to the log.
Product/Name/User Selection
At the top of the results three drop down boxes
are available to provide the ability to refine your
view and selection criteria by Product, Name of
the Script, Map or Application, or User Profile
who ran the test.
Test Status Describes the current status of the Test Run. The
Warn column will be either blank or contain the
text ‘Warnings’ to indicate that either there was
an issue with the execution of the Test or that
some of the expectations were not met. Run
provides the latest status of the Test Run, values
can range from Queued and Running through to
Result options
The following options are accessed by right clicking the mouse over the
required Test Run from the list.
- Pass/Fail
Notes Edit the notes associated with the Test Run - see later
section for more information.
Report Produce a test report for the Test Run - see later section
for more information.
All Screens in the result details have the ability to pass to previous and
next test runs. Simply click on the Icon at the bottom of the panel to
display the desired run information.
Summary
The summary provides a view of the Test Run results and Test Case set
up. The summary screen is the first tab on the collection of details from
the Results list. This details tab holds the high level summary and has the
ability to allow a filter view of sub-run activity across the other tabs on
this set of results.
Warnings
All warnings apart from screen differences will only be generated if the
Script has been integrated with a Test Case.
Timeline
Screens
Display the detailed results for screens from the Test Run or Sub-Run
using the product which created the script, either TestDrive-Green or
TestDrive-Gold. Please refer to the relevant chapters for more
information on these products.
Database
If a Script has been integrated with a Test Case on the iSeries or Oracle
and this Test Case uses Environment Protection, all database updates
made by the server programs in the associated Environment are tracked
and stored by TestBench. This option provides a summary display of this
database activity.
To see record level details including before and after images you need to
select the file from the top panel and then the Row ID from the lower left
panel. Each difference is denoted by the update symbol to the left of the
detail row.
Data Rules
Although Data Rules can be used on both the iSeries and Oracle, they can
only be maintained in TestBench-PC on Oracle and therefore this tab is
not visible when the server is the iSeries.
All of the Data Rules for this Test Case and the Project in which it resides
are listed. The symbol to the left of the row indicates whether the rule
has passed (grid with a green tick), failed (grid with a red cross), or was
not applied (empty grid) due to the fact that the table over which it was
based was not updated by the test. The Rule Filter at the top of the screen
can be used to control which of these rule types are visible in the list.
Click on a rule in the list to display the rule definition to the right of the
screen. The conditions which have caused the rule to fail are highlighted
in red. The rows in the table which have failed the rule are listed in the
bottom left panel. Click on one of these to display the database effect,
with the failed columns also highlighted in red. The Field Filter at the top
of the screen can be used to show only Passed, Failed or All fields.
Server
Every server program on the iSeries initiated by the test is listed on the
screen above along with information about the program execution. This
information includes Call Sequence, Start Time, Elapsed Time, CPU
Cycles and Auxiliary I/O.
Log
All iSeries messages which have been received by the Job Log or Oracle
SQL errors for this test can be viewed here.
Response
Pass/Fail
When a test is executed, it is possible to define a list of Test Items against
a Test Case which at the end of the test can be passed or failed. In
addition to this, the Test Run as a whole can also be given a status of Pass
or Fail. These values can be updated on the following screen.
Current Run Status Display either Pass, Fail or N/A (not yet
defined) to represent the relative success of the
Test Run.
Change Run Status Change the current status of the Test Run to
either Pass, Fail or N/A. Once the status has
been altered, the comments box is enabled and a
comment must be entered before the status
change can be updated.
Update Status Click this button once the new status and
comment have been entered in order to update
the current status of the Test Run.
The following options are available from the right click menu for an
Action Map Run.
Map Entries Show the details of the Action Map Run in another tab
as described below.
Delete Remove the Action Map Run and all of the Action Map
Entries.
Map Entries
Right click any of the Action Map Runs to display the following screen
showing results for each individual Action Map entry.
Result options
The following options are accessed by right clicking the required entry in
the list. However some options are not applicable to every type of entry
and may therefore not always be available.
- Summary
- Warnings
- Timeline
- Screens
- Database
- Data Rules
- Server
- Log
- Response
- Pass/Fail
Report Produce a test report for the Action Map Entry - see
later section for more information.
Reporting
The reporting function for a Test Run can be accessed from the Results
List panel by right clicking a Test Run or Action Map Run. The options
are stored and remembered on each PC.
File Effects If database effects have been stored for this run,
choose which level of information to include on
the report.
Data Rules If Data Rule results have been stored for this
run, choose which level of information to
include on the report.
Click the Next button to complete the wizard selections and then the
Finish button to begin the report generation. Please note that only the
options available on your server are enabled. Once the report has been
generated the following screen is displayed. Use the arrows at the bottom
of the report to navigate through the report and the magnifying glass to
alter the zoom level.
Data Cases
When a valid TestBench for Oracle license exists, additional options to
enable test data to be extracted are available in TestBench-PC. When
carrying out testing to ensure that the correct data is being written to a
database and that functionally the system is performing as it should, it is
not usually practical to perform these tests on large volume files (such as
full copies of a live database) as you cannot see the wood for the trees. It
is necessary to know what the expected result of any test should be in
order to determine whether the test was successful or not, something
which cannot be easily defined with large volumes of data. Later, once
you are happy that the correct functions are being performed, volume
testing has a meaningful place to check performance, capacities and
operational/environment issues, but until then, you need small volumes
and appropriate data which you can use and re-use.
Once you have created test tables in the format you require, Test
Environments can be used to manage the test data and provide the option
to re-set the database to pre-defined check points.
Introduction
There are two steps to performing a data extraction using TestBench for
Oracle:
1. Take a copy of the source database (the schema that you wish to
extract from) and create the target test database. This is called
cloning a schema and is only required if a brand new database is
required. Existing schemas can be re-used if the tables are
unchanged and you simply wish to update the data within.
Both of these tasks are performed from the Data Case panel. This is
accessed in a number of ways:
• Highlight a Project and click the ‘New Data Case’ option from the
Commands panel.
• Right click a Project and select the ‘New Data Case’ option.
• Highlight an existing Data Case and click on the ‘Edit’ option from
the Actions panel.
Toolbar
Information Area
Data Case Name In Add mode you should key an identifier for
the Data Case which must be unique within the
current Project and is a maximum of 10
characters in length. This field cannot be altered
in Change mode.
Data Case Description Key appropriate text to describe the data that
will be extracted by this Data Case.
Data Source TNS Name The TNS Name of the source schema from
where data will be extracted. This is displayed
here for information purposes and can only be
change by clicking the ‘Change Data Source’
toolbar button.
Data Source Schema The name of the schema from where data will
be extracted. This is displayed here for
information purposes and can only be change by
clicking the ‘Change Data Source’ toolbar
button.
Default Target Schema The name of the target schema to where data
will be extracted. This is a default value and
can be changed at execution time.
Source Credentials
The following panel is automatically displayed the first time you attempt
to interact with a new Data Case session, because no actions can be
performed until a connection has been made to the source database. To
alter these details once a connection has been established, click on the
‘Change Data Source’ toolbar button on the main panel.
Clone Schema
Before a Data Case can be executed, the target schema must exist. A
copy of an existing schema can be created using the ‘Clone Schema’
toolbar button, this can then be used to hold the extracted data.
Alternatively this option can be used to update an existing schema with
new table definitions.
Click ‘Next’ on the first screen of the wizard to reach the following panel.
All of the following values are defaulted from the Source Credentials
already specified but can be overridden here.
TNS Name The TNS Name of the source schema that will
be copied to create the target schema.
Relationships
When a new Data Case is created, the Relationships panel will be empty.
This can be populated manually by right clicking and adding files using
the ‘Add Top Level Table’ and ‘Add Child Table’ options, these are
described further below. However, a faster method is to use the ‘Auto
Analyse’ option from the right click menu. This will automatically add
all files in the source schema into the Data Case. Additionally, if the
source tables have constraints defined, these will be used to build a
relationship tree for the database as shown below.
New parent files can be added by right clicking and selecting the ‘Add
Top Level Table’ option. The following panel is displayed which lists all
tables in the source schema, click on one to select it and then click
‘Finish’.
New child tables can be added to the Data Case by right clicking and
selecting the ‘Add Child Table’ option. Step 1 of the wizard lists all
tables in the source schema, click on one to select it and then click ‘Next’.
On the following screen, the fields which define the link between the two
tables are specified. Right click in the Parent Field column to select from
a list of fields in the parent table which are the same type as the one in the
child table. There is also a ‘Clear’ option on the right click menu which
enables a previously defined relationship to be removed. It is possible
that more than one pair of fields is required to correctly defined the
relationship between the two tables. Once all file pairs have been
defined, click ‘Finish’ to add the table to the Data Case.
Any table can be removed from the tree by right clicking and selecting
the ‘Remove Table’ option. A confirmation screen is displayed prior to
the deletion taking place, please note that for parent tables, all children
will also be removed. Care should be taken when deleting tables because
if a constraint is broken, the data extraction may fail.
Data Selections
The purpose of creating the relationship tree above is to ensure that the
correct data gets extracted from the source to the target database with the
minimum of effort. For each table there are three options which control
the data that will be extracted. There are all available from the right click
menu for each table:
In the example below, a specific query has been defined to specify which
data should be extracted from the Customer table (OECUSTP), denoted
by the magnifying glass icon. Moving upwards through the relationship
tree, the Customer Type table (OECTYPP) has the default setting of Only
Required, denoted by an icon with one orange line in the grid. This
means that only the customer type records that are required for those
customers matching the selection criteria will be included in the target
database. The table OEUNITP at the top of the hierarchy is set to All
Possible, denoted by the fully orange grid icon. All records in this table
will be extracted.
Moving downwards through the relationship tree, the Order Header and
Order Lines tables are set to All Possible, this is reflected by the grid
which is half filled, half empty. All orders for the customers that match
the selection criteria will be extracted.
Note that although a table can appear in the relationship tree more than
once, its extraction criterion remains the same. Therefore when defining
selections it is sometimes easier to view the tables in a flat structure, this
is achieved by selecting the ‘Data Selections’ tab.
Each table in the Data Case appears in this list only once. The data
selections can also be modified from here from the right click menu.
Custom Queries
Specific data can be extracted from any table in the Data Case by defining
a Custom Query, accessed via the right click menu for the table or from
the Queries tab as shown below. This tab lists all custom queries in the
Data Case across all tables.
When a new query is being defined from the screen above, the following
panel is displayed. If however this is being done from the right click
menu for a table, this screen is bypassed as the table has already been
selected. Click on the table for which the query is being defined and then
click ‘Next’.
The selection criteria can then be defined on the Custom Query panel
shown below. In the query it is possible to not only use fields from the
table for which the query is being defined, but also any of its parents,
simply expand the list of fields by clicking on the plus icon next to each
of the parent tables.
To use a field in the query, click on it in the right hand list and drag it to
the desired location in the query definition. The first ‘Value’ entry for
any line in the query must be a field on the table, but the second ‘Value’
can be either another field or a specific value as shown above. Multiple
lines preceded by either ‘And’ or ‘Or’ can be defined and unwanted lines
can be removed by right clicking and selecting ‘Delete Line’. When the
query is complete click on ‘Next’.
Use the above screen to specify whether or not you wish to obtain all
possible data from the child tables of the table for which the query has
been defined. Either select the table individually or right click to select or
de-select them all. Please note that this will only affect tables which are
currently set to ‘Only required’, any other settings will not be modified by
changes made on this screen. Click ‘Finish’ to add the query to the Data
Case.
Queries can be removed in two ways. Right click the query on the
Queries tab and select the ‘Delete’ option. Alternatively right click a
table on one of the other two tabs and change to an alternative selection
method. An existing query can be changed by selecting the ‘Edit’ option
from the right click menu.
Click ‘Next’ to be taken to the confirmation screen and then click ‘Finish’
to begin the extraction. A progress bar is shown while the extraction is
occurring, when it is finished a message box is displayed as shown below.
If for any reason there was a problem with the extraction, an error
message is displayed on the Extract Data panel and this should be
reviewed before committing any changes to the target database. If
however the extraction was successful and you wish to add the records to
the target database, click ‘Yes’ on the above Data Extraction window.
Whether the changes are committed or rolled back, a record is added to
Data Case Results in TestBench-PC.
Right click any of the result lines and select the ‘Details’ option to view
more information about the run. Alternatively if these results are no
longer required, select the ‘Delete’ option.
The Summary panel displays more detailed status information about the
run such as target schema.
The Warnings panel lists any problems that prevented the extraction
completing successfully, often meaning that the run has a status of
Abnormal.
Click on the Tables tab to view the detailed statistics about how many
rows were extracted for each table.
Notes
Text Notes can be stored for an Action Map, Project, Test Case, Script
and Test Run and either viewed online or included on the test report.
Typical uses for Notes are:-
The text can be formatted with left and right margins, left adjust, right
adjust, centering, underline, capitals and bullets.
- Left Adjust The text on each line in the paragraph will start
in the left most position allowed by the margin
settings.
3 TestDrive-Green
Introduction
TestDrive-Green is part of TestBench, the only integrated testing suite for
the IBM iSeries. TestDrive-Green provides:-
System Requirements
Minimum Spec Recommended Spec
Pentium III 1.0 Ghz Pentium IV 2.0 Ghz
128MB Memory 512MB Memory
Getting Started
TestDrive-Green is installed when TestBench-PC is installed.
TestBench-PC provides a GUI interface into some of the features that can
also be accessed by TestBench on the iSeries. Once a validation code has
been registered, TestDrive-Green is launched by clicking on the
TestDrive-Green button at the bottom of TestBench-PC. See the
TestBench-PC User Guide for more information.
Toolbar Buttons
Menu Options
Note: This section only lists the options which have not already been
explained above.
File
Create Launch File Launch files are used for two purposes. They
support integration with Mercury Quality Center
and enable you to launch a particular Script
without going through TestBench-PC. To use
the file in Quality Center simply add it as an
attachment as you would any other PC file. To
launch outside of Quality Center simply right
click the file and select ‘Launch’. By specifying
additional options in the wizard that is displayed
launch files can also be used for unassisted
playback via a scheduler. See the section in the
TestBench-PC chapter for more information.
Script
Tracked Field History This provides a history of all of the tracked field
activity during the current execution of the
Script. The option is only enabled during
Playback.
Tools
Help
Product Options
The Product Options control global settings that affect all TestDrive-
Green connections on this PC and they remain in effect until they are
updated. Access this panel from the Tools menu, by clicking the Options
button on the toolbar or by right clicking the name of the currently
opened script. The following System options are supported:
Title Search The position on the screen where the screen title
can typically be found. Screens in the Script
will automatically be saved with this name.
Configure for DBCS data Increases the size of the text on the
screen view window and causes the
text in the tree view to be displayed
in the selected DBCS font. DBCS
characters contain more detail than
SBCS characters and if the scripts
contain a lot of DBCS data then
this option may make it easier to
read.
Prompt For Test Items Check this option to be prompted to select test
items at the beginning of a test run and to record
the status of the items at the end of the run. See
later section for more information.
Recording
Prerequisites
• Connection to a 5250 screen emulator session.
• Connection to an iSeries.
The iSeries system application under test to be at the correct screen
from which recording is to commence.
• Error messages must appear in the body of the 5250 display or they
will not be recorded. (There is an option on some emulators, for
example Client Access Express, to show error messages in the
status bar.)
Instructions
1 Press the Record button. You will be asked to confirm this choice.
2 After confirmation, focus will be placed on the 5250 session, which
you can operate as normal. Note, keystrokes and cursor movements
are not recorded, only the data when you press the next “Action On”
key (Eg. Function Keys, Enter, Page Up/Down), so don’t worry
about keying mistakes.
3 When finished, press the Stop button. Remember to always record
the exit from the last screen, e.g. F3, if you are recording integrated
scripts.
Notes
1 The Stop button can be used at any time, even when the 5250
session is in an Input Inhibited state.
3 No additional load is placed on the iSeries while a Script is being
recorded.
4 The time required to save a Script is based on many factors and
includes:
• The size of the Script.
• The current performance of the iSeries.
• The current network traffic.
5 If you are running TestDrive-Green on a very low powered or low
memory PC, you may see a slight delay between pressing the keys in
the 5250 session and their appearance on the screen. This is not a
problem and can, if required, be eliminated by upgrading the PC
used.
6. Until the script is saved, it will not be possible to store any results
following playback. Nor will it be possible to store attached input,
such as variable data, or use any tracked fields.
Script Structure
While a Script is open it will be shown in the format illustrated below,
expanded to the screen level.
Layout Options
The panes on the display can be moved to different positions and also
overlain. To move a pane, click on its title and drag it to the new
location. To overlay a pane, click on its title and position it over another
pane, a tab for each one will appear at the bottom of the pane. The panes
can be hidden by clicking on the Pin icon, when this is done a tab for the
pane will appear on the left hand side of the screen. To restore the pane
to the main view, click on the tab and then on the Pin icon.
TestDrive-Green can store three possible formats for this window. The
options can either be selected from the Tools menu or by right-clicking
on any of the orange window labels.
While both the Original and Advanced layouts can be manipulated, they
will continually return to their default setting so the Preferred layout
should be used to store any permanent changes to the view.
The following screen prints show the Original layout followed by the
Advanced layout.
Icons
The functionality for each of the panes is explained in more detail below.
Screens
All screens in the script are listed. Click on a screen in the list to show
information for that screen in all of the other Content panels.
Right click any screen in the list to obtain the following options.
Verify From Here This option enables a Script to be re-run and the
old screens to be automatically updated with the
contents of the actual screens when differences
are encountered. See separate Script
Verification section for more information.
View the details about any screen by placing the focus on the Screen
name.
Any fields with the ‘Do NOT test this field’ option checked on the Field
Properties window will appear in italics, which means that differences
between the actual and expected versions of this field on playback will
not be reported as errors.
Click on any field on the screen to highlight the details for this field on
the Fields and Input panels described below.
Double click any field on the screen to access Field Properties for the
field (see later section). Alternatively right click and select the
‘Edit/View’ option.
Fields
All individual pieces of information present on the screen are listed. The
tick or cross next to each field indicates the status of the ‘Do NOT test
this field’ option that can be found on the Field Properties window. A
cross means that differences between the actual and expected versions of
this field on playback will not be reported as errors.
This field is a data type field and has both an At Start value
and Keystroke value.
Click on any field in the list to highlight the details for this field on the
Screen and Input panels described above and below.
Double click any field in the list to access Field Properties for the field
(see later section). Alternatively right click and select the ‘Edit/View’
option.
Input
All input actions for the screen are listed, the final input in the list being
the key that was used to exit the screen. The progression from this screen
to the next will most commonly occur as a result of a function key or
other action key being pressed.
Click on any input in the list to highlight the details for this field on the
Screen and Fields panels described above.
Double click any item in the list (with the exception of the exit action) to
access Field Properties (see later section) from where the input can be
changed. Alternatively right click and select the ‘Edit’ option. For the
exit action, the Screen Options panel is displayed from where the exit key
can be changed.
Variable Data
All variable data fields in use in this script are listed on this screen in
bold. Beneath each field is the screen number and name on which this
variable data field is in use, of which there could be more than one.
Beneath the screen name are listed the start values of the fields for which
the variable data will be used. For example, the variable data field ‘Cust
Num’ is in use on screens (2) and (5). Click on any screen or field in the
list to highlight the details for that screen or field on the other Content
panels.
Edit Variable Data Maintain or view the variable data set associated
with this script.
Tracked Fields
All Tracked Fields in use in this script are listed on this screen in bold.
Beneath each field is the screen number and name on which this tracked
field is in use, of which there could be more than one. Beneath the screen
name are listed the start values of the fields for which the variable data
will be used. See later Tracked Fields section for more information.
Click on any screen or field in the list to highlight the details for that
screen or field on the other Content panels.
Thumbnails
This panel enables small images for each screen to be viewed in a
scrollable list to assist with navigation through the script. For
performance reasons the thumbnails are not created automatically when
the script is opened, the first time the panel is accessed after a script has
been loaded the following screen is displayed.
Once the icon has been clicked, the view is expanded to show the
following display. Click on any screen image in the list to show
information for that screen in all of the other Content panels.
Field Properties
Field Properties can be accessed by double clicking a field on the screen
display or field list view or by right clicking and selecting the Edit option.
Keystrokes are not valid for constant type fields and cannot be seen on
this screen.
Start
You have the ability to change what value will be used for this field for
verification when the Script is executed and the screen containing this
field appears. If this value does not appear then an error will be reported.
Fixed Value Use this option to check the start value against
the characters in the adjacent box.
Field Error Actions If this field is in error the Script can either be
paused, at which point the option to continue is
available, or aborted. These values can also be
set at the screen level and by default any screen
actions will be followed.
Do NOT test this field If the field is not to be checked for differences,
it will appear in italics on the graphical screen
display and this will also be indicated with a
cross on the field list.
Keystrokes
You have the ability to change what keystrokes will be sent to this field if
it is a Data field rather than a Constant. Care must be taken with these
values as they may affect what screens will subsequently appear. The
Keystrokes tab is not shown for Constant fields.
Track
Use these options when you wish to use either the ‘At Start’ or
‘Keystrokes’ values of a screen field to populate a Tracked Field. This
Tracked Field can then be used elsewhere in the Script to populate other
screen fields with the stored value. See separate section on Tracked
Fields for more information.
Store Start Value In Select a Tracked Field into which the initial
value of this field will be stored. To modify any
Tracked Fields, click on the Edit button.
Store Whole Field Place the entire contents of the screen field into
the named Tracked Field.
Store Part Field Place only the specified section of the screen
field into the named Tracked Field.
Store End Value In Select a Tracked Field into which the final value
of this field will be stored. To modify any
Tracked Fields, click on the Edit button.
Location
These options can be used to change the length, screen position and
colour of fields on the screen.
Screen Options
You can access the Screen Options by double clicking the Screen name in
the tree view.
General Options
Final cursor position The position of the cursor when the screen was
ended. Use in conjunction with function key
option when this action is cursor sensitive. For
example, pressing F4 on different fields may
yield different results.
Take the action/ Select a value between 0 and 999 and use this to
If errors exceed force an action to be taken should the number of
errors at a screen level surpass the number
specified here. The script can either be paused,
at which point the option to continue is
available, or aborted.
Play / Pause Options Normal: Play back all the keystrokes for the
screen with no delay between them.
Pause at start: Suspend playback at the start of
the screen before any keystrokes have been
played. At this point playback can either be
halted or continued.
Step: Before playing each keystroke TestDrive-
Green will prompt to confirm whether the
keystroke should be played back or not.
Fine Step: Similar functionality to ‘Step’
except that it provides a higher level of detail.
If one of the step modes (Step or Fine Step) is selected, when the screen
is reached the following form will be displayed.
Do not play keystrokes Skip these keystrokes and continue to the next
keystrokes that are available for the screen.
Change step mode Switch from fine step to normal step or vice
versa.
Column Input
When to use column input
The column input feature may be set up and activated screen by screen in
a script. If activated, the normal input (whether keystrokes, variable data
or a tracked field) will still be played except for the input specified under
the column input tab.
For example, if you need to amend an order number ‘000123’ with option
‘2’ from a list of orders for that customer, you can search and find the
order and continue processing by playing the input value of ‘2’ into the
appropriate field on the same line.
1. Identify the data or value which the script will need to search for. If
fixed, this can be entered as is, if variable use either variable data or
tracked fields to store the value you are looking for.
2. Access the screen options window and select the Column Input tab.
3. Check the check box to enable Column input
4. In the search for grid, find the column heading (these are determined
automatically for sub-files) which represents the column in which the
value you are looking for will be contained. Select the type of value
and the value or field containing the value.
5. In the playback section, select the field into which input is to be
performed and similarly to above define what the required input
should be.
6. In case the row you are looking for is not found, you can specify a
‘not found’ action. Typically this will be page down together with a
maximum attempts at this action.
Use Column Input If checked then Column Input will be used for
this screen. If not checked then the Column
Input information will be retained but the
standard keystrokes for the screen will be
played.
If not found play It is possible that the value you are searching for
is not on the screen. This option enables you to
press a Function key a specific number of times
to try and find the value.
Before playing key Use this option if the cursor needs to be moved
to a specific position before playing the
Function key.
Script Options
First/Last Transaction For Scripts which use variable data, specify the
first and last transaction in the data set to be
used during playback.
Tracked Fields This button will allow any Tracked Fields that
are associated with the script to be edited. See
the separate section on Tracked Fields.
Match Field Length If a field has changed in length but its contents
remain the same, leaving this option unchecked
will not report this as a difference during
playback. However, checking this option will
cause a difference to be reported.
Displays the name of the current Script along with the Project and Test
Case in which it is stored.
Exceptions
These Script and Global (system) level exclusions are an important
addition or alternative to that achieved by excluding individual fields.
For example, the time of day may appear on every screen in a Script. One
option would be to tune each screen to ignore any differences in the time,
but Script level exclusions allow this to be achieved via a single entry.
The Global exclusions are displayed here for information purposes only.
These must be updated on the iSeries by taking option ‘4’ on the System
Values screen. This can be reached by taking option ‘51’ from the
TestBench Main Menu and then keying option ‘21’ for System Values.
Start Line The start line in the 5250 session line range to
examine fields that start in the range for
exclusion data. This can range from 1 to 27.
End Line The end line in the 5250 session line range to
examine fields that start in the range for
exclusion data. This can range from 1 to 27 and
must equate to, or exceed the End Line.
Compare Data For Item exclusion types, this can be any value.
For Date exclusion types this must be ‘/’, ‘:’ and
‘.’. For Time exclusion types this must be set to
‘:’, ‘/’ and ‘.’. For *Any exclusion types this
can be blank, in which case all fields beginning
in the line range will be excluded.
Alternatively, if a value is specified only fields
containing this character string will be excluded
from the comparison.
Notes
1. The Date exclusions only apply to fields that are 8 or 10 characters in
length and have the Data character (e.g. ‘/’) in the format nn/nn/nnnn
or nnnn/nn/nn where all the other characters are numeric apart from an
allowed leading blank.
2. The Time exclusions only apply to fields that are 5 or 8 characters in
length and have the Data character (e.g. ‘:’) in the format nn:nn:nn
where all the other characters are numeric apart from an allowed
leading blank.
Amending Scripts
It is important to remember that TestDrive-Green only records the iSeries
screens on the pressing of the ‘Action On’ key or keying into an auto
record advance field and therefore keystroke mistakes made during the
entry of data when recording a screen are NOT captured. This leads to
fewer Script amendments being required.
In the event that changes occur to the system under test for which scripts
already exist, it is possible to easily change these scripts.
If a system application screen has been changed so that new fields added
have resulted in the original screen being split into two screens then the
originally recorded screen can be deleted and two new screens inserted in
its place. Alternatively the Screen Replacement or Script Verification
functions can be used.
Deleting a Screen
To delete a screen from an existing Script:
1 Open the Script that requires changing and click on the relevant
Screen name.
2 Press the ‘delete’ key on the keyboard. You will be asked to confirm
that deletion is required.
3 On confirmation the screen will be deleted.
4 Save the Script.
Care must be taken in the event that Top and Bottom Loops have been
specified within the Script, especially if the screen being deleted is
marked as either one (see Variable Data section).
Inserting a Screen
To insert a screen into an existing Script:
1 Open the Script that requires changing and optionally click on the
Screen name before the point at which you want to insert a new
screen.
2 Ensure that the system application under test is on the correct screen
for recording.
3 Press the Record button. The following screen will be displayed.
Care needs to be taken to use the same data starting point as the screens
recorded previously in the Script.
Screen Replacement
Screen replacement is performed by right clicking a screen in the list and
selecting ‘Replace Content’ from the menu. TestDrive-Green must be
connected to an iSeries session for this function to be enabled.
screen to the new screen and then the old screen is replaced by the new
one. If one or more fields cannot be successfully mapped, the screen
display form is opened and the fields that could not be matched are
highlighted as shown below.
More information about how field mapping can be viewed and modified
and the rules which govern field mapping can be found in the later Match
Rules section.
The user has the option of ignoring the fields (screen will be replaced but
no data for these fields will be mapped), manually specifying a mapping
for the unmapped fields or canceling the replacement.
Manual Mapping
If a field cannot be auto-matched it is possible to define the correct
mapping for the field manually. This only needs to be done for fields
where keystrokes, tracked fields or variable data definitions need to be
transferred to a field on the new screen. This option can also be used to
change the mappings for fields that have been incorrectly matched by
TestDrive-Green
• Right click on the new field that should contain the properties of
the old field and select the ‘New target’ menu option.
Alternatively select the ‘Cancel’ option if you do not wish to
manually map the selected field.
Script Verification
A verification run is started by right clicking a screen in the list and
selecting ‘Verify From Here’. The confirmation screen that is displayed
allows the user to control the conditions under which playback will halt to
request manual intervention. One of three possible conditions for halting
playback can be specified.
If the decision is made that the Script should be changed, one of the
following actions must be chosen:
• Replace
• Ignore
• Insert
• Delete
Insert is intended to deal with screens that have been added to the
application since the Script was recorded. The Delete option removes
unwanted screens. When a screen is removed the playback/verify action
is repeated on the next screen in the Script. When a new screen is added
the screen definition must be edited to specify any keystrokes for fields
which need them and to specify an exit action for the screen. A screen is
generally ended by playing an action key but it is also possible to leave
the screen by keying values into an Auto-RA (Record Advance) field or
by typing a keystroke into a non-input-capable area of the screen. The
drop-down list box labeled ‘Exit Action’ allows either of these options to
be specified. Selecting one of these options will display a separate form
where the location of the field or the keystrokes can be specified.
TestDrive-Green will then add the screen definition to the Script before
playing back the new screen and will repeat the validation routine against
what was the current screen.
Any fields on the screen can also be manually mapped - please refer to
this paragraph within the previous Screen Replacement section for more
information.
Match Rules
The Match Rules are used when TestDrive-Green attempts to map
expected to actual fields within the following functions:
• PowerPLAY
• Screen Replacement
• Script Verification
The setting of field mapping conditions can be changed from the System
tab in TestDrive-Green Product Options or when playback has been
interrupted in Verify mode to display the error screen. The following can
all be specified as match conditions which mean that the property must be
the same on the expected and actual fields for a match to be identified.
Const/Data Data fields are those which can accept input whereas
constant fields are display fields only.
Label
A label is the piece of text associated with a field. Label matching is
probably the most flexible way of matching expected and actual fields
and is particularly useful for matching the keystrokes, variable data and
tracked fields which are associated with data fields. Since the fields on a
green screen do not have accessible names, the matching process
generally relies on the label to match the actual and the expected field,
although in some cases (e.g. if the data fields are populated with values) it
may be possible to rely on the start value for matching.
Title: ---
Title: ---
TestDrive-Green will still be able to play back the keystrokes for the field
correctly because, even though the data fields have moved they still have
the same positions relative to their labels.
If the field is a constant then the label is the start value of the field. If the
field is a data field the label is calculated as follows:
• Look to the left of the field. If the next field found is a constant
then the label is the start value of the constant.
• If no constant is found to the left then look at the field directly
above.
• If there is no field, the data field does not get a label.
• If there is a constant field above the data field, the label of the data
field is the start value of the constant.
• If there is a data field above the data field, then the data field which
is being assigned acquires the same label as the data field above it.
As well as the label text, a field acquires a label offset value. If the field
is a constant the offset value is zero. If the field is a data field, if the
constant that provided its label is on the same row, then the label offset
value is also zero. If the constant is on a different row, the label offset
value indicates the number of rows between the data field and the
constant that defines its label. In other words, the label offset value
defines the positional relationship between a field and its label. When
TestDrive-Green matches a field by ‘label’ both the label text and the
label offset value have to match.
All other fields within this vertical range which form into continuous
columns are identified as being part of the subfile and the labels they are
given are always derived from the column name.
This matching process works well where the subfile is uninterrupted, but
may work incompletely or not at all where the subfile is interrupted by a
window. In such cases manual intervention will be needed by the user to
decide whether the fields need to be manually mapped or if the
differences can simply be ignored.
Playback
Prerequisites
• Connection to a 5250 screen emulator session.
• Connection to an iSeries.
• The Script you are about to replay has been opened.
• If using an integrated Test_IT run, make sure that TestBench is on the
Main Menu.
Instructions
PowerPLAY
PowerPLAY allows an existing Script to be played back against an
application that has changed, therefore enabling testing during the early
parts of the development cycle where fields may move around on the
screen and frequent cosmetic changes may be made.
There are two fields on the playback tab of the TestDrive-Green Options
panel that control how PowerPLAY operates. These options both affect
playback and can be applied independently of each other.
• PowerPLAY Active
• Halt on Unmatched Input
Each time the warning is generated the user needs to decide whether to
continue or stop the Script. If Continue is selected, TestDrive-Green will
try to play on without sending the keystrokes for the field. If Stop is
selected the script playback will halt and the Keep/Discard buttons will
appear.
Test Items
Test Items which have been created for a Test Case can be accessed via
TestDrive-Green, and the status updated and comments added. The
following window appears after the Script Options when the Play button
is pressed and provided the option has been selected in the TestDrive-
Green Options window. The list of documented items for this run can be
tailored either by left/right clicking individual items, or by using the two
buttons to select or de-select all items. New Test Items can be added or
removed for this run only.
If not performing an Integrated run and Test Items have already been
selected during Test Case initiation in the 5250 session, any selections or
alterations made now will replace those made earlier. Any changes are
stored back on the iSeries at the end of the run when the results are saved.
Once the playback of the script has been completed a similar window is
displayed which enables the status and comments to be entered (see later
Pass/Fail Results section).
Results
Once the execution of a Script is complete or the execution has been
interrupted by the user, a panel in the following format is displayed:
Each screen which was presented during the execution of the Script is
listed together with a symbol indicating:
Green tick All Data and Constant items that were actually
presented matched your expectations as defined
in the Script or expanded through Variable Data
or Tracked Fields.
View the details about any screen by clicking on the screen name in the
list. The results window can have three possible formats depending on
the layout option that is selected:
The following screen prints show the Original layout followed by the
Advanced layout.
Icons
Click on any field on either of the screens to highlight the details for this
field on the field list panels.
All fields present on the Actual and Expected screens are listed with a
green tick indicating a match between the two and a red cross indicating a
difference. The Errors view lists the Actual version of only those fields
with a red cross. The highlighting of differences is affected by the
checking options in place.
Double clicking any field on the Expected list will cause the Field
Properties panel to appear in view only mode (see later section).
Alternatively right click and select the ‘Edit/View’ option.
Click on any field in the list to highlight the details for this field on the
screen images.
Pass/Fail Results
If the option has been selected in the TestDrive-Green Options window -
see earlier section TestDrive-Green Options - after clicking on the button
to save results, a window will be displayed so that the status and
comments can be recorded for each Test Item selected at the start of the
run. The list of selected Test Items can also be changed here.
Variable Data
Variable Data allows you to record shorter, more flexible Scripts and is
thus a key feature of TestDrive-Green.
For example, let’s imagine that you want to stress test the ‘Add a
Customer’ function. One option would be to record a Script that adds a
single customer and then set the Script Option, ‘Number of Runs’ to 100.
However, there is a problem with this approach. The Script would
successfully add the first customer, but all the following 99 attempts
might be rejected with a ‘Customer already on file’ error message as the
Script uses a fixed value for the Customer Number.
Without Variable Data the only alternative is to record a Script that is one
hundred times longer and contains the details of one hundred different
customers. This would take much longer to record and be almost
impossible to maintain if the screen layouts for the ‘Add a Customer’
function were changed at any time in the future.
Variable Data allows you to feed selected Data and Constant fields with
external information, rather than using the fixed value that was recorded.
You can therefore record a Script that adds a single customer but then
select the Customer Number and perhaps the Customer Name to be fed
not as the record constant, but by fields.
Local Variable Data is stored with the individual Script and can only be
used by the one Script.
Public Variable Data Sets are stored at the Project level and can be
utilised by any Script within the Project.
2. Both Local and Public Variable Data Sets can be accessed from the
Script Options panel in TestDrive-Green - see screen print below.
Right click on any of the Variable Data Sets in the list to obtain the
following options:
Right click on the single grey box in the top left of the Edit Variable Data
window to begin creating variable data fields and transactions for new
Variable Data Sets. The following options are available.
occurred.
Column Options
When at least one field has been added some additional options are
available on right click on the field name column headings. This screen
is cursor sensitive, the options listed apply to the field on which the right
click occurred.
Row Options
When at least one row has been added some additional options are
available on right click on the transaction numbers. This screen is cursor
sensitive; the options listed apply to the row on which the right click
occurred.
In addition to the right click options, rows can be moved simply by left
clicking the transaction number and dragging to the desired new position.
Multiple rows are selected by using left click in combination with
Shift/Ctrl.
Data Options
This screen is cursor sensitive; the options listed apply to the cell on
which the right click occurred. Multiple cells are selected by using left
click and dragging the cursor.
=date (dd/mm/yy)
A - is valid in addition to a +.
The valid operators are day, days, month,
months, year, years. The expression is not case
sensitive.
Playback
When the playback of a Variable Data Script is selected, the Script
Options panel enables the starting and ending Variable Data transaction
numbers to be selected. If these are left blank then all transactions will be
processed. This screen also enables any Variable Data fields containing
blank values to be ignored.
Tracked Fields
Tracked Fields enable system generated values to be stored and re-used
later in the Script. This has two benefits. Differences during playback as
a result of these unpredictable values can be avoided. Also, these values
can then be used as input on subsequent screens, whereas without
Tracked Fields testing would need to be paused while the correct value
was added to the screen in question.
Script Security
If security has been activated within TestBench then the access for a
specific User to Scripts within TestDrive-Green may be restricted. There
are several authority states which are indicated by the text at the bottom
left of the TestDrive-Green panel.
View The User can view Scripts within this Project but
cannot change them or play them back.
Execute The User can view and play Scripts but cannot
change them.
All authority The User has full access to the Scripts within this
Project.
4 TestDrive-Gold
Introduction
TestDrive-Gold comes with a simple, easy-to-learn interface, behind
which lies a powerful leading edge Windows and Web testing solution.
TestDrive-Gold is designed to deal with standard Windows controls and
any application which has implemented the Microsoft Active
Accessibility Standard, including languages such as .NET, Visual Basic,
Delphi, Visual Lansa, Coolplex, JWalkTM or typical ‘screen-scrape’
products. The major elements - record and playback, maintaining &
editing scripts, storage, analysis and presentation of results - have all been
designed to offer maximum efficiency and functionality with minimum
complexity. With TestDrive-Gold, you will speed up the application
development process immediately - and improve quality dramatically.
And you do not even have to learn a scripting language!
Platforms
The TestDrive-Gold repository can run on the following platforms:-
Browser Requirements
TestDrive-Gold can be used with Microsoft Internet Explorer version 5.5
or later with a recent service pack. No other web browser products are
supported.
System Requirements
Minimum Spec Recommended Spec
Pentium III 1.0 Ghz Pentium IV 2.0 Ghz
128MB Memory 512MB Memory
Getting Started
TestDrive-Gold is installed when TestBench-PC is installed. TestBench-
PC provides a GUI interface into some of the features that can also be
accessed by TestBench on the iSeries. Once a validation code has been
registered, TestDrive-Gold is launched by clicking on the TestDrive-Gold
button at the bottom of TestBench-PC. See the TestBench-PC User
Guide for more information.
Toolbar Buttons
Menu Options
Note: This section only lists the options which have not already been
explained above.
File
Create Launch File Launch files are used for two purposes. They
support integration with Mercury Quality Center
and enable you to launch a particular Script
without going through TestBench-PC. To use
the file in Quality Center simply add it as an
attachment as you would any other PC file. To
launch outside of Quality Center simply right
click the file and select ‘Launch’. By specifying
additional options in the wizard that is displayed
launch files can also be used for unassisted
playback via a scheduler. See the section in the
TestBench-PC chapter for more information.
Script
Playback.
Tools
Help
Options
The Options control the settings that affect the recording and playing
back of scripts using TestDrive-Gold. Access this panel from the Tools
menu, by clicking the Options button on the toolbar or by right clicking
the name of the currently opened script.
General
The options held within the General tab are the default options that will
be applied to all new scripts and are stored with the script when it is saved
for the first time. Once the script has been created they can be viewed by
right clicking the script name and selecting the Edit option. They are
separated into three tabs. The first three sections, Analysis, Take picture
on and Wakeup on, can be viewed but cannot thereafter be changed for
that script. The latter three sections, Busy sense, Alerts and
Miscellaneous can be changed to alter the settings that will be applied
when the script is played back.
General Tab
Playback
Analysis
This first set of options controls the level and nature of the information
that is recorded.
Take picture on
These options determine at which points during the test a new picture is
recorded of the target application. TestDrive-Gold monitors for activity
within the target application and takes a picture when this activity has
finished and the application goes “quiet”. These options control which
types of activity are included in this process.
Hide child element Similar to the above option but a new picture is
taken when the elements are hidden.
Active window changes If the active window (the window that currently
has the focus) has changed a new picture is
taken. For example, the main application
window and the About window are open and
you click between the windows without closing
either one.
Resize child element Similar to the above option but a new picture is
taken when the elements are resized.
Move child element Similar to the above option but a new picture is
taken when the elements are moved.
Wakeup on
hence no new picture will be taken if they result in any screen changes
occurring. Therefore the wakeup options can be used to monitor for these
types of activity and take a picture when the application becomes quiet
again. For performance reasons the last three options are left unchecked
by default.
Busy sense
Alerts
Miscellaneous
5250 Tab
These options are used by TestDrive-Assist when it is monitoring 5250
applications.
Title Position/Line The position on the screen where the screen title
can typically be found. Screens in the Script
will automatically be saved with this name.
Screen Analysis
TestDrive-Gold has many options which enable it to record and playback
scripts and analyse the content of screens for applications written in a
wide range of languages. It uses Microsoft Active Accessibility (MSAA)
and Class Rules to define how to interact with the elements on screen and
the Messages help to control when screen pictures will be taken.
If you are using TestDrive-Gold for the first time over a new application
you are likely to achieve a good result because by default it uses
Microsoft Active Accessibility to analyse screen contents. This means
that TestDrive-Gold will enumerate the contents of every screen using
Microsoft Active Accessibility and every control which implements this
standard will be analysed without the need for any additional setup. If
there are then any specific window components which do not support this
standard, Class Rules can be created to retrieve the data by sending and
receiving Microsoft Standard and Common Control messages (MSCC).
Any controls which do not support either of these techniques will be
recorded as a panel and their contents will not be retrieved. In summary:
Class Rules
When TestDrive-Gold is recording and playing back scripts and the
MSAA standard cannot be used, it looks at the names of the windows
components on each screen and must convert these to an internal type so
that it knows how to interact with the component and obtain the
information that is listed on the Content screen. It does this by obeying
the Class Rules listed below. For example, a class name of
“Thunder*Option Button” will be treated as a Radio Button.
Filter
It is possible to filter via the drop down lists as displayed in the screen
below, for example by Description or Type. Once a filter has been
chosen, click on ‘Apply’ to save the filter. Click on the ‘Hide’ button to
return to the full Class Rules display.
Options
The new rule will be appended to the end of the list of Class Rules by
clicking on the Accept button. The sequence of the Class Rules can be
altered using the up and down arrow buttons.
Pattern Matching
The following characters are allowed for Class Rules that use pattern
matching:
Character Matches
? Any single character
* Zero or more characters
# Any single digit (0-9)
[Charlist] Any single character in Charlist
[!Charlist] Any single character not in Charlist
If the class name that you want to match includes any of the special
characters above, you must include it into brackets.
i.e.: Dialogs have class #32770. The class rule for dialogs would be
[#]32770.
The order of the Class Rules is very important and the more specific rules
must be higher up in the list.
When rules are exported they are added to a file with the extension .rul,
the location of this file can be selected by the user at the time of export.
All Class Rules will be added to the file but if only some rules are
required the file can be opened using an editor such as Notepad and
modified.
When the Import option is selected you are prompted to locate the file
containing the Class Rules. If the format of any of the rules being
imported is invalid an error message will be displayed, the rule must then
be corrected prior to trying the request again. Once all errors have been
fixed a message box containing the number of new rules that will be
added and the number of existing rules that will be updated is displayed,
the Class Name is used to identify a matching rule. At this point click on
Cancel to stop the import or Proceed to continue. The changes to the
Class Rules will not however be stored in the database until the Accept
button is clicked on the Class Rules screen.
Messages
TestDrive-Gold knows when to take a new screen picture by listening to
the application under test and monitoring for when it becomes “quiet”,
i.e. there is no message or CPU activity. Some applications send certain
messages constantly and therefore these need to be ignored. These
messages are defined in the following table which should only be
modified with assistance from your support contact.
Validation Rules
By default when a script is re-played in TestDrive-Gold, every screen
which actually appears is compared against the expected screen stored in
the script and any differences are highlighted. Validation Rules provide
an alternative to this method in situations where either there is no single
correct answer, or the correct answer might be different dependant on
other variables. They also enable more complex checks to be performed
on the information that is displayed on screen.
The following options are available from the right click menu.
Validation Functions
Validation Functions are used to perform more complex processing that
cannot be easily achieved by the Validation Rule wizard. They enable
small programs to be created, the results of which can be used by a
Validation Rule to compare against a screen field in order to validate it.
For example, a Function could be created to calculate today’s date or to
retrieve the correct salesperson code for a specific order number. There
are two types of Validation Function. A Custom Function must be
entirely created by the user and keyed into TestDrive-Gold via a VB
script. Alternatively an SQL Function can be created using a wizard
which guides the user through checking values in a database file.
All of these Functions are created at the Project level and displayed on the
screen below. They can then be used by any scripts within that Project.
See the later section for more information on using Validation Functions
within Validation Rules.
The following options are available from the right click menu.
Custom Functions
Custom Functions enable the user to write a VB script to perform more
complex processing that is not possible with a simple Validation Rule.
When a Custom Function is created or edited the following screen is
displayed.
SQL Functions
SQL Functions enlist a wizard to assist the user in performing a check
against a database table. When an SQL Function is created or edited the
following is the first screen to be displayed.
Click Next to define the table from which you would like to retrieve a
value, the table to be interrogated. Click on a file as shown on the screen
below in order to select it or key in the name of the file at the top of the
screen.
Click Next to define how the data should be retrieved from the file by
specifying the parameters that should be used for the selection.
There are three possible values for the Include option for each field in the
file.
Blank This field will not be used for the data selection.
Click Next to define which information will be retrieved from the file
being validated. Click on the field whose contents will be returned as the
result of the SQL Function and which can then be used within a
Validation Rule.
There are five possible values for the option which determines how the
return value should be treated.
Just return the value The contents of the selected record and field will
be returned. In the above example this would be
the First Name of the salesperson being
retrieved. If there is more than one record
which matched the selection, the contents of the
first one is returned.
The Function definition has now been completed but can be modified at
any time using the Back and Next buttons. Clicking Next on the above
screen will display the final screen shown below, from where it is
possible to run a test on the SQL Function to ensure that it is working as
expected prior to plugging it into a Validation Rule.
All of the parameters which have been defined as ‘Runtime’ are listed
and for each one a test value must be keyed. At this point click the ‘Test’
button. The file will be interrogated and the results returned. If there is a
problem returning the desired results a red cross and an error code and
reason is displayed.
Click Finish to store the Function in the database at the Project level.
Recording
Instructions
1. Ensure that TestDrive-Gold does not currently have a script open and
then click the Record button on the toolbar. The Record Wizard is
then displayed.
3. Click the Start button to begin recording, at which point either focus
will be placed on the selected application or the specified application
will be launched and all activity recorded. Alternatively click Cancel
to end the recording process.
5. When you have finished recording, press the Stop button. Click ‘No’
to continue recording further actions in the application or ‘Yes’ to end
the recording process and return to the main TestDrive-Gold display.
6. At this point the script has not yet been saved. It can be modified and
re-played, however any features such as variable data and tracked
fields which require Project specific data to be retrieved cannot be
utilized until the save has taken place. Also the results generated by
the execution of the script cannot be stored.
Notes
- Windows APIs
- Ajax
- Web Navigation
• If a control has a scroll bar, the action of clicking and holding the up
or down arrows to scroll the contents list will be re-played in real time
as the time delay between the down and up click is recorded.
However this may not cause exactly the same result on playback due
to differences in application response times. Therefore whenever
possible it is preferable to use an alternative method to scroll the list,
for example drag the rocker button, use single clicks on the up and
down arrows or click within the scroll bar to move one page at a time.
• If Alt/Tab is used while recording to access another application, only
the Alt is recorded because after this point focus is no longer on the
application being recorded. This will not replay correctly and
therefore this action should be avoided while in record mode.
• If a browser window is maximized or minimized while in record
mode, this action is not repeated on playback and therefore if a
specific browser size is required it should be set prior to initiating the
record process.
• When an adobe document is launched from a browser, there is a
setting inside Adobe which determines whether the document is
hosted inside a browser, or simply hosted inside Adobe Reader. Only
PDF documents hosted inside a browser can be recorded and analysed
correctly. Also, only Version 7 or later of Adobe Reader is supported.
• Please note that when recording on Windows Vista, there is a specific
animation function which must be switched off to enable TestDrive-
Gold to work correctly. If this option is switched on, a message is
displayed. The following steps can be followed to disable the option.
There are two types of drag and drop, the Win32 implementation and the
OLE method. The first is fully supported but there is currently no support
for the latter.
Script Structure
While a Script is open it will be shown in the format illustrated below.
Layout Options
The panes on the display can be moved to different positions and also
overlain. To move a pane, click on its title and drag it to the new
location. To overlay a pane, click on its title and position it over another
pane, a tab for each one will appear at the bottom of the pane. The panes
can be hidden by clicking on the Pin icon, when this is done a tab for the
pane will appear on the left hand side of the screen. To restore the pane
to the main view, click on the tab and then on the Pin icon.
TestDrive-Gold can store two possible formats for this window, which
can be selected by right-clicking on any of the orange window labels.
Script
All screens in the script are listed, the + button to the left of each screen
can be used to expand the view to also show the input for the screen.
Right click any screen or input in the list or the script name to obtain the
following options. The options displayed depend upon the type of
element selected.
Loop Point Select this screen as the start or end loop point
for variable data transactions, which causes a
red down or up arrow to be added to the screen
icon. See later Variable Data section for more
information.
Click on a screen in the list to show information for that screen in all of
the other panels. Click on any input to display an orange box around the
Picture
An image of the recorded screen is displayed. If an input action or
element in the Contents list is highlighted, the target element is
surrounded by an orange rectangle on the display. Conversely, clicking
an item on the picture will also highlight that element in the Contents list.
There are 3 buttons at the top left of the panel that determine how the
picture is displayed.
Two additional icons above the screen image control the mode that the
panel operates in.
Markups
You may wish to annotate specific areas of any of the recorded screens,
for example to highlight information which is incorrect and needs to be
changed, or for training or documentation purposes. To do this, click on
the outline icon above the screen picture. Then use the mouse to drag a
box around the area of the screen that you wish to add a comment about.
When the mouse is released, a comments box is displayed as shown
below.
Click Accept to add this markup to the screen. The markup is shown
below the screen and clicking on it will highlight the selected area of the
screen and fade out the rest. Click anywhere else on the screen to return
to the full view.
The following options are available from the right click menu for a
markup.
Any screen which contains a markup will have its screen icon overlain
with a dotted box. If any of the markups have been flagged as being of
high importance, the box will contain a red exclamation mark.
Contents
The elements included on the highlighted screen are presented in list
form. Click on an element to highlight it on the screen picture with an
orange rectangle and to populate the Expected Element Properties panel.
The Filter button on the main toolbar can be used to modify the
appearance and contents of the element list. See the Main Panel section
near the beginning of this document for more information.
Right clicking an item in the list displays a floating menu with two
options. The ‘Edit’ option displays the Edit Element screen, see later
section for more information. ‘Checking’ enables the element to either be
checked or not checked for differences on playback. If an element is not
being checked it is displayed with a yellow cross icon in this list.
The properties that appear in bold are those that will be checked for
differences on playback. These are different for each element type and
are considered the core properties for that type. The only non-core
properties that can be optionally checked are the size and location, this is
controlled via the TestDrive-Gold options.
Edit Element
Right click an element either on the screen picture or within the Expected
Contents list and select the ‘Edit’ option in order to display the following
screen.
Use this tab to store the contents of this element for use later in this script
or in another script in an Action Map. See the section on Tracked Fields
for more information.
Store Start Value The initial contents of this field when the screen
was first displayed will be captured and stored
in the specified tracked field.
Store End Value The final contents of this field after all
keystrokes have been replayed will be captured
and stored in the specified tracked field.
Track Field The name of the tracked field which has been
defined at the Project level and which will be
used to store the contents of this screen element.
Store Whole Field The entire contents of the screen element will be
stored.
Store Part Field Only the specified subset of the screen element
value will be stored.
Element Caption The label associated with the target element for
the input. For example, for a button this could
be the visible text such as ‘Accept’ or ‘Cancel’,
for a text box it might be the label which
appears to the left of it.
Edit Input
The following display is accessed by right clicking an input action on the
Script panel and selecting the Edit option, then clicking on the Input
Properties tab. Please note that existing input can be changed or removed
but new input cannot be added.
Keyboard States Defines if the Ctrl, Alt or Shift keys were active
Play Dynamic Input This option is only visible for input that is
considered to be dynamic. This is input over
elements that were not present on screen when
the analysis took place and therefore are not
included in the Expected Contents list. For
example, combo box items or drop down menu
items.
Monday
Tuesday
Wednesday
Thursday
Monday
Wednesday
Thursday
Pause options The default value for this option is ‘Do not
pause before playing’. However two other
options enable the playback of this input to be
delayed. ‘Pause until user confirms’ will
display a dialog box at playback time and the
input is not played until the message is cleared.
Alternatively the user can ‘Delay playing this
item for’ a specified number of seconds.
The properties of the target item for the selected input can be displayed
by clicking on the Target Element tab.
Summary
The Script or Screen Summary panel has several sections within it, each
one is described in more details below. The title of each section for
which information exists is highlighted, headers for other sections where
no data exists are not.
Tracked Fields
All Tracked Fields in use in this script are listed on this screen. Beneath
each field are listed the screen elements for where the tracked field is
being used. See later Tracked Fields section for more information. Click
on any field in the list to highlight the details for that field on the other
Content panels. Click on the ‘click here’ link to define new or modify
existing tracked fields for the Project.
Variable Data
All variable data fields in use in this script are listed on this screen.
Beneath each field are listed either the input values or the start values of
the fields for which the variable data will be used. Click on any field in
the list to highlight the details for that field on the other Content panels.
Click on the ‘click here’ link to manage the Local and Public variable
data sets.
Validation Rules
When the script name is in focus, this panel lists all of the Validation
Rules defined for the entire script. There are two levels of Rules:
• Tracked Field. These are effectively defined for the whole script,
but will only be applied to screens where at least one of the tracked
fields utilised by the rule is populated on that screen. If none of
the tracked fields are populated then the Rule is bypassed for the
screen. Therefore the Rules at the Tracked Field level are only
applied when a relevant tracked field changes.
The following options are available from the right click menu.
When a specific screen is in focus, only the Rules for that screen plus any
Tracked Field Rules are listed. Tracked Field Rules that will not be
applied to the screen because none of the tracked fields are stored there
will have their icon greyed out.
When either the Add or Edit options are taken, the following screen is
displayed. Please note that each column is only enabled once all previous
Blocks
There are occasions when the key goal is to match a ‘block’ of similar
data elements within a web page, not necessarily the individual items
themselves. For example, when viewing a long list of products where
sometimes a new product is added, it is necessary to ensure that on
playback all of the details of the same product on the expected and actual
pages are matched, even if when the script was recorded the product
appeared at the top of the list but on playback it was half way down. To
facilitate this matching process, ‘blocks’ of data can be identified on the
screen and these are used on playback when matching actual and
expected elements. Click on the ‘click here’ link on the Blocks section of
the summary panel to launch the Block Wizard.
When the anchors have been defined, click Continue to view the actual
blocks that have been identified as shown below. What this actually
means is that during playback, a match of the entire block will be
searched for. This means there is no danger that the radio button from
one block will be incorrectly matched with that from another block,
which could result in a mouse click selecting entirely the wrong
company.
If the number of blocks that have been identified does not match the
number of anchors that were found on the previous screen, a red icon
instead of a green one will appear next to the text above the screen picture
and the Continue button will be disabled. If this occurs, click the Back
button to modify your anchor selection. Only when matching numbers of
anchors and blocks are found will the wizard allow you to move to the
next stage.
If identity elements were not found in every block, a red icon instead of a
green one will appear next to the text above the screen picture and the
Continue button will be disabled. If this occurs, click the Back button to
modify your selection. Only when matching numbers of blocks and
identity elements are found will the wizard allow you to move to the next
stage.
Click Continue to display the final screen in the Block Wizard. This
enables error checking to be switched off for all of the elements within
the blocks although the rest of the screen will still be checked for
differences on playback. This is a sensible option to take if the list of
items for which blocks have been defined is likely to change, or if
variable data will be used to enable input to be played over alternative
blocks.
Once a Block Set has been defined, it is listed in the Summary panel as
shown below. The Block Set header is followed by any input for
elements within the block, which is turn followed by the identity elements
for the input.
The following right click options are available depending on which item
in the block definition was highlighted at the time. To view the blocks
that have been defined in the wizard, click on the Block Set header in the
Blocks panel.
Block Set
Input
Identity Element
Screen Properties
Information about the current screen which is required by TestDrive-Gold
and is placed here for information purposes only.
Script Options
Script Options can be accessed in several ways:
1. Shown when the ‘Playback’ button is pressed for a selected Script.
2. Double clicking the Script Header in a selected script or right
clicking and selecting the ‘Edit’ option.
3. Clicking on the Options button when saving the Script.
Tracked Fields This button will allow any Tracked Fields that
are associated with this Project to be edited. See
the separate section on Tracked Fields.
Use Variable Data If the script has been set up to use variable data
this field will be automatically checked. Un-
checking this field will cause the script to be
replayed with its original fixed values.
Use Tracked Fields If the script has been set up to use tracked fields,
this option will be automatically checked. Un-
checking this option will cause the script to be
replayed with its original fixed values.
Exceptions
These Script level exclusions are an important addition or alternative to
that achieved by excluding individual fields. For example, the time of
day may appear on every screen in a Script. One option would be to tune
each screen to ignore any differences in the time, but Script level
exclusions allow this to be achieved via a single entry.
To add an exception right click anywhere in the data entry grid on the
exceptions tab and then click the add option. To edit or delete an existing
exception right click on the exception and then click ‘Edit’ or ‘Delete’
from the floating menu.
Options
The values listed here are those values that were set up using the Options
toolbar at the time the script was recorded. They are used to determine
how the script should be recorded and played back. They are split into
six sections, the first three of which are used for recording purposes and
therefore cannot be changed here. These are listed for information only.
The latter three sections can be changed for this script and these changes
will be applied to subsequent executions of the script. See the earlier
Options section for a full explanation of all of the values.
Playback
Instructions
1. Click the Playback button to display the following screen.
2. On the Playback Wizard select the required options. Most of these are
explained in the Script Options section. The Run Description and
User Reference are user controlled text fields that are displayed
against this run in results. They can be used for any purpose, for
example to group multiple runs together via a work request number.
The variable data transactions specify which rows in the attached
variable data set should be played back for this execution of the script.
Click Next to continue with the playback or Cancel to return to the
script.
3. The Wizard is displayed from where the application against which the
script should be re-played can be selected.
4. If the ‘Edit test items and scores’ Option is checked the following
screen is displayed.
The list of documented items for this run can be tailored either by
Any changes to the Test Items are stored back on the server if there is
an active connection when the Keep button is clicked at the end of the
run to save the results. Once the playback of the script has been
completed a similar window is displayed which enables statuses and
comments to be entered (see later Scoring Results section).
Results
Once the execution of a Script is complete or the execution has been
interrupted by the user, a panel in the following format is displayed.
Each screen which was presented during the execution of the Script along
with its associated input is listed together with a signal light indicating:
The panes on the display can be moved to different positions and also
overlain. To move a pane, click on its title and drag it to the new
location. To overlay a pane, click on its title and position it over another
pane, a tab for each one will appear at the bottom of the pane. The panes
can be hidden by clicking on the Pin icon, when this is done a tab for the
pane will appear on the left hand side of the screen. To restore the pane
to the main view, click on the tab and then on the Pin icon.
TestDrive-Gold can store two possible formats for this window, which
can be selected by right-clicking on any of the orange window labels.
Markups
You may wish to annotate specific areas of any of the screens in the
results, for example to highlight information which is incorrect. To do
this, click on the outline icon above the screen picture. Then use the
mouse to drag a box around the area of the screen that you wish to add a
comment about. When the mouse is released, a comments box is
displayed as shown below.
Click Accept to add this markup to the screen. The markup is shown
below the screen and clicking on it will highlight the selected area of the
screen and fade out the rest. Click anywhere else on the screen to return
to the full view.
The following options are available from the right click menu for a
markup.
Any screen which contains a markup will have its screen icon overlain
with a dotted box. If any of the markups have been flagged as being in
error, the box will contain a red exclamation mark.
Pass/Fail Results
If the option has been selected in the TestDrive-Gold Options window,
after clicking on the Keep button to save screen results, a window will be
displayed so that statuses can be recorded for each Test Item selected at
the start of the run. The list of selected Test Items can also be changed
here.
Amending Scripts
In the event that changes occur to the system under test for which Scripts
already exist, it is possible to easily change these Scripts.
Deleting a Screen
To delete a screen from an existing Script:
1 Open the Script that requires changing and click on the relevant
Screen name.
2 Right click and select the ‘Delete’ option. You will be asked to
confirm that deletion is required.
3 On confirmation the screen will be deleted.
4 Save the Script.
Care must be taken in the event that Start & End Loops have been
specified within the Script, especially if the screen being deleted is
marked as either one (see Variable Data section).
Inserting a Screen
To insert a screen into an existing Script:
1 Open the Script that requires changing and optionally click on the
Screen name before the point at which you want to insert a new
screen.
2 Ensure that the system application under test is on the correct screen
for recording.
3 Press the Record button. The following screen will be displayed.
4 Click the Replace option of you wish to delete the existing script and
create a new one. Click the Insert option if you wish to add some
new screens to the current script and specify the desired location of
the new screens. Click Next to continue or Cancel to return to the
original script.
5 The Record Wizard is displayed from where the application
containing the new screens to be recorded can be chosen. At this
point TestDrive-Gold will go directly into ‘Record’ mode with the
focus placed on the selected application. Commence the required
testing and when finished press the ‘Stop’ button.
6 Play back the script to verify that the screens have been captured
correctly and then save the changes.
Script Verification
This function can be used when more significant changes have been made
to the application rendering existing scripts out of date. When the script
is played back in verify mode, the old screens will be replaced with the
new ones and TestDrive-Gold will attempt to transfer all field
information from the old to the new elements. In order to do this it
attempts to match elements on the old screens to those on the new ones,
once a match has been found it attaches existing input, variable data,
tracked fields and checking flags to the new element.
Verification Options
A verification run is begun by right clicking the Script name and selecting
the ‘Verify’ option. The Verification Options screen is then displayed
which enables the options that control the verification run to be modified.
Notification
These options control the conditions under which playback will halt to
request manual intervention when elements cannot be matched. If
matching does not successfully occur any input, variable fields, tracked
fields or checking flags defined for an existing element will not be
retained by the corresponding element on the new screen.
Core Fields
Click on the Next button to select the application under test from the
Verification Wizard. When the verification run has complete, choose
whether or not to overwrite the existing script. The changes will not be
saved however until the Save option is taken.
Notification
When elements cannot be matched, the following variation of the main
TestDrive-Gold panel is displayed to enable manual intervention in the
matching process. It is initially displayed in compact mode but can be
expanded to full screen mode by clicking on the expand icon at the top
right of the display. The notification level (which is explained in more
detail above) determines whether this screen is displayed when any
unmatched fields are encountered or for core fields only.
The areas of this screen that have not been explained in either the Script
Content or Results sections of this chapter are described below.
Core Fields
The following display is split into two parts, Missing Elements and
Matched Elements. For all elements listed, the four icons in the header
bar determine whether the object has any associated input, variable data,
tracked fields or if the element is not being checked for differences.
These four properties are used to determine whether or not an element is a
core field, depending on the options that were selected at the start of the
verification run.
Any core fields that were present on the expected screen for which a
match cannot be found on the actual screen are listed here. It is possible
to proceed with the replacement of the expected screen without matching
these elements; however any core information will be lost. If an element
is present on the actual screen to which this element should be matched,
then the two elements can be matched manually and in doing so all core
information will be transferred to the new field.
Manual Matching
To match a field manually, left click to highlight the element on the Core
Fields display and then drag the element to either the Actual Picture or
Actual Contents list until the target element is also highlighted. Then
release the mouse button to complete the manual matching process. If
you matched a missing element, the element will be removed from this
list and added to the list of matched elements. If you selected an element
from the list of matched elements because the matching was incorrect and
have now matched it to a different element on the actual page, it is
possible that you will see an element added to the list of missing
elements. This will be the field that was previously matched to the actual
field you have just selected.
Verifying Screen
Replace and proceed This is the default option. The expected screen
will be replaced with the actual screen and any
core information transferred to the new
elements for all fields which have been
successfully matched.
Insert new screen The actual screen is inserted into the script
before the current expected screen. This
inserted screen will have no associated input so
control is returned to the application under test
and the user must manually key in any input
and navigate to the next screen. When a new
screen is displayed the verification process
resumes, this screen is then matched to the same
expected screen as before.
Matching Options
The options on this panel control the level of accuracy of the matching
process.
This panel enables the matching options to be modified. Change the level
of accuracy and click on the Rematch button to match expected and
actual fields on the current pages at the new level. Click the ‘Apply to
future pages’ option to use the new level of accuracy on all subsequent
pages in the script. This panel is minimized by default but can be
expanded using the double arrow icon.
Notification Options
The notification level that was chosen at the beginning of the verification
run can be changed here, click on the Set button to apply this change to
all subsequent screens in the script. This panel is minimized by default
but can be expanded using the double arrow icon.
Notification Reason
Displays a message indicating why the verification run has been paused.
Verification Checkpoints
Right click any screen in the Script panel and select the ‘Verification –
Always Stop Here’ option from the floating menu to create a verification
checkpoint. This ensures that when the script is verified it will always
stop at the selected screen. This is important, for example, when a new
mandatory field has been added to the application. The verification
window is only displayed when existing core fields cannot be matched
but verification does not stop when new fields are found. Therefore there
is no opportunity to add input for this new field. Using a verification
checkpoint the run can be automatically stopped at this screen and input
manually added to the new field.
Variable Data
Variable Data allows you to record shorter, more flexible Scripts and is
thus a key feature of TestDrive-Gold.
For example, let’s imagine that you want to stress test the ‘Add a
Customer’ function. One option would be to record a Script that adds a
single customer and then set the Script Option, ‘Number of Runs’ to 100.
However, there is a problem with this approach. The Script would
successfully add the first customer, but all the following 99 attempts
might be rejected with a ‘Customer already on file’ error message as the
Script uses a fixed value for the Customer Number.
Without Variable Data the only alternative is to record a Script that is one
hundred times longer and contains the details of one hundred different
customers. This would take much longer to record and be almost
impossible to maintain if the screen layouts for the ‘Add a Customer’
function were changed at any time in the future.
Variable Data allows you to feed selected Data and Constant fields with
external information, rather than using the fixed value that was recorded.
You can therefore record a Script that adds a single customer but then
select the Customer Number and perhaps the Customer Name to be fed
not as the record constant, but by fields.
Once this has been done simply define the Customer Numbers and Names
that are to be keyed in and they will be automatically merged with the
Script when it is run.
There are five basic steps involved in creating a Variable Data Script:-
1 Record a simple Script to process a single transaction.
2 Access the Script Properties panel by right clicking the Script name
and selecting the ‘Edit’ option. Either create a Public Variable Data
Set or add data to the Local set. For controls such as Radio Buttons
and Select Boxes the transaction values must be equal to one of the
possible list of values for this field. One transaction is equal to one
iteration of the Script. Variable data can also be defined for the full
contents list of controls such as list boxes. In these cases the
transaction data consists of all possible values for the control
separated by the | symbol.
Local Variable Data is stored with the individual Script and can only be
used by the one Script.
Public Variable Data Sets are stored at the Project level and can be
utilised by any Script within the Project.
2. Both Local and Public Variable Data Sets can be accessed from the
Script Properties panel in TestDrive-Gold - see screen print below.
Right click on any of the Variable Data Sets in the list to obtain the
following options.
Right click on the single grey box in the top left of the Edit Variable Data
window to begin creating variable data fields and transactions for new
Variable Data Sets. The following options are available.
Column Options
When at least one field has been added some additional options are
available on right click on the field name column headings. This screen
is cursor sensitive, the options listed apply to the field on which the right
click occurred.
Row Options
When at least one row has been added some additional options are
available on right click on the transaction numbers. This screen is cursor
sensitive; the options listed apply to the row on which the right click
occurred.
In addition to the right click options, rows can be moved simply by left
clicking the transaction number and dragging to the desired new position.
Multiple rows are selected by using left click in combination with
Shift/Ctrl.
Paste Row Insert the data in the previously copied row into
the row where the right click occurred.
Data Options
This screen is cursor sensitive; the options listed apply to the cell on
which the right click occurred. Multiple cells are selected by using left
click and dragging the cursor.
=date (dd/mm/yy)
A - is valid in addition to a +.
The valid operators are day, days, month,
months, year, years. The expression is not case
sensitive.
In the above example you might wish to loop around the script several
times, each time adding an order for a different product. In this case the
variable data set should contain the list of Identity Elements for each
block that you wish to be selected.
Right click the Identity Element in the Blocks section of the Summary
panel and select ‘Edit’. From here the variable data field ‘Product’ can be
selected.
Playback
When the playback of a Variable Data Script is selected, the Playback
Wizard enables the starting and ending Variable Data transaction
numbers to be selected. If these are left blank then all transactions will be
processed.
Tracked Fields
Tracked Fields enable system generated values to be stored and re-used
later in the Script. This has several benefits. Differences during playback
as a result of these unpredictable values can be avoided. Also, these
values can then be used as input on subsequent screens, whereas without
Tracked Fields testing would need to be paused while the correct value
was added to the screen in question. They can also be used within
Validation Rules, please see earlier section for more information.
TestBench Integration
TestDrive-Gold is integrated with TestBench on the iSeries or Oracle,
providing several benefits:-
JWalk
If the JWalk product is being used to create iSeries screens in your
application, scripts recorded over these screens can be played back fully
integrated with the TestBench product. In this way full advantage can be
taken of all other Test Case features on offer. To achieve this, a ‘Start-
up’ script is automatically loaded onto the local PC at the time of
installation. This script will then be used to navigate from the TestBench
Main Menu, through launching the relevant Test Case to the first screen
of the application under test. At this point the script created by the user
will take over to perform the testing.
record an alternative script to be used instead of the one that was installed
with the product. Please speak to your support contact for details should
you need to do this.
a Project) to check all the updates to protected files during a test run.
The following results may be stored for your Test Run, depending on the
functions utilised when setting up the Test Case:-
• Warning messages
• Screen results (Screen content can be viewed in TestBench-PC `
only)
• Performance information
• Timeline
• Server activity
• Database effects
• Job Log messages
• Data Rules
• Pass/Fail information
Test Reporting
Full reporting capabilities for all stored results are present in TestBench-
PC or TestBench on the iSeries - please refer to the relevant sections of
these User Guides for more information. Please note, TestDrive-Gold
screen results can only be reported from TestBench-PC.
Script Security
If the server that you are connected to is the iSeries and security has been
activated then the access for a specific User to Scripts within TestDrive-
Gold may be restricted. There are several authority states.
Execute The User can view and play Scripts but cannot
change them.
Full authority The User has full access to the Scripts within
this Project.
Troubleshooting
If the expected results are not being achieved while either recording or
playing back using TestDrive-Gold, this can often be resolved by
modifying the Options that are being used. TestDrive-Gold is shipped
with some standard Options sets for known application types. The
following list describes how to resolve some of the most common
scripting issues.
Recording Issues
A picture is not being taken when clicking on a tab control within a
web page.
Most web tab controls use some form of DHTML to simulate the known
Windows tab controls. This can be monitored for using the option ‘Take
picture on content changes’.
Extra pictures are being taken when changing the focus between the
target application and TestDrive-Gold or another application.
Turn the option ‘Take picture on Active window changes’ off.
Playback Issues
A picture is being taken too early (or additional pictures are being
taken) so the playback of input fails.
Try setting the ‘Wait for replacement screen similarity to be’ option to
50% (experiment with this value). This will make TestDrive-Gold wait
for a replacement screen that is more similar to the expected screen than
the current actual screen. If you find you need this option to enable
reliable playback, you might want to configure your ‘During playback,
set activity timeout to’ setting to something more appropriate.
processing to occur.
Elements within a web page are not being matched correctly. This
results in input not playing back.
Some web sites are designed around a single page where the multi-page
feel is simulated by URL parameters. As default we ignore these
parameters in our matching. This results in a link to index.htm?home
being treated the same as index.htm?contactus. To resolve this problem,
disable the option ‘For web pages, exclude parameters in comparisons’.
To load a script into TestDrive-Gold which has been saved locally, first
of all ensure that no script is currently open. Then hold down the Ctrl key
and Right Click to obtain a list of local scripts in the default location.
5 Action Maps
Introduction
Action Maps allow a set of components to be run in a conditioned
sequence. Components are either TestDrive-Green Scripts, TestDrive-
Gold Scripts or TestBench functions such as Warp Cases, Plan Cases or
Environment Checkpoint manipulation actions. Action Maps allow
scripts from different Test Cases and other components within the same
Project to be grouped and run together.
Action Maps also allow the creation of branches where results in one
script can be used to decide which subsequent components will be run.
This means that the run sequence is driven by conditions pre-determined
by the user – e.g. different script scenarios may be followed depending on
whether an invoice is raised containing a positive, negative or zero
balance. The logic that enables this conditional execution is what forms
the basis of an Action Map.
Getting Started
Action Maps are accessed at the Project level in TestBench-PC. See the
TestBench-PC chapter for more information.
Notes Edit the notes associated with the selected Action Map -
see the Notes section in the TestBench-PC chapter for
more information.
Delete Remove the Action Map from the Project. This will not
delete component objects which exist in their own right.
Confirmation will be required.
Results View results for all previous runs of this Action Map.
See the Action Map Results section in the TestBench-
PC chapter for more information.
Toolbar Buttons
New Creates a new Action Map. Where an existing
Action Map is open and has been changed the
application will display a warning and enable
the Map to be saved.
Menu Options
Note that only those menu options which are not available through
buttons (see section above) are detailed here.
File
Create Launch File Launch files are used for two purposes. They
support integration with Mercury Quality Center
and enable you to launch a particular Map
without going through TestBench-PC. To use
the file in Quality Center simply add it as an
attachment as you would any other PC file. To
launch outside of Quality Center simply right
click the file and select ‘Launch’. By specifying
additional options in the wizard that is displayed
Map
Help
Right-click Options
Once an Action Map has been opened the Action Map main panel will
have several right-click options as displayed below.
Remove Remove the item from the Action Map. Only the
reference to the component is removed and the
actual component is not affected by this action.
A confirmation screen will be displayed. If this
option is selected for a branch, the branch and
all of its components will be removed.
Inside the branch The component will only run if the condition in
the branch is true.
Below the branch The component will run after the branch and its
components, regardless of whether or not the
branch condition is true.
Adding Components
To create an Action Map you will need to select the components which
will make up the Map. These components can be held within different
Test Cases but must all exist within the same Project as the Action Map
itself. The component type must be one of Script, Test Case, Data Case,
Warp Case, Plan Case or Compare Case. For more details on these
different component types see the relevant section within the TestBench
User Guide.
Right click on any of the nodes in the Action Map and select the Add
Component option to insert a new component after the highlighted item.
The following screen is displayed which lists all components within the
current Project.
For all components other than Test Cases, double click the component or
right click and choose Select to add it to the Action Map. As you double
click the components will become displayed on the left side of the panel.
Continue to add as many components as required. For Test Cases, right
click and choose Select to add them to the Action Map, or double click to
reveal the list of Scripts which are stored against the Test Case which can
in turn also be selected.
Toolbar Options
Change the view Click on the button or the down arrow to toggle
the list of components between the Detail view
shown above, the List view which includes the
component name but not its description and the
Icon view.
Right click on any of the nodes in the Action Map and select the Set
Checkpoint / Rollback option to insert one of these actions after the
highlighted item. The following screen is displayed which lists all Test
Environments within the current Project.
Toolbar Options
Change the view Click on the button or the down arrow to toggle
the list of components between the Detail view
shown above, the List view which includes the
component name but not its description and the
Icon view.
Adding Branches
A powerful feature of the Action Map application is the ability to create
branches within the Map. This enables events to be driven by the user’s
creation of logical arguments. The following panel is displayed when the
Add Branch option is selected.
The window is divided into two panels. The panel on the right shows the
complete set of conditions which make up the branch. (A condition is a
logical test such an A > 10 or b = “Customer”.) An individual condition
can allow a Tracked or Variable Data Field value to be compared with a
fixed value, or it can allow the values of two Fields to be compared. A
branch may include several conditions, linked together with And/Or
statements.
Use the left hand area to enter the correct selections for the argument you
wish to build. As you move to the next column the left hand panel will
automatically present the next set of applicable values from which you
can select by clicking.
And/Or The default and only possible value for the first
condition of any branch is ‘If’. When additional
conditions are added to the branch using the Add button
a choice of And/Or is available. If an Or condition is
specified, any previous lines form a separate condition
Op The valid values are =, <, >, <=, >=, <>, Like. Like
enables wild card values to be used. For example,
Error* would include values of Error and Errors.
Comp The two valid entries are Value and Field. Value
enables any string or numeric value to be entered
against which the Tracked or Variable Data Field will
be compared. Field requires another Project and Field
to be selected against which the contents of the first
Field will be compared.
Details If the option ‘Value’ has been selected from the ‘Comp’
field then the value in here can be free-text. If the value
chosen for the ‘Comp’ field was ‘Field’ then the value
for this selection must come from those available in the
drop-down.
Right-Click Options
Right click any of the expressions in the branch to obtain the following
options.
Buttons
The following are buttons are available for new branches that have not
yet been added into the Action Map.
Cancel Do not add the newly created branch into the Action
Map.
The following are buttons are available for branches that already exist in
the Action Map.
Evaluating Branches
If an Action Map includes branches, the values of the Tracked or Variable
Data Fields at the point where the branch occurs will be used to determine
which of the later components should be run. If a branch is Not True then
the components which depend on the branch will not be executed and
they will not appear in the results. The following rules apply when the
application evaluates a branch. If the Field value has not been set when
the condition is evaluated (this applies to Tracked Fields only), the
default Field value is used, blank for alphanumeric fields and zero for
numeric fields. If the condition is defined against a number but the Field
does not contain a number then the condition is Not True. If the condition
is defined against a string but the Field does not contain a string then the
condition is Not True. If the condition compares two Fields and one
Field contains a number and the other one does not, the condition is Not
True.
Action Properties
Click on a component in the list and select the Action Properties tab to
view and change the options which control the execution of the
component within the Action Map.
Variable Data Source This option is valid only for Script components
and it determines which variable data set will be
used during Action Map playback. As well as a
script containing its own variable data, it is also
possible to attach a variable data set to the Map
itself. However, only one variable data set can
be used by each script. Therefore set a value of
‘Script’ if the scripts own variable data set
should be used, in which case when the script is
executed it will loop around all transactions
prior to the running the next component.
Alternatively select ‘Map’ if the variable data
Play Final Screen? This option is valid only for Script components
and it determines whether or not the script will
play the keystrokes on the penultimate screen or
not. For example, if all scripts have been
recorded in an integrated fashion, they may
press F3 on the final screen in the script which
causes an exit from the application menu and a
return to TestBench. However, if several scripts
have been combined into an Action Map and
Map Integration has been selected, the exit
should only be performed by the final script in
the Map. The default value is checked, i.e. the
last screen of the script will be played. Uncheck
the box to instruct the application not to play the
last screen of the script in playback, this is
indicated by a yellow horizontal arrow on the
icon in the Action Map list.
Options
Click the Options button on the toolbar to control some of the system
level Action Map options.
Default Run Mode There are three valid run mode options, ‘Map
Integration’, ‘Script Integration’ and ‘Map No
Integration’. For full details on the definitions of
these terms see the following Action Map
Control section. This screen controls the default
value that will be applied to all new Maps but it
can be subsequently changed by right clicking
the Map name and selecting the Edit option.
General
Use Variable Data This option is only relevant if the Map has
variable data attached to it, see the Variable
Data section below. If this option is checked, the
Map will loop for the number of transactions
within the variable data set and any scripts
which are set to use Map variable data will use
the data within the current transaction or row
number.
Advanced
If error count exceeds Select a value between 0 and 999. Use this to
force an action to be taken should the number of
fields in error exceed the number specified here.
The script can either be continued to be played,
Variable Data
Variable Data allows you to record shorter, more flexible Scripts and is
thus a key feature of the Original Software suite of products.
For example, let’s imagine that you want to stress test the ‘Add a
Customer’ function. One option would be to record a Script that adds a
single customer and then set the Script Option, ‘Number of Runs’ to 100.
However, there is a problem with this approach. The Script would
successfully add the first customer, but all the following 99 attempts
might be rejected with a ‘Customer already on file’ error message as the
Script uses a fixed value for the Customer Number.
Without Variable Data the only alternative is to record a Script that is one
hundred times longer and contains the details of one hundred different
customers. This would take much longer to record and be almost
impossible to maintain if the screen layouts for the ‘Add a Customer’
function were changed at any time in the future.
Variable Data allows you to feed selected Data and Constant fields with
external information, rather than using the fixed value that was recorded.
You can therefore record a Script that adds a single customer but then
select the Customer Number and perhaps the Customer Name to be fed
not as the record constant, but by fields.
This is taken one step further by Action Maps, which enable many scripts
using variable data to be linked together. Let us assume that you have
two variable data scripts, one which adds multiple customers and another
which adds orders. When these are combined into an Action Map, by
default each script will complete all of its own transactions before the
next one is executed. Therefore we would have many customers added
by the first script, and then multiple orders added to a single customer by
the second.
However, it may be desirable to run the first script using one transaction
to add a single customer and follow this with the second script to add an
order for that customer (customer number could be monitored for using
tracked fields). Then once this is complete the first script can run again
and add the second customer followed by the second order, and so on
until all transactions are complete. In effect we wish to loop around the
entire Map, thus enabling us to add one order for each new customer.
This can be accomplished using the variable data attached at the Map
level. The following steps outline how to achieve this.
1. Record the required scripts and add variable data in the normal way.
2. Add these scripts in the correct sequence to an Action Map.
3. Create a new public variable data set which contains all of the fields
and data transactions required by all of the component scripts. This
can be the same data set attached to the individual scripts.
4. Attach this data set to the Map using the Action Map Control form as
shown below. Right click the required data set and choose the ‘Select’
option. Please refer to the TestBench-PC chapter for general
information on creating variable data sets.
5. Access the Action Properties tab for each script that should use the
Map level variable data and change the ‘Variable Data Source’ value
to ‘Map’.
6. If required, set loop points on your Action Map Components using the
Action Properties tab to define which elements of the Map should be
repeated for each transaction. If no loop points are defined the entire
Map will be looped.
Properties
If the Map contains any TestDrive-Gold scripts and the ‘Show target
application prompt’ option is checked, the screen below will be displayed
to indicate which of the available applications you wish to connect to
during playback. Alternatively specify an application or web site to
launch when playback begins. For each TestDrive-Gold script with a
‘Start Mode’ of ‘Attach’, the script will play back over the application
that was last used, ie. the one that was used by the previous script in the
Map. If the first TestDrive-Gold script is set to ‘Attach’, this will be the
application selected below. All scripts with a ‘Start Mode’ of ‘Launch’
will continue to launch the application over which they will play back.
Playback can be halted at any time by clicking the Stop button within
Action Map. When playback has finished, a message is displayed asking
whether or not you wish to keep the script results.
Tracked Fields
It is possible to view the history of any tracked field activity for the
current Action Map run. To do this, choose the Tracked Fields option
from the ’Map’ menu option.
Results
The results for the Action Map can also be accessed through TestBench-
PC. From the TestBench-PC main window, click the Action Map and
then select the View Results option from the Commands panel.
This will display a list of all Action Map Runs as shown on the following
screen. The results are stored in the TestBench database by reference to
the Action Map and a unique run number.
The following options are available from the right click menu for an
Action Map Run.
Map Entries Show the details of the Action Map Run in another tab
as described below.
Delete Remove the Action Map Run and all of the Action Map
Entries.
Map Entries
Right click any of the Action Map Runs and select ‘Map Entries’ to
display the following screen showing results for each individual Action
Map component.
Result Options
The following options are accessed by right clicking the required entry in
the list. However some options are not applicable to every type of entry
and may therefore not always be available.
6 Manual Testing
Introduction
The previous chapters in this User Guide have largely dealt with
automated testing. These are scenarios where consistent, repeatable tests
can be run over a stable initial environment. However this may not
always be possible for a number of reasons. For example, when new
functionality has been introduced it is inevitable that manual testing will
be required during the early stages of the test cycle. Only when stability
is confirmed and any interface changes applied might the user wish to
record automated tests to use during the later stages of the test cycle and
for future releases. Also when an automated test tool is first employed, it
is likely that the core areas of the application will be included first,
meaning that the peripheral testing tasks still need to be completed
manually.
Getting Started
Manual Testing is managed from TestDrive-Assist. This can be launched
from the shortcut on the TestBench-PC menu within the Original
Software program collection. Alternatively it will launch automatically
when a task containing steps is executed within TestPlan. This will be
covered in more detail in a later section.
When TestDrive-Assist is opened for the first time it will have the
following appearance. It occupies the right hand side of the screen and
all other active applications are automatically re-sized to fill the
remainder of the screen size. The width of TestDrive-Assist can be
changed by clicking and dragging the left hand border.
Connection Parameters
The following window enables the database containing the TestBench
application to be specified.
Connection Parameters
Data Source Lists the name of each server for which there is
an ODBC connection available. Only ODBC
Data Sources which include the word
‘TestBench’ will be displayed.
User Name The User ID that will be used to log onto the
selected server.
Other
Applications
The following window lists all of the Applications that have been defined
by all users on this connection. Those applications that will actually be
monitored by this PC must be selected by checking the box to the left of
the row.
The following options are available from the right click menu.
If either the Add or Edit options are selected, the following window is
displayed.
Application Tab
Once your application parameters have been defined and you have
selected the application for manual testing, as soon as the application
gains focus, TestDrive-Assist will begin recording its screens and input.
Options
The Options control the settings that affect the recording of screens.
They can be accessed from the Application panel. Only those which are
most relevant to manual testing within TestDive-Assist will be explained
here, for further information relating to all of the Options please refer to
the TestDrive-Gold chapter.
Analysis
This first set of options controls the level and nature of the information
that is recorded.
should be unchecked.
Take picture on
These options determine at which points during the test a new picture is
recorded of the target application. TestDrive-Assist monitors for activity
within the target application and takes a picture when this activity has
finished and the application goes “quiet”. These options control which
types of activity are included in this process.
Hide child element Similar to the above option but a new picture is
taken when the elements are hidden.
Active window changes If the active window (the window that currently
has the focus) has changed a new picture is
taken. For example, the main application
Resize child element Similar to the above option but a new picture is
taken when the elements are resized.
Move child element Similar to the above option but a new picture is
taken when the elements are moved.
Wakeup on
Busy sense
Miscellaneous
5250 Tab
These options are used by TestDrive-Assist when it is monitoring 5250
applications.
Title Position/Line The position on the screen where the screen title
can typically be found. Screens in the Script
will automatically be saved with this name.
Dictionary
Click on the rosette icon on the toolbar to add exceptions for spell and
link checking.
Four base dictionaries are included for use with spell checking, English
(United Kingdom), English (United States), Spanish and Dutch. It is
possible to add words to these dictionaries which do not exist in the pre-
shipped list, for example names and places.
Link Exceptions
When a link is checked for validity, one of a large number of return codes
is received. All codes which are considered successful must be added to
the following panel, these will not be reported as an error by TestDrive-
Assist.
Toolbar
There are three additional icons on the toolbar that have not already been
described.
System Tray
When TestDrive-Assist is minimized it is placed in the System Tray and
can be identified by the yellow circle icon that also represents TestDrive-
Gold. The right click menu for the icon contains the following options.
• During the recording process, five icons on the right side of the status
panel of TestDrive-Assist indicate the type of activity which is being
monitored for. Typically it is when all activity stops and the
application goes quiet that a new picture is taken. Therefore, if
pictures are not being taken at the correct times, noting down which
icons are active and conveying this information back to your support
contact will help to set the Options correctly. In order from left to
right, these icons represent:
- CPU Activity
- Messages
- Windows APIs
- Ajax
- Web Navigation
If link checking is enabled, the links on each web page are checked by
another thread in the background. This means that although you may
have navigated away from the page, its links are still being checked. If
link checking is in progress, the left-hand icon in the status bar beneath
the list of screens is enabled. The application should not be closed until
this icon is disabled to ensure that all links have been checked.
Screen Icon This screen does not contain any items that have
failed the spell or link checking and no markups
have been flagged as errors.
If at any point during the test a screen has been updated with some
information that you wish to record but a picture of that screen has not
been captured, click on the Add button on the toolbar to add a picture of
the screen to the screen list.
Click on any screen in the list to change the screen image at the bottom of
the panel. Please note that this screen appears on top of any other
application on the PC and therefore it must be closed before accessing
any other application. Click on any input to highlight that input in orange
on the screen image. To view a larger image and further information
about any of the screens, double click the screen name in the list.
The two buttons to the right of these control which mode the screen
contents view operates in.
The elements included on the highlighted screen are presented in list form
in the Contents section. Click on an element to highlight it on the screen
picture with an orange rectangle and to populate the Element Properties
panel.
The Element Properties pane displays the properties for the currently
selected object in the Contents pane. An additional section beneath the
properties panel is populated when a specific property is highlighted.
This enables the full list of values to be viewed for any properties that are
too large to fit on the main display. Such properties are displayed with
three dots at the end to indicate that there is more information that cannot
be shown. The properties that appear in bold are those that will be
checked for differences on playback, should this manual set of results
ever be turned into an automated script (see later section for more
information). These are different for each element type and are
considered the core properties for that type.
Highlight an input action from the screen list to view further details about
that input in the Input Properties section. The properties displayed vary
depending on the type of input selected. These properties will become
very important if this set of results is ever converted to an automated
script.
Element Caption The label associated with the target element for
the input. For example, for a button this could
be the visible text such as ‘Accept’ or ‘Cancel’,
for a text box it might be the label which
appears to the left of it. GUI and Web
applications only.
Ignore Remove this word from the Errors list for all
screens within this test, but do not add it to the
Dictionary. Therefore it will still be reported as
an error for all subsequent tests.
Please note that when results are saved for Web and GUI applications, the
results of the spell checking are also saved. However, this is not true of
5250 or iSeries applications and therefore spelling errors for these tests
must be reviewed from within TestDrive-Assist.
Link Errors
The Errors panel lists all of the links on the screen which have failed the
check. Click the plus icon to expand the list to show all of the instances
of this link. Click on the instance to highlight its location on the screen
image and populate the Element Properties panel with the Link Status
(the reason for the failure) and the Response Time.
Please note that each link is only checked once. If the same link appears
on more than one screen, each will have the same status.
Markups
You may wish to annotate specific areas of any of the recorded screens,
for example to highlight information which is incorrect. This can either
be done from the main Screen Contents panel above or by using the full
screen display mode. To access the full screen mode, click on the expand
symbol to the bottom right of the screen picture on the main TestDrive-
Assist panel. Please be aware that this panel will always sit on top of any
other applications on the PC, so to access any other application it must
first be closed.
Use the mouse to drag a box around the area of the screen that you wish
to add a comment about and optionally flag as an error. When the mouse
is released, a comments box is displayed as shown below.
Click Accept to add this markup to the screen. The markup is shown to
the left of the screen and clicking on it will highlight the selected area of
the screen as shown below.
The following options are available from the right click menu for a
markup.
When you close the above window and return to the main tree view, any
screen which contains a markup will have its screen icon overlain with a
dotted box. If any of the markups have been fagged as being in error, the
box will contain a red exclamation mark.
Please note that this functionality is not available for 5250 results.
Windows Vista
Please note that when monitoring Web or GUI applications on Windows
Vista, there is a specific animation function which must be switched off
to enable TestDrive-Assist to work correctly. If this option is switched
on, a message is displayed as soon as the application receives the focus.
The following steps can be followed to disable the option.
Toolbar Options
Once recording has begun for an application and its screens have been
added to the Current tab, three toolbar buttons become available.
History Tab
Results can reach the History tab in one of two ways.
1. Click on the Clear toolbar option from the Current tab to move the
most recent set of results to history.
Each set of results is stored separately and is identified by the name of the
application as it was defined in the Applications list. Click on a result set
to display the full set of screens with their input and screen images in the
panels below. All of the functionality regarding viewing the details about
each screen that was available on the Current tab is also available here.
For example, double click a screen to display another window giving
further details about the screen contents.
Saving Results
The results for any manual test can be saved, whether on the Current tab
or moved to History. If on the History tab, the results set that you wish to
save must be in focus prior to clicking on the Save button. When the
Save option is selected, the following window is displayed.
Reference Also saved with the results, this text can be used
to hold the version number or change control
reference for example.
Pass / Fail Use this option to Pass or Fail the test or section
of the results you intend to save (see following
screen for more information). A blank in this
field means that the test results are stored with
no status.
test.
Use the top section of the screen to specify against which Project and Test
Case the results should be saved. The properties for a Folder in TestPlan,
which are obtained from the Folder right click menu, hold the default
location for saving results for Tasks within it. Any Test Case specified is
automatically selected but can be changed here. New Projects and Test
Cases can be added here and existing ones changed or deleted by using
the right click menus. See the following section for more information.
Click the Next button to specify which screens to save with the test
results. Using these options you may wish to save a result set multiple
times for different reasons or errors, perhaps selecting a different subset
of screens each time.
When you have finished click on the Finish button to store the results. At
any time the Back button can be used to modify data on a previous
screen.
New Test Case Add a new Test Case to contain manual test
results. The following screen is displayed.
Add a code and title to describe the type of results that the Test Case will
contain. All other fields are only required if utilising other features that
can only be set up using TestBench-PC. Please refer to this chapter in the
User Guide for more information.
Viewing Results
The results saved from a manual test are stored in the TestBench database
and can be viewed from TestDrive-Assist and from TestBench-PC. To
view them from within TestDrive-Assist, click on the Results tab on the
toolbar. All manual results for this User are listed, grouped by
application name.
Toolbar Options
Summary
This screen contains high level information about the test and a picture of
the first screen that was saved for reference purposes.
Warnings
either a spelling mistake, link check failure or a markup that has been
flagged as an error.
Screens
Display the detailed screen results which can also be viewed directly from
the Current or History tabs in TestDrive-Assist. See the above sections
for more information.
Response
Pass/Fail
Current Run Status Display either Pass, Fail or N/A (not yet
defined) to represent the relative success of the
test.
Change Run Status Change the current status of the test to either
Pass, Fail or N/A. Once the status has been
altered, the comments box is enabled and a
comment must be entered before the status
change can be updated.
Update Status Click this button once the new status and
comment have been entered in order to update
the current status of the test.
Report
File Effects If database effects have been stored for this run,
choose which level of information to include on
the report.
Data Rules If Data Rule results have been stored for this
run, choose which level of information to
include on the report.
Click the Next button to complete the wizard selections and then the
Finish button to begin the report generation. Please note that only the
options available on your server are enabled. Once the report has been
generated the following screen is displayed. Use the arrows at the bottom
of the report to navigate through the report and the magnifying glass to
alter the zoom level.
The Execute option is available either from the right click menu for a
Task or from the Steps tab for a Task as shown below (please refer to the
TestPlan chapter for detailed information regarding the Steps panel).
When the execute option is selected, the Steps for the selected Task are
copied to the TestDrive-Assist panel ready to become part of the next
manual test. If TestDrive-Assist is not currently active then it will be
automatically launched. Please note however that the Execute option is
only valid for Tasks which contain Steps. Also if another Task is
executed prior to the current test being moved from the Current to the
History tab, a message asks if the current results should be cleared.
As well as the Test Item type and description, two additional columns are
present on the above display. The Pass column provides the ability to
pass or fail an individual Test Item. Therefore as the testing begins, once
each test item is completed its status can be changed accordingly. The
right hand column can be used at this time to add any notes which may
provide further information about the testing of the individual item.
Simply click in the notes field to activate an additional box into which
comments can be keyed. If the Steps defined to TestPlan have more than
one column, double click the column title on the above display (in this
case ‘Step Description’) to display another fully expanded window
containing all step columns. Close this additional window to revert to the
above view.
Once the test has been completed, the results can be saved as described in
the previous section. This time the Test Items and their results will also
be saved and can be viewed from the Pass/Fail tab in results. The results
are also automatically stored against the Task which was executed in
TestPlan and they can also be viewed from the Results tab there. If
multiple sets of results are saved for this manual test then they will all be
stored against the Task, providing a complete audit trail of testing
activity.
Clearing results which contain Test Items moves both Screen and Test
Item results to History. Both can be viewed at any time for each result set
by clicking on the relevant tab. If Test Items exist without any screen
results, no history entry is created.
Adding Defects
Once a Task has been executed, a new toolbar button is created for
raising a defect. Clicking on this button at any point during or after a test
displays the following panel, which contains a subset of the functionality
provided for raising a defect in TestPlan. Please refer to the TestPlan
chapter of this User Guide for full details about the defect maintenance
screens.
Once the defect description details have been entered, you can optionally
click on the Results tab and right click to attach your manual test results.
All results which have been generated for this manual test are listed.
Once all defect details have been entered, click Accept to create the
defect. It will automatically be added to the Task which was executed
from TestPlan. However, an alternative Task, Folder or in fact Plan can
be selected by clicking the Back button on the Defect wizard once, twice
or three times respectively.