Software Testing 452

Wordpress for Android

What is Wordpress?
A popular blogging website The Android App is Open source Programmed in Java Can post, view, and edit content

Scope
We restricted our testing to the Android application Other platforms of the app are not within our scope We tested the primary features of the application

Primary Features
New Post New Page Post Comment Post Video Post Photo View Post View Comment View Page Edit Post Edit Comment Delete Post Delete Comment

Environment
Eclipse as our workspace Android Virtual Device Android Development Tools are used to work with android Our project uses the ADT eclipse plugin with SDK 13 which is Android 3.2

Static Analysis
Code review of NewAccount.java class Used by every user at least once but not very much so it could be improved Quality was pretty good Very few comments, hard to follow Variables had vague names

FindBugs
Eclipse plugin for static code analysis There were some surprising results
Found a string comparison bug Found a null pointer bug

FindBugs - String Comparison

FindBugs - Null Pointer

Specification Document Analysis

We could not access the true specification document Found a final description of what was included in the app The addition of a "featured image" was emphasized. We were successful on posting a featured image by following the strict instructions

Unit Testing
AndroidTestCase
augmented version of JUnit Java Components vs Android components testActivity vs testClass

Unit Testing Example

Unit Testing
Overall not useful Most of the Wordpress web service logic exists on the server. The Android app merely provides another interface. The classes reference the actual web application The android application just generates views if certain conditions are met There is not much local logic

Usability Testing
Consistency
Everything was consistent: color and layout Creating and editing have similar interfaces
makes changing blog content simple

Consistent with Wordpress website UI

Correctness
Pretty good app crashed once when overloaded with input

Usability Testing Continued

Correctness
If app crashed in middle of long post, entire content would be lost

Usefulness
very useful for on-the-go blogger Many reviews describe mobile usefulness

Usability Testing Continued

App meets the Android design guidelines "When people use your app for the first time, they should intuitively grasp the most important features"
Wordpress definitely meets this expectation

Very simple to use and very intuitive

Accessibility Testing
The use of icons in the app was great! The icons were intuitive Every action had an icon Colors are dark text on light background

Accessibility Testing Continued

The app was given to someone with lowvision and they were able to navigate easily Talkback failed to speak appropriate text main screen buttons were all spoken as "WordPress" many buttons had no Talkback names post titles and text were not spoken

Black Box Testing

Allowed us to cover most of the program Issue with adding a video Took a very long time - more than expected with similar apps on same device No progress indicator - misleading On the edit page screen, sometimes when hitting

the back button the keyboard disappears and a large text box remains Requires hitting back again
confusing for users

Malicious Testing
Tested extreme data values and situations Purpose to see how the app reacts
inserted 21-page title with 1 char of content
program allowed this surprisingly Significantly slows down app when viewing aggregate posts Maybe consider title-character limits

Stress Testing
Was interesting because most android devices try to maintain stress themselves While running the app it could take a phone call Also while writing a post and receiving a call, content remains ready to be posted
Very good feature, would be annoying to retype

Fuzz Testing
We used dumb fuzz testing
all inputs random

Tool we used is Monkey

Generates random UI events and logs errors Events are clicks, touches, gestures, etc. Random, but in a repeatable manner

We used 700 random input events

behaved well
monkey -p org.wordpress.android -v 700

Security Testing
Android Lint and Database Injection plugins
Test basic and common security faults Found XML and database vulnerabilities There were some cases of injecting code into the database which could either break it or affect user information

Regression Testing
Happened late in the testing process because a newer version of the app was released Re-ran our tests most passed some had to be tweaked to reflect changes in the app Not really useful because difficult unit tests had to be rewritten

Testing Methods Not Used

Mutation Testing Not much logic to alter Load Testing Did not have enough users to test app at the same Coverage Testing Tools would be little use for our few unit tests
time, not enough resources Makes more sense for WordPress server

Conclusion
Educational Value learned a lot about Android UI libraries, the
Android Developer Tools, Virtual Devices, and how Android apps work

Risks building the program on the computers,

understanding code, scheduling meetings, working with new technology, and testing everything we wanted

Conclusion Continued
Reporting Errors WordPress app developers appreciate feedback Made blog for reporting problems General App Quality Overall quality was pretty good Performed very well