How to Deliver High-Quality Mobile User Experience

HOW TO DELIVER
HIGH-QUALITY MOBILE
USER EXPERIENCE
An Olenick & Associates White Paper
September 2013
DEPLOYMENT. ASSURED. ™
Deployment. Assured. ™
Contents
EXECUTIVE SUMMARY .............................................................................................................. 1
USER EXPERIENCE AND TEST PLANNING ................................................................................2
Dimensions of Mobile Applications ......................................................................................... 2
User Experience ...................................................................................................................... 2
User Experience in Software Testing ...................................................................................... 3
MOBILE USER EXPERIENCE TESTING FRAMEWORK ................................................................ 3
OTHER USER EXPERIENCE OPTIONS........................................................................................ 7
CONCLUSION ............................................................................................................................ 8
ABOUT OLENICK & ASSOCIATES .............................................................................................. 8
APPENDIX ................................................................................................................................ 10
User Experience Test Cases .................................................................................................. 10
Exhibits
EXHIBIT 1: MOBILE APPLICATIONS ARE MULTI-DIMENSIONAL .............................................2
EXHIBIT 2: ELEMENTS OF THE MOBILE USER EXPERIENCE TESTING FRAMEWORK ..............4
EXHIBIT 3: USER EXPERIENCE GUIDELINES AND RESOURCES ............................................... 7
Copyright © 2013 Olenick & Associates, Inc. All rights reserved.
205 W. Wacker Drive, Suite 2000
Chicago, IL 60606
+1 (312) 332 0096
http://www.olenick.com
@OlenickIT
Additional copies of this white paper can be downloaded from
http://www.olenick.com/
Deployment. Assured. ™
EXECUTIVE SUMMARY
User experience is an essential consideration for independent software vendors and Enterprise
IT developers in mobile projects. If user experience is not optimized during the development of
a mobile application or tested before release, users will likely reject the application and the
producer’s brand will be damaged. Such a rejection can be fatal, as mobile applications usually
do not get a second chance.
Delivering a high-quality mobile user experience is particularly challenging due to a lack of
related standards, processes and tools that can be quickly deployed or incorporated into the
project lifecycle.
This whitepaper explains the critical elements of user experience, how they can be incorporated
into the existing software test process and how applications can be validated during the mobile
application development phase.
To address these aspects of user experience, Olenick & Associates has developed the Mobile
User Experience Testing Framework:








Main function
Interaction
Logical Path
Consistency
Display
Multi-Tasking
Settings
Touch
In addition to the framework, this white paper also provides generic test cases and validations
that can be re-used by testers.
How to Deliver High-Quality Mobile User Experience
1
Deployment. Assured. ™
USER EXPERIENCE AND TEST PLANNING
Dimensions of Mobile Applications
With the rapid growth and adoption of smart phones and tablets into our personal and work
environments, it has become imperative for businesses to reach their customers via mobile
applications and with mobile compatible websites. Similarly, it has become essential for
enterprises to provide additional tools for employees to be productive while away from their
desks using mobile devices. This need for businesses and enterprises to develop and maintain
mobile applications has created a demand for trained, experienced and competent mobile
application testers.
Mobile application and website testing is an intricate process, as it involves testing across
various platforms and operating systems. When applying traditional web testing concepts and
processes to mobile test planning, additional factors to consider include:
Devices
•Form factors
•Phone vs. tablet
Usage Type
•Business
•Personal
•Mobile trends
Location
•GPS
•Language
•Unit of measure
(mi vs. km)
Connectivity
•Wifi, GSM,
CDMA
•Signal strength
Platform
•Android
•iOS
•Blackberry
•Windows
Exhibit 1: Mobile Applications are Multi-Dimensional
All these additional factors should be assumed to be in scope from a testing perspective, until
they can be reviewed with the sponsors, end users and other stakeholders to make a final
determination on what will be considered to be in scope.
User Experience
Even the best conceived applications will struggle to gain acceptance if they are difficult to
install or use. A bad mobile experience can damage a company's brand and create negative
feelings among customers or employees.
Many customers are frustrated when they encounter
Ease of use and integrated
a site that is not mobile ready or friendly. This
experience might cause them to wonder how much
user experience across
the sponsoring company cares about their business. A
devices significantly
poor mobile experience may end up impacting a
enhances user experience.
customer's continuing relationship with that
company.
So testing to ensure a positive user
experience can not only make or break the success of a mobile site or application, it can also
secure a customer's continuing positive relationship with the company.
“
How to Deliver High-Quality Mobile User Experience
2
Deployment. Assured. ™
Some of the factors that can impact how an end-user feels when interacting with mobile
applications are:







Device display and keypad (soft, hard, QWERTY)
Ease of navigation
Functionality, performance and compatibility
Location and signal strength (if server connectivity is required)
Data quality and security
Number of input fields
Number of actions to complete a transaction
The impact of these factors can be minimized by incorporating best practices for mobile user
experience early on in the software design and testing lifecycle.
User Experience in Software Testing
For the most part, software testing processes from the traditional desktop and web
applications can be applied to mobile projects. Just like in traditional projects, the layout and
interface of mobile applications are sometimes designed by technical resources instead of user
interface (UI) specialists, which may result in a lack of attention to navigation and aesthetics
during the design phase. In desktop and web application projects user experience is usually not
a primary consideration for testers. But user experience in mobile projects must be given
attention and focus during the testing phase.
User experience testing can be a challenge, especially
for testers new to mobile projects, due to the lack of
sufficient requirements and design specifications
related to user experience. Moreover there is no
widely accepted industry standard or quantitative
way to measure or rate user experience. Testers thus
face the challenge of identifying test scenarios that
will address all user experience factors.
“
There is no widely
accepted industry
standard or
quantitative way to
measure or rate user
experience.
MOBILE USER EXPERIENCE TESTING FRAMEWORK
To address the challenges of user experience testing, software testers should look for creative
ways to incorporate user experience into the software testing. In the absence of user
experience specifications, one way is to leverage the widely accepted UI guidelines as user
experience requirements. The UI guidelines are very detailed and address all aspects of UI, but
are not necessarily followed by application designers and developers.
How to Deliver High-Quality Mobile User Experience
3
Deployment. Assured. ™
Below are some of the user experience elements to consider when designing or testing a mobile
application. These elements address all the important aspects of user experience that usually
result in common defects and complaints from end users. Together, these elements form a
holistic approach to user experience testing: the Mobile User Experience Testing Framework.
This framework will facilitate test teams to easily incorporate user experience into software
testing.
Mobile User Experience
Testing Framework
Main Function
Interaction
Logical Path
Consistency
Display
Multi-Tasking
Settings
Touch
Exhibit 2: Elements of the Mobile User Experience Testing Framework
Main Function





Main functionality should be easily and immediately apparent.
Controls and gestures should follow accepted platform standards and thus behave as
expected.
Controls should be minimal in number and labeled so users understand exactly what
they do.
Common tasks must be achievable through simple actions.
Shortcuts should be meaningfully related to longer procedures.
How to Deliver High-Quality Mobile User Experience
4
Deployment. Assured. ™
Interaction





Communication should be simple and in the user’s own language.
Content should have the option of being selected or highlighted.
Feedback about actions or inputs, changes of state or condition should be provided in
clear, concise and unambiguous language and within a reasonable time.
The system must respond within an appropriate time for all actions.
Errors must be clear.
Logical Path




Application must have intuitive flow, allowing users to complete tasks effortlessly.
Example (i): Tasks in intuitive applications match the user’s mental model of how similar
tasks are done in other contexts (i.e. reading a book on a mobile device in some ways
matches how a physical book is read), thus reducing the new knowledge needed to use
the app. 1
Example (ii): Intuitive applications stealthily train users on any new knowledge
required (i.e. an application such as Grades 2, whose start screen is just one button
and a finger pointing at the button). 2
Application must be predictable, producing results that fit with previous commands
and states.
Application must be easy to learn. Minimal effort should be required of the user once
the system is learned.
A structure must be organized purposefully, grouping contents in a meaningful and
useful way that puts related things together and separates unrelated things based on
clear, consistent models that are apparent and recognizable.
Consistency




Navigation, look and feel should be consistent throughout the application.
Options, behavior and controls should follow the general pattern of the operating
system (i.e. the application may be slightly different on Android and iOS).
For widespread adoption, identical user experience must be provided across all devices.
Usage paradigms of built-in applications should be followed.
Example: Users already understand how to navigate a hierarchy of screens, edit lists
and switch among application modes using the tab bar.
How to Deliver High-Quality Mobile User Experience
5
Deployment. Assured. ™
Display





Application should be “responsive”, i.e. automatically adjust to the form factor of any
device.
Entire device display should be used. Screen size discrepancies should be a top
consideration when testing mobile applications or mobile web.
Scrolling from left-to-right or top-to-bottom should be avoided.
Animation, symbols, buttons and colors should be legible, pleasant and easily understood.
Orientation and changes in orientation should not affect the app’s state, focus or
functionality.
Multi-Tasking




Switching between applications should be easy and should retain application state.
Closed applications should retain their state so users do not have to repeat actions upon
return.
Memory footprint should be small.
Interruptions and multi-tasking should be made easy.
Settings





User preferences and settings must be easy to set and modify.
Menu items like help, about, etc., should be easy to find and navigate.
Work should be recoverable if the user makes a mistake or the application fails or is
interrupted.
Application must adapt to various environments and users.
Interface must be modifiable in order to improve efficiency.
Touch




Touch controls should be fingertip-sized or else meet the minimum requirements of the
UI guidelines of the platform.
Buttons should be used for any important actions instead of links.
Action links should take up enough screen space to be tapped easily.
Example: Thick rows that span the width of the screen or square boxes can both be
tapped easily.
Multiple finger touches, like pinch or stretch, should be accounted for.
How to Deliver High-Quality Mobile User Experience
6
Deployment. Assured. ™
The Mobile User Experience Testing Framework allows testers to gain a basic understanding of
user experience. This will allow testers to incorporate user experience into testing even when
there are no documented user experience requirements. Testers can also incorporate this
framework into the test process and make it an essential part of the testing.
For additional information on the elements of the Mobile User Experience Testing Framework
discussed above, testers can refer to the UI guidelines defined by popular mobile platform
vendors.
Platform
Apple
Resource
• iOS Human Interface Design
Android
• Unified Testing Criteria
Windows
• Windows User Experience Design Principles
• How to Design a Great User Experience
• Testing Applications for Windows Phone
Exhibit 3: User Experience Guidelines and Resources
3
OTHER USER EXPERIENCE OPTIONS
If enterprises or businesses would like to explore more options to perform user experience
testing, they can consider hiring or engaging user experience testing specialists. User
experience services engage real users to gather feedback and identify areas of improvement.
This could be an expensive proposition to consider, however, and is only feasible after the
application is built and rolled out to end-users or to targeted groups of users.
A less expensive alternative could be in-house user experience testing, in which employees,
family and friends rate the application (as end-users) based on pre-defined criteria. The criteria
can be identified using the elements of the Mobile User Experience Testing Framework.
However, this option may have limited benefits because the users identified may not be
representative of the actual customer base.
How to Deliver High-Quality Mobile User Experience
7
Deployment. Assured. ™
CONCLUSION
User experience is critical to the adoption and success of a mobile application. The Mobile User
Experience Testing Framework provides a solution to assure that a mobile application meets
usability expectations and garners a “five star” rating from its users. The framework applies to
both native and mobile-enabled web applications and benefits developers and testers alike.
Implementing the Mobile User Experience Testing Framework into training, designing and test
planning activities will add a new dimension to the design, verification and validation processes.
This will ensure that the test engineers, developers and designers working on the application
are not only focused on business functionality but are also taking into consideration user
experience. This framework will also empower project stakeholders and testers to get a handle
on user experience during the design and testing phases, rather than much later in the project.
For a set of generic test cases that align with the user experience testing framework in this
white paper, see the generic user experience test cases in the Appendix, below.
ABOUT OLENICK & ASSOCIATES
We prepared this white paper for independent software vendors and enterprise IT developers
and testers to understand how to set an approach for testing for user experience in a mobile
environment.
In our experience, many managers underestimate this challenge. To find out more about how
Olenick & Associates can support your mobile testing initiatives, please contact us at
+1 (312) 332 0096 or via email at info@olenick.com.
To learn more about our full range of deployment assurance services, please visit us at
http://www.olenick.com/
How to Deliver High-Quality Mobile User Experience
8
Deployment. Assured. ™
Endnotes
1
For more on intuitive user design, read:
•
•
•
What Makes a Design Seem “Intuitive”?
http://www.uie.com/articles/design_intuitive/
The Secret to Designing an Intuitive UX : Match the Mental Model to the Conceptual Model
http://uxmag.com/articles/the-secret-to-designing-an-intuitive-user-experience
Intuitive Web Design: How to Make Your Website Intuitive to Use
http://conversionxl.com/intuitive-web-design-how-to-make-your-website-intuitive-to-use/#.
2
“Making an application ‘dead simple’ requires the axing nearly all of the cool features its creators
envisioned, leaving just the essence.”
•
3
7 Keys to Unlock an Apple Design Award
http://uxmag.com/articles/7-keys-to-unlock-an-apple-design-award
User Interface guidelines:
•
•
•
•
•
iOS Human Interface Design:
http://developer.apple.com/library/ios/documentation/userexperience/conceptual/mobilehig/M
obileHIG.pdf
Unified Testing Criteria for Android:
http://www.appqualityalliance.org/files/utc_android_v1.pdf
Windows User Experience Design Principles:
http://msdn.microsoft.com/en-us/library/windows/desktop/dd834141.aspx
How to Design a Great User Experience for Windows:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa511335.aspx
Testing Applications for Windows Phone
http://msdn.microsoft.com/en-us/library/windowsphone/develop/jj247547(v=vs.105).aspx
How to Deliver High-Quality Mobile User Experience
9
Deployment. Assured. ™
APPENDIX
User Experience Test Cases
These test cases were written generically in order to be used on various devices and operating
systems.
MAIN FUNCTION
Test Description:
Main functionality should be easily and immediately apparent
Required For:
All Applications
Testing Steps
Move between screens on the application
Testing Results
Icons
Controls and gestures behave as expected (per standards)
Labeled so that users understand exactly what they do
Shortcuts
Simple actions to do common tasks
Meaningfully related
Visual Indication
There is visual indication that the function is being performed
Effectiveness
All tasks are accomplished successfully within a specific time period
Device Variations
Different devices perform equally
User Load
App is able to perform during high user volume
Stability
App does not crash or freeze at any time while running on the device
Results of Test
Pass ☐ Fail ☐ Deferred ☐
How to Deliver High-Quality Mobile User Experience
10
Deployment. Assured. ™
INTERACTION
Test Description:
Communication should be simple and in the user’s own language
Required For:
All Applications
Testing Steps
Move between screens on the application
Testing Results
Feedback
Keeps user informed of what is going on
Delete Confirmation
Asks the user to confirm before deleting an item
Save Confirmation
Asks the user to confirm an update if updates are made and the user navigates to
another item before explicitly saving
Error Messages
Clearly understandable
Clearly explain to user the nature of the problem, and indicate what action needs to be
taken
Results of Test
Pass ☐
Fail ☐
Deferred ☐
How to Deliver High-Quality Mobile User Experience
11
Deployment. Assured. ™
LOGICAL PATH
Test Description:
UI must have intuitive and logical flow, allowing users to complete tasks effortlessly
Required For:
All Applications
Testing Steps
Move between screens on the application
Testing Results
Intuitive
Application flow is natural and allows the user to effortlessly complete tasks
User Interface
Predictable
In accordance with previous commands and states
Easily learnable by the user
Effort
Minimal effort is required by the user to perform app functions
Results of Test
Pass ☐
Fail ☐
Deferred ☐
How to Deliver High-Quality Mobile User Experience
12
Deployment. Assured. ™
CONSISTENCY
Test Description:
Navigation, look and feel are consistent throughout the application
Required For:
All Applications
Testing Steps
Move between screens on the application
Testing Results
Abbreviations
Consistent for all screens in application
Buttons
Appear in a consistent order from screen to screen (e.g. Submit, OK, Cancel, etc.)
Color
Colors match from screen to screen in header, body background, footer, etc.
Dialog Box
Different dialog is used on different screens. (Save/Cancel dialog on one screen and an
OK/Cancel on another is inconsistent)
Grammar/Spelling
No grammar/spelling errors are present from screen to screen
Icons
Common icon set is used through application
Links
Links appear in the same order from screen to screen
Color of the links is consistent
Links on screen (Save as, Email, etc.) have consistent spacing between them and other links
Menus
Order in which each menu item appears is consistent from screen to screen
Menu items that are not applicable for specific screens are disabled
Screen Font Size
Font size matches from screen to screen for headers, body text, footers, etc.
Screen Font Type
Font family matches from screen to screen
Results of Test
Pass ☐ Fail ☐ Deferred ☐
How to Deliver High-Quality Mobile User Experience
13
Deployment. Assured. ™
DISPLAY
Test Description:
Correct screen display and graphics are used throughout the application
Required For:
All Applications
Testing Steps
Move between screens on the application
Testing Results
Button Display
Animation, symbols, buttons and colors are legible, pleasant and easily understood
Screen Display
User does not need to constantly scroll from left-to-right or top-to-bottom to view information
Screen Rotation
Screen rotates automatically to match user’s viewing orientation
Screen Size
Entire display is utilized regardless of the device used
App is displayed correctly on differing screen sizes and devices without obvious errors
Results of Test
Pass ☐
Fail ☐
Deferred ☐
How to Deliver High-Quality Mobile User Experience
14
Deployment. Assured. ™
MULTI-TASKING
Test Description:
Switching between apps should be easy and should retain app state
Required For:
All Applications
Testing Steps
Move between screens on the application, switch between applications, answer calls, take a
picture and reply to text messages
Testing Results
Background Apps
User is able to run various backgrounds apps simultaneously (e.g. Music, GPS, Skype, social
network sites)
App Switching
When switching between apps, user can pick up where they left off instead of starting over
Interruptions
App is responsive to factors such as incoming calls, battery failure, signal loss, etc.
Results of Test
Pass ☐
Fail ☐
Deferred ☐
How to Deliver High-Quality Mobile User Experience
15
Deployment. Assured. ™
SETTINGS
Test Description:
Users have ability to easily select, set or modify user preferences and settings
without resulting in any app failures
Required For:
All Applications
Testing Steps
Use the application to set/modify user preference and settings
Testing Results
Flexibility
UI is flexible and adapts to different users and environments
Menu Items
Menu items are easy to find (Help, About, etc.)
Recovery
App can be recovered after a mistake or app failure
User Preference and Settings
Easily set and modified
Results of Test
Pass ☐
Fail ☐
Deferred ☐
How to Deliver High-Quality Mobile User Experience
16
Deployment. Assured. ™
TOUCH
Test Description:
Optimal touch experience is evident throughout the application
Required For:
All Applications
Testing Steps
Move between screens on the application
Testing Results
Interactive Touch
Screen changes or vibrates upon touch
Multi Touch
App recognizes multi-finger touch and responds appropriately
Size
Touch area is defined and responsive for different finger sizes
Touch Swipe
App recognizes finger-swipe touch and responds appropriately
Results of Test
Pass ☐
Fail ☐
Deferred ☐
How to Deliver High-Quality Mobile User Experience
17