Sunday, September 26, 2010

KPIs for Great Experiences

Educating and equipping yourself and your existing organization to build great experiences and set KPIs for how those experiences are rendered and consumed requires some faith in your own R&DIY methodology - Research and Develop It Yourself.  Even for organizations with no intention of building teams with dedicated user experience skills, it is necessary to have some framework for stakeholders in any project to evaluate skills, work plans, resource models, wireframes and other artefacts of the user experience outcomes and process. 

I would encourage anyone thinking about this for the first time to stretch your mind a bit - often new methodologies that can bring considerable benefit don't work right away simply because organizations and individuals hold onto beliefs, practices and organizational paradigms that no longer fit the modern software development process.  So here are 10 things that I am confident everyone will benefit from and that will light the path to better customer experiences. There are some specific focus areas for each theme as well as some ideas for key user-centric KPIs that you can set for each.

1. Branding
User Experience is the front-facing evidence of a true focus on great customer experience.  Some people interchange UX with brand as it relates to interfaces on the web, essentially acknowledging that brand is something we influence in everything we do. The brand becomes an experience when designers wrap the online or in-store presence of an institution or organization in the promise of the brand and this powerful reality can lead to both good and bad brand experience.  A positive user experience has a direct correlation to positive brand perception. Authors such as Dirk Knemeyer who has worked on software design for Microsoft, Apple, Oracle and many other well known brands, expanded on this theme recognizing that both online and offline customer experiences contribute to brand image and \highlighting the importance of consistency between the customer experience across all touch-points.  Therefore working from the premise that an organization engages in a broad, complex set of interactions with its customers, of which the brand experience portrayed through its websites and applications is only one.  At minimum here, we need to verify that:
a.   The system provides all types of visitors, customers and employees with an engaging and memorable experience.
b.   The visual impact of the system is consistent with the brand identity and actually delivers on the learned and trusted promise of the brand.
c.   The system leverages the capabilities of the internet or application in order to enhance or extend the promise, values and mission of the brand.

Example Branding KPIs - Brand affinity, brand awareness, referral, comment/rating, sentiment, NetPromoter

2. Functionality
Less may in fact be more, but simple is never simple.  This subjective evaluation that drives a lot of application and online thinking needs to consider the goals of both the user and the business, and needs to reflect the capabilities and policies (e.g. security, privacy) of the IT organization responsible for deploying and managing the application.  When building interaction based on functions we should strive to minimize the cognitive load required to absorb information and complete tasks.  One way to think about this is to understand how much concentration the task requires to complete. A good way to reduce the amount of ‘thinking work’ the user has to do is to focus on what the computer is good at and build a system that uses the computers skills to the best of its abilities. To optimize interaction with sophisticated functionality consider the following key attributes of a system or application:
a.   The system helps its visitors accomplish common goals and tasks.
b.   Online functions are integrated seamlessly with both real-time data and transactions as well as offline or asynchronous business processes.
c.   Can users apply knowledge that they might have gained from using other sites or in-person situations?
d.   Accelerators—unseen by the novice user—may often speed up the deployment of an application and introduce best practices that cater to both inexperienced and experienced users. Can a one size fits all approach be used?  Can you extend systems and products in such a way to specifically target your use cases?
e.   Visibility of both system health and progress allows you to keep users informed about what is going on at all times and reduces abandonment during issues that may be out of their, or even your, control.  Ensure that task progress is clearly communicated at all stages.

Example KPIs for functionality - Adoption, abandon rates, customer service escalations, social feedback

3. Visual Delight
The balance between psychology, human factors, HCI (human computer interaction) and the magic of design is all based on one common principle, the notion that people want to be engaged by the things that they do.  People may learn in different ways, keep track of tasks in different ways but there is a fundamental need to be and feel immersed in experiences.  In a world of constant distractions, it is no small feat any longer to stay focused on tasks.  How well they can do that is a reflection of design, not just the design of the system or process they are engaging with, but also the surrounding distractions and plethora of options that are often unnecessarily pushed into interfaces.  Bear in mind that personalization can enhance the impact of design - allowing users to customize presentation maximizes utility to them.  Site design should accommodate personalization and utility, not force users to accommodate the design.  While design and engagement are subjective and need to reflect both brand and functionality as outlined above, we can do a few basic checks to ensure we are on the right path, such as:
a.  All graphics and multimedia add value to the specific experience or potential adoption of a system or tool. Engage and guide user's eye, mind and navigational system appropriately.
b.  Page length, content depth and screen elements are relevant to the task at hand - such as providing data visualizations to aid decision-making or help content and comparisons to help with product selection and configuration.
c.  Does the application use standard iconography and adhere to brand guidelines, as well as style guides that may be in use across other interaction points in both the organization or even the industry in general?
d.  Remember, not everyone gets visually engaged and that there are myriad forms of difficulty people may have with complex interfaces.  Balance the need to engage and delight with a practical approach to accessibility and use of standards.

Example KPIs for Visual Delight - Adoption, abandonment, page depth, time on site, transaction success, transaction times

4. Navigation
Now that we have some of the basics in place it's time to tackle the difficult tasks of information architecture, interaction design and navigational tools in order to shorten the distance between users and their goals.  Someone once said that architecture is 80% of usability - whether or not that is true is irrelevant to users but what they do know is that if they cant find what they are looking for, then it doesn't work. The Pareto principle is that oft repeated gem that informs us that 20% of the functionality is used 80% of the time - some people interpret this as 80% of people only ever use 20% of the features. Therefore we should make the most common or important functions easiest to find.  In various studies it has been proven that problems with navigation can be the #1 reason for abandoning transactions or support requests online, resulting in costly interactions with frustrated users to solve problems they should have been able to tackle on their own.  While this is a matter of much debate, and many experts weigh in with different approaches, there are some fundamental points that are universal to how we navigate applications and websites.  Consider the following:
a.   Is there a convenient, obvious way to switch between related pages or different sections based on an efficient navigational paradigm?
b.  Are you using any unknown, non-standard keyboard shortcuts?  Education is in order.
c.   Minimize user 'wet memory' load (brainpower requirement) by making all objects, actions, and options appropriately visible and reflective of their significance.
d.   Have you provided easy entry into the system and are fundamental links obvious in their intent and destination?
e.   Can a user get help quickly and efficiently, perhaps even collaboratively within the session?
g.   Have you given users freedom?  Do you support undo, redo, go back and reset appropriately and with safeguards in place to prevent loss of effort?
h.   DO you aggressively monitor traffic through your system. Which pages pique user interest? Which pages make users leave? Adjust your system accordingly.

Example KPIs for Navigation - Purchase, registration, page depth, time on page, time on site, path through site, adoption, abandonment, social feedback

5. Consistency
This is harder than it sounds.  To tackle consistency let's look at the sources of inconsistency.  Acquisitions often result in different product/service lines being represented, voiced and navigated differently.  Pre-registration (marketing) and post-registration (business process) applications are often broken into silo'd operations and this can result in confusion right at the point of transaction.  The need to release projects at different times can also result in inconsistencies across everything from brand to content presentation to navigation.  The source of inconsistency is valid.  The challenge is keeping a unified approach to how we present ourselves to users, and this is hard enough with a single project.  Maintaining consistency ensures that users learn and adopt more quickly and can apply learning to one part of the application based on their prior experiences from another.  We should use inconsistency only where users will clearly understand that things do not work the way they might expect.  How do we tackle this?
a.  Know the user (YOU are not the user)  and ensure that interfaces conform to user-centric and goal-oriented tasks.
b. Provide constant hints and reassurances that the current user is in fact the intended audience.
b. Speak from the outside in, in the users' language, rather than system-oriented terms.
c.  Users should not have to wonder whether different words, situations, elements or actions mean the same thing.
d.  Consider earlier areas like navigation and brand.
e.  Help users understand not only what the can and should do, but what they cannot do.

Example KPIs for Consistency - Success rate, abandon rate, support escalation, social feedback

6. Content
Sweat the small things.  This is a big area.  Well presented content is the exception rather than the rule simply because it is hard to do. Also many teams do not recognize the value of investing in a great copywriter or strong multi-media (lean-back) content.  The reality is that people wspend an overwhelming amount of their online time reading words on a page. On a PAGE - it is no accident that we refer to them as pages.  You need to adopt, develop or use a style guide in order to codify the rules and standards that are right for you.  You need to make several key decisions - languages and grammar, editorial voice, column layout, font, image handling, media handling, page depth/length, above and below the fold, the ability to rate or interact or share content and the ability to print for example, but most importantly you need to determine how to best present it based on your user's goals in order to get the right content to the reader as quickly as possible and in the most easily consumable manner.  Here are a few general themes that are must-haves:
a.   Content is appropriate to customer needs and business goals and help user to make a decision.
b.   Content is complete, up-to-date, accurate, with no redundancy.
c. Content, media, navigation and transactional capabilities work as an integrated whole
d. Nothing is included that is irrelevant.
e. Tags, keywords, categories and subheads are distinctive, descriptive, balanced and easily findable.
f.   Content across multiple languages is comprehensive.
g. Search works!  Do not underestimate the effort required or the power of getting search right on your site or within your application.

According to the W3C guidelines/standards for content, content should be:

    •    Perceivable – Information and user interface components must be presentable to users in ways they can perceive
    •    Operable – User interface components and navigation must be operable
    •    Understandable – Information and the operation of user interface must be understandable
    •    Robust – Content must be robust enough that it can be interpreted reliably by a wide variety of user agents, including assistive technologies (see Accessibility below)

Example KPIs for content - Engagement, time on article, sharing, social feedback, purchase/conversion, site path optimization

7. Performance
Scalability, availability, load times, wait times and how these are represented to the user ultimately determine if users refer to your site as 'slow and annoying' or 'awesome and easy' - great performance empowers users to give you great feedback and ensures that you are always on and building trust and credibility with your users.  This is ultimately an IT task as it relates to network, server horsepower, application server choices, etc but a lot of performance issues can be introduced by content owners and designers.  Two simple rules to observe from a user POV are:
a.   Does the system perform fast enough for users and when it is not performing correctly do you have a way to let them know what the problem is?
b.  Are all load times appropriate to the value of content, even on a slower connections - or do you provide specific services to those with slower connections?

KPIs for Performance - page/application loads, load time, simultaneous users, transaction processing time, abandons, feedback

8. Error handling
Well-written, helpful error messages are crucial to a quality user experience. Poorly written error messages result in low product satisfaction, and are a leading cause of avoidable technical support costs. First and foremost there are essentially two types of errors - system errors and user errors - and we should group our work into the functional areas of error prevention, detection and recovery.  You, the designer and developer, are ultimately responsible for how these are put in place.  Also there are two types of messages that happen when an error takes place - error messages and warning messages. An error message alerts users of a problem that has already occurred. By contrast, a warning message alerts users of a condition that might cause a problem in the future. Error messages can be presented using modal dialog boxes, in-place messages, notifications, or balloons.  Effective error messages inform users that a problem occurred, explain why it happened, and provide a solution so users can fix the problem. Users should either perform an action or change their behavior as the result of an error message.  Consider:
a.   Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
b.   How are you capturing user error levels, breakpoints, feedback and assessing that information and acting on it?
c.   Can a user recover all by themselves? What do users have to do to recover from errors? For example, Error messages should actually mean something to the user and tell the user how to fix the problem.

Microsoft and many other software developers have already put an extraordinary amount of effort into this space and we can learn from some best practices in the packaged application world to help us sort out graceful handling of errors.

    •    Is the user interface (UI) presenting a problem that has already occurred?  If not, the message isn't an error. If the user being alerted of a condition that might cause a problem in the future, use a warning message.
    •    Can the problem be prevented without causing confusion? If so, prevent the problem instead. For example, use controls that are constrained to valid values instead of using unconstrained controls that may require error messages. Also, disable controls when clicking would result in error, as long as it's obvious why the control is disabled.
    •    Can the problem be corrected automatically? If so, handle the problem and suppress the error message.
    •    Are users likely to perform an action or change their behavior as the result of the message? If not, the condition doesn't justify interrupting the user so it's better to suppress the error.
    •    Is the problem relevant when users are actively using other programs? If so, consider showing the problem using a notification area icon.
    •    Is the problem not related to the current user activity, does it not require immediate user action, and can users freely ignore it? If so, use an action failure notification instead.
    •    Does the problem relate to the status of a background task within a primary window? If so, consider showing the problem using a status bar.
    •    Are the primary target users IT professionals? If so, consider using an alternative feedback mechanism, such as log file entries or e-mail alerts. IT professionals strongly prefer log files for non-critical information.

KPIs for Errors - number of errors, time to error, recovery from error, abandon rate, customer service escalation reductions, social feedback, success rates

9. Accessibility
When we build interfaces it is important that we use systems to display information in the way that it was originally intended, to encode meaning rather than specifically focusing on appearance. Jakob Neilsen has long been a proponent of the theory that as long as a page is coded for meaning, it is possible for alternative browsers to present that meaning in ways that are optimized for the abilities of individual users and thus facilitate the use of the Web by disabled users.  Bear in mind though that buildings still have stairs, that is to say that it is probably best if you can optimize your onramps and sites for different types of users with different types of limitations, while still maintaining a high fidelity and optimal experience for users without these challenges.  It is a matter of balance and preference, but an interface that has been optimized for every possible use case is probably not ideal for any one type.  Some simple and basic rules to follow are:
a. Check and understand any legislation impacting your presence and be sure to comply with that
b. Study and understand the various challenges users may have, and how they overcome them, and use that information to guide your ability to help them further.
a.   Know the technology limitations: Identify and optimize for target browsers and user hardware.
b.  Think about adding collaboration or live help features if possible to ensure an optimal experience in the event of a problem or barrier that is unforeseen.
c.   Remember that not all disabilities impact users the same way or to the same degrees, for some users being able to magnify type might be enough, for others you need to be sure your site or applications work with the major screenreaders.
d. Allow users to customize presentation, without destroying the usability of the content.
e. Text is used to convey information rather than images of text.

KPIs for Accessibility - all KPIs so far as it pertains to unique audience requirements, compliance to standards

10. Help and documentation
I deliberately left it last to make a point - people always leave it until the end - and often don't spend enough time on it at all.  Ideally a system and the model for interaction should be discoverable and if it is based on common metaphors for navigation and interaction and follows the first 9 guidelines effectively users may never have a problem but you still need to be prepared for this.  For your organization it is an insurance policy on usability, and its also a great way to verify that a system works the way it was intended.  If the help file sounds too complicated, the system likely is too complicated. 
a.   Instructions for use of the system should be visible or easily retrievable whenever appropriate.
b.  Any information required in order to make a decision needs to be there when the decision is being made.
b.   Any documentation should be findable, searchable and written in neutral form.
c. Focus on the user's task not how the system works
d. List concrete steps to be carried out and help users understand at all times where they are in the journey

KPIs for Help and Docs - users helping themselves, users helping others, customer service escalations, social feedback

You may also want to better understand UX theories such as affordance which is essentially the quality of something that tells us or hints as to it's function and interaction model, or more sophisticated theories such as hierarchy of control or Fitts' Law which both have to do with the controls for an object and how we interact with them through mouse or touch or other means.  Once you are on the right path with the functions, design, brand and other key aspects of your application it makes sense to dig deeper into the patterns of interaction and controls that you are surfacing and start thinking about how to standardize them.

Growth in the user experience space and attention to the craft continues to grow.  I constantly find new folks to follow on Twitter, the discussion continues to flood my inbox, there are constantly new topics springing up in the IxDA forums, and more and more enterprise project discussions are revolving around the end user goals.  The job boards are also lighting up and this is fuelling even more interest in education, books and ways that people can get started in or transition their careers to a focus on user experience.  A lot of these discussions around acquiring skills lead to comparisons of education institutions but many of them also point to a few of the key thought leaders in this space.

No newbie to this space could go wrong understanding two critical thinkers in this space in order to get a grasp on the depth of information and help that is available and to understand some of the fundamentals.  Alan Cooper and Peter Morville, in order to help a get a grip on the basics.  Interaction design, information architecture and usability are essentially the fundamentals of user experience and as three key focus areas they complement each other to

Alan Cooper - Twitter - Interaction Design
An advocate of interaction design and agile methodologies, Alan runs cooper and writes books such as The Inmates Are Running The Asylum and the About Face series, which are mostly about how to make software user interfaces address user's goals. Cooper is sometimes called "the father of Visual Basic", and was the leading force behind Visual Basic 1.0, pioneering the use of an IDE to create a GUI via wrapped calls to system routines in the API (this is now referred to as the Adapter pattern).  Check out his blog for a rich history of the world of Interaction Design as many have come to know it.

I had the pleasure of working with Alan and his team when I was in product strategy at Yahoo! on a very interesting project - essentially thinking about design tools for designers and advertising tools for agencies.  Together with members of my team who were driving the project, we mapped out the entire agency ecosystem and actually built persona of the people that generally build persona.  Alan's iterative workstyle and deep insight provided the frank picture of opportunity that we needed to progress the business.  One of my favorite presentations from Alan is located here - The Wisdom of Experience.

Peter Morville - Twitter - Information Architecture
Peter Morville is president and founder of Semantic Studios, a leading information architecture, user experience, and findability consultancy.  He is probably best known for helping to create the discipline of information architecture as we know it, and he is a tireless advocate for the critical roles that credibility, findability, trust and other human desires play in defining user experience.  Peter's new book Search Patterns (co-created with Jeffery Callender) illustrates patterns of behavior and design across the categories of web, e-commerce, enterprise, desktop, mobile, social, and covers such diverse topics as relevance ranking, faceted navigation, multi-touch, and augmented reality.  Peter is a founder and past president of the IA Institute and serves on the advisory boards of Rosenfeld Media, Faz, Clueray, the Journal of IA, the Encyclopedia of Library and Information Sciences,, and Project Information Literacy.

Morville's honeycomb serves as both a companion and checklist for making great experiences.

honeycromb – Peter Morville
Peter Morville (P.Morville – Facets of the User Experience)
Let's take a look at how Peter explains the 7 facets of the UX honeycomb:

    •    Useful. As practitioners, we can't be content to paint within the lines drawn by managers. We must have the courage and creativity to ask whether our products and systems are useful, and to apply our deep knowledge of craft and medium to define innovative solutions that are more useful.
    •    Usable. Ease of use remains vital, and yet the interface-centered methods and perspectives of human-computer interaction do not address all dimensions of web design. In short, usability is necessary but not sufficient.
    •    Desirable. Our quest for efficiency must be tempered by an appreciation for the power and value of image, identity, brand, and other elements of emotional design.
    •    Findable. We must strive to design navigable web sites and locatable objects, so users can find what they need.
    •    Accessible. Just as our buildings have elevators and ramps, our web sites should be accessible to people with disabilities (more than 10% of the population). Today, it's good business and the ethical thing to do. Eventually, it will become the law.
    •    Credible. Thanks to the Web Credibility Project, we're beginning to understand the design elements that influence whether users trust and believe what we tell them.
    •    Valuable. Our sites must deliver value to our sponsors. For non-profits, the user experience must advance the mission. With for-profits, it must contribute to the bottom line and improve customer satisfaction.

You might also find his User Experience Treasure Map useful to guide you through the tools and artefacts of UX.

Another prominent fixture in the UX space is Jakob Neilsen and his writings and work around better practices for HTML may be useful if you are concerned with basic approaches for usability, accessibility and standard ways of presenting textual information.

It is important to understand how some of the more prolific and thoughtful leaders in the space are thinking but don't stop there.  Now more than ever there are tons of people posting around the web about and on user experience. The badge below keeps track of all tweets that are tagged #ux and for the most part the UX conversation on Twitter can be both enlightening and engaging.  It also provides a lot of jumping off points to other articles on the subject.

No comments: