Do computers cause dead parrots?

At a train station a few weeks ago I hit a wall of bureaucracy that enraged me beyond my usual breaking point. I had booked a ticket on the web, and at the collection machines, I was issued only half of the tickets (no, this isn’t my breaking point…).

At the ticket office, a Southern trains employee was initially helpful – he gave me the missing tickets. However, when I pointed out that I didn’t think they were the right tickets,  he told me that web bookings were handled by a different company, so he couldn’t help.

If that’s true, how could he even issue the tickets?

The problem I encountered may be twofold:

  1. as a customer I unknowingly used different legal entity when I booked on the web (confused by the use of an identical brand!), and….
  2. the employee may be boxed in by software, so that he cannot help even if he wanted to.

This incident has made me think about how customer experience worked before employees and customers had computers (mostly before my time then…). A few questions….

  • How rigid were the rules for employees regarding customers? If rules were on paper or communicated verbally, were they more or less likely to be followed?
  • Were interactions slower? Presumably. Did customers even expect speedy responses?
  • What about the quality of the interactions? I wonder whether working from a computer means that employees are less empathetic (because you’re referring to a binary system, rather making qualitative judgements yourself).
  • How does computerisation change the motivations for employees? If interactions are recorded in binary form, does it change the way they react to customers?

The cynical me is tempted to think that whoever codified the rules governing the issue of tickets in software saw an opportunity to reduce the chance of employees giving them away. In my case, there are unintended consequences; the honest customer is being punished at the expense of risk avoidance.

It may even be that this risk has been calculated; who knows. The only thing I can say with certainty is that the anger I felt would have instantly killed any prospect of future purchases were trains a free market (but of course, they are not).

Despite the cynical thoughts, I’m left with a feeling that computerization may be inadvertently narrowing the way employees can think and respond to customers. I wonder what Kafka would have to say about our modern digital life. I’d love to see a study about it (I couldn’t find one…).

Posted in customer experience, software, technology, ux | Leave a comment

Hands on! My experiences usability testing a printed booklet

At UX London this year Jon Kolko talked about using our UX skills to solve problems beyond the commercial, digital world. I think he’s tapping into a common desire in our industry to work outside of our typical domain.

I got lucky in this respect a couple of months ago; I usability tested a booklet that teachers use. I wish I could claim responsibility for this idea; however, it came from a rather smart client of mine! He recognised some potential issues with the booklet, and he wanted to test it and use the results to advocate internally for design changes.

When googling I couldn’t find much about anyone testing print products, but I am vaguely aware that marketers do use A/B testing (quantitative). When doing direct mail campaigns they will test the response rate from two different designs. I don’t know if I’m the first to do qualitative testing on a print product (I’d guess not), but if anyone wants to do this in future, I hope this post proves useful.

The setup

I did my best to replicate the standard recording tools when usability testing a digital product; i.e. two cameras – one on the participant to capture emotional reactions, and one on the booklet in their hands (ala silverback - full disclosure – I work for clearleft!). I didn’t manage to create a rig that captured a readable version of what the participant was looking at on the page, but it was certainly good enough to make a reasonable judgement about what happened during analysis. With a bit more hacking I’m sure you could get closer to a mobile testing rig. Here’s what I managed to achieve.
I used my macbook pro with the iSight camera on the participant, and an external USB camera on the booklet in their hands. I mixed the video together using camtwist studio (its free) in studio mode, sending the combined video to quicktime, which recorded the output and audio.
A few ‘gotchas’ I encountered:
  • the setup was fairly temperamental with two cameras. I’m told by the experts that its best to have one on firewire and one on USB – so its likely that there were hardware complications with both on USB. A third of my sessions were conducted with a single camera when the twin camera setup failed to work.
  • Quicktime runs in 64-bit these days, and doesn’t recognise the video coming from camtwist in this mode. You can make Quicktime startup on 32-bit mode by going to ‘Applications’ > right click on ‘Quicktime Player’ > ‘Get Info’ > check ‘run in 32-bit mode’.
  • camtwist prefers having the cameras use the same resolution – check the defaults and change as required.

The test script

Broadly speaking, I made no significant changes to the usability testing method. I tested mostly with new users: I interviewed them about their domain experiences first, and used their story to create a realistic scenario in which they would encounter the booklet. At this point I handed them the booklet (or a competitors), and gave them a task to complete, asking them to think aloud as they go.

The results

We had already suspected that the booklet was difficult to use, and the sessions confirmed this from the word go. The first participant got lost immediately when she mistook the distracting inside cover for the table of contents (which was visible on the facing page!).

Assumed knowledge: The traditional use of this booklet has required that users know a lot of things up front to get anything done. This problem raised it head very quickly – the terminology was confusing even to those domain-savvy participants. This problem wasn’t clearly visible to the organisation as the market is steeped in very old traditions that are rarely questioned, and new users are rarely encountered – the booklet is introduced to a teacher perhaps once in a lifetime.

Visual design: The typography, visual hierarchy, and layout were clearly very problematic, easy page scanning was near-impossible. We could all see this problem without testing it, but we needed actionable evidence so that stakeholders could clearly empathise with the users.

Navigation: The navigation of the booklet also proved interesting to watch: the participants spent a lot of time jumping back and forth between sections, and in a couple of cases searching in vain for an index at the back. It’s hard, in retrospect, to gauge how much of this was the booklet design, and how much was typical print-consumption behaviour. The booklet is not designed to be consumed in a linear fashion (and probably couldn’t be – as the sections would then contain huge amounts of repetitive information).

What was different?

Print behaviours: because I don’t have a history of watching how people consume print, it was harder to distinguish between genuine problems and common print use, although some of the flicking back and forth was clearly the result of a design issue. In the digital world we call this pogosticking – when a user is forced to click back and forth repeatedly to find information. We’ve observed enough behaviour to spot these common patterns – I’m sure there is research out there for common print behaviours, but I didn’t check beforehand.

The feel: users commented on the way the paper felt in their hands: “its like bible paper” and how the transparency inhibited readability. Perhaps the tactile nature, and the use of additional senses made it easier for people to express their relationship to the product.

The use: teachers who already use the booklet showed me their annotations in the margins. Seeing this made me wonder more about how limited digital products might be in other respects – you can’t just do what you want with a web page, but you can cut/rip/write on/bookmark paper in any way you like. Unexpected use might be easier to spot/guesstimate in the print world.

Emotional responses: when testing digital products, we know just how inclined people are to blame themselves for the problems they encounter: “I’m terrible with computers“. Although the frustration was at least partly caused by the design of the booklet, the negative emotional responses were expressed far more clearly than when I observe people using digital products. It was the first time I observed people sighing loudly in frustration, and at one point shove the booklet away!

I’m inclined to believe that people expect more from print because it is not historically perceived as complicated – and so they are much more critical of the product when having a problem.

Because of this difference, I’m inclined to do more of this kind of thing in future, particularly  to capture reactions to brand – it seems from this project that its easier to discern the reactions when the user is holding your product (and not a mouse)!

Next time

  • Setting up camtwist with quicktime is pretty easy now I’ve worked it out, but the camera issues need addressing. I might use a firewire-connected video camera next time as the second camera.
  • I will look for research into how people consume print before conducting testing.
  • I think this might be interesting to combine with emotional response testing for a clearer assessment of a brand.
Posted in usability, ux | Leave a comment

Apple’s Keynote… needs more testing?

At the end of last week, Steve Jobs (RIP) made me look like an idiot. In front of a large group of people. OK, I’m exaggerating (I knew a bunch of them personally!), but Steve had a close hand in Keynote’s design (apparently) and he wasn’t into usability testing, so he can take the blame. Boo Steve (well, really, yay Steve. Thanks for Keynote. And Apple.. etc. Ahem…)

So I was doing a short presentation/discussion to a department on UX and how it can help the organisation. I was ushered into the room (everyone’s waiting…) and shown the projector (never touched it before). Things were running a little late from a previous session.

I then proceed to make an idiot of myself for several minutes, trying to show the presenter notes on my laptop, and the slides of my presentation on the projector. When I plugged in, I just could not get it the right way round. Everyone in the meeting got a good look at my presenter notes and slides in advance. Way to spoil the fun!

Now in my rush to get started, here’s where I went to in Keynote to adjust the settings:

I played with the first checkbox, closing the preferences window several times to play the presentation, to see if I had corrected the problem. I di this a few times. No luck. So I start playing with ‘Customize Presenter Display’ – again no luck.

Now here’s the thing: I have set this up correctly many, many times (but probably always without the time pressure!).There are actually three places that you have to go, and I knew this already:

  1. System Preferences > Displays (to set which screen is the primary one)
  2. Keynote > Preferences > Presenter Display Preferences (I got fixated on this one, see above)
  3. Keynote > Preferences > Slideshow (completely forgot about it, see below)

My cognitive abilities to solve this problem were severely reduced under the pressure of having twenty people staring at me. My view of the problem narrowed and my memory disappeared – I thought that minor changes to ‘Presenter Display’ settings would fix it. All of my experience and problem solving ability flew out of the window. I know I’m not the only one; I’ve seen people struggle with exactly this problem at conferences.

Two thoughts then….

  1. Why the *hell* are all these display-related settings not grouped in one place?
    I think it would be totally legit to put that ‘Alternate Display’ setting with the slideshow preferences. Yes, you can argue the grouping both ways, but hey, *three* different places to set this up?
  2. Would testing have uncovered this problem?
    Not sure. If you were looking to simulate the pressure of this situation perhaps, but that’s not something we simulate often (if ever) in usability testing.I guess the only way you would know about this is customer feedback (i.e. this blog post). And we all know how Apple feels about customer feedback…

    *bush rolls through windy desert…*

Posted in design, software, ux | Leave a comment

Apps…. extinction ahead?

A couple of weeks ago I had the good fortune to attend the first designpush - working with some designers and some developer folks from chrome+mozilla. Our chosen topic was the emerging standard for webintents (follow that link for a better summary of it than I could give) – a standard for connecting applications based on what the user is intending to do. “Oh you want to share? Just press this button and your browser will find the service you use for that.”

With webintents, the pain of a developer needing to know which services to present to users goes away (how many variations on the share button are there in the world??), as does the idea that developers have to build in lots of functionality themselves. If it takes off (a major strategic/design challenge in itself), the impact would be huge. Developers could add an awful lot of value to an existing app without much effort.

For users the (not-so) interesting part is this: you might no longer need to remember what you use to solve these little first-world problems. Perhaps I could edit a flickr photo using photoshop.com right within flickr, instead of having to download, upload, edit, save, and upload again. Or more interestingly, I could pay for something without having to have an account with *insert abusive, monopolistic payment gateway here* as the site owner requires me to.

One thing that occurred to me during the opening talks was this: if that pain-point is gone, and services can plug-in to each other at will, then what exactly is an app any more? When I use my iphone, I’m often forced to remember which app I’m supposed to use for a task.  My iphone doesn’t have ten screens of apps (unlike some geeks I know!), yet I still struggle… “where is that thing again… what’s it called? What does the icon look like?”.

This is, of course, the problem that Siri is trying to solve, with decidedly mixed results. I think the interest in the Siri API is instructive: if Siri could take care of which app you use to remember the milk, or which one tells you your bank balance, then we’re not really caring about the app any more, are we? Maybe never even need to look at it. Install once, then….. forget?

My tired little brain got thinking about this after more ‘open web vs apps’ hand-wringing at the end of this article (warning, YASJA). From my point of view, the future is likely to make the ‘app’ as a recognisable, branded, marketed, identifiable concept somewhat redundant, if Siri is any indication of the future. Certainly the way we think of them now is feeling old already: apps might be in the background of whatever we’re up to in future.

This idea isn’t new. In The Humane Interface - Jef Raskin proposes the idea that stand-alone desktop applications should die – “every software package should be structured as a set of tools available to users on any document”. A less pessimistic way of thinking about it is this; there might be destination services (get them eyeballs, future facebook!) and then tool apps which simply facilitiate anonymously between them.

I have no idea what the commercial future is, but here’s my (probably broken) analogy: imagine our future use of the internet somewhat like the way an open source operating system currently works – lots of independently written bits of opaque code working together to produce a whole. It will require some incredible API code-fu (go developers!), and some magical interface/platform thing that changes how we interface with technology (go designers!), but hey, we all know it’s coming, right?

When, not if.

Posted in design, software, technology, ux | Leave a comment

Apologies readers, my blog images are toast: the wordpress theme was hacked!

I’ve been dutifully keeping my wordpress up to date, but I had no idea that wordpress themes were also vulnerable.

 

This hack allowed someone to delete all the uploaded images on my blog, but fortunately is fixed now and was no threat to the server. Time to go through my hard disk and find all those images again!

I’ve not been really happy with this theme anyway: perhaps its time to focus on a simpler, more typographical-oriented them; watch this space.

Posted in Misc | Leave a comment

I just *underpaid* my tax because of a poor interface

What if thousands of people were underpaying their tax by accident, and getting fined for it later? I can’t be the only one.

Paying your tax as a self-employed person is never fun. Some years ago I switched to a stricter accounting regime in order to ensure I was saving the correct amount of tax in advance into a separate account.

I submit my UK tax return once a year, which tells me what I should pay at the end. There’s a lot of things about accounting, tax, and the process that I am prone to forget (once a year only!), so there’s an additional burden on this interface to guide me through this complicated task successfully, without having to remember lots of little details.

Clearly I got it wrong this year: I’ve just received a letter informing me that I’d underpaid my tax, by a not insignificant amount. I won’t go into the boring details of tax, but the problem is caused at the end of the process:

  1. they shock a user by incorrectly asking for a very large payment that is two tax years added together – no idea why they even show this (see ‘WHAT THE HELL??’ in my screenshot)
  2. they don’t calculate and display the figure you should pay, even though the numbers needed for calculation are displayed right there. In fact, you’re mislead by a different figure. (see ‘OK, will pay right away’ in my screenshot)

Do they have any idea how error-prone this screen turns out to be?  Once again, I wonder if this is a page with an initially poor design, that’s resulted in a lot of expensive, labour-making mistakes. A few questions then…

  • Are they unaware?
  • What are the bureaucratic/cultural/cost barriers to fixing it?
  • Is there some technicality that means they can’t show the correct calculation?
  • Is it deliberate? Are they profiting from the resulting mistake?

Here’s my guess: there’s a stakeholder somewhere in the chain who believes that its the user’s job to know all of the little details about tax, including how to calculate. But here’s the rub: this screen actively misleads a user, as opposed to just leaving them to calculate on their own.

It would be better to show less, or nothing, forcing a user to to check with their accountant, or perhaps re-read how to do the calculation. But I’d prefer it if they showed the right number!

Posted in design, ux | 4 Comments

The delicate balance between design and build

There are some projects where significant design input is not required in the early stages. Perhaps the project is at such an early stage that you don’t even know whether the idea will fly.

Perhaps you’re trying to save money and just put something out there quickly, especially if the scope is fairly limited. To quote a startup guy I know: “I generally introduce (visual) designers to a new project pretty late, they cause big delays if they’re involved too early”. I have experienced this phenomenon: it wasn’t that design was unhelpful, we just did too much of it for an untested idea.

However, there’s a big risk involved that I’ve seen play out a number of times. Perhaps fundamental design flaws were built into the project, and it grew before anyone had a chance to ask critical questions. The structure of the product might be inappropriate, or the interface lacks consistency. In the race to get something working, the focus on actual use has been lost.

Suddenly a user base has grown around something that is broken. Perhaps the design inhibits growth. Redesigning is expensive (i.e. the problem is not just cosmetic) and may present the users with a pretty big change in design (never a happy event, even when the new design is better).

Its also clear that a certain level of design can be important when you’re looking to sell or advocate an early idea. I enjoyed Aza Raskin’s talk about rapid prototyping which touches on this.

In the corporate world UX designers are pushing to be involved earlier in the process, at a strategic level. I can’t disagree with that effort: I’ve seen plenty of projects that needed a smart design head much earlier in decision making (this is what makes Apple so successful: designers have the most revered role in the organisation, ahead of management). Often, too many assumptions about how people will behave were made by non-designers. The entire project might have been based on wishful thinking.

To solve this dilemma, I’d like to propose a framework for thinking about whether you need more ‘design thinking’. By design thinking, I don’t mean a visual designer, I mean someone who considers the *actual* use of a product, and understands the value of observing users, regardless of method (that isn’t limited to UX people, many others get this too).

Its just a first stab, so I welcome any feedback.

 

To involve a design thinker or not, that is the question (png).

 

 

 

Posted in design, software, ux | Leave a comment

Thoughts before buying the new Kindle

As I’m pretty close to getting one of these (I’ll admit that the new pricing pushed me over the edge), I thought I’d make some notes about the experience of reading books, and review after a few months of getting a Kindle.

 

 

 

  • My feelings about a book are intimately linked to how far through it I am. If something is feeling like a slog, and the bookmark indicates that I’m not very far through it, I’m quite likely to give up. I’f I’m further through it, I may stick with it. I wonder how the Kindle will indicate my progress, and if this affects my determination to finish a book. I have seen the UI (Don Norman showed me his DX at UX London 2009!), but not used it much, or seen the new UI in action.
  • Page turning is related to progress. With a 6 inch screen (diagonally) – which I presume will display less content than a typical book page – I wonder about how this will affect my progress through a book. Will ‘flipping pages’ more often make me feel like I’m making more progress through it?
  • The visibility of books is important. I’ve had plenty of conversations on trains where a shared interest in a book is the icebreaker. Occasionally, my wife has surreptitiously left books around the house that I would never normally read, but after a couple of pages I’m hooked. This kind of exposure to things not to my taste is enlightening!
    I also worry about this for my daughter’s sake, music just as much as books. I explored my parents’ numerous bookshelves and vinyl extensively as a child: how will she discover things hidden away on memory somewhere? James Bridle touched on this problem at dconstruct from a historiographical perspective: when all your data is hidden in folders on a computer, how do you know if you’ve lost anything?
  • Sharing. There’s a whole other discussion about copyright here, but I love giving books away. It occurs to me that perhaps the Kindle may act as a kind of filter for books – I’ll only buy the physical books that I feel are important to keep around or lend (I do this with DVDs).
  • I quite like the fact that I may not be able to do much more than read with it. Some may agonise over whether to get a tablet device or an e-reader at this moment in time, but for me there’s no dilemma. When I’m reading a book, I absolutely do not need the distraction of the web (this is why I probably won’t buy the 3G version!).
  • Visual content. I read a lot of things with diagrams and illustration, especially books related to my profession: how well will this work on the Kindle?
  • Articles. Another thing that pushed me into buying: instapaper can now send articles from your browser to the Kindle (with a single click). So often at work I encounter things that I want to read but don’t have time for, and would rather not read at my desk: this is a potentially great solution.
  • Control of content. It was only after I discovered Calibre that I decided to buy one (it’s like an open source iTunes for books). I have no interest in Amazon monopolizing what I can and cannot read, so the freedom to manage the books with Calibre is a big plus.

 

Posted in art, culture, ergonomics, hardware, media, productivity, ux | 3 Comments

When sitting around waiting produces better UX

Recently, I’ve been conducting some research for a medium-sized organisation that asked me to help them develop the next phase of their website.

As I sat waiting for a meeting in reception, I got to overhear some of the phonecalls they get from customers; mostly people needing help with a web-based booking system they’ve implemented (away from the website itself).

Hearing these calls, it became clear that the process of using this system (which is quite time-sensitive for customers) provokes quite a lot of questions for an unfamiliar user: they need to ask a human about specific form fields in the system in order to complete a booking.

An improved user flow + design will reduce these calls (time/cost-save) and instil greater confidence in users (increased customer loyalty).

The lesson here is that I identified a pain-point for users simply by sitting in reception. I didn’t use a computer. I didn’t even talk to anyone.

The website-specific work was unlikely to identify this problem: I got lucky really. However, its a great example of how thinking holistically about the customer experience can produce interesting results.

Posted in design, software, ux | 2 Comments

When Social Proof Is Unfair

 

I just received an email from 38 degrees asking me to vote on which action the government should take to mitigate the effects of deep cuts to public services (see screenshot).

Fair enough… it strikes me as pretty odd that the previous government’s spending policy is being blamed for financial crisis that was really caused by risky financial instruments that don’t add real value(ok ok ok… so the previous government didn’t exactly discourage this sort of thing….).

Back to the matter at hand: in asking for a vote from a user, they’ve presented the voting so far right when you’re standing at the ballot box. I can imagine this does encourage people to vote, but it will also be influencing what they vote for: people naturally go for options that they see other people taking.

You could be swayed away from the issue you felt strongly about just by seeing what everyone else is doing, which isn’t representative of the issues you might care most about.

Posted in culture, politics, psychology, ux | Leave a comment