Web Usability

[BBS-Dom]

At BigBathroomShop.com we are constantly trying to improve our conversion rates and user experience, we are currently using three / four services to track users and see how they are doing and if there are any problems with our design..

We are currently Using

CrazyEgg.com (Crazy Egg – visualize your visitors)
Clicktale.com (ClickTale | Web Analytics by ClickTale | Visitor Movies, Heatmaps & Form Analytics)

and an eye tracking and dedicated usability testing company called SimpleUsability (Eye Tracking Usability and User Experience Services - SimpleUsability Usability Testing and Eye Tracking in Leeds, Yorkshire, UK)

Does anyone else have any recommendations or what they like to use

[Niall_at_realeyes_it]

Hello BBS,
though it might appear somewhat biased I would suggest you look into quantitative eye tracking reports.

At Realeyes we specialise in eye tracking data reports on 50 Participants leaving the web redesign and improvement decisions to company website designers managers. We try to help them in making these decisions in making our reports as simple as possible to digest and making sure the eye tracking data contained is of a sufficient sample that solid decisions can be taken based on this.

Here is a typical benchmark report of ours:
http://docs.realeyes.it/Cameras_Benchmark.pdf

and here is a detailed report of the Argos page performance:
http://docs.realeyes.it/Argos/Argos.pdf

It would be interesting to hear how our samples and analysis techniques compare with the ones you are currently using.

Alternatively, for other ideas on Web Analytics that could be of use I would reccomend the Web Analytics Wednesday forum in London organised by SCL Analytics once every two months (next one is actually this wednesday). If you are interested I can post here the details of this event.

I look forward to hearing from you,
Niall

niall@realeyes.it

[Eyetracking]

Hi Niall

Nice reports. Quant data is great for supporting findings but the real benefit of eye tracking is in the way it allows us to understand subconcious and concious behaviour - your analysis gives no insight into understanding behaviour - so I'm not sure how much this truely helps webmasters make decisions.

We find that heatmaps and aoi analysis, just like your examples are fine, but where do you develop a model of why people are behaving in the way they do? All you can do with heatmaps and aoi analysis is hypothisize. That's why BBS have chosen us, an eye tracking agency ( Eye Tracking Usability and User Experience Services - SimpleUsability Usability Testing and Eye Tracking in Leeds, Yorkshire, UK ) that supports quantitative analysis (just like yours) with a qualitative retrospective analysis technique called PEEP (Post Eyetracking Experience Protocol).

Quant eye tracking is good for benchmarking, but we find site owners need to understand why users behave the way they do. That's the only way you can truely evolve user engagement - especially as more companies start to design for different online behavioural patterns.

Niall, how do find your studies stack up against traditional online multivariant testing where it could be argued online has a much larger sample set?

Guy

Guy@SimpleUsability.com

[Niall_at_realeyes_it]

Hello Guy!
Interesting reply!

I respectfully disagree with your main point that from quantitative eye tracking it is not possible to understand why a certain web function is not being performed optimally online.

In the analytics we present there are time based animations, that show how quickly various elements are visually digested; time to view time to click analysis which show if an element should be moved (average time to view high) or redesigned (average time to click high); and statistics (on a decent sample) that show the relative importance of various areas.

From these, given a clear understanding of the 3-5 key processes that generate revenue or are of strong user importance it is possible to understand,without the need to ask questions, what is getting in the way of the user's visual attention.

When a user is attempting to perform a task of business importance (buy the CD, rent a hotel, book a flight etc) I am not convinced that "deep insights" gathered from a user retrospectively justifying his eye movements deliver real business value and therefore I am not convinced a qualitative approach is necessarily a superior one.

The PEEP approach is academically sound, but does not mean user understanding insights cannot be found in other ways or that is necessarily delivers maximum customer value. However it has advantages with repect to its sample size (costs) and experienced consultant fees (revenues) which makes it clearly attractive...

The website design either performs and generates minimum failure %, or needs to be improved and made even more efficient and easy to use. I am convinced that with a quant analysis like ours with a large enough sample, a good understanding of why your user comes to your site and a good idea of the top business priorities of a website and experienced website designers the Why a function is not performing will emerge.

Regarding multivariant testing we have found cases where customers combine the two approaches to understand what is happening before the clicks are generated, especially for pages that are apparently underperforming. For example a page with a suspiciously high drop out rate or a reasonable % of users mis using the page design (going via secondary paths for an important function).

Would it be possible for you to share a qualitative report to discuss further?

I look forward to hearing your opinions!

Best Regards,
Niall

[Eyetracking]

Hey Niall

your approach is good for benchmarking, shows site owners 'what' people are looking at against a scenario - but at no point, does the user tell you 'why' they are doing it. We know that humans are not very good at understanding their our own behaviour - which is why peep is so much better than traditional research methodologies, such as focus groups. It is a tool for letting people explain both concious and subconcious behaviour.

We find that people use the internet at different speeds - so any measurements you take like 'time taken to do things' on a site, need to take into account the 'skill' and 'speed' of the user. There are also the recognised behaviour personalities - humanistic, methodical, spontaneous and competitive - which you could profile against for quant analysis to see marked differences in heatmaps. But even with improved segmentation it only shows you what they were looking at - not why.

We think user centred design is all about the why.

Back to your approach, I'm not sure how you can derive the 'why' from a red hotspot and some time/gaze metrics. Are the users fixating because they are confused or loving it ? I think what you're suggesting is that you and the web developers will make assumptions from your analysis - which isn't very user centred.

PEEP gives stakeholders a lot of things. Huge insight into concious and subconcious behaviour, a huge understanding of the natural vocabularly of users (seo and affiliate people love this), huge buy-in from the dev team, etc.

Our aproach does take more effort and needs skilled professionals - but the roi from our work justifies it and hopefully we'll be publishing some figures for the affiliate relevant work we've done, soon.

Guy
SimpleUsability Ltd.
Eye Tracking Usability and User Experience Services - SimpleUsability Usability Testing and Eye Tracking in Leeds, Yorkshire, UK

[Niall_at_realeyes_it]

Hello Guy,
Apologies for the delay, super busy lately, so as interesting as this topic is, I have other priorities

Interesting post! However, once again I respectfully disagree.

I think you are making 2 statements that I beleive are incorrect.
1) The Why cannot be understood from Quantative eye tracking

The best way for me to argue this is a concrete example of recent study we carried out on the online retail sector.
(It would be nice if you could reciprocate with PEEP).

We benchmarked the performance of M&S / Bhs / Debenhams and Next on the task "Buy a pair of trousers". http://docs.realeyes.it/Retailers_Benchmark.pdf

From the comparison of the designs we could see that the M&S page was struggling on most metrics wrt its peers.

By looking at the heatmaps we can see how users tend to go for menus as opposed search bars (in some cases not present).

By comparing the time to view time to click graphs for the best and worst performing pages on the menus we can see what is happening.

Here is the Debenhams Menu http://docs.realeyes.it/debenhams_menu.png
Here is the M&S Menu http://docs.realeyes.it/ms_menu.png
Here is a graph showing speed of seeing the Debenhams menu and speed to click on it.
http://docs.realeyes.it/debenhams_viewtoclick.png
Here is a graph showing speed of seeing the M&S menu and speed to click on it.
http://docs.realeyes.it/ms_viewtoclick.png

The comparison tells you that people are getting to the 2 menus quickly but the M&S one keeps people looking at it for longer which in turn is affecting their ease of use score.

Task is not "Shop for something you like" in which case a longer dwell time could be considered positive. The task is "buy a pair of trousers".

I don`t think there any other insights are needed to tell a web designer and his manager his menu is not as simple as it should be. I believe an experienced web interface designer with this report would have come to the sensible conclusion:
- Declutter the menu, make it simpler, space it out, fonts biggers, etc.. (btw I am not a web designer)

2) A why can only be understood from notes taken while users retrospectively comment their eye gaze movements.

Most of the tasks we perform and want to optimise on websites are extremely simple.
Examples:
- Buy the tiles / book / cd / find the store in your area / Contact us page / Book the holiday / etc....

Despite the apparent simplicity of the tasks at hand the web is plagued with pages that are sub optimal.

Once you spot that a webpages is not performing as well as it should (with Eye tracking Benchmark / Conversions % / Customer complaints etc... ) I don`t believe a psychological input is necessarily needed to fix the problem.

- What was he trying to do ?
- What did he focus on most ?
- What clues do I have from data available ? (see above)
- Where is the delta ? (where he should look Vs where did he look)
- What are my other business priorities ?
- If I move this item will my other priorites be affected? (therefore is this change a good idea?)

After this set of questions and answers a good designer and manager can make a call as to what to do.

I beleive the why a webpage does not work is simple: The design is not as users want it to be.
How you fix it? Look at how they are using the current design and modify it accordingly.

I feel eye tracking because of its relatively recent presence as a commercial tool has a quasi scientific/psychological nuance to it. However users on websites are trying to perform very simple tasks for these most "deep insights" are, I fear, unnecessary overkill (and using PEEP to figure out SEO keywords is also not efficient...).

I believe Eye tracking is more about customer value and ROI.

I suppose to proove me wrong you could post a PEEP document and set of insights deemed from this for us to discuss further.
Look forward to your thoughts and comments.

Best Regards,
Niall Bellabarba
Niall_at_realeyes_it

[Azam.net]

Some great info there. Thanks so much.

I have got really interested in web usability after going to a brilliant presentation on the subject at the a4u expo and can't wait to start using some of the tools above and those recommended by Ben Jesson and Dr Karl Blanks at the seminar.

[Eyetracking]

Hi Niall

We obviously have conflicting views on user centred techniques. We value the users' thoughts and emotions and use them to evolve our understanding.

Originally Posted by Niall_at_realeyes_it I don`t think there any other insights are needed to tell a web designer and his manager his menu is not as simple as it should be. I believe an experienced web interface designer with this report would have come to the sensible conclusion: - Declutter the menu, make it simpler, space it out, fonts biggers, etc.. (btw I am not a web designer)

Originally Posted by Niall_at_realeyes_it After this set of questions and answers a good designer and manager can make a call as to what to do. I beleive the why a webpage does not work is simple: The design is not as users want it to be. How you fix it? Look at how they are using the current design and modify it accordingly.

In essence, I think you are saying that you can spot areas that need improving through your eye tracking analysis. We agree, and also have this quantitative aspect included in our service mix.

Where I strongly dissagree with your process is the assumption that you or the designers are the best people to fix a problem, without understanding why your users struggled with the task.

I think it's arrogant to state that site owners can fix a problem with no insight.

Why do teams of people create broken/unoptimised websites/menus/functionality in the first place? Why will they be able to fix it when you show them a hotspot/metric that shows them that there could be a problem?

We believe that website stakeholders need tools/evidence for understanding the behavioural models of their different user groups to improve their online activity. PEEP is one of the best tools for doing it - ask any good eye tracking agency

You state in your response that managers/designers will know how to fix the problems you find without any indepth qualitative research, then I would have thought that you should be able to spot these problems without having to conduct a study for your customers.

Don't get me wrong, eye tracking is a fantastic quantitative analysis tool, but metrics will only get you so far along the ROI improvement path. Using eye tracking to understanding the users' behavioural models, will get site owners a lot further and open up new areas to explore. Staying ahead is no longer a game of fixing menus and optimised placement of functionality on a web page.

Anyway, I think we will have to agree to disagree. We're also quite busy

Guy

[D-Mac]

Interesting stuff! It seems that both the companies above offer a very valuable service

[Niall_at_realeyes_it]

Hello Guy!

Once again very interesting reply!
Yes I do agree that we are using a common technology from two different angles. I believe in a commercial arena the end choice will be of the customer. If the PEEP methodology holds customer value only the test of time will tell.

I suspect as more players emerge in the eye tracking space, more companies like Realeyes will challenge the need for an expensive interpretation of data given the simplicity of the tasks on the web and given that customers want SIMPLE designs for their clients (the basis of usability as I see it).

If simplicity is desired by all parties I struggle to see why a consultant using a psychological technique is necessary.

I believe there will be a time when outsourcing your eye tracking data collection and analysis will make sense (what we specialize in) and your core competencies of trusted advisers and designers helping your customers will increase your profitability.

You mentioned "I think it's arrogant to state that site owners can fix a problem with no insight."
I agree, there is always need for quality insight. But this is present in eye tracking data (given a large enough sample), a consultant can use this to advise the customer without being part of the data collection. You are of the opinion a consultant is needed to help the customer "understand" the data...

I think the differences of our opinions are inline with our respective business models which is also understandable.

You mentioned " Staying ahead is no longer a game of fixing menus and optimized placement of functionality on a web page. "
Completely agree, I think there is a lot more in helping companies websites "stay ahead", and eye tracking is one of the tools for the modern web designer and manager to understand online user behavior.

I take it you really don`t want to share a PEEP analysis document for us to understand its customer value ?

On a more conciliatory note, I would suggest you join the Linkedin Group about Eye tracking. LinkedIn: Groups: Sign In Eye-tracking
I would expect interesting discussions like this in there!

Would you be interested in discussing the next benchmark report we publish ? Ideally you could carry out a PEEP analysis on these and we could compare the data and the PEEP results?

Best Regards,
Niall

[Eyetracking]

Hi Niall

we're definitely talking two very different languages. The web isn't simple and we're not trying to dumb it down. Good usability is about ensuring the users don't have to think, that's very different to the 'simplicity' that you are striving for.

To illustrate what PEEP offers, I'll use an example.

In testing of a B2C retailer, one of the tasks involved a user browsing to find something they'd like in their house. All participants were potential customers of this retailer, our recruitment ensured that. In one session, a user scanned over a page of light fittings and bought something she liked, completing the task. When we reviewed the individual gazeplot of how she had chosen a product, we noticed a few small fixations in a pattern that suggested something was of interest to us. When we asked the user why she had briefly looked at the light fitting in this way she told us that she liked the light fitting, but wondered if it was available as a wall lamp. This insight was from a few milliseconds of eye movement, something that would not show on your sample of 50 on a heatmap. When asked, the user recalled memories from the task through looking at the gazeplot. This insight, with many others, is shaping the feature set of their new website. We think this level of understanding is amazing, and so do our customers.

For our customers, this is how they evolve their online proposition. This level insight is not about the efficiency of a 'buy a pair of jeans' task - it's about understanding the mind and language of a shopper to ultimately make their experience a pleasure and on-brand.

again, we have to agree to disagree on which service is of most value. Our propositions are aimed at different needs.

Guy

[BBS-Dom]

What have I started?

[Niall_at_realeyes_it]

Hello Guy!
Thanks for the PEEP insight - example, quite interesting!

I think the value of your offering is very much embedded in your staff as experienced consultants who can help your customers with elements that are omitted from pure eye tracking data. As you mentioned: features customers would like, "on brand" feeling, pleasure of online experience are all elements that need a consultant to be briefed on, investigate, collect data points on and present in a professional manner.

The customer either has those skills in house, as they are close to his core competence or feels there is a necessity to hire them outside specifically. I think the propositions are indeed aimed at different needs and more importantly I believe the two offerings are complementary.

It could be interesting to have a common project where the two approaches are show cased to the customer and we can collectively learn what are the value points for him. I would like to offer for us to come and present in your offices to see in what circumstances our approaches can be combined to offer the customer a higher value eye tracking product. Please contact me if this is of interest.

In response to BBS-Dom, I think instead this discussion is very appropriate and is what an industry should have more often to ensure we are delivering real customer value with an innovative technology.

Look forward to your thoughts!

Best Regards,
Niall Bellabarba