Select language

You are here:

User research overview

During January we have been conducting a discovery phase. We have looked at what users do (search logs and analytics) and what they say (questionnaires and telephone interviews) to work out the most important solutions to design, build and test during the alpha phase.

We’ve used three main techniques:

  1. Data from web analytics for worldskillssaopaulo2015.com and worldskills.org - Helps to understand users, uncover usage patterns, see most popular content and use search logs to see what people are looking for.
  2. Web survey - Over 120 people answered the website survey in a week.
  3. Telephone interviews - We have conducted interviews with users from the different audiences around the world including students, teachers, participants, parents, journalists, policy makers and partners.

We’ve also talked to stakeholders at WorldSkills Abu Dhabi 2017 and WorldSkills International about what they need from the website.

Note: the analytics and the interviews focused on worldskillssaopaulo2015.com and worldskills.org, but during the survey we asked users about “WSI websites” to try to get as many people as possible to answer. When asked to specify, users referred to sites including worldskills.org, worldskillssaopaulo2015.com, worldskillsleipzig2013.com, National WorldSkills websites, euroskills.org. Information based on the survey needs to be considered with this distinction in mind.  

How do users feel about the WorldSkills websites?

Many WorldSkills users found the websites “informative”, “user-friendly” and “easy to use”. Some felt they were “boring”, “frustrating” and “bland”.

Here’s some of the most common words from the survey.

Common survey responses word cloud

You can see a bigger version here.

(Source: Survey)

Most common complaints:

  • “use of ‘inside’ language”
  • “It takes time to find the information I need because WSI has its own special terms”“complicated organisational structure”
  • “too many websites”
  • “too much information”
  • “difficult to find information, particularly older information”
  • “search doesn’t work very well”
  • “can be overwhelming for someone who only uses it occasionally.”

(Source: Survey)

How easy was it to find what you were looking for?

Ten is very easy and one is very hard. The mean was 6.1. That’s not a bad result. Two thirds of the sample group described themselves as “WorldSkills practioners” so it’s people who already know about WorldSkills and are familiar with it. 

Bar chart of How easy was it to find what you were looking for

(Source: Survey)

How likely would you be to recommend the website to your friends?

Ten is very likely and one is very unlikely. The mean was 6.6, so again that’s a good result.

Bar chart of How likely would you be to recommend the website to your friends

(Source: Survey)

Who are WorldSkills users?

New or returning visitors

New visitors accounted for most of the traffic (69%) with some users coming back more than once (31%). This opens up interesting questions about our approach to content: do we want to try build an audience that keeps coming back or do we try to make the most of the single visit?  

Pie chart of new or returning visitors

(Source: worldskillssaopaulo2015.com 20131206-20160105)

Location for worldskillssaopaulo2015.com

Map of visits to worldskillssaopaulo2015.com

This map shows visits to worldskillssaopaulo2015.com over the two years before the event. The darker the blue, the greater the traffic. As you can see, people came from almost every country in the world, but most of the traffic came from within Brazil. This audience could be people who live in Brazil as well as people visiting from other countries for the competition.

  1. Brazil (62.82%)
  2. Japan (2.37%)
  3. Switzerland (2.24%)
  4. United States (2.18%)
  5. France (2.17%)
  6. Russia (2.13%)
  7. Germany (1.87%)
  8. United Kingdom (1.71%)
  9. India (1.68%)
  10. Canada (1.10%)

(Source: worldskillssaopaulo2015.com 20131206-20160105)

Neighbouring countries

There wasn’t much interest from neighbouring countries, though it’s difficult to predict how much this behaviour would be repeated in the context of the UAE where “local” includes many different nationalities and where a stronger bond exists across the Emirates, GCC and MENA due to a common language and shorter distances. 

  • Argentina - 0.08%
  • Bolivia - 0.01% 
  • Colombia - 0.61%
  • Guyana - 0.00%
  • Paraguay - 0.03%
  • Peru - 0.05%
  • Suriname - 0.01%
  • Uruguay 0.02%
  • Venezuela 0.01%

(Source: worldskillssaopaulo2015.com 20131206-20160105)

Language

The local language was a bit more popular than English. Brazilian Portuguese was 55.44% and English was 44.56%. Note: many browsers in Brazil are set to pt even though people use the English site, so these figures may not be very accurate. Nevertheless this evidence supports complete translation: all content needs to be in English and Arabic.

Pie chart of languages on WSSP2015

(Source: worldskillssaopaulo2015.com 20131206-20160105)

Device

Just over 40 per cent of visits came from a mobile. We expect even more visits from mobiles for this website because mobile’s share of webviews is greater in this region and there is a general trend towards increased mobile usage. All of this evidence supports a device-agnostic approach.

Pie chart of languages on WSSP2015

(Source: worldskillssaopaulo2015.com 20131206-20160105)

Time online

Many WorldSkills users (over 70%) live online, ie over five hours per day. They access it at home, at work and on the move. The audiences are “connected”. We just have to reach them. (Source: Survey and user interviews)

Browsers

Chrome is the most popular browser, followed by IE (IE8 gets 13% and IE7 gets 2% of IE usage but IE6 has really dropped off), Safari and then Firefox and Android.

Pie chart of Browsers on WSSP2015

(Source: worldskillssaopaulo2015.com 20131206-20160105)

Traffic source

Most traffic comes from search or directly. Given that users go mostly to home and landing pages, it suggests search terms don’t match their queries accurately or they have general search terms. WSSP put considerable resources into digital paid-for advertising which is reflected in the fact that 20 per cent of referrals came from PPC. This raises questions about the type of engagement we consider successful: is it worth paying to draw people to the site for them to look at the homepage and leave? are we aiming for a deeper engagement from fewer users?

Pie chart of traffic sources on WSSP2015

  • google / organic - 36.54% - Normal search
  • (direct) - 21.46% - Direct (type in url)
  • google / PPC - 19.97% - Paid search
  • m.facebook.com / referral - 2.14%
  • facebook.com / referral- 2.10%

(Source: worldskillssaopaulo2015.com 20131206-20160105)

Social network usage

Most people use YouTube and Facebook. These figures don’t show Facebook’s dominance with certain users (such as parents) who only go online through Facebook.

  • YouTube - 78.5%
  • Facebook - 77.7%
  • LinkedIn- 48.8%
  • Twitter - 31.4%
  • Instagram - 28.9%
  • Flickr - 28.9%
  • None of the above - 9.1%

Given the new user types we are trying to reach who live in the UAE and don’t know much if anything about WorldSkills, it makes sense for WSAD2017 to focus on Facebook. Facebook need to be the testing area for all content, ideas and experiments with tone and voice. We need to ask ourselves who will share something and why? Then we need to test it: post it to Facebook, see shares, likes and comments and use that knowledge to improve. We also need to keep in mind that a lot of users may never come to the website and only see our content within the context of a feed. How can we design for this situation and how does this influence our design decisions? 

(Source: Survey and user interviews)

Top content

These are all the articles which had more than 1,000 page views. Traffic comes mostly through the home and landing layouts with big drop-offs at each interaction.

  1. WorldSkills São Paulo 2015 Medals and Recognition of Champions
  2. Press Accreditation
  3. Countdown for WorldSkills São Paulo 2015
  4. Opening and Closing Ceremonies: Two special shows!
  5. Ibirapuera Gymnasium venue for the Opening and Closing Ceremonies
  6. 12 August 2015: Future Trends

User flows

As you can see, not many specific articles received a lot of views. This could indicate problems around the navigation as well as the lack of specific interesting content.

Looking at the user flows (based on 20% of the visits), two thirds of users dropped off before their first interaction. These usage patterns raise questions about the effectiveness of digital advertising and the types of engagement we consider successful.

A user journey on WSSP2015  

Another quarter went after the first interaction.

A user journey on WSSP2015  

There was a separate, additional site for the 2015 competition at http://worldskills-experience.com/ We did not have access to the analytics for this site, but perhaps this site attracted users to specific content. 

We recommend keeping the focus within http://www.worldskillsabudhabi2017.com/ It will be challenging enough to attract people to the WSAD2017 site and make people aware that it exists without diluting their attention on to another site. For the navigation, we suggest surfacing important information (rather than having packing everything under a top navigation for “competition”) to avoid making people dig.     

Top YouTube videos

General explanatory videos performed well. A total of 820 videos received total views of 825,220. The top 50 videos accounted for almost half (397,000) of those views. 649 videos received less than 1,000 views. 534 videos received less than 500 views. It suggests we need to focus on producing fewer videos which get seen by more people.  

  • 2014-05-30 WorldSkills Sao Paulo 2015 Presentation Video - 25.8K
  • 2014-10-17 WorldSkills Sao Paulo 2015 - Short Presentation Video 15.9K
  • 2015-04-22 What is WorldSkills 9.3K
  • 2015-08-17 Closing Ceremony (full) - WorldSkills Sao Paulo 2015 8.7K
  • 2014-05-16 WorldSkills Sao Paulo 2015 8.4K
  • 2015-08-21 WorldSkills Sao Paulo 2015 highlight video 7.8K
  • 2015-08-14 Highlights - Day 2 4.7K
  • 2014-05-19 WorldSkills Sao Paulo 2015 4.2K
  • 2014-09-26 WorldSkills Sao Paulo 2015 - Presentation Video - PT-BR 4.2K
  • 2015-07-02 A New Look At Skills 3.9K
  • 2015-08-12 Opening Ceremony highlights from WorldSkills Sao Paulo 3.5K
  • 2015-08-17 WorldSkills Abu Dhabi 2017 Preview 3.0K
  • 2015-08-12 Highlights - Day 1 2.8K

Source: http://socialblade.com/youtube/user/worldskillstv and scraping YouTube

Word frequency / keywords

Search logs didn’t offer much. “Results” appeared a lot, which matches a point made in user interviews that it was difficult to find the results of the competitions.To try to discover topics of interest to WorldSkills users, we took the top 100 pages on worldskills.org and looked for keywords and word frequencies. It didn’t really show anything interesting or surprising, but “career skills explained” and “media news” emerged as the most important keywords supporting the idea of focusing on skills and their connection to the world. 

The full list is available here.

Results of this work

This overview has confirmed various principles around things like a device agnostic approach and translation. We have a list of user needs along with some personas for important types of users which we will be publishing soon.We will also be talking about what this analysis means in the context of designing, building and testing solutions in the coming weeks during the alpha phase. Thanks for reading!