Customer Journey Map – table of contents:

1. What is a Customer Journey Map?
2. Advantages of creating a Customer Journey Map
3. How to Create a Customer Journey Map?
4. Summary

What is a Customer Journey Map?

A Customer Journey Map is a visual representation of a customer’s experience with a company or a specific product. Such a map is a diagram that typically covers touchpoints, customer sentiment, pain points and actions in a set order. The main goal of devising a CJM is to empathize more with the user and form a clear vision for improving their experience. There are no rigid rules on what a CJM should look like – it can be either a simple, schematic table or an elaborate diagram, depicting many paths.

Above all, the map should be tailored to the company, the product as well as the project – to help the research and design team match the product to the customers’ needs.

Advantages of creating a Customer Journey Map

A customer journey map primarily provides project team members with one common source of information with which to work. It should help the team visualize the customer journey as a whole and better understand (and then solve) its problems. Perhaps the greatest benefit associated with developing CJMs is empathy. It is what makes us realize and reminds us that our customers are real people with real problems.

Customer Journey Map allows us to understand multiple customer paths and many personas. Maps help us understand where the similarities and differences between customer experiences lie, allowing us to reach different personas and solve their problems more effectively. They allow us to visualize customer pain points and understand which personas and types of users (customers) are most important from the company’s point of view. This data will help prioritize planned product improvements based on this most influential.

CJMs are also a great way to summarize and understand the research that has been done previously. They can also facilitate auditing your current operations to make sure they are in line with current customer needs. In the case of such an audit, it helps to see what the actual customer journey currently looks like – with all the flaws, problems as well as areas for improvement.

How to Create a Customer Journey Map?

Creating a customer journey map does not have to be complicated, but it will be essential to plan it properly and think about what purpose you want to use it for. To create a valuable CJM, several steps should be taken care of:

1. Conduct interviews with stakeholders

At the very beginning, you need to figure out how your team will apply the resulting map. If the team doesn’t do it, it won’t be effective – so it’s a good idea to determine what each team member should take from the customer journey map to find it useful.

The list of stakeholders will vary depending on the organization, the project and how the map is incorporated, as it may include people associated with the product, the UX team, analysts, customer service, sales or marketing departments. The main goal of stakeholder interviews is to understand in which form the customer journey map will be most functional and to identify key points of contact.

2. Decide which user personae to include on the map

The next step is to determine which customer journey you will map. To do this, you must decide which user personae (representing key audience segments) should be included. Defining the personas or customer segments at this stage will also help narrow down the pool of participants for the next step, which is user research.

3. Refine your ideas with user research

After conducting stakeholder interviews, identifying touch points and selecting personas for mapping, it’s time to get serious about dedicated customer journey mapping research. There are various ways to survey users to devise CJMs – the choice of method depends, of course, on our needs and time resources.

The best solution will combine qualitative and quantitative research. You can use the customer data you already have (from Google Analytics, Hotjar, sales or customer service data) and combine it with some in-depth interviews to expand the context and knowledge of your customers. You can also employ unmoderated usability tests, diary or ethnographic surveys, or a combination of these.

4. Create an empathy map for each persona

It’s time to use the insights gained from user research to complete empathy maps. The purpose of an empathy map is to learn more about how a person experiences certain things – this allows you to supplement your list of dry data about a persona and make it a little more human. You will need a separate empathy map for each persona/customer segment. The empathy map consists of 4 elements, covering what the user says, thinks, does and feels.

5. Sketch a customer journey map

At this stage, you should already have enough information to make a preliminary sketch of the customer journey map. Making a preliminary sketch at this point will help you create a more complete final map and focus on the most relevant areas. This step is a kind of audit of the process so far and helps guide the next steps.

This first, schematic version of the customer journey map doesn’t have to be detailed, it’s just meant to illustrate what you’re currently working on. The initial draft also helps identify areas where information needs to be supplemented.

Once you have a map sketched out, think about the lists of points of contact you made earlier. Consider whether your map is missing any of the most important ones. At this stage, you can also ask yourself questions about the individual steps on the map – is there anything that happens in between? Does anything need more or less detail? Review your initial notes from the brainstorming and stakeholder interviews to determine if your map covers all the things previously identified as essential.

Remember, this is just a draft – if you don’t know something, aren’t sure, or are missing information, you’ll be able to supplement and improve your map in further steps.

6. Choose the type of customer journey map that best suits your needs

Depending on the goal you want to achieve, there are several types of SJMs you can choose from.



Customer Journey Map – current status

These customer journey maps are fact-based maps that show what the current customer journey looks like. This is the most common type of CJM. Current state maps help identify, document and come up with ways to solve current customer problems.

Choose this type of customer journey map if you wish:

• communicate the existing problems to stakeholders, show exactly how the product does not meet the customer’s needs and convince them to take action
• focus the team around a common, better understanding and empathy with the customer
• explore new research and product opportunities based on the current state of the art

Customer Journey Map – future state

Customer journey maps in the future state visualize what the ideal customer journey could look like in the future. This type of map helps the team align on a shared vision of what your final product should be. generally, the best option would be to create a current state map first – since much of the same data will show up in a future state map. Future state maps require much more creativity from the team and hypothesizing about what the customer journey would look like and how it would differ from the existing (current) one.

Decide on a customer journey map in a future state when you want to:

• focus the team around a shared vision of where your product is going
• explore possible reactions, hopes and expectations of customers
• create a map for a brand-new product or experience that doesn’t yet exist

Customer Journey Map – a day in the Life of a Customer

A day in the life of a customer in the form of a customer journey map follows someone throughout the day. It documents everything a customer does – from their morning coffee to their dinner plans – whether these things will be related to your product. Such a map will differ from current as well as future CJMs in that their scope is not limited to your company’s touchpoints.

Choose this type of customer journey map if you wish to:

• learn how other elements of a customer’s life affect their experience with a product
• identify the moments when your product could be most useful to your customers

Service blueprints

A service plan can be a useful supplement to a classic customer journey map. Service plans cover all the little things that make up a successful customer journey.

Choose a service plan when you need it:

• learn about the elements that influence the current customer journey
• identify structural changes you can make to improve the user experience
• plan procedural or organizational changes
7. Select customer journey mapping tools

Choosing customer journey mapping tools is the final straightforward part of the whole process. There are scores of different tools useful for creating CJMs – select the ones you find easiest to use or have access to. Remember that the goal of journey mapping is to get a common, accurate picture of the customer experience, not to create a work of art. Therefore, if you don’t have design experience, select simple tools and focus on the content rather than the style of your map. We list some tools worth considering below.

Simple and publicly available tools to help create customer journey maps include: Paper, PowerPoint, Google presentations, Miro, LucidChart

Customer journey prototyping tools: Sketch, AdobeXD, Figma, AxureRP

Dedicated tools for creating customer journey maps: UXPressia, FlowMapp

8. Build a customer journey map!

The last step will be the easiest of all because you are just putting together all the elements you prepared earlier. Start by looking at your draft customer journey map – have you completed all the data you were missing? Do you already have all the information to create a complete map? If so, it’s time to translate all the data into a CJM and create a customer journey using the tool of your choice.

You can create your map from scratch, however, there is a wide selection of ready-made customer journey map templates only to fill in. This will save you a lot of time and allow you to focus on the content. When you’re done, be sure to review the completed map with your team to make sure it’s clear, accurate and functional for everyone.

Summary

Creating a customer journey map is not difficult, and it allows you to better understand the user and diagnose the most problematic elements in the product. It will also help build empathy and increase stakeholder engagement. Make sure to properly prepare for creating a CJM and do your research diligently. Having that, you will surely visualize the customer path, which you can apply in creating a better final product that meets the users’ needs.

If you like our content, join our busy bees community on .

UX research report – table of contents:

1. What is a UX research report?
2. Written reports or live presentations?
3. Types of research reports
4. How to prepare a UX research report?
5. How to present the research results to stakeholders?
6. UX research report – summary

What is a UX research report?

A UX research report is a document that summarizes all the details of the study conducted, including the research questions, the chosen methodology, important insights together with recommended next steps. The main purpose of reporting is to communicate the results to all stakeholders and provide accurate, objective information to help take further steps in the project. A UX research report is:

• A summary of the survey data and results,
• A well-written document with a standardized structure,
• Information often including links, charts, images and other data sources,
• A basis for decision-making supported by real data.

In the context of UX research, a report shared with those outside the research team should resemble a summary, i.e., a shorter document with more emphasis on the next steps and business application than on the methodology itself.

Written reports or live presentations?

Gathering and analyzing research findings is only half the job. The prepared report must be comprehensible to the audience who should extract the key findings from it and remember everything relevant to the project. It is therefore necessary to adapt the format of the research summary to the audience. They usually take the form of written reports and/or presentations.

Written reports will work well when talking to smaller groups, a distributed team or technical stakeholders such as engineers or other researchers. They can take the form of PDF files, emails, or pages in tools such as Confluence or Notion.

Presentations, on the other hand, work well when speaking to a large audience. This can include a slide show, presenting pre-recorded videos or conducting a workshop. Creating both a detailed written report and a presentation ensures that different types of audiences can access and use the findings.

Types of research reports

Research reports can come in different forms, which are typically tailored to the type of conducted research. For example, usability research will require a usability report, while competitive analysis will end with a competitive analysis report, etc. The most common types of UX research reports include:

Usability reports – documents that outline the background and methodology of the usability test, as well as key findings on user behavior, expectations and challenges.

Analytics reports – with qualitative and quantitative data to analyze past performance, these reports should summarize insights and make recommendations.

Competitive analysis reports – reports based on evaluation of data related to customer perceptions of the brand and product, the competitive environment or industry trends.

Qualitative research reports – written reports on research using qualitative methods such as 1-1 interviews or diary studies.

Quantitative research reports – written for research studies using quantitative methods, such as first-click tests or A/B tests.

How to prepare a UX research report?

A basic outline of a UX research report should include an introduction, research objectives, business value, methodology, key findings and recommendations. Below we will describe what exactly should be included in each of the above-mentioned points.

Introduction:

• a general overview of the research,
• business objectives,
• user pain points,
• potential business application.

Research Objectives:

• key decisions that must be made,
• research questions.
• hypotheses and expectations.

Business value:

• company-level goals,
• team-level goals,
• product development goals.

Methodology:

• the research methods used (including an explanation of why they were used).

Key findings:

• recurring, recurring themes and trends,
• relevant quotes, audio-video recordings,
• links, such as to a drive containing more source data.

Recommendations:

• clear, specified solutions and next steps in the project,
• recommendations indicated directly by users,
• suggestions for further studies to conduct in the future.

How to present the research results to stakeholders?

Once you have finished creating the UX research report, you should inform stakeholders about it and draw their attention to how valuable the document is. How to do this? Here are some tips to consider.

1. Understand stakeholder expectations

Typically, during the research process, interviews are conducted with stakeholders. Through them, the researcher can learn about their needs and goals, and thus understand what they can expect from the report presented to them. Therefore, it is worth including the most relevant (from the stakeholders’ point of view) findings and the impact of these results on their daily lives. You should also tailor the report to their preferences.

Stakeholders may have different needs and expectations concerning the conducted research. That’s why, you should place great effort to match the presentation to different stakeholder groups by providing, for example, several different formats and ways to interact with the report.

2. Define your own goals

At the beginning of the presentation, it is handy to remind the audience why the research in question was conducted in the first place. Then answer the following questions: What were the research objectives? How will the collected information be used? How will they influence design decisions?

Objectives are, of course, something the researcher constantly refers to during the research. Their value, then, is probably already known. Tying the study to the goals in this way leaves no room for stakeholders to doubt the importance of the work being done.

3. ‍ Explain the research methods used

The choice of research methods was not random. The researcher selected them to support specific research objectives. It is therefore worth showing stakeholders the process behind the selection of the methods in question, so they can better understand how the researcher approached his task. At this stage, therefore, it is important not only to explain what the method is but also to show how the participants were presented, how the research sessions were conducted and how the results were analyzed.

4. Choose an appropriate form of presentation

The presentation format chosen will have a big impact on whether stakeholders will understand the information they are given and how much of it they will remember. In most cases, they will not have the time or knowledge to analyze extensive sheets of raw, unprocessed data. Stakeholders often only need (and expect) a summary of key insights, presented in a way that is relevant, practical and easy for them to understand.

‍
5. Use storytelling

No matter what presentation format a researcher decides on, it is good practice to use narrative elements to keep the audience engaged. Storytelling is communication that moves people.

6. Attach research artifacts

While conducting the survey, research artifacts were likely collected in the form of videos, audio recordings, transcripts, emails, screenshots, photos or charts. Including such items will add variety to the presentation and keep the audience focused. It is also a way to show, “tangible” evidence to support the conclusions drawn and illustrate the findings in question.

7. Propose the next steps

This part of the presentation probably contains the greatest value for stakeholders. Based on the data discussed, the researcher makes clear and specific recommendations for further action, including future research, if necessary. This stage should therefore probably receive the most attention.

UX research report – summary

There is nothing more daunting in the work of a UX researcher than working hard on research and discovering interesting relationships, only to have stakeholders ignore or overlook them. This is why it is so important to prepare a UX research report in an accessible format. By properly preparing and presenting the results to stakeholders, the researcher will ensure a better understanding and support of the project.

If you like our content, join our busy bees community on .

Data analysis in UX research – table of contents:

1. Why analyze the collected data?
2. When to analyze the data?
3. Data analysis in UX research
4. Defining the objectives of the analysis
5. Qualitative analysis of research data
6. Summary

Why analyze the collected data?

Making a product decision based solely on raw data is a huge UX mistake. Skipping the analysis stage can result in providing users with an incomplete or ineffective solution, or even cause the project team to focus on solving the wrong problem or recognizing the real users. For these and other reasons, data analysis is an essential process that keeps the entire project on the right track. It does so by taking into account the real needs of users and gathering information that helps to develop the best and most optimal solution possible.

When to analyze the data?

Many people harbor a gross misconception that analysis should take place after completing the research, i.e, upon collecting information from many sources. However, this approach is ineffective, as examining such a large amount of data requires huge effort, manpower and time. It is more efficient to investigate data on an ongoing basis, for example, taking a few minutes after each in-depth interview.

Also, remember to take notes during your research. This way, you can put fresh observations down and make sure nothing is omitted. These reflections make you easily select information and choose from it those that will be most relevant for later design recommendations. Analyzing on an ongoing basis, after every little research step, allows you to conduct the final summary analysis in a far more organized and structured way, but above all, much faster.

Data analysis in UX research

Data analysis in UX research transforms previously unprocessed data into meaningful information that will support business decisions. Conducting a comprehensive data analysis consists of five basic steps – these steps are:

1. Defining the objectives of the analysis
2. Organizing data
3. Investigation
4. Clusterization
5. Identification of results and insights

Defining the objectives of the analysis

The first step defines the goals of our analysis – these should come in strict accordance with the goals of UX Research. At this stage, remember not to deviate from the motives that made you set off conducting research – for example, what are the user’s needs; on which page the rejection rate is more significant and why; what improvements to make to increase the conversion rate; or how to make our product more attractive than the competition. Holding on to these, and research objectives will help you understand how to conduct data analysis in a way that is useful for the project. To define exactly what you are looking for.

Organizing data

Each survey provides different types of data, more and less relevant to the project. Thus, you have to manage, select and filter them cleverly for usability. Organizing data also enables its thoughtful arrangement to swiftly scoop for the desired information when necessary. For example, you can catalog data by the sub-page of the website to which it pertains. Segregation is key to conducting efficient data analysis and improving its visualization which makes stakeholders understand the entire process better.

Investigation

The investigation phase lies at the heart of the entire data analysis process. Its main goal consists of identifying the words, ideas or phrases that appear most frequently in users’ responses and that are most likely in line with the purpose of the analysis. This process is not just about looking for words and their synonyms, but about understanding what they mean to users in their context.

Having that figured out words and expressions means depending on the studied user group. It happens because people vary. They have unique experiences and behaviors, as well as ways of expressing themselves. Hence, you should avoid transcribing user responses to your vocabulary. Instead, stick to the original as much as possible, because any variation, even the smallest one, can harm the investigation phase reshaping the entire data analysis altogether.

Clustering

The next step is to devise so-called clusters to label the answers according to those identified in the investigation phase. These clusters help the team to differentiate prioritized issues. For example, if more than half of the user responses fit into the created cluster labeled “Interface performance,” the team should probably prioritize this topic and look for issues specifically related to interface performance.

Identification of results and insights

Let’s not forget that results aren’t insights. Results concern the discovered, investigated, then grouped and cataloged facts that the research team brought to light through the process of analysis. Insights, on the other hand, refer just to the act of recognition of causes that occasioned the results. This is quite a distinct feature since user responses do not always lead to the source of the problem. The designer’s job, then, is to look deeper and search for insights.

Users are usually unable to identify the source of their difficulties on their own. The research team must therefore review the results during the data analysis process, discuss them, and then search for insights and match them to the research objectives. A workshop to identify the most relevant insights helps accomplish this task. Effective use of this tool involves conducting several rounds of discussion separated by short breaks.

The steps described above are a fairly general and standard data analysis process that works with any research method (both qualitative and quantitative). All you need to do is properly adapt the steps to your process.

Quantitative vs. qualitative data analysis

Although the process of analyzing quantitative data, is not significantly different from analyzing qualitative data, due to the nature of this research, designers may receive different insights. Quantitative research focuses on collecting and analyzing numerical data, using statistics and probability. Indicators such as the rejection rate of a given page, for example, or the demographic profile of a user, provide researchers with concrete and quantifiable information about how people interact with the product and the audience itself.

Qualitative research focuses more on abstract concepts, such as human behavior. For this reason, take a little more time to study and evaluate to fully comprehend user experience and opinions. It’s worth asking helpful questions at this stage, such as:

• What do users like most about the product and what do they like least?
• Why do some users react differently than others?
• Did (and when) users have an emotional reaction?
• Are (and why) users satisfied with the product?

Given the difference in the data received, it makes sense to use both quantitative and qualitative anecdotes as part of UX research. That way the gathered data complement each other and give clear and deeper insight into the results.

Summary

Properly conducted data analysis allows for better, more optimal design decisions. Omitting its findings leads to developing an incomplete, ineffective product that doesn’t respond to users’ actual needs. This is why data analysis is such a critical process that determines the success of the entire project. It empowers you to collect and select key information that, when translated into concrete design recommendations, helps to develop the best possible solution – tailored to users’ needs and requirements. The data analysis steps we described will help you carry it out in a structured manner and focus on what matters most.

If you like our content, join our busy bees community on .

Continuous research in UX – table of contents:

1. What is continuous research in UX?
2. User analytics
3. Customer opinion surveys
4. Use of data already in the organization
5. Continuous qualitative interviews with users
6. Summary

What is continuous research in UX?

The need to respond to user requirements, preferences as well as problems doesn’t vanish after a product launch. It means that UX research has to follow and stay put. As customers’ requirements change over time, some things have to get fixed, updated, or improved – all of which require research.

Continuous research (also known as ongoing research) is a particularly crucial element in keeping customer service high and ensuring that users have a positive experience when interacting with a product. They help maintain the product’s value and relevance to users.

User analytics

User analytics provides insight into product performance in its environment. Many people are unaware of how much data they already have – user and site behavior data collected by product teams can be harnessed to analyze users in continuous UX studies.

User analytics (also called user behavior analytics) is a form of continuous, quantitative data tracking and evaluation that takes place after a product is released. Teams use analytics tools to passively collect data about users’ interactions with a product, app or website. This data is then examined to better understand their engagement and sentiment.

User analytics can apply not only to UX researchers but also to outsiders teams (such as marketing staff, UX designers, UI designers, product managers, engineers or developers). It can feature, for example, to identify problems with a product, investigate hypotheses about a design or technical issue, monitor the user path, evaluate the user experience, target appropriately tailored marketing messages, present data to stakeholders and convince them of the validity of certain decisions.

So how does user analytics differ from UX research in the traditional meaning? Firstly, user analytics provides quantitative data by answering “how many” questions. The UX research answering “how” and “why” questions provides qualitative data. With analytics, researchers work on large datasets, while they take on smaller ones during UX research. They also differ in the method of data collection – in the case of UX research, the data is collected on an ongoing basis, through direct observation. In user analytics, on the other hand, data is collected passively and continuously.

User analytics allows you to adjust goals and metrics across the organization, better understand actual user behavior toward the product, improve product design and development, identify and forecast trends and, as a result, retain current customers and attract new ones – which will translate into the company’s bottom line and market position.

The process of conducting user analysis is as follows:

1. Knowing the users’ goals.
2. Defining the analytics metrics to track.
3. Configuring the system for measurement analytics.
4. Setting up a rhythm for viewing analytics.
5. Identifying trends, user segments and other patterns in the data.
6. Conduct formalized UX studies to explore trends, questions and opportunities.

By analyzing quantitative user analytics data – and combining it with qualitative data from UX research – you can identify areas for improvement, create exceptional user experiences, and ultimately increase company revenue.

Customer opinion surveys in continuous research

Continuous customer feedback surveys can help a company stay abreast of user sentiment and opinions on a product. The feedback can also support future UX research, product refinement and speed up the cycles of subsequent iterations. One of the best ways to gather feedback is through surveys. Below are some of the most popular solutions.

1. Net Promoter Score (NPS)

Net Promoter Score segments participants based on the likelihood that they will recommend a product to a friend. It’s a simple way to assess whether an experience is likely to spread through word-of-mouth, which can make a big difference. Often users are also allowed to provide reasons why they chose a particular answer.

If respondents rated the interaction high on the scale, it’s worth asking what they particularly appreciate the company for and what inspired their confidence to recommend the brand to family or friends. On the other hand, when they rate NPS low, you can inquire about the reason and expectations (what is worth changing). So this is a method to get not only quantitative data but also valuable qualitative information.

2. Customer Satisfaction Score

The Customer Satisfaction Score (CSAT), as the name suggests – measures customer satisfaction with a product or service. It involves asking users to rate their experience based on a predetermined scale. The CSAT is relatively simple and easy to use, but because the question is so broad, the reason for the answer can be difficult to decipher. Still, in some ways, it is a more straightforward question than the NPS and can help measure overall user satisfaction more simply.

3. Customer Effort Assessment

The Customer Effort Score (CES) measures how much effort it takes respondents to perform certain tasks, such as contacting technical support to resolve a problem.

4. Site capture surveys

Site capture surveys are mostly pop-ups that appear at key points in a user’s journey through the site to gauge their mood and feelings. This may seem like an annoying addition to the site, but if planned and implemented correctly, it can be relatively seamless for the participants and provide key feedback for researchers.

5. Email surveys

Email surveys involve questionnaires sent to users via email. With them, you can send customers the NPS, CSAT or CES surveys discussed above, but also provide them with more detailed qualitative surveys.

Continuous feedback surveys are a great way to monitor user sentiment at each stage of contact with a product. When choosing the right type of feedback survey, be sure to implement it carefully and according to your stated goals. Continuous surveys will provide an in-depth understanding of users, and when done correctly, help accelerate subsequent product iteration cycles, leading to better, more competitive products and loyal customers.

Data management within the organization

The use of sales, support, customer service and product data in user analysis is based on information from support tickets, error reports or consumer quotes. Knowledge about customers held by employees from other departments in the company is an immense source of valuable quantitative and qualitative data to acknowledge and handle wisely.

Sales, support, product or marketing teams, as well as UX researchers themselves, among others, may have useful data. It’s worth sharing the information gained, because the more different departments work together, the more the overall knowledge of users increases, resulting in the creation of an increasingly better product, consistent communication and improved user experience. To acquire such data, the researcher must cultivate good relationships with other employees and know who has the information and how they collect it.

Continuous qualitative interviews with users

Continuous qualitative user interviews are frequent one-on-one meetings with customers designed to enrich one-time surveys with fresh insights and prioritize the product development plan. Unlike user interviews conducted as part of a dedicated study, continuous user interviews are faster, iterative and open-ended. They are an essential part of continuous research in UX. They can take various forms:

• Broad interviews, gathering customer feedback, conducted weekly and focusing on general aspects of the product.
• Team-based customer interviews, conducted weekly by individual teams, focusing on topics specific to that team.
• Structured user interviews are held at specific stages of the design and product development process, focusing on answering more specific, precise questions.
• Repeated interviews with the advisory board, held regularly, to build long-term relationships with customers and improve future work on the product.

When deciding what continuous interviews should look like, keep the main focus on the objectives: What does the company want to achieve? What are the interviews supposed to help with? What questions does the company want answers to? etc.? Each organization is unique in terms of how the team is structured and what it plans to achieve through continuous research. You may choose to implement a combination of different types of interviews to encourage a variety of recurring customer conversations across the organization.

It is worth noting that continuous user interviews should never replace one-time surveys or other types of research. In addition to cyclical (weekly, monthly or quarterly) user interviews, you should exploit a variety of qualitative and quantitative approaches to maximize audience insights and extract as much valuable data as possible.

Summary

The continuous research discussed still does not receive enough attention from many organizations. Even though you have flawlessly executed the UX research process, designed a product that perfectly matches the users’ requirements and released it successfully, you should not stop researching because there will always be something to improve!

In an era of ever-changing trends, new technologies, ever-increasing user demands as well as ever-increasing competition in the market, it is necessary to constantly work on improving and developing the product, especially after it is released into the hands of users.

Continuous research will help obtain feedback from users and learn about their processes and feelings about the product. A good solution is to combine different research methods at the same time – to acquire quantitative and qualitative data at the same time, data from different users related to different situations.

This approach to continuous, repeated and cyclic testing will deliver the most comprehensive, yet optimal data set for planning effective product development.

If you like our content, join our busy bees community on .

Emotion evaluation in UX – table of contents:

1. What is emotion evaluation in UX?
2. Methods for studying emotions in UX
3. Aspects of emotions according to Don Norman
4. Summary

What is emotion evaluation in UX?

If by user experience we mean the entire experience of interacting with a website or application, then we must also take into account the emotions of our users. This is why it is vital to analyze and match the emotional side of the product with the users’ requirements. To achieve this, we should therefore “measure” users’ emotions during UX studies – especially during usability tests when they interact with the product or prototype. Although emotions may seem unmeasurable, there are tools to support this element of the research process and allow for the evaluation of emotions in UX.

Methods for studying emotions in UX

Unfortunately, most emotion evaluation methods are subjective and rely on the researcher’s accurate reporting of users’ emotional states during interaction with the product. However, keep in mind that the participants undergoing this kind of assessment must feel comfortable during the procedure, which forces the researchers to adjust accordingly and less formally testing conditions and schedule.

There is a specially created tool for this purpose called the User Experience Questionnaire (UEQ). It allows the researcher to evaluate a product through a special questionnaire in terms of its: attractiveness, expressiveness, efficiency, reliability, innovation as well as stimulation. Comparative pictures are an equally common method of measuring the participant’s emotions. The subject chooses how they feel at a given moment by pointing to specific pictures or animated images. The development of technology enables us to provide better and more authoritative solutions. For example, the FaceReader app allows us to read facial expressions directly and assign them to one of seven basic emotions (which include anger, surprise, neutrality, fear, disgust, sadness and joy).

Studying users at the level of their micro-expressions can prove handy in designing sales processes. For instance, when they can’t find something while adding products to the shopping cart, we can see their frustration or anger. However, microexpressions are often ambiguous because they don’t provide insight into the broader context – such as their previous experiences, expectations or habits. Despite assurances of staying calm, users may feel tormented by other feelings. Many of these are hard to explore as they remain in the unconscious. Researchers and designers should try their best to understand them, but the subject should get treated with caution and uncertainty.

Aspects of emotions according to Don Norman

Don Norman believes that the activity of designers is broadly human, not just their emotions. However, we think it’s worth having an idea about them and at least trying to study and analyze them in the UX process – sometimes the results can be amazing and can be a very useful clue!

Don Norman distinguished three aspects of emotion in design:

• Sensory – when there is an instinctive reaction at the sight of an object/product, e.g., pretty, colorful, complex. This aspect is based on sensual experience and senses – the look or smell of the product.
• Behavioral – involves satisfaction that an object works the way it should. It is functional, intuitive, easy to use, well-designed and practical.
• Reflective – stems from emotions that arise from the user’s identity: his values, the way he thinks about the world, or the meaning of a particular object

Any product can be analyzed through the prism of such an impression-behavior-reflection matrix. In addition to functional needs, the users want to satisfy their emotional needs. They may get frustrated not only by the fact that the application does not work as it should but also that it’s difficult to navigate or visually unattractive. Therefore, the product should be intuitive, easy to use, attractive and user-friendly. By combining functionality with visual appeal, we can expect to achieve our business goals, while building a positive image of the brand and the product itself.

Summary

Emotion evaluation in UX research is an often overlooked element. However, it is worth adding emotional analysis to the research process to understand better the overall user experience of interacting with our product and to create products that are not only efficient and useful but also pleasing to the eye and evoke positive emotions. Despite the few methods and tools for capturing emotions available on the market, there are still ways to help learn about the emotions of surveyed participants. The simplest of these would be to talk to users, asking them to describe their emotions and their changes over time – often it is the simplest solution to provide us with the most interesting information.

If you like our content, join our busy bees community on .

Task analysis in UX research – table of contents:

1. What is task analysis in the context of UX?
2. When to conduct task analysis?
3. How to perform task analysis?
4. Summary

What is task analysis in UX?

Task analysis is the process of learning about users by observing them in action (performing tasks). This method proves effective in finding out how potential users interact with the product and see whether they succeed in applying its functionalities (and to what extent). With the help of task analysis, UX researchers can get information about, among other things:

• What personal (or cultural) experiences do users bring to the task?
• How does the participants’ prior knowledge affect the approach to solving the task?
• What impact does the physical environment have on users’ actions?
• At what place and time do the participants perform the task?
• What is the estimated (from the participant’s point of view) time to solve a given task of the assignment?

When to conduct task analysis?

Task analysis is worth doing first and foremost before creating a user flow. If they don’t exploit a particular product function in the expected or intended way or can’t complete the process of achieving a certain goal, that something was overlooked during the task analysis.

Task analysis will be a good choice for early prototyping or research validation. By knowing the likely path of a user from point A to point B, you can base a design that meets realistic expectations (and not just assumptions) on it.

Preparation for task analysis

The data gathered for conducting a task analysis can come from user interviews, observational studies or another research method. To start analyzing a task, you should have enough information to answer the following questions without too much trouble:‍

• What makes users start the task?
• How do users know when a task has been completed?
• What do users need to know to perform this task?
• What tools do users need when performing a task?

Selecting the type of analysis

There is more than one approach to task analysis, and each of them, depending on the project, the nature of the research problem, overall circumstances or conditions, can work. One of the easiest methods to carry out is hierarchical task analysis.

1. Hierarchical Task Analysis
• Identify tasks and subtasks – the study should begin by identifying the main task, divided into smaller subtasks since each event needs its specific purpose. If the task contains more than eight subtasks, the issue under study is likely too broad or complex. In this case, it is better to separate the process, into separate cases.
• Draw a diagram – the next step is to create a diagram of all the activities required to complete the task and each of its subtasks. Not everything will be equally important to the project, so you should incorporate your knowledge, experience together with guidance from the data you already have, to determine which steps are critical. The diagram should show how the tasks relate to each other and in what order they should progress – if, of course, the order matters in this case.

You can draw the diagram in any way that suits the entire team. There is no set standard or any guidelines for what it should look like. It can be sticky notes taped to a whiteboard, a sketch in a notebook or a diagram made in a graphics program. What matters is that it should be editable if necessary and understandable to project team members.

• Write a story – diagrams are just figures and do not tell the whole story. They will mean little to someone who is not familiar with the task at hand. As an addition to the diagram, you should describe a story that will expand and complement it.
• Provide external feedback – having the diagram and story prepared, it is a good idea to contact a person (or even several people) from outside. This person may not belong to the project team but must know and understand the situation. Make sure to get feedback on whether the prepared description of the task and all its subtasks are clearly understood. Such guidance will enable you to catch ambiguities and understatements that need improvement.
2. Cognitive Task Analysis

Cognitive task analysis is similar to the hierarchical analysis described above. However, it differs in that in addition to looking at how the different steps relate to each other and how they are interrelated, it also examines how the user makes his or her decisions at each step, how many cognitive challenges are involved in each step, and how the overall process may vary depending on the user’s individual experience and level of knowledge.

3. Parallel analysis

Parallel analysis means that the same task is analyzed multiple times (with any method or even several different methods) to reflect the perspectives of different user groups. The tasks get examined by sampling various groups. In such a way, the final product becomes tailored for different target groups.

Another reason for conducting parallel analysis is to obtain and compare the other team’s data. Each can conduct its separate analysis and then compare the results.

4. Applying analysis in the project

At this stage, look for elements in the predefined steps (subtasks) in which the user can be helped in some way. Examples of solutions are to include recommendations and guidance or to remove subtasks that have proven to be unnecessary. However, this must come from the data, i.e. from the user’s perspective, and not from the assumptions or suspicions of the project team.

A well-created and consistent diagram allows you to identify the steps that can create problems and the tasks that can be automated in some way. At the end of the analysis, record all observations and, based on this, decide which design challenges should be improved and which are not currently relevant.

Summary

The task analysis is relatively easy. The more difficult part is certainly gathering the data necessary as it is worth remembering that common mistakes are widespread because intelligent, well-meaning people make them, sometimes even after learning the warning message. If one task analysis is properly applied it can become a key factor that will make your design functional and intuitive and your product enjoyable.

If you like our content, join our busy bees community on .

First Click Testing – table of contents:

1. What is First Click testing?
2. When to use First Click testing in UX research?
3. How to conduct First Click testing?
4. Tools to help with First Click test
5. Summary

What is First Click testing?

First click tests are a quick research method you can apply to any product with an interface – including a website, app or mobile site. Its function is to assess whether the planned navigation and link structure is effective in helping users navigate the site and find the information and elements they need.

The whole idea is quite simple: during the test, you show participants a model or design of the site and ask them which place they would click to perform a certain task. By analyzing the places clicked first, we can get answers to relevant questions such as:

• Which buttons, content and other navigation elements users find most intuitive?
• Which navigational elements do surveyed users overlook, avoid or use incorrectly?
• Where is the best place to put given buttons, icons, menus and other navigation elements?
• What is the most optimal path to accomplishing a given task and how does your project support or interfere with that path?‍

The results of a first-click test often take the form of a heat map (called a “click map” or “dark map”), which illustrates the most common click locations. Subsequent analysis of where users clicked will help understand whether the tested design provides an optimal user experience.

You may wonder why a first click is so crucial that we devote an entire article to it. This is because it determines the overall success of the session. A properly executed first click is a key aspect of designing a user-friendly and functional site.‍ Conversely, when the first click fails, the rest of the session tends to follow. Users are about twice as likely to complete the entire process if they get that first click right!

When to use First Click testing in UX research?

Since first-click tests are relatively quick and inexpensive, they can feature in a wide variety of projects and situations. In addition, the information gathered in the tests does not require in-depth analysis and allows corrective action to be taken quickly. They will prove effective at almost any stage of product development, as well as after the launch – to improve the product and functionality. It is worth noting that first-click tests will work well not only for websites but also for any other product with a user interface.

How to conduct the First Click testing?

A first-click test is relatively simple to plan and design, and it only requires the preparation of two basic elements: the page under test, a screenshot or blueprint, and the content of the tasks you want to test on that page.

Start by creating and writing down the tasks (research scenario). Determine which element of the site you want to study and analyze, and then determine the best path to perform this task (for example, you want to study where users look for information about the hours of operation of the bank’s hotline, in which case the best path to perform the task is, for example, to click on the “contact” tab on the home page).

During the study, observe where users click and record how long it took them to click. If it takes the user several seconds to find such basic information as the hours of operation of the hotline – it will mean that the information is hidden, too little highlighted on the page or the designed information architecture is not fully intuitive.

Also, pay attention to the user during the test – more specifically, their level of confidence and the level of difficulty they experienced with the task. Unfortunately, you won’t always have the opportunity to observe first-click tests in real-time, and thus gain insight into participants’ thoughts and behavior. If, on the other hand, you do have the opportunity to observe them during the tests, pay attention to several elements:

• Observe the facial expressions and body language of the test subjects during the test,
• Ask participants additional questions (e.g., why did they click at that particular place, why did they hesitate before clicking, why did they change their minds about clicking),
• Once clicked, don’t say if it was the right place, because participants must feel unhindered and must believe that there are no right or wrong answers – this will make them behave as naturally as possible

The answers you get to these additional, follow-up questions can be just as valuable as the quantitative data collected during the test (regarding when and where that first click occurred). However, if you are testing remotely, you can apply an automation tool or surveys during the test – to get additional information from testers.

Tools to help with First-Click testing

If you can observe first-click tests in person, it is possible to track clicks manually, but the most efficient (and also the most accurate) option is to employ one of the many tools available to record them:

UsabilityHub is a remote platform for user research that offers first-click testing functions, as well as, for example, preference testing, which we wrote about in the article “When and how to run a preference testing?”

Optimal Workshop offers tools for conducting various UX research methods, including card sorting, tree testing or online quantitative surveys. Their first-click testing tool is called Chalkmark.

UserZoom’s click testing feature allows you to conduct remote click tests on a wide variety of project variants, ask follow-up questions, and visualize the results on heat maps.

Useberry offers remote UX testing tools for various user testing methods – including just first-click testing.

Proven by Users is a user research platform that allows you to test many aspects of your project. It offers tools for first-click testing, but also includes card sorting, tree testing, surveys and many other methods.

UXTweak offers a variety of UX testing tools, including first-click testing tools.

HotJar lets you track and analyze user behavior on a live website – with heat maps for visualization and remote recordings of user behavior.

Summary

In conclusion, first-click testing is a quick, simple and relatively inexpensive method for understanding and improving a product’s interface. You can apply it at many stages of a project and quickly get questions answered. Remember, the first correct click doubles the likelihood of overall completion! Therefore, by testing and optimizing the first click, you double the success of your user interface – and thus the entire end product.

If you like our content, join our busy bees community on .

What is tree testing? – table of contents:

1. What is Tree Testing?
2. Tree testing vs Card Sorting and First Click testing
3. When to apply Tree Testing?
4. Disadvantages of Tree Testing
5. How to conduct Tree Testing
6. Summary

What is Tree Testing?

Tree testing means assessing the architecture of a designed website or application – in particular, its menu – that is, a kind of “branching” of tabs and subpages in the product (hence the name referring to a tree). This method is relatively simple to carry out and you can employ it early in the development process to save a lot of time and effort later on. Tree testing allows you to get the feedback you need to design a functional website or application that has a menu.

Tree Testing vs Card Sorting and First Click testing

Tree testing is also sometimes referred to as “backward card sorting” and to some extent resembles First Click Testing, so it is difficult to describe it without discussing other methods first them. However, note that these solutions are strongly related and may seem quite similar on the surface, but they have very different functions in UX design.

First click testing involves giving subjects a task – for example, searching for opening hours on a store’s website – and recording whether the first place a user clicks on is the right path for that task. Failure to click in the right place or taking too long to find information will suggest that something is wrong with the layout of our site. Either the site architecture is unintuitive or the design of the site is confusing or distracting in some way. It may also mean that the right place to click has been accidentally hidden or obscured. First click tests can take place both when wanting to examine a new design and to investigate possible problems in an existing design.

On the other hand, card sorting involves handing the participant a group of cards (paper or electronic on a computer screen), each containing a specific slogan/category/word. We then ask the subjects to sort the cards and group them in the most logical and meaningful way. Although this method has several possible applications, it is often employed as the first step in designing the structure of a website or application.

Having a basic grasp of card sorting and first click testing, we can move on to a broader discussion of tree testing. Tree testing involves showing the participants the architecture of the site and asking them where they would click to achieve a given goal (a mandated task). However, unlike first click testing, it does not end at this stage. The tester must take the entire path, from the first page to the last click. It is worth remembering that performing the first click correctly, does not guarantee success – the user can still get lost before completing the task. An incorrectly laid out page structure results in the loss of users.

Another significant difference between tree testing and first click testing is that the primer only covers the structure of a page, not its content or layout. During first click testing, invalid clicks can result, for example, from too big or too small buttons, their incorrect placement or wrong color – even if the basic architecture of the page is correct. In tree testing, none of these variables are relevant, because the participants don’t see the entire layout of the page but only a diagram showing which headings will contain which subheadings.

Tree testing is often referred to as backward card sorting because both of these methods focus very closely on the site’s architecture and its branching “tree” of options – or menus. In card sorting, you ask users to create “trees” for you, which you further test (tree testing).

When to apply Tree Testing?

You can conduct it very early in the design process since the site – or even a sketch of it – doesn’t have to exist yet. All that is needed for the test is a “tree” prepared at an earlier stage, such as when sorting cards with users. If the tree doesn’t pass the test, the problem is relatively simple and, above all, cheap to fix, since you don’t have to go very far back in the process.

Keep in mind that in the UX industry, there is no single ideal process or guidelines for the methods you must use every time on a project. It all depends on your situation, your time and human resources, your budget as well as the scope of the project itself. We should choose methods depending on the problem, what we are investigating, and what we want to achieve and improve in the product.

However, if you are working on a more elaborate site – which will feature a menu and at least several sub-pages it is worth working on the architecture specifically. Sorting the tabs by users, combined with a subsequent tree test, will enable you to prepare an intuitive, user-friendly and clear site architecture, resulting in a clear interface that is simply easy to navigate.

Disadvantages of the tree test method

Tree testing focuses only on the architecture of the site – which can come as invaluable help to a UX designer because if the test reveals a problem, we know exactly where it lies. However, many other things can go wrong in design, so don’t just use tree testing. To ensure the usability of the entire product, you also need sources of information about other aspects of the design and its functionality. Another potential disadvantage of tree testing could be that, because of the automated and remote process, it’s hard to get qualitative data from it that could pinpoint the causes of problems.

How to conduct a tree test?

1. Design the test

Testing a tree logistically is quite simple, as it doesn’t require moving, gathering materials or coordinating more people. All you need to do in preparation is design your test.

2. Select the tools

While it is possible to perform tree testing with a hand-drawn site map and a notepad, however, most testing is done online using specialized tools. They allow you to enter categories and subcategories (generally all menu branching elements), from which a clickable tree is created, adapted for testing.

Participants get a link to the test and perform the assigned tasks. They can do this in an unmoderated setting, in the comfort of their own home using a private computer, or in a moderated setting where the entire test can be observed. The tool tracks where participants click, how long it takes them, the order in which they click each object/element as well as how many of them click in the right places.

3. Create the tree

Of course, before you test the tree you will need to design it and decide what part of it you want to try – if your site is very complex and extensive, you don’t need to try all the paths at once. You can run an A/B test and asses two different trees at the same time – to evaluate which one works better. However, remember not to show both versions to the same tester.

4. Write assignments

When preparing tasks for your sampled group, try not to make them too easy, “find store opening hours.” Remember to avoid the same wording as the answer key. Second, the tester must be in the same mood and the same situation as the real user – the mind works differently depending on whether the person is solving a test or trying to solve an actual life problem. Give the tester a realistic scenario instead of one simple test question.

Also, don’t create scenarios that are too complicated: leave out irrelevant details that may confuse you. Remember that many participants will just browse through the questions instead of reading them carefully with understanding, and thus may mistake the extra detail for the central point of the task. One or two sentences are enough.

The same test can – and indeed should – include several tasks to obtain a bigger and more detailed picture of whether the tree is working as planned. However, you should keep to no more than 10 commands, so that the entire test doesn’t take too long and tire the tester out too much. If you plan to test more than 10 tasks, plan more than one test. You will never be able to test everything, every possible path on the page, so focus on the elements that are the most crucial to the usability of the product, and those that you suspect may be a particular problem.

Summary

Tree testing has the advantage over most other tests thanks to the modern tools available on the market as they are easy to prepare and conduct. We believe that we have succeeded in presenting tree testing, as a helpful, low-cost and simple method of design verification. Bear in mind though, that it will not work in every situation and – as a principle of most UX research methods – need supplementing with other tests for more reliable results and a broader understanding of the problems the users face.

If you like our content, join our busy bees community on .

Eyetracking in UX testing – table of contents:

1. What is eyetracking?
2. What does eyetracking consist of?
3. How to include eyetracking in usability testing?
4. Summary

What is eyetracking?

Eyetracking is a technology that records eye movements and enables the researchers to find out where a person is looking, at what or for how long. Thanks to such visualization, we can see the exact paths a user’s eye follows on a page. Eyetracking in UX research is very often employed in usability testing because it allows us to find out how users scan pages, and which elements catch their attention the most and which are overlooked.

Eyetracking is the only method to objectively and accurately record and analyze a user’s visual behavior. This helps in detecting usability problems without interfering with the user’s natural behavior. With this methodology, you can see live what the user sees and immediately understand where these problems are occurring.

How does eyetracking work?

The eye tracker directs light onto the eye, and sensors record the direction of its reflection off the cornea. At the same time, the camera takes multiple images of the user’s eye at very high resolution. The position of the eye gets documented many times per second.

Eyetracking tools generate interesting visualizations that can benefit the design process. By analyzing eyetracking visualizations, product designers can better understand how people perceive and understand the content and individual interface elements. The most popular visualizations are heat maps and gaze point diagrams. A heat map is a view of the location the study participant was looking at. It is created based on fixations – the places participants focus on and look at the longest.

Most eyetracking tools cannot only show the most often viewed areas but also combine fixation points to show the exact path a participant’s eye follows when looking at a particular page. Viewpoint charts allow the researcher to see what elements on the page piqued the participant’s interest as well as the order in which they saw them.

How to include eyetracking in usability testing?

It may seem that there is no better way to test UX than to see it through their eyes. However, to get the most out of eyetracking, there are a few basic things to keep in mind.

1. Consider other possible testing methods

Eyetracking surveys unfortunately require a lot of time. You need to recruit study participants, prepare the testing environment, and take the time to analyze the results after testing. Therefore, when you determine the scope of your research, don’t start with eye tracking right away. Consider whether you can employ cheaper, simpler testing methods. For many projects, low-cost methods such as user interviews or A/B testing may sufficiently extract valuable information that will help the project team make informed decisions.

2. Identify the purpose of testing and select appropriate participants for the test

As with other usability tests, it will be very important to determine what questions you want to answer. With a goal in place, you can define the tasks the participants perform during the test. Determine who your target user is and who will help you get the answers.

3. Recruit a sufficient amount of test participants

You should pick the number of test participants based on the nature of the tests and your individual goal. For the recordings you plan to analyze manually, 5 participants will suffice for qualitative eyetracking tests. For creating valuable heatmaps and gaze paths (charts), analyzed automatically, it is recommended to invite at least 39 participants.

4. Select a test environment

Determine whether you want to set up your in-house eyetracking team or outsource the testing to an external company. Determine where the testing will take place and prepare the environment – provide the appropriate equipment, the necessary personnel for the testing, ensure the comfort of the participants and quietness during the testing.

5. A retrospective repeat of the gaze

After completing the test, it is advisable to interview the participants and ask some follow-up questions to the survey. We also encourage you to ask the subjects about their emotions and feelings concerning performing the various tasks during the test.

6. Don’t use eyetracking as a substitute for other testing tools

You need to be aware that eye tracking is not a substitute for other usability testing methods. Remember that eye tracking is more of a quantitative test than a qualitative one. Eye tracking data will certainly help you understand that users are spending more time focusing on a particular element – while it will not tell you why this is happening. Qualitative information can only be obtained by interviewing users.

Summary

Eyetracking is still a relatively new technology that could change the way we create digital products. The ability to see a user’s natural interaction with a product enables researchers to identify real usability issues. Applied properly – along with other complementary research and testing methods – eyetracking can provide very valuable information to help inform design decisions and improve the user experience of a product.

If you like our content, join our busy bees community on .

]]> /eyetracking-in-ux-testing-ux-research-27/feed 0 /what-is-a-b-testing-in-ux /what-is-a-b-testing-in-ux#respond Mon, 06 Feb 2023 13:17:40 +0000 /?p=54622 A/B testing is an excellent research method for testing two alternative versions of a given solution at the same time. Read our article to learn how to conduct A/B tests and see their benefits and limitations.

A/B testing in UX – table of contents:

1. What are A/B tests n the context of UX research?
2. When to apply A/B testing?
3. How to conduct A/B testing?
4. Summary

What are A/B tests in the context of UX research?

A/B testing allows you to test two versions of a product/solution (version A and version B) and evaluate the one that wins over greater approval from users. The way to measure it includes conversion rate, the time spent on the site or participants’ feedback and their propensity to recommend the site/product. Before the test, you need to define and determine what “success” will mean for a particular version.

When to apply A/B testing?

You can deploy A/B tests for prototype testing, during the product development phase, as well as for building marketing and promotional strategies. They are the perfect tool for arriving at decisions that can affect an organization’s bottom line. A/B tests come in handy especially well when we already have a hypothesis based on previous research and want to confirm that it is the right solution. Research questions constructed for A/B testing might look like these:

• Which version of the product generates a higher conversion rate?
• Which of the two differently worded push notifications increases engagement in the app?

A sound A/B test should include as plain comparisons as possible, e.g. instead of comparing two completely different versions of the site, it is better to test two various header styles or two different distinct locations of the CTA button. With minor comparisons, we will precisely recognize which font, color, element, or location influences the UX most.

This research method comprises tests of two kinds: univariant and multivariant. The first one focuses on the differences between two variants of an item – for example, a red button and a blue button. The multivariant one, however, compares more than 2 variants of a button at the same time – e.g. red, blue, green, and white ( additionally, they can still differ in headings, e.g. “Check this” and “See more”).

The key upsides of A/B testing are swiftness and low costs. They also enable evaluating several product variants on a large group of real people. Still, be aware to focus on these aspects that can have a real impact on the overall perception of a product. Don’t compare random elements. Make a hypothesis, carry out other complementary research, then consult your design and development team. Together, you’ll settle which essential features to examine in numerous versions by conducting single-variant or multi-variant A/B tests.

A/B testing seems a quick form of research – though it’s not a rule. You may need to run them for as long as a few weeks to get enough data for UX analysis (but you can just as well get through a few days or even a few hours). The time it takes to run a survey depends on many factors.



How to conduct A/B testing?

1. Identify your problem.

Make sure to apply the right analytical tools to precisely establish the nature of the problem.

2. Find out as much as you can about the problem as well as the users. Get a good feel for them.

Pinpoint precisely the location of the flow and try to figure out why it happens. Its detailed understanding will contribute to a properly rigid analysis.

3. Formulate a hypothesis by answering how to solve the problem.

A hypothesis is a testable assumption. You can formulate it in the form of a condition – “if X happens then Z”, i.e., for example, “if the headline is in font 22 instead of 18, the conversion will increase”. A/B testing will let you know if the conjecture presented in the hypothesis is correct.

4. Define your goal.

Determine what you want to achieve with the study as well as through the entire research and design process – for example, you want more users to click on the CTA button on the homepage.

5. Define statistical accuracy.

Determine the numbers and figures you need for both the practical evaluation of the survey and for the business stakeholders to showcase – e.g., will a 2% increase in conversions satisfy them and be worth investing in a survey?

6. Define the required scale of results.

What number of respondents will ensure statistical accuracy? What percentage of the daily, weekly or monthly user base will make these results valuable and conclusive? It is imperative to determine this before proceeding with the survey.

7. Create version B and test your hypothesis.

Prepare an extra variant (variant B) of the site/product/functionality for your hypothesis and start testing. At this stage, developers step in to implement a second, alternative solution for the existing product – and users unknowingly split into two groups (group A and group B) the site/app as before. During the assessment, try to look at your data only after you have collected enough of it to get statistical validity and a viable result.

8. Analyze and act on the test results.

If your version B meets the established effectiveness threshold and they confirm your hypothesis, you can proceed to implement it for all users (no longer split between versions A and B). However, if the hypothesis is disproven, stay with the original version A or devise and test a new hypothesis. Also, check out alternative research methods to supplement the data.

Summary

A/B testing is a fairly technical subject. It necessitates possessing certain knowledge of statistics, as well as more specialized technical / programming know-how (or a good relationship with the company’s development team). It’s a direct method – on top of that it is quite simple, fast and cheap. It enables comparing two alternative versions of a product at little cost with satisfactory results. What’s more, its findings come out on the grounds of the real users, they are as precise as you can get. Still, remember that you can’t test every feature, element or tiny detail on the site – that’s why, when conducting A/B tests, it’s a standard to carry out other complementary research methods.

Read also: Discovery research methods

If you like our content, join our busy bees community on .

]]> /what-is-a-b-testing-in-ux/feed 0 /when-and-how-to-run-preference-testing-ux-research-25 /when-and-how-to-run-preference-testing-ux-research-25#respond Mon, 30 Jan 2023 08:07:29 +0000 /?p=53969 Preference testing is a popular UX research method. The key to its success lies in its good understanding and careful preparation. When and how to conduct preference testing? How do they differ from A/B testing? How can they improve the final product functionally and visually? Read on to find out!

When and how to run preference testing? – table of contents:

1. What are preference tests?
2. Qualitative vs. quantitative preference tests
3. When to conduct preference tests?
4. Preference testing vs A/B testing?
5. How to conduct preference tests?
6. Summary

What are preference tests?

Preference testing is a research method that involves sharing several (usually two to three) design options with test subjects and asking them about their preferences – which design they like better and why. Unlike other tests, preference testing focuses specifically on the visual aspects of products and design. This study can help you learn more about the user and their perception of the brand, and discover what feelings and emotions they have. Preference tests also allow you to evaluate a design in terms of its visual appeal, brand consistency, and overall credibility.

Preference testing proves handy in UX, as it gives researchers and designers direct insight into user tastes and information about what they think of different designs (visually). As a result, the method allows important decisions to be made early in the design process and saves the company from the unnecessary investment of time and energy in a design that is unlikely to appeal to the final user.

Preference testing aims to understand what visually appeals to the target user and why. The results of testing can be used at many stages of the design process, from general planning of color schemes or hierarchies on a page to specific decisions such as font and icon selection.

Qualitative vs. quantitative preference tests

We can distinguish between qualitative and quantitative preference tests. Qualitative ones usually take the form of an interview, during which we show the user different versions of the product and are then asked which one they like best. Qualitative tests also examine the user’s impressions and attitudes toward each of the designs shown, and allow them to answer not only which design they like best, but “why this one?”

Quantitative preference tests can take the form of a survey in which users select their preferred design and what attitude they have toward each. This allows feedback from a much larger survey group than qualitative tests, resulting in greater confidence that the results can be generalized to all users. Quantitative preference testing is appropriate when the design is relatively simple and does not contain numerous screenings, and when the company knows the reasons why users prefer one version more than another.

When to conduct preference tests?

You can run preference tests at various stages of the design process. However, they usually come early in the design phase to get initial feedback from users, even before the company invests time and money in the project. In this way, it can be determined which direction is more cost-effective and why.

Companies that are not building a product from scratch, but plan to redesign an existing solution, can conduct a preference test to check their design against the competition.

Preference testing vs A/B testing?

In the simplest terms – A/B testing is done later when the final design is almost ready and users can interact with it live, while preference testing is done early in the process – on a prototype, model, or even a sketch of the design. Preference testing is about understanding which designs a user prefers and why – before the product is finished.

Another key difference is that A/B tests rely on KPIs (Key Performance Indicators). They help determine how different variations affect user behavior and the achievement of business goals. As an example, a company that wants to increase the number of newsletter signups on its online store decides to A/B test the CTA button, examining which color of the item generates more conversions.



How to conduct preference tests?

1. Define research objectives

As with other methods, the most important thing is to define the objectives and research questions and write them down in a prominent place and present them to the test participants. Is the priority to understand which design variant users prefer? Is the goal to find out how they perceive each project individually?

It is also worth determining what kind of feedback the company wants – qualitative or quantitative. You should also make sure that all design options are functional.

2. Determine how to measure responses

Participants’ sharing of insights can vary depending on whether qualitative or quantitative tests were conducted. Here are some possibilities:

• Open-ended questions – this way participants can argue their choices, e.g. Which design do you prefer? Why do you like it? Was the information on this page clear and understandable to you? How easy was it for you to navigate this screen? What do you like about the design of this site?
• Closed list of adjectives – having specific adjectives at their disposal, e.g. classic, minimalist, elegant, participants will be able to assign them to the evaluated patients.
• Request to describe the variants in 3-5 words (adjectives).
• Numerical evaluation – will help determine which design best represents certain brand characteristics.

Participants in the preference test (as well as other research methods in UX) are mostly polite and usually refrain from direct criticism of the design, even if they are explicitly encouraged to be ruthlessly honest. For this reason, it is sometimes better to use indirect methods to find the real reasons for choosing one product over the others.

3. Gather test participants

According to the general principles of research with users, it is necessary to find test subjects who reflect the target customers as closely as possible. Depending on the stated goal, they can be current customers (if the company wants them to understand the context of use) or people unrelated to the product (if the study is to tip fresh insights, without bias).

Before recruiting, you need to determine the number of participants (20-30 respondents are recommended), the method of reaching them, and a possible incentive for participating in the study.

4. Conduct a preference test

Before starting the test, it’s a good idea to explain the entire process and expectations to participants. The test should not just show designs, but also ask follow-up questions. This will help the researcher find out why the participants chose a particular option, and how the prototype can be improved by considering their preferences.

5. Analyze the results of the preference tests

Conducting tests without analyzing the results does not bring much information to the company, so don’t skip this step. For qualitative testing, group similar responses and find the resulting patterns. With quantitative testing, it is worth extracting the most preferred answer. However, if the difference is not noticeable enough, retest on a revised draft.

Summary

Preference testing is worth conducting early in the design process, as it can help design based on user preferences rather than personal guesswork. Additionally, this solution is easier to implement and less costly than A/B testing.

If you like our content, join our busy bees community on .

]]> /when-and-how-to-run-preference-testing-ux-research-25/feed 0 /how-to-conduct-usability-testing /how-to-conduct-usability-testing#respond Thu, 19 Jan 2023 10:15:21 +0000 /?p=53345 How to conduct usability tests? Before we get to the answer to this question, we’ll explain what usability tests are in general, when to apply them and what distinguishes quantitative from qualitative tests. We will also hint at how to prepare for such tests.

How to conduct usability testing? – table of contents:

1. What are usability tests?
2. When to employ usability testing?
3. Quantitative vs. qualitative usability testing
4. Preparing for usability testing
5. Moderating usability tests
6. Summary

What are usability tests?

Usability testing is deeply rooted in the UX research process. At the testing stage, the team examines to what extent the developed product is applicable. Thanks to this, the researcher learns, for example, whether it is easy for users visiting the site to find a button or a given piece of information. Usability can vary in complexity depending on the designed solution. Nevertheless, the study always boils down to one thing – to investigate how well (or not) the target users can perform the tasks indicated by researchers as well as how eager they are to do so.

How do usability tests look in practice? In a nutshell, researchers ask participants to complete a series of assigned tasks, then observe their work and assess how well and efficiently they complete these tasks. Their ability to perform and their rate of completion informs the researcher of the next steps they should take in the project – whether the project needs further modification, and if so, what changes to make.

A significant aspect of these tests concerns their conditions. They do not require a special laboratory or another facility, but they can take place there provided it’s possible and the prospects for the valuable result look more promising. Still, typically the team decides on conference rooms to conduct the tests or carry out the whole process online.

As a rule, a facilitator with a ready list of predetermined tasks that research participants should perform conducts the usability tests to examine a given product or prototype. If the user is unable to perform something or has a problem with it, for the researcher this is a sign that the product needs additional work, such as deepening research, devising new solutions, or making modifications to the existing one.

Usability testing enables investigating any aspect of a product’s functionality, but this does not mean that you should test everything at once. For clarity of results and better organization of ideas, each test should focus on specific tasks/questions. This will make both the test and its results easier to collect and analyze, indicating an appropriate decision on the next steps in the project.

When to employ usability testing?

Usability tests should take place only after specific issues/questions have been prepared. For example, they may sound as follows:

• Are users supposed to find a given piece of information through several different paths (ways), e.g., through the search engine, main menu, footer, etc.? Does each path work equally effectively?
• By design, are users expected to purchase in less than 4 steps (clicks) from product selection to payment?
• Since submitting documents is the most important action on the site, the” submit” button must be the most obvious element on the site.

Before starting usability testing, you should have a functional version of the product or at least a prototype. For this reason, this test will not work well in the discovery phase. However, it will find its application at all later stages. Often, usability testing is even conducted at all stages related to product design, whenever there is an opportunity to interact with the product. This helps to improve and optimize the design on an ongoing basis and to prevent inventive errors.

Quantitative vs. qualitative usability testing

The difference between quantitative and qualitative tests lies in the questions to which the researcher seeks answers. Quantitative usability tests are, of course, linked to numerical data, and their analysis is primarily statistical and can be completely objective. The participants function as a representative sample, so the results apply to the entire population. Quantitative usability tests enable concluding, for example, that 45% of users have a problem with a certain function.

Qualitative data is not numerical but comes in a narrative or descriptive form. This analysis involves extracting valuable information from the data in a way that minimizes bias as much as possible. Such data could be, for example, narrative descriptions of a tester attempting to perform a series of tasks – in this case, the analyzed results would consist of a report and the information gathered regarding which tasks were problematic to perform. The qualitative analysis yields results that cannot represent the entire population but provides insights that help deepen and complement quantitative results, among other information. Qualitative data can, for instance, indicate why users might have a particular difficulty.



Preparing for usability testing

Proper preparation and planning, as well as a written plan, will help not only to run the test efficiently but also to explain the purpose of the test to the team understandably. The plan should include several elements such as the scope of testing, the number of testers as well as the purpose of the test – including the expectations and examined functions.

Before the survey, prepare the necessary equipment and plan the location. This can be the researcher’s office, the participant’s natural surroundings or a virtual link that allows you to conduct the test online, such as Zoom, MS Teams, or Google Meet. Remote testing is a good solution if the company cares about saving time or money and when the product is aimed at a niche audience from all over the world.

Some tests take place in lab settings where observers follow the actions of the moderator and participant through a Venetian mirror. The test subjects mostly know they are being observed, but they cannot hear what the team members are saying.

In addition to the participants, you need equipment to conduct the test, which is usually limited to a device with access to the product (prototype) and a few sheets of paper, pads, and pens so that observers can take notes. Optionally, the study can also be supplied with a recording device or screen capture software. This is practical especially when the study does not involve observers who can take notes in real-time.

The next step is to arrange and schedule meetings with the testers. It is necessary to determine how long each test will take, how many such tests to carry out are to be conducted per day, and what the expected time reserve is between test subjects.

Participants in the study can come from, for example, people obtained from the customer base. For qualitative usability tests, the study group should consist of between 5 and 10 participants. We have already written more about recruiting participants for the study in the article UX study participant recruitment.

Moderating usability tests

The moderator is responsible for the entire course of the test, from setting the tone of the test to asking questions, to providing the attendees with all the information necessary to complete the tasks. A good moderator encourages the participant to share their thoughts in real-time – in other words, to think out loud.

When ordering a task, state the goal of the task without going into the instructions for execution. Then observe whether the user succeeds in achieving it by solving the task. It is worth remembering about the language – it must be clear, simple and non-technical so that everything is understandable from the point of view of the participant, does not require additional translation and does not cause confusion. A scenario prepared in advance and tested (e.g., through pilot tests)will prove helpful.

In a situation when the subjects perform a task incorrectly, the facilitator shouldn’t correct or direct to solve it. The facilitator should let him or her work independently so that the outcome of the test is not affected. Although it may be difficult to observe a tester struggling with the task at hand, it is important to allow him or her to work independently to get the best possible data and guidance on what to improve in the project.

The test moderator cannot moderate and put down notes concerning the test at the same time. A good practice is to invite someone to the test as an observer (or several observers) to make notes during the procedure. If this is not possible, you can record the session and writie down notes after the test. For remote testing, many tools, such as Zoom or MS Teams, allow you to record the meeting. Some of them even have a function to generate an automatic transcription.

Summary

Usability testing is a crucial element of the UX process. It is the first encounter between the product and the user to fish out errors or shortcomings early in the design early and make modifications before developing the final product. It is worthwhile to carefully prepare for usability testing and plan the course of the study to draw the most valuable results and design guidelines on its basis.

If you like our content, join our busy bees community on .

]]> /how-to-conduct-usability-testing/feed 0