Skip to content

How to Conduct Remote Research

 By Userlytics
 Jun 08, 2020
 5865 views

How to Conduct Remote Research

Are you a UX researcher who mainly conducts in-person studies and you’re looking for some online alternatives to get you through this unusual period of lockdown and social distancing? Or perhaps you’re someone who is starting their path down UX enlightenment, looking to understand the benefits of remote research? Maybe you stumbled across this page by accident, in which case I argue that your so-called accident is actually fate and you should really consider incorporating UX research to improve your products (if applicable), your services (if applicable), or better yet, your overall life. 

Lofty hyperbolic claims aside, UX research is a great way to understand users, their needs, wants, expectations and attitudes; but you probably already knew that. Well did you know that Userlytics recently hosted a webinar titled “How to Conduct Remote Research” where we discussed the advantages, challenges, and best practices for in-person, remote, and unmoderated testing? Don’t feel bad if you didn’t because we’ve summarized said webinar below so that you can walk away with more options to add to your research toolkit. 

In-Person Moderated Studies 

In person moderated studies typically take place in a formal lab environment. You might have a one-way mirror that allows observers sitting in the adjacent room to see the participant and a number of cameras that allow observers to see the participant and their interactions from different angles. You can set up the testing environment to simulate a more natural setting, but at the end of the day, participants are acutely aware of the fact that they’re not actually at home. 

Some of the advantages of in-person studies include:

  • Having more control over the testing environment.
  • Physical interactions between participant, moderator, and stimuli.
  • The ease of testing with physical products/hardware.

Challenges associated with in-person studies include:

  • Limited geographic reach with regards to participants.
  • Being more expensive considering you’ll likely need to pay for plane tickets, hotels, and other accommodations. 
  • The potential bias of participants being in a corporate/lab setting.
  • Situations like the current Covid-19 pandemic where being in the same room with a stranger can be dangerous (and quite possibly illegal). 

To mitigate some of these challenges, here are some best practices we’ve come up with:

  • Greet participants prior to starting the session and ensure that they are comfortable with sharing their thoughts.
  • Stay mindful of the moderator’s body language in order to avoid introducing additional bias. 
  • Try to keep your sessions to no more than 60 minutes since participants and moderators alike will experience fatigue. 

Remote Moderated Studies

In remote moderated sessions, you still have a moderator who’s guiding the participant through the questions and tasks, but instead of the two being in the same room, participants are typically at home. The difference in environment introduces an entirely different set of advantages and challenges. 

Advantages of remote moderated studies include:

  • Less potential bias introduced by the moderator’s body language.
  • Less potential bias introduced by the corporate/lab setting.
  • The ability to test on a wider range of devices since participants will likely be using their personal smartphones, computers, tablets, etc. 
  • The ability to test with participants all across the globe. 
  • With fewer logistical challenges to overcome, remote studies are generally quicker and less expensive to complete.

Challenges of remote moderated studies include:

  • Having less control over the testing environment which can lead to distractions.
  • Inability to use more advanced testing capabilities such as blood pressure trackers, eye-tracking, etc.
  • Testing stimuli are mostly limited to software.

Best practices for remote moderated studies include:

  • Quickly familiarize participants with your platform (Zoom, Skype, Userlytics, etc.) prior to the session.
  • Consider which platforms may be inhibited by corporate firewalls (Userlytics’ no-download recorder operates from your browser, so no concerns here).

Remote Unmoderated Studies

With remote unmoderated studies, participants rely strictly on on-screen instructions that are programmed into the platform prior to the session. There’s no moderator present to steer the session and participants are completely on their own. However, because you’re not limited to the moderator’s availability, you can essentially have sessions running 24/7 and have multiple sessions running at the same time.

Advantages of remote unmoderated studies include: 

  • Shorter turnaround time for recruitment and results.
  • The ability to run larger studies with more participants.
  • Generally less expensive on a per-participant basis.
  • Highly scalable. 

Challenges of remote unmoderated studies include: 

  • Less test flexibility since there’s no moderator present to steer the session.
  • Inability to probe in unexpected or unforeseen directions.
  • Poorly worded tasks/questions may confuse participants.
  • Inability to react to technical issues in real-time.

Best practices for remote unmoderated studies include:

  • Ensure tasks/questions are clear and succinct. 
  • Use skip/branching logic to guide participants through the session.
  • Consider the amount of time a participant may spend on each task. For a 30 minute session, we recommend designing the test to take 20 minutes to complete to allow for slower participants. 

Deciding Which Method to Use

If you’ve made it this far, you’re probably wondering which method you should go with. Well, there’s no one clear way to decide which method to use because all studies are different and should be addressed on a case-by-case basis. To help point you in the right direction, we came up with 3 questions you should ask yourself:

(1) Does your study involve software (app, website) or hardware (physical product)? 

In-person moderated studies are inherently better equipped to handle hardware because the moderator is in the same room as the participant which allows them to observe the physical interactions as well as stepping in when participants get stuck. Remote studies work best with software. 

(2) Which characteristic of your study is most important? Physical environment, flexibility, or logistics?

Does your study require you to physically be beside the participant in a specific environment? If so, in-person moderated studies sound like the way to go. Perhaps you’re more concerned with having the option to probe and explore certain topics. If that’s the case, you’ll want to go with a moderated study, either in-person or remote. Maybe you’re working with a limited budget, pressed for time, or looking to test with a large number of participants. If the answer to any of those questions is yes, consider running a remote unmoderated study. 

(3) Are you looking to identify opportunities (generative) or test a product (evaluative)? 

Whether it’s in-person moderated, remote moderated, or remote unmoderated, you’ll still have opportunities to conduct either generative or evaluative research. Below, you’ll find a breakdown of each method and the types of studies you can run. 

Conclusion

User Experience, and particularly UX research, is a field that is constantly evolving to fit the ever-changing needs of businesses and users alike. Change is unavoidable and although situations like the Covid-19 pandemic may introduce seemingly insurmountable obstacles, the silver lining is that it forces us to explore other options and expand the ways in which we achieve our research goals. We’re fortunate to be living an age where human insights are only a click away; so why not make the most of it?  

A recording of the full webinar can be accessed here

Didn’t find what you were searching for?

Related posts

Things to be Thankful for: Advantages of Professional Peer Support in UX Research Thanksgiving is a time to reflect on all the things we are thankful for. As user researchers, we are thankful for the ability to connect with our peers and
2022 has been a busy and exciting time at Userlytics. We’ve launched several new features and capabilities, including sentiment analysis, accessibility testing, the ULX score, and VR testing. These newly implemented tools were designed to enhance your experience and provide you with
As staples of nearly all research conducted today, many people think they know about the differences between qualitative (qual) and quantitative (quant) testing. However, in the context of user research, these two methodologies are often used interchangeably. So how does one know
Understanding what democratizing user research means is not as simple as searching for the definition online, or asking a colleague for a quick summary. In a general sense, the universal definition is “providing all stakeholders with access to user insight and UX
It's no secret that a good website or app user experience is crucial for boosting sales. In an offline location such as a store or restaurant, being physically present may keep a prospect even if the experience is subpar. But in an
For many researchers working in UX, quantitative data analysis can cause a lot of anxiety. UX researchers provide interpretations that can impact businesses with said data, which can be an overwhelming amount of pressure. There are many reasons UX researchers are often
As a UX researcher or designer, it's important to have a clear understanding of your target customer before beginning a remote UX study. This helps you make informed decisions about what to test, how to test it, and who to target your
If you’re a UX researcher, you probably already know how important it is to carefully consider every step of a research plan: You need to make sure you understand the problem space, choose the most suitable research methods, identify and recruit the
In remote user experience and usability testing, there are a variety of factors that contribute to valuable, authentic user data that can be used to improve your brand’s user and customer experience. Among those factors are the quality and authenticity of the
Are you thinking about doing some user testing on your website, app or prototype? Are you currently weighing up the different options available to you? Should you use a free video-conferencing tool or a paid user testing platform? What are the differences
If you visited a complex website and found everything you were looking for with ease, it was unlikely a coincidence. A well-designed website structure is often the result of careful user research and testing, following information architecture principles and best practices. So
While remote user experience testing significantly enhances researchers' ability to reach test participants, these participants’ technical skills and access to technology can often vary greatly. Therefore, it is important to design and administer remote tests that facilitate participants' varying technical skills and
Userlytics prides itself in being a truly global platform. With over one million panel testers in over 150 countries, we are happy to say that we are able to find and test your target customers, wherever they may be. With the inception
In today’s highly digital world, cybersecurity is of the utmost importance when running day to day business operations. As usability testing grows in popularity, so does the need to choose a UX platform that values your company's privacy and security. But how
Userlytics makes user experience testing easy, efficient and fun with its state-of-the-art no-download web recorder. This revolutionary user experience testing recorder allows participants to jump right into usability tests directly with a simple click of a button; this means you can more
Moderated testing is a great opportunity to get thorough and instant feedback from users all around the world. This type of testing involves a live, online interview where you are face-to-face with your users via webcam, asking them questions, guiding them through
Researching the ins and outs of your business is integral to setting your company up for success. Increasingly, fine-tuning and improving your customers’ brand experience has become one of the top priorities for businesses. With customer expectations being at an all-time high,
INTRODUCTION Given the current world health and associated economic crisis, Userlytics ran a qualitative moderated ("Live Conversations") user experience study with participants in the US to determine how respondents’ online shopping behavior has changed as a result of the pandemic. The following
Imagine a scenario where a user is shopping online and stumbles across a product she likes, but the product’s description doesn’t tell her everything she needs to know before making a purchase (strike 1). To find out more, the user decides to
The Best and Worst UX in the Travel Industry Finding the best flight deals online can be an exhilarating experience. Almost every booking site today promises the lowest fares available and, in some cases, even a price match guarantee. With such a
Last November 18th our CEO was interviewed by Ditsa Keren, of Website Planet; Here are some excerpts from the interview: “If we take a step back and think about how software has evolved over the past 15 years, the cost and time
Designers are prone to falling into the trap of cluttering their interfaces. Limitations of space often push them to the wall, and they find themselves trying to fit everything they can into a tiny usable space. The unique problem that the modern
How Can Bad UX Influence Your Conversion Funnel? Source: Shengjun Shi Regardless of whether you sell a product, offer a service or run a blog, any visitor comes to your site with simple goals like consuming or gathering more information. The user experience

Let’s work together on your next UX study.

Create positive user experiences and keep customers loyal to your product and brand.

Analytics tells you what,
Userlytics tells you WHY.