How can we help you?
Browse our most frequently asked questions so you can use Userlytics like a pro.
PLATFORM
What kind of User Testing does Userlytics do?
Both unmoderated as well as moderated (“Live Conversations”). Userlytics enables you to usability test the user experience of your websites and competitor websites, released and unreleased Mobile Apps, advertisements, and social media content, at all stages of the design and development process.
We capture the interaction, the verbal commentary of the participants and of the moderator if any, AND the participants themselves and their context through their webcam, in a picture-in-pìcture format, so you can be sure they are your target Persona.
Userlytics also provides Quant study tools, making it simple to gather useful UX insights from your study participants. This includes surveys, Card Sorts, Tree Tests, Time on Task measurements, Success/Fail rates, and other related metrics. When you need reliable statistics without using screen, audio, or face recording, you can quickly receive hundreds of Quant results from our panel of over 2 million participants.
How can I User Test on Mobile Devices?
We have native applications for Android & iOS for mobile testing of apps, prototypes, and mobile web browsing. You can also leverage our online/offline capability, allowing you to send participants to a physical store and compare the e-commerce experience with the in-person equivalent! We also offer an alternative methodology that allows the user to natively test the UI on their mobile device while viewing instructions and answering questions on their desktop/laptop computer.
Who benefits from User Testing?
Brand managers, Product Managers, Advertising and Marketing Agencies, User Experience professionals, Marketers, Mobile App and Website Designers and Developers all benefit from the capability to conduct remote usability testing on their prototypes, websites, mobile apps and digital assets within the context of participants’ homes and offices.
Do you have any recommendations or suggestions about how to create a good test script?
Yes, one of our senior UX Consultants hosted an in-depth webinar about this topic. You can view it here.
What changes do I have to make on my website or mobile App to conduct user testing?
None. You do NOT need to add snippets of code anywhere on your website. You do NOT need to insert an SDK into your Mobile App. You can conduct remote usability testing at any stage of the design and development and launch of your website, mobile app or other digital asset, with no type of intrusive modifications to your proprietary code. Full compatibility with all user experience design, prototyping and prototype delivery platforms.
How does remote unmoderated user testing work?
There are a number of different ways you can leverage Userlytics for remote usability testing:
You can use your own participants, or have them provided by a 3rd party user experience respondent panel, or ask Userlytics to recruit and incentivize the participants, or leverage our over 2 million member proprietary panel. You can configure participant redirects to specific landing pages for completes, abandons, etc. Respondents download a small app that checks if they have the necessary equipment (e.g.: webcams, when applicable) and that takes them through the tasks and questions you have defined
You can test a website, a wireframe, a set of static images, a video, a PowerPoint or Google Slides presentation, or any other type of digital asset accessible from a computer, including desktop applications.
You can test a mobile App that is available on the App Store or Google Play, or an unreleased App using TestFlight or similar, on both smartphones and tablets.
You can simultaneously view the screen interactions, listen to what the participant hears, and see the participants’ face and surroundings in a “Picture-in-Picture” (PiP) format video.
You can use our customizable templates for all types of use cases, work flows and scenarios for easy task and exit survey question formulation.
You can leverage skipping logic in the design of your unmoderated usability testing script, as well as in the design of a custom screener survey
You can create an invitation link for participants that forces them to register their name and email, or one that incorporates a unique identifier in the link (e.g.: for use with 3rd party panel providers) which you can send or post anywhere.
You can use a customizable respondent feedback invitation widget that you can add on any website page.
You can access the full video and survey question results, share access to specific projects with clients and colleagues, and download videos, quantitative metrics and survey results for your own reporting format.
You can access a hyperlinked “Video Highlights” section with key comments and events linked to specific videos and timeline moments.
You can download specific video clips with or without captions of annotation titles.
You can automatically create “Highlight Reels” of clips from different participants and/or studies linked by a common theme through the use of tags.
You can request an AI generated audio transcript of the videos with time stamped text for easy searching of videos and links to specific timeline moments.
You can request an analysis and executive summary, crafted by one of our Professional UX Consultants, of results with detailed recommendations for each issue uncovered.
How does Live Conversations (Moderated Usability Testing) work?
Very easy and very similar to an unmoderated user experience test, the main difference is you have a scheduling tool where you define the availability of your moderator (unless you wish to have Userlytics moderate). Invited participants then confirm the specific date/time they will do the test and both the moderator and the participant receive reminders prior to the selected date/time.
One very cool and unique aspect of the Userlytics moderated testing platform (“Live Conversations”) is that participants do NOT need to download ANY application, extension or plug-in; the system is completely “in-browser” and 100% HTML based, so participant onboarding is extremely seamless and can take place within corporate firewalls!
What languages do you support?
You can write your test instructions and questions in any language or alphabet. In addition, for formative studies, you can set the language of the recorder UI and participant landing pages into any of the following languages:
English
Spanish
French
German
Italian
Dutch
Japanese
Portuguese (Brazil)
Chinese (Simplified)
Swedish
Turkish
Finnish
In addition, our AI based automatic transcription of video sessions, provided by Amazon Alexa, is available in the following languages:
Gulf Arabic
Modern Standard Arabic
Chinese Mandarin-Mainland
Dutch
Australian English
British English
Indian English
Irish English
Scottish English
US English
Welsh English
French
Canadian French
Farsi
German
Swiss German
Hebrew
Indian Hindi
Indonesian
Italian
Japanese
Korean
Malay
Portuguese
Brazilian Portuguese
Russian
Spanish
US Spanish
Tamil
Telugu
Turkish languages
Can I have Observers in my Moderated Usability Testing (“Live Conversations”)
Yes, you can have up to 10 observers in every session, who can watch and listen to the moderator and the participant as they go through the session. If needed, more observers can be purchased as well.
What kind of instructions should I use for remote Usability Testing?
You can direct participants to perform any goal or task or question, in any order, including leveraging branching logic, to help you optimize the user experience of your websites, mobile apps, online advertising and content, and discover the reasons behind your analytics and funnel conversion results. You can also leverage pre-formatted activity types such as system usability scale questions, net promoter score questions, Matrix questions, Image testing tools, and have results automatically tabulated (as well as time on task, success/failure, SEQ, surveys, etc.)
We provide a set of customizable templates.
You might ask about:
Site navigation, Mobile App downloads and interactions,
The understanding of an advertising message, How do customers find your website or App or those of your competition,
The process for satisfying a need or objective, Ease of use and intuitiveness. Design, video, colors, layout, Specific processes such as: registration, login, check out, utilities, etc. , If they would use your site or recommend it to a friend.
Other user experience & usability questions specific to your user testing objectives, Comparisons with competitors, production and prototype assets, offline and online media
What is prototype usability testing?
Prototype user testing allows you to easily and quickly identify issues with the user interface and user experience of your prototype, mobile app, ad concept, storyboard, or wireframe during the early stages of design or development, and prior to spending significant sums of money on multivariate testing. Early detection of issues provides the opportunity to make necessary course corrections early in the design and development process, when change is less costly.
Early user experience testing of websites and mobile apps gives a better understanding of what features your users will appreciate, how and why they react to your message, and limits problems identified after your next product launch, campaign or upgrade. They also answer the question of “Why” participants engage in the behaviour you detect through analytics.
How does user testing of an unreleased Mobile App prototype work?
We can work with any platform you use, whether low fidelity prototypes hosted on Invision, Adobe XD, Figma, Proto, or similar, or unreleased high fidelity prototypes on the App Store or Google Play leveraging TestFlight or similar. Userlytics is compatible with any prototype platforms that can be tested through a link.
We project manage the entire process of setting up targeted participant profiles and Personas with the software required for conducting the mobile app usability test.
What kind of files can be uploaded for remote user testing?
You can user test any type of file (JPG, .GIF, .PNG, PDF, HTML, JS, CSS, etc), or Mobile App (Android, iOS, Free, Paid, released, unreleased…)
If you have a proprietary file format that you are unsure of, please contact us and we will be happy to review your needs and work with you to make your remote usability test project a success.
What if I make a mistake setting up a remote usability test?
You can run a “preview” of the test flow/journey/script, and then edit your directions, goals, tasks and questions or demographic selections until you are satisfied. You even change the test script or maximum session time or number of respondents in real time after your test has been launched and is live.
How quickly are results received with unmoderated remote user testing?
If you are using your own usability test participants, it depends on how quickly they respond to your invitation. Our platform processing time takes between 1 to 5 minutes, so you can receive results extremely quickly.
When using the Userlytics Panel, with broad based demographics, after a thorough QA review of every user test video, results are available in as little as 24 hours, sometimes within hours.
For narrowly based demographic targets, it depends on the filters and behavioural screener used to set up the project, and thus how narrowly defined the target Persona is.
For how long can I access usability test results?
You may access your remote user testing results at any time within 365 days after they have been delivered. You can also download the user experience test video results anytime within the 365 day period and keep and edit and share as needed.
How long is a typical unmoderated usability test session?
We can conduct user testing with the maximum session time set to 5, 10, 15, 20, 30, 60 minutes or any maximum duration you choose.
However, we recommend designing unmoderated usability test sessions for approximately 20 min session times, with the maximum session time set to 30 minutes to catch outliers. For moderated usability test sessions (“Live Conversations”) a default session time of 60 minutes is used.
What are the browser requirements to use and access all functions of Userlytics?
We recommend using Microsoft Internet Explorer 8 or higher, Mozilla Firefox 4.x or higher, Apple Safari 4.x or higher, and Google Chrome 4.x or higher.
What does “Paused Test” mean?
It means that the test has been completed (for now). We will keep our testers updated if there is an opening of new slots. When you receive an invitation to take one of our tests, please make sure to respond as soon as possible.
Can Userlytics manage user experience studies with persons with disabilities?
We are working on making the platform 100% accessible for testing of participants with disabilities.
If you are thinking of launching this type of study, please consult us first at operations@userlytics.com so we can assist you.
What is a starting point URL, and do I have to define one?
The “Starting Point URL” is the website, app or prototype your participant will immediately be pointed to when they begin a user experience study. The decision to include a Starting Point URL depends on whether you are conducting an unmoderated test or a moderated test, and what your test goals are.
If you are conducting an unmoderated study and would like to immediately point your participant to a URL at the beginning of the test, you will need to fill the starting point URL. When you place your URL in this field, the user will be sent to that location at the beginning of the test. However, if you would prefer to ask a few questions before directing the user to a site or app, toggle “Define Destination URL(s) inside the TestScript.” This will allow you to place your URL at a later task.
If you are conducting a moderated study, you do not need Starting Point URL. By default,“Define Destination URL(s) inside the TestScript” option will be toggled if you select “Live Conversation” as your test type. This is because our updated Live Conversations recorder allows you to direct your participant to your website, app, or prototype at any point during the session by clicking the “Share URL With Participant” button.
What recording methodologies can be used for tests taken on mobile devices?
You can use video, audio, and screen recording for tests on mobile devices, except when testing a website on iOS devices. Due to technical limitations, you can only use audio and screen recording for website tests on iOS devices. If video recording on iOS devices is essential for your study, please contact our operations team at opsteam@userlytics.com, and we will help you.
How do I conduct custom usability tests?
Please contact us to learn more.
When moderating or observing a mobile moderated user testing session, should the Moderator or Observer also participate in the study using a mobile device?
Both the Moderator and the Observer(s) should participate using desktops or laptops during a mobile user testing session, with the participant accessing from a mobile device.
How far in advance can I cancel a moderated session to avoid getting charged for it?
The current policy for cancellations is that you're free to cancel any sessions more than 4 hours in advance. For cancellations under 4 hours, we do have to compensate the participants, so we might charge 50% of the total session cost so we can cover our costs.
What if I wish to cancel during the session?
If you believe the participant misrepresented their skills during a moderated session, was unresponsive, or had other ethical difficulties, you should discontinue the session within the first 10 minutes of the session. Userlytics will review the session and replace the participant.
How is the duration of quantitative studies calculated?
The duration for quant studies is measured automatically by an internal algorithm based on the number and types of tasks in the study. For example, if you add a few quick rating questions the system will allocate several seconds per question, but if you add some verbal response questions or a card sort then the algorithm might allocate half a minute or so. You don't agree with the calculations and would like to make any changes, you can simply let us know and we'll be happy to adjust this for you directly. One thing to keep in mind is that we give participants a buffer of about 25% more time to complete Quant tests, so even if the test duration is set to 15 minutes, the "total time will be closer to 18–19 minutes.
Does Userlytics have diary study capabilities? Or any case study where an organization used the platform to execute a diary study?
Userlytics does have diary study capabilities, but this service entails an extra cost. We would require the below information to prepare a quote, as pricing differs depending on the requirements. Can you please help us with the below requirements and any other information you find relevant? Research objective Timelines Profiles Number of participants Country Once we have this information we will forward it to the pertaining teams to get a quote.
How do we go about adding an additional consent form or an NDA to the study?
You would add it as a screener question (copy/paste). If this is about our 10K character limit, you can reach out to our support team so we can add it for you. Once this is added, you can set the responding logic to qualify or disqualify whether they agree or disagree. If you would like us to collect signed / PDF versions of the consent form, or if participants need to e-sign the document, our support team will need to recruit and have participants sign before they see the screener. This has an additional fee due to the manpower needed to manually collect the e-signatures of participants. If you'd like to proceed this way, let us know and we can get it set up for you.
How do I test with my own users? Do they have to create an account?
There's no need for a BYOU participant to sign up into Userlytics or create an account. You can simply share a link that allows them to access the test directly. You can learn more about BYOU testing in the following help post.
Do you have a way to reach out to past testers?
Yes, our operations team can reach out to the participant in question to try to find out more about the issue they faced and find a solution. You should share the participant's name and the information you require.
Can we reinvite a good participant?
Yes! IAnd in fact it is a very common practice by our clients to re-invite past participants to a follow-up study. Though we can only guarantee 99% probability, this happens because we do not know their availability. Our support team can help you with this request. You should share their name and the study you'd like for us to re-invite them to.
For BYOU Testing and troubleshooting, are there certain browsers or operating systems that dont test well with the Userlytics platform that we should avoid?
Chrome and Edge are our recommended browsers, we first recommend participants clear browser cookies and cache and re-install our web extension with the below links:
Chrome Extension
Edge Extension
After that, the participants can test the configuration with this debug test: https://remove.userlytics.com/launch/ro3ltpik
What happens if a participant takes more than 30 minutes – is there some sort of prompt that the end is nearing, or do they just get cut off?
The participant gets an alert when there are around 5 minutes remaining for the study. At the 30-minute mark, they will get cut. You can always set up the study for 45 or 60 minutes, although this would double the session credits. I'd say it's best to: set the study for 30-minutes - have around 20 questions (if 1-2 minutes per question) run a dry run with 1 participant to see how it goes, then clone it, make any edits, and launch it to five or more participants.
Is there any way I can practice my test on a colleague before I launch it out to panelists?
Yes! We have clients doing this all the time. If you'd like, you can clone the test and swap it over to "Bring your own participants" instead of the Userlytics Panel, and then launch the test and take it yourself (or send it to a colleague/friend to see how they take it). The user (you or your colleague) will take the full test as a normal participant would, and you'll even get a recorded result on your platform to look back on if you need to.
SECURITY
Where is European data stored and processed?
Userlytics stores and processes its European data in Ireland, with a backup system in Frankfurt. We are the only major remote UX testing platform that keeps all EU data 100% within the EU.
Are you GDPR Compliant? CCPA? ICO?
Yes, we are fully GDPR compliant, as well as registered in the ICO of the UK and fully CCPA compliant.
Are you certified under the EU-US Privacy Shield?
Yes.
Is remote user testing secure?
We have implemented commercially reasonable technical and organizational measures designed to secure uploaded files from accidental loss and from unauthorized access, use, alteration or disclosure. Our participants have signed up to non disclosure clauses, and we can project manage a customized NDA from usability test respondents as part of the test workflow. Another alternative or complement is to set up a test on a password protected client site, with a temporary password provided to user test participants.
How can I conduct remote usability testing of purchases?
For privacy reasons, user experience test panel participants are normally prohibited from completing purchase processes with their own personal credit card information. As an alternative, we can facilitate the use of dummy credit card information for such purposes, including the use of an application that delivers unique codes to each participant.
However, if a real purchase is necessary, we can project manage participant reimbursement.
FEATURES
What Operating systems and Prototyping platforms is Userlytics compatible with?
We are compatible will all major operating systems, including Mac, Windows, iOS, and Android.
Userlytics is also compatible with most major prototyping platforms, including Figma, Invision, Proto.io, Axure, Marvel App, Justinmind, and Adobe Xd. In addition to these platforms, we are compatible with any prototyping platforms that can be tested through a link.
What Unmoderated Usability Testing Features are available?
Capture of computer or mobile device screen interactions; Capture of participant audio commentary; Capture of participant and context; Integrated “Picture-in-Picture” (PiP) video of participant, verbal “think aloud” commentary, and screen device interactions; Skipping logic; Configurable testing methodology (webcam, screen, audio recording + metrics, or screen, audio recording + metrics or audio recording + metrics). Personal Identifiable Information (PII) protection; User testing of production sites, mobile apps and desktop applications; UX testing of prototypes, wireframes, PNG sketches, unreleased mobile Apps; Usability testing of Android or iOS, free or paid, production or prototype mobile apps; In-linetime stamped hyperlinked annotations; Downloadable videos (MP4), as well as downloadable CSV files (time on task, success/failure rates, System Usability Scale, Net promoter Score, demographic specs and other quantitative data); Downloadable or shareable video clips with annotation subtitling; Downloadable or shareable highlight reels with annotation subtitling; AI generated testing session audio transcriptions; Usability Test Design, Scenario – Script preparation; User Experience Test review and analysis and executive Summary report with recommendations; User test respondent recruitment and incentives with a global panel of over 2 million; 3rd party user testing respondent panel integration; White labelling – private labelling of dashboard results; Project management.
Can I annotate my remote user testing videos?
You can annotate each user experience test video with key events and comments, and then share an aggregate list of annotations with each one hyperlinked to the relevant video and timeline moment.
What does picture-in-picture (PiP) recording include?
It refers to a recording of the webcam view of the usability test participant, together with the recording of the screen the participant is interacting with, as well as the audio “think aloud” commentary of the user testing participant and the UX moderator, if any.
What does PII Protection mean?
This unique feature enables the blocking of screen recording during sensitive moments in the test journey, to protect personal identifiable information, while continuing to record the webcam view of the participant and their “think aloud” commentary.
Can I create a Highlight Reel?
Yes. You can create and download or share individual video clips, even captioning them with annotation subtitles, as well as automatically create and download or share a highlight reel of clips from different participants and/or studies grouped by a common theme through the use of tags.
Can I have my picture-in-picture video recording sessions transcribed?
Yes. You can have them transcribed through the use of our automated video transcription capability, provided by Amazon Alexa. Our advanced transcriptions allow you to search for key terms spoken aloud by participants, and go directly to the relevant video and timeline moments.
Is it possible to conduct an unmoderated usability test with children?
Yes it is possible, however, a test with children requires certain protocols to be met in order to ensure all guidelines and laws are being followed.
Can I conduct quantitative tests across multiple devices?
Yes, you can conduct quantitative tests with participants using various devices. In the Userlytics study builder, select "Multi-Device" when designating your participant’s device. You can then choose to recruit participants using either the BYOU (Bring Your Own Participants) option or the Userlytics Worldwide Panel of over 2 million participants. Pick the one that suits your needs. Benefits of multi-device quantitative testing: 1. Faster Results: Since participants can use desktops, laptops, mobile phones, or tablets, study results can be gathered more quickly, leading to better UX insights. 2. Simpler Onboarding: Participants don’t need to download our mobile app to take a quantitative usability study on a mobile device, making onboarding faster and easier. 3. Smooth Integration: Multi-device testing allows easy addition of our Live Intercept to your website. You can use BYOU credits to pull website visitors straight into a Userlytics-hosted survey, regardless of their device. The onboarding process is just one click from intercept to survey! If you would like to learn more about multi-device Quant studies, read our help guide here. Note – if you do not see this option and would like to conduct quantitative tests with your participants, please contact your Account Manager so they can adjust your account settings.
How does the AI UX Analysis tool work?
Userlytics' AI UX Analysis uses machine learning algorithms to auto-analyze session transcriptions and summarize key themes and actionable insights. This allows you to quickly identify the root causes of UX problems and scale the number of studies and sessions you can analyze.
What is Userlytics' Sentiment Analysis feature? What are its benefits?
Userlytics' Sentiment Analysis feature instantly determines whether a participant’s tone is positive, negative, or neutral. This allows you to quickly gain insights into your users' feelings without manually reviewing each session. Using Sentiment Analysis is beneficial because: 1. It saves time and energy: It eliminates the need to manually review each session. 2. You get quick insights: Instantly draw key insights about your users' feelings. 3. It is easy to use: The tool has a simple UI and takes moments to work, making it easy to use. 4. You can scale your research: It allows you to scale your user research with more studies to gather actionable insights for improving UX.
PANEL
Can I use participants based outside of the United States?
Yes, you may invite participants from our Global Participant Panel, or use a 3rd party panel provider, or invite participants yourself by sending them invitation links or posting them on your website or social media sites. Contact us for a customized recruitment quotation from anywhere in the world, at any incidence level.
How many participants should I use for a Usability Test?
Experts generally agree that using 5-15 participants per qualitative user test iteration will uncover 80% of usability issues, and is the most efficient manner to conduct agile user experience testing. We have found that it is more efficient and cost-effective to conduct numerous small sample studies iteratively, and then use multivariate testing at the end of the process with larger samples to conduct final optimization.
What kind of user test participant profiles can I select?
You can filter our remote user testing panel for education, age, gender, employment status, household income, education, civil status and similar. In addition, you may define a behavioural screener to select usability test participants based on highly specific demographic profile criteria, both B2B and B2C, from around the world to target your specific target Person for your remote usability test.
Where do usability test panel participants come from?
The Userlytics user experience test panel comprises crowd-sourced respondents from all backgrounds. However, most of our clients ask us to custom recruit highly specific demographic participants from around the world.
Who are Userlytics’ highest rated user testing participants?
Our remote user testing participants are rated by our clients and we use those ratings as a selection basis for remote user test projects. In some cases our clients ask us to recruit first time usability test participants, in other cases we are asked to provide the same participants for a set of repeat sessions.
Can I get a result I am unhappy with replaced?
Yes, you can! If you rate a result as a “1 Star”, within 7 days of receiving the result (or simply by letting operations@userlytics.com know), we will conduct a second in-depth review. We will replace the result if: Participant was not following a “Think Aloud” protocol, Participant was not making a good faith effort to follow instructions, There were technical issues with the recording. If however, the instructions were confusing (and you had not taken advantage of our optional Professional Services and had us design the test script), or, there was a bug in the test asset you directed participants to, then participants will be paid their incentives, and the credit(s) used will not be replaced (We can however still delete the result from your account if you wish)
Can I use the same participant invitation link for multiple sessions?
Yes. Participants can reuse the invitation link multiple times. It will generate a different code each time, which can be used each time with the Userlytics Recorder.
Is there a way for us to see demographic information for the participants?
Yes, of course! Once you click on the test, you are taken to each participant's recording. You can simply click on the desired participant and on 'Participant information', where you will see all the filter details.
How do I find participant’s answers to screener questions?
Those responses will be shown alongside the name, under the Participant Information.
What demographic information would I need to explicitly ask participants for, and what would I be receiving directly from Userlytics?
By default, you will have access to our participant's first name, last name, initials, country, age, gender, and type of device they are using. Per our Customer Terms & Conditions, clients cannot request PII (address, credit card number, social security number, email address, driver's license number, etc.) unless they have previously coordinated with Userlytics and requested consent from our participants. In case you ever want to gather any other information from our participants you can ask for it in the screener and in case you want to ask for personal information you would have to add a consent question in the screener as well. Please just keep in mind this would make recruitment more difficult as most participants don't want to share their personal information.
PRICING
I reside outside of the US. In which currency do I pay?
We can manage payments in either US dollars or Euros, depending on your location, using a Credit Card. For Custom Projects or Annual Enterprise plans, wire transfers or PayPal may also be used.
How much does a Usability Test cost?
Pricing for remote user testing depends on the following factors:
Usability test participant demographic profile – Target Persona. Maximum user experience test session time.Number of user experience test participant completes desired
Whether usability test design – scripting is required.
Whether user test review, analysis and recommendations are required.
Pricing can be monthly or annual-licensing based, or Ad-Hoc, on a per usability test project basis.
Contact us for information on how we can help you achieve your unmoderated user experience testing goals.
When do I pay for the test credits?
If you have a DIY account, then you would pay for the credits when you go to launch the test, after you have fully setup the test.
If you have a Custom Project or Enterprise Plan, then you would purchase the credits upfront and they would be available in your account for when you are ready to run your upcoming test(s).