It is finally here. The VR application you’ve put so much effort in. Now one last step and you can present it to the world. But should you? Like any other product, VR applications can turn out great, amazing, or so-so. Before giving it a go and sharing with others, you want to make sure the app is as good as you expect it to be. You want the users to have the best experience when you take them to a ride to the virtual world.
If you don’t know where to start looking, this article is for you: what to pay attention to and how to evaluate your VR application from different points of view.
How the app influences the user
Of course, everything in your application should be about the user and their experience. There is a couple of aspects, anyway, that are exceptionally important for any VR application, no matter what is the content and what is the headset you use.
People value their comfort a lot. Even if your experience is the most engaging in the world, but it makes people feel sick or tired halfway through the session, it won’t work.
Comfortable experience includes many aspects. To begin with, take into account the duration of the experience and how pleasant it is for a user to spend this much time in VR. Virtual reality headsets are getting better with every new launch, but some are still not the best for continuous use.
Possible mistake: Your experience lasts for half an hour and you choose to make it on Google Cardboard. This headset has no head straps and people will just get sore hands and probably won’t wait until it finishes.
Possible solution: Use the same application with Google Daydream or Samsung Gear VR. These headsets are also affordable mobile headsets but equipped with headbands for more comfortable extended use.
Also, motion sickness. Up to 80% of people experience motion sickness (or cybersickness) at some point. This is something to keep in mind because virtual reality experiences tend to provoke nausea.
Motion/VR sickness or cybersickness is a type of nausea that occurs when the picture you see doesn’t match with what your brain feels. The inner ear, which registers body position and movement, reacts to head movement faster than eyes catch visuals changing. That mismatch makes people feel sick. Of course, today the delay is milliseconds only, but people have different levels of sensitivity, and for some those milliseconds are enough to feel nausea.
It is impossible to avoid motion sickness altogether as some people with high sensitivity will still experience it. However, you can minimize the effect by reducing possible triggers such as low frame rate, the delay between motion and visuals, flickering of the screen, etc. Again, the time users need to spend in VR also matters. If people start feeling nausea ten minutes into the experience, which is designed for five minutes of active use — you’re in a good place.
Possible mistake: A VR application with long delays that make people sick seconds after start. If 8 out of 10 people experience nausea as they launch the app, it’s not likely to be about individual sensitivity.
Possible solution: Adjusting technical specifications of the application for a comfortable VR experience.
Have you heard stories of keen VR players who walked into a wall or smashed a lamp with a controller? Probably you’ve even seen a couple of videos. This happens when people neglect the safety aspect of setting up their VR stations. To ensure the users won’t get hurt when trying out your application, consider the following.
The pose the user takes should correspond to the experience. If you suggest people to take a ride on a rollercoaster, it’s better for them to take a seat. Think of VR arcades and people swaying side to side hardly able to stand upright. Not the best way to enjoy VR.
This is the case with the walls and lamps. If the experience implies broad gestures, active use of controllers, or movement, there should be enough room to do it freely. It can be either a seated position with free space around for the controller manipulation or a dedicated VR room big enough and empty enough for walking around in the experience.
Talking about walking in VR — usually, a person does not see what happens around when in a headset (though recent HMDs do have cameras that allow looking around without taking off a headset). So when designing an application where the user needs to walk, or jump, or climb, think of a corresponding real environment. If you duplicate elements of the virtual environment in reality, it will make the experience first of all safer, as the user will have a real-life anchor, and secondly, much more immersive.
For example, Merrell “Trailscape” VR experience includes walking across a rope bridge in VR, while a similar bridge is built in reality to support the visual.
Possible mistake: A hiking VR experience that includes actual walking to move in the app with a setup in a crowded room. The user is likely to trip over when making a step or to bump into another person.
Possible solution: A dedicated space for a VR setup free of other objects. Or making physical movement unnecessary, substituting it with in-app teleportation.
How the user interacts with the application
This is probably the most critical aspect to evaluate in terms of content quality. You do want your application to be safe and comfortable to use, but what’s the point of it if nobody enjoys the experience itself? Besides, quality content also means that the application will meet the business goals you set. So, here is about the user-app interaction.
In a nutshell — the less explanation you need to incorporate in your application, the better. The criteria can be applied to both controls and content/script of the experience.
Modern UX/UI experience is all about the absence of explanations. Many actions are already taken for granted, and people feel confused if something doesn’t meet their expectations. Even though VR is relatively young, there are still movements and actions people suppose should work. Your goal is to make human-computer interaction as simple and obvious as possible.
If your experience doesn’t include the use of a physical controller but implies gaze control, make it noticeable. Draw a dot or a sign in the middle of the screen to provide an anchor. Include some animation to let the user know they are doing everything correctly.
A physical controller is a little bit complicated, especially if you use one of the tethered headsets with highly-functional controllers. Valve Index controllers, for instance, have a few buttons, a trigger, a thumbstick, and a bunch of sensors. In case your experience does require all of the provided inputs, make the interaction as straightforward as possible. Pull/release a trigger to grab/let off an object instead of using a button for each step. Press a button to choose a menu option instead of staring at it and so on. And it is always a good idea to start an experience with a brief tutorial to explain the main ways to operate within the app.
Possible mistake: The application requires the player to use gaze control to open the door. The door itself has no markers and no hints that it can be interacted with. The user is lost and needs help to find the next step.
Possible solution: When the door comes in sight of the user, it starts glowing to draw attention.
You do not create a VR application just for the sake of creating. Your application should be designed to meet the business goals and objectives you have set. There are many different types of VR applications for different business goals: to entertain, to teach, to show, to train, to shop. The script for each is built in its own way to guide the user through a sequence of required actions. And the user should understand them without a lengthy explanation.
It doesn’t mean there should be no explanation, but there should be just enough to guide the user without ruining the immersion. And, what is essential, these instructions should be logically incorporated into the experience. Guided meditation, for instance, is all about verbal instructions. And an entertaining application should not have a lot besides initial control explanations.
For example, if you create an application for doctor’s training, you can add a nurse character who will help the user once they are uncertain what to do.
When evaluating your script, take into account your target audience. What seems obvious and straightforward for a hardcore gamer, can become an invincible obstacle for a VR novice. The same is about the industry you create an experience for: a training program for a manufactory worker will likely confuse someone from retail.
There is a great way to check if your application is intuitive enough. Take a person from your target audience, who has never had experience with VR, and let them try the application out. Watch if they are able to figure out in-app controls, if they understand what to do and whether they can walk through without additional help. If you feel like you want to give a hint or to guide them at any point, adjust the experience, so these comments become unnecessary or add them to the script as subtle hints.
Possible mistake: A VR firefighting simulation requires the user to do a particular sequence of actions to put out the fire successfully. The correct sequence, though, is not identified other than by trial-and-error search. In the third round, the user gives up and refuses to continue.
Possible solution: The fire extinguisher has readable instructions printed on it. There is also an action plan as part of the emergency drawing on the wall.
2. Level of immersiveness
Immersiveness is the real power of virtual reality. The feature that awes people and makes them believe in what they see. It is impressive, and it is fragile. It is easy to go too far or, on the contrary, not far enough, and break the charms of the virtual world. The success of your application heavily depends on how immersive, engaging, and interesting it is for the user.
A quality experience doesn’t have to be full of action or exploit high-quality graphics. But it has to be gripping. It has to be immersive and interesting to the extent your business specifics allow it to be.
For example, a VR therapy application doesn’t need that much of an action, but it can but evoke interest with wonderful scenery and some in-app interaction.
Immersiveness consists of many elements. Every little detail you see or hear or feel when in VR either enhances or reduces the immersiveness of the virtual experience. Some are already mentioned in this article. Like using props that add to the experience and link it to the real world — a sure way to make people forget they look at a computer-generated image.
Explanations also contribute to this characteristic. If you have to tell the player what to do or insert a voiceover that plainly explains every action, forget about the immersiveness. The user will be reminded again and again that there is something outside of this virtual experience.
A great way to add extra vividness to a VR program is spatial sound. Most, if not all, VR experiences should include it to some extent.
Spatial sound is immersive audio that floats around the listener creating an illusion of things happening around. This type of sound can communicate objects moving left to right, approaching, or getting farther.
Again, it depends on the type of an app you create. A walk-in-the-forest experience will become much better if you add background spatial audio with sounds of nature. However, in a VR hotel tour, flat background music will be enough. Instead, you can add a few spatial audio elements to enrich the interactive details of the tour.
A good experience is a balanced one. Where nothing sticks out too much and the customer has no urge to focus on separate elements like the graphics, the sound, the instructions, the characters, the gameplay. A VR application reaches maximum immersiveness when it is coherent, entertaining, and relevant to the user you introduce it to.
Possible mistake: An educational VR experience that takes children on a tour around the Solar system. The colors of the planets are too bright and uncomfortable to look at. This makes the user stop the experience to give rest to the eyes.
Possible solution: Mute the colors, so nothing distracts from the space tour.
This point can sound as “Make your experience replayable” and “Don’t make your experience replayable.” It all depends on the goals you set for the application.
Replayability is a quality feature of a VR experience that denotes whether the user is willing or able to play the game/walk through the experience more than once.
Some VR applications are designed to be played once. If the primary goal of the experience is to impress people and make them speak about it, or to showcase the possibilities of a particular hard- and software, it is perfectly fine to create a single-use app.
For example, The North Face put a stake on an element of surprise in their marketing VR experience. This application does what it was designed for: it excites and makes visiting the store memorable. Replayability is not as important here as the initial impression it makes on a customer.
In other cases, a good VR experience has to be replayable. When it comes to training or instructional programs, the user is supposed to go through it a few times to acquire and reinforce a target skill. A non-variable VR experience turns learning into memorizing steps and repeating them mechanically. Which defeats the initial purpose of the application. To avoid it, you can add some variations and options in the gameplay, like a new starting point every other run.
Possible mistake: An instructional VR application that teaches first aid. There is one linear script the user follows. In the third run, they memorize the steps and just click through the experience.
Possible solution: Slight variation in the sequence of actions, different starting conditions the patient has to practice a range of skills.
How the app performs
Now we are about to get technical. The app performance is an important characteristic to look at as it can change the overall impression the app has on the user. Here are the main specs you should evaluate.
This is what probably came to your mind as soon as you read about tech specs. No doubt it is important. But not in the way you think.
The better the graphics, the better the app. Surprisingly, it doesn’t correlate this way. It is more about the visual being complementary to the content. Think about old school games with mediocre, by modern standards, visual component. However, many of them stay relevant and popular even today, all thanks to well-thought game design.
The same applies to VR. Graphics is important in terms of making the experience more immersive and more believable. The virtual environment shouldn’t distract or draw excessive attention. It should correspond to the goal this app is supposed to meet. If it is an outdoor experience, the background can be low-detailed, while the main foreground objects have more detail and more quality to them. If you want to showcase something, like a VR real estate tour, then it is expected to have a lot of high-quality 3D objects throughout the tour.
For example, you wouldn’t call Superhot VR a game with very realistic or detailed graphics. It, anyway, is a popular game with almost three thousand positive reviews.
Possible mistake:: A guided meditation VR application with high-quality graphics all the way. The user, instead of focusing on the meditation, will keep examining every object, thus getting distracted from the main goal.
Possible solution: Make the background a bit vague to provide enough context for the experience without drawing unwanted attention.
The market today is flooded with numerous HMDs options. If you make a niche application, check on if your prospects already own any VR headset and stick to their choice. Custom solutions are often made for one or two headsets in mind that meet the specific needs of the industry. The choice of a headset is already a different story, especially for a medical application.
If your target audience is wide public, you might want to make your application compatible with as many devices as possible. It may mean that for different platforms, you will have to launch slightly different versions of the application. What works for Oculus Quest and 6 degrees of freedom controllers won’t work for Oculus Go and its 3 DoF controller. And it’s an example within one manufacturer. You may imagine how different controls are for HTC Vive and PSVR.
Possible mistake: A VR fitting room application where a user chooses an item color with a tap on a touchpad. When ported to another device without a touchpad, the option becomes unavailable.
Possible solution: Look for cross-platform input options: instead of a touchpad, make the action possible with a button click.
3. Frames per second and responsiveness
These two features, once implemented well, can help a lot to reduce motion sickness. FPS rate allows displays to refresh more often, providing the user with actual visual they should see. And the smoother the picture is, the more comfortable it is to watch.
Responsiveness shows how fast the software reacts to player input. That includes head, hand, and body movement, which can be translated into the virtual experience. Ideal numbers for comfortable usage are less than 20 milliseconds of delay. 50 ms delay is noticed visually and can result in nausea, especially for more sensitive users. Delay above this number just makes the whole experience very uncomfortable even for the toughest of users.
Possible mistake: An entertaining VR application, with an image twitching and freezing randomly. The user will leave with not the most favorable impression about the app and the brand.
Possible solution: Optimize the app to avoid visual distractions and have it running smoothly. Even if it means you have to reduce the image quality.
4. Loading time
Nobody likes waiting. Especially if the app itself takes 20 seconds to complete and one has to wait 15 seconds until it launches. Of course, it’s all respective. If your experience lasts for ten minutes, 15-second loading time is expected.
In case the user needs to wait too long, they won’t wait. Optimize the application, so it takes as little as possible to start. It’s also great to add a progress bar if the loading takes more than a few seconds to give the user some information on how long they need to wait.
Possible mistake: A city guide VR experience with several locations. Each location loads separately up to 40 seconds and includes a couple of video clips and interactive elements to it. The user won’t go past the second location unless they are really into that city.
Possible solution: Optimize the app by reducing video size and have the next video loading while the user watches a previous one. If possible, load all the videos for one location as a whole package. As an alternative, port the application to a more powerful headset to make it work faster.
5. Battery life
This point refers to apps for mobile and standalone HMDs. You want to make sure that the battery of the device is capable of lasting as long as your experience does. It would be disappointing for a user if their full-charged headset doesn’t make it to the end of your VR program. Also, pay attention to the way a device behaves when running an app. If a phone battery gets as hot as a pan, then something is probably wrong and needs fixing.
Possible mistake: A VR test drive and car selection application for Google Cardboard drains a phone battery in 15 minutes. The user can’t finish the test drive and can’t use their phone, which is now dead.
Possible solution: App optimization again. Or using another more suitable device.
This is your complete guide on how to evaluate a VR application for all the points of view. To recall everything we mentioned, take a look at this checklist.
What a quality VR application should have
How the app influences the user
- most users don’t have motion sickness at a time period enough to complete the experience;
- the experience doesn’t feel tiring;
- the real-world environment corresponds to the app content and is safe for movement if such involved.
How the user interacts with the application
- the user easily understands how to manipulate objects and use controls in the app;
- the walkthrough is clear and doesn’t require additional external comments;
- the user understand the primary goal of the application;
- the experience is coherent and immersive in all its elements;
- the gameplay (if any) is designed according to the business goals of the application.
How the app performs
- the loading time corresponds to in-app time;
- the FPS rate and level of responsiveness are good enough to provide smooth experience;
- the application doesn’t drain the device battery too fast;
- if the application is cross-platform, it is ported with necessary adjustments for every device.
How do you like the article?