Field study with eye-tracking to leverage product development

Tytyri Mines in Lohja is a museum located approximately 100m below sea level, in an actual original mine. It exhibits the mine’s history, geology and machinery in atmospheric settings, allowing the visitors to roam the dim corridors.

As part of the Virtual Nature project, a virtual mine experience was developed to enrich visits, encourage longer stays, and provide reasons for return visits. To ensure the concept delivered on these goals, a field study was conducted in the authentic mine environment with real visitors. The aim was to understand how people experienced the application and how it could be improved before finalising the product.

The virtual experience in a nutshell

The pilot was designed as an interactive map-based application. Visitors could navigate a 3D map of the mine, rotate and zoom it, and use it both for orientation and for discovering hidden clues in the tunnels. Entering the codes found in the mine unlocked additional content. The application was web-based and intended mainly for mobile use, so that it could be easily integrated into the visit.

How the study was done

Because of the mine’s unique environment, it was essential to carry out the testing in Tytyri itself rather than in a simulated setting. Factors such as temperature, humidity and acoustics would have been almost impossible to reproduce elsewhere, and the more authentic the conditions, the more reliable the results. Testing the concept in the actual environment with real visitors ensured that the feedback came directly from the intended user group, making the insights highly relevant for product and service development while helping secure a smoother path to market. The authentic setting additionally offered the service provider valuable observations about how the application might function in practice once commercially launched.

To pilot the concept and collect data, a testing day was organised in the mine. Five participants were recruited to explore the site independently with the virtual map while wearing eye-tracking glasses, and afterwards they took part in in-depth interviews. The use of eye-tracking allowed us to study the participants’ visual focus, ultimately helping us to understand their behaviour while using the application and exploring the mines. The interviews provided richer accounts of their feelings and experiences throughout the visitor journey. These discussions focused on the difficulty of finding the clues, the usefulness of the map for navigation and for accessing extra content, and the overall usability, memorability and recommendability of the experience.

Alongside the individual sessions, data was also collected from a wider audience. An online questionnaire, based on a simplified version of the interview themes, was distributed to all visitors who used the application during their visit. Responses were gathered per visiting group, resulting in a final tester base of 148 visitors. This combination of eye-tracking, interviews and survey data provided both detailed behavioural insights and a broader understanding of how the pilot was received by the wider visitor population.

Findings

The findings of the field study could be roughly divided into three categories.

Valuable customer feedback

A majority of the testers said that they would recommend the experience. The experience was boosting the value of the visit. It was reported to allow the adults to explore the mine museum in more depth, while making it more attractive for children. Most of the testers considered the level of difficulty in finding the clues in the mine appropriate. However, the locations may have been easier to find for adults but difficult to see in the dark. The children may see better, but are not yet that skilled in spotting hidden signs in a complex environment.

Roughly two-thirds of testers reported that the information provided in the application was interesting. Though the information was detected to be partly overlapping with the information already available in the physical information signs in the mine. Also, the nature of the information was very fact-oriented and less storytelling, which, on average, might attract more adults than children. It was detected that, as the product was mainly operated by adults, and funny characters implemented in the application did not reach the intended audience, the children. The children concentrated more on searching for the clues in the physical environment, which added excitement for discovering the mines.

Detected technical issues or inconveniences

The application seemed to work differently on different operators’ connections. When the network connection was unstable, the application was unreliable, incomplete or completely disconnected. One of the goals of the product was to provide accurate location information for the users, functioning as a map. This feature was not usable with an unpredictable connection.

Another issue mentioned frequently by the testers was the sensitivity of the touch screen command for zooming. Many reported problems with modifying the view to fit their desired proximity, as even a light touch caused the map to zoom out completely, being too inaccurate to view details of the map, or to zoom all in on specific areas, which made it difficult to understand the location in the map due to unfamiliarity with the place. Also, the screen on mobile use felt too small for many. Some of the testers mentioned the inconvenience of using other applications simultaneously with the map. If they switched to the camera to shoot some pictures, the map application rebooted and forced a restart from the beginning.

Insights from the eye-tracking

The eye-tracking insights revealed how the participants observed the environment and found the hidden keywords from the mines. We analysed all the locations of the signs and their vicinities. While approximately half of the signs were placed optimally, delivering some challenge for finding them, a few of them did not pop out of their environments enough. This was especially problematic for the first sign, which was placed near the mine entrance. Being more visible, it could both serve as an easy accomplishment, introducing the users to the outlook of signs and enhancing trust and can-do towards the application, while potentially increasing curiosity towards use by visitors who missed the information about the digital mine experience by the ticket booth.

Implications for development

Based on the findings, two key development ideas were presented together with several smaller improvement ideas that could be put into use immediately. The development suggestions were 1) a digital offline map, and 2) storytelling of the contents.

To beat the challenges of unstable connection and provide a map the visitors can use also without, a suggestion of an offline map was made. It could be added to the application, showing the approximate location of each code word. This will enhance reliability and the chances of successfully finding the codewords, while increasing the potential to use it as a traditional map.

The current map application does not show which code words are still missing and where to go next to find them. The tour becomes a rummage, and only some of the words are found. The suggestion was to add the possibility to use the code list linked to the offline map as a checklist.

The content development by using storytelling was suggested as another improvement for the product. Storytelling adds layers to the content. For example, users of the app might be attracted by a fictional, progressive story told by a troll, where the user is taken on a tour of the troll’s journey. The narrative content could then partly replace the informational content that can be read on site at the signposts.

Updatable story content according to changing themes (e.g. Christmas, Easter) is also an idea worth exploring. The result is a renewed, multi-channel and coherent package that has the potential to attract families to keep returning to Tytyri. This could be an easy way to create topical and innovative layers for the experience.

Engaging content makes visitors stay longer, and with storytelling, the content is made more attractive to families and children. As lengthening the stay of visitors was one of the initial goals for the product development, these improvements could encourage wider and deeper exploration of the mines, which ultimately would lead to longer visiting times.

Consumer experience research is a valuable tool in leveraging services

The findings of this case study once again highlight the benefits of user-centric approaches when designing services and products. This is a good example of how insightful it can be to involve the real users in the process, even if they have been studied and on the focus of developers during the process. Many issues or inconveniences cannot be imagined beforehand, and detecting them requires testing sessions where the product is exposed to real audiences and non-professional end-users. This makes the testing phase an utterly valuable part of the product development process.

In this case, it would have been even more valuable to be able to pilot the product already in earlier phases of the process, to gain more understanding before the product was close to being finalised. In future projects, the process should be designed to be more participatory, including user testing phases with real users from desired user groups at earlier points of the process. This is sometimes challenging and may feel unnecessary, but sometimes it is too hard to really understand the users without hearing them speak out loud.

Kuva: Shutterstock