Making user research work in a large corporation

KPN used to be heavily dependent on external parties to carry out research. The outcomes were presented and handed over in reports that often ended up on a pile, rarely to be looked at again.

In response to the Experience Economy and the increasing importance of UX within organizations, the need to validate ‘desired’ experiences awakened. For KPN, this meant that a more efficient method was welcome. Although such a change in working methods and mindset is not always easy to achieve, especially in a large corporate organization such as KPN, the first big steps were taken starting 2017 by:

1. Creating the possibility for everyone to talk to the customer and thus becoming more user-centered;
2. Facilitating research internally, aiming for a better/broader empathic capacity for the user among employees and across departments.

After doing this for 2 years, we reflect on what we have experienced so far from our role as UX Researchers within KPN.

2017 – Experience Lab

Two years ago, the Experience Lab was opened within KPN. This lab can be used by everyone who wants to talk to customers. Therefore, training on qualitative research is provided to everyone within the company who’s interested in talking to the customer. One can learn how to set up and conduct research, how to create an interview protocol, formulate questions, and the like.

A lab manager is in charge of the lab agenda, oversees the topics, and leads a biweekly meeting in which upcoming reservations are discussed with researchers from different departments. This meeting enables us to connect with colleagues and teams with overlapping interests. The lab manager also checks if the colleague who wants to do research has done research before, and if not, if they would like to be linked to a researcher.

Within the past 2 years, we’ve seen that the possibility for everyone within the organization to talk to customers led to great outcomes:

1. Customer insights became more easily accessible and the number of insights gathered in the last two years are much higher compared to previous years.
2. By talking to customers themselves, colleagues are more engaged in the research process, resulting in a shift from thinking that we know what the user wants, to exploring and validating our ideas with actual customers.
3. It also took some pressure off the researchers within the company to create more flexibility and room to focus on complex and/or strategic research projects.
4. When research colleagues conducted a study, we were more aware of what research was ongoing, ensuring we didn’t research the same topic twice.

From the business’s perspective, these are all good points. However, there is room for improvement.

1. We often see that the research questions and the methods to be used are not yet clear when a lab reservation is made, making it hard to determine what organizational benefit the study might bring and if the method to acquire answers is valid/reliable.
2. From a more scientific point of view, we can also see some challenges regarding reliability and validity. Even though providing training is a good way to ensure a basic quality, it can still be difficult for a novice researcher to conduct research on their own. Apart from the fact that it can be tricky to identify the correct method for a certain research question, some methods are prone to leading questions and affecting outcomes. The biggest issue primarily lies in the fact that we are depending on one’s self-dependence.
3. Another concern is that observations are often not analyzed. For example, invalid conclusions are drawn when researchers quantify qualitative data. They might state that 50% of the participants said something, while n=6.
4. Sharing insights within the entire organization instead of a single team/department still remains a hassle due to silos across different departments (we are currently working on resolving this issue). Additionally, sharing insights should be done while taking the GDPR into account. However, there are no clear protocols on how to do this yet.
5. The lab is located at our KPN office in Amsterdam, giving rise to bias, as participants know the research is linked to KPN.

2018 – Popup lab

Since a few months, we work together with a partner to accommodate a faster way to talk to our customers within a more natural context (e.g. events, markets, and KPN stores). It should be noted that this kind of research is most suitable for small topics, as the conversations last 15 minutes at most. Even though, doing this for a few hours, this method provides a way to explore and validate ideas and designs with 10 to 15 customers. Working in an agile environment, where sprint cycles of 2 weeks make it difficult to do lengthy user/usability testing, this rapid-pace testing provides a solution for gathering insights in short amounts of time without interrupting the current way of working.

Additional to the challenges mentioned above, speaking to more than 10 users in an unfamiliar setting can make it more difficult and even overwhelming for a novice researcher to translate the correct insights. This makes the study prone to false data interpretation. The higher number of participants compared to lab research can also lead to the assumption that the data can be quantified, which is often not the case.

2019 – Online platform

At the moment, there is another initiative deployed: an all-in-one platform with multiple online research methods. Using this platform, we can send out surveys, ask UX-related questions, and start dialogues with customers within a panel. This way, researchers can explore and validate ideas and designs with an even bigger number of customers (quantitative). For now, this platform is only accessible by researchers within the company. We’ll evaluate how this goes in the coming months.

2020 and beyond

Our next big steps are providing a follow-up training on how to analyze data, and a platform for sharing insights within the organization. Yet some challenges remain for which we don’t have answers right now. Our opinion is that even though we encourage every form of UX evangelism throughout the company, the fact remains that certain methods require more expertise. Therefore, we think they should not be done by the novice researcher alone. As there is a large number of UX research vacancies at the moment, this is a clear indication that a certain amount of expertise is preferred and should be maintained and strengthened. For the novice researcher, we think that peer review and mentoring could play an essential part in learning by doing in order to maintain a certain level of expertise.

What do you think? How is this organized within your organization? And what solutions would you propose in order to take away some of the challenges mentioned above? Let us know!