Measuring employee user enablement at Microsoft with the Microsoft 365 admin center

If you’re rolling out a product or initiative, measuring how many employees are using it is likely something you want to watch closely.

But figuring out what caused your daily active usage (DAU) to change on a particular day—that hasn’t always been easy to figure out.

Until now.

When it comes to products like Microsoft 365 and Microsoft Viva, you can now measure the impact of your user enablement actions using the Microsoft 365 Experience insights dashboard in the Microsoft 365 admin center—that’s exactly what we do here in Microsoft Digital, the company’s IT organization responsible for deploying and driving adoption of products like Microsoft Viva and Microsoft 365 here at Microsoft.

Recently, we used the Microsoft 365 admin center to track the impact of an event promoting employee resource groups at Microsoft using Microsoft Viva Engage. The Experience insights dashboard showed us the event as an overlay on the DAU chart for Viva Engage, correlating the event with an increase of 33,000 users.

“We saw a huge spike in usage of Viva Engage from the whole company having a day where we focused on using it,” says Amy Ceurvorst, a director of business programs for Microsoft Digital, the company’s IT organization.

The Actions tab in the Experience insights dashboard can show more than just usage: it also tracks how net promoter score (NPS), product feedback, and help article views change in response to actions you’ve taken.

Experience insights dashboard Action tab

Measuring internal communications

Ceurvorst first started working on graphing user enablement events against product usage for MyHub, an internal employee experience application that has now become part of Microsoft Viva Connections.

“For MyHub, I created the first concept of this action overlay—on the DAU graph, I overlaid the date when we did a training or a leader sent a communication,” Ceurvorst says. “That led to us liking the concept so much that we asked the product group to build it into our internal reporting for Viva Connections.”

Employees use Viva Connections to check payroll information; for example, to view their pay stubs and dates of company holidays. Before the action overlay, understanding adoption data for Viva Connections was difficult.

Usage of Viva Connections started out slow but picked up when a mobile version was added. That started “a lot of organic user growth,” but it was hard to tell what was driving that uptick.

“It used to be pretty hard to understand why all of a sudden more people would start using Viva Connections,” says Paul Chantakraiwat, a senior product manager in Microsoft Digital. “I would have to go back to what information was sent through email, or what was in the news, or check if there was a drop in usage because of a holiday. And sometimes I couldn’t find anything.”

That changed when the team started graphically charting user enablement events as they happened for MyHub, and later Viva Connections. This allowed the team to analyze what was happening in real time, which improved the experience for everyone involved.

“The entire Viva Connections adoption team of product managers, program managers, engineers, researchers, and communicators could understand where the traffic was coming from,” Chantakraiwat says. “The communicators could see how effective their campaigns were, and so on.”

Microsoft 365 admin center impact

From Viva Connections, our team brought this adoption event mapping capability to the Microsoft 365 admin center, a function that became the Actions tab in the Experience insights dashboard.

“It allows customers to drive adoption—to understand how the products are faring across the customer base,” says Jonathan Allou, a software engineer in the Amplify product group.

The product group started working with Microsoft Digital, the company’s IT organization, on actions over a year ago.

“Amy Ceurvorst actually started as a customer and it just made sense to onboard her to drive the direction of the product itself,” Allou says. “It was a great partnership, because she’s doing this work every day. It taught me a lesson: oftentimes what you think would be good for the customer is not necessarily what they think would be good for themselves. So having Amy as a user tell us, ‘People in this position would not consider this useful,’ it’s a very good thing to have.”

A new role was created in the Microsoft 365 admin center for success specialists—User Experience Success Manager—which grants users driving adoption of Microsoft 365 applications with access to the Experience insights dashboard. Traditionally the Microsoft 365 admin center was seen as a place for administrators only.

“I got asked, is this the right place to do something like this?” Ceurvorst says. “The Admin Center is the source of truth for your tenant. It’s the one place you go to look at success for your product.”

The Microsoft 365 admin center gives visibility across roles in the product, so success specialists can see the impact of what everyone is doing.

Deeper insights

Although many users are happy just to have one place to go to see product usage, the Actions tab has value beyond that.

“This is great to see that we are going to start enabling customer IT organizations to not just build action plans for adoption but be able to assess the implications on usage,” says Andrew Osten, the general manager of business operations and programs for the Employee Experience Success organization in Microsoft Digital.

And yes, it’s about measuring usage, but that’s just the start.

“What we’re currently measuring is DAU,” says Johan Bosch, a senior director of employee experience success in Microsoft Digital. “But that’s not the only thing that’s in there. Because if you drive action by giving more training or more communication, are there some actions that have a stronger impact than other actions? We can start deriving business value from that. If we drive up DAU, we can start saying to ourselves, ‘What is the business value of getting 10% or 20% more people to use this specific tool or component?’”

In addition to business value, the Actions tab can also lead to insights about user behavior, particularly for new technologies where there isn’t a lot of existing data.

“As we start unpacking some of these actions, creating a higher level of usage and a higher level of value, there’s a reason we want to understand them,” Bosch says. “Using the Actions tab can give deeper insights from users as to why they use the product, in terms of when they use, why they use, and how much they use certain features.”

What’s next

Actions in the Experience insights dashboard was made available for public preview in September. It’s a new concept for the Microsoft 365 admin center to be seen as a workspace. People are used to getting data and reports there, but this is the first feature that asks for data input and the Experience insights team wants to socialize this concept. 

What’s next? The Experience insights dashboard could offer suggested actions to success specialists. Microsoft already provides information about adoption, but to offer that in a success specialist’s flow of work would help them share the best practices they find.

The actions team is also exploring whether people in other roles would find this tool useful.

“I thought of it initially as something we’re doing for communications or training,” Ceurvorst says. “But what if an IT manager put a new patch on a product or enabled a feature in a product? If you tracked that, I think you could potentially see impact on the product usage. So, I’m trying to figure out how to reach additional audiences.”