A typical and frequent comment I get from customers, colleagues and L&D practitioners in general is something like this: “I know we have data, a lot of data in fact, but I just do not know where to start with analytics?

And I fully understand the challenge. If you consider Learning Analytics to be an activity that can help you get data driven insights to take action or make decisions…there are a lot of possible actions and decision to make. So where to start with Learning Analytics when you have that ambition and when you have the data?

This post provides you with pragmatic step by step guidance on how to start with learning analytics. I’ve also included 2 cheat sheets you can start using today to use your data to calculate meaningful metrics and analyze trends: on Audience Reach, Completions Rates and Time Spent on Learning.

The Learning Analytics Process

Before we dive into the actual data & analysis, I want to quickly explain the standard 6 step learning analytics process that we use in SLT. The process is not new nor unique. It’s actually a fairly standard data analysis process.

Step 1: Define a solid research question: The first thing to do is to come up with a solid research question that will kick off your learning analytics journey and will hopefully lead to an actionable outcome. This immediately poses a challenge as there are so many questions you can ask. So where to begin? One of the reasons people find it hard to start with learning analytics is exactly this: what questions are worth asking for? In this series, I’ll provide you with some fairly simple, but very useful example questions to get you on your way.

Step 2: Collect the data: Based on the question you want to answer, you will need to collect data. Luckily, many of us have LMS, LXP and other platforms (like survey tools) that collect a lot of data. All this data is at your disposal. But again, having so much data available could lead to uncertainties on where to start. In this series I’ll show you the most foundational data we have in L&D and explain how to use it!

Step 3: Clean, Structure and Model the data: This is for sure one of the most technical steps in the process that requires the data to be cleaned, structured and modeled. This step is also called data engineering and in many cases you might want to get an expert in. However, it could be very useful and valuable to learn a few tricks yourself for the following 2 reasons. First by understanding even a little bit about data quality, data engineering and data modeling you will be able to far better instruct, guide and validate the work of a true expert. Secondly, it will give you the skills required to sometimes do this yourself, for example when timelines are tight, of data scientists are scarce, or simply when you feel it’s worth doing it yourself!

Step 4: Analyze the data: So this is where the through magic happens! But data analysis does not have to be very complicated! I will show you some simple examples that already provide a lot of actionable insights! I will show you some techniques that are called ‘exploratory data analysis‘: a concept where you use simple statistical methods to better understand the data you have at your disposal and where you see if there’s anything in there worth exploring further.

Step 5: Data visualization and Storytelling : Data visualization and Storytelling is a true art and science. This is a very interesting area on it’s own. One where I also still learn every day! For this series I will not discuss all in’s and out’s, but I will share some simple data visualization techniques that you can try and test yourself!

Step 6: Take action!: This is the 6th step of the process that I specifically added to emphasize the importance of always doing analytics with and action or decision in mind. Doing analytics just to produce numbers that look good on a slide is a waste of time and effort.

So much data….

So imagine your are leading the AI upskilling program at NexTech, or in fact any other L&D program in any organization. And you know you have a lot of data available. And you want to start doing something with analytics. Maybe you want to do it because you want to learn new skills (highly recommendable!), or you have a gut feeling that somebody might ask for it soon. Whatever the reason…you want to use that data, but you have not yet a clear idea on where to start.

This is were a bottom up approach using ‘exploratory data analysis‘ for learning analytics comes in handy! This approach will enable you to start small, start simple and while you are doing it, you will learn how data works, how to do simple analysis and how to visualize data. And while you are working on all these steps of the process, you will get more confident and comfortable with it. No doubt soon you will be able to pick up more complex analytics!

But, to be honest, there are some rules that apply with any bottom up approach for learning analytics.

  1. Start small. Even when you have literally thousands and thousands (or millions?) of data records at your disposal, start small. Make sure you take a slice of the data that is manageable. Not to many rows, not to many columns. This allows you to first of all use excel as a tool (still a great tool for analytics!), and equally important, it will enable you to simply test and validate your analytics outcomes, if you want to check a calculation or statistic! This is also the reason why you should never start learning analytics by trying to determine the business impact of a program. This is seriously complex stuff and you will not be able to validate your work!
  2. Bring L&D context to the table. Always choose a topic that you are very familiar with! Having a solid understanding of the program, the process, but also typical challenges will help you to first of all to focus and in choosing ‘right things to look at’. It will also help you to validate some of your results. In essence, you can then compare the outcome of the analysis with your understanding and perception. If you have a really good understanding of the context, and your analysis is correct, the outcome should show what you expect. If the outcome differs from your expectation, either your analysis is at fault, or you might not have the right perception of reality!
  3. Make mistakes. Do not expect to be doing it all right the first time. Making mistakes is ok. In fact, analytics work in general is a very iterative process. You often do not know if what you’re doing is going to work as expected. In many cases you only know if things work when you have tried. So in many cases you only find out it did not work after you have tried.

In this series I will share 3 examples of simple analytics that you can do with data you most like have. Insights that already can generate interesting and actionable outcomes. These metrics are:

  1. Audience Reach: What % of employees is upskilling in AI through your programs?
  2. Percentage Completions: from all employees who registered for AI learning, what % has actually completed the experience?
  3. Average time spend per person: How much time do employees spend on AI upskilling?

Why these three? Well….they are fairly easy to measure and track. And secondly they will provide you with a lot of insights that you can use to take various actions. But maybe the most important reason to cover these 3 metrics is that are among the most common metrics used in L&D. And that because they can answer 3 fundamental questions:

  1. Do employees ‘appreciate’ our AI programs and experiences?
  2. Do our programs reach the intended audience?
  3. Do employees spend sufficient time on AI upskilling

Now trued to be told, there are some assumptions hidden in these questions, and if you read my blogs, you know that assumptions can be tricky! But I will explain each relevant assumption carefully!

How to proceed

To help you get started, I’ve created 3 cheat sheets (one for each topic) that contain a step by step guide and instructions on what to do. Each cheat sheet discusses the following aspects of the learning analytics process:

  1. What Are We Measuring and Why?
  2. What Information Do You Need?
  3. What Tools Will You Use?
  4. Steps to Take
  5. Questions to Ask
  6. Actions You Can Take with the Insights
  7. Common Mistakes and How to Prevent Them
  8. Ideas for Further Analysis

Happy Analysis!

Learning Analytics Cheat Sheet – Audience Reach

Learning Analytics Cheat Sheet – Audience Reach by Peter Meerman

Learning Analytics Cheat Sheet – % Completions

Learning Analytics Cheat Sheet – Completion Rates by Peter Meerman

Learning Analytics Cheat Sheet – Time Spent on Learning

Learning Analytics Cheat Sheet – Time Spent on Learning by Peter Meerman
I have the data, but how do I start with Learning Analytics?
Tagged on: