Machine Learning Allows Employers to Leverage Skill Data in Ways We Never Dreamed

In 2019, our team at Lighthouse Research & Advisory set out to determine if artificial intelligence (AI) is better than humans at some HR tasks. We know that automation can process some actions faster than a human, but what about finding insights in employee engagement data? Can a computer really do that better than we can? 

In the experiment, we gathered 1,000 responses from workers across North America on workplace stressors, manager relationships, and more. We then shared that data with a few consulting firms for expert inputs and recommendations. The insights were broadly helpful and interesting, but they weren’t actionable. For instance, we found that 60% of our “workforce” was stressed at work, but we didn’t get any clarity into who was stressed or why. 

We then ran that same data through an algorithm that looked at sentiment within the comments. The output was incredibly dense and rich with insights. For instance, we could see specific stressors (manager relationships, working conditions, family responsibilities, etc.) by gender, race, job, or geographic location. 

This proved to us that AI is better than humans at analyzing feedback data to help us take action as organizational leaders. 

Just like in the experiment above, there are other areas where rich data sets exist within the business that haven’t been mined for insights. In the rest of this piece, we’ll explore some of the areas where AI can suggest actions, offer insights, and create value for employers that leverage technology to examine one of their richest, untapped data sources: employee skills. 

Skills and the Spatial Relationship

Every possible unique skill on the planet has some relationship to others. This explanation from Workday helps to explore that concept more fully: 

Understanding the spatial representation of skills provides a clear picture of how closely skills are related to one another, as well as to those entities represented with skills (jobs, for example).  This enables us to determine a more optimal path toward a target result—in this case matching workers or candidates to jobs, content, learning, mentors, etc.; and vice versa, matching jobs to candidates, learning to workers, and more.

Let’s use a basic example to explore the concept. In a marketing job, these might be relevant skills: 

• Writing
• Research
• Analysis

However, in a software engineering job, these may apply: 

• Coding
• Collaboration
• Analysis

In each of these two jobs, there’s a need for analysis of outputs, but otherwise the skills don’t overlap heavily. That means the relationship between coding and writing skills (while both of them actually mean sitting and typing at a computer) is relatively far apart spatially. However, within each job, the skills have a much closer relationship. It helps to think about jobs not as discrete and distinct entities, but as clusters of skills. 

Now, expand this very basic explanation across the thousands of possible skills across the millions of jobs people do, and you start to see how the challenge of identifying and relating this universe of skills is something that is well-suited for an algorithm to support. 

AI and Machine Learning Insights

As with the experiment we covered above, humans can do some limited analysis of skills, but there is so much more information than we can easily process or use. Machine learning and AI allow us to use this raw skills data in a wide variety of ways to support talent and business decisions. For example, it can uncover: 

• Suggested skills – supports individual skill development paths
• Skills gaps – uncovers what skills a company has and what skills they need
• Suggested learning content or development opportunities – personalized recommendations for learning
• Skills verification – confirms suspected/inferred skills
• Suggested gigs/opportunities – personalized recommendations to leverage strengths
• Job matching – personalized recommendations for relevant jobs and advancement opportunities

While not an exhaustive list, this helps to paint the picture of how an intelligent organization with a foundation of data-driven AI can derive and utilize the value from large skills-focused data sets. 

Future Vision: The Self-Developing Workforce

The potential for this type of approach is incredible, but it also opens up something that we’re seeing increasingly in the talent market today. Workers are increasingly expecting control over the velocity and direction of the work they do. The future vision of this AI-enabled and skills-focused strategy allows us to have a self-developing workforce. Machine learning understands what skills exist and where gaps may be and helps to target development, hiring activity, and other interventions to solve problems before the company may even realize the problems are happening. 

Is this a bit fantastical? Yes. But even a few short years ago, we would have said the same about an algorithm identifying  what skills a person has with any degree of clarity. 

Our research shows that understanding the skills of your organization is a critical business problem, not just a siloed talent issue. Leveraging technology to solve this challenge will create a competitive advantage for those employers willing to adopt and use it. It’s not really a question of “if.” It’s a question of “when.”