An Introduction to Data Roles

With the number of different data roles open on the market today, it’s difficult for an outside individual to figure out which roles are fungible across one-another and which are not. We wrote this article for non-data folks to resolve confusion around the roles in data, where they overlap, and provide a starting point for understanding how to build and scale a data organization from a staffing perspective.

The 4 core data roles

Though there are at least 10 different roles in a data practice, all of those roles are effectively related-to, or a derivative of one of the four core roles in data: Data Analyst, Data Engineer, Data Scientist and ML Engineer.

Across these four core roles and their derivative counterparts there is a certain degree of fungibility, but there is little overlap between the roles themselves. As organizations expand in size and complexity, new roles adjacent to the core roles above develop. These are roles that at large scale are assigned to an individual or group, but at a small scale may be folded into the original four.

A common pitfall in hiring and staffing data teams starts with hiring a core role that is not aligned with the actual needs of the business. Most often this trap manifests as a company hiring a data scientist or data analyst without any existing infrastructure for them to work from, and then expecting them to either build the infrastructure themselves, or produce production-scale outputs on top of messy data.

The chart below shows how these roles relate to one another, and which ones fold into the core role at smaller scales. The X axis describes where each role sits in terms of relative business/domain expertise, and the Y axis describes where each role sits in terms of relative technical competency. Everyone wants to hire the unicorn person who can build everything from scratch across every discipline, and translate findings into actionable tools and or decisions for the business to take.

In general though, there is an inverse relationship between technical expertise and business knowledge because data professionals that spend time building back-end systems rarely get the chance to develop their interpersonal communication skills, or learn about the business, and the reverse is true of business analysts and operators.

The darkest dot in each chain is the role that absorbs the others when headcount is tight; and the lightest is the role that gets folded back in first. Data Scientist and Analytics Engineer are commonly conflated in job postings; same for Data Engineer and Data Analyst.

Both axes in the chart above are skills that take time and experience to develop, and therefore are somewhat mutually exclusive. Technical depth and business/domain expertise are both things people build with exposure and effort.

A Platform Engineer can learn enough of the business to write a useful dbt model, a Business Analyst can learn enough SQL to pull their own numbers. A brilliant Data Engineer isn’t fluent in the business just because they’re fluent in the warehouse, and a sharp Business Analyst can’t write production pipelines just because they know the P&L.

The key to hiring data roles correctly is being able to reconcile the requirements of the business from a data perspective with the competencies built by candidates in each of these roles.

Below we've added a tool that lets you play around with various configurations of the 10 roles. Select any of the tiles individually or together, or use one of the presets we've added to see how they overlap.

Where AI-era roles fold in

In the AI era, these roles are further complicated by the addition of new roles that are adjacent to existing data roles. In many cases the AI role is really the existing data role wearing a moustache, (i.e. a Data Platform Engineer and a AI Platform Engineer use basically the same processes across different substrates), but for others, the jury is still out on whether they'll become their own role, or get folded into an existing one.

The chart below shows the AI roles and their relationships in addition to the existing data roles, hover over any dot to see a description of the role and its overlap.

Why these roles are so different from one another

There are two reasons these roles are so heavily differentiated:

The first is that data roles developed over the past 2 decades following a relatively non-traditional path. For this reason, many data professionals either migrated left or right in their organizations to end up in data; starting as a business analyst or statistician and moving left up the stack or starting as a software engineer and moving right. As a result, data professionals come from diverse backgrounds, and therefore can have wildly different core competencies.

The second reason is that until recently, there was no classical training at the secondary or continuing education levels for data disciplines specifically, and standards and norms within the industry change extremely rapidly with subsequent advancements in technology. This contributes additionally to the spread of backgrounds and skills across data professionals.

Cheat Sheet