Data scientist turnover: Burnout in a burgeoning profession

• Author:
• Author name

  Quantumrun Foresight
• April 25, 2022

Insight summary

The escalating burnout among data professionals is creating a ripple effect across industries, with data scientists often leaving their positions after just 1.7 years on average. This trend is rooted in organizational misunderstandings of the role and capabilities of data professionals, coupled with unrealistic expectations and restrictive policies. The long-term implications include risks to future strategies, instability in data sources, and an erosion of trust in data-driven decision-making.

Data scientists turnover context

Data is fast becoming among the most essential resources defining and driving future economies. Data scientists are critical to unlocking the potential power of this new commodity, yet these professionals are coming under increasing strain as demands on their time increase. With over 2.5 exabytes of data created every day (2021), more and more companies and organizations are beginning to extract value and insights from the data they gather or augmenting their systems to optimize already existing processes.

As more opportunities for data gathering become available, so does the need grow for qualified data science professionals, analysts, and engineers. However, according to an October 2021 survey by 365 Data Science, an online data skills training provider, 97 percent of the 600 data engineers surveyed experienced burnout in their day-to-day job. Seventy-nine percent are considering leaving the industry entirely. 

While turnover in this essential position can cause major disruptions for companies, it's mainly due to the substantial impact that data scientists and engineers have on data productivity and overall business agility. These professionals play a key role in managing and interpreting data, which is vital for quick decision-making and adaptability in the business environment. However, the underlying cause of burnout among data professionals is widespread and structural.

Disruptive impact

The primary causes of burnout among data professionals stem from an apparent lack of organizational understanding of their role and capabilities. Data scientists are likely to stay with their current employer for only 1.7 years on average, according to a 2021 survey. Frequent requests for analytics with unrealistic expectations and restrictive data governance policies limit their ability to positively impact their respective employers.

The high demand for data scientists means that new employment is relatively easy to find for professionals seeking to change employers. However, this ease of transition may lead to a lack of stability and continuity in their careers. For companies, the need to retain data scientists means that organizational structures should support rather than inhibit these professionals, including implementing efficient and reliable workflow processes to improve analytics collaboration and productivity. Given the scale of the problems faced and the high demand for their skills, financial incentives alone are unlikely to keep data scientists in unsatisfying jobs, emphasizing the importance of a supportive work environment.

For governments and policymakers, the trend of high turnover among data professionals presents a challenge in maintaining a stable and skilled workforce in a field that is vital to modern economies. The constant shifting of professionals between roles can lead to a loss of institutional knowledge and expertise, affecting long-term projects and national competitiveness. Educational institutions may need to focus on not only training data scientists in technical skills but also educating employers and executives about the importance of creating a conducive environment for these professionals. 

Implications of high data scientist turnover 

Wider implications of high data scientist turnover may include: 

• Organizations placing future growth and strategies at risk due to the inability to leverage their data professionals and function effectively, leading to missed opportunities in market expansion and competitive positioning.
• Companies being unable to build a solid foundation for data analytics to thrive internally, and this knowledge being spread throughout the business to create additional value, hindering the development of new products and services tailored to consumer needs.
• Unstable data sources and analytics limiting the flow of critical insights that impact an organization's ability to make decisions, seize opportunities, and extend suitable offers to customers, resulting in reduced customer satisfaction and loyalty.
• A continuous cycle of hiring and training new data professionals, leading to increased operational costs and reduced efficiency in utilizing data for strategic planning and execution.
• The creation of a competitive job market for data professionals, leading to salary inflation and increased costs for companies, potentially affecting the affordability of data-driven solutions for small and medium-sized enterprises.
• A potential shift in educational focus towards specialized training for data professionals, leading to a more skilled workforce but possibly creating an oversaturation in the market and neglecting other vital areas of study.
• Governments struggling to maintain a stable and skilled workforce in critical data roles, leading to potential delays in policy development and implementation, particularly in areas reliant on data analysis such as healthcare and infrastructure planning.
• A shift in labor demographics with a concentration of data professionals in urban areas, leading to potential regional imbalances in skill distribution and economic development.
• The potential erosion of trust in data-driven decision-making due to frequent changes in data professionals and lack of consistency in analysis, leading to skepticism among consumers and stakeholders about the reliability of data insights.

Questions to consider

• Do you think that offering higher wages to data science professionals will make a significant difference concerning the challenges these professionals face?
• Given how central data and data analysis is becoming to most organizations, do you think executives/directors are keeping pace with the necessary changes to optimally position employees for success?