Distributed Advanced Analytics Through Citizen Data Scientists
For analytics teams to be successful, they need to learn how to scale their human resources using citizen data scientists. This will require new thinking and new practices.
When you hear the term “distributed advanced analytics,” what do you imagine? Do you think of large, horizontally scaled Hadoop clusters with Spark running on them processing complex models and generating advanced insights? What if the future of distributed analytics in 2023 was less about scaling machines horizontally but rather scaling human resources horizontally?
Since the beginning of the last decade, universities around the world have pushed to train a new generation of data scientists. They incorporated courses in their curricula focused on teaching students about R, Python, and Scala syntax and usage. They built courses on data engineering processes and the process of building advanced ecosystems of open source and proprietary technology that can process big data in its various facets. This has greatly increased the supply graduates who can think analytically, but not all of them are destined to join the analytics team.
The challenge is that as enterprises strive to be more data-driven, the demand for data and analytics resources outpaces the supply. The answer to this challenge is to adopt the citizen data scientist model. In this case, the term “citizen” really refers to anyone who is not part of the analytics organization. They can be internal or external resources with the skills needed to augment efforts to become a data-driven organization.
To accomplish this scale-out, there are three questions an organization must answer: where do we find citizen data scientists, how do we enable them, and how do we protect the organization from their potentially destructive behavior, whether intentional or unintentional?
Finding Citizen Data Scientists
When searching for citizen data scientists, you need to find individuals who are analytical by nature and have complementary skills to those of your analytics team. You are looking for resources with skills that can add a new perspective to the problems you are trying to solve.
There are many resources in other parts of your business, in scientific fields of study, or in the IT department that can provide these complementary skills (see Finding Talent on the Periphery). By incorporating these individuals into your analytics team, you add diversity of thought and new tools and techniques for solving problems.
Identifying these resources is the first step. The next step is making the opportunity attractive and compelling. You need to establish a value statement about what you are doing that is attractive to these resources and engages them to want to help. This could include setting an attractive vision, establishing a compelling compensation structure, and incorporating them into your culture and team structure.
Enabling Citizen Data Scientists
When working with citizen data scientists, you will have a mix of skill sets. You could have individuals versed in different languages (e.g., R, Python, Scala, SAS) and at different levels of maturity in data preparation and data engineering capabilities. You want to start with the skills they have and build from there.
Your goal is to establish tools that provide these citizen data scientists with the appropriate data and functionality to assist in your advanced analytics …….