Data Analytics, Data Democracy and the Velocity of Innovation
By Kelvin Tan, Head of FinTech & Data, Technology, Singapore Exchange
Data and analytics are increasingly being recognized by senior management as essential to sustaining the growth and competitiveness of companies. For many organizations, including those in the financial sector, setting up a data and analytics group, data management office or similar is typically a top-down directive. A Chief Data Officer, or Chief Data Scientist, is put in place to have C-suite driving the agenda. Along with this comes a multimillion-dollar budget to install a data warehouse or data lake, deploy analytic tools and hire a roomful of data scientists.
But what if we tried a bottom-up approach, leaning on a handful of good people on the ground who believed in the power of data? Can similar objectives be met with passion, commitment and perseverance? Based on my experience, such a journey would be a challenge – but the answer is yes.
Rolling up Sleeves
The first thing we did was to build a simple database: collect and store as much data as possible, both historical and on an ongoing basis, at the most granular level. This meant collecting files and reports that our legacy systems and operations staff had produced every day since the beginning.
Historical data is especially important for advanced analytic capabilities such as machine learning. However, due to the cost of storage in the past, much data had been deleted or archived as IT departments had to manage costs; back then, no-one saw a need to store data on disks beyond a short period of time. Today, hardware and storage has become quite affordable, while database and analytics software is often freely available.
Once we had enough data to work with, our next challenge was to analyze it and demonstrate the value of data analytics to the business. Since there was only so much we could do on our own, we went about empowering our end-users – and this is where I believe the paradigm shift happened.
To prove how data analytics can help them, we had to sufficiently understand their jobs.
It is important to realize that cyber security is not an IT issue but a strategic issue
This involved providing training and, at times, doing some of the work to get them started. The idea was to show that data can make a difference, and secure their buy-in and support. Let data, rather than opinions, drive decision making across the organisation.
Empowerment leads to culture change and, admittedly, this was not easy. Culture change is tricky when top-down driven and can be significantly more difficult with a bottom-up approach. But when end-users, especially those in the businesses units, feel empowered, change can happen. Data empowers!
More data, more power
Eventually, as senior management started seeing the dashboards, charts and analysis being produced, end-users began to request for even more data and analytics. This opened the way for additional resources to be deployed for an enterprise data warehouse, dashboard and analytic tool. A dedicated data team is now required to manage the ever increasing amounts of data, and it was imperative to maintain quality and accuracy – our analysis depended on it.
With more data comes more power, and with that comes more responsibility. But these need not be vested in the data team alone. I am a strong believer in a federated data organisation, where the core team manages the databases, ensures the quality of the data and implements the analytics tools and reusable models. At the same time, we should also rely on end-users from the business units who have the skills to analyze data and derive relevant insights. Deep domain expertise can be crucial.
While we have empowered end-users who are able to work with analytic tools, many still lack the coding skills to be able to leverage the big data we now have. So, we began to train designated “super users ” from our business units – those who can do some coding to perform a few of the more complex analytics where the tools fall short, or code up automated scripts to improve operational or administrative efficiencies. Coming from the business units means they can become key people to drive innovation.
Velocity of innovation
Now that we have the data and have embarked on creating a data-driven culture, the next step would be to move towards an innovative culture. Data can increase the velocity of innovation.
For most large organizations, the easiest and most obvious starting point for innovating is to automate processes for efficiency gains. Automation usually requires either the collection or usage of data. Many analytics are also automated as an ongoing process and fed to a dashboard that is updated periodically or in real-time. Where these processes have bureaucracies that inhibit innovation, automation can help break down barriers.
Today, we are further empowering our super users by providing them with “sandboxes” for software development on proof of concepts with production data. (This is important as software development and testing conducted with artificial test data tend to fail miserably at launch with actual production data.) This can help a business validate assumptions quickly without taking too much time or resources from its IT department.
While every company and its culture are unique, I hope my experience can help organizations that are embarking on their first steps. A bottom-up approach in building capabilities in data and analytics can be successful. Data is the new oil and Innovation is the new engine of growth.