Using data for the public good

Microsoft. Facebook. Google. What do these companies have in common? Yes, they’re vast, powerful, innovative tech companies who define our digital existence. They’re also amongst the biggest recruiters of data scientists in the world.

This isn’t an appeal to study the analysis and science of data because it will get you a job at one of these companies. It’s an appeal to study it because those skills should be put to work on something far more important. The people you see around you every day as you drive to work, or drop your kids off at school, or sit in your GP’s waiting room – those people don’t need another Google. They need good roads, good schools and good healthcare, and increasingly that means seriously good data science too. Data underpins every facet of our society, and we owe it to our communities to use that data to make their lives better.

The public deserves impeccable data science. You have the power to provide it.

I’ve spent the past few years working for various charities and political campaigns as a fundraiser. I did everything I could to inspire people to give their time and money to change the world for the better, but in the process I discovered something incredible, and honestly, a little frightening. Everything I did, from what I wrote, to the design decisions I made, to the people I spoke to, all of it could be supercharged with data science. I worked with people using reinforcement learning to test donation suggestions. Clustering to experiment with customer segments. Multilevel regressions to predict election outcomes. I became incredibly conscious that there was a limit to the power of the communications I could produce, to the results I could achieve with creativity alone. But my data scientist colleagues? We could have tripled them and still had slack to do more. We had copywriters and designers coming out our ears. Where were the data scientists?

Microsoft. Facebook. Google. And if not there, Barclays. JP Morgan. Deutsche Bank. And to be fair, academia. But I still can’t shake the feeling that this is a gross misallocation of resources. I’m uncertain how much the public needs Forbes’s forex algorithms updated. I am absolutely convinced of the public’s need for a fair, efficient tax system.

You should be a data analytics apprentice because the public sorely needs these skills.

There are lots of pragmatic reasons that learning this work in the Civil Service is a great idea. The data sets are massive and varied, there’s a lot of inspiring and talented people, the pay is good, and the support and exposure to professionals across the government is fantastic if you look for it. These are all things other blog posts go into in more depth. The more convincing argument, to me, is the moral one. The public sector needs unparalleled data science. Whether your policy line is global Britain or buy British, levelling up or devolution, outsourcing contracts in Whitehall or insourcing them in local councils, the world we live in requires data science to pursue any of these positions with any success. In short, this is necessary, exciting, fulfilling work. You should do it.

Managers, recruiting apprentices is the right thing to do.

And yes, there’s pragmatic reasons too. We’re guaranteed, eventually, to have rigorous and diverse competencies. We’re cheap. We’ve chosen to pursue a particular academic discipline, and should therefore be curious and motivated. But in my opinion, if you can inspire a new generation of public sector data scientists, you should do it because it’s the right thing to do. You have the chance to shape a lifelong learner, equipped with not just the attitudes and methods a good data scientist needs, but the commitment to the public spirit that the field is crying out for. This is what my manager inspires me to do, and if this is an ideal you as a manager share, you should step up to the powerful and nurturing role you could be playing in the future of the field.

The world doesn’t need more billionaire tech giants. Data science should make the world a better place for everyone, not as a footnote to profit. There’s a need to fulfil, a job to be done, and there’s something you can do about it right now.

Become an apprentice, or recruit one. You won’t look back.