What I do or: science to data science

tl;dr After finishing my PhD in quantum information, I turned to data science. I couldn't be happier about this transition! Compared with academia, data science world looks to me like a wonderland.

When meeting acquaintances, the typical conversation goes as follows:

• acq: I saw you finished your PhD in Dec 2014. So, what do you do right now?
• me: It's complicated: various places, various things, but finally I feel free!
• acq: You mean, you are unemployed?
• me: No, no! I have .
• acq: Oh, I see... and an awkward silence
• me: But it's legal!

I do data science freelancing. That is, I take contracts related to machine learning (predicting things, e.g. user growth of a company), data visualization (custom charts in D3.js), preparing and conducting trainings in data analysis (e.g. in scikit-learn and Spark), and other data expertise tasks (e.g. merging data from various source). It's rather a consulting-like service than being a code-monkey.

I aim at short projects - I like energy, deadlines (sic!), and novelty. Typically I run 2-3 projects simultaneously. Some projects are purely commercial, some are academic or for NGOs (but still with grant money), some are my personal side-projects. In almost all of them I have a lot of freedom in setting how I am going to solve a problem. In fact, usually I am required to propose a solution to their problem, e.g. to design a data visualization, not only code it.

Right now, I love it! Here is why:

• I'm free (I can move anywhere, anytime and I don't need to ask anyone for permission; it has some very direct benefits, like being able to live with my girlfriend, instead of a few thousand km away).
• I have sense of being needed (people contact me when they want someone to solve a task they can't do by themselves).
• I'm being appreciated (maybe it's shallow, but instead of being "yet another PhD student" I am Piotr Migdał).
• Data science is a rapidly growing field, with yesterday's challenges being today's industry standards (rather than yesterday's hopes being today's... hopes).
• I have a lot of intellectual stimulation (the last time I had such is 8 years ago or so - when starting my undergraduate studies); partly because each project is different, partly because I am not confined to a subsubdiscipline.
• No bureaucracy, it takes almost an instant from agreeing to starting a project.
• I feel that the programming world is much more meritocratic than academia:
  • Interviews are hard and technical, instead of looking at with whom I worked (i.e. recommendations).
  • I am being contracted because my particular skills are needed, regardless of other formal criteria.
  • Having a PhD is an impressive discussion-starter, but it's almost never required - and I consider it a good sign!
• No politics - as a freelancer I can avoid it almost completely.
• Projects are fast - if sometimes does not work, instead of getting more and more frustrated over the course of years, we change approach or pivot.
• Much more $; while I was never too greedy, it has a direct consequence:
  • I can buy a lot of time for my own projects (e.g. Quantum Game).
  • I am the chooser - all projects I accepted in the last year where ones I really enjoyed!
• I interact with various people (IT, marketing, management and scientists) rather than only with fellow scientists.
• I still travel a lot (mostly to lead my trainings).

The overall difference is tremendous - from swings between depression and frustration (with occasional rays of hope) and a very unproductive state by default, to feeling great and learning a lot. Of course, it is also a function on my personal traits (chaotic, hot-headed, multidisciplinary, loving data, disliking hierarchy, a bit narcissistic¹) and my particular experience of academia.

I was afraid that the escape from academia would burn bridges, especially as I was openly saying I wanted to move out straight after my PhD defense². Yet, from time to time I am being invited for scientific talks or collaboration, even when I make it explicit that I am no longer in academia (or aim to be again). Furthermore, I feel that it's easier for me to contribute to education as an independent freelancer than from within the system³.

Very often I am being asked by academics how to make the transition from physics or mathematics to data science or programming. I do my best to give as detailed, personal and practical answers as possible. When I get such emails, it's flattering when it comes from undergraduate students, I feel sympathetic when it's from PhD students, and it is sad - when from people, who have invested in academia a number of years after their PhDs. Especially ones that love academia with its quirks, but have family to support and cannot bear the lack of security and constant move.

I have never regretted my transition. Instead of asking myself "why I am doing this" I am happy.

Caveat

Don't try to treat it as an unconditional recommendation of data science freelancing!

Data science is great, but you need to enjoy programming, dealing with real & dirty data and with "good enough" solutions. (But no, you don't need to work in adverts or finance.)

Freelancing is a perpendicular dimension (the "normal way" to do data science is via full-time works). Whether you love or hate freelancing boils down to your psychological traits and whether you are able to get clients. As I like to say:

Freelancing is something exactly between having holidays and deadlines all the time.

Empowering the readers

There are a few follow-ups I'm considering:

• How did I learn about data science?
• How to move from academia to data science? (mostly: practical links)
  • EDIT: Data science intro for math/phys background
• Why I got disappointed by academia?

If you like to hear about one of these topics, mail me! :)

I would like to thank Michał Kotowski, Marta Czarnocka-Cieciura and Jacek Migdał for comments on the draft. This blog post started as an email to Artem Kaznatcheev entitled "what I do" I wrote when flying from Dublin to NYC. Credit for the doge PhD picture goes to Ryszard Paweł Kostecki.

Footnotes

1. Sometimes I say that freelancing is a way to monetize one's narcissism: advertising oneself is a part of this job.
2. To be absolutely fair, I did apply to a single postdoc. But even then I collected notes in a file entitled Stockholm syndrome.
3. I still volunteer for the Polish Children's Fund - not less compared to when I was in academia. I already created from scratch and conducted a one-week intensive introduction to data analysis in Python for social scientists (github.com/DELabUW/szkola-letnia-2015, in Polish). It would be hard to have such freedom as a young lecturer.