What I do or: science to data science
14 Dec 2015 | by Piotr Migdał
tl;dr After finishing my PhD in quantum information, I turned to data science. I couldn't be happier about this transition! Compared with academia, data science world looks to me like a wonderland.
When meeting acquaintances, the typical conversation goes as follows:
- acq: I saw you finished your PhD in Dec 2014. So, what do you do right now?
- me: It's complicated: various places, various things, but finally I feel free!
- acq: You mean, you are unemployed?
- me: No, no! I have
. - acq: Oh, I see... and an awkward silence
- me: But it's legal!
I do data science freelancing. That is, I take contracts related to machine learning (predicting things, e.g. user growth of a company), data visualization (custom charts in D3.js), preparing and conducting trainings in data analysis (e.g. in scikit-learn and Spark), and other data expertise tasks (e.g. merging data from various source). It's rather a consulting-like service than being a code-monkey.
I aim at short projects - I like energy, deadlines (sic!), and novelty. Typically I run 2-3 projects simultaneously. Some projects are purely commercial, some are academic or for NGOs (but still with grant money), some are my personal side-projects. In almost all of them I have a lot of freedom in setting how I am going to solve a problem. In fact, usually I am required to propose a solution to their problem, e.g. to design a data visualization, not only code it.
Right now, I love it! Here is why:
- I'm free (I can move anywhere, anytime and I don't need to ask anyone for permission; it has some very direct benefits, like being able to live with my girlfriend, instead of a few thousand km away).
- I have sense of being needed (people contact me when they want someone to solve a task they can't do by themselves).
- I'm being appreciated (maybe it's shallow, but instead of being "yet another PhD student" I am Piotr Migdał).
- Data science is a rapidly growing field, with yesterday's challenges being today's industry standards (rather than yesterday's hopes being today's... hopes).
- I have a lot of intellectual stimulation (the last time I had such is 8 years ago or so - when starting my undergraduate studies); partly because each project is different, partly because I am not confined to a subsubdiscipline.
- No bureaucracy, it takes almost an instant from agreeing to starting a project.
- I feel that the programming world is much more meritocratic than academia:
- Interviews are hard and technical, instead of looking at with whom I worked (i.e. recommendations).
- I am being contracted because my particular skills are needed, regardless of other formal criteria.
- Having a PhD is an impressive discussion-starter, but it's almost never required - and I consider it a good sign!
- No politics - as a freelancer I can avoid it almost completely.
- Projects are fast - if sometimes does not work, instead of getting more and more frustrated over the course of years, we change approach or pivot.
- Much more $; while I was never too greedy, it has a direct consequence:
- I can buy a lot of time for my own projects (e.g. Quantum Game).
- I am the chooser - all projects I accepted in the last year where ones I really enjoyed!
- I interact with various people (IT, marketing, management and scientists) rather than only with fellow scientists.
- I still travel a lot (mostly to lead my trainings).
The overall difference is tremendous - from swings between depression and frustration (with occasional rays of hope) and a very unproductive state by default, to feeling great and learning a lot. Of course, it is also a function on my personal traits (chaotic, hot-headed, multidisciplinary, loving data, disliking hierarchy, a bit narcissistic¹) and my particular experience of academia.
I was afraid that the escape from academia would burn bridges, especially as I was openly saying I wanted to move out straight after my PhD defense². Yet, from time to time I am being invited for scientific talks or collaboration, even when I make it explicit that I am no longer in academia (or aim to be again). Furthermore, I feel that it's easier for me to contribute to education as an independent freelancer than from within the system³.
Very often I am being asked by academics how to make the transition from physics or mathematics to data science or programming. I do my best to give as detailed, personal and practical answers as possible. When I get such emails, it's flattering when it comes from undergraduate students, I feel sympathetic when it's from PhD students, and it is sad - when from people, who have invested in academia a number of years after their PhDs. Especially ones that love academia with its quirks, but have family to support and cannot bear the lack of security and constant move.
I have never regretted my transition. Instead of asking myself "why I am doing this" I am happy.
Caveat
Don't try to treat it as an unconditional recommendation of data science freelancing!
Data science is great, but you need to enjoy programming, dealing with real & dirty data and with "good enough" solutions. (But no, you don't need to work in adverts or finance.)
Freelancing is a perpendicular dimension (the "normal way" to do data science is via full-time works). Whether you love or hate freelancing boils down to your psychological traits and whether you are able to get clients. As I like to say:
Freelancing is something exactly between having holidays and deadlines all the time.
Empowering the readers
There are a few follow-ups I'm considering:
- How did I learn about data science?
- How to move from academia to data science? (mostly: practical links)
- Why I got disappointed by academia?
If you like to hear about one of these topics, mail me! :)
I would like to thank Michał Kotowski, Marta Czarnocka-Cieciura and Jacek Migdał for comments on the draft. This blog post started as an email to Artem Kaznatcheev entitled "what I do" I wrote when flying from Dublin to NYC. Credit for the doge PhD picture goes to Ryszard Paweł Kostecki.
Footnotes
- Sometimes I say that freelancing is a way to monetize one's narcissism: advertising oneself is a part of this job.
- To be absolutely fair, I did apply to a single postdoc. But even then I collected notes in a file entitled Stockholm syndrome.
- I still volunteer for the Polish Children's Fund - not less compared to when I was in academia. I already created from scratch and conducted a one-week intensive introduction to data analysis in Python for social scientists (github.com/DELabUW/szkola-letnia-2015, in Polish). It would be hard to have such freedom as a young lecturer.