Hi! Oriana here. Although I have been behind the scenes at Analytics Engineers Club for a while now, I haven’t properly introduced myself. You may have seen me helping students debug errors in Slack or read my recent blog post about using Codespaces at AEC!
The TLDR: Prior to AEC, I worked as a sole data practitioner and wished that I had known about this course earlier.
Side-stepping into Data
As Claire has written about in the past, there are many “side-steppers” in the data community, and I am one of them. In fact, I have sidestepped a few times now. I started out in the technical fields of electrical and biomedical engineering. Then, I switched to business strategy and churned out PowerPoint decks at a very large consulting company. Eventually, I found my dream job as a product manager of the digital business of a coffee shop. As a PM, I had always been drawn to doing my own data analysis and picked up some SQL along the way.
After we launched our mobile app, I delved even deeper into the analytics part of my job and wrangled our Segment event data into meaningful user metrics. If you can see where this is going, I sidestepped once more into a new role as head of our data team.
A Shaky Start
I entered this new role at an interesting juncture. At the time, our data stack was built on an ETL model, with everything happening in Airflow and Redshift. Eventually, business priorities shifted, engineering roles were reassigned, and I suddenly found myself as the sole dedicated data person. Without engineers maintaining our Airflow instance, our pipelines became brittle and data jobs would frequently fail or stall.
On top of this, we were performing complicated transformations in our BI tool. Imagine having to navigate and debug long, unwieldy SQL queries embedded in Tableau data sources: no versioning, no testing, and no documentation. And because these complicated SQL queries were costly to execute, we scheduled the queries to execute at certain times via Tableau. So not only were we scheduling data jobs to run in Airflow, but we had another scheduler built into Tableau.
Not a week would go by that I wouldn’t wake up to the dreaded Slack messages telling me that “numbers looked off” in so-and-so report. Each time, I’d have to diagnose which problem it was: Airflow DAG needed refreshing? A mandatory field left empty in our ERP? Outdated business logic in one of the many Tableau SQL data sources? It didn’t take me very long to realize that I was in an untenable situation, especially as the only person focused on data and analytics at the company.
Befriending the Modern Data Stack
Around the same time, tools for the modern data stack, the concept of self-service analytics, and the role of an analytics engineer became more prominently featured in our internal conversations. The intersection of these ideas went hand-in-hand in supporting a lean data team. We realized that we could offload the data engineering work with a tool like Fivetran. We introduced a BI tool that would be friendlier to business users, thereby empowering folks outside of our data team to pull their own numbers and build their own reports. We started moving our transformations from Airflow and Tableau into dbt, which improved maintainability and reliability in our pipelines.
As part of this transition, I moved squarely into the role of analytics engineer, overseeing the entire ELT data pipeline. In another sense, I became a data generalist — I wasn’t specialized in the technical details nor was I specialized in any one business area, but I knew enough to be dangerous. This didn’t happen overnight, and at times it felt like I was feeling around in the dark. I spent a lot of time trawling the dbt Slack community, perusing countless blog posts, and Googling the Internet ad infinitum. It was all this Googling that brought me to the likes of Claire and Michael!
Eventually I learned what I needed on the job, but you can take the easy way. We’ve designed AEC to be the course that we wished we had when we were learning on the job. At AEC, we’ve put together a comprehensive course that covers everything a one-person data team needs to know to make the modern data stack work for them. Are you a team of one? Did this article strike a chord? Drop us a line — we’d love to hear from you!
