Technical communication

Data scientists on my team develop and refine models that extract significantly more value from existing data. We data engineers translate their experimental code written in Databricks notebooks into production-grade code that runs on AWS Glue. These pipelines generate transformed datasets which our company packages as data products for customers.

Writing code is not the hard part of my job. The real challenge lies in understanding the data models developed by the data scientists. Here, clear technical communication between the two roles is important, especially since these models can be complex enough to form the basis of PhD projects. Once we understand a given model’s intent, it’s straightforward to iteratively refactor the code for production.

Good communication is a skill honed over time. Our team structure, divided between data engineering and data science, offers ample opportunities to exercise that skill.