Impersonation in Data Engineering: No More Credentials in Your Code!
2024-07-10 , North Hall

Imagine stepping into your dream job as a python data developer, ready to dive into coding and show your talent, only to run into missing database credentials that leave you idle for days due to slow interdepartmental communications and permission issues. Frustrating, right?

In my talk, I'll showcase how we can make this whole process much easier. I’ll explain how using something called “Identity and Access Management” (IAM) lets everyone in a company, including machines, get to work without these annoying holdups.

Surprised to hear that a machine like Airflow can have its own identity? I'll explain how we use something known as Workload Identity as a crucial part in this ecosystem integrating Airflow within our infrastructure.

A central pillar of the discussion will be the role of impersonation in our setup - how it ties together various elements to enable a harmonious, secure, and maintainable infrastructure. The resulting architecture not only fosters an improved developer experience, faster project delivery, increased productivity and transparency, but also serves as a foundation for more advanced concepts such as data mesh implementation.

Join me in this talk to discover the synergy of IAM, Workload Identity, and impersonation. Let's equip you with a model that promotes easy team onboarding, transparent access management, and a secure, frustration-free workspace focused on delivery.

And for those interested in having their code perform consistently, whether on a local machine or in the cloud, I will share a small but powerful Docker hack to achieve things consistently no matter where your code is running.


Expected audience expertise:

Intermediate

See also: Presentation slides (6.7 MB)

Marian Spilka, a Senior Data Engineer at Kiwi.com, is an adept in orchestrating data workflows with managed Airflow and helping teams through Data Mesh architecture integration. Marian is passionate about everything around data. He finished his doctoral degree in Computer science with a focus on face recognition processing.
Beyond professional expertise, Marian is also an enthusiast in smart home technology, merging the worlds of DIY projects with his love for innovation. Marian cherishes everyday moments with his family and loved ones, balancing work with a fulfilling home life.