Teddy Crepineau
Teddy is a Software Engineer at Collate (the software company building OpenMetadata, a metadata catalog). He is currently working on the ingestion part, more specifically on everything related to data profiling and quality tests.
Teddy has been working in the data field for 5 years in Analytics, Business Intelligence, and Engineering teams. He previously worked on building a data platform infrastructure, an ELT overlay framework and data pipelines. He loves contributing to open source projects in his downtime and studying software engineering and Python fundamentals.
Session
When processing data, validating its structure and its type is critical. Bad record types or changes in structure can often result in processing errors or worst in wrong data output. Yet, solving this problem cleanly and efficiently can be challenging. It often results in complicated code logic and increases complexity; consequently decreasing code readability. Pydantic is an efficient and elegant answer to these challenges
We expect you'll leave this talk with a good understanding of:
- Existing challenges in data validation
- What Pydantic Models, Validators, and Convertors are
- How to leverage Pydantic in your day to day (using real-life examples)
- [Bonnus] How to use Code Generation to create Pydantic Models from any data sources