Simple data validation and setting management with Pydantic
2022-07-14 , Liffey Hall 1

When processing data, validating its structure and its type is critical. Bad record types or changes in structure can often result in processing errors or worst in wrong data output. Yet, solving this problem cleanly and efficiently can be challenging. It often results in complicated code logic and increases complexity; consequently decreasing code readability. Pydantic is an efficient and elegant answer to these challenges

We expect you'll leave this talk with a good understanding of:

  • Existing challenges in data validation
  • What Pydantic Models, Validators, and Convertors are
  • How to leverage Pydantic in your day to day (using real-life examples)
  • [Bonnus] How to use Code Generation to create Pydantic Models from any data sources

Expected audience expertise: Domain:

some

Expected audience expertise: Python:

some

Abstract as a tweet:

Leverage the power of Pydantic to validate your data efficiently

Teddy is a Software Engineer at Collate (the software company building OpenMetadata, a metadata catalog). He is currently working on the ingestion part, more specifically on everything related to data profiling and quality tests.

Teddy has been working in the data field for 5 years in Analytics, Business Intelligence, and Engineering teams. He previously worked on building a data platform infrastructure, an ELT overlay framework and data pipelines. He loves contributing to open source projects in his downtime and studying software engineering and Python fundamentals.