Dynamically generated methods with a non-generic signature
07-19, 15:30–16:00 (Europe/Prague), PyCharm (Forum Hall)

In other words, Descriptors + PEP-362 (function signature object) and a seasoning of PEP-487 (simpler customization of class creation via __init_subclass__).

There are different ways to have generated methods and attributes attached to all classes in a library, and this talk presents the way we’re doing it in scikit-learn. Here you’ll understand the use-case, and see the details and challenges presented by it, and how we approached them.


Title:
Dynamically generated methods with a non-generic signature

Abstract:
In other words, Descriptors + PEP-362 (function signature object) and a seasoning of PEP-487 (simpler customization of class creation via __init_subclass__).

There are different ways to have generated methods and attributes attached to all classes in a library, and this talk presents the way we’re doing it in scikit-learn. Here you’ll understand the use-case, and see the details and challenges presented by it, and how we approached them.

Long:
The use-case we study here goes as: we would like to add methods to all Estimators, which are all subclasses of the BaseEstimator. The signature of the methods generated depends on the signature of other methods existing in those subclasses, but we also want to give the option of modifying generated methods without having to change the existing methods.

The solution we present involves a few concepts which we’ll explain during the talk:
Investigate the existing methods’ signature using inspect
Manually traverse MRO (method resolution order) and inspect class attributes allowing for modifications on what the inspect has concluded
Use a descriptor to generate methods accordingly
Use PEP-362 to attach a signature object to the generated methods
Dynamically generate docstrings for those methods
Use PEP-487, aka __init_subclass__, to attach those methods to child classes when appropriate

This is a hands-on talk, explaining each concept in isolation and then showing how they fit together, and we’ll be presenting and testing code during the talk.


Expected audience expertise

advanced

Adrin works on skops which tackles some of the MLOps challenges related to scikit-learn models. He has a PhD in Bioinformatics, has worked as a consultant, and in an algorithmic privacy and fairness team. He's also a core developer of scikit-learn and fairlearn.