In the domain of scientific and high-performance computing (HPC), software packages are primarily written in compiled languages such as C, C++, and Fortran, complemented by end-user APIs implemented in Python. Such packages frequently incorporate CPU-specific code (e.g. SIMD extensions) and utilise GPU-specific programming models, such as OpenMP and CUDA, to achieve enhanced performance. Despite the recent proliferation of build backends for creating pure Python packages, the distribution of Python packages containing binary extensions poses a unique set of challenges and currently lacks a standardised solution. In this talk, I will share insights and experiences gained from building portable and performant Python wheels for a set of computational neuroscience projects, aiming for compatibility and usage across a diverse of compute platforms, from desktop to large compute clusters.