Will Rogers: Clúid Housing @ PyCon Ireland 2024 скачать в хорошем качестве

Will Rogers: Clúid Housing @ PyCon Ireland 2024

Clúid Housing is an award-winning, not-for-profit housing body providing over 11,000 affordable, high-quality homes to people in housing need. Recently Cluid's Data Engineering team played a big role in a complex data migration to a new Housing Management System in dynamics365. Python was a big part of this journey. Some simple custom classes evolved into a package that glued our migration together, all in one codebase. We'd like to share our experience, how it helped with collaboration and enabled us to iterate towards success. We'll cover: • A summary of the project including the data involved. • The constraints that directed us towards Python. • Using Pytest and Docker to test locally, without data! • The overall architecture of our solution including SQL server, Azure Synapse Analytics and Data Factory. • Simplifying SQL using custom dependencies. • The benefits of iteration!