Vasileios Mourtakos - A data engineering framework in Python скачать в хорошем качестве

Vasileios Mourtakos - A data engineering framework in Python

Abstract ======= Prioritizing speed over scalability and robustness can lead to maintenance issues and difficulties when updating tasks. To prevent these issues, it is recommended to use a data standardization framework that includes commonly used tools in a scalable and adaptable form. This framework can be custom-made for an organization's needs and can include various features such as authorization pipelines, SQL queries, file handling, and setups for caching, logging, testing, and more. In this presentation, a comparison between the utilization of a custom data standardization framework to a quick, but potentially problematic solution will be demonstrated. Speaker ======= Vasileios Mourtakos Vasileios is Junior Data Analyst at refurbed and PhD student at NTUA. www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.