Dirty Data Part 2: A framework for measuring data quality over time скачать в хорошем качестве

Dirty Data Part 2: A framework for measuring data quality over time

NYC Open Data Week 2025 (March 27) Part II of a presentation on data quality presented by David Tussey, formerly of the NYC Department of Information Technology and Telecommunications and Jun Yan, professor of statistics from the University of Connecticut. Part I focused on how to undertake a data cleansing effort. This presentation focuses on measuring data quality over time. In this presentation we will examine the challenges of measuring data quality over time; essentially attempting to answer the question “Is my data getting better or worse?”. We discuss some data quality measures, how to capture and visualize these over time, and how to detect deviations in data quality using a process known as statistical process control. We will demonstrate the framework we propose with real-time demonstration of data quality scripts executed against the 311 SR dataset. At the end, we hope to illustrate both issues and a potential solution for measuring the changes in data quality over time. https://2025.open-data.nyc/event/dirt... ABOUT OPEN DATA WEEK Open Data Week is an annual festival of community-driven events organized and produced by the NYC Open Data Team at the Office of Technology and Innovation (OTI), BetaNYC, and Data Through Design. Every March, New Yorkers come together online and across the five boroughs to celebrate public data in New York City and mark the anniversary of the City’s first open data law, which was signed on March 7, 2012. Coincidentally, the first weekend of March is also International Open Data Day. Throughout Open Data Week, we rally NYC’s civic tech and open data communities and offer opportunities for more people to learn about and engage with municipal public data. The 2025 Festival kicked off at the opening of Data Through Design‘s annual art exhibition. Throughout the week, participants engaged with NYC Open Data at dozens of virtual and in-person events. https://www.open-data.nyc/ We closed out the week with BetaNYC’s annual civic tech conference, NYC School of Data, which took place at CUNY School of Law in Long Island City. https://schoolofdata.nyc/ #opendatanyc #opendataweek