Casting a light on Aotearoa’s darkest assets
Custom Analytics + Software Solutions | Data Quality | Data Visualisation + Reporting Solutions
Improving the transparency of Three Waters asset data quality across New Zealand.
The Building Innovation Partnership (BIP) is a research programme based at the Universities of Canterbury and Auckland. BIP initially began working with the Waikato-based Road Asset Technical Accord (RATA) to develop a tool that assesses the pipe asset data quality across councils in the Waikato region.
The Challenge
Data is essential for assessing the condition of Three Waters (wastewater, drinking water and stormwater) pipe networks, planning renewals and scheduling maintenance. Traditionally, pipe asset data is often inconsistent and sometimes even non-existent. This has made it difficult for councils and utilities to understand the limitations of the data they are using, as they depend upon it to make significant investment decisions that affect people’s lives. Harmonic was tasked with developing a solution to assess data quality to provide councils with an overview of which areas require data improvement.
The Solution
Harmonic developed a data quality assessment framework and accompanying tool, using BIP’s Code of Practice for Three Waters asset data as a central schema of asset data.
The data quality assessment framework provides a methodology for assessing, comparing and communicating data quality by:
- Implementing consistent checks to assess the quality of data.
- Creating metrics, such as accuracy and completeness, to measure and compare the quality of various data assets.
- Using various visualisation techniques to communicate data quality.
Harmonic then used this framework and the Code of Practice to develop a Data Quality Assessment tool. This included developing an R package to define the various tools used to carry out data quality checks and calculate metric scores. Harmonic then built a dashboard in Shiny to visualise the data quality results.
Since working with the RATA councils, several new councils, including the Wellington region, Christchurch and Southland, have used the data quality assessment tool. This provides more comparative information upon which councils can benchmark their relative data quality performance and make improvements.
The Result
- Gained insights into the quality of individual data assets through a data quality assessment framework.
- An R Package that runs cross-asset checks to identify the compatibility of different assets types.
- Effective visualisation and communication of data quality and areas for improvement through an interactive dashboard.
- Comparing data quality performance across other councils.