From Catchment to Consumer: Visualising a Water Network
Custom Analytics + Software Solutions | Data Visualisation + Reporting Solutions
HydroBalance is an interactive visualisation tool that traces water balance from source to customer, facilitating the exploration of areas to target for leakage reduction efforts, efficiency improvements, and a better understanding of the path of water through the network.
Harmonic was selected as the Data Science squad within the Waters Services Reform National Transition Unit (NTU). During this time, the squad developed a suite of data science solutions centred around the identification and reduction of water leakage. These solutions are now available for the benefit of all councils.
The Challenge
The water industry faces a range of challenges in its goal of supplying clean drinking water to customers. This includes being able to supply sufficient water year-round, in the face of potential droughts and leakages.
The Water Services NTU wanted a tool to help visually understand where leakages were occurring. There was also interest in representing the path of supplied water from dams and rivers, through various steps of water processing and distribution, and finally to customers in homes and businesses. This could both help as a leakage-targeting tool and as an onboarding tool to help new staff understand the complexities of the water treatment and distribution network.
Prior to building the tool, our first challenge was to clean and collate the data. The required data was available for the Auckland water network, but was spread across a range of different spreadsheets, maintained by industry experts.
The Solution
Harmonic data scientists wrote data processing code to validate, transform, and load the data from the numerous spreadsheets into Snowflake tables. This provided a trustworthy and standardised source of flow data spanning two and a half years. After data processing, the HydroBalance solution is agnostic to the specific water network structure, designed to be easily applied to other city networks in the future.
The HydroBalance solution represents network flows using a Sankey chart, a powerful visualisation tool for showing the path of flows of any quantitative value from source to destination.
Within the Auckland water network, the flows travel from water supply sources (dams and rivers), via water treatment plants, bulk supply facilities and regional supply, to customers (domestic and non-domestic). At each stage, unaccounted water flows are attributed to leakages. Furthermore, quantities representing losses at treatment plants are calculated using information provided by domain experts.
The Sankey chart has several interactive features to enhance the user experience:
- Date selector to be able to select flows from a single month or aggregate over a range of months.
- Hover data for each flow in the Sankey, providing the ability to read the exact amount in cubic metres.
- Flow summaries showing the volume at each network stage and the percentage of total flow.
In addition to this interactive Sankey chart, the HydroBalance tool allows for a deeper dive into understanding non-revenue water. A second page in the dashboard uses graphics to show trends of non-revenue water quantities over time and includes a table to look up specific network locations and dates.
The Result
- Implements a cohesive data wrangling process and dashboard user interface in Python.
- Reflects volumetric flows in the water network spanning several years, with data as recent as three months prior.
- Facilitates understanding of how the distribution of water is divided at various stages, and how much is lost at each stage. This can help leaders target key areas where losses can be reduced or identify utilities that are being over or under utilised.