Formula for eradicating leaks

NEL's Alick MacGillivray looks at how data validation helps water utilities combat leakage with reliable diagnosis of pipe network failures, enabling accurately targeted maintenance

Over the last few years water companies in the UK have come under increasing pressure from regulatory bodies to improve the accuracy of metering. This has meant large investment in new plant, data control systems and general data acquisition infrastructure. One cost-effective way of increasing confidence in data accuracy is to use a technique known as data validation. This method is commonly used in the power generation and process sectors throughout Europe and the USA, and is growing in popularity in the UK water industry. It uses matrix algebra and statistics to correct measured values in such a way that the full set of measurements fulfils all of the conservation laws such as mass balances. Using this technique engineers may quickly identify which meters are reading outside their uncertainty bands and take remedial action. It may also be used to estimate the amount of water leakage in a system. NEL, the engineering services company based in East Kilbride has played a leading role in developing this technology for use in the water industry.

Data validation may be applied to many different types of plant, from simple systems consisting of only a few measurements to complicated systems with several hundred. An example of a simple pipe network is shown in Figure 1.

The technique is best implemented as a computer program, either as a stand-alone application or by integrating validation code into the existing data control system and validating the data as it is acquired. In either of these cases the implementation may be broken down into several stages:

  • system selection,

  • formulation of conservation equations,

  • pre-processing of measurements and uncertainties,

  • validation.

The first stage is to select the set of measurements which are to be validated. In a water distribution zone this may be a network of inter-connecting pipes with meters in the individual lines. The set of measurements should be chosen so that each may be determined in more than one way. Normally this will be by direct measurement and by one or more mass balances involving other dependent measurements.

The next stage is to describe fully how each measurement depends on others in the system. This involves the application of energy conservation equations to the full set of measurements. For water flow in pipe networks this is particularly simple and the process reduces to the formulation of a set of mass balances. In the example given the process yields the equations.

Here M1 can be determined in three different ways. It may be done by direct measurement, by adding measurements M2, M3 and M4 and by measurement M6. Being able to determine the value of a measurement in more than one way is called data redundancy. When redundancy exists, data validation will give more accurate values than measurement alone.

Data Validation

To perform calculations with large numbers of measurements efficiently, data validation uses matrix algebra. Apart from the conservation equations the method requires two inputs: the n measured values and their associated uncertainties. Before the validation calculations are performed these are read into two nx1 matrices X and U respectively.

Data validation improves the accuracy of measurements by modifying each by an amount determined by calculations involving the uncertainties and the conservation equations. In virtually every case the conservation equations are not strictly obeyed by the measurements. So there are discrepancies in the data that require to be rectified by the validation process. What data validation does is to find a nx1 correction factor matrix V such that:


where X satisfies the conservation equations. It does this by applying the principle of minimised error squares to:

This is similar to performing a least squares fit of a line through data points on a graph. This means that each measurement has to be altered by the amount in the

correction factor matrix before it can satisfy the conservation equations.

Data validation provides quality indices that give an indication of how accurate the original measurement was, taking into account the size of the associated uncertainty. These indices are compared with set limits to determine whether there is an anomaly with a given measurement. They may also be graphed against time to evaluate any fall-off in instrument performance.


Due to the large number of calculations involved, data validation is particularly suited to software application. The basic function of validation software is two-fold: to provide clients with more reliable and accurate data and to alert operators to instrumentation reading outside uncertainty bands. Most packages, including that developed by NEL, allow the user’s data historian to be interrogated non-intrusively. Validation may also be performed on live data, i.e. that just acquired from the plant. Mass balances describing the flows in the selected network may also be developed and stored. Results are normally written back into the data historian where database tools may be used to analyse it. The graph in Figure 3 shows a software generated comparison of measured and validated mass flowrate in a pipe network.

Leakage Detection

If, in a particular system, all meters have been checked and calibrated and their assigned uncertainties verified then you would reasonably expect the data validation technique to show no anomalies. Often, however validation still highlights large corrections to certain measurements. In a large pipe network this is most likely to be caused by leakage in the line. Using this approach it is therefore possible to estimate the size and location of major sources of leakage in the system. It should be stressed that

the common practice of performing mass balances with unvalidated data to estimate the size of losses is of

limited value and may give misleading results.

Benefits to users

So what does data validation tell us? First and foremost it may be used as a diagnostic tool to pinpoint exactly which meters are operating outside their uncertainty bands. This may indicate that operators have made incorrect assumptions about the uncertainty of the meter. This may be changed and the validation rerun with the new value. Alternatively it may mean that the meter has drifted out of calibration or that a fault has developed. Either way the ability of the data validation technique to highlight anomalies will allow operators to target maintenance at specific equipment. The data redundancy required by the method also gives the validated data greater accuracy and reliability than unchecked data.

Data validation does not introduce data that is not there already but allows operators of pipe networks to make the most of the data they have – with the accompanying financial and operational benefits

Action inspires action. Stay ahead of the curve with sustainability and energy newsletters from edie