Hildebrand Solves a Key Problem in Smart Metering Research

Hildebrand, the key technology partner of the DEHEMS project has been working with IBM address the issues of traditional databases dealing with time-series data  (i.e. ‘pulses’ of data arriving at regular intervals from one or more sources), because their structure makes it difficult to store and index this data efficiently. As a result, Hildebrand knew that it would be impossible to create a solution around a standard database engine without massive investment in high-end server hardware. The costs of this would be prohibitive, so a new kind of solution was required.  Following research in to leading-edge database platforms Hildebrand looked at the IBM Informix TimeSeries DataBlade and Real-Time Loader, which are specifically designed to handle time-series data.   Working with the software lab at IBM Hursley and they helped run a number of proof-of-concept projects using the facilities in the IBM Hursley Innovation Centre to see if it would be possible to handle the volume of data needed in a cost-effective manner.

The Informix TimeSeries technologies create a single database object for each data-source, and then simply update it with the latest readings whenever a new ‘pulse’ of data arrives. This provides a more manageable data structure, which makes it easier to store, extract and analyse data.

With help from the IBM Hursley team, it was quickly found that Informix TimeSeries could deliver spectacular results. In the first proof-of-concept three million homes were stimulated sending readings once a minute, and was able to capture nearly 40,000 readings per second using only a quad-core, dual-processor Intel server. In the second, using a slightly larger server and it was found it could deliver analytics response times of between one and three seconds for a load of 50,000 readings per second. Without understanding the technical details, energy monitoring for three million homes or more became a practical proposition.

Further details are available in the IBM press release.