The Future of Data Integrity

Compliant Data Lakes IV

July 02, 2020

The Digital World needs Proof Pools

Last week the focus was on exploring the notion of separate repositories of digital proofs for data lakes or proof pools. This week we want to expand on real-life applications for such repositories.

Today there are several applications that would benefit immensely from such a repository. Most if not all companies today have huge issues with having compliant eCRM systems, often being overwhelmed by the rapid changes and growing amounts of data. Having an easy-to-integrate and fully customizable storage solution for digital proofs can give you easy access to compliant verification of data for internal processes or external audits. Considering the trend that most data will be stored in geographically distributed cloud storage servers like Amazons AWS or Microsoft Azure to name some.

However, if you prefer having digital proofs stored behind your firewall, that is possible and, in some cases, desirable. In fact, a proof-pool may be most useful if it is parallel to the data lake in a different place acting as both, a risk-free area of interaction while fulfilling its primary objective to store and supply digital proofs. Doing so would to some extent address issues of data localization and support compliance efforts.

M2M Communication

A trend that indicates a change towards distributed systems and processes envisioned for Web 3.0 is the increase in M2M communication. This can be seen especially in industrial applications and robotics but is encroaching more and more into everyday life. Most phones communicate even when on ‘standby’ and enormous companies like Amazon utilize M2M for their automated package facilities.

Regarding the immense degree of autonomy of machines that M2M is deeply interwoven with, data integrity becomes even more important. You have to make sure that these processes, the data and communication can be accounted for and that machines are enabled to verify the integrity of received and to-be-processed data. Having a portable digital proof for such data, would enable each machine to automatically verify data and eliminate data errors from impacting processes. This may be of great importance for heavy dangerous machinery, processing of highly volatile materials and the construction of complex buildings to name a few.


Insurance is another market where the amount of data has not only evolved existing businesses but also created a spore of new strategies. In the case of existing insurance types, the constant creation of sensor data has led to smart cars feeding data into the evaluation of claims and may also be used in court as evidence. Having data verifiable would ensure that automate claim processing via smart contract is only fed verified information and that in case of any claim
disputes data integrity cannot be questioned. Having a proof pool would further enable independent qualified searches for specific digital proofs by owner approved third parties.

A completely new idea can be found in the CAPEX to OPEX trend where sophisticated AI or machine-learning will accurately predict the lifecycles of important machines. The predictions optimize operational output and minimizing operational costs. Hence, enabling insurances to guarantee the productiveness due to them being sure that the data they base their analytics on cannot be tampered with. Again, you see in this example the necessity for trustworthy data to make it a feasible business at all.

Supply Chains

In the case of supply chains, you will quickly discover how many parties are involved in the same process. To enable different parties, often in a global chain, data is required for optimization, quality assurance, accountability and compliance. Solutions like track and trace revolutionized the industry but created the challenge of not only handling great amounts of data but also ensuring that it can be trusted. This further reduces the margin of error, ensures accountability and establishes transparency. Having an easily accessible repository for digital proofs enables easy gapless verifiable provenance and a trusted data exchange. Cold chains in particular and perishable goods like meat or essential pharmaceuticals products required to be kept at certain temperatures, would benefit immensely from digital proofs and the resulting data integrity.

After examining the usefulness of a repository of proofs we found several industries that could benefit from it today. Futher, M2M communication – which will inevitible be in almost every process – may be the biggest beneficiary. The discussion around this topic has however only been started and needs further expansion.

Read the entire paper here.