Data DNA key to preventing ‘Digital Memory Loss’
20 May 2010
16 of Europe’s top Libraries, Archives, Universities and Technology Institutions collaborate to map the ‘Digital Genome’ – preserving the electronic building blocks required to unlock our digital heritage.
Over the last decade the digital age has seen an explosion in the rate of data creation.Estimates from 2009 suggest that over 100 GB of data has already been created for every single individual on the planet ranging from holiday snaps to health records – that’s over 1 trillion CDs worth of data, equivalent to 24 tons of books per person!
Yet research by the European Commission co-funded Planets project, co-ordinated by the British Library, highlights deep concerns regarding the preservation of these digital assets. Findings suggest that as hardware and software are superseded by more up-to-date technology, and older formats become increasingly inaccessible, the EU alone is losing over 3 billion euros worth of digital information every year.
Looking to ensure the preservation of our digital heritage, on 18 May 2010 the Planets project will deposit a time capsule containing a record of the ‘Digital Genome’ inside Swiss Fort Knox – a high security digital storage facility hidden deep in the Swiss Alps – preserving the information and the tools to reconstruct highly valuable data long after the lifeline of supporting technology has disappeared.
Inside the Digital Time Capsule:
- Five major at risk formats - JPEGs, JAVA source code, .Mov files, websites using HTML, and PDF documents
- Versions of these files stored in archival standard formats – JPEG2000, PDFA, TIFF and MPEG4 – to prolong lifespan for as long as possible
- 2500 additional pieces of data – mapping the genetic code necessary to describe how to access these file formats in future
- Translations of the required code into multiple languages to improve chances of being able to interpret in the future
- Copies of all information stored on a complete range of storage media – from CD, DVD, USB, Blu-Ray, Floppy Disc, and Solid State Hard Drives to audio tape, microfilm and even paper print outs
Adam Farquhar, Head of Digital Library Technology at the British Library and Planets project coordinator says:
“Anyone using a relatively modern PC who has ever gone back and tried to read material stored on a floppy disc will instantly recognise the frustration of trying to access obsolete formats. Yet the death of the floppy disc is just the tip of the iceberg. Even if you possess the necessary hardware to access a particular storage format and the files haven’t become corrupt, without the supporting software and compatible operating systems, knowing what is on the disc, let alone reading the files in question will be impossible.”
Since 2007 the volume of data produced globally has risen from 281 exabytes to over 700 exabytes – much of this is now considered to be at risk from the repeated discontinuation of storage formats and supporting software. Current studies suggest that common storage formats such as CDs and DVDs have an average life expectancy of less than 20 years, yet the proprietary file formats to access content often last as little as five to seven years and desktop hardware even less. Backing up this data is a start, but without the information and tools to access and read historical digital material it is clear huge gaps will open up in our digital heritage.
To meet this threat, in 2006 the European Commission established the Planets project – Preservation and Long-term Access through Networked Services – bringing together a coalition of European libraries, archives, research organisations, and technology institutions including the Austrian National Library, the University of Technology of Vienna, and the British Library to develop the software solutions to guarantee long-term access. Marking the end of the first phase of the project the deposit of the Planets ‘Digital Genome’ in Swiss Fort Knox will help to highlight the fragility of modern data and help to protect our digital heritage from a whole range of human, environmental and technological risks.
Adam Farquhar, Head of Digital Library Technology at the British Library and Planets project coordinator says:
“Over the past century, the advance of technology has helped to double life expectancy. Yet the march of technology has resulted in the lifespan of information decreasing by 95%, from over a thousand years to fewer than twenty. The time capsule being deposited inside Swiss Fort Knox contains the digital equivalent of the genetic code of different data formats, a ‘Digital Genome’ that will enable researchers to experiment how to prolong the existence of digital data, as well as providing a blueprint by which it can be reconstructed.
“Highlighting both the physical and technological impermanence of the data we create, the deposit of the ‘Digital Genome inside Swiss Fort Knox will preserve the information and tools for potentially unlocking our digital heritage for generations to come.”
Dr. Andreas Rauber, Professor at the Institute of Software Technology and Interactive Systems of the University of Technology of Vienna says:
“Unlike hieroglyphics carved in stone or ink on parchment, digital data has a shelf life of years not millennia. As well as being more prone to physical decay, the reliance on multiple layers of compatible technology to access each file leaves vast swathes of our digital heritage at risk from hardware and software obsolescence. With technology continuing to develop at a blistering pace and the rate of data creation showing no signs of slowing down, failure to implement adequate digital preservation measures now could cost us billions in the future.”
For any media enquires and interview requests please contact Ben Sanderson in the British Library Press Office on 020 7412 7111 or via email on ben.sanderson@bl.uk. Alternatively contact Jacob Lant on 07972 148 245 or via email on Jacob.lant@bl.uk.
Notes to editors
Planets is a four-year, €15 million project, co-funded by the European Commission to build and provide access to commercial software to preserve digital content. Members are: the national libraries of Great Britain, Austria, the Netherlands Denmark, the State and University Library of Denmark; the national archives of the United Kingdom, the Netherlands and Switzerland; commercial companies: IBM, Microsoft Research, Tessella and Austrian Institute of Technology and research institutes: the Vienna University of Technology, Glasgow, Freiburg and of Cologne. Its work will be continued from June 2010 by the Open Planets Foundation: www.planets-project.eu
The Planets time capsule has been created by a team of researchers based at the Vienna University of Technology. A virtual replica will make it possible to track the fate of its contents online and experiment with resurrecting or prolonging their lifespan: www.ifs.tuwien.ac.at/dp/timecapsule/.
The British Library is the national library of the United Kingdom. It provides world class information services to the academic, business, research and scientific communities and offers unparalleled access to the world's largest and most comprehensive research collection. The British Library's collections include 150 million items from every era of written human history beginning with Chinese oracle bones dating from 300 BC, right up to the latest e-journals. Further information can be found at www.bl.uk