Wednesday, January 31, 2024

Digital Disinheritance? A response to proposals to digitise, then destroy, the nation’s wills

This blog was never intended to become a hot-bed of political activism, but for the second time in a year I find myself calling on sympathetic readers for action against a proposed act of destruction — not of railway ticket offices this time, but of the national archive of wills.

The Ministry of Justice has recently announced proposals to reform the procedures, and indeed the law, governing the archival storage of wills in England and Wales.  The main intention is to reduce the costs of this storage, which have been estimated at £4.5 million annually.  However, the current plan — to dispose of the originals once a digitisation programme has been completed — has caused many archivists and researchers considerable alarm.

The current procedure is as follows.  Every will notarised since 1858 is, by law, retained permanently in paper form (at present at a repository in Birmingham).  Since 2021 it has also been procedure to make digital copies of new wills as they are accessioned (i.e. incorporated into the archive), which enables speedier delivery should they need to be seen in court or by members of the public.  But that digital version is still understood as a surrogate for the original paper document, not as the document itself.

The new proposals (set out in detail here) would bring about a drastic change to this arrangement.  Firstly, a massive digitisation project would be undertaken, making digital copies of every document in the collection: some ten million wills.  Then — and here the trouble starts  the originals, or at least those more than twenty-five years old, would be destroyed, excepting only those deemed to belong to ‘notable individuals’ or to be otherwise of ‘significant historical interest’.  The twenty-five-year cut-off point would thereafter become a standard, rolling retention period, after which hard copies would be destroyed.  The fundamental character of the archive would be transformed: the digital image, not the paper original, would be regarded as the master copy of any given will.  Ultimately, it would become the sole copy.

“What is the problem with this?” one might reasonably ask.  After all, our whole lives are being digitised as it is.  If digital files would be easier to access and, according to the consultation documents, cheaper to store, why not ‘move with the times’ , as the Minister for Justice Mike Freer puts it, and make the switch from paper to digital records?

One way to answer this is to consider the two imperatives that hover like a pair of angels over the shoulders of every archivist: Preservation and Access.  Our twofold task is both the permanent preservation of records and the provision of access to them when required.  Although the two principles sometimes contradict each other — handling causes damage, and damage can prevent handling — both principles are vital, including to each other: each is meaningless without the other.  It seems little use to preserve documents if nobody will ever be permitted to read them, nor is it any good providing access to them if they have been allowed to decay into illegibility.

This means that, for archivists, ‘moving with the times’ requires careful thought.  Simply discarding old practices in favour of the new will never do: we have to work out how the arrangements we make will serve the Twin Angels in the long term.  This does not make us Luddites — part of any archivist’ s job is to respond nimbly to the challenges of record-keeping in an age of break-neck technological change — rather, we need to make intelligent and imaginative assessments of what is likeliest to prove worthwhile in the long term.  Archivists understand that technology, like anything else, should be assessed by its merits, rather than by its novelty.

So, to return to the question of the wills, it is not the digitisation itself that is the problem.  No archivist would object to a properly-organised programme of this kind.  Anything that makes the records more easily accessible is, in principle, to be welcomed.  (Indeed, the use of digital surrogates also tends to help to preserve the originals by reducing the risk of damage from handling.)  The problems arise with the decision to dispose of the paper record, and to rely on the digital image alone as the permanent record.

Digital information is generally very easy to access, but its preservation in the long term is a much more uncertain affair.  Anyone who has wondered how to retrieve information from an old floppy disc will have an idea of the sort of problems that can arise, as will anyone who reads this article once the consultation has ended and the hyperlinks have expired.  Digital preservation as a field is, in spite of the extensive work of organisations such as the Digital Preservation Coalition, still relatively new, and in any case, these experts and organisations make it very clear that digital preservation is even more complex, demands demands even more expertise, requires even greater vigilance and maintenance, and is generally even more intensive than care for traditional paper archives.  In the right conditions, paper records can last for centuries, as these very wills prove, but no digital file has yet had the chance to be preserved for longer than about three or four decades: it is a bold claim to declare that any given digital file can be relied upon as a ‘permanent’ record.  

As it is, we have found by experience that there are innumerable threats to digital records — more, in fact, than to paper records.  They are vulnerable to obsolescence of hardware and software, to the physical deterioration of complex and finely-tuned equipment, to accidental deletion, to corruption, to tampering and to cyber-crime.  (Only last October a cyber-attack on the British Library put its systems, including its main catalogue and digital repository, out of action for three months; recovery from is expected to take until June 2025 and entail significant costs).  It is by no means certain — or, at the very least, the consultation documentation provides no hard evidence — that the cost of protecting the data from these threats will necessarily be lower than the cost of retaining the paper records.  And the stakes are incredibly high.  The ultimate cost, of course, would be the total loss of all the records, a scenario which most archivists would agree is likelier in the case of a digital archive than a physical one.  

My view, in summary, is that although the digitisation project is certainly a constructive and beneficial idea, the resulting digital files should never be regarded as the master copies, as it is far harder to ensure the preservation of digital files than of paper records, and the risk of corruption or total loss is far higher.  Above all, the originals should under no circumstances be destroyed.  Other methods of reducing costs, such as more efficient, shared, or more centralised storage, ought to be pursued instead.

In so far as the new proposals improve access to the wills, they are entirely to be welcomed.  But in so far as they pose a risk to their preservation, they should be revised or rejected.  Anything that threatens the principle of preservation also threatens the principle of access: records cannot very well be accessed if they corrupted, damaged or destroyed.  For this reason, while welcoming the idea of a digitisation project, I believe very strongly that the originals must be preserved permanently, so as to pass on to future generations an irreplaceable legacy.

---

The Government has said that it ‘welcomes views from court users, the legal and archivist professions, all other probate practitioners and historians, as well as the judiciary and anyone else with an interest in this topic.’   If any readers feel inclined to respond to the consultation, I would urge you to do so by writing to:

Will Storage Consultation,
Ministry of Justice,
Civil Justice and Law Division,
Postpoint 5.25,
102, Petty France,
London – SW1H 9AJ.

— or emailing civil_justice_poli@justice.gov.uk — by the deadline of 23rd February 2024.

Here are my answers to the consultation’s ten questions:

Question 1: Should the current law providing for the inspection of wills be preserved?

From the point of view of an archivist, I have no reason to object to the law as it currently stands.

Question 2: Are there any reforms you would suggest to the current law enabling wills to be inspected?

No, I have no particular suggestions to make in this regard.

Question 3: Are there any reasons why the High Court should store original paper will documents on a permanent basis, as opposed to just retaining a digitised copy of that material?

Yes, in my view (as a qualified archivist) there are several significant reasons why the original paper records should be preserved, instead of being entirely substituted by a digital copy.  These reasons arise mainly from the difficulties and complexity in preserving the integrity and authenticity of digital records in the long term.

I have no objection to the proposed digitisation programme; indeed, I would regard a properly-organised project of this kind as an entirely beneficial development.  Anything that makes archival records more easily accessible is, in principle, to be welcomed.  It is the idea of destroying the originals that is the problem.  The consultation document gives almost no consideration to the question of digital preservation, i.e. of how the permanent preservation of the digital copies of the original wills can be ensured.  The considerable work of experts in this field — notably of the Digital Preservation Coalition (https://www.dpconline.org/) — indicates that in spite of significant advancements, digital preservation is extremely complex, demands technological as well as archival expertise, depends upon a particularly high level of vigilance and maintenance, and is overall even more intensive than care for traditional paper archives.   It is therefore generally acknowledged in the archival profession that the long-term storage of digital files is more difficult, and carries more uncertainty and risks, than that of analogue files.  

There are several reasons for this.  Firstly, a digital file in its inert state is not ‘humanly readable’: it cannot be deciphered without technological assistance.  Even if it can be read, it must also invariably be decoded, a process which depends on a particular combination of hardware and software.  The present rapid evolution of both of these categories is well known, as are the consequences of the resulting obsolescence: it is now very difficult to find a machine capable of reading the once-ubiquitous floppy disc, for instance.  Unlike a paper document, a digital document must effectively be rewritten every time it is opened, and in order for this to remain possible over the long term, both hardware and software must be kept continually up to date and digital files migrated from old to new storage.  This is a process which requires vigilance, expertise, expense and an unblemished success rate.  Where paper records are concerned, neglect may run the risk of degradation and destruction, but with digital files, it will inevitably result in total loss..

Another main area of concern concerns security and authenticity.  Firstly, it is simultaneously easier to tamper with digital records than with paper files, and harder to prove their authenticity — an issue which, where wills are concerned, has particular pertinence.  Secondly, there is the question of cyber-crime, a considerable and growing threat.  At the time of writing, the British Library continues to suffer the effects of a serious cyber-attack, a security breach which put the majority of its collections out of public reach for three months, and recovery from which is expected to take at least eighteen months.  It is not hard to imagine the severity of the consequences, let alone the financial cost, of any such attack on a digital repository of wills.  

Question 4: Do you agree that after a certain time original paper documents (from 1858 onwards) may be destroyed (other than for famous individuals)? Are there any alternatives, involving the public or private sector, you can suggest to their being destroyed?

I disagree very strongly with this proposal.  Under no circumstances should the originals be destroyed.

The costs of the retention of physical copies are appreciable, but in my view they can be justified by the longevity assured by the physicality of those copies.  The costs of digital preservation — which are not assessed in the consultation document — will scarcely be less; indeed, they are likely to be higher.  To rely on digital versions as the permanent copies would ultimately prove a false economy.

The existing costs could be alleviated by letting out unused capacity to other government departments or third parties, provided that appropriate measures are taken to isolate personal data in the wills; alternatively, a broader centralisation of record storage, perhaps involving collaboration with other institutions and organisations, might be explored.  The National Archives’ ‘DeepStore’ facility in former salt-mines at (or underneath) Winsford in Cheshire has emerged as one solution to the problems of storing large volumes of historic records.  More generally, the National Archives and Archives and Records Association are also excellent sources of professional advice. 

Question 5: Do you agree that there is equivalence between paper and digital copies of wills so that the ECA 2000 can be used?

From an archival rather than a legal point of view, any document whose integrity and authenticity can be proven may be regarded as an authoritative record, regardless of its format.  However, in the case of digital files, proving authenticity is far more complex than with paper.  Since it is easier to tamper with digital files than with analogue records, and since such tampering is harder to detect, digital files require additional safeguards such as encryption or checksums.  The Digital Preservation Coalition is able to offer extensive advice in this field.

Question 6: Are there any other matters directly related to the retention of digital or paper wills that are not covered by the proposed exercise of the powers in the ECA 2000 that you consider are necessary?

Not to my knowledge or within my particular area of expertise.

Question 7: If the Government pursues preserving permanently only a digital copy of a will document, should it seek to reform the primary legislation by introducing a Bill or do so under the ECA 2000?

Not to my knowledge or within my particular area of expertise.

Question 8: If the Government moves to digital only copies of original will documents, what do you think the retention period for the original paper wills should be? Please give reasons and state what you believe the minimum retention period should be and whether you consider the Government’s suggestion of 25 years to be reasonable.

Since I do not believe that the original wills should be destroyed at all, I do not think there should be a limited retention period of this kind for these documents.  I might make the point that no digital document has yet had the chance to be preserved for longer than about thirty years, so even twenty-five years seems to me an extremely ambitious figure.

Question 9: Do you agree with the principle that wills of famous people should be preserved in the original paper form for historic interest?

I agree that the instinct to preserve wills ‘in the original paper form for historic interest’ is sound, and understand the reluctance to dispose of the wills of prominent figures.  It is worth interrogating the reasons why this element of the plan has been included.  To make a provocative point, though constructively intended, if the digital copies are thought to be sufficient, why is any need felt to preserve even the physical wills of well-known people?  Is it because we do, in fact, sense the relatively high risk of loss?  If so, are we really confident that the digital records will last in perpetuity, or is there actually something important, something reassuringly and reliably durable and tangible, about the hard copy?  

I think this element of the proposals reflects a deeper instinct to preserve all the wills physically, an instinct which I think is sound and should be followed.

Question 10: Do you have any initial suggestions on the criteria which should be adopted for identifying famous/historic figures whose original paper will document should be preserved permanently?

Only that any adequate criteria would be so complex, and no doubt controversial, as to be scarcely justifiable.  As far as historical research is concerned, the past is full of figures who died obscure and penniless but later came to be regarded as significant; there are just as many who enjoyed great fame when living but are now forgotten.  In any case, to some researchers it may be the wills of ordinary people which come to be of greatest interest. 

Fundamentally, however, I believe that it is of paramount importance that the original wills should be preserved in their entirety, regardless of the existence of any digital copies.  I would urge all involved with these plans to reconsider the proposal to destroy them, and to maintain the current policy of permanent preservation.

Update 16th February, 2024: The Archives and Records Association, the professional body for archivists and records managers in the United Kingdom and Ireland, has just issued its own response to these proposals.