Data retention and archiving for growing businesses

A business asks how much storage it needs next year. The honest answer is that nobody knows, because nothing has ever been deleted. Every database row, every backup set, every call recording, every mailbox belonging to someone who left four years ago is still there, and the only reason is that keeping it required no decision while deleting it would have.

The cost shows up in three places. Storage grows on a curve nobody planned. Every additional record is another record an attacker can take, so the breach you eventually suffer is larger than it needed to be. And in several jurisdictions, keeping personal data with no defensible reason is not merely untidy — it is a regulatory failure, because the obligation runs the other way.

Retention is the discipline of deciding, in advance and in writing, how long each kind of data is kept and what happens at the end of that period. It is unglamorous, it is cheap to establish, and it is very expensive to retrofit across an estate that has been accumulating for a decade.

Keeping everything is a decision, not a default

It is worth naming the asymmetry that produces the problem, because it is organisational rather than technical. Deleting data is an action somebody has to take, justify, and be accountable for if it turns out to have been needed. Keeping data is what happens when nobody does anything. The incentives point one way, and they point that way in every business regardless of size.

That would be harmless if data were free to hold. It is not. Each additional year of retained records adds storage cost, adds backup volume, lengthens restore times, and enlarges the set of information that a single compromise exposes. The last of these is the one that changes the character of an incident: the difference between a breach involving twelve months of customer records and one involving eleven years is not a difference of degree.

Where personal data is involved, the law is explicit that indefinite retention is not the neutral option. GDPR Article 5 requires that personal data be adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed, and kept in a form which permits identification of data subjects for no longer than is necessary for the purposes for which the personal data are processed. Read plainly, that makes holding data without a current purpose a breach of the principle rather than a filing habit.

Must keep, may keep, must delete

Every category of data sits somewhere on a spectrum with two hard edges, and the useful work is establishing where those edges are before arguing about the middle.

The middle row is the one that surprises people. There is a widespread assumption that retention law only sets minimums, and that keeping data longer is always safe. For financial records that is broadly true. For personal data it is not, and the two categories are usually stored in the same systems.

Building a schedule that survives contact with reality

A retention policy that lists every data type in the business will not be maintained. One that covers the categories carrying the most volume and the most risk will be. Start narrow and extend.

1. 1

   Inventory by category, not by system

   Customer records, employee records, financial records, correspondence, system logs, backups, call recordings. The same category usually lives in several systems, and the retention rule follows the category rather than the storage location.

2. 2

   Establish the floor for each category

   The minimum a law, regulator or contract requires. This is research, and it is jurisdiction-specific — a business delivering internationally may face different floors for the same category in different markets.

3. 3

   Establish the ceiling for each category

   The point beyond which there is no purpose. For personal data this is the harder question and the one that carries regulatory weight. Where no purpose can be articulated, the ceiling has already passed.

4. 4

   Decide what happens at the end

   Delete, anonymise, or archive. These are genuinely different outcomes with different costs and different residual risks, covered below.

5. 5

   Name an owner per category

   A category with no owner will not be reviewed. The owner is the person who can answer why the period is what it is, which is the question an auditor will ask.

6. 6

   Automate the expiry

   A retention period enforced by somebody remembering is a retention period that does not exist. Where a system supports lifecycle rules, configure them; where it does not, the deletion job is a scheduled task with monitoring like any other.

The last step is where most policies fail. Businesses commonly produce a well-researched schedule, publish it, and change nothing operationally — so three years later the schedule says twenty-four months and the database holds everything since inception. An unenforced retention policy is worse than none, because it documents a standard the business is visibly not meeting.

Archiving, backup and deletion are three different things

These are used interchangeably in conversation and behave entirely differently, which produces a specific and common failure: a business deletes data from production, considers the retention obligation met, and continues to hold the same data in backups for another five years.

Archiving is the mechanism most businesses under-use. Data that must be retained for a regulatory period but has no operational purpose does not belong in the production database, where it inflates every query, every backup and every breach. Moving it to a restricted archive keeps the obligation met while removing it from daily exposure — and it is usually cheaper storage as well.

The backup problem

Backups are the reason deletion is harder than it looks. If retention says twenty-four months and backups are held for five years, the effective retention is five years, and a subject access request or a deletion request is answered incorrectly if the backups are ignored.

The pragmatic position taken by most regulators is that data in backups may persist through the normal backup cycle provided it is not restored into active use and is removed when that cycle turns over. That is workable, and it requires two things most businesses have not established: a backup retention period that is defined rather than accidental, and a documented process ensuring that a restore does not silently reintroduce records that were deleted deliberately.

Deletion has to mean something

Marking a row as deleted, moving a file to a recycle location, or removing a pointer while leaving the data on disk are all common and none of them is deletion. Where the data is sensitive and the medium is being disposed of or repurposed, the relevant guidance is NIST SP 800-88 Revision 2, Guidelines for Media Sanitization, finalised in September 2025 and superseding the 2014 revision. It describes media sanitization as a process that renders access to target data on the media infeasible for a given level of effort, and frames the choice of technique around the sensitivity of the information rather than applying one method universally.

For most business data the practical hierarchy is straightforward. Logical deletion within an application is sufficient where the underlying storage remains under your control and encrypted. Cryptographic erasure — destroying the key rather than the ciphertext — is effective and fast where data was encrypted at rest with a key you can destroy. Physical destruction is for media leaving your control that held sensitive data and cannot be reliably sanitised.

Where retention meets the rest of the estate

Retention is not a database concern. It applies to every system that holds a copy, and the copies are usually in more places than the policy anticipates.

• Call recordings, which accumulate quickly and frequently contain payment or health information spoken aloud. Covered further in the call centre article.
• Mailboxes and shared drives, which typically have no retention rule at all and hold the largest volume of unclassified personal data in most businesses.
• Log and monitoring data, where the useful investigative window is weeks but default retention is often indefinite.
• Analytics and reporting stores, which are frequently built by copying production data and then forgotten.
• Automation platforms, which retain execution histories containing whatever passed through them.
• Departed employees' accounts and personal drives, which are usually disabled rather than dispositioned.

The last two are the ones found during audits rather than during planning. An automation platform's execution log can contain full customer records, and it is almost never included in a retention schedule because nobody thinks of it as a data store.

Common mistakes

A representative scenario

A composite of situations we see repeatedly; no client detail is included.

A services business is asked by a prospective enterprise client to describe its retention policy as part of procurement. There is a policy document. It states that customer data is held for the duration of the relationship plus twenty-four months.

The review found that no system enforced it. The production database held records from inception. Backups were retained indefinitely because the retention setting had never been changed from the default. Call recordings occupied more storage than the application itself and had no policy at all. Mailboxes of departed staff were disabled but retained in full, including several years of correspondence containing client personal data.

The remediation was sequenced by exposure rather than by volume. Categories were separated so that records carrying a statutory floor could be identified and the rest made expirable. Lifecycle rules were configured where the platform supported them and scheduled jobs written where it did not, both monitored. Call recordings were given a defined period with payment-related segments excluded from capture at source. Backup retention was set deliberately and documented as part of the effective period. Departed-staff mailboxes were dispositioned rather than left disabled.

The policy document barely changed. What changed was that the periods it described started to happen.

Implementation checklist

• Data is inventoried by category rather than by system, and each category has a named owner.
• Each category has a documented minimum retention derived from law, regulation or contract, with the source recorded.
• Each category has a documented maximum retention derived from purpose, and personal data with no articulable purpose has been identified.
• The end-of-period action is defined per category: delete, anonymise, or move to restricted archive.
• Expiry is automated through lifecycle rules or monitored scheduled jobs, not left to memory.
• Backup retention is defined deliberately and accounted for in the effective retention period.
• A documented process prevents a restore from silently reintroducing deliberately deleted records.
• Anything described as anonymised has been tested for re-identification against the fields actually retained.
• Call recordings, mailboxes, shared drives, logs, analytics stores and automation execution histories are all in scope.
• Departed employees' data is dispositioned, not merely made inaccessible.
• The schedule is reviewed on a fixed interval, and the review is recorded.

Frequently asked questions

Conclusion

Retention is one of the few areas where doing nothing has a compounding cost and doing something is cheap. The work is not technical: it is deciding, per category, how long the business must keep data and how long it may, then making a system enforce the answer rather than a document describe it.

The test of whether a retention policy is real is simple. Pick a category, look at the oldest record in production, and check it against the period the policy claims. In most businesses that check fails the first time it is run, and it fails quietly, which is exactly why it is worth running deliberately.