Designing a 7-Year Audit Trail That Won't Bankrupt You (or Fail Compliance)

Overview

For regulated organisations, particularly those navigating the stringent requirements of crypto-asset platforms, audit logs are the incontrovertible proof of who did what, and when. Thanks to mandates like MiCA Article 70 and CDR 2025/305, you are now required to retain a tamper-proof audit trail for a minimum of seven years. However, merely hoarding petabytes of log data and leaving it to rot in a digital attic is not a strategy. The solution needs to be testable, repeatable, and cost-effective across a near-decade lifecycle.

We recently worked with a crypto-asset platform facing this exact reality. We needed to develop a centralised approach to collecting and retaining audit records, managed entirely through Terraform and validated through separate development and production environments.

---

The Challenge of Indefinite Storage

The client’s existing logging environment provided decent visibility for engineers, but retention policies varied wildly across different Google Cloud Projects. Demonstrating to an auditor that specific records would unequivocally be available in 2033 was — to put it mildly — an exercise in blind optimism. When a regulator comes knocking for specific compliance evidence, you don’t want a “four candles versus fork handles” situation where you are handing over application traces instead of the required tamper-proof access logs.

Furthermore, storing vast quantities of audit data indefinitely introduces a rather unappealing financial burden. For a regulated entity, paying to store data you no longer legally require is akin to leaving the heating on while you go on holiday for a month. We needed an approach that could:

• Provide a centralised and highly reliable audit trail.
• Lock down and protect those records for exactly seven years.
• Prevent premature deletion or modification.
• Automatically purge the data the moment the regulatory window closes.
• Keep long-term storage costs under strict control.
• Be managed entirely as infrastructure as code.
• Be tested safely before being unleashed on the production environment.

The Solution: Centralised, Automated, and Eventually Ephemeral

We built a centralised audit logging architecture across Google Cloud, routing all relevant logs into a dedicated compliance storage bucket.

Because engineers do not typically read five-year-old audit logs over their morning coffee – unless they have very peculiar hobbies – the bucket utilises Archive storage. We implemented strict retention policies ensuring that absolutely no records can be deleted or modified before the seven-year mark is reached.

Crucially, the configuration also includes automated deletion policies. Once the seven-year requirement has been fully satisfied, the logs are immediately destroyed rather than being kept indefinitely. This ensures the platform only pays for the storage it legally requires, ruthlessly trimming long-term operational costs.

 

Here is the Terraform configuration used for the organisation-wide audit logging, sink and retention bucket:

 resource "google_organization_iam_audit_config" "all_services" {
org_id = local.org_id
service = "allServices"
audit_log_config { log_type = "DATA_READ" }
audit_log_config { log_type = "DATA_WRITE" }
audit_log_config { log_type = "ADMIN_READ" }
}

resource "google_logging_organization_exclusion" "data_access" {
name = "exclude-data-access-global"
org_id = local.org_id
description = "Drop Data Access logs from _Default sinks to avoid duplication"
filter = "log_id(\"cloudaudit.googleapis.com/data_access\")"
}

resource "google_logging_organization_sink" "org_logs" {
name = "org-audit-logs"
org_id = local.org_id
destination = "storage.googleapis.com/${google_storage_bucket.org_audit_logs.name}"
filter = "log_id(\"cloudaudit.googleapis.com/activity\") OR log_id(\"cloudaudit.googleapis.com/data_access\") OR log_id(\"cloudaudit.googleapis.com/system_event\")"
include_children = true
depends_on = [google_organization_iam_member.logging_manager]
}

resource "google_storage_bucket" "org_audit_logs" {
project = data.google_project.production_audit_logs.id
location = "EU"
name = "${data.google_project.production_audit_logs.id}-org-sink"
storage_class = "ARCHIVE"
retention_policy {
# 7 years in seconds (7 * 365.25 * 24 * 60 * 60)
retention_period = 220903200
is_locked = false
}
lifecycle_rule {
action {
type = "Delete"
}
condition {
age = 2556 # 7 years in days (365.25*7 rounded up)
}
}
lifecycle {
prevent_destroy = true
}
}


 

Shifting Compliance down into the platform: Testing Through the SDLC

A major component of this implementation was ensuring the logging configuration could be thoroughly tested before we applied a seven-year retention lock to the production environment. A configuration mistake here usually results in either missing vital compliance records or paying for phantom storage until the end of the decade.

Rather than making changes directly to production, we created a dedicated development project and storage bucket. This provided a safe stage in the software development lifecycle to validate the exact behaviour of the infrastructure.

To prevent the development environment from generating absurd amounts of useless test data – which is a bit like testing a fire alarm by burning down the kitchen – we tightly scoped the configuration. The bucket filters logs from the development environment, restricting the query purely to relevant data access logs for the Cloud Storage API.

 

Here is the code for the dev audit log sink and bucket that is used for testing log configuration changes in development before modifying the org-wide logs:

resource "google_storage_bucket" "dev_audit_logs" {

  project       = data.google_project.development_audit_logs.id

  location      = "EU"

  name          = "dev-audit-logs-sink"

  storage_class = "Standard"

 

  retention_policy {

    # 3 days in seconds (3 * 24 * 60 * 60)

    retention_period = 259200

    is_locked       = false

  }

 

  lifecycle_rule {

    action {

      type = "Delete"

    }

 

    condition {

      age = 3 # days

    }

  }

 

  lifecycle {

    prevent_destroy = true

  }

}

 

resource "google_logging_folder_sink" "dev_logs" {

  name             = "dev-audit-logs"

  folder           = data.google_folder.dev.id

  destination      = "storage.googleapis.com/${google_storage_bucket.dev_audit_logs.name}"

 

  filter = "log_id(\"cloudaudit.googleapis.com/data_access\") AND protoPayload.serviceName=\"storage.googleapis.com\""

 

  include_children = true

} 

 

This staging capability allowed us to confidently verify that:

• The required logs were being accurately captured.
• The filtering logic behaved exactly as intended.
• Retention locks successfully blocked premature deletion.
• Deletion policies triggered perfectly upon expiration.
• The configuration could be safely promoted through development and into production.

Once the behaviour was proven, the exact same Terraform configuration was applied to production.

 

Results

The new approach provides a centralised and controlled audit trail across the organisation, with visibility into important activity and access to sensitive resources.

It also provides:

• Seven-year retention for required audit records
• Protection against premature deletion
• Automatic deletion immediately after the retention period, preventing unnecessary ongoing storage costs
• A dedicated development environment for testing logging and retention behaviour
• Controlled log filtering to keep development testing representative without generating excessive data
• A repeatable deployment process through Terraform
• A clear path for promoting tested configuration from development into production

The result is a compliance solution that considers not only the regulatory requirement, but also testability, operational reliability and cost.

The Final Conclusion: Compliance as an Engineering Asset

Long-term regulatory retention can create a significant operational and financial burden if it is approached simply as a requirement to store data for seven years.

A better approach is to design the entire lifecycle of the audit data: collect it, centralise it, protect it, test the configuration, retain it for the required period and automatically remove it when it is no longer needed.

Testing the configuration through a development stage provides confidence that filtering, retention and deletion behave correctly before the production configuration is finalised. At the same time, automatic deletion prevents organisations from paying to store data beyond the period for which it is required.

For regulated organisations, this combination of compliance, testability and cost management is an important part of building sustainable cloud infrastructure.

 

---

At Mesoform, we help organisations build secure and compliant cloud platforms where regulatory requirements are incorporated into the infrastructure and its lifecycle from the outset. For regulated entities, this combination of compliance, testability, and cost management is the foundation of sustainable cloud engineering.

Book a 15-minute technical audit of your compliance strategy today