Humaans / Case study
Reliable audit logging
Designing understandable audit data and leading the decision to use a transactional outbox so application changes and audit events stayed connected.
OutcomeThe production outbox buffered pending events during a BigQuery outage, separating audit delivery from BigQuery availability.
Two writes, one consistency problem
I joined the audit-log project after BigQuery had been selected. I contributed to the data structure design, focusing on an understandable model that would be straightforward to extend.
Calling BigQuery during an application request created a consistency risk: the audit event could be written but the application transaction could fail, or the application change could succeed without its audit event reaching BigQuery.
Make the intent part of the transaction
I led the decision to use an outbox. We wrote the pending audit event in the same database transaction as the application change, then delivered it to BigQuery separately. This tied the change and the intent to record it together, with eventual consistency between the application database and the audit store.
Proven during an outage
The design also allowed pending events to accumulate when BigQuery was unavailable, instead of making its availability part of the application request. A BigQuery outage in production demonstrated that buffering behaviour.