Data Architecture Principles: Decisions and Tradeoffs

Data architecture principles are durable decision rules for designing and governing data systems. They make tradeoffs consistent and reviewable; they do not make data valuable, secure, or compliant by themselves.

Write principles as testable decisions

For each principle, record the statement, rationale, scope, owner, exceptions, controls, measurable fitness functions, and review date. For example: “Every published data product has an accountable owner, documented semantics and lineage, service expectations, access policy, retention rule, and tested recovery path.”

Design from workloads and obligations

Document operational and analytical workloads, latency, consistency, scale, sensitivity, residency, retention, recovery, interoperability, skills, and total cost. A warehouse, lakehouse, federated layer, event stream, and domain product solve different access patterns; organizational size alone does not select one.

Make meaning and authority explicit

Define entities, events, keys, units, time semantics, valid states, ownership, and versioning. “Single source of truth” should not imply one physical repository. Identify which system is authoritative for each field or event, plus reconciliation, conflict, freshness, and consumer rules.

Treat interoperability as tested behavior

Open formats such as Parquet and schema systems such as Avro can improve portability, but file syntax alone does not preserve semantics, authorization, transactions, indexes, lineage, or performance. Version contracts, test readers and writers, and rehearse export and restoration.

Build quality into interfaces

Set requirements for completeness, validity, timeliness, uniqueness, consistency, and reconciliation according to use. Publish observable service expectations, exception handling, and lineage. A centralized platform can simplify some controls while increasing concentration, cost, latency, privacy, and blast-radius risks.

Apply identity-aware security and privacy

Combine authentication, least privilege, workload identity, segmentation, encryption, key management, monitoring, retention, deletion, and tested recovery. Encryption protects specified data states; masking changes presentation; neither guarantees privacy. De-identification is context-dependent, and HIPAA provides two defined methods for protected health information.

Plan for change and failure

Prefer reversible decisions where uncertainty is high. Record dependencies and exit constraints, test backup restoration, define degradation behavior, and measure recovery objectives. “Cloud,” “microservices,” “mesh,” and storage/compute separation are options whose benefits depend on requirements and operating capability.

Review with fitness functions

contract compatibility and schema-change failure rate;
lineage and ownership coverage;
freshness and reconciliation performance;
access-revocation and recovery tests;
cost per workload and concentration risk;
time and completeness of a tested exit.