Layer model
Every system we build follows a six-layer model. The layers are not deployment tiers — they are logical boundaries that constrain coupling. A change in one layer should not require changes in another.
L6 Client Interfaces, consoles, portals
L5 API Versioned contracts, auth, rate limits
L4 Orchestration Workflows, sagas, schedulers
L3 Services Domain boundaries, SLAs
L2 Data Stores, streams, schemas
L1 Infrastructure Compute, network, secrets
Layers communicate downward only. L4 never calls L6. L2 never reaches into L3's tables. If you need cross-layer communication, you go through a contract, not a direct reference.
Service boundaries
We define service boundaries by domain, not by technical function. A "payment service" owns everything about payments — the data model, the business logic, the API surface. Other services interact with it through its public contract.
Rules we follow:
- Services own their data. No shared databases.
- Cross-service reads go through APIs, not direct queries.
- Cross-service writes go through events, not direct calls.
- Every service has an explicit latency budget and error budget.
- Failures in one service never cascade to others.
Data modeling
Data is modeled once per domain. We use event sourcing for write-heavy domains and traditional CRUD for read-heavy ones. Streams carry facts. Caches are disposable. Migrations are forward-only and reversible in rehearsal.
We treat the database as a private implementation detail of the service. If you need data from another service, you call its API or subscribe to its events. You never reach into its tables.
Sagas & workflows
Long-running work is a workflow with a name, a history, and a compensating action. Every workflow has:
- A unique execution ID for tracing and replay.
- Idempotent activity implementations.
- Compensating actions for every write step.
- A dead-letter path with operator replay.
- Timeout and retry policies per activity.
Cron jobs are not an architecture. If something needs to run on a schedule, it's a workflow with a timer trigger.
Observability
Telemetry is a product requirement, not a dashboard you add later. Every service emits structured logs, metrics, and traces from day one. We use OpenTelemetry as the standard instrumentation layer.
traces → Jaeger / Tempo
metrics → Prometheus → Grafana
logs → Structured JSON → Loki
alerts → PagerDuty / OpsGenie
If you can't see it, stop it, and roll it back, it's not in production — it's a demo.
Deployment
We deploy progressively. Canary → percentage → full rollout. Every deployment has a rehearsed rollback procedure. Every service has an on-call owner who wrote the code.
Infrastructure is code. Secrets are injected at runtime, never baked into images or config files. Environments are reproducible from a single source of truth.