Banking Data Platform Engineering
Almost Every Question a Bank Asks Is an As At Question.
Most Platforms Answer As of Now.
Data platforms for banking built around point-in-time correctness, reconciled to the general ledger, with the daily cycle modelled rather than engineered around, and lineage that survives a restatement.
What was the balance at close of business on the fourteenth. What was the exposure at the moment the limit was breached. What did the customer’s risk rating say on the day the decision was taken.
The problem
Two time dimensions, and most platforms model one
Banking data has two independent clocks and conflating them is the most consequential modelling error in this domain.
One is when something was true in the world. The other is when the bank recorded or discovered it. They differ constantly, because transactions are back-dated, corrections are posted, reversals happen and information arrives late.
The platform can only answer as of now
So any question about a past position is reconstructed by hand.
Numbers do not tie to the general ledger
Which means nobody uses them, whatever else is true.
Streaming was applied to an end-of-day definition
Producing a live number the business has no definition for.
The core cannot be read the way the design assumed
No change capture, a narrow window, and a production system to protect.
Restatement was not designed for
So a corrected past figure overwrites the one that was reported.
Customer identity was treated as a data problem
When the same person is a borrower, a depositor and a guarantor.
Reconciliation was scheduled after go-live
Which is when the programme discovers whether it was worth doing.
Our approach
Model the time and the cycle first, then build the pipelines
Pipelines are the visible work and the straightforward part. The decisions that determine whether the platform can answer a banking question are taken in the data model, before any pipeline exists.
Step 1
Assess the architecture and the questions it is actually asked
Step 2
Establish the reconciliation position with finance at the start
Step 3
Model both time dimensions explicitly
Step 4
Model the operating cycle rather than engineering around it
Step 5
Establish what each source system can actually provide
Step 6
Design the platform layers to the questions, not to a reference diagram
Step 7
Resolve party and customer identity as a first-class problem
Step 8
Build pipelines with lineage captured as a by-product
Step 9
Migrate history with the reconciliation evidence retained
Step 10
Validate against the ledger and against the source systems
Step 11
Deploy with reconciliation running continuously from day one
Step 12
Monitor freshness, reconciliation, lineage completeness and cost
What we build
Platform work, grouped by what it has to guarantee
Grouped by the guarantee each layer provides rather than by technology, because in banking the guarantee is what the platform is bought for.
Foundation and ingestion
Data lake, warehouse and lakehouse architecture, selected on the workload rather than on the category
Ingestion from core banking, lending, cards, payments, treasury and channel systems, within what each source can actually provide
Streaming and event processing where the consumption is genuinely real-time
A raw layer retained exactly as received, because a source that restates is the main reason you need it
Model and guarantee
Bi-temporal modelling: valid time and recorded time, so any date can be reconstructed as it was known then and as it is understood now
Business date and cut-off modelled as data, so a transaction belongs to a period independently of its arrival
Reconciliation to the general ledger as a continuous control with an owned break process
Party and customer resolution across products, roles and relationships
Lineage captured as a by-product of the pipeline rather than documented beside it
Modernisation and delivery
Migration from legacy warehouses and marts, with reconciliation evidence retained as a deliverable
Modernisation of pipelines and orchestration, including the ones nobody can safely change
Consumption layers shaped to the question: regulatory reporting, risk aggregation, finance, analytics and the AI foundations our Banking AI pages depend on
Platform operations: orchestration, observability, cost attribution and capacity management
Where this gets used
Eight workloads, and the guarantee each one needs from the platform
The third column is the point of the table. These workloads look similar from the outside and they place genuinely different demands on the data model, which is why one platform design rarely serves all of them well.
| Workload | What it consumes | The guarantee it needs |
|---|---|---|
| Core banking integration | Accounts, balances, transactions, positions | Reconciliation to the core and to the ledger |
| Transaction consolidation | Movements across products and channels | One business date definition across all sources |
| Customer data platform | Party, relationship, product and interaction | Party resolution with the purpose attached |
| Risk data aggregation | Exposures, limits, collateral, ratings | Point-in-time correctness at the moment of breach |
| Regulatory reporting | Whatever the return requires, period-bound | Lineage to transaction, and restatement handling |
| Finance and ledger analytics | Balances, accruals, allocations | Tie to the ledger, with differences explained |
| Financial crime and AML | Transactions, parties, networks, alerts | Completeness, and party linkage across products |
| Analytics and AI foundations | Modelled, documented, accessible data | Documented meaning and stable definitions |
How it is built
Keep the raw layer. It is the only defence against a restating source.
Architecture debate in this space concentrates on the modelling layer and the consumption layer. The decision that saves a banking platform most often is the one nobody argues about: retaining the source exactly as received.
What each layer contributes
A raw layer holding every extract exactly as delivered, immutable, with the extraction time recorded, because a source restating history is the normal case and not an exception
An integration layer resolving identity, business date and currency, with both time dimensions applied consistently
A modelled layer shaped to banking subject areas rather than to source systems, since a platform organised by source reproduces the fragmentation it was built to remove
Consumption layers per workload, because regulatory reporting, risk and analytics need different shapes of the same facts
Orchestration with dependency awareness, since a banking cycle has a sequence and a late upstream feed must not produce a partial downstream position
Platforms and systems
Cloud data platforms and lakehouse architecture, with open table formats where they fit the workload
Warehousing and query engines, selected per consumption pattern rather than standardised on one
Streaming and event platforms for the flows that genuinely need them
Orchestration, transformation and data quality tooling
Integration: enterprise APIs, change data capture where sources support it, file-based extraction where they do not
Enterprise integration with core banking, lending, cards, payments, treasury, finance and risk systems
Governance
Lineage has to reach the transaction, and survive the correction
Which regulations apply to a given dataset or report, how it is classified, and what controls are required are determined by your compliance, legal and risk functions, based on your operating markets. We do not interpret those requirements and this page does not state which apply to you.
Technology supports a compliance programme. It does not independently establish compliance, and no platform makes an institution compliant.
Lineage and quality
- Lineage from any published figure to the transactions behind it, captured as a by-product of the pipeline and traversable rather than documented
- Lineage that survives a restatement, so the path to a previously reported number remains intact after the correction
- Data quality controls with rules, thresholds and an owned exception queue, since a rule with no owner is a monitor rather than a control
- Reconciliation to the ledger and to source systems as continuous controls with defined break ownership
Access and protection
- Access management resolved per user and per dataset, with customer and transaction data treated as the sensitive data it is
- Encryption in transit and at rest with key ownership stated, and controlled handling wherever data leaves the platform
- Masking and minimisation per consumer, so an analytics consumer does not receive what only operations requires
- Export and extract treated as part of the access model, because a file leaves every control behind
Records and residency
- Retention applied per data class rather than per table, with the raw layer retained to its own standard
- Data residency and processing location established before build, including any third-party processing
- Disposition as a controlled, evidenced act, since deletion is as much an obligation as retention
- Archive that remains readable and queryable for its retention period, which may outlive the platform holding it
Definitions and change
- Business definitions owned by named functions, versioned with effective dates, and implemented once rather than per consumer
- Change control over definitions, mappings and rules with regulatory or financial significance
- Manual adjustments captured as data with a reason and an approver, never applied invisibly
- Audit trails over access, definition changes, adjustments, reprocessing and exports
Keeping it trustworthy
A pipeline that succeeded is not the same as a position that is right
Platform monitoring concentrates on whether jobs ran. In banking the more important question is whether the published position is complete, reconciled and current, and a green pipeline answers none of that.
| Failure | How it presents |
|---|---|
| A partial position was published | Every job succeeded and a product is missing |
| Reconciliation breaks are ageing | The platform and the ledger are drifting quietly |
| A source restated history | Past figures moved with no change on your side |
| Freshness degraded within the window | Nothing failed and the data is older every cycle |
| The cycle window is being exceeded | Downstream consumers start late and nobody owns it |
| Lineage gaps on a reported figure | The number cannot be explained to an examiner |
| A manual adjustment became permanent | Applied monthly for years with no remaining rationale |
| Access permissions drifted | Sensitive data reachable by people nobody intended |
| Cost growing faster than volume | An inefficient pattern is spreading across pipelines |
Outcomes
A platform finance will argue with once, and then stop
The outcomes below follow from three choices: model both time dimensions, reconcile to the ledger as the acceptance criterion, and keep the raw layer.
A trusted data foundation, evidenced by reconciliation rather than asserted by architecture
Any date reconstructable as it was known then and as it is understood now, which is what regulatory and dispute questions require
Regulatory reporting defensible, with lineage to transaction and restatement handled rather than overwritten
Faster integration of new sources and new consumers, because the time, date and identity patterns are established once
Technical debt reduced where it was costing something, with legacy marts retired rather than surrounded
Analytics and AI foundations that are usable, since documented meaning and stable definitions are their prerequisite
Operating cost lowered and attributable, with cost per pipeline and per consumer visible from the start
Ask us for a figure as it stood on a date you have already reported.
We will take one reported figure and one business date and establish what your platform can actually do: reproduce it as reported, explain the difference against today, trace it to transactions, and reconcile it to the ledger. You get the findings whether or not we build anything.
Why choose CaliberFocus for Banking Data Platform Engineering?
CaliberFocus builds banking data platforms that can reconstruct any date, tie to the general ledger and explain a number after it has been restated. We model both time dimensions and the daily cycle before building pipelines, respect what the core can actually provide, retain the raw layer exactly as received, and run reconciliation to the ledger continuously from day one.
- Bi-temporal modelling, so any date reconstructs as known then and now
- Reconciliation to the general ledger as the acceptance criterion
- Business date and cut-off modelled as data, not as a schedule
- Lineage from reported figures to transactions that survives restatement
Security & Compliance
