Architecture / Architecture Operating Models / Measurement
05 · Evidence
Measurement and data lineage
Instrument the lifecycle, trace outcomes to trustworthy sources and expose data quality.
17leading indicators
18lineage definitions
1illustrative scorecard
Workbook source · ODM Contributing Indicators
Through-life instrumentation
Leading and diagnostic phase-gate indicators.
Tier-2 leading and diagnostic indicators instrumented at Service Through Life phase gates. These do not carry PLAs; they explain ODM movement and give EA/SA/OA leads early warning. Phase numbers refer to the Through Life tab.
| CI ID | Through Life Phase | Architecture Layer | Indicator | Measure | Role Owner | Feeds ODM(s) | Source System |
|---|---|---|---|---|---|---|---|
| CI-01 | 1. Strategy | Business | Capability-to-investment mapping | % of proposals entering portfolio intake with target capability and roadmap item identified; count of unmapped proposals by sponsor. | EA | ODM-01, 03 | Portfolio intake; architecture repository |
| CI-02 | 2. Planning | All layers | Architecture baseline at Planning gate | % of Tier-1 initiatives with architecture vision, baseline requirements (including NFRs) and roadmap position recorded before Design entry. | SA | ODM-01, 02, 07 | Architecture repository; requirements repository |
| CI-03 | 3. Design | All layers | Design-gate review coverage | % of in-scope changes passing the Design gate with completed architecture review; findings per review; median review turnaround (working days). | SA | ODM-04, 08, 15 | Delivery pipeline gates; Design Authority review log |
| CI-04 | 3. Design | All layers | Dispensations raised at design | Count and severity of dispensations granted at gate; % with owner, expiry and remediation path recorded. | EA | ODM-05, 08, 09, 11 | Dispensation register |
| CI-05 | 4–5. Development/Deployment | Information Systems | Design escapes | Design defects found in build/test/deployment that should have been caught at design (escape rate); NFR test pass rate at go-live gate. | SA | ODM-04, 05 | Defect tracker; performance and NFR test results |
| CI-06 | 5. Deployment | Technology | Pattern conformance at go-live | % of deployments matching the specifying reference pattern or baseline at first configuration scan after go-live. | OA | ODM-06 | Configuration compliance scanning; CMDB |
| CI-07 | 5–6. Deployment/Transition | Information Systems | Verification evidence at handover | % of requirements (including NFRs) with verification evidence at operational acceptance; completeness of runbook, monitoring and support-model handover. | OA | ODM-04, 07, 13 | Test evidence; service transition records |
| CI-08 | 7–8. Operation/Maintenance | Technology | Drift detection | Mean days from configuration drift (vs pattern/baseline) to detection; count of unremediated drift items >30 days. | OA | ODM-06 | Configuration compliance scanning |
| CI-09 | 7–8. Operation/Maintenance | Technology | End-of-support horizon | Tier-1 components reaching end of vendor support within 12 months without an approved exit plan; count past end of support. | OA | ODM-10 | Technology lifecycle register; CMDB |
| CI-10 | 6–7. Transition/Operation | All layers | Early-life incident rate | P1–P3 incidents in the first 30 days after go-live per release; % attributable to operability gaps (monitoring, runbook, capacity). | OA | ODM-13 | ITSM incident records |
| CI-11 | 8–9. Maintenance/Optimisation | All layers | Debt identified vs retired | Debt items identified per assessment cycle vs items retired; net debt movement on critical services; lapsed dispensations converted to debt. | EA | ODM-09, 10, 11 | Technical debt register; dispensation register |
| CI-12 | 1–2. Strategy/Planning | Business | Roadmap milestone health | Milestone RAG across the architecture roadmap; slippage >1 quarter flagged; unresolved cross-programme dependency conflicts. | SA | ODM-02 | Portfolio tooling; roadmap register |
| CI-13 | 7. Operation | All layers | Post-incident architectural review | % of P1/P2 incidents with architectural root-cause analysis completed within 30 days; pattern/standard changes raised per incident. | OA | ODM-14 | Incident and problem records; standards register |
| CI-14 | 9. Optimisation | All layers | Run-cost trend | Unit run-cost per service against baseline; savings realised from rationalisation and optimisation actions. | OA | ODM-12 | Finance / FinOps tooling; CMDB |
| CI-15 | 7–9. Operation–Optimisation | Technology | Recovery exercise completion | Critical services exercised in rolling 12 months; RTO/RPO achievement per exercise. | OA | ODM-14 | BC/DR test records |
| CI-16 | 1–3. Strategy–Design | Business | Early engagement and self-service signals | % of initiatives engaging architecture at Strategy/Planning; pattern instantiations without Design Authority escalation; repeat engagement by delivery team. | EA | ODM-15, 16 | Engagement log; pattern repository telemetry; accreditation records |
| CI-17 | 10. Retirement | All layers | Decommissioning completion | % of retired services with completed decommissioning checklist (dependencies removed, data dispositioned, access removed, licences exited, repository updated). | OA | ODM-11, 12 | CMDB; decommissioning records |
Workbook source · ODM Data Lineage
Operational data to decision evidence
Sources, extraction, calculations and quality ownership.
Operational Data → Metrics path per ODM (Risk Ontology). An ODM without a reliable automated source should not enter a PLA until its lineage is fixed - data quality kills credibility faster than a bad number.
| ODM ID | ODM Name | Operational Data Required | System of Record | Extraction | Frequency | Transformation / Calculation | Quality Checks & Data Owner |
|---|---|---|---|---|---|---|---|
| ODM-01 | Strategic alignment of investment | Approved investments with value; capability and roadmap links | Portfolio tooling; architecture repository | API / export | Quarterly | Σ approved value traced to capability/roadmap item ÷ Σ Tier-1 approved value | Capability link mandatory field at investment gate; untraced items reviewed by EA. Owner: EA lead |
| ODM-02 | Transition architecture delivery | Roadmap milestones: planned vs actual dates | Portfolio / programme tooling; roadmap register | API | Quarterly | Milestones delivered within +1 quarter ÷ milestones due | Baseline change control on plan dates. Owner: SA lead |
| ODM-03 | Capability assessment currency | Capability list, tier, last assessment date | Architecture repository (capability model) | Repository query | Semi-annual | Tier-1 capabilities assessed ≤ 12 months ÷ all Tier-1 capabilities | Capability tiering agreed with COO annually. Owner: EA lead |
| ODM-04 | Design-stage defect capture | Design defects by detection phase; problem records tagged 'design-attributable' | Design review log; defect tracker; problem management | API / export | Monthly | Defects detected ≤ Design gate ÷ all design defects in period | Phase tagging mandatory at defect entry; RCA tagging QA-sampled 10%. Owner: SA lead |
| ODM-05 | Architecture rework cost | Change records flagged architecture/design-driven post-build; effort/cost per change | Delivery tooling; finance cost codes | API + finance extract | Quarterly | Σ architecture-driven rework cost ÷ total in-scope delivery spend | Design-driven flag reviewed at change board; finance code reconciliation. Owner: EA lead |
| ODM-06 | Pattern & baseline conformance | Deployment records; pattern/baseline versions; configuration scan results | CMDB; compliance scanning; architecture repository | Automated scan | Monthly | Conformant deployments ÷ pattern-specified deployments; drift age from scan timestamps | Baselines version-controlled; scan coverage ≥ 95% of Tier-1 estate. Owner: OA lead |
| ODM-07 | Requirements & NFR traceability integrity | Requirements (incl. NFRs) with links to design elements and verification evidence | Requirements repository; test management; architecture repository | API | Quarterly | Tier-1 changes with complete req → design → evidence chain ÷ all Tier-1 changes at go-live | Broken-link report zero at gate; evidence attachments mandatory. Owner: SA lead |
| ODM-08 | Design Authority decision cycle time | Submission-complete and decision timestamps; tier | Design Authority decision log | Export | Quarterly | Median working days submission-complete → decision, by tier | 'Complete submission' definition published; clock-stop rules documented. Owner: EA lead |
| ODM-09 | Dispensation health | Dispensations: owner, expiry, remediation path, status | Dispensation (exception) register | API | Monthly | Active dispensations owned, in date and with remediation path ÷ all active dispensations | No record without owner and expiry; monthly lapsed report to Design Authority. Owner: EA lead |
| ODM-10 | Technology currency | Tier-1 components with product version; vendor support dates; exit plans | CMDB; technology lifecycle register | API + reference data | Monthly | Components in support or with funded exit plan ÷ all Tier-1 components | Vendor support dates refreshed quarterly; CMDB completeness ≥ 95%. Owner: OA lead |
| ODM-11 | Technical debt exposure | Debt items: severity, age, service, owner, expiry | Technical debt register; dispensation register | API | Monthly | Σ (severity weight × age factor) on critical services, indexed to baseline = 100 | No item without owner and expiry; monthly orphan report. Owner: EA lead |
| ODM-12 | Rationalisation delivery | Rationalisation actions: planned vs actual; savings realised | Application portfolio register; CMDB; finance / FinOps | API + finance extract | Quarterly | Actions completed within +1 quarter with savings evidenced ÷ actions due | Savings validated by finance business partner. Owner: OA lead |
| ODM-13 | Operational acceptance first-time pass | Acceptance submissions, outcomes, waivers per release | Service transition records; ITSM | API | Quarterly | Tier-1 releases accepted first time without waiver ÷ Tier-1 releases submitted | Service acceptance criteria version-controlled; waiver reasons coded. Owner: OA lead |
| ODM-14 | Resilience design verification | Exercise records with RTO/RPO results per critical service | BC/DR test records; service catalogue | Export | Quarterly | Critical services exercised ≤ 12 months meeting RTO/RPO ÷ all critical services | Criticality list owned by Resilience Board; results signed by service owner. Owner: OA lead |
| ODM-15 | Early engagement rate | Initiative register with engagement timestamps vs Design gate entry | Pipeline gates; engagement log | API | Quarterly | Initiatives engaged before Design gate ÷ in-scope initiatives | In-scope (Tier-1) definition reviewed annually. Owner: SA lead |
| ODM-16 | Reuse & capability uplift | Accreditations; pattern usage without escalation | Training records; pattern telemetry | Export + telemetry | Annual | Composite trend: practitioner count; self-served design % | Accreditation register verified by HR. Owner: EA lead |
| MP-01 | Metric automation & data quality | Pipeline run logs; manual-adjustment log | Metrics pipeline | Pipeline logs | Quarterly | ODMs produced without manual adjustment ÷ all ODMs | Adjustment log reviewed at Design Authority. Owner: OA lead |
| MP-02 | Decision uptake | Governance decision records with evidence citations | Decision log / minutes | Export | Quarterly | Decisions citing ODM evidence ÷ all recorded decisions | Citation field mandatory in decision template. Owner: EA lead |
Workbook source · ODM Dashboard
Illustrative scorecard
A worked portfolio view; current values are placeholders.
Current values (blue) are illustrative placeholders - replace with pipeline feeds. RAG and rollups are formula-driven from the thresholds in the ODM Register.
| ODM ID | ODM Name | Type | Unit | Direction | Target (Green) | Amber Threshold | Current (illustrative) | RAG |
|---|---|---|---|---|---|---|---|---|
| ODM-01 | Strategic alignment of investment | KRI | % | Higher | 90 | 80 | 92 | Green |
| ODM-02 | Transition architecture delivery | KPI | % | Higher | 80 | 65 | 78 | Amber |
| ODM-03 | Capability assessment currency | KCI | % | Higher | 90 | 75 | 70 | Red |
| ODM-04 | Design-stage defect capture | KPI | % | Higher | 80 | 65 | 74 | Amber |
| ODM-05 | Architecture rework cost | KRI | % | Lower | 3 | 5 | 2.6 | Green |
| ODM-06 | Pattern & baseline conformance | KCI | % | Higher | 85 | 70 | 81 | Amber |
| ODM-07 | Requirements & NFR traceability integrity | KCI | % | Higher | 95 | 85 | 96 | Green |
| ODM-08 | Design Authority decision cycle time | KPI | days | Lower | 10 | 15 | 12 | Amber |
| ODM-09 | Dispensation health | KRI | % | Higher | 95 | 85 | 88 | Amber |
| ODM-10 | Technology currency | KRI | % | Higher | 95 | 85 | 83 | Red |
| ODM-11 | Technical debt exposure | KRI | index | Lower | 100 | 130 | 112 | Amber |
| ODM-12 | Rationalisation delivery | KPI | % | Higher | 80 | 65 | 83 | Green |
| ODM-13 | Operational acceptance first-time pass | KCI | % | Higher | 85 | 70 | 88 | Green |
| ODM-14 | Resilience design verification | KCI | % | Higher | 95 | 85 | 89 | Amber |
| ODM-15 | Early engagement rate | KPI | % | Higher | 90 | 75 | 86 | Amber |
| ODM-16 | Reuse & capability uplift | KPI | trend | Higher | 10 | 5 | 12 | Green |
| MP-01 | Metric automation & data quality | KCI | % | Higher | 80 | 60 | 65 | Amber |
| MP-02 | Decision uptake | KPI | % | Higher | 70 | 50 | 55 | Amber |
Portfolio rollup
| Green | 6 |
|---|---|
| Amber | 10 |
| Red | 2 |
| Total ODMs | 18 |