Example: An IPV Project#

This page runs one project through the approaches in the earlier pages. The project is to improve independent price verification (IPV) by adding an independent source of long-dated swaption volatility, automating the comparison with the desk marks, and tracking the exceptions in Jira. It continues the examples in Financial Statements, where a 20-year EUR swaption failed its IPV check.

Everything here is invented and meant as an illustration. The numbers are consistent across the tables, as they would be in a real plan.

The project in brief#

Problem: the IPV process has no independent evidence for a part of the long-dated swaption book. Those positions are marked on the desk’s own volatility surface and are reviewed by hand, so the check is slow and the evidence is thin. This is a risk to the Level 2 and Level 3 classification (see Fair-value hierarchy: Levels 1, 2 & 3) and to the audit of the fair values.

Objective: bring independent evidence to the long-dated swaption volatility, and make the daily comparison and the exception process automatic.

Business case measures#
Measure	Baseline	Target	Minimum acceptable
Share of fair value with independent IPV evidence	82%	95%	92%
Time to complete the daily IPV run	T+3	T+1	T+2
Exceptions open for more than 10 working days	14	0	3
Manual effort at month end, in person-days	12	4	6

The minimum acceptable column is the tolerance. If the project is forecast to fall short of it, the sponsor must decide whether to continue, change the scope or stop.

Approach 1: waterfall#

Waterfall plan, 18 weeks#
Phase	Weeks	Main deliverables	Gate
Requirements	1 to 3	Business requirements, scope, data needs and acceptance criteria.	Sponsor approves the requirements.
Design	4 to 6	Data mapping, IPV methodology, tolerances, and system design.	Design review. Model validation reviews the methodology.
Build	7 to 12	Data feed, comparison, exception workflow, adjustment calculation and dashboard.	Build complete.
Test	13 to 16	System testing, user acceptance, and reconciliation of the IPV adjustment to the ledger.	User acceptance sign-off.
Deploy	17 to 18	Go-live, parallel run and handover to the team.	Go-live approval.

The plan is clear, and each gate gives the sponsor and audit a document to sign. The weakness is when the project learns things. The team first sees real vendor data at week 7, and the users first see a working comparison in week 13. If the vendor’s quotes are stale beyond 15 years, the project finds out half way through, when a change is expensive.

Approach 2: Scrum#

The product owner is the IPV lead. The product goal is “the daily IPV run for long-dated swaptions uses independent data, flags breaches and tracks exceptions”. The backlog is in Jira, grouped into four epics.

Product backlog#
Issue	Epic	Story	Points
IPV-101	Data	Agree the vendor contract for swaption volatility.	5
IPV-102	Data	Map the vendor quotes to the internal product, tenor and strike grid.	8
IPV-103	Data	Load the vendor feed daily, with validation for gaps and stale quotes.	8
IPV-104	Method	Propose tolerance thresholds by product and level.	5
IPV-105	Method	Document the swaption IPV methodology and get it approved.	5
IPV-106	Automation	Compare the desk mark with the independent mark and flag breaches.	8
IPV-107	Automation	Track each exception in Jira, with an owner, ageing and status.	5
IPV-108	Automation	Calculate the IPV adjustment and pass it to the ledger bridge.	8
IPV-109	Control	Build the coverage and exceptions dashboard.	5
IPV-110	Control	Reconcile the IPV adjustment to the general ledger.	5
IPV-111	Control	Write the user guide and get the control sign-off.	3
Total			65

The team’s velocity is estimated at 13 points a sprint, so the 65 points need five two-week sprints:

Sprint plan#
Sprint	Weeks	Sprint goal	Issues	Points
1	1 to 2	The vendor is agreed and mapped to the swaption grid.	IPV-101, 102	13
2	3 to 4	Vendor data loads daily with validation, and tolerances are proposed.	IPV-103, 104	13
3	5 to 6	The methodology is approved, and the comparison runs in test.	IPV-105, 106	13
4	7 to 8	Exceptions are tracked in Jira, and the adjustment is calculated.	IPV-107, 108	13
5	9 to 10	The dashboard, the ledger reconciliation and the sign-off are ready for go-live.	IPV-109, 110, 111	13

After each sprint, the team reviews the increment with the users, and updates the burndown:

Burndown of the 65 points#
After	Completed	Remaining
Start	0	65
Sprint 1	13	52
Sprint 2	26	39
Sprint 3	39	26
Sprint 4	52	13
Sprint 5	65	0

Real velocity varies from sprint to sprint, so a real burndown is not this smooth. The product owner would reorder the backlog as the team learns.

What the iterations show early#

Real data arrives in sprint 2 and a working comparison in sprint 3. Suppose that the sprint 2 review shows that the vendor’s quotes beyond 15 years are often stale. The team can act on it straight away: it adds a staleness check to the acceptance criteria of IPV-106, and the IPV lead asks the vendor about coverage. It would also be an input to the Level 3 judgement. Under the waterfall plan, the same finding would come in the build or test phase.

Approach 3: hybrid#

In many banks the governance is fixed by a method such as PRINCE2, and the teams use Scrum inside it. The same project would look like this:

Hybrid plan, 14 weeks#
Stage	Duration	Contents and decision
Initiation	2 weeks	Business case, scope, tolerances and the risk log. The project board decides whether to start.
Delivery stage 1	6 weeks	Sprints 1 to 3. At the end, the board sees working data and a comparison, and decides whether to continue.
Delivery stage 2	4 weeks	Sprints 4 and 5. At the end, the board decides whether the project is ready for go-live.
Go-live and closure	2 weeks	Parallel run, handover, a benefits review plan and lessons learnt.

The project board is the head of valuation control as executive, product control as senior user, and the head of the technology team as senior supplier. The board sets these tolerances, and the project manager escalates only if one is going to be breached:

Time: plus or minus 2 weeks on the end date.
Cost: plus or minus 10% of the budget.
Benefit: independent evidence for at least 92% of fair value, as in the table of measures.

The 14 weeks in this illustration, against 18 for waterfall, come from overlapping design, build and test, and from not waiting for a full sign-off of each phase before starting the next. It is not a general result, because a project with unstable scope or unavailable users can take longer in sprints.

RAID log#

RAID log at the end of initiation#
Ref	Type	Description	Rating	Action and owner
R1	Risk	Vendor quotes are thin or stale beyond 15 years.	High	Second source for long tenors. Staleness check in the feed. IPV lead.
R2	Risk	Model validation cannot approve the methodology before go-live.	Medium	Involve validation from sprint 1. Share the draft early. Valuation control lead.
A1	Assumption	The vendor contract is signed before sprint 1.	Medium	Sponsor to confirm. If wrong, it becomes an issue.
D1	Dependency	The market data team supplies the ticker mapping in sprint 1.	Medium	Agree the date at the initiation board. Market data lead.
D2	Dependency	Technology sets up the Jira exception project before sprint 4.	Low	Raised as a task in sprint 3. Technology lead.

The same log is used in all three approaches. In Scrum it is reviewed at the sprint review, and in the hybrid plan it is reviewed at each board.

Evidence for the control#

The result is a change to a control over fair value, so the project has to leave evidence. For each increment, the Definition of Done includes:

the Jira issues, linked to the test results
the approved methodology, with the date and the approver (see Governance: Model Risk, Accountability and Data)
the reconciliation of the IPV adjustment to the ledger, as in the roll-forward in Financial Statements
the control owner’s sign-off, held where audit can find it (see Sarbanes-Oxley: Internal Controls over Financial Reporting)

Measuring the benefit#

After three months of use, the sponsor reviews the benefits against the baseline:

Benefits review after three months#
Measure	Baseline	Target	Actual	Result
Share of fair value with independent IPV evidence	82%	95%	96%	Met
Time to complete the daily IPV run	T+3	T+1	T+1	Met
Exceptions open for more than 10 working days	14	0	2	Within tolerance
Manual effort at month end, in person-days	12	4	5	Within tolerance

Two measures are better than the minimum acceptable but short of the target, which is a typical result. The review should say why, for example that two old exceptions depend on a vendor query, and decide on any follow-up.

Comparing the approaches#

The same project, three ways#
Question	Waterfall	Scrum	Hybrid
When is real data first used?	Week 7	Week 3	Week 3
When do the users first see a working comparison?	Week 13	Week 5 to 6	Week 5 to 6, at the stage 1 board
How are changes handled?	Change control request, with an impact assessment.	Backlog, reordered by the product owner.	Backlog inside the board’s tolerances. A breach goes to the board.
Where is the approval?	At each phase gate.	Sprint review and the product owner’s acceptance.	Stage boundaries and the product owner.
Where is the evidence?	Phase documents.	Jira issues and the Definition of Done.	Both.
Elapsed time	18 weeks	10 weeks plus a go-live	14 weeks

The choice depends on the points in Waterfall and Agile: Two Ways to Run a Project. If the same project were driven by a fixed regulatory date and fixed scope, waterfall would be easier to defend. For improving data and tools, where the users learn what they need by seeing it, Scrum or the hybrid is a better fit.