In November 2019, complaints went viral that the Apple Card credit algorithm was assigning lower limits to women than to equally qualified men, in some cases within the same household. When customers called Goldman Sachs, which issued the card, service representatives said they had no access to the algorithm's reasoning and no authority to change its output. New York's Department of Financial Services opened an investigation. Apple pointed to Goldman's underwriting responsibility. Goldman pointed to Apple's technology. Eighteen months later, the regulator closed the inquiry, finding no evidence of intentional discrimination and no evidence of disparate impact on any protected group. The more instructive outcome was what the process revealed about accountability structure: three parties, no connected record, no single function that held the full picture of what the system was designed to do, how it had been monitored, and what it had actually done.
That pattern is not unique to Apple and Goldman. It surfaces regularly when regulators examine AI deployments in financial services, and it is becoming more consequential as EU obligations tighten. The accountability question the Apple Card episode exposed is, more than six years later, still unresolved in most regulated firms, and it is starting to carry real enforcement consequences.
A Gap That Existing Governance Did Not Anticipate
Risk management teams in regulated FinTech firms are doing genuine governance work. Model risk frameworks, internal audit processes, compliance reviews, and senior manager accountability regimes are all functioning. The issue is more specific: those governance structures were largely designed before AI product teams became the primary vehicle through which AI is deployed at scale into regulated processes. The frameworks are sound. The connection between the frameworks and the teams operating inside them is where the gap appears.
In July 2025, the Massachusetts Attorney General reached a $2.5 million settlement with Earnest Operations, a student-loan lender, over allegations that its AI-driven underwriting models produced disparate harm to Black, Hispanic, and non-citizen applicants. The models had not been tested for disparate impact. Underwriters frequently overrode the models without clear standards or documentation. Adverse action notices did not adequately explain why applications were denied. As part of the settlement, Earnest agreed to discontinue two of the contested model variables and to build a written corporate governance system for its AI use, going forward. Earnest denied the underlying allegations. What the case shows is not malicious intent but a familiar pattern: the governance infrastructure existed, and the AI system was deployed, but the connection between the two had never been formally established.
The Bank of England and FCA's 2024 survey of AI in UK financial services found that 84 per cent of firms reported having an accountable person for their AI framework, yet most of those same firms named three or more accountable parties simultaneously. McKinsey's March 2026 State of AI Trust research found that organisations with explicitly assigned AI governance accountability scored 2.6 on a four-point maturity scale, against 1.8 for those without it. The gap was not explained by resources or intent. It was explained by whether someone held the complete picture continuously.
As practitioners at global financial institutions have described it, the distribution looks like this: the product function owns the model, the engineering function owns the infrastructure, the compliance function owns the policy. When a regulator challenges a decision, each can credibly point elsewhere. The accountability gap is not a failure of any individual function. It is a structural property of how the three functions were arranged before AI deployment at speed became the norm.
Why the Gap Has Proved Difficult to Close
The instinctive response when a governance gap appears is to strengthen the compliance programme: clarify obligations, document controls, assign responsibilities. That response addresses the regulatory environment well. It is less effective for a second problem that AI deployment creates, one that operates on a different timescale and arrives through a different channel.
Compliance frameworks were designed for systems that behave predictably after deployment. A rules engine, a calculation, a deterministic API call does what it was built to do, and the governance review at design time remains valid because the system stays within its assessed boundaries. Risk registers, model validation procedures, and audit frameworks were built on this assumption. The system is fixed. The framework can be periodic.
AI systems, particularly the adaptive models now used in credit decisioning, fraud detection, and customer onboarding, do not stay within their assessed boundaries in the same way. They drift when input data shifts. They behave differently at the edges of their training distribution. A system that was adequate when the risk function reviewed it may, months into production, be making consequential decisions in conditions the original assessment did not cover. The risk function cannot see this unless the product team is actively monitoring for it and has a channel to report what it finds. The product team cannot respond to regulatory changes unless the risk function has a way to translate those changes into the technical parameters of the deployed system.
The EU AI Act illustrates the regulatory architecture of this problem. Article 26(5) specifies that for financial institutions, deployer obligations for high-risk AI systems are met by complying with rules on internal governance arrangements under existing EU financial services law. Those governance arrangements were drafted before AI governance was a distinct regulatory obligation, and the EBA was still mapping how they apply to AI systems as recently as November 2025. The compliance deadline for standalone high-risk systems under Annex III, which includes credit scoring, was originally set for August 2026. Under the EU's Digital Omnibus, finalised in mid-2026, that deadline has been deferred to 2 December 2027, with high-risk AI embedded in already-regulated products deferred to August 2028. The reason for the deferral is itself telling: the harmonised technical standards and the national supervisory infrastructure needed to assess conformity were not ready in time. The obligations did not shrink. The clock moved because the ecosystem around them was not built.
That is not a reason for regulated firms to relax. The UK Parliamentary Treasury Committee stated in January 2026 that it was not confident the financial system was prepared for a major AI-related incident. The FCA, for its part, has so far chosen not to introduce AI-specific rules, relying instead on existing accountability regimes such as the Senior Managers and Certification Regime, a position the Treasury Committee's report suggests has left the practical question of where AI accountability sits still unresolved. A longer runway to a fixed compliance date is not the same as a settled answer to that question, and firms that treat the deferral as permission to wait will be building the governance routine under deadline pressure instead of in the years they have now been given.
These are not criticisms of the existing risk infrastructure. They are observations that the regulatory architecture has not yet specified how existing governance frameworks connect to the AI deployment layer. Until that connection is specified, both risk and product functions are working in parallel rather than together.
From Compliance to Governance Capability
Management research on how organisations adapt to rapidly changing environments offers a useful reframe. The core insight from what researchers call dynamic capabilities is that organisations which perform well under conditions of change are not those with the best procedures at a fixed point in time. They are those with the organisational routines to continuously sense what is changing in their environment, commit to a response, and adjust their internal arrangements accordingly.
Applied to AI governance, the question shifts from whether a firm complies with current requirements to whether it has built the routines to govern continuously. Compliance satisfies the regulatory environment at a point in time. Governance capability keeps the firm aligned as both the AI system and the regulatory environment keep moving. The distinction matters in practice because AI deployment creates two simultaneously moving targets, and a compliance programme calibrated to one will miss signals from the other.
Two Environments, Neither Following the Other
Every regulated FinTech firm deploying AI is operating in two environments that change independently and at different speeds. Understanding both is the foundation of effective AI governance, and it is where the risk function and the product team need to work as a unit rather than in sequence.
The first is the AI capability environment. It changes rapidly and is driven by technical events: model drift as production data diverges from training data, performance degradation in underrepresented segments, behavioural shifts following retraining or upstream model updates. The signals from this environment arrive in inference logs, monitoring dashboards, and data quality alerts. They are visible to the product team and the data science team. They are rarely visible to the risk function unless a formal channel connects them.
The second is the regulatory environment. EU AI Act obligations evolve through implementing acts and technical standards, as the recent Digital Omnibus deferral itself demonstrates. DORA requirements are refined through supervisory authority publications. EBA guidelines are updated through consultation cycles. These changes carry legal force and arrive through compliance and legal channels. They reach the risk function and legal team. They may not reach the product team in time to change how a system is operating.
This is the structural source of the accountability gap. It is not that either function is poorly run. It is that the two streams of information needed to govern AI responsibly arrive in different parts of the organisation, are processed by people with different expertise and different priorities, and rarely meet in a shared governance record. The product team cannot exercise governance accountability it does not hold. The risk function cannot exercise oversight over technical behaviour it cannot see. The gap between them is where the accountability question disappears.
McKinsey's 2026 governance maturity data makes the consequence quantifiable. The 0.8-point gap between firms with assigned AI accountability and those without is not a function of organisational size, regulatory regime, or available resources. It is explained by whether a firm has built the routines that connect both environments into a shared view, and whether a named person holds responsibility for maintaining that connection.
Three Conditions for Joint Governance
For risk and product functions to govern AI together rather than sequentially, three structural conditions need to be in place. These are not new organisational units or expensive technology deployments. They are shared routines that both functions build and exercise together, starting from the early stages of product development rather than at the point of deployment review.
| Condition | What it requires | What risk and product each bring |
|---|---|---|
| Decision authority at design time | Before an AI system enters development, the acceptable behavioural parameters are jointly documented as an architectural commitment: what data the model can access, what decisions it can influence autonomously, where human confirmation is required, what counts as a deviation that triggers escalation. | Risk brings the regulatory constraint. Product brings the technical feasibility assessment. Together they produce a design-time contract both functions understand and can monitor against. |
| Ongoing monitoring mandate | A named owner, jointly accountable to both functions, reviews the gap between documented parameters and observed runtime behaviour on a continuous basis, not only at scheduled checkpoints. | Product translates technical signals, declining confidence in a segment, a runtime condition the design-time assessment did not cover, into terms risk can act on. Risk interprets what those signals mean regulatorily. |
| Evidentiary chain | Design-time parameters, monitoring records, and decision logs connected traceably in a single form, not reconstructed after the fact from separate archives. | A living record maintained jointly, so that when a regulator asks what governed a specific decision, the answer is retrievable rather than assembled under deadline. |
Decision authority at design time matters because it is the only moment at which governance can be built into the system rather than imposed on it after deployment. A governance policy written after the system is live describes what the organisation wishes were true. A design-time commitment determines what the system can actually do. The EU AI Act's requirements for human oversight under Article 14 and technical documentation under Article 11 are both, in practical terms, requirements for this early agreement to exist in a documented form.
Without a named monitoring owner, review happens informally or not at all. When it does happen, the findings stay within the technical team and never reach the governance record. The risk function cannot act on signals it does not receive. The product team cannot understand the regulatory implications of technical changes without the risk function's interpretation. The monitoring mandate is the connecting function both environments need.
The evidentiary chain is not a compliance archive assembled before a review. It is a living record, maintained jointly, that connects what the system was designed to do, what it has been monitored doing, and what it has actually done at any consequential moment. The Earnest Operations case is, in part, a description of what it costs when this record does not exist. The settlement required the firm to build a governance system after the fact. Building it before deployment is a fraction of the cost and produces a substantially better outcome for both functions.
What Risk and Product Teams Should Do Together
The governance model described here does not require a reorganisation or a new technology platform. It requires a shared operating rhythm between two functions that are already present in most regulated FinTech firms, starting from product ideation rather than at the compliance review stage.
The most important shift is in when the two functions engage. In most firms, the risk function reviews an AI system at defined checkpoints: before deployment, at periodic model validation, following a significant change. The product team builds and monitors between those checkpoints. By the time the next review arrives, the system may have drifted from the parameters that were approved. The review then reflects a snapshot of a system that has already moved on. Bringing risk and product into a shared conversation from the product ideation stage, and maintaining that conversation through the development cycle, means the design-time commitment reflects genuine joint understanding rather than a compliance sign-off on a technical document the risk function did not fully interpret.
A practical starting point is a standing monthly governance review attended by the product owner, a member of the risk function, and the data science lead responsible for the model. The agenda has three questions: what is the system doing that it was not designed to do; does anything in the current regulatory environment change how we should respond to that; and what does the governance record need to reflect this month. That is not an enterprise transformation. It is a governance routine that costs a few hours a month, produces the connected record that regulators will eventually require, and creates the shared understanding between risk and product that neither function can develop alone.
To be concrete: a mid-sized EU FinTech running a credit scoring model under the EU AI Act's high-risk classification would assign a product manager as the monitoring mandate holder for that model, with a standing brief to the head of risk. The product manager produces a monthly one-page summary connecting the model's current performance metrics to the design-time parameters and to any relevant regulatory updates. The risk function reviews and countersigns. The document becomes the evidentiary chain. It takes both functions an hour a month to maintain. It takes a great deal more than that to reconstruct after a regulatory examination has identified it was missing.
Starting the Conversation Earlier
The accountability gap in regulated FinTech AI governance is not a failure of risk management or product development. Both functions are doing serious work. The gap arises because the two kinds of knowledge needed to govern AI responsibly, technical visibility over what the system is doing and regulatory depth over what the framework requires, have not yet been consistently connected into a joint governance practice.
Firms that close this gap will not do it by extending either function's remit into the other's territory. Risk managers are not product engineers. Product managers are not compliance specialists. What they can build together, starting from product ideation, is a shared governance routine that uses each function's existing expertise in the places where it is most needed, connects the two streams of information that AI governance requires, and produces the evidentiary record that demonstrates, to regulators and to themselves, that the governance held.
The EU AI Act's high-risk obligations for credit scoring now have a later, fixed deadline: 2 December 2027, not August 2026. That is not a reason to wait. It is a reason to start now, precisely because the routines described in this article take time to become genuine practice rather than paperwork, and because the delay itself was caused by an ecosystem that was not ready, a warning about how long readiness actually takes. For firms that have not yet built the joint governance routines described here, the right moment to start the conversation between risk and product is well before the system is deployed and well before the deadline, not after the regulator asks who owned it.
About the Author. Snehal Kalmegh is an independent AI governance researcher and the founder of 4iGov, a practitioner-focused research and advisory initiative on responsible AI governance (4igov.cloud). She brings over a decade of industry experience across banking, insurance, wealth management, and IT governance, including roles connected to HSBC, FTSE Russell, Credit Suisse, Edward Jones, and Zurich Financial Services. She is also engaged in doctoral research on AI governance in regulated FinTech environments.
Disclosure: Snehal Kalmegh is the founder of 4iGov, an AI governance research and advisory initiative. The analysis in this article is independent of that commercial activity.