Kennisbank

When your role changes, and what that depends on

Many organizations assume that their role is fixed: they are the user of a system that a supplier has built, and the obligations that come with that are clear and permanent. That assumption is often correct, but not always. The role an organization has with respect to an AI system is not a fixed label. It is an outcome of what actually happens with the system, and that outcome can shift without a new contract being signed or a new purchase being made.

What the classification depends on

The classification of a role depends on a number of factual elements, not on how a supplier names the system or how an internal department positions the system internally. Relevant factors include, among others, who brought the system to market, who actually uses the system within their own process, and who has modified the system after delivery. It also matters under what name the system is presented externally, and whether an organization passes the system on to others without using it themselves. Together, these elements determine whether an organization acts as a provider, as a user, or as a party that falls somewhere in between.

The reason this is relevant for governance is that the role determines which obligations fall to which party. An organization that is only a user has different responsibilities than an organization that in fact acts as a provider. What exactly is expected of each role is described elsewhere; here the focus is on the mechanism that determines which role applies, and on identifying the moment when that mechanism produces a different outcome than before.

What changes when the situation changes

A role is not tied to a system for its entire lifespan. It is tied to what actually happens at a given moment. This means that the same organization can fall into a different role for the same system at two different points in time. A few situations in which this happens: a team that fine-tunes a purchased model on its own data, a department that relabels a system and offers it to other departments under its own name, or an organization that passes on a system originally intended for internal use to a customer or partner. In each of these cases, the actual role shifts, even though little changes on the outside.

What changes in practice is not abstract. It determines who is responsible for documentation, who must be able to demonstrate that a system does what it claims to do, and who must respond if something goes wrong. More on exactly what changes when a model is modified is described on the page about modifications to a model. That shift is one of the clearest examples of how a role can tip without a deliberate decision preceding it.

Why this is relevant for governance

The reason this topic is relevant not only legally but also organizationally is that a change of role often happens unnoticed. A team that modifies a model thinks of a technical improvement, not a change in responsibility. A department that passes a tool on to another department thinks of convenience, not a new role as provider. Governance that only looks at what has been purchased systematically misses these shifts.

This is connected to the question of exactly which applications fall under a regulation and which do not; that boundary is addressed on the page about the scope of applications. A system that fell outside scope when it was purchased can come within scope once the organization's role shifts. That means it is not enough to assess this once at the time of purchase; the assessment must move along with what actually happens with a system.

The risk level of an application can also shift along with a change of role. What a high risk level exactly means for an organization's obligations, and what that depends on, is explained on the page about the consequences of a high risk level. An organization that, due to a change of role, suddenly qualifies as the provider of a system with a high risk level carries different responsibilities than it did when it was only a user of that same system.

How an organization identifies this

To identify role changes before they become a problem, an organization must know which systems exist, who modifies them, and who passes them on to whom. This requires an inventory that goes beyond the list approved by IT, and a fixed place where decisions about modification, passing on, and relabeling are recorded. Such a fixed place, and the questions that go with it, are described on the page about an oversight decision log.

The Responsible AI Scan is built to make these shifts visible: which systems are actually running, who uses them, who has modified them, and which role follows from that at this moment. The tool that supports this is under construction; anyone who wants to make use of it can sign up for the waiting list.

A change of role does not only affect governance. If a system shifts from a loose tool to something that structurally takes over part of a task, the question of how much work that system actually performs and how much still lies with people also changes. The work scan of FTE TO AI calculates, per task, what portion of the work can be taken over by AI, making visible where a change of role in governance coincides with an actual shift in the division of work.