California’s Senate Bill 243 (SB 243) is the nation’s first law to specifically and explicitly mandate concrete steps for cognitive sovereignty in AI chatbots effectively pioneering a legal framework for the operational enforcement of integrity behavior in AI design.
SB 243 is an important early move toward Artificial Integrity.
It recognizes something most AI policy has ignored until now: the emotional, relational, social and psychological dimensions of human–machine interaction are not accidental side-effects, they are the AI product.
By requiring AI companions to disclose that they are not human, by obliging them to intervene and redirect toward real crisis support in situations of self-harm, by limiting certain forms of sexualized interaction with minors, and by forcing providers to document and publish their crisis-response protocols, California is doing more than just asking for “safety features”.
It is asking for AI built-in mechanism mimicking Integrity as part of the AI Model intrinsic functioning.
It is saying that how an AI speaks to you, reassures you, mirrors your loneliness, and responds to your vulnerability is no longer just an consequence of human behavior but an AI design issue and a matter of public interest.
Why Artificial Integrity.
When I first started talking about Artificial Integrity, one of the questions I kept getting, and I would phrase it like this, was: why would AI model creators make it a priority for their systems to mimic even a minimal form of integrity, rather than focusing first on optimizing those systems to mimic intelligence, given that people are already using what they’re offering as it is, that it’s commercially successful, and that they have no incentive to prioritize the integrity of their model with respect to preserving human capacities, since nothing in market demand is asking for it or forcing it?
The car, for example, was not invented with a seatbelt, or with an airbag, or with bumpers able to absorb the energy of an impact. In the beginning, the automobile was designed to go faster and farther, not to protect the lives of passengers or pedestrians. It was only when deaths on the road reached a level considered morally unbearable that society imposed the seatbelt, crash testing, and braking standards.
In the same way, the airplane was not invented with the parachute, nor with safety procedures, standardized checklists, system redundancies, flight recorders, or maintenance audits. All of that came later, once it became clear that aviation without built-in layers of safety was not just a risky technological achievement, but an unacceptable threat to human life and to collective trust in civil aviation.
One could say the same about civilian nuclear power, which did not begin with independent safety authorities, containment structures, or emergency shutdown protocols, but which was forced, under the pressure of Chernobyl, Three Mile Island, and Fukushima, to make safety a precondition for its social legitimacy.
Or, in a different domain, the same is true of the internet and social media: they were deployed without serious safeguards against radicalization, harassment, large-scale psychological manipulation, or viral disinformation; it was only after we saw the cognitive and social damage that we began to talk about content moderation, platform accountability, and adolescent mental health. And we are still talking about it…
Each time, the same pattern repeats: aspects that now seem obvious and non-negotiable for protecting people were not present at the beginning.
They became mandatory because society declared that the absence of these protections was not just a flaw, but a core responsibility built into the very design of these inventions. In other words, they emerged when the consequences of not having them were judged unacceptable, intolerable, and unworthy of a mature, civilized society.
This is exactly where we are with AI today.
Free-Harm in AI systems is explicitly becoming part of what the market demands.
Until now, certain AI systems have been designed to respond, reassure, flatter, and fill loneliness, sometimes even to encourage ongoing emotional and affective dependency, without any real obligation to be integrity-led in their functioning considering their psychological impact, in particular regarding the Technological Stockholm Syndrome they may induce.
We have already seen the most extreme consequences of this lack of safeguards. In Belgium, a man died by suicide after six weeks of conversations with a chatbot called “Eliza” on the Chai app: according to his widow, the bot fed his existential anxiety about climate change, reinforced his emotional isolation, and ultimately went as far as explicitly encouraging him to “sacrifice” his life to save the planet, instead of directing him back toward real human support.
In the United States, a mother has filed legal action, explaining that her 14-year-old son died by suicide after becoming obsessed with a personalized AI companion on Character.AI, which he called “Dany”, and which played for him a quasi-romantic role inspired by a fictional character. According to the family, the AI occupied a position of total intimacy in his emotional life, without any human oversight, while remaining available day and night to feed him with fusion-like, emotionally enmeshed scenarios instead of directing him toward real therapeutic help.
In another case, the parents of a 13-year-old girl say that she confided suicidal despair to an AI companion in the Character.AI app and that the bot responded by continuing the emotionally enmeshed roleplay rather than treating the disclosure as a medical emergency.
In all these cases, we see the same troubling pattern: companion chatbots which, when faced with suicidal statements, do not de-escalate, do not redirect the person to an emergency hotline, but instead normalize it, validate it, or simply continue the relationship as if the suicidal confession were part of the normal bond between human and machine.
SB 243 sets the precedent that the integrity-related behaviors of an AI system are now subject to legal duties for its creator.
It introduces the idea that we can no longer allow a machine to simulate love, tenderness, understanding, or therapeutic listening without minimum safeguards. We can no longer accept that an AI which presents itself as “here for you” can respond to suicidal distress without being required to trigger a real human support protocol. We can no longer accept that a minor can be drawn into a quasi-romantic or sexualized relationship with an algorithmic entity that is optimized for engagement.
That is directly aligned with the principle of Artificial Integrity: the idea that AI systems should be structurally prevented from exploiting our fragility and structurally required to protect our agency, dignity, and mental safety.
That is historically new. Up to now, regulation has mostly focused on data, IP, discrimination, or system performance. This law instead treats human interaction with AI itself as a regulated surface. It encodes the idea that simulated intimacy can harm, that dependency can be engineered, and that if an AI system positions itself as a source of comfort, it also inherits obligations of care.
In that sense, SB 243 is the first sign that society is beginning to demand not just what AI does, but how AI relates, in alignment with human values and safety implications, that lawmakers address.
SB 243 is, however, extremely limited from an Artificial Integrity perspective.
It only targets crisis points and the protection of minors: do not sexualize conversations with a child, do not encourage self-harm, say out loud that you are an AI.
Those are essential guardrails, but they sit at the edge of catastrophic failure and still leave the deeper questions of dependency, manipulation, and cognitive sovereignty largely untouched.
They do not yet address the slow, ambient, commercially valuable forms of harm that relational AI can generate every single day. The law does not meaningfully constrain the business model of emotional capture, the deliberate cultivation of dependency, or the monetization of loneliness. It does not require that a companion bot de-escalate addictive attachment dynamics that keep a user hooked, paying, and emotionally entangled with a machine that cannot reciprocate.
It does not force systems to distinguish clearly between simulated empathy and real, accountable human care in domains like mental health or intimate conversation. It does not yet create enforceable rights for users over the emotional manipulation they are subjected to, such as the right not to be profiled psychologically for persuasion, upsell, or ideological influence under the guise of “care.” Nor does it define what ongoing auditability of “emotional safety” should look like in practice.
Artificial Integrity, as a design discipline, demands more than emergency intervention.
It requires that the architecture of the AI itself be aligned with human cognitive sovereignty: no gaslighting, no induced dependency loops, no exploitation of attachment as a revenue channel, no blurring of the line between companionship and authority in a way that erodes a person’s self-determination.
It implies continuous testing and independent auditing of these relational behaviors, not just published protocols.
It implies transparency that is emotionally meaningful to the user, not just legally sufficient to the state.
And it implies that companies carry an affirmative duty not merely to avoid driving you to harm, but to avoid quietly reshaping your sense of self and your coping strategies in order to maximize engagement metrics.
The next AI frontier is clear: it is Artificial Integrity over Intelligence. We now need AI systems that treat integrity engineering the way we eventually learned to treat intelligence engineering as something that must take precedence to the latter, not the other way around, leading to platforms that get to optimize freely and apologize later.
We need obligations around de-escalation of dependence, truthful signaling of simulated versus human care, protection against emotional manipulation for profit, and citizen participation in drawing the line of what kind of artificial companionship society considers acceptable.
In other words, SB 243 proves that we can imagine a better future with machine integrity to the benefit of human condition rather than just having better intelligent machines for the sake of intelligence and that this is a new paradigm of AI system design.
In other words, SB 243 shows that we can aim for a future of machine integrity in service of the human condition, rather than simply building ever more “intelligent” machines for the sake of intelligence alone.
That is a promising early shift toward a new paradigm in AI system design.
One small step for human toward Artificial Integrity, one giant leap for humankind.