Ethics vs. Safety is a False Choice, and Corporate AI Governance is Paying the Price

Most corporate AI governance teams are composed of legal, privacy, and compliance professionals on one side, and technical people focused on 𝗰𝘆𝗯𝗲𝗿𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆, 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲, and 𝗽𝗿𝗼𝗱𝘂𝗰𝘁 on the other.

As I said in a recent post: in most occasions, no one is purposely trying to build “unsafe” or “non-compliant” products, and you’ll often find that people take their jobs rather seriously.

These are competent professionals that specialized in delivering their part. The concepts aren’t beyond them either: the problem is 𝘁𝗶𝗺𝗲 𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝘁𝘀 and 𝘀𝗰𝗼𝗽𝗲 𝗺𝗶𝘀𝗺𝗮𝘁𝗰𝗵.

The security person is buried in security reviews and actual deliverables. The product team commissioned testing to other functions precisely because they own shipping and project management.

The architects need the panoramic view of product functionality and every component in it. And testing ownership varies wildly by industry and company, but it usually answers traditional questions heavily shaped by 𝗰𝗼𝗺𝗽𝗹𝗶𝗮𝗻𝗰𝗲 and 𝗾𝘂𝗮𝗹𝗶𝘁𝘆 𝗮𝘀𝘀𝘂𝗿𝗮𝗻𝗰𝗲:

Does the product work as intended? Is it fit for purpose? What do we need in place to prevent complaints about product functionality?

All of this is necessary and logical.

Then you have the other side: privacy, risk & compliance, legal and the newer 𝗔𝗜 𝗴𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 function (especially since IAPP popularized the AIGP certification for Data Protection and Privacy roles).

These professionals bring experience assessing products against different 𝗹𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗿𝗲𝗴𝗶𝗺𝗲𝘀, thinking about how things fail in practice, and applying mitigations. They carry a 𝗿𝗶𝘀𝗸 𝗺𝗶𝗻𝗱𝘀𝗲𝘁 by design.

The problem is we have a 𝗿𝗲𝘀𝗽𝗼𝗻𝘀𝗶𝗯𝗹𝗲 𝗔𝗜 𝗱𝗶𝘃𝗶𝗱𝗲 right now. People are pushing 𝗲𝘁𝗵𝗶𝗰𝘀 against 𝘀𝗮𝗳𝗲𝘁𝘆 and vice versa. And when you wore the “DPO” (data protection officer) hat for years, it’s easy to be “boxed into” trainings that teach the intersection between AI, Privacy and Human Rights.

When we think about ethics, we may think about 𝗮𝗹𝗴𝗼𝗿𝗶𝘁𝗵𝗺𝗶𝗰 𝗯𝗶𝗮𝘀, violations of 𝗳𝘂𝗻𝗱𝗮𝗺𝗲𝗻𝘁𝗮𝗹 𝗿𝗶𝗴𝗵𝘁𝘀, misuse, poor design. This is exactly why privacy professionals are strong contenders to own AI governance in companies. They already think in terms of risk mitigation against rights violations.

But if this is the only lens, you end up thinking about those problems 𝗶𝗻 𝗶𝘀𝗼𝗹𝗮𝘁𝗶𝗼𝗻.

If you also engage with safety, even though the field is disproportionately focused on LLMs and foundation models, you learn principles that go well beyond bias and IP.

You start understanding how AI companies apply 𝗿𝗲𝗶𝗻𝗳𝗼𝗿𝗰𝗲𝗺𝗲𝗻𝘁 𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗳𝗿𝗼𝗺 𝗵𝘂𝗺𝗮𝗻 𝗳𝗲𝗲𝗱𝗯𝗮𝗰𝗸 and other 𝗽𝗿𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁 methodologies, and what those companies consider human values to be. You learn that these training methodologies get documented in 𝗺𝗼𝗱𝗲𝗹 𝗰𝗮𝗿𝗱𝘀 and 𝘀𝘆𝘀𝘁𝗲𝗺 𝗰𝗮𝗿𝗱𝘀, which, if you read carefully, you can evaluate against downstream use cases and the products built on top of them.

You start understanding the 𝗸𝗻𝗼𝘄𝗻 𝘃𝘂𝗹𝗻𝗲𝗿𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝘀𝘂𝗿𝗳𝗮𝗰𝗲 of these models: prompt injection, jailbreaking, reward hacking, specification gaming. You can do better 𝗿𝗶𝘀𝗸 𝗺𝗼𝗱𝗲𝗹𝗶𝗻𝗴 and 𝘀𝘁𝗿𝗲𝘀𝘀 𝘁𝗲𝘀𝘁𝗶𝗻𝗴 of how harms may actually materialize in deployment, because you understand the safety layer, not just the compliance layer.

This doesn’t happen if your testing scope is limited to commissioning evaluations for violent speech or discriminatory outputs.

When you learn about safety, you understand how 𝗹𝗶𝘁𝘁𝗹𝗲 𝗰𝗼𝗻𝘁𝗿𝗼𝗹 𝗮 𝗱𝗲𝗽𝗹𝗼𝘆𝗲𝗿 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗵𝗮𝘀 over an LLM’s behavior.

You understand that how much you can mitigate downstream harms depends on the 𝗮𝗿𝘀𝗲𝗻𝗮𝗹 𝗼𝗳 𝗿𝗲𝘀𝗽𝗼𝗻𝘀𝗶𝗯𝗹𝗲 𝗔𝗜 𝗹𝗮𝘆𝗲𝗿𝘀 your company has built: guardrails, output filtering, retrieval architecture design, monitoring, human-in-the-loop escalation paths.

You stop treating the LLM as just one more interchangeable component. You start considering the 𝗿𝗶𝘀𝗸 𝗽𝗿𝗼𝗳𝗶𝗹𝗲 𝗼𝗳 𝘁𝗵𝗲 𝗲𝗻𝘁𝗶𝗿𝗲 𝗽𝗿𝗼𝗱𝘂𝗰𝘁 and how it shifts depending on model selection, because no matter how much you tune other elements, some product-level benchmarks are fundamentally 𝗯𝗼𝘁𝘁𝗹𝗲𝗻𝗲𝗰𝗸𝗲𝗱 𝗯𝘆 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹’𝘀 𝗼𝘄𝗻 𝗲𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 𝗯𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸𝘀.

You understand that there are 𝗵𝗮𝗿𝗱 𝗹𝗶𝗺𝗶𝘁𝘀 𝘁𝗼 𝗲𝘅𝗽𝗹𝗮𝗶𝗻𝗮𝗯𝗶𝗹𝗶𝘁𝘆 when LLMs sit in the architecture. You can observe and trace only so far. You start valuing 𝗰𝗵𝗮𝗶𝗻-𝗼𝗳-𝘁𝗵𝗼𝘂𝗴𝗵𝘁 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 once you are the person defining accountability frameworks. You don’t get that mindset shift if you stay within the lane of pure ethics and preventing violations of fundamental rights alone, because the 𝘁𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗱𝗲𝗽𝘁𝗵 simply doesn’t reach as far without complementing ethics with safety research.

Responsible AI needs 𝗯𝗼𝘁𝗵.

Responsible AI teams need to become 𝘁𝗿𝗮𝗻𝘀𝗹𝗮𝘁𝗼𝗿𝘀 𝗮𝗻𝗱 𝗲𝗻𝗮𝗯𝗹𝗲𝗿𝘀 between legal, compliance, ethics, architecture, product, and business development. That won’t happen if we keep structuring responsible AI departments as 𝗺𝗼𝗻𝗼𝗹𝗶𝘁𝗵𝘀.

This is why I was grateful to Bálint Gyevnár and Atoosa Kasirzadeh for their paper “Bridging the Gap in the Responsible AI Divides“, which maps the research landscapes of AI Ethics (AIE) and AI Safety (AIS) across 3,550 papers and makes the structural divide visible in data.

What the paper surfaces is exactly what I see playing out in corporate governance. The 𝗺𝗶𝘁𝗶𝗴𝗮𝘁𝗶𝗼𝗻 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗲𝘀 each field prioritizes tell the story. AIE focuses heavily on 𝗳𝗮𝗶𝗿𝗻𝗲𝘀𝘀, 𝗯𝗶𝗮𝘀 𝗱𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻, 𝗽𝗮𝗿𝘁𝗶𝗰𝗶𝗽𝗮𝘁𝗼𝗿𝘆 𝗺𝗲𝘁𝗵𝗼𝗱𝘀, 𝗮𝗻𝗱 𝗴𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 𝗿𝗲𝗰𝗼𝗺𝗺𝗲𝗻𝗱𝗮𝘁𝗶𝗼𝗻𝘀.

AIS focuses on 𝗮𝗱𝘃𝗲𝗿𝘀𝗮𝗿𝗶𝗮𝗹 𝗿𝗼𝗯𝘂𝘀𝘁𝗻𝗲𝘀𝘀, 𝗺𝗲𝗰𝗵𝗮𝗻𝗶𝘀𝘁𝗶𝗰 𝗶𝗻𝘁𝗲𝗿𝗽𝗿𝗲𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆, 𝗿𝗲𝘄𝗮𝗿𝗱 𝗺𝗼𝗱𝗲𝗹𝗶𝗻𝗴, 𝗮𝗻𝗱 𝗯𝗲𝗵𝗮𝘃𝗶𝗼𝗿𝗮𝗹 𝗺𝗼𝗻𝗶𝘁𝗼𝗿𝗶𝗻𝗴 like chain-of-thought inspection. Both fields study risk assessment and human preference elicitation, but they approach them from different angles and rarely cross-reference each other.

The paper identifies five major areas of overlap: 𝗵𝘂𝗺𝗮𝗻 𝘃𝗮𝗹𝘂𝗲 𝗮𝗻𝗱 𝗽𝗿𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴, 𝗮𝗱𝘃𝗲𝗿𝘀𝗮𝗿𝗶𝗮𝗹 𝗿𝗼𝗯𝘂𝘀𝘁𝗻𝗲𝘀𝘀, 𝗮𝘂𝗱𝗶𝘁𝗶𝗻𝗴 𝗮𝗻𝗱 𝗿𝗶𝘀𝗸 𝗮𝘀𝘀𝗲𝘀𝘀𝗺𝗲𝗻𝘁, 𝗺𝗮𝘁𝗵𝗲𝗺𝗮𝘁𝗶𝗰𝗮𝗹 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻𝘀, and 𝗲𝘅𝗽𝗹𝗮𝗶𝗻𝗮𝗯𝗶𝗹𝗶𝘁𝘆/𝗶𝗻𝘁𝗲𝗿𝗽𝗿𝗲𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗺𝗲𝘁𝗵𝗼𝗱𝘀. These overlaps exist in the research, but they are not translating into how corporate teams are actually structured or how they scope their work.

And this is the point. In a corporate responsible AI team, the person assessing a product against 𝗔𝗿𝘁𝗶𝗰𝗹𝗲 𝟱 𝗽𝗿𝗼𝗵𝗶𝗯𝗶𝘁𝗲𝗱 𝗽𝗿𝗮𝗰𝘁𝗶𝗰𝗲𝘀 or running a 𝗳𝘂𝗻𝗱𝗮𝗺𝗲𝗻𝘁𝗮𝗹 𝗿𝗶𝗴𝗵𝘁𝘀 𝗶𝗺𝗽𝗮𝗰𝘁 𝗮𝘀𝘀𝗲𝘀𝘀𝗺𝗲𝗻𝘁 is doing work that may map better to the AIE side.

But if they don’t also understand the safety side, they won’t ask whether the 𝗿𝗲𝘄𝗮𝗿𝗱 𝗺𝗼𝗱𝗲𝗹 driving the LLM’s behavior was trained on preference data that conflicts with the use case.

They won’t question whether 𝗼𝘂𝘁-𝗼𝗳-𝗱𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻 𝗶𝗻𝗽𝘂𝘁𝘀 in production could trigger failure modes that no bias evaluation would catch.

Your privacy team won’t understand why 𝗚𝗗𝗣𝗥 𝗱𝗮𝘁𝗮 𝗱𝗲𝗹𝗲𝘁𝗶𝗼𝗻 𝗿𝗲𝗾𝘂𝗲𝘀𝘁𝘀 are nearly undoable with LLMs, because they’ve never encountered what 𝗺𝗮𝗰𝗵𝗶𝗻𝗲 𝘂𝗻𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴 research or 𝗺𝗲𝗰𝗵𝗮𝗻𝗶𝘀𝘁𝗶𝗰 𝗶𝗻𝘁𝗲𝗿𝗽𝗿𝗲𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 actually say about how LLMs learn and store data.

They won’t know why most LLM providers have resorted to issuing standard 𝗜𝗣 𝗶𝗻𝗱𝗲𝗺𝗻𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀 in their Terms, not because ethics required it, but because there is a 𝘁𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗹𝗶𝗺𝗶𝘁𝗮𝘁𝗶𝗼𝗻 around removing data learned during training that no contractual clause can engineer away.

The reverse is also true. Safety-focused technical teams that ignore the ethics and governance literature will miss that 𝗽𝗮𝗿𝘁𝗶𝗰𝗶𝗽𝗮𝘁𝗼𝗿𝘆 𝗺𝗲𝘁𝗵𝗼𝗱𝘀 for stakeholder elicitation, 𝗮𝗹𝗴𝗼𝗿𝗶𝘁𝗵𝗺𝗶𝗰 𝗮𝘂𝗱𝗶𝘁𝗶𝗻𝗴 𝗳𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸𝘀, and 𝘀𝘂𝗽𝗽𝗹𝘆 𝗰𝗵𝗮𝗶𝗻 𝘁𝗿𝗮𝗻𝘀𝗽𝗮𝗿𝗲𝗻𝗰𝘆 requirements are not bureaucratic overhead. They are the mechanisms through which deployment risk actually gets governed.

The paper calls this constructive path “𝗰𝗿𝗶𝘁𝗶𝗰𝗮𝗹 𝗯𝗿𝗶𝗱𝗴𝗶𝗻𝗴.”

In corporate, the responsible AI function is where that bridge either gets built or doesn’t. If your team only hires from one tradition, you get a monolith.

If it hires from both but keeps them in separate lanes, you get 𝗰𝗼𝗺𝗽𝗮𝗿𝘁𝗺𝗲𝗻𝘁𝗮𝗹𝗶𝘇𝗲𝗱 𝗰𝗼𝗲𝘅𝗶𝘀𝘁𝗲𝗻𝗰𝗲, which the paper flags as insufficient.

The only version that works is the one where people are expected to develop literacy across both fields and translate between them for every other function in the business.

That is what responsible AI teams should be. Neither ethics police nor safety researchers. 𝗧𝗿𝗮𝗻𝘀𝗹𝗮𝘁𝗼𝗿𝘀 𝘄𝗶𝘁𝗵 𝗲𝗻𝗼𝘂𝗴𝗵 𝗱𝗲𝗽𝘁𝗵 𝗶𝗻 𝗯𝗼𝘁𝗵 𝘁𝗼 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗰𝗵𝗮𝗻𝗴𝗲 𝗵𝗼𝘄 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝘀 𝗴𝗲𝘁 𝗯𝘂𝗶𝗹𝘁

Beckett LeClair / https://betterimagesofai.org / https://creativecommons.org/licenses/by/4.0/.