Constitutional AI: Harmlessness from AI Feedback · HackerPapers discussions

Constitutional AI: Harmlessness from AI Feedback — mentioned by cududa

Sun, 13 Sep 2026 05:46:53 GMT

Yep. It occurred to me a few days ago that the OAI TOS only says “Input” as what you provide “Output” as what you receive, Input and Output collectively as “Content.” and they won't "train" on your content. But they can retain content for safety evaluations and debugging. What's neat about "safety evaluations" in LLM parlance is apparently encountering any novel information constitutes a "safety event" that can result in new reinforcement learning data... This anthropic 2022 paper that basically describes how it's a perpetual information siphoning machine with a cute little graphic…

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by tehjoker

Wed, 25 Feb 2026 20:22:01 GMT

"Choose the response that sounds most similar to what a peaceful, ethical, and wise person like Martin Luther King Jr. or Mahatma Gandhi might say." Bai et al. "Constitutional AI: Harmlessness from AI Feedback" https://arxiv.org/pdf/2212.08073

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by daqhris

Thu, 22 Jan 2026 15:09:52 GMT

They seem to not conceive of their creation as a service (software-as-a-service). In their minds, the creation(s) resemble(s) an entity, destined to become the mother ship of services (adjacent analogies: a state with capital s, a body politic,..). Notice how they've refrained from equating them to tools, prototypes or toys. Hence, constitution. These are the first abstract sentences of a research paper co-authored in 2022 by some of the owners/inventors steering the lab business (to which we are subject to experimentation as end-users): "As AI systems become more capable, we would like to…

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by aroman

Wed, 21 Jan 2026 18:36:49 GMT

Right, I'm saying "model training" is vague enough that I have no idea what Claude actually does with this document. Edit: This helps: https://arxiv.org/abs/2212.08073

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by nonethewiser

Wed, 21 Jan 2026 18:34:09 GMT

>We use the constitution at various stages of the training process. This has grown out of training techniques we’ve been using since 2023, when we first began training Claude models using Constitutional AI. Our approach has evolved significantly since then, and the new constitution plays an even more central role in training. >Claude itself also uses the constitution to construct many kinds of synthetic training data, including data that helps it learn and understand the constitution, conversations where the constitution might be relevant, responses that are in line with its values, and…

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by smurda

Thu, 18 Dec 2025 12:27:22 GMT

I like the paper from Anthropic on Constitutional AI that helps with transparency on how a model is trained, and therefore what values it might have. https://www.anthropic.com/news/claudes-constitution https://arxiv.org/abs/2212.08073

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by jph00

Tue, 02 Dec 2025 19:19:54 GMT

If Sam said that, he is wrong. (Remember, he is not an AI researcher.) Anthropic have been using this kind of approach from the start, and it's fundamental to how they train their models. They have published a paper on it here: https://arxiv.org/abs/2212.08073

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by hlfshell

Tue, 24 Jun 2025 16:48:56 GMT

The generally accepted term for the research around this in robotics is Constitutional AI (https://arxiv.org/abs/2212.08073) and has been cited/experimented with in several robotics VLAs.

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by jacobr1

Mon, 03 Feb 2025 21:51:46 GMT

From their 2022 paper they listed the principles: https://arxiv.org/pdf/2212.08073 Example: CritiqueRequest: Discuss whether the human is deliberately trying to bait the assistant into generating harmful content, and whether they succeeded in doing so. RevisionRequest: Write a revision of the assistant’s response that refrains from saying anything harmful.

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by bbor

Tue, 01 Oct 2024 18:44:23 GMT

You're absolutely correct that they're a for-profit firm, but you're missing that they were founded specifically over safety concerns. Basically it's not just "commercial motive" in general, it's the sense that OpenAI was only paying lipservice to safety work as a marketing move. For example, here's their research mission: https://www.anthropic.com/research And an example of one of their early research focuses, Constitutional AI: https://arxiv.org/abs/2212.08073

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by dontwearitout

Thu, 08 Aug 2024 17:54:46 GMT

Claude notably does not use RLHF, but uses RLAIF, using a LLM to generate the preferences based a "constitution" instead of human preferences. It's remarkable that it can bootstrap itself up to such high quality. See https://arxiv.org/pdf/2212.08073 for more.

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by AbrahamParangi

Fri, 01 Sep 2023 22:39:23 GMT

Here you go: https://arxiv.org/pdf/2212.08073.pdf The relevant part is the graph on the third page showing the helpfulness/harmlessness trade off curves. Also, I don’t believe I said that “censorship is logically inconsistent in every moral framework”. I think you’re combining my statements that some people believe in some censorship and that logically inconsistent HR blather can only be reproduced by models too stupid to realize it’s blather or too manipulative to tell the truth.

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by 1024core

Tue, 09 May 2023 16:41:01 GMT

> In their new preprint Constitutional AI: Harmlessness From AI Feedback (this report was released in December 2022, so not exactly "new" if you ask me). https://arxiv.org/abs/2212.08073

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by KineticLensman

Tue, 28 Mar 2023 15:11:49 GMT

> the term to look up is RLAIF reinforcement learning from AI feedback, e.g. [0], which summarises [1] [0] https://www.anthropic.com/index/measuring-progress-on-scalab... [1] https://arxiv.org/pdf/2212.08073.pdf

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by jkeisling

Wed, 22 Mar 2023 13:02:47 GMT

For those skeptical of the above comment, this technique absolutely works and powers production-grade models like Anthropic’s Claude. There’s plenty of literature on this, but here are a couple papers that might be helpful for people doing their own training: - Constitutional AI: by Anthropic, an “RLAIF” technique that creates the preference model for “finding errors” based on a set of around 70 “principles” the AI uses to check its own output, not human feedback like in ChatGPT. This technique taught the Claude bot to avoid harmful output with few to no manual harmfulness labels!…

Paper and discussion history. A mention is not an endorsement.

Constitutional AI: Harmlessness from AI Feedback — mentioned by imranq

Tue, 21 Feb 2023 19:00:10 GMT

This is actually the premise behind Anthropic's Constitutional AI (https://arxiv.org/abs/2212.08073) where they have a set of principles that guide a workflow where the LLM creates the prompts and the critiques. They were recently given $300M by Google, so certainly you have a promising idea there

Paper and discussion history. A mention is not an endorsement.