The only way we are going to figure out “what good looks like” in the context of technical AI safety is with real-world experience. You cannot purely think your way to safety, just as no one could have invented a cybersecurity ecosystem at the dawn of software. Anthropic’s public release of Fable had extremely constrained guardrails in some regards, and while I disagreed (vociferously) with some of their decisions, it is probably true that erring on the side of caution with a public release is prudent. It is also probably true that real-world experience with the guardrails would have given Anthropic the ability to rapidly iterate on them, improving usability without sacrificing safety. That iteration cannot happen in a vacuum; it needs real-world, large-scale human usage data.
Comments
David Oberlin
From the essay:
The only way we are going to figure out “what good looks like” in the context of technical AI safety is with real-world experience. You cannot purely think your way to safety, just as no one could have invented a cybersecurity ecosystem at the dawn of software. Anthropic’s public release of Fable had extremely constrained guardrails in some regards, and while I disagreed (vociferously) with some of their decisions, it is probably true that erring on the side of caution with a public release is prudent. It is also probably true that real-world experience with the guardrails would have given Anthropic the ability to rapidly iterate on them, improving usability without sacrificing safety. That iteration cannot happen in a vacuum; it needs real-world, large-scale human usage data.