Frontier AI Safety Frameworks: What Product Managers Must Know When Building on Powerful Models
TL;DR
On September 3, 2026, OpenAI released GPT-6 Astra — the first model to cross their "Critical" cybersecurity capability threshold. Anthropic's ASL-3 safety level has been active since 2025. These are not abstract policy documents. They govern access controls, what models can do by default, how fast providers release capabilities, and what guardrails your product inherits when you build on top of these models. This guide explains how the major safety frameworks work, what they mean for the products you build, and how to make safety-framework-aware product decisions.
The AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.
The Critical Threshold: What Happened with GPT-6 Astra
OpenAI's Preparedness Framework, introduced in 2023, defines four risk levels for frontier model capabilities: Low, Medium, High, and Critical. The Critical threshold for cybersecurity is met when a model can "identify and develop functional zero-day exploits of all severity levels in many hardened real-world critical systems without human intervention." GPT-6 Astra, released September 3, 2026, is the first model OpenAI has deployed that meets this threshold.
OpenAI delayed the Astra release by several weeks — following a security incident in July 2026 — to implement additional safeguards before deployment. The cybersecurity-sensitive capabilities are gated behind a Trusted Access Program (the "Daybreak" cohort), not available to all API developers. This is a direct consequence of the Critical threshold designation.
What this means for AI PMs building on Astra
You are now building on a model that has Critical-level cybersecurity capability by design. That does not mean your product is dangerous — it means the guardrails your product relies on are more critical than ever. The model's default safe behavior is a product decision from OpenAI, not a technical constraint. And policy decisions can change. Building a robust application-level safety layer — not just relying on provider-level defaults — is now a product requirement, not an option.
How the Major Safety Frameworks Work
There are two dominant safety frameworks in frontier AI: OpenAI's Preparedness Framework and Anthropic's Responsible Scaling Policy (RSP). Both define capability thresholds at which additional safeguards must be in place before a model can be deployed. The frameworks have different structures but the same goal: prevent a model from being deployed with dangerous capabilities before adequate mitigations exist.
OpenAI Preparedness Framework
Levels: Low, Medium, High, Critical
Domains evaluated: Cybersecurity, CBRN (chemical/biological/radiological/nuclear), persuasion, and model autonomy
Deployment rules: Models rated Medium or above in any domain require enhanced mitigations. Models rated Critical in cybersecurity are gated to a Trusted Access Program. The framework governs what API access tiers can reach which capabilities.
PM implication: When you get API access to a new OpenAI model, the capability envelope your API key can reach depends on your access tier. Cybersecurity-adjacent features you want to build may require joining the Trusted Access Program. Plan for this in your roadmap timeline.
Anthropic Responsible Scaling Policy (RSP)
Levels: ASL-1, ASL-2, ASL-3, ASL-4
Domains evaluated: CBRN uplift, autonomous replication, cyberoffense, catastrophic deception
Deployment rules: ASL-1 is current non-frontier models. ASL-2 is current frontier models (Claude Opus 5, Fable 5.1). ASL-3 triggers when a model can provide meaningful uplift to nation-state level threats — requiring stricter access controls, internal red-teaming, and security procedures before deployment. ASL-4 would require models to be able to contribute to their own training or autonomous replication, and currently has no deployed instances.
PM implication: Anthropic's models are currently at ASL-2. If a future Claude model hits ASL-3 thresholds, API access may be restricted, output filters may become stricter, and certain use cases may require enterprise-level agreements. Track Anthropic's RSP announcements the same way you track model release notes.
What Safety Frameworks Mean for PMs Building Products
Safety frameworks affect your product in four concrete ways. Most PMs do not encounter these until they hit them in production. Understanding them ahead of time prevents costly roadmap surprises.
Capability gating
Some model capabilities are only available at higher access tiers. If your product requires agentic capabilities, computer use, or domain-specific outputs in regulated areas, you may need an enterprise agreement or Trusted Access Program membership before your feature can ship.
Default refusal behavior
Provider safety frameworks drive the model's default refusal settings. A model rated High in a domain has stricter default content filters for that domain. Your product may inherit refusals that your users experience as bugs — but that are actually safety policy. Understanding which framework decisions drive which behaviors saves weeks of debugging.
Deployment delays
OpenAI delayed Astra by weeks because of a Critical threshold finding. If you are building a feature that depends on a specific model release, framework-driven delays can move your launch date. This risk is not in most PMs' roadmap assumptions.
Compliance requirements
Enterprise customers — especially in regulated industries — increasingly ask about providers' safety frameworks in procurement. Having a clear answer about which models you use, at what safety level, and what application-level guardrails you add is becoming a sales prerequisite.
Navigate the Frontier Model Era in the Masterclass
The AI PM Masterclass covers model selection, safety frameworks, and how to build products that survive the rapidly shifting frontier. Taught live by a Salesforce Sr. Director PM.
The Safety Tradeoffs You Own as a Product Manager
When you build on top of a frontier model, you inherit a safety stack from the provider. But provider-level safety is not sufficient for most products. You own a second layer — the application-level safety decisions that determine how the model behaves inside your product context. These decisions have real tradeoffs.
Strictness vs. utility
The more you restrict model behavior in your application context, the safer your product is but the less useful it becomes. A legal research tool that refuses to discuss litigation strategy is safe but useless. Finding the right restriction level for your specific user base and risk profile is a product decision, not a technical default.
Speed vs. safety review
Adding an application-level safety review pass — a classifier, a rule-based filter, a human review queue — adds latency and cost. The question is not whether to have a review layer but what the acceptable latency and false-positive rate are for your product's use case.
Transparency vs. user experience
When your product refuses a request for safety reasons, how do you communicate that? Saying 'this request was blocked by our safety system' is transparent but breaks the UX. Saying 'I cannot help with that' is seamless but opaque. High-stakes domains (healthcare, legal, financial) typically require more transparency, not less.
Model lock-in vs. safety flexibility
Different models have different safety profiles. GPT-6 Astra, Claude Fable 5.1, and Gemini 3.8 Flash have different default behavior for the same input. If your product's safety architecture is tightly coupled to one provider's defaults, a provider policy change can break your safety guarantees without a code change. Model-agnostic application-level safety layers reduce this risk.
How to Build a Model-Safety Policy for Your Product
A model-safety policy is the document that defines how your product handles the safety tradeoffs above. It is not an ethics policy — it is an operational specification. It should be one page, version controlled, and reviewed on every model upgrade.
1. Catalog your use cases by risk level
List every task your product asks the model to perform. Rate each task Low, Medium, or High risk based on potential for harm if the model responds incorrectly or is misused. Use this catalog to determine where you need the most restrictive safety settings.
2. Define your application-level guardrails
For each High-risk task, specify your application-level safety mechanism: input filter, output classifier, human-review queue, or hard refusal. Document the mechanism, not just the intent. 'We check for harmful content' is not a guardrail. 'All outputs in category X pass through classifier Y with threshold Z' is a guardrail.
3. Map provider defaults to your requirements
For each model you use or plan to use, document what the provider's default safety behavior is for your use cases. Run test inputs across your risk catalog and record the default model behavior. This is your baseline — any difference from your required behavior is a gap your application layer must fill.
4. Build a model-change protocol
When your provider updates the model or changes safety policy, you need a repeatable process for re-running your safety test set and comparing outputs. Make this a launch gate. If a model update changes safety behavior in your use cases, it is a breaking change — treat it like one.
5. Review on every safety-relevant provider announcement
Set a Google Alert for '[your provider] safety' and '[your provider] policy.' When OpenAI releases a Preparedness Framework update or Anthropic publishes a new RSP version, review it within 48 hours and determine if it changes your product's safety posture. Add this review to your team calendar as a recurring lightweight process.
The competitive angle most PMs miss
Enterprise procurement teams increasingly ask AI vendors about their safety framework compliance. Having a documented model-safety policy — and being able to articulate your provider's safety level for the models you use — is becoming a sales advantage, not just a risk management practice. The companies that are ahead of this conversation close enterprise deals faster.
Build Responsibly on the Frontier
The AI PM Masterclass covers the full model selection and safety decision framework — from reading provider safety documentation to building application-level guardrails that survive model updates.
Related Articles
Before you go: get the AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.