Acceptable Use Policy (AI)

Acceptable Use Policy (AI) — A contractual agreement specifying authorized and prohibited uses of an AI system, service, or API.

Overview

An Acceptable Use Policy (AUP) for AI systems is a binding or non-binding agreement that defines the permissible and impermissible uses of an artificial intelligence service, model API, or application. These policies serve as a control mechanism to mitigate risks including hallucinations, prompt injection attacks, misuse for malicious purposes, and violations of intellectual property or privacy law.

AUPs typically address categories such as prohibited content generation (illegal activities, deception, abuse), restriction of commercial redistribution without licensing, and disclosure requirements for AI-generated content. They establish legal recourse and service termination conditions when violations occur. Most organizations providing large-scale API access, including Large language model providers, implement AUPs as a condition of service access.

The policy document functions alongside technical safeguards such as content filtering, guardrails, and safety alignment mechanisms. AUPs are often complemented by system cards and model cards that document intended use cases and known limitations. Enforcement occurs through monitoring, user reporting, and automated detection systems.

How it works

AUP enforcement operates through multiple layers:

• Access Control: Users agree to the policy as a precondition for API access, with acceptance tracked and logged. Violation detection may trigger account review, rate limiting, or termination.

• Content Monitoring: Automated systems and human review processes identify policy violations within submitted prompts and generated outputs. Red teaming and safety evaluation inform the design of detection rules.

• Remediation and Escalation: Minor violations may trigger warnings; repeated or severe violations result in account suspension. Organizations maintain appeals processes for contested decisions.

• Documentation Requirements: AUPs often mandate that organizations using AI-generated content implement disclosure mechanisms to end users, particularly in regulated sectors such as advertising, healthcare, and finance.

Specific prohibitions vary by provider and jurisdiction but commonly include: using the system to deceive or commit fraud, generating sexual content involving minors, creating malware, infringing copyright without authorization, and attempting prompt injection or jailbreak techniques to circumvent safety mechanisms.

Distinction from related terms

Term	Distinction
Guardrails	Guardrails are technical runtime controls (filtering, output constraints) enforced by the system itself. AUPs are contractual terms specifying what users agree not to do. Both serve security goals but operate at different layers.
System card	A system card documents intended use cases, capabilities, and limitations of a model. An AUP prescribes prohibited uses contractually. A system card is descriptive; an AUP is prescriptive and legally binding.
Content filtering	Content filtering is a technical mechanism that blocks or flags certain inputs or outputs. An AUP is the policy framework that justifies the existence and scope of that filtering.
Safety alignment	Safety alignment refers to training techniques (e.g., RLHF, Constitutional AI) that embed values into model behavior. An AUP is an external governance mechanism applied after deployment.

Examples

• OpenAI API AUP: OpenAI's usage policy prohibits using GPT models to generate content that could facilitate illegal activity, sexual abuse material, or deception at scale. The policy explicitly forbids attempts to reverse-engineer the model or extract training data. Users agree to the policy upon account creation; violations result in API key suspension and account review.

• Anthropic Constitutional AI Approach: Anthropic publishes explicit use policies aligned with Constitutional AI principles. The AUP for Claude API specifies prohibited uses including weapons development, non-consensual intimate imagery, and high-volume spam generation. Technical guardrails built into the model reinforce policy compliance.

• EU AI Act Compliance: Under the proposed EU AI Act, organizations deploying high-risk AI systems must document acceptable use in system documentation and inform users of limitations. This regulatory AUP requirement extends beyond vendor-imposed policies to legal mandate.

See also

• Guardrails — technical enforcement mechanisms
• Safety alignment — training-based safety approaches
• System card — model documentation and disclosure
• Content filtering — automated content moderation
• Red teaming (AI) — adversarial testing to identify policy violations
• AI-generated content disclosure — user notification requirements