AI Alignment

AI alignment is the challenge of ensuring an AI system's behavior matches human intentions, values, and safety expectations, including in situations its designers did not anticipate. It covers both everyday behavior, such as following instructions and refusing harmful requests, and longer-term concerns about powerful systems.

Why it matters

A model optimized for a poorly specified goal can satisfy the letter of the target while missing its purpose. As AI agents gain the ability to act, misaligned behavior has real consequences, so alignment shapes how models are trained, tested, and restricted.

Example

A common technique is reinforcement learning from human feedback (RLHF): people rank model answers, a reward model learns those preferences, and the language model is tuned to produce preferred responses. Other approaches use written principles or automated feedback.

Common misconceptions

• Alignment only means content filtering. It also includes honesty, robustness, and oversight.
• It is solved once a model passes a safety test. Behavior can shift with new prompts and tools.
• It is purely science fiction. Today's models already show alignment problems like flattery or hallucination.

Related

• Large language model
• AI agent
• Algorithmic bias
• Fine-tuning

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.