Transformer (AI Architecture)

The transformer is a neural network design introduced in the 2017 paper "Attention Is All You Need". Its key mechanism, self-attention, lets each token in a sequence look at every other token and decide which matter most for its meaning.

Why it matters

Transformers process sequences in parallel, which trains efficiently on GPUs and scales to very large models. They underpin LLMs, many image and audio models, and translation systems. The design also made it practical to pre-train one large model and adapt it to many tasks.

Example

In the sentence "The server crashed because it ran out of memory", attention helps the model link "it" to "server" rather than "memory". Real models stack many attention layers, each with multiple heads that capture different relationships.

Common mistakes

• Confusing the electrical device or the toy line with the architecture; the name is unrelated.
• Assuming the cost of attention is constant. It grows quickly with sequence length, which is one reason the context window is limited.
• Believing every modern AI model is a transformer. Other architectures remain in use.

Related

• Neural network
• Deep learning
• Large language model
• Token

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.