GPU (Graphics Processing Unit)

A GPU (graphics processing unit) is a chip designed to perform many calculations simultaneously. Originally made for rendering graphics, its parallel design suits the matrix multiplication at the heart of neural networks.

Why it matters

Training large models on ordinary CPUs would take impractically long. GPUs and similar accelerators make it feasible, and their memory (VRAM) often sets the limit on which model size you can run. Availability and price shape the cost of inference services.

Example

A rough rule of thumb: a model's weights need memory equal to parameter count times bytes per parameter. A 7-billion-parameter model stored in 16-bit format needs about 14 GB for weights alone, before extra working memory.

7,000,000,000 params x 2 bytes = ~14 GB

Common mistakes

• Sizing hardware from the weights alone and forgetting activations and long-context memory.
• Using a GPU for tiny workloads where a CPU is cheaper and sufficient.
• Leaving expensive GPU instances running idle.

Related

• Deep learning
• Inference
• AI model
• Latency

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.