Big Data

Big data refers to data collections whose volume, velocity (arrival speed), and variety exceed what a conventional database or spreadsheet on one machine can handle comfortably. Handling them usually means spreading storage and computation across many servers.

Why it matters

Clickstreams, logs, sensor feeds, and transaction records can reach billions of rows. Analyzing them reveals trends invisible in samples, but the infrastructure adds cost and complexity. There is no official size cutoff; what counts as "big" depends on your tools.

Example

A site logging every request might produce:

50 million requests/day x ~500 bytes = ~25 GB/day
~9 TB per year of raw logs

Querying that with distributed engines over a data lake or data warehouse is routine; opening it in a spreadsheet is not.

Common mistakes

• Adopting big data platforms for data that fits comfortably in one PostgreSQL or SQLite database.
• Collecting everything with no plan to use it, then paying to store it.
• Neglecting quality and governance as volume grows.

Related

• Data lake
• Data warehouse
• Data pipeline
• Data science

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.