ETL (Extract, Transform, Load)

ETL stands for extract, transform, load. Data is pulled from source systems, cleaned and reshaped (types fixed, duplicates removed, fields joined), then written to a destination, usually a data warehouse.

Why it matters

Source systems store data in different formats and quality levels. ETL gives analysts one consistent dataset. A related pattern, ELT, loads raw data first and transforms it inside the warehouse, which modern cloud warehouses make practical. Documenting each transformation makes numbers easier to trust and debug.

Example

A nightly job:

Extract:   SELECT * FROM orders WHERE updated_at > :last_run
Transform: convert currency to USD, drop test orders
Load:      INSERT into warehouse.fact_orders

If this job silently stops, dashboards go stale, so scheduled jobs are worth watching with cron monitoring.

Common mistakes

• Jobs that are not idempotent, so a rerun creates duplicate rows.
• No alerting on failure or on unexpectedly low row counts.
• Transformations buried in code with no documentation.

Related

• Data pipeline
• Data warehouse
• Data cleaning
• Heartbeat monitoring

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.