Data Cleaning

Data cleaning, also called data cleansing or wrangling, is the work of correcting or removing inaccurate, incomplete, duplicated, or badly formatted records so a dataset is reliable.

Why it matters

Analysts commonly spend a large share of project time on cleaning, because flawed inputs produce flawed conclusions and weak models ("garbage in, garbage out"). Cleaning is also a major defence against hidden bias. Doing it systematically, with scripts rather than manual edits, makes the work repeatable.

Example

Typical fixes, shown in pandas:

df = df.drop_duplicates()
df["country"] = df["country"].str.strip().str.upper()
df["age"] = df["age"].fillna(df["age"].median())
df = df[df["price"] > 0]

Each step should be recorded so the result is reproducible.

Common mistakes

• Overwriting the raw data instead of keeping the original untouched.
• Filling missing values blindly; the fact something is missing can itself be informative.
• Removing outliers that are real events, such as a genuine traffic spike.

Related

• Dataset
• ETL
• Training data
• CSV

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.