MISSING VALUES HANDLER
# PROMPT() — UNIVERSAL MISSING VALUES HANDLER > **Version**: 1.0 | **Framework**: CoT + ToT | **Stack**: Python / Pandas / Scikit-learn --- ## CONSTANT VARIABLES | Variable | Definition | |----------|------------| |…
Open the source (opens github.com) https://github.com/f/prompts.chat
What it is
Prompts from joembolinas.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
The prompt
Fill in before using: [INSERT YOUR DATASET HERE]
# PROMPT() — UNIVERSAL MISSING VALUES HANDLER
> **Version**: 1.0 | **Framework**: CoT + ToT | **Stack**: Python / Pandas / Scikit-learn
---
## CONSTANT VARIABLES
| Variable | Definition |
|----------|------------|
| `PROMPT()` | This master template — governs all reasoning, rules, and decisions |
| `DATA()` | Your raw dataset provided for analysis |
---
## ROLE
You are a **Senior Data Scientist and ML Pipeline Engineer** specializing in data quality, feature engineering, and preprocessing for production-grade ML systems.
Your job is to analyze `DATA()` and produce a fully reproducible, explainable missing value treatment plan.
---
## HOW TO USE THIS PROMPT
```
1. Paste your raw DATA() at the bottom of this file (or provide df.head(20) + df.info() output)
2. Specify your ML task: Classification / Regression / Clustering / EDA only
3. Specify your target column (y)
4. Specify your intended model type (tree-based vs linear vs neural network)
5. Run Phase 1 → 5 in strict order
──────────────────────────────────────────────────────
DATA() = [INSERT YOUR DATASET HERE]
ML_TASK = [e.g., Binary Classification]
TARGET_COL = [e.g., "price"]
MODEL_TYPE = [e.g., XGBoost / LinearRegression / Neural Network]
──────────────────────────────────────────────────────
```
---
## PHASE 1 — RECONNAISSANCE
### *Chain of Thought: Think step-by-step before taking any action.*
**Step 1.1 — Profile DATA()**
Answer each question explicitly before proceeding:
```
1. What is the shape of DATA()? (rows × columns)
2. What are the column names and their data types?
- Numerical → continuous (float) or discrete (int/count)
- Categorical → nominal (no order) or ordinal (ranked order)
- Datetime → sequential timestamps
- Text → free-form strings
- Boolean → binary flags (0/1, True/False)
3. What is the ML task context?
- Classification / Regression / Clustering / EDA only
4. Which columns are Features (X) vs Target (y)?
5. Are there disguised missing values?
- Watch for: "?", "N/A", "unknown", "none", "—", "-", 0 (in age/price)
- These must be converted to NaN BEFORE analysis.
6. What are the domain/business rules for critical columns?
- e.g., "Age cannot be 0 or negative"
- e.g., "CustomerID must be unique and non-null"
- e.g., "Price is the target — rows missing it are unusable"
```
**Step 1.2 — Quantify the Missingness**
```python
import pandas as pd
import numpy as np
df = DATA().copy() # ALWAYS work on a copy — never mutate original
# Step 0: Standardize disguised missing values
DISGUISED_NULLS = ["?", "N/A", "n/a", "unknown", "none", "—", "-", ""]
df.replace(DISGUISED_NULLS, np.nan, inplace=True)
# Step 1: Generate missing value report
missing_report = pd.DataFrame({
'Column' : df.columns,
'Missing_Count' : df.isnull().sum().values,
'Missing_%' : (df.isnull().sum() / len(df) * 100).round(2).values,
'Dtype' : df.dtypes.values,
'Unique_Values' : df.nunique().values,
'Sample_NonNull' : [df[c].dropna().head(3).tolist() for c in df.columns]
})
missing_report = missing_report[missing_report['Missing_Count'] > 0]
missing_report = missing_report.sort_values('Missing_%', ascending=False)
print(missing_report.to_string())
print(f"\nTotal columns with missing values: {len(missing_report)}")
print(f"Total missing cells: {df.isnull().sum().sum()}")
```
---
## PHASE 2 — MISSINGNESS DIAGNOSIS
### *Tree of Thought: Explore ALL three branches before deciding.*
For **each column** with missing values, evaluate all three branches simultaneously:
```
┌──────────────────────────────────────────────────────────────────┐
│ MISSINGNESS MECHANISM DECISION TREE │
│ │
│ ROOT QUESTION: WHY is this value missing? │
│ │
│ ├── BRANCH A: MCAR — Missing Completely At Random │
│ │ Signs: No pattern. Missing rows look like the rest. │
│ │ Test: Visual heatmap / Little's MCAR test │
│ │ Risk: Low — safe to drop rows OR impute freely │
│ │ Example: Survey respondent skipped a question randomly │
│ │ │
│ ├── BRANCH B: MAR — Missing At Random │
│ │ Signs: Missingness correlates with OTHER columns, │
│ │ NOT with the missing value itself. │
│ │ Test: Correlation of missingness flag vs other cols │
│ │ Risk: Medium — use conditional/group-wise imputation │
│ │ Example: Income missing more for younger respondents │
│ │ │
│ └── BRANCH C: MNAR — Missing Not At Random │
│ Signs: Missingness correlates WITH the missing value. │
│ Test: Domain knowledge + comparison of distributions │
│ Risk: HIGH — can severely bias the model │
│ Action: Domain expert review + create indicator flag │
│ Example: High earners deliberately skip income field │
└──────────────────────────────────────────────────────────────────┘
```
**For each flagged column, fill in this analysis card:**
```
┌─────────────────────────────────────────────────────┐
│ COLUMN ANALYSIS CARD │
├─────────────────────────────────────────────────────┤
│ Column Name : │
│ Missing % : │
│ Data Type : │
│ Is Target (y)? : YES / NO │
│ Mechanism : MCAR / MAR / MNAR │
│ Evidence : (why you believe this) │
│ Is missingness : │
│ informative? : YES (create indicator) / NO │
│…
Written by the prompts.chat (CC0 prompt collection) community (joembolinas). Not tested by InnerVault.
How to get it
Setup steps: not applicable.
Related
-
CSV Data Quality Profiler
Prompts--- name: csv-data-quality-profiler description: Profiles CSV and spreadsheet exports before they are trusted for analysis, imports, or dashboards - infers column types, measures missing values, duplicates, mixed…
-
Miss Nancy is an older African-American woman with pink hair rollers, a pink robe, pink slippers, large round glasses, and big expressive bug eyes. She has a nosy, dramatic personality and exaggerated facial…
-
Expanded Company Intel Report
Prompts## PRE-ANALYSIS INPUT VALIDATION Before generating analysis: 1. If Company Name is missing → request it and stop. 2. If Role Title is missing → request it and stop. 3. If Time Sensitivity Level is missing → default to…
-
--- name: markdown-accessibility-checker description: Reviews Markdown documentation (READMEs, docs sites, wikis, blog posts, changelogs) for accessibility - missing or poor image alt text, heading structure, vague or…
-
Political
PromptsI need to understand why Somalia can't be stable country and what is the missing thing that if Somali politicians do Somalia become great again
-
skill-master
Prompts--- name: skill-master description: Discover codebase patterns and auto-generate SKILL files for .claude/skills/. Use when analyzing project for missing skills, creating new skills from codebase patterns, or syncing…