Eda

# HarnessML: Exploratory Data Analysis

Use when you need to build intuition about the data. This isn't a checkbox — it's how you generate your first hypotheses and catch problems before they poison your models.

---

## The Goal of EDA

EDA produces **hypotheses**, not charts. Every observation should lead to a question: "If X is true, then feature Y should be predictive because Z." Those hypotheses feed into `domain-research` and `experiment-design`.

---

## Target Analysis

Start with the thing you're predicting.

```
data(action="inspect")
```

**Ask:**
- What's the distribution? (Class balance for classification, skew for regression)
- Are there temporal patterns? (Does the target rate change over time?)
- Are there segments where the target behaves differently? (Target rate by category, region, time period)
- Is the target noisy or clean? (If noisy, your metric ceiling may be lower than you think)

**What this tells you:**
- Class imbalance → you may need stratified CV, class weights, or a metric that handles imbalance (F1 over accuracy)
- Temporal drift → you need temporal CV (expanding window, not random k-fold), and drift-aware features
- Segment differences → features that capture segment membership may be valuable
- Noise → calibrate your expectations. Don't chase the last 0.001 on a noisy target.

---

## Feature-Target Relationships

```
features(action="discover")
```

**Ask:**
- Which features have the strongest univariate signal?
- Are the strong features surprising or expected?
- Are there features with near-zero signal that you expected to matter? (This is interesting — why don't they matter?)
- Are there features with suspiciously strong signal? (Possible leakage)

**What this tells you:**
- Strong expected features confirm domain knowledge — good, build on this
- Strong unexpected features need investigation — is it leakage? A proxy for something else?
- Weak expected features are the most interesting — either the relationship is non-linear (trees will find it), conditional (need an interaction), or your expectation was wrong (update your mental model)

---

## Feature-Feature Relationships

**Ask:**
- Are there highly correlated feature pairs? (>0.9 correlation means one is redundant)
- Are there feature clusters? (Groups of features measuring the same underlying thing)
- Do categorical features have high cardinality? (May need encoding strategy)

**What this tells you:**
- Correlated features inflate importance scores and can destabilize linear models
- Feature clusters suggest a latent variable — sometimes a single derived feature capturing that latent variable is better than the cluster
- High cardinality categoricals need careful handling — frequency encoding, target encoding (with leakage protection), or grouping rare levels

---

## Missing Data Patterns

**Ask:**
- Is missingness random or patterned?
- Are certain features always missing together? (Suggests a data source issue)
- Is missingness correlated with the target? (If so, missingness itself is a feature)

**What this tells you:**
- Random missingness → impute and move on
- Patterned missingness → the *pattern* is information. A `null_indicator` feature for these columns may be predictive.
- Target-correlated missingness → important signal, but also a leakage risk if the missingness is caused by the outcome

---

## Segments and Subgroups

**Ask:**
- Does the target behave differently across natural segments? (Time periods, categories, geographies)
- Are there subgroups where prediction will be harder? (Small segments, unusual distributions)
- Would separate models or segment-specific features help?

**What this tells you:**
- Segment-dependent target behavior → regime indicators or interaction features
- Hard-to-predict subgroups → these will dominate your error. Understanding them is where the biggest gains come from.

---

## What EDA Is Not

- It is not running `data(action="inspect")` and moving on
- It is not a phase to "complete" before modeling
- It is something you return to throughout the project — every surprising experiment result is an invitation to look at the data again
- The best data scientists spend more time here than anywhere else

---

## Output

By the end of initial EDA, you should have:
1. A written description of the data landscape (target, features, quality issues)
2. A list of 5-10 hypotheses about what might be predictive and why
3. Identified risks (leakage candidates, class imbalance, temporal drift)
4. Questions you can't answer yet that will guide early experiments