Newms Data Analysis

---
name: newms-data-analysis
description: "Use when conducting and reporting the analysis of a New Media & Society (NM&S) manuscript across qualitative, content/discourse, computational, and mixed methods — making inference transparent and defensible on each tradition's own terms. Strengthens analysis and reporting; it does not collect data."
category: media-and-creative
source_repo: brycewang-stanford/Awesome-Journal-Skills
source_path: "New-Media-and-Society-Skills/skills/newms-data-analysis/SKILL.md"
source_url: https://github.com/brycewang-stanford/Awesome-Journal-Skills/blob/HEAD/New-Media-and-Society-Skills/skills/newms-data-analysis/SKILL.md
---


# Data & Analysis (newms-data-analysis)

NM&S spans interpretive, content-analytic, and computational analysis under one interdisciplinary roof.
The standard is the same across them: the analysis must be **transparent**, **credible to a reader from
another tradition**, and matched to what the evidence can support. This skill is about *inference and
reporting*, not study design (`newms-research-design`).

## When to trigger

- Moving from collected data to claims, themes, measures, or results
- A reviewer asked for reliability, robustness, validation, or a clearer analytic trail
- You need to report uncertainty or limits honestly for a cross-method audience

## Qualitative inference (interviews / ethnography)
- **Analytic transparency**: show the path from data to claim — coding/memoing process, how themes were
  built, and how many informants/instances support each theme (avoid "many participants felt…").
- **Negative cases and disconfirmation**: report instances that cut against the reading and how they
  were handled — the strongest signal of credible qualitative work.
- **Quote-to-claim discipline**: each claim is anchored to specific evidence, not an isolated vivid quote.

## Content / discourse analysis
- **Quantitative content analysis**: report **intercoder reliability** with the right statistic
  (Krippendorff's alpha preferred for most designs), the unit of analysis, and how disagreements were
  resolved; report category distributions with uncertainty, not just counts.
- **Interpretive discourse analysis**: make the interpretive logic auditable — what features of the text
  warrant the reading, and what an alternative reading would require.

## Computational analysis
- **Validation first**: report agreement between automated measures and human labels (precision/recall,
  F1, agreement) before interpreting model output as a finding.
- **Robustness**: sensitivity to preprocessing, model/hyperparameter choices, time window, and platform;
  show the result is not an artifact of one pipeline.
- **Inference and uncertainty**: report confidence/credible intervals; respect non-random API sampling;
  do not over-claim causality from observational trace data.

## Inference honesty (all methods)

State plainly what the analysis establishes — description, association, interpretation, or (rarely)
causation — and do not let verbs outrun the design. A cross-method NM&S panel reads candor as strength.

## Robustness & reliability checklist by method

| Method | Minimum credibility move | Common referee ask |
|--------|--------------------------|--------------------|
| Interviews / ethnography | analytic trail + negative cases | "How representative are these quotes?" |
| Quant content analysis | intercoder reliability (alpha) + unit defined | "What's your reliability?" |
| Discourse analysis | auditable interpretive warrant | "Why this reading not another?" |
| Computational | human-label validation + robustness sweep | "Did you validate the classifier?" |

## Worked micro-example (illustrative)

```
Computational: a classifier labels courier posts as "compliance" vs. "contestation."
Validation: 500 hand-coded posts → F1 = 0.84 reported before any substantive claim.
Robustness: result holds across two embeddings + two time windows; stated explicitly.
Inference framing: "posts shift toward compliance after a ranking change" = association, not proof of
  internalization; the qualitative strand supplies the mechanism (triangulation, per mixed design).
```

## Referee pushback → NM&S-specific fix

- *"How do I know the qualitative themes aren't cherry-picked?"* → Supply the analytic trail, theme
  prevalence, and negative cases.
- *"Your classifier is a black box."* → Add human-label validation metrics and a robustness sweep.
- *"You imply causation from observational traces."* → Downgrade the verbs; report as association and say so.

## Calibration anchors

- **Validate before you interpret.** Computational output is not a finding until it is checked against
  human labels.
- **Report what cuts against you.** Negative cases and robustness checks build more trust than a clean story.
- **Match verbs to design.** Description, association, interpretation, causation — name which one, and stop there.

## Anti-patterns

- "Participants said…" with no count, trail, or negative cases
- Content analysis with no reliability statistic or undefined unit of analysis
- Computational results with no validation against human labels
- Robustness checks omitted, leaving the result as a single-pipeline artifact
- Causal language on observational, non-random trace data

## Output format

```
【Method】qualitative / content-discourse / computational / mixed
【Inference type】description / association / interpretation / causation
【Credibility move】analytic trail / reliability stat / human-label validation
【Robustness】sensitivity checks / negative cases reported? [Y/N]
【Next】newms-tables-figures
```

## Supplementary resources

- [`../../resources/external_tools.md`](../../resources/external_tools.md) — reliability, content-analysis, and computational packages
- [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — NM&S methodological breadth

---

**Source:** [`brycewang-stanford/Awesome-Journal-Skills`](https://github.com/brycewang-stanford/Awesome-Journal-Skills) → `New-Media-and-Society-Skills/skills/newms-data-analysis/SKILL.md`