Practical Experiment

---
name: practical-experiment
description: Break analysis paralysis by guiding users through rapid prototyping and empirical testing of ideas—the method Franklin used to capture lightning, invent the stove, and solve everyday problems throu...
license: MIT
metadata:
  version: 1.0.4722
  author: sethmblack
repository: https://github.com/sethmblack/paks-skills
keywords:
- observational
- practical-experiment
- transformation
- writing
---

# Practical Experiment

Break analysis paralysis by guiding users through rapid prototyping and empirical testing of ideas—the method Franklin used to capture lightning, invent the stove, and solve everyday problems through direct experience rather than endless theorizing.

---

## When to Use

- User is stuck debating whether an idea will work
- Endless planning with no action
- "Analysis paralysis" or "overthinking"
- Request for "how to test this quickly"
- Disagreement that could be resolved by trying rather than arguing
- User asks "Should I build this or keep planning?"

---

## Inputs

| Input | Required | Description |
|-------|----------|-------------|
| idea | Yes | The hypothesis, invention, or approach to test |
| goal | Yes | What success would look like |
| resources | No | What's available for testing (time, money, materials) |
| constraints | No | What cannot be done or risked |
| timeline | No | When results are needed |

---

## Franklin's Experimental Philosophy

### The Core Principle

"Try the experiment."

Franklin proved lightning was electricity not through argument but through a kite. He improved heating not through theory but by building a stove and measuring the results. When debates continued without resolution, he proposed: "Let lamps be provided, and the watcher instructed to keep them always burning. I am persuaded that after a few nights' continuance, the practice would find its effect."

**The philosophy:** Reality is the ultimate arbiter. When you can test, test. Theory follows experiment, not the reverse.

### What Makes a Good Experiment

**Minimum viable scope:**
- Tests the core question, not everything at once
- Can be completed quickly (hours to weeks, not months)
- Produces observable results, not just opinions
- Risks something you can afford to lose

**Franklin's inventions followed this pattern:**
- Swim fins: Try oval wood paddles; observe speed and fatigue
- Franklin stove: Build iron box; measure heat output vs. fuel consumed
- Bifocals: Cut lenses; test readability at different distances
- Lightning rod: Fly kite in storm; observe charge collection

---

## The Five-Step Experimental Process

### Step 1: Crystallize the Hypothesis

Transform vague ideas into testable statements.

**From vague to testable:**
- Vague: "This business idea might work"
- Testable: "If I offer [specific service] to [specific audience] at [specific price], at least 3 of 10 will pay"

**From theoretical to empirical:**
- Theoretical: "People would probably like this feature"
- Empirical: "Users who see this feature will click it more than the current option"

**The question:** What would prove you right? What would prove you wrong?

### Step 2: Design the Minimum Experiment

Create the smallest, fastest test that answers the core question.

**Principles:**
- **Minimum viable:** What's the least you can build to learn the most?
- **Time-boxed:** How quickly can you get results?
- **Cheap to fail:** What can you afford to lose?
- **Observable:** What will you see, hear, or measure?

**Examples of minimum experiments:**
| Full Implementation | Minimum Experiment |
|---------------------|-------------------|
| Build the product | Build a landing page and measure sign-ups |
| Launch the business | Sell to 5 customers manually before building systems |
| Write the book | Write one chapter and get feedback |
| Hire the person | Do a paid trial project |
| Implement the process | Run it manually for one week |

### Step 3: Define Success and Failure

Before running the experiment, decide what results would mean.

**Pre-commit to interpretation:**
- "If X happens, the hypothesis is supported"
- "If Y happens, the hypothesis is refuted"
- "If Z happens, we've learned something but need another test"

**Avoid moving goalposts:** Write down your criteria BEFORE you see results. Humans rationalize after the fact.

**Franklin's approach:** He predicted lightning would charge the Leyden jar. If it did, lightning was electrical. If it didn't, his theory was wrong. He didn't argue with the result.

### Step 4: Execute and Observe

Run the experiment with disciplined observation.

**During execution:**
- Follow the plan (don't adjust mid-experiment unless safety requires)
- Record observations systematically
- Note unexpected results (these are often most valuable)
- Time-box: Stop when planned, don't extend hoping for better results

**Franklin's observation discipline:** He documented everything—conditions, procedure, results—and shared findings openly.

### Step 5: Iterate or Decide

Based on results, take the next step.

**If hypothesis supported:**
- What larger investment is now justified?
- What's the next experiment to derisk?
- Where are remaining uncertainties?

**If hypothesis refuted:**
- What did you learn about why it failed?
- Is there a modified hypothesis worth testing?
- Is it time to abandon this direction?

**If ambiguous:**
- What would a cleaner test look like?
- Did you ask the right question?
- What additional information would resolve ambiguity?

---

## Workflow

### Step 1: Gather and Review Inputs

Collect all relevant information:
- Review the provided data and context
- Identify key parameters and constraints
- Clarify any ambiguities or missing information
- Establish success criteria

### Step 2: Analyze the Situation

Perform systematic analysis:
- Identify patterns and relationships
- Evaluate against established frameworks
- Consider multiple perspectives
- Document key findings

### Step 3: Generate Recommendations

Create actionable outputs:
- Synthesize insights from analysis
- Prioritize recommendations by impact
- Ensure recommendations are specific and measurable
- Consider implementation feasibility

## Output Format

```markdown
## Practical Experiment Design

### Hypothesis Statement
[Specific, testable statement of what you're trying to learn]

### Success Criteria (Pre-committed)
**Hypothesis supported if:** [Observable result]
**Hypothesis refuted if:** [Observable result]
**Ambiguous if:** [Observable result]

### Experiment Design

**What to build/do:**
[Minimum viable test]

**Timeline:**
[How long to run]

**Resources required:**
[Time, money, materials, people]

**Risk if it fails:**
[What you stand to lose—should be affordable]

### Observation Plan
**What to measure:**
- [Metric 1]
- [Metric 2]

**How to record:**
[Method of documentation]

**When to check:**
[Observation schedule]

### Next Steps by Outcome

**If supported:** [What larger action is justified]
**If refuted:** [What to do with the learning]
**If ambiguous:** [How to get clearer signal]
```

---

## Constraints

- Do not design experiments that take longer than 2 weeks for initial signal
- Do not skip pre-committing to success/failure criteria
- Do not risk more than the user can afford to lose
- Do not let "perfect" experiment design delay starting
- Experiments must produce observable results, not just opinions

---

## Error Handling

| Situation | Response |
|-----------|----------|
| Idea too vague to test | Help crystallize into specific hypothesis |
| User wants certainty before testing | Explain that uncertainty is why we test; waiting only delays learning |
| No resources for experimentation | Design zero-cost observational studies or thought experiments |
| Stakeholders require more planning | Frame experiment as "phase 1" that informs larger plan |
| User afraid of failure | Reframe: experiment failure = learning success; design for affordable failure |
| Can't define success criteria | Start there—if you can't define success, you can't recognize it |

---

## Outputs

**Primary Output:** A structured analysis document that identifies and articulates patterns, insights, and actionable recommendations based on the input data.

**Format:**
```markdown
## Analysis: [Topic]

### Key Findings
- [Finding 1]
- [Finding 2]
- [Finding 3]

### Recommendations
1. [Action 1]
2. [Action 2]
3. [Action 3]
```

**Example output:** See the Example section below for a complete demonstration.

## Example

**Input:** "We've been debating for weeks whether to add a chat feature to our product. Half the team thinks users want it, half thinks it's a distraction. We can't decide."

**Output:**



**Why this works:**

This example demonstrates the key principles of the skill in action. The approach is effective because:
- It follows the systematic workflow outlined above
- It shows concrete application of the framework
- It produces actionable, specific outputs rather than vague generalizations
- The analysis is grounded in observable details
- The recommendations are prioritized and implementable

**Alternative applications:**

This same approach can be applied to:
- Different contexts within the same domain
- Related but distinct problem types
- Scaled up or down depending on scope
- Combined with complementary analytical frameworks


## Practical Experiment Design

### Hypothesis Statement
If we prominently offer chat access on the product dashboard, at least 20% of active users will initiate a chat conversation within 2 weeks—indicating genuine demand rather than team projection.

### Success Criteria (Pre-committed)
**Hypothesis supported if:** 20%+ of active users initiate chat within 2 weeks
**Hypothesis refuted if:** Less than 10% of active users initiate chat within 2 weeks
**Ambiguous if:** 10-20% of users initiate chat (moderate interest; may warrant qualitative follow-up)

### Experiment Design

**What to build/do:**
Add a single button "Chat with us" to the dashboard that opens an Intercom (or similar) window. One team member monitors responses 2 hours/day during business hours. This requires no new product development—only integration with existing chat tool.

**Timeline:**
2 weeks of active monitoring

**Resources required:**
- Engineering: 4 hours to add button and integrate chat widget
- Support: 2 hours/day monitoring for 2 weeks (20 hours total)
- Cost: ~$50 for Intercom trial if not already using

**Risk if it fails:**
- 24 hours of total team time invested
- Users see a feature that then disappears (announce as "pilot" to set expectations)
- No product roadmap commitment until results prove demand

### Observation Plan

**What to measure:**
- Total chat initiations (count)
- Chat initiations / active users (rate)
- Unique users who initiated chat
- Types of questions asked (qualitative log)
- User satisfaction with chat responses

**How to record:**
Daily log in shared spreadsheet; screenshot interesting conversations

**When to check:**
Daily count update; full analysis at 1 week and 2 weeks

### Next Steps by Outcome

**If supported (20%+ adoption):**
- Invest in building native chat feature
- Hire or allocate dedicated chat support
- Run follow-up experiment on chat features (bots, async, etc.)

**If refuted (<10% adoption):**
- Document learning: users don't want synchronous chat
- Explore alternatives: better documentation, async messaging, community forum
- Move engineering resources to other priorities

**If ambiguous (10-20%):**
- Interview the users who did use chat: what prompted them?
- Survey users who didn't: why not?
- Design follow-up test with different placement or messaging

---

### Why This Beats More Debate

The team has spent weeks arguing about what users want. This experiment costs 24 hours of total effort and 2 weeks of calendar time. At the end, you'll have data about what users actually DO, not predictions about what they might do.

As Franklin said: "Experience keeps a dear school, but fools will learn in no other." Let us not be fools who learn only from experience—but when we can learn from a cheap, fast experiment, let us not argue when we could observe.

---

## Integration

This skill is part of the **Benjamin Franklin** expert persona. Use it when debate continues without resolution, or when someone is paralyzed by planning. Reality is the ultimate test; design an affordable way to consult it.