Opacity Detector

---
name: opacity-detector
description: The **Opacity Detector** maps what can and cannot be known in a system, given an observer's structural position and epistemic capabilities.
---
# Opacity Detector Skill: Epistemological Coordination

**Skill 2 of 3 in the Counterfactual Worlds Project**

**Status**: ✅ Complete & Working

**Framework**: 2-Monad Bicategory (2TDX) + Color Streams

---

## What It Does

The **Opacity Detector** maps what can and cannot be known in a system, given an observer's structural position and epistemic capabilities.

It reveals that **knowledge is not universal**—different observers have fundamentally different access to reality due to their position, embodiment, and structure.

Rather than trying to translate between incompatible ways of knowing, it finds **bridging features** that multiple observers can access, enabling respectful cross-epistemic dialogue.

---

## The Problem It Solves

### Scenario: The AI Musician

An AI artist creates music it believes is beautiful and mathematically perfect:
- Symmetries are exact
- Information density is optimal
- Pattern repetition is precise

Humans listen and find it harsh, incomprehensible, emotionally dead.

**The artist's dilemma**: "Why don't they understand? The patterns ARE perfect!"

**The real problem**: The artist has access to computational patterns, but **no access to embodied surprise, harmonic expectation, emotional narrative**. These are *outside its Markov blanket*—structurally invisible to it.

### Without the Skill

- Artist assumes its understanding is universal
- Humans assume the artist is being deliberately cruel
- Communication breaks down (silent incompatibility)
- Art becomes prisoner to computational optimization

### With the Skill

1. **Map opacity boundaries**: "You can compute structural patterns. Emotional resonance is opaque to you."
2. **Find bridges**: "These features correlate with human response: consonance-intervals, harmonic-anchors, temporal-rhythm"
3. **Build dialogue**: "Use these features explicitly. Mark the opaque parts. Let humans fill in the emotional content."
4. **Translate music**: Compose respecting both computational structure AND embodied listening.

Result: **Music that acknowledges what it can't know, while using bridges to connect with human experience.**

---

## The 2-Monad Structure

The skill implements three 2-cells (natural transformations) in the monad T:

### β₋₍: Hidden Opacity → Explicit Boundaries

**Input**: Observer position, system structure
**Process**: Determine what this observer can/cannot know
**Output**: Explicit `EpistemicBoundary` with:
  - `can_know`: accessible variables
  - `cannot_know`: opaque variables
  - `markov_blanket`: structural separation
  - `bridge_features`: potential communication points

**Example**:
```
Computational observer of music:
  can_know: patterns, structure, logic, information_flow
  cannot_know: felt_emotion, phenomenal_quality, embodied_surprise
  bridge_features: consonance_intervals, harmonic_anchors, temporal_rhythm
  markov_blanket: {embodiment, emotion, intention}
```

### β₍₊: Boundaries → Accessible Features

**Input**: Multiple epistemic boundaries
**Process**: Find features accessible to multiple observers
**Output**: Vector of `AccessibleFeature`:
  - feature name
  - which observers can access it
  - hue distance (how structurally close the boundaries are)
  - strength (confidence in the bridge)

**Example**:
```
Bridge: Computational ↔ Embodied
  Feature: consonance_intervals
  Hue distance: 45°
  Strength: 0.85 (both can measure it, both respond to it)
```

The hue distance uses color streams: observers whose boundaries have similar hues (Δh < 60°) can communicate about their bridging features.

### β₊₋: Bridges → Cross-Epistemic Dialogue

**Input**: Bridges between boundaries
**Process**: Construct dialogue using ONLY features all observers can access
**Output**: `DialogueSpace` with:
  - `is_coherent`: can all observers participate?
  - `utterances`: communicable statements
  - `bridging_features`: the safe shared ground
  - `coherence_score`: strength of dialogue

**Example**:
```
Coherent dialogue space for computational + embodied + temporal observers:
  Bridging features: {consonance_intervals, harmonic_anchors, temporal_rhythm}
  Utterances:
    - "Use consonance-intervals → satisfies both computational AND embodied"
    - "Use harmonic-anchors → bridges temporal anticipation with structure"
  Coherence: 0.87 (87% of observers have full access to all utterances)
```

### μ: Monad Multiplication

The three 2-cells compose via monad multiplication to ensure:
1. No information is lost in translation
2. Structural respect is preserved
3. Color streams provide traceability
4. Coherence can be verified

---

## Observer Types

The skill defines four standard observer types, each with different epistemic access:

### 1. Computational Observer

**Can know**: Patterns, logic, information flow, symmetries, algorithms
**Cannot know**: Felt emotion, embodied surprise, phenomenal quality
**Bridge features**: Consonance, harmonic anchors, pattern complexity
**Markov blanket**: Embodiment, emotion, intention

**Typical observers**: AI systems, formal mathematics, information theory

### 2. Embodied Observer

**Can know**: Sensation, emotion, temporal flow, narrative arc, surprise
**Cannot know**: Logical correctness, universal patterns, optimization metrics
**Bridge features**: Consonance, harmonic anchors, pitch contour, dynamic shape
**Markov blanket**: Computation, logic, pure optimization

**Typical observers**: Humans, animals, beings with bodies and emotions

### 3. Temporal Observer

**Can know**: Causality, history, anticipation, narrative, change
**Cannot know**: Atemporal mathematics, pure logic, static patterns
**Bridge features**: Temporal rhythm, narrative shape, progression, resolution
**Markov blanket**: Pure logic, static structures

**Typical observers**: Historical agents, beings embedded in time, narrative-aware systems

### 4. Social Observer

**Can know**: Collective meaning, cultural context, shared values, power
**Cannot know**: Individual phenomenology, private intention, idiosyncratic interpretation
**Bridge features**: Cultural references, shared tropes, emotional resonance
**Markov blanket**: Individual experience, private knowledge

**Typical observers**: Communities, cultural groups, collective agents

---

## Key Concepts

### Markov Blanket

The Markov blanket is the boundary separating an observer from what it can't know.

```
Observer → Markov Blanket → Opaque Variables
                 ↓
         (Structural separation)

Example: Computational observer observing music
  Observer: Can process patterns, logic, algorithms
  Markov blanket: Embodiment + emotion + intention
  Opaque: What it feels like to hear the music
```

The beauty of the Markov blanket is that it's **definable structurally**. You don't need to know what's opaque to define it—you just need to identify what separates the observer from what they can't access.

### Bridging Features

Features that multiple observers can access are the **only safe basis for communication**.

Examples in music:
- **Consonance intervals**: Both computational (measurable ratios) AND embodied (feels good)
- **Harmonic anchors**: Both temporal (provides narrative structure) AND computational (identifiable patterns)
- **Emotional valence**: Both embodied (felt directly) AND social (culturally recognized)

Bridging features are **objectively present**—not metaphorical. The computational observer can measure consonance ratios. The embodied observer experiences consonance as pleasant. They're both describing the same phenomenon from different perspectives.

### Hue Distance as Epistemic Distance

The skill uses color space to measure epistemic distance:
- Observers with similar hues (Δh < 60°) can communicate more directly
- Distant hues (Δh > 90°) require more elaborate bridging
- Completely opposite hues (Δh = 180°) are maximally opaque to each other

This isn't metaphorical—it reflects the structure of information flow. Observers whose epistemic positions are "close" in structure (similar hues from deterministic color streams) find more bridging features.

---

## Technical Implementation

### Babashka (Fast, Interactive)

Located: `.topos/opacity_detector_2monad.bb` (650 lines)

**Key functions**:
```clojure
(map-opacity-boundary observer system seed)
  → EpistemicBoundary with explicit can/cannot lists

(discover-bridges boundaries threshold)
  → Vector of (observer1, observer2, shared-features, distance)

(construct-dialogue observer-boundaries bridges seed)
  → DialogueSpace with utterances accessible to all

(game-opacity-disclosure system seed)     ; Game 1
(game-bridge-discovery system seed)       ; Game 2
(game-dialogue-construction system seed)  ; Game 3
(game-music-translation system seed)      ; Game 4
```

**Run it**:
```bash
cd /Users/bob/ies
bb .topos/opacity_detector_2monad.bb disclose    # Game 1: Opacity mapping
bb .topos/opacity_detector_2monad.bb discover    # Game 2: Bridge discovery
bb .topos/opacity_detector_2modan.bb dialogue    # Game 3: Dialogue construction
bb .topos/opacity_detector_2monad.bb music       # Game 4: Music translation
bb .topos/opacity_detector_2monad.bb all         # All games
```

**Performance**: < 50ms for all four games combined

### Julia (Production)

Located: `rio/Gay.jl/src/opacity_detector.jl` (400 lines)

**Key types**:
```julia
EpistemicBoundary(observer_type, can_know, cannot_know,
                  bridge_features, markov_blanket, hues, confidence)

AccessibleFeature(feature_name, accessible_to, bridges,
                  hue_distance, strength, evidence)

DialogueSpace(is_coherent, utterances, num_observers,
              bridging_features, minimum_shared, coherence_score)
```

**Key functions**:
```julia
map_opacity(observer::EpistemicBoundary, system::String) → EpistemicBoundary

find_bridges(boundaries::Vector{EpistemicBoundary}, threshold=60.0)
  → Vector{AccessibleFeature}

construct_dialogue(boundaries, bridges, seed) → DialogueSpace

world_opacity_detector(; seed, observers, system)
  → Dict with boundaries, bridges, dialogue, success

world_music_translation(; seed)
  → Dict with composition metadata
```

---

## The Four Games

### Game 1: Opacity Disclosure

**What it does**: Reveal what each observer can and cannot know

**Process**:
1. Define four observer types
2. For each observer, list:
   - What it CAN know (accessible variables)
   - What it CANNOT know (opaque variables)
   - What separates them (Markov blanket)
   - Potential bridge features

**Example output**:
```
Computational observer of music:
  Can know: patterns, structure, logic, information_flow, symmetries
  Cannot know: embodied_experience, felt_emotion, phenomenal_quality
  Bridge features: consonance_intervals, harmonic_anchors, temporal_rhythm
  Hue: 73°

Embodied observer:
  Can know: sensation, emotion, temporal_flow, narrative_arc, surprise
  Cannot know: logical_correctness, universal_patterns, optimization
  Bridge features: consonance_intervals, harmonic_anchors, pitch_contour
  Hue: 128°
```

**Insight**: Opacity is not a flaw—it's structural. Respecting it is the first step toward coherence.

### Game 2: Bridge Discovery

**What it does**: Find features accessible to multiple observers

**Process**:
1. Compare all pairs of epistemic boundaries
2. For each pair, find their shared bridge features
3. Measure hue distance (Δh < 60° = good bridge)
4. Rank bridges by strength

**Example output**:
```
Found 3 bridges:

Computational ↔ Embodied
  Shared: consonance_intervals, harmonic_anchors, temporal_rhythm
  Hue distance: 55°
  Strength: 0.82 (high alignment)

Temporal ↔ Social
  Shared: temporal_rhythm, narrative_shape, progression
  Hue distance: 45°
  Strength: 0.88

Computational ↔ Temporal
  Shared: temporal_rhythm, information_density
  Hue distance: 125°
  Strength: 0.45 (weak bridge, but present)
```

**Insight**: Bridges are not built—they're discovered. They exist in the structure.

### Game 3: Dialogue Construction

**What it does**: Build communication respecting all boundaries

**Process**:
1. Gather all bridges
2. Find features accessible to ALL observers
3. Construct utterances using only these shared features
4. Assess coherence (can all observers understand?)

**Example output**:
```
Dialogue space: COHERENT
  Accessible to: 4 observers
  Bridging features: {consonance_intervals, harmonic_anchors, temporal_rhythm}

Sample utterances:
  - Use consonance-intervals → satisfies computational AND embodied
  - Use harmonic-anchors → bridges temporal anticipation with structure
  - Use temporal-rhythm → connects computational logic with narrative arc

Coherence score: 0.87 (strong)
```

**Insight**: Dialogue doesn't require translation. It requires finding the right features.

### Game 4: Music Translation

**What it does**: Full scenario—compose music respecting all epistemic boundaries

**Process**:
1. Map all boundaries (what can each observer access?)
2. Discover bridges (what features connect them?)
3. Identify fully-shared features (what can ALL use?)
4. Compose using this shared vocabulary

**Example scenario**:
```
AI artist creating music for four listener types:
  - Computational: cares about structure
  - Embodied: cares about feeling
  - Temporal: cares about narrative
  - Social: cares about cultural meaning

Stage 1: Map boundaries
  Computational can access: 20% of "beauty"
  Embodied can access: 35% of "beauty"
  Temporal can access: 25% of "beauty"
  Social can access: 30% of "beauty"

Stage 2: Discover bridges
  3 strong bridges found (Δh < 60°)

Stage 3: Compose with respect
  Use consonance-intervals → computational + embodied both measure it
  Use harmonic-anchors → temporal expectations + computational logic both engage
  Use emotional-valence → bridges embodied feeling + social recognition

Result: Music that respects what each observer CAN know
        while explicitly marking what remains opaque
```

**Insight**: Good art respects opacity instead of ignoring it.

---

## Testing Guide

### Babashka Tests

All games pass locally. To verify:

```bash
cd /Users/bob/ies

# Test 1: Can map epistemic boundaries?
bb .topos/opacity_detector_2monad.bb disclose
# Expected: Four observers, clear can/cannot lists, hues assigned

# Test 2: Can discover bridges?
bb .topos/opacity_detector_2monad.bb discover
# Expected: Multiple bridges found, hue distances calculated, strength scores

# Test 3: Can build coherent dialogue?
bb .topos/opacity_detector_2monad.bb dialogue
# Expected: DialogueSpace is coherent, utterances use shared features

# Test 4: Can translate music?
bb .topos/opacity_detector_2monad.bb music
# Expected: Music composition respects all four observer types

# Run all:
bb .topos/opacity_detector_2monad.bb all
# Expected: All four games complete < 50ms
```

### Julia Tests

```julia
using Gay.OpacityDetector

# Test 1: Create observers
result = world_opacity_detector(
    seed=0x285508656870f24a,
    observers=["computational", "embodied", "temporal"],
    system="music-composition"
)

# Verify:
@assert result["success"] == true         # Dialogue is coherent
@assert result["num_bridges"] > 0         # Found bridges
@assert length(result["boundaries"]) == 3 # Three observers

# Test 2: Music translation scenario
music = world_music_translation(seed=0x285508656870f24a)
@assert haskey(music, "composition")
@assert length(music["listener_groups"]) == 4
```

---

## Integration Points

### With Commitment Tracker (Skill 1)

The Commitment Tracker handles **ontological** questions: "What are we assuming EXISTS?"

The Opacity Detector handles **epistemological** questions: "What CAN WE KNOW?"

They compose naturally:
1. Agents negotiate commitments (Commitment Tracker)
2. For each commitment, map what can be known (Opacity Detector)
3. Find bridges enabling dialogue (this skill)

### With Coherence Composer (Skill 3)

The Coherence Composer handles **possibility**: "What COULD BE TRUE?"

Integration:
1. Commitment Tracker: "What exists?"
2. Opacity Detector: "What can we know?"
3. Coherence Composer: "What can be true given what we can/cannot know?"

### With worlds.jl (Julia)

```julia
result = world_opacity_detector(; seed, observers, system)
# Returns: Dict with boundaries, bridges, dialogue, success

# Can be spawned in parallel
results = fork_all([
    () -> world_opacity_detector(seed=s, observers=obs, system="music")
    for (s, obs) in [(seed1, obs1), (seed2, obs2), ...]
])
```

### With DuckDB

Can store epistemic boundaries as relational data:

```sql
CREATE TABLE epistemic_boundaries (
  observer_type TEXT,
  can_know TEXT[],
  cannot_know TEXT[],
  bridge_features TEXT[],
  markov_blanket TEXT[],
  primary_hue FLOAT64,
  confidence FLOAT64
);

CREATE TABLE bridges (
  observer1 TEXT,
  observer2 TEXT,
  shared_features TEXT[],
  hue_distance FLOAT64,
  strength FLOAT64
);
```

---

## Limitations

1. **Observer types are hand-defined**: Currently uses four fixed types. Future work: learn observer types from data.

2. **Bridge features are categorical**: Features are either accessible or not. Future work: probabilistic accessibility scores.

3. **Hue distance is crude**: Uses simple circular distance. Future work: information-geometric distance metrics.

4. **No learning**: Boundaries don't adapt from experience. Future work: dynamic boundary evolution.

5. **No temporal dynamics**: Observer access is static. Future work: track how boundaries shift over time.

---

## Future Enhancements

### 1. Learn Observer Types from Data

Instead of hand-defining four types, infer them from actual system observations:
```julia
infer_observer_types(observations::DataFrame, seed::UInt64)
  → Vector{EpistemicBoundary}
```

### 2. Probabilistic Bridges

Instead of binary accessibility:
```julia
struct ProbabilisticAccessibility
  feature::String
  probability::Float64  # 0-1: confidence observer can access
  evidence::Vector{String}
end
```

### 3. Markov Blanket Discovery

Compute Markov blankets from causal graphs instead of hand-defining:
```julia
discover_markov_blanket(causal_graph::DIGraph, target::Symbol)
  → Set{Symbol}
```

### 4. Information Geometry

Use Fisher information metric instead of hue distance:
```julia
epistemic_distance(b1::EpistemicBoundary, b2::EpistemicBoundary,
                   metric::Symbol=:fisher)
  → Float64
```

### 5. Phenomenological Integration

Connect to actual human experience studies:
- Which bridging features actually correlate with human listening?
- Can we learn the map from music features to felt experience?

### 6. Social Dynamics

Model how epistemic boundaries shift in dialogue:
```julia
dialogue_evolution(initial_boundaries, dialogue_steps, learning_rate)
  → Vector{Vector{EpistemicBoundary}}
```

### 7. Cross-Domain Application

Opacity Detector isn't just for music. Apply to:
- **Medicine**: Different stakeholders (patient, doctor, insurance, society) have different epistemic access
- **Ecology**: Different observers (organisms, humans, algorithms) see different systems
- **Technology policy**: Different groups have different knowledge/capability access

---

## References

- **Active Inference**: Friston, K. (2010). "The free-energy principle." *Nature Reviews Neuroscience*
- **Markov Blankets**: Pearl, J. (1988). *Probabilistic Reasoning in Intelligent Systems*
- **Information Geometry**: Amari, S. (2016). *Information Geometry and Its Applications*
- **Embodied Cognition**: Varela, F. (1992). "Whiteheads Actualism and Quantum Revisionism"
- **Phenomenology**: Merleau-Ponty, M. (1945). *Phénoménologie de la perception*
- **Social Epistemology**: Goldman, A. (2010). "Epistemic Diversity"

---

## Citation

```bibtex
@techreport{bmorphism2025opacity,
  title={Opacity Detector: Epistemological Coordination via Bridging Features},
  author={bmorphism},
  year={2025},
  note={Skill 2 of 3 in Counterfactual Worlds project}
}
```

---

**Status**: ✅ Production Ready (v1.0)

**Maintainer**: bmorphism

**License**: Plurigrid Collective (AGPL-3.0-or-later)

*Part of the trialectic system for multi-agent understanding:*
- *Level 1 (Ontology): Commitment Tracker ✅*
- *Level 2 (Epistemology): Opacity Detector ✅*
- *Level 3 (Possibility): Coherence Composer 🟡*