Freemorph Tuning Free Image Morphing

---
name: freemorph-tuning-free-image-morphing
title: "FreeMorph: Tuning-Free Generalized Image Morphing with Diffusion Model"
version: 0.0.2
engine: skillxiv-v0.0.2-claude-opus-4.6
license: MIT
url: "https://arxiv.org/abs/2507.01953"
keywords: [Image Morphing, Diffusion Models, Interpolation, No Fine-tuning, Smooth Transitions]
description: "Generate smooth morphing sequences between images without fine-tuning or alignment. Uses guidance-aware spherical interpolation and step-oriented attention blending to handle diverse semantic and layout scenarios, completing morphs 50× faster than fine-tuning methods."
---

# FreeMorph: Morphing Between Any Two Images Without Training

Image morphing traditionally requires either careful manual alignment or extensive fine-tuning per image pair. This limits practical application—each new pair means retraining. FreeMorph solves this by working within standard diffusion models' self-attention mechanisms. By blending features from both images and carefully managing attention across diffusion steps, it generates smooth morphing sequences in under 30 seconds without any per-image training.

The key insight: diffusion models' attention mechanisms can naturally blend images if guided properly. Early denoising steps preserve input image structure; later steps add variation. By modulating which input image dominates at each step, smooth transitions emerge automatically.

## Core Concept

Morphing requires three capabilities: (1) feature-level understanding of both images, (2) smooth interpolation between their features, and (3) handling semantic differences (objects in different positions or shapes). FreeMorph achieves this through:

1. **Guidance-Aware Spherical Interpolation**: Blend self-attention Key/Value features from both images using spherical geometry, avoiding the "shortest path" problem where linear interpolation creates unrealistic intermediate states

2. **Step-Oriented Variation Trend**: Rather than uniform weight throughout diffusion, gradually shift emphasis from left image (early steps) to right image (late steps), creating directional morphing

3. **Improved Diffusion Process**: Strategic application of different attention mechanisms at forward (corruption) and reverse (denoising) stages, plus high-frequency noise injection for flexibility

This enables smooth, realistic morphing across diverse image pairs without training.

## Architecture Overview

The FreeMorph system consists of these components:

- **Feature Extraction from Both Images**: Initial encoding of left and right images into latent space
- **Spherical Feature Aggregation**: Geometric blending of Key/Value attention features
- **Prior-Driven Self-Attention**: Modulation emphasizing interpolated features during forward diffusion, input images during reverse
- **Step-Oriented Weight Scheduling**: Temporal weighting that shifts from left to right image
- **High-Frequency Noise Injection**: Gaussian noise added to allow greater morphing flexibility
- **Attention Mechanism Switching**: Different strategies for forward vs reverse diffusion processes
- **Inference Without Fine-tuning**: Complete morphing in single forward pass with no training

## Implementation

This section demonstrates how to implement FreeMorph morphing.

**Step 1: Implement spherical feature interpolation**

This code blends features from two images using spherical geometry:

```python
import torch
import torch.nn as nn
import torch.nn.functional as F
import math

class SphericalFeatureInterpolation:
    """
    Blend features from two images using spherical interpolation (slerp).
    Avoids "shortest path" artifacts that come from linear interpolation.
    """

    @staticmethod
    def spherical_interpolate(feat_left, feat_right, t):
        """
        Spherical linear interpolation between two feature vectors.

        feat_left: (B, N, D) features from left image
        feat_right: (B, N, D) features from right image
        t: scalar in [0, 1], interpolation parameter (0=left, 1=right)

        Returns: (B, N, D) interpolated features
        """

        # Normalize features to unit sphere
        feat_left_norm = F.normalize(feat_left, dim=-1)
        feat_right_norm = F.normalize(feat_right, dim=-1)

        # Compute angle between features
        cos_omega = (feat_left_norm * feat_right_norm).sum(dim=-1, keepdim=True)
        # Clamp to avoid numerical issues with arccos
        cos_omega = torch.clamp(cos_omega, -1.0, 1.0)
        omega = torch.acos(cos_omega)

        # Spherical interpolation
        # If features are nearly parallel, fall back to linear interpolation
        sin_omega = torch.sin(omega)
        sin_omega = torch.where(sin_omega < 1e-6, torch.ones_like(sin_omega), sin_omega)

        # Slerp formula: (sin((1-t)*w) * a + sin(t*w) * b) / sin(w)
        weight_left = torch.sin((1 - t) * omega) / sin_omega
        weight_right = torch.sin(t * omega) / sin_omega

        # Blend using computed weights
        interpolated = weight_left * feat_left_norm + weight_right * feat_right_norm

        # Scale by average magnitude to preserve signal
        magnitude = 0.5 * (feat_left.norm(dim=-1, keepdim=True) + feat_right.norm(dim=-1, keepdim=True))
        interpolated = interpolated * magnitude

        return interpolated

    @staticmethod
    def interpolate_attention_features(kv_left, kv_right, t):
        """
        Interpolate Key/Value attention features from both images.

        kv_left: (B, N, 2*D) Key-Value features from left image
        kv_right: (B, N, 2*D) Key-Value features from right image
        t: interpolation parameter

        Returns: blended KV features
        """

        # Split Key and Value
        K_left, V_left = kv_left[..., :kv_left.shape[-1]//2], kv_left[..., kv_left.shape[-1]//2:]
        K_right, V_right = kv_right[..., :kv_right.shape[-1]//2], kv_right[..., kv_right.shape[-1]//2:]

        # Interpolate Key and Value separately
        K_interp = SphericalFeatureInterpolation.spherical_interpolate(K_left, K_right, t)
        V_interp = SphericalFeatureInterpolation.spherical_interpolate(V_left, V_right, t)

        # Concatenate back
        kv_interp = torch.cat([K_interp, V_interp], dim=-1)

        return kv_interp

# Test spherical interpolation
feat_left = torch.randn(1, 64, 768)  # 64 patches, 768-dim features
feat_right = torch.randn(1, 64, 768)

for t in [0.0, 0.25, 0.5, 0.75, 1.0]:
    interp = SphericalFeatureInterpolation.spherical_interpolate(feat_left, feat_right, t)
    # At t=0: close to left, at t=1: close to right
    dist_to_left = (interp - feat_left).norm(dim=-1).mean()
    print(f"t={t}: distance to left = {dist_to_left:.3f}")
```

This blends features using geometry instead of linear interpolation.

**Step 2: Implement step-oriented weight scheduling**

This code creates temporal weighting that shifts from left to right image:

```python
class StepOrientedMorphing:
    """
    Gradually shift emphasis from left image (early diffusion) to right image (late diffusion).
    Creates directed morphing: left → morphing sequence → right.
    """

    @staticmethod
    def compute_morphing_weights(current_step: int, total_steps: int, morphing_direction: str = 'left_to_right'):
        """
        Compute interpolation weight as function of diffusion step.

        current_step: 0 = start (noise), total_steps-1 = end (clean image)
        total_steps: total diffusion steps
        morphing_direction: 'left_to_right' or 'right_to_left'

        Returns: t in [0, 1], interpolation parameter
        """

        # Normalize step to [0, 1]
        progress = current_step / (total_steps - 1)

        if morphing_direction == 'left_to_right':
            # Early steps: t=0 (left image)
            # Late steps: t=1 (right image)
            t = progress
        else:
            # Early steps: t=1 (right image)
            # Late steps: t=0 (left image)
            t = 1.0 - progress

        return t

    @staticmethod
    def apply_prior_driven_attention(
        self_attention_output,
        interpolated_features,
        current_step,
        total_steps,
        forward_pass: bool = True
    ):
        """
        Modulate attention output based on diffusion direction and step.

        forward_pass: True during corruption (forward diffusion), False during denoising (reverse).

        During forward: emphasize interpolated features (generate morphing structure)
        During reverse: emphasize input images (preserve original content)
        """

        # Get weight for this step
        t = StepOrientedMorphing.compute_morphing_weights(current_step, total_steps)

        if forward_pass:
            # Forward diffusion: blend toward interpolated features
            # This makes the model learn to generate morphing transition
            blend_weight = 0.7  # Emphasize interpolation
            output = (1 - blend_weight) * self_attention_output + blend_weight * interpolated_features
        else:
            # Reverse diffusion: smoothly transition which image to emphasize
            # Early: use left features; late: use right features
            # This is implicitly handled by interpolated_features changing over time
            output = self_attention_output

        return output

    @staticmethod
    def create_gradual_transition(
        image_left,
        image_right,
        num_frames: int = 30,
        diffusion_steps: int = 50
    ):
        """
        Generate a morphing sequence by gradually changing interpolation weight.

        Returns: list of num_frames morphed images
        """

        morphing_sequence = []

        for frame_idx in range(num_frames):
            # Interpolation parameter for this frame
            frame_t = frame_idx / (num_frames - 1)

            # During diffusion, interpolate based on step
            morphed_frame = {}
            morphed_frame['t'] = frame_t
            morphed_frame['left_weight'] = 1.0 - frame_t
            morphed_frame['right_weight'] = frame_t

            morphing_sequence.append(morphed_frame)

        return morphing_sequence

# Test step-oriented morphing
frames = StepOrientedMorphing.create_gradual_transition(None, None, num_frames=20)

print("Morphing frame schedule:")
for i, frame in enumerate(frames[:5]):
    print(f"  Frame {i}: t={frame['t']:.2f}, left={frame['left_weight']:.2f}, right={frame['right_weight']:.2f}")
```

This creates a temporal schedule that smoothly transitions from one image to the other.

**Step 3: Implement improved diffusion with attention modulation**

This code integrates spherical interpolation into the diffusion process:

```python
class FreeMorphDiffusionModule(nn.Module):
    """
    Diffusion module with morphing-aware attention modulation.
    Handles both forward (corruption) and reverse (denoising) processes.
    """

    def __init__(self, diffusion_model):
        super().__init__()
        self.diffusion_model = diffusion_model
        self.spherical_interp = SphericalFeatureInterpolation()
        self.step_morph = StepOrientedMorphing()

    def encode_images(self, image_left, image_right):
        """Encode both images into latent space."""
        # Use diffusion model's VAE encoder
        latent_left = self.diffusion_model.vae_encoder(image_left)
        latent_right = self.diffusion_model.vae_encoder(image_right)

        return latent_left, latent_right

    def forward_diffusion_step(
        self,
        x_t,
        t,
        latent_left,
        latent_right,
        timestep_in_sequence
    ):
        """
        Single forward diffusion step with morphing guidance.
        """

        # Compute interpolation weight for this step
        morph_t = self.step_morph.compute_morphing_weights(timestep_in_sequence, 50)

        # Get interpolated features at this morphing parameter
        # Extract attention features from diffusion model
        kv_left = self.diffusion_model.extract_attention_features(latent_left, t)
        kv_right = self.diffusion_model.extract_attention_features(latent_right, t)

        kv_interp = self.spherical_interp.interpolate_attention_features(kv_left, kv_right, morph_t)

        # Inject interpolated features into attention
        x_t_morphed = self.diffusion_model.forward_with_custom_attention(
            x_t,
            t,
            kv_interp,
            is_forward_pass=True
        )

        return x_t_morphed

    def reverse_diffusion_step(
        self,
        x_t,
        t,
        latent_left,
        latent_right,
        timestep_in_sequence
    ):
        """
        Single reverse diffusion step (denoising) with morphing guidance.
        """

        morph_t = self.step_morph.compute_morphing_weights(timestep_in_sequence, 50)

        # During denoising, use prior-driven attention
        x_t_denoised = self.diffusion_model.denoise_step(
            x_t,
            t,
            condition=None,
            prior_guidance=morph_t  # Gradually shift from left to right
        )

        return x_t_denoised

    def generate_morphing_sequence(
        self,
        image_left,
        image_right,
        num_morphs: int = 20,
        diffusion_steps: int = 50
    ):
        """
        Generate complete morphing sequence between two images.
        """

        # Encode input images
        latent_left, latent_right = self.encode_images(image_left, image_right)

        morphing_sequence = []

        for morph_idx in range(num_morphs):
            # Initialize noise
            x_t = torch.randn_like(latent_left)

            # Forward diffusion
            for t in range(1, diffusion_steps):
                x_t = self.forward_diffusion_step(
                    x_t, t, latent_left, latent_right, morph_idx
                )

            # Reverse diffusion (denoising)
            for t in reversed(range(1, diffusion_steps)):
                x_t = self.reverse_diffusion_step(
                    x_t, t, latent_left, latent_right, morph_idx
                )

            # Decode to image space
            morphed_image = self.diffusion_model.vae_decoder(x_t)
            morphing_sequence.append(morphed_image)

        return torch.stack(morphing_sequence)

# Test morphing module (placeholder)
# In practice, this would use actual diffusion model
print("FreeMorph diffusion module ready for morphing")
```

This integrates morphing guidance into the diffusion process.

**Step 4: Add high-frequency noise injection**

This code improves morphing flexibility through adaptive noise:

```python
class HighFrequencyNoiseInjection:
    """
    Inject high-frequency noise adaptively to allow greater morphing flexibility.
    Helps bridge semantic gaps between very different images.
    """

    @staticmethod
    def compute_frequency_content(image_tensor):
        """Analyze frequency content of image using FFT."""
        fft = torch.fft.fftn(image_tensor.float(), dim=(-2, -1))
        magnitude = torch.abs(fft)

        # High frequency: edges of spectrum
        high_freq_mask = magnitude > magnitude.median()
        return high_freq_mask

    @staticmethod
    def adaptive_noise_injection(
        x_t,
        image_left,
        image_right,
        injection_strength: float = 0.1
    ):
        """
        Inject noise proportional to semantic difference between images.
        Greater difference → more noise allowed.
        """

        # Compute semantic difference (simplified: L2 distance)
        diff = (image_left - image_right).abs().mean()

        # Adapt noise strength based on difference
        noise_strength = injection_strength * (1.0 + diff.item())

        # Generate Gaussian noise
        gaussian_noise = torch.randn_like(x_t) * noise_strength

        # Add to current diffusion state
        x_t_injected = x_t + gaussian_noise

        return x_t_injected

    @staticmethod
    def high_frequency_component_injection(
        x_t,
        image_left,
        image_right,
        diffusion_step: int,
        total_steps: int
    ):
        """
        Inject high-frequency components to preserve detail during morphing.
        Especially important for handling layout differences.
        """

        # Extract high-frequency components from both images
        freq_left = HighFrequencyNoiseInjection.compute_frequency_content(image_left)
        freq_right = HighFrequencyNoiseInjection.compute_frequency_content(image_right)

        # Blend frequency masks based on morphing progress
        progress = diffusion_step / total_steps
        freq_blend = (1 - progress) * freq_left + progress * freq_right

        # Create high-freq noise from blend
        hf_noise = torch.randn_like(x_t) * 0.05
        hf_noise = hf_noise * freq_blend.unsqueeze(1)

        # Add high-frequency component
        x_t_enhanced = x_t + hf_noise

        return x_t_enhanced

# Test noise injection
x = torch.randn(1, 4, 64, 64)
img_left = torch.randn(1, 3, 256, 256)
img_right = torch.randn(1, 3, 256, 256)

x_noisy = HighFrequencyNoiseInjection.adaptive_noise_injection(x, img_left, img_right)
print(f"Original x range: {x.min():.3f} to {x.max():.3f}")
print(f"Noisy x range: {x_noisy.min():.3f} to {x_noisy.max():.3f}")
```

This injects adaptive noise to handle diverse image pairs.

**Step 5: Complete end-to-end morphing pipeline**

This code combines all components:

```python
class FreeMorphPipeline:
    """
    Complete FreeMorph pipeline: morph between any two images without fine-tuning.
    """

    def __init__(self, diffusion_model):
        self.morph_module = FreeMorphDiffusionModule(diffusion_model)
        self.noise_injection = HighFrequencyNoiseInjection()

    def morph(
        self,
        image_left,
        image_right,
        num_keyframes: int = 20,
        num_interpolations_per_keyframe: int = 1,
        diffusion_steps: int = 50,
        guidance_scale: float = 7.5
    ):
        """
        Generate complete morphing sequence between two images.

        Returns: sequence of morphed images
        """

        print(f"Morphing {image_left.shape} → {image_right.shape}")
        print(f"Generating {num_keyframes} keyframes with {num_interpolations_per_keyframe}× interpolation")

        start_time = time.time()

        # Generate morphing sequence
        morphing_frames = self.morph_module.generate_morphing_sequence(
            image_left,
            image_right,
            num_morphs=num_keyframes,
            diffusion_steps=diffusion_steps
        )

        # Optionally interpolate between keyframes for smoothness
        if num_interpolations_per_keyframe > 1:
            # Linear interpolation in latent space
            smooth_frames = []
            for i in range(len(morphing_frames) - 1):
                frame1 = morphing_frames[i]
                frame2 = morphing_frames[i + 1]

                for j in range(num_interpolations_per_keyframe):
                    alpha = j / num_interpolations_per_keyframe
                    interpolated = (1 - alpha) * frame1 + alpha * frame2
                    smooth_frames.append(interpolated)

            morphing_frames = torch.stack(smooth_frames)

        elapsed = time.time() - start_time

        print(f"Morphing complete in {elapsed:.1f} seconds")
        print(f"Generated {len(morphing_frames)} frames")
        print(f"50× faster than fine-tuning methods (30s vs 25min)")

        return morphing_frames

# Example usage
pipeline = FreeMorphPipeline(pretrained_diffusion_model)

img_left = torch.randn(1, 3, 512, 512)
img_right = torch.randn(1, 3, 512, 512)

morphing_sequence = pipeline.morph(
    img_left,
    img_right,
    num_keyframes=20,
    num_interpolations_per_keyframe=2,
    diffusion_steps=50
)

print(f"Output shape: {morphing_sequence.shape}")
```

This provides a complete morphing pipeline.

## Practical Guidance

**When to use FreeMorph:**
- Interactive morphing applications requiring fast turnaround
- Content creation where morphing many image pairs
- Scenarios where images have different semantics or layouts
- Web/mobile applications with latency constraints
- Batch morphing of image collections

**When NOT to use:**
- Ultra-high-quality studio morphing (fine-tuning methods may be better)
- Real-time applications (30 seconds still substantial)
- Situations where perfect semantic alignment is critical
- Very low-resolution images (morphing quality depends on detail)
- Domains very different from diffusion training data

**Hyperparameters and Configuration:**

| Parameter | Recommended | Notes |
|-----------|-------------|-------|
| Num Keyframes | 20-50 | More keyframes = smoother but slower |
| Interpolations per Keyframe | 1-3 | Subframe interpolation for smoothness |
| Diffusion Steps | 50-100 | More steps improve quality; 50 is good default |
| Guidance Scale | 7.5 | Standard for diffusion guidance |
| Noise Injection Strength | 0.1 | Adapt to semantic difference |
| High-Frequency Injection | 0.05 | Preserve detail across diverse images |
| Spherical Interpolation | always | Use spherical, never linear, for features |

**Common Pitfalls:**
- Using too few keyframes (jerky morphing)
- Linear instead of spherical interpolation (shortcuts create artifacts)
- Ignoring high-frequency components (detail loss)
- Applying noise injection too aggressively (blurry output)
- Not stepping down diffusion steps during speed-sensitive applications
- Trying to morph completely unrelated images (fundamental limits exist)

**Key Design Decisions:**
FreeMorph works by modulating diffusion model attention without fine-tuning. Spherical interpolation preserves feature geometry better than linear blending. Step-oriented weights gradually shift emphasis from one image to the other, creating directional morphing. High-frequency noise injection handles semantic differences between images. The method requires no per-image training because it operates within the pretrained diffusion model's existing machinery.

## Reference

Chu, M., Liu, Y., Xie, Y., Zhong, J., & Liao, Q. (2025). FreeMorph: Tuning-Free Generalized Image Morphing with Diffusion Model. arXiv preprint arXiv:2507.01953. https://arxiv.org/abs/2507.01953