What Is Spatial Video? A Deep Dive into Immersive Video Technology

Spatial video is revolutionizing how we capture and experience digital memories. This cutting-edge format goes beyond traditional 2D video, offering an immersive 3D experience that brings your memories to life. In this post, we'll explore what spatial video is, how it works, and how you can start creating your own spatial content today.

Understanding Spatial Video

At its core, spatial video is a 3D video format that captures depth information along with visual data. Unlike standard 2D videos, spatial videos create a sense of depth and dimensionality when viewed on compatible devices, such as virtual reality (VR) or augmented reality (AR) headsets.

Technical Implementation

Spatial video relies on several key technologies:

1. Multiview HEVC (MV-HEVC):

- This is the primary codec used for spatial video.

- MV-HEVC encodes multiple views (typically left and right eye views) into a single bitstream.

- It leverages inter-view prediction to efficiently compress the multiple viewpoints.

2. Stereoscopic Capture:

- Spatial videos are typically recorded using two cameras or lenses, simulating human binocular vision.

- These dual feeds capture slightly different perspectives of the same scene.

3. Depth Mapping:

- Some spatial video systems incorporate depth sensors or use AI algorithms to generate depth maps.

- These depth maps provide additional 3D information, enhancing the immersive experience.

4. Metadata:

- Spatial videos include metadata that describes how to display the content in 3D.

- This metadata ensures proper rendering on compatible devices.

Apple's Spatial Video Implementation

Apple has recently introduced spatial video capture with the iPhone 15 Pro and iPhone 15 Pro Max. Here's how their system works:

- Uses both the main and ultra-wide cameras in landscape orientation.

- Records at 1080p and 30 frames per second.

- Employs computational photography to align and process the dual camera feeds.

- Saves videos in HEVC format, with a file size of about 130MB per minute.

- Can be played as regular 2D videos on non-compatible devices.

- Designed for viewing on the upcoming Apple Vision Pro headset.

Viewing Spatial Videos

The true magic of spatial video comes alive on compatible devices like VR headsets. On Apple's Vision Pro, for example:

- Videos can be displayed at life-size scale.

- Viewers can choose between windowed playback or an immersive full-view mode.

- The 3D effect creates a sense of presence, as if looking through a window into the recorded moment.

Creating Spatial Videos with Spatial Media Toolkit

While native spatial video capture is currently limited to high-end devices like the iPhone 15 Pro, there's a powerful alternative available: the Spatial Media Toolkit. This innovative software allows you to convert your existing 2D videos into immersive spatial format.

Key features of the Spatial Media Toolkit:

1. AI-Powered Conversion:

- Utilizes advanced machine learning algorithms to analyze 2D video and generate depth information.

- Creates a convincing 3D effect from standard video input.

2. Compatibility:

- Converts videos for viewing on various VR and AR platforms, not just Apple's ecosystem.

- Supports a wide range of input video formats.

3. Customization:

- Offers adjustable parameters for depth effects.

- Allows fine-tuning to achieve the best results for each video.

4. Preview and Export:

- Provides real-time preview of the 3D effect within the app.

- Exports in formats compatible with Vision Pro and Meta Quest devices

5. Batch Processing:

- Convert entire libraries of 2D videos to spatial format efficiently.

6. High-Quality Output:

- Supports 4K and beyond resolution for both input and output.

7. Cross-Platform Availability:

- Available on iOS, macOS, and visionOS.

- Boasts a 4.8-star rating with over 500 ratings on the Vision Pro App Store.

The Spatial Media Toolkit democratizes spatial video creation, allowing anyone with a collection of 2D videos to enter the world of immersive content. Whether you're looking to breathe new life into old memories or create engaging 3D content for VR platforms, this tool bridges the gap between traditional and spatial video. You can download it here for visionOS, macOS and iOS.

The Future of Spatial Video

As technology advances, we can expect improvements in spatial video:

- More devices capable of native spatial video capture.

- Enhanced AI for even more realistic 2D to 3D conversion.

Spatial video represents the next frontier in personal media, offering a more immersive and lifelike way to capture and relive our memories. While native capture options are currently limited, tools like the Spatial Media Toolkit make it possible for anyone to start creating and experiencing spatial content today. As this technology continues to evolve, it's set to transform how we share and experience our digital memories.