Live Photos on the Web

_Update March 2020: After migrating this blog post, the code on this page no longer works. I'm leaving it here for historical reference. Mealwhile, if you want to display Live Photos on the web, Apple has tools for you.

TL;DR: I made Live Photos work on the web and it brought my computer to a grinding halt. Just here for the demos? Skip ahead!

What are Live Photos?

Live Photos, Apple's flagship iPhone photo party trick arrived on the scene with the 6s in the fall of 2015. The concept is simple, each time you take a picture, you get a high quality photo and 1.5 seconds of less-than-high quality video before and after the moment you snapped the shutter.

Apple implements this as a 12 megapixel, 4032 pixels wide by 3024 high JPEG image file and an up-to-three-seconds-long Quicktime MOV video file. As long as you stay within Live Photos approved areas of the Apple ecosystem, the technical details of this is all transparent to you. At the swipe of your wand…err…mouse, your Live Photos spring to life on command like they were straight off the front page of Harry Potter’s Daily Prophet.

Dead outside of the Apple ecosystem

As soon as you step out of these Live Photo friendly zones, by exporting your Live Photos from Photos, you’re left with two unwieldily source files and no way to recreate the Live Photo experience you know and love.

Half a year after launch, Live Photo support isn’t very widespread outside of the iOS apps of some large services. While some people can share Live Photos on Facebook (I’m not one of them), viewing them is limited to others on iOS.

Even within Apple’s Ecosystem, iCloud Photo Sharing’s public website and iCloud.com’s Photos app ignore remove the life from your Live Photos, showing only still images.

There’s also no good way to share your beautiful Live Photos with your Android or Windows using friends. You can either send them the still photo or convert the video to an animated GIF or a video clip. Either way, you lose the flavor that makes Live Photos magic.

Making photos live on the web

I decided to take a stab at recreating the Live Photos magic in a webpage. This webpage to be precise. After all, how hard could it be?

My approach would be to use a combination of the HTML img and video elements, displaying the still photo in the img tag and briefly displaying the video element over the img tag when the Live Photo is activated. I’d also make the code self-contained and reusable so that it could easily be used on any website, not just my own. Simple in theory…in practice not so simple.

Problem: Web Components only work in Chrome

The obvious approach was to use a technology called Web Components, which makes it easy to make self-contained reusable building blocks out of HTML and Javascript, to create a custom live-photo HTML tag. Unfortunately, as of April 2016, Chrome is the only browser that supports the technology out of the box. So I was stuck using slow, buggy, Javascript polyfills to emulate the functionality in other browsers. These polyfills have some negative side effects, including breaking Twitter javascript in Safari.

Problem: Only Safari on Mac can play Quicktime

The video container format used by Live Photos won’t play on most browsers and platforms. This means, the Quicktime MOV files we exported from Apple Photos need to be converted to a container format that works on most browsers, like MP4. To make matters more complicated, while the video format used by Live Photos will play in most browsers, the audio format, PCM, won’t. So besides changing video container formats, we also have to transcode the audio into a format that will play. No worries, our friend ffmpeg saves the day:

$ ffmpeg -y -i IMG_0008.MOV -vcodec copy -acodec libfaac IMG_0008.mp4

Problem: iPhone can’t play inline videos

The next problem I ran into is that the device that creates Live Photos, the iPhone, can’t play videos inline in its web browser. This restriction on iOS Safari on the iPhone and iPod (but not iPad), while ostensibly to save users from excessive battery and bandwidth usage, is really a holdover from the earlier days of the platform.

Our best option for iOS is to turn the video into an animated GIF. This loses a lot of quality and the audio, but will let us appear to play video in web page. It’s the best we can do.

Problem: Caching videos uses up a lot of bandwidth and memory

To get the magical effect of Live Photos, the photo needs to spring to life as soon as the user interacts with it. Achieving that magical “spring” means having the video downloaded and ready to play before the user interacts with it.

An average Live Photo takes up about 10 megabytes including the image, video and animated GIF. By conditionally loading both the animated GIF and video only on platforms where it was necessary, I was able to reduce that size down to about 6 or 7 megabytes. 7 megabytes for one asset is a lot even for a high speed connection.

However, put dozens of these on one page and it’s easy to create a web page that requires hundreds of megabytes to load and even more in memory to render. I learned this the hard way as Safari repeatedly made my Mac grind to a halt during development when I tried to load almost 200 Live Photos on one page.

iPhone, with its limited RAM, fares even worse. A page with 60 or Live Photos is enough to bring iOS Safari to a halt as it maxes out memory on the latest iPhone 6s Plus hardware.

Lessons

It’s difficult to deliver a magical Live Photo experience in a web browser a way that is both simple and doesn’t put large demands on the user’s bandwidth and computing resources. A really robust Live Photos on the web solution (which this is not) requires a lot of processing, converting and rendering on both the server side and complicated client side logic to select the right combination of video and photo for the user’s situation.

Enough talk, I want to see Live Photos!

I thought you’d never ask! I’ve put together a collection of more than 150 of my favorite unedited, taken-with-my-phone Live Photos.

Safe Demo

These demos are relatively safe. You’ll see scaled down thumbnails of each Live Photo and need to click visit each photo’s page to load the Live Photo.

{% for category in site.data.photos_by_category %}{{ category[0] }} {% endfor %}

Dangerous Demo

Instead of scaled down thumbnails, this demo puts multiple real, living Live Photos on each category page. You’ll regret clicking on these links if you’re on a metered connection or something much slower than 100 megabit. Memory constrained devices might also crash. You have been warned!

{% for category in site.data.photos_by_category %}{{ category[0] }} {% endfor %}

How can I get this on my site?

If you’re interested in using this commercially or have other questions about this, email me at livephotos@larrysalibra.com.

If you’d like to be notified if I publish the code and tools I used to enable Live Photos on my blog, submit your email below.

{% include mailing-list-live-photos.html %}

I’ve found a problem

Please accept my apologies! I’m not surprised! If you’re so inclined, open an issue on Github describing the problem. Thanks!

Found a better way to make Live Photos work on the web? Tweet at me or get in touch!