Saving Adaptive HDR Photos

We’ve been updating Kodist, our passion project that takes no-AI photos on an iPhone. We take in Bayer RAW data from the iPhone camera, apply colour profiles, and save it as a photo into the Photo Library — simple, right? That’s until we wanted to support HDR (High Dynamic Range, or EDR for Extended Dynamic Range, if you prefer).

One challenge we had was to save the Bayer RAW photo in an HDR colour space — colours can go beyond typical 1.0 in the SDR range, and the photos look instantly bright and lifelike.

Recap on ISO HDR vs. Adaptive HDR

Shortly put the best I know how, ISO HDR is the original HDR standard, and Adaptive HDR is what Adobe began adopting with Apple to recently follow.

At the 9:38 mark, this WWDC 2024 video (the year of iOS 18) explains it the best: Use HDR for dynamic image experiences in your app. Essentially, an Adaptive HDR photo is composed of an SDR image + a gain map image. It solves the problem of the hardware chaos so that the same photo can look good on both HDR and SDR displays.

The options for saving Adaptive HDR

In the aforementioned video, at 31:37 mark, Apple Engineer David Hayward explained that there are 3 ways you can edit and save an HDR photo:

1. Edit HDR only, and save as a non-adaptive HDR photo
2. Edit SDR photo, edit the gain map, and save both as an Adaptive HDR photo
3. Edit SDR and HDR photos, and save them as an Adaptive HDR photo

In Kodist, we chose Option 3 because:

• We want the photos to look good on both SDR and HDR devices. Option 1 is out.
• Our input is a Bayer RAW photo data (i.e. not Apple Pro RAW), and gain map is not something readily available. As a result, Option 2 is out.

The complete workflow

You will need 3 components to make Adaptive HDR work:

• The edited image in SDR
• The edited image in HDR
• The correct, SDR colour space (i.e. Display P3, not Display P3-PQ)

We are using CIRAWFilter to get the CIImage from Bayer RAW data. To generate the photo in HDR, you’ll need to tweak some parameters:

let rawFilter = CIRAWFilter(imageData: rawImageData, identifierHint: nil)!
rawFilter.isGamutMappingEnabled = true // Required for tone-mapping
rawFilter.extendedDynamicRangeAmount = 1.0 // Required

let hdrImage = rawFilter.outputImage!

(Note that extendedDynamicRangeAmount can be any float value between 0 and 2. We found that 2 is a bit excessive for our use case; 1 seems best.)

With this HDR image, you can do editing with CIFilter chains for your own purposes. Once you’re settled, you will need to generate the corresponding SDR image:

// import CoreImage.CIFilterBuiltins

let toneMapFilter = CIFilter.toneMapHeadroom()
toneMapFilter.inputImage = hdrImage
toneMapFilter.targetHeadroom = 1

let sdrImage = toneMapFilter.outputImage!

This tone map filter sets the headroom to 1, or the SDR headroom.

Now you have all 3 components! Time to render the Adaptive HDR photo to disk:

let options: [CIImageRepresentationOption : Any] = [
	.hdrImage: hdrImage, // Pass in HDR image here
]

let p3ColorSpace = CGColorSpace(name: .displayP3)!

let heifData = ciContext.heifRepresentation(
	of: sdrImage,
	format: .RGB10,
	colorSpace: p3ColorSpace,
	options: options)!

If you’re wondering what the key .hdrImage does, you are not alone! This isn’t mentioned in Apple’s developer documentations, but was alluded to in this WWDC session:

• If this key is used, Core Image calculates the per-pixel difference between the HDR and SDR images, and generates a gain map for you.
• The value of this key is a CIImage object in the HDR range.

Remember that the code samples in the video was released during WWDC, the week of iOS 18 beta 1. A few things must have changed including the lower-cased .hdrImage spelling, and that format: is no longer an optional parameter for this image rendering method.

Other links

Apart from Core Image, Apple provides another framework, Image I/O, for lower-level image data and metadata (auxiliary data) handling. While it seemed tempting to try things, we decided that Image I/O remained something for people who want better data control or who look to juice the best performance out of the app.

If you are looking, JuniperPhoton has a great discussion and code examples in his Substack. Rio Ogino also has a great 2-part discussion on ISO HDR vs. Adaptive HDR, as well as low-level rendering in Metal.

Related, if you are interested in displaying (not saving) HDR photos, checkout this WWDC 2023 session, Support HDR images in your app. This should have all the things you need in your UI.

Closing thoughts

I hope you found this article helpful and that you’re one step closer to making better HDR photos. There are still many standards surrounding how HDR works, but I think Apple is leaning towards Adaptive HDR with iOS 18. As with anything in the world, you should probably keep learning as everything can change year to year.