Photo Corners headlinesarchivemikepasini.com


A   S C R A P B O O K   O F   S O L U T I O N S   F O R   T H E   P H O T O G R A P H E R

Enhancing the enjoyment of taking pictures with news that matters, features that entertain and images that delight. Published frequently.

Apple Announces ProRaw Share This on LinkedIn   Tweet This   Forward This

14 October 2020

At Apple's product announcement yesterday, the company introduced a new photo format it called Apple ProRaw. It will be available on the iPhone Pro and Pro Max models by the end of the year and the company will also offer an API to facilitate incorporating the format in third-party image editing software.

During the presentation, Alok Deshpande, Apple's senior manager of camera software engineering, described the new format as combining a Raw file format with "the power of computational photography."

It's not the first time Apple has riffed on an established image format. JPEG inspired Apple's HEIF format that added live motion to stills.

The new format is designed to appeal to professional photographers who shoot Raw but miss the advantages of Apple's computational photography they get on their phone.

WHAT'S WRONG WITH RAW

A Raw file is not an image. It's a data file. To use the data to render an image, software has to read the data and construct an image.

That doesn't phase people who are used to processing Raw files but it discourages others whose first peek at a rendering of the raw data is always disappointingly soft and flat.

Generally that data is not optimized, nor can any one rendering on screen show the complete range of the data. In a Raw image editor like Adobe Camera Raw, you can appreciate the range of the data by changing the Highlight slider to recover highlights or the Shadows slider to reveal details in the darker areas of the image that are not at first apparent.

We say "generally' not optimized because Raw files from some manufacturers have in fact applied optimizations like lens corrections to handle distortion for wide angle lenses.

So Raw files provide a great deal of latitude, more than you can see at any one time, but are rendered plainly. It's that second issue that ProRaw addresses.

THE PIPELINE

On its Web site, Apple describes the new format (which is calls ProRAW) this way:

ProRaw gives you all the standard Raw information, along with the Apple image pipeline data. So you can get a head start on editing, with noise reduction and multi-frame exposure adjustments already in place -- and have more time to tweak color and white balance.

Pipeline data?

Alok Deshpande, Apple's senior manager of camera software engineering, ~talked about the pipeline~ during the keynote yesterday:

In order to achieve this, we constructed a new pipeline that takes components of the processing we do in our CPU, GPU, ISP and neural engine and combines them into a new deep image file, computed at the time of capture, without any shutter delay. And we do this for all four cameras, dynamically adapting for various scenes while maintaining our intuitive camera experience.

It's also worth watching whole but brief segment of yesterday's Apple Event in which Deshpande describes the format:

The computational refinements are, he notes, stored as instructions in the Raw file which includes:

  • Raw data captured by the sensor without processing, which may include multiple frames
  • Multi-frame processing
  • The results of computational photography like Deep Fusion and Smart HDR performed in camera after capture by the CPU, GPU, ISP and neural engine which processes the Raw data

That's what happens when you take an iPhone photo anyway. The Raw sensor data is passed to the image processing chip which applies recipes to optimize the image before you ever see it on the screen. But in an iPhone JPEG, the raw data is not saved, just the image data after optimization.

So the optimization is "baked in" as Deshpande put it. In ProRaw it isn't baked in. Think of it as a very sophisticated preset that you can then work with.

It looks as good as an iPhone JPEG when you open it in an image editor like Apple's Photos but it has the latitude of a Raw file for editing. That's the "deep image file."

CONCLUSION

Computational photography simply describes automatically manipulated image data. Artificial intelligence is employed at some level to make decisions about the manipulations.

It isn't entirely clear what happens to the computational refinements once you start fiddling with the image in editing software. Or what the API will allow other applications (and possibly even cameras) to do.

But we'll know soon enough. Just wait -- and see.


BackBack to Photo Corners