Tone Mapping HDR Luminance to an SDR-compatible Range

Android 13 introduces a vendor-configurable static library called libtonemap, which defines tone mapping operations and is shared with the SurfaceFlinger process and Hardware Composer (HWC) implementations. This feature enables OEMs to define and share their display tone mapping algorithms between the framework and vendors, lessening a mismatch in tone mapping.

Prior to Android 13, display-specific tone mapping operations weren’t shared between the HWC, SurfaceFlinger, and apps. Depending on the rendering path, for HDR content, this led to mismatches in image quality, where the HDR content was tone mapped to an output space in different ways. This was perceptible in scenarios such as screen rotation, where the composition strategy changes between the GPU and the DPU, and in differences in rendering behavior between TextureView and SurfaceView.

This page describes the interface, customization, and validation details of the libtonemap library.

Interface to the tone mapping library

The libtonemap library contains CPU-backed implementations and SkSL shaders, which can be plugged in by SurfaceFlinger for GPU-backend composition and by the HWC for generating a tone mapping look-up table (LUT). The entry point to libtonemap is android::tonemap::getToneMapper(), which returns an object that implements the ToneMapper interface.

The ToneMapper interface supports the following capabilities:

Generate a tone-mapping LUT

The interface ToneMapper::lookupTonemapGain is a CPU implementation of the shader defined in libtonemap_LookupTonemapGain(). This is used by unit tests in the framework, and can be used by partners for assistance with generating a tone-mapping LUT inside their color pipeline.

libtonemap_LookupTonemapGain() takes in color values in absolute, unnormalized linear space, both in linear RGB and in XYZ, and returns a float describing how much to multiply the input colors in linear space.
Generate an SkSL shader

The interface ToneMapper::generateTonemapGainShaderSkSL() returns an SkSL shader string, given a source and destination dataspace. The SkSL shader is plugged into the Skia implementation for RenderEngine, the GPU-accelerated compositing component for SurfaceFlinger. The shader is also plugged into libhwui, so that HDR-to-SDR tone mapping can be performed efficiently for TextureView. Because the generated string is in-lined into other SkSL shaders used by Skia, the shader must adhere to the following rules:
- The shader string must have an entry point with the float libtonemap_LookupTonemapGain(vec3 linearRGB, vec3 xyz) signature, where linearRGB is the value of the absolute nits of the RGB pixels in linear space and xyz is linearRGB converted into XYZ.
- Any helper methods used by the shader string must be prefixed with the string libtonemap_ so that framework shader definitions don’t conflict. Similarly, input uniforms must be prefixed with in_libtonemap_.
Generate SkSL uniforms

The interface ToneMapper::generateShaderSkSLUniforms() returns the following, given a metadata struct describing metadata from different HDR standards and display conditions:
- A list of uniforms that are bound by an SkSL shader.
- The uniform values in_libtonemap_displayMaxLuminance and in_libtonemap_inputMaxLuminance. These values are used by framework shaders when scaling the input into libtonemap, and normalizing the output as applicable.
Currently the process of generating uniforms is agnostic to the input and output dataspace.

Customization

The reference implementation of the libtonemap library produces acceptable results. However, because the tone mapping algorithm used by GPU composition can differ from that used by the DPU composition, using the reference implementation can cause flicker in some scenarios such as the rotation animation. Customization can resolve such vendor-specific image quality issues.

OEMs are strongly encouraged to override the implementation of libtonemap to define their own ToneMapper subclass, which is returned by getToneMapper(). When customizing the implementation, partners are expected to do one of the following:

Modify the implementation of libtonemap directly.
Define their own static library, compile the library as a standalone, and replace libtonemap library’s .a file with the one generated from their custom library.

Vendors don’t need to modify any kernel code, but multiple vendors must communicate details about the DPU tone-mapping algorithms for proper implementation.

Validation

Follow these steps to validate your implementation:

Play HDR videos on screen of any HDR standards that your display system supports, such as HLG, HDR10, HDR10+, or DolbyVision.
Toggle GPU composition to ensure that there's no user perceptible flicker.

Use the following adb command to toggle the GPU composition:
```
adb shell service call SurfaceFlinger 1008 i32 <0 to enable HWC composition,
1 to force GPU composition>
```

Common issues

The following issues can occur with this implementation:

Banding is caused when the render target used by GPU composition is of lower precision than the typical value for HDR content. For instance, banding can occur when an HWC implementation supports opaque 10-bit formats for HDR such as RGBA1010102 or P010, but requires that GPU composition writes to an 8-bit format like RGBA8888 to support alpha.
A subtle color shift is caused by quantization differences if the DPU operates at a different precision than the GPU.

Each of these issues is related to the relative precision differences of the underlying hardware. A typical workaround is to ensure that there's a dithering step in the lower precision paths, making any precision differences less human perceptible.