Layers and displays are two primitives that represent composition work and interactions with the display hardware.
Layers
A Layer is the most important unit of composition. A layer is a
combination of a surface and an instance of
SurfaceControl. Each layer has a set of properties that
define how it interacts with other layers. Layer properties are described in the table below.
| Property | Description |
|---|---|
| Positional | Defines where the layer appears on its display. Includes information such as the positions of a layer's edges and its Z order relative to other layers (whether it should be in front of or behind other layers). |
| Content | Defines how content displayed on the layer should be presented within the bounds defined by the positional properties. Includes information such as crop (to expand a portion of the content to fill the bounds of the layer) and transform (to show rotated or flipped content). |
| Composition | Defines how the layer should be composited with other layers. Includes information such as blending mode and a layer-wide alpha value for alpha compositing. |
| Optimization | Provides information not strictly necessary to correctly composite the layer, but that can be used by the Hardware Composer (HWC) device to optimize how it performs composition. Includes information such as the visible region of the layer and which portion of the layer has been updated since the previous frame. |
Displays
A display is another important unit of composition. A system can have multiple displays and displays can be added or removed during normal system operations. Displays are added/removed at the request of the HWC or at the request of the framework. The HWC device requests displays be added or removed when an external display is connected or disconnected from the device, which is called hotplugging. Clients request virtual displays, whose contents are rendered into an off-screen buffer instead of to a physical display.
Virtual displays
SurfaceFlinger supports an internal display (built into the phone or tablet), external displays (such as a television connected through HDMI), and one or more virtual displays that make composited output available within the system. Virtual displays can be used to record the screen or send the screen over a network. Frames generated for a virtual display are written to a BufferQueue.
Virtual displays may share the same set of layers as the main display (the layer stack) or have their own set. There's no VSYNC for a virtual display, so the VSYNC for the internal display triggers composition for all displays.
On HWC implementations that support them, virtual displays can be composited with OpenGL ES (GLES), HWC, or both GLES and HWC. On nonsupporting implementations, virtual displays are always composited using GLES.
Case study: screenrecord
The screenrecord command allows the user to
record everything that appears on the screen as an .mp4 file on
disk. To implement this, the system receives composited frames from
SurfaceFlinger, writes them to the video encoder, and then writes the encoded
video data to a file. The video codecs are managed by a separate process
(mediaserver), so large graphics buffers have to move around the
system. To make it more challenging, the goal is to record 60 fps video at
full resolution. The key to making this work efficiently is BufferQueue.
The MediaCodec class allows an app to provide data as raw bytes in buffers,
or through a surface. When screenrecord requests access to a video
encoder, the mediaserver process creates a BufferQueue, connects
itself to the consumer side, then passes the producer side back to
screenrecord as a surface.
The screenrecord utility then asks SurfaceFlinger to create a
virtual display that mirrors the main display (that is, it has all of the same
layers), and directs it to send output to the surface that came from the
mediaserver process. In this case, SurfaceFlinger is the producer
of buffers rather than the consumer.
After the configuration is complete, screenrecord triggers when
the encoded data appears. As apps draw, their buffers travel to SurfaceFlinger,
which composites them into a single buffer that's sent directly to the video
encoder in the mediaserver process. The full frames are never
seen by the screenrecord process. Internally, the
mediaserver process has its own way of moving buffers around that
also passes data by handle, minimizing overhead.
Case study: simulate secondary displays
The WindowManager can ask SurfaceFlinger to create a visible layer for which SurfaceFlinger acts as the BufferQueue consumer. It's also possible to ask SurfaceFlinger to create a virtual display, for which SurfaceFlinger acts as the BufferQueue producer.
If you connect a virtual display to a visible layer, a closed loop is created
where the composited screen appears in a window. That window is now part of the
composited output, so on the next refresh the composited image inside the window
shows the window contents as well. To see this in action, enable
Developer options in Settings, select
Simulate secondary displays, and enable a window. To see
secondary displays in action, use screenrecord to capture the act
of enabling the display then play it back frame by frame.