Notion(3-3): Pixel Transfer -- the act of taking pixel data from an unformatted memory buffer and copying it in OpenGL-owned storage governed by an image fromat.

[References]
1. "Pixel Transfer, from opengl.org/wiki/Pixel_Transfer"

A Pixel Transfer operation is 

• the act of taking pixel data from an unformatted memory buffer and copying it in OpenGL-owned storage governed by animage format. 
• Or vice-versa: copying pixel data from image format-based storage to unformatted memory. 
• There are a number of functions that affect how pixel transfer operation is handled; many of these relate to how the information in the memory buffer is to be interpreted.

Contents

 [hide] 

• 1Terminology
• 2Pixel transfer initiation
• 3Pixel transfer arguments
  • 3.1Pixel format
  • 3.2Pixel type
• 4Pixel transfer parameters
  • 4.1Pixel layout
  • 4.2Endian issues
  • 4.3Sub-image selection
• 5Format conversion

Terminology

Pixel transfers can either go from user memory to OpenGL memory, or from OpenGL memory to user memory (the user memory can be client memory or buffer objects). Pixel data in user memory is said to be packed. Therefore, transfers to OpenGL memory are called unpack operations, and transfers from OpenGL memory are called pack operations.

Pixel transfer initiation

There are a number of OpenGL functions that initiate a pixel transfer operation. These functions are:

Transfers from OpenGL to the user:

• glReadPixels​: Reads pixel data from the currently bound framebuffer object or the default framebuffer. If this is a color read, then it reads from the buffer designated by glReadBuffer.
• glGetTexImage​: Reads all of the pixels from a mipmap level of the bound texture object.

Transfers from the user to OpenGL:

• glTexImage*​: Allocates mutable storage for a mipmap level of the bound texture object and optionally writes pixel data to that mipmap level.
• glTexSubImage*​: Writes the user's pixel data to some part of the given mipmap of the bound texture object.

There are also special pixel transfer commands for compressed image formats. These are not technically pixel transfer operations, as they do nothing more than copy memory to/from compressed textures. But they are listed here because they can use pixel buffers for reading and writing.

• glCompressedTexImage*​: Allocates mutable storage for a mipmap level of a texture that uses a compressed image format and writes compressed data to that mipmap level.
• glCompressedTexSubImage*​: Writes compressed data to some part of the given mipmap level.
• glGetCompressedTexImage​: Reads all of the compressed texture data into the user's memory.

The discussion below will ignore the compressed texture functions, since none of what is discussed pertains to them.

Pixel transfer arguments

With the exception of the compressed texture functions, all functions that initiate pixel transfers take 3 parameters:

 GLenum format​, GLenum type​, void *data​

The data​ pointer is either a client memory pointer or an offset into a buffer object. The switch for this is based on whether a buffer object is bound to the GL_PIXEL_PACK/UNPACK_BUFFER binding, depending on whether the pixel transfer is a pack or unpack operation. For ease of discussion, let us call this "client memory" regardless of whether it refers to a buffer object or not.

The format​ and type​ parameters describe the data in client memory. These do not refer to the format of the texture object (in the case ofglTexImage*​ calls). Together, these two values describe how each pixel is specified in client memory.

Pixel format

Pixels of the client data can be color values, depth values, combined depth/stencil values, or just stencil values. Color values can have up to four components: R, G, B and A. Depth and stencil values only have one component. Combined depth/stencil values have two components.

The format​ parameter of a pixel transfer function defines the following:

• The basic type of data that is being read/written from/to: Color, depth, stencil, or depth/stencil. This must match the image format of the image being read/written from/to.
• The order of the individual components within each pixel.
• For color values, whether or not the data should be converted to/from floating-point values when being read/written. For more details, see below.

If only depth values are being transferred, then GL_DEPTH_COMPONENT is used. If only stencil values are being transferred, then GL_STENCIL_INDEX is used. If combined depth/stencil values are being transferred, then GL_DEPTH_STENCIL is used. The latter can only be used with Textures that explicitly use a depth/stencil format, or in Framebuffer pixel read operations for Framebuffer Objects that have an explicit depth/stencil format.

For color formats, there are more possibilities. GL_RED, GL_GREEN, and GL_BLUE represent transferring data for those specific components (GL_ALPHAcannot be used). GL_RG represents two components, R and G, in that order. GL_RGB and GL_BGR represent those three components, with GL_BGR being in reverse order. GL_RGBA and GL_BGRA represent those components; the latter reverses the order of the first three components. These are the only color formats supported (note that there are ways around that).

All of the above format specifiers implicitly mean that the pixels are floating-point values. For integer pixel types, using a floating-point format means that the pixels will be assumed to be normalized integers. And thus they will be interpreted as normalized values.

If you want to transfer integral data to integral image formats, you must suffix the pixel format with "_INTEGER". This states that the client-side pixel data is integer rather than floating-point. You should only use the "_INTEGER" format suffix with integral image formats.

Note: Values for the format​ parameter look a lot like image formats. They are not! Do not confuse the two. While in many cases they must match to some degree, they do completely different things. If you always use sized image formats for texture, then they will never match, since the format​ parameter cannot have a size.

Pixel type

The type​ parameter of a pixel transfer function defines how many bytes each of the components defined by the format​ take up. There are two kinds oftype​ values: values that specify each component as a separate byte value, or values that pack multiple components into a single value.

For example, GL_RGBA format​ combined with GL_UNSIGNED_BYTE type​ means that each pixel will take up 4 unsigned bytes. The first byte will be R, the second will be G, and so on. GL_UNSIGNED_BYTE as a type​ is a per-component type; each component has this size.

The possible per-component formats use the enumerators for OpenGL types:

• GL_(UNSIGNED_)BYTE: 1 byte
• GL_(UNSIGNED_)SHORT: 2 bytes
• GL_(UNSIGNED_)INT: 4 bytes
• GL_HALF_FLOAT: 2 bytes
• GL_FLOAT: 4 bytes

However, there are packed arrangements of pixel data that are useful, where each component is packed into non-byte-length values. A common example is a 16-bit RGB color, where the red and blue components take up 5 bits and the green is 6.

To specify this kind of data in OpenGL, we use a packed type​ value. Packed type fields are specified as follows:

 GL_[base type​]_[size1​]_[size2​]_[size3​]_[size4​](_REV​)
 

The parenthesis represent an optional value.

The base type​ is the OpenGL type enumerator name of the fully packed value. These values are always an unsigned integer type, of a size large enough to hold a whole color. So the 5-6-5 RGB colors would be stored into a 16-bit unsigned integer, which means using UNSIGNED_SHORT.

The size​ values represent the sizes of the components, in that order. We want the components to be 5 for the first, 6 for the second, and 5 for the third. Since there is no fourth component, there is no size4​ value.

Therefore, the type​ that represents 5-6-5 colors is GL_UNSIGNED_SHORT_5_6_5.

This can have a _REV​ at the end of it. This would mean to reverse the order of the components in the data. In "REV" mode, the first component, the one that matches the first 5, would go into the last component specified by the format​.

So if we have GL_RGB format​ combined with the GL_UNSIGNED_SHORT_5_6_5_REV type​, the first 5 bits will go into the blue component of the color. Note that this is functionally identical to GL_BGR with GL_UNSIGNED_SHORT_5_6_5.

With the exception of 2 special cases, the number of components in the format​ must match the number of components provided by a packed type​. Some of the packed types even put restrictions on the component ordering, or the kinds of components the format​ can be used with.

OpenGL defines the possible sizes. They are:

• 3_3_2 (2_3_3_REV): unsigned bytes. Only used with GL_RGB.
• 5_6_5 (5_6_5_REV): unsigned shorts. Only used with GL_RGB.
• 4_4_4_4 (4_4_4_4_REV): unsigned shorts.
• 5_5_5_1 (1_5_5_5_REV): unsigned shorts.
• 8_8_8_8 (8_8_8_8_REV): unsigned ints.
• 10_10_10_2 (2_10_10_10_REV): unsigned ints.
• 24_8 (no _REV): unsigned ints. Only used with GL_DEPTH_STENCIL.
• 10F_11F_11F_REV (no non-REV): unsigned ints. These represent floats, and can only be used with GL_RGB. This should only be used with images that have the GL_R11F_G11F_B10F image format.
• 5_9_9_9_REV (no non-REV): unsigned ints. Only used with GL_RGB; the last component (the 5. It's REV) does not directly map to a color value. It is a shared exponent. Only use this with images that have the GL_RGB9_E5 image format.

There is one very special packed type​ field. It is GL_FLOAT_32_UNSIGNED_INT_24_8_REV. This can only be used in tandem with images that use theGL_DEPTH32F_STENCIL8 image format. It represents two 32-bit values. The first value is a 32-bit floating-point depth value. The second breaks the 32-bit integer value into 24-bits of unused space, followed by 8 bits of stencil.

Pixel transfer parameters

The format​ and type​ parameters describes only the representation of a single pixel of data. The layout of the data otherwise is controlled by various global parameters, set by the glPixelStore[if]​ functions.

Pack and unpack operations (reads from OpenGL memory and writes to OpenGL memory, respectively) use different sets of parameters to control the layout of pixel data. All pack (read) parameters begin with GL_PACK, while all unpack (write) parameters begin with GL_UNPACK. Both kinds of operations have the same parameters which have the same meaning, but they only affect that particular kind of operation.

The GL_PACK_ALIGNMENT parameter will not affect any uploads (unpack) to OpenGL memory.

Pixel layout

The layout of pixel data is as follows.

The data is arranged in "rows". Each row represents a horizontal span in the pixel transfer, based on the width​ parameter in the transfer operation. Each pixel within a row is directly adjacent to the other pixels. So there is no space between pixels.

If the format​ is GL_RGB, and the type​ is GL_UNSIGNED_BYTE, then the size of a pixel is 3. A width​ of 16 pixels means that the total byte length of a single row of pixels is 48 bytes. If the format​ were GL_RGBA, then the pixel size would be 4 and the size of a row would be 64.

If the pixel transfer operation is two-dimensional or higher, then there will be height​ number of rows, where height​ is a parameter of the pixel transfer function. The first row is the bottom of the image in OpenGL space. The next row is the row above that, and so on.

Unlike pixels however, rows are not necessarily directly contiguous in memory. The bytes on each row must begin on a specific alignment. This alignment is user defined with the GL_PACK/UNPACK_ALIGNMENT parameter. This value can be 1, 2, 4, or 8.

For example, if the format​ is GL_RGB, and the type​ is GL_UNSIGNED_BYTE, and the width​ is 9, then each row is 27 bytes long. If the alignment is 8, this means that the second row begins 32 bytes after the first. If the alignment is 1, then the second row begins 27 bytes after the first row.

If the pixel transfer operation is three-dimensional, then there is a depth​ as well as width​ and height​. This changes nothing about the layout; it only changes how many rows there are. Instead of height​ rows, there are height​ * depth​ rows.

Endian issues

Client pixel data, the "packed" data, is always in client byte ordering. So on a little-endian machine, unsigned integers are in little-endian order. On a big-endian machine, unsigned integers are in big-endian order.

If you wish to change this, you can use glPixelStore to set GL_PACK/UNPACK_SWAP_BYTES to GL_TRUE. This will cause OpenGL to perform byte swapping from the platform's native endian order to the order expected by OpenGL.

Sub-image selection

TODO: This section needs to be filled in.

Format conversion

Pixels specified by the user must be converted between the user-specified format (with format​ and type​) and the internal representation controlled by the image format of the image.

The pixels in the client memory are either in some form of floating-point representation or integral values. The floating-point forms include normalized integers, whether signed or unsigned. If the format​ parameter does not specify the "_INTEGER" suffix, then all integer values are assumed to be normalized integers. If the format​ parameter specifies "_INTEGER", but the type​ is of a floating-point type (GL_FLOAT, GL_HALF_FLOAT, or similar), then an error results and the pixel transfer fails. Also, if "_INTEGER" is specified but the image format is not integral, then the transfer fails.

When data is being transferred to an image, pixel values are converted to either floating-point or integer values. If the image format is normalized, the values that are written are clamped to [0, 1] and normalized. If the image format is integral, then the integral input values are copied verbatim.

This process is reversed for writing to client data.

Note: If the OpenGL implementation can get away with it, it will not do the conversion. So if you upload normalized integers to a normalized integer internal format, the implementation won't bother with the conversion since it would be a no-op. Always try to match the given format with the image format.

Category: 

• Textures

....
....