Verilog code for various color-space manipulations, optimized for efficiency.
I wasn't satisfied with the quality of the image coming from the MIPI-to-HDMI pipeline example for the Spartan Edge Accelerator dev board, so I created some filter circuits in Verilog that would allow me to manipulate the image.
This code was implemented within a complete camera pipeline which streams frames from a MIPI/CSI interface to an HDMI output in real time. The filter block is inserted between the demoisaicing stage (bayer2rgb) and the HDMI interface (rgb2dvi), as shown in the diagram above.
In short, our eyes perceive brightness non-linearly, which affects the way we perceive color in images displayed on screens. Monitors are intentionally designed to attenuate that brightness unevenly across the color space. Then, the color information (from an image, for instance) has to be gamma-corrected for the final result to become similar to what the original subject looked like.
The theoretical formulas for doing gamma correction on RGB are as follows:
As you can imagine, the floating-point power law can't easily or efficiently be implemented in a digital circuit. To compute the gamma-corrected values, a piece-wise linear function can be used. This was developed in excel (see folder) for gamma values of 2.2 and 1.4, with 2.2 being the most common.
The slopes are intentionally designed to be implemented as logical left/right shifts.
Contrast is adjusted using a simple linear function, clipped to the 0 to 255 range.
Brightness is increased by using a simple formula. Each color component is increased by 25% and clipped to 255 if required.
Compared to simply multiplying RGB, Y'UV goes one step further by assigning weights to each color component (as a way of accounting for how red and green participate more towards our perception of brightness than blue does). Y'UV is used widely in TVs. The standard for Y'UV implemented here is based on BT.601 (the standard for SDTV).
The conversion between Y'UV and RGB is a linear transform between the two spaces, which boils down to the following matrix multiplications:
Signed floating-point matrix multiplication is a hard task in a digital circuit. It may be synthesizable, but it would take up a lot of die/FPGA area (or a lot of DSP slices), and it would be slow. Therefore, to create a more efficient design, an approximation of the transforms can be used.
The above matrix coefficient values were obtained by multiplying the previous linear equations by 32, rounding to the nearest integers, then dividing by 32. This is equivalent to the following equations:
An even more accurate implementation could be obtained by multiplying by 64, or even 128. These equations, in turn, can be implemented very efficiently in RTL by using a combination of signed addition/subtraction and arithmetic left/right shifts. Despite the relative complexity of normal matrix multiplication, using this approximation approach, the delays become minimal.
Once in the Y'UV space, the value of Y' (the luma, or luminance) can be manipulated to improve image quality.
Utilization on a Spartan-7 FPGA was seen to be ~500 LUTs and ~100 FFs. This includes all features listed above as well as some extra logic. Since they're implemented as automatic verilog functions, unused features can easily be removed to reduce the LUT usage.