Tweet
Broadcast and Distribution - Think of this as Part 3 of my recent series of Display Dailys on expanded color gamuts and the future of video encoding. The series started, innocently enough, with Canon’s support of the BT.2020 standard and continued with how Nanosys quantum dots can almost meet the BT.2020 color gamut standard. In that second article, I mentioned that I’d heard from my former colleague at Philips, Jeroen Stessen, about how he, Charles Poynton and Rutger Nijland were working on a color encoding system for high dynamic range (HDR) and wide color gamuts (WCG) that was based on the u’v’ CIE values. Traditionally, colors for broadcast have been encoded in color difference formulas, rather than encoding the CIE color co-ordinates directly.
Stessen kindly provided me with the paper as presented at the recent IBC conference in Amsterdam, plus an updated and expanded version to be presented by Poynton at the upcoming SMPTE 2014 Technical Conference and Exhibition in Hollywood, October 20 – 23. He also provided email explanations of some of the fine points of the system.
The need for encoding wide color gamut information is obvious, as I discussed last week. Many colors content creators want to display on a consumer’s TV simply cannot be encoded in the current system. The flip side of the coin is also there – many consumer TV and mobile displays are wide color gamut, but there is no wide color gamut material available to show on them. These include not only OLED displays, but displays based on quantum dots from Nanosys or QD Vision and even displays based on conventional LEDs. The electronics driving these displays stretch normal color gamut material to fill the wide color gamut of the display, but this is not the same as having native material. While much has been made of the lack of native UltraHD content to show on UltraHD displays, I’ve heard little about the lack of WCG content to show on WCG displays.
xvYCC or Extended-gamut YCC (also called x.v.Color) is a color space that was proposed by Sony and approved as a standard in January 2006 as IEC 61966-2-4. This is a backward-compatible extension to HDTV color encoding that supposedly gives a color gamut of 1.8x the color gamut of Rec. 709. This standard has received only modest support, in part because it is not supported by DVDs. One problem with xvYCC encoding is it is backward compatible with an 8-bit encoding standard and 8 bits aren’t really enough to encode these extra colors. Blu-Ray can support xvYCC and in 2013 Sony announced “Mastered in 4K” Blu-ray disks that not only have native UltraHD content but have xvYCC colors as well.
One of the problems with xvYCC as a backward compatible system, is it sticks with the 8-bit encoding of HDTV or 24 bits per pixel for 4:4:4 encoding. Unfortunately, a wide color gamut display can display many additional color/brightness combinations at the pixel than a normal color gamut system and 24 bits is not really enough to encode these extra combinations. This is one of several of the justifications for HDR, or 10 bits per color, 30 bits per pixel with 4:4:4 encoding.
One of the problems with u’v’ encoding and other systems such as DCI and ACES that can encode all possible colors, is the reverse of a problem with Rec. 709 encoding. While Rec. 709 cannot encode some colors that should be displayed, u’v’ and these other systems can encode many colors that are physically impossible and cannot be encoded. These non-physical colors take up space in code tables and are part of the reason why 30 bits are needed as opposed to 24.
Of course, these extra code groups could be used for something other than color. Closed captions and multi-language subtitles? Multilingual sound? Metadata? Second screen information? Picture in picture? If I know the CE industry, once a channel is established, they will find data to fill it. With advertising, if nothing else.
Figure 7 from Poynton, et. al. Rec. 709 and Rec. 2020 are both color spaces and color gamuts. Only colors within these triangles can be encoded by these systems. P# is the digital cinema color gamut but the encoding DCI color space includes all possible colors, as defined by the CIE horseshoe curve. U’v’ encoding can encode all colors within the dotted line, including some outside the CIE gamut of real colors.
Figure 7 from Poynton, et. al. Rec. 709 and Rec. 2020 are both color spaces and color gamuts. Only colors within these triangles can be encoded by these systems. P# is the digital cinema color gamut but the encoding DCI color space includes all possible colors, as defined by the CIE horseshoe curve. U’v’ encoding can encode all colors within the dotted line, including some outside the CIE gamut of real colors.
One must distinguish between color space and color gamut. Color space is the range of colors that can be encoded into a video signal and color gamut typically refers to the colors a display is capable of reproducing. In CRT days, and the HDTV Rec. 709 standard was created in CRT days, color space and color gamut were identical. All CRTs used essentially the same red, green and blue phosphors and the color space of Rec. 709, NTSC, PAL and SECAM encoding were designed to encode the colors that could be created by these CRT phosphors and no other colors. Content editors in post-production used Sony Trinitron CRTs with these same phosphors so they saw, in terms of color, pretty much what the consumer saw.
Times have changed, to put it mildly. The Trinitron has pretty much vanished from post-production facilities, replaced by LCD displays. While many CRT TVs are still in the field (I have one!) no new ones are being sold, all new display sales are flat panel displays such as LCD, OLED, plasma or projection displays based on mercury lamps, xenon lamps, LEDs or lasers. “Laser” projectors is pretty much a misnomer, most of them are phosphor-based and sometimes have an even smaller color gamut than CRTs. Displays with three, four, or more primary colors are available to the consumer. There is a huge variation in the color gamut for all of these displays, but they are all limited by the colors of the input signal, which is intended to be displayed on a CRT.
The TV, smartphone and tablet manufacturers , of course, process this input video to make it look as colorful as possible on their display. Consumers like bright, saturated colors, even if they are not contained in the original content. There are some very sophisticated algorithms out there to expand the color space encoded at the input to match the color gamut of the display, most of them are proprietary and none of them use a common defined standard. Since this gamut expansion is not standardized, the content creator has no idea what his content will look like on a “typical” consumer display.
A question arises at this point. Why has the consumer, who has shown a definite preference for wide color gamuts and colorful content, been deprived of native WCG content? Most consumers are relatively indifferent to UltraHD content but there has been a strong push to provide them with native UltraHD content.
One reason is UltraHD content is available since most Hollywood films are made with UltraHD resolution, even if they are often shown in the local multiplex at 2K resolution. Of course, the Hollywood content is also made with a WCG and HDR, but these are lost when it is transferred to video.
Another of the reasons is standards. Existing standards, e.g. Blu-ray and HDMI, support UltraHD native content but existing standards for consumer content do a poor job of supporting WCG or HDR content. Standards in the consumer electronics business evolve slowly. No one expects that at the upcoming SMPTE technical conference, after the u’v’ WCG HDR system from Poynton, Stessen and Nijland is presented, the audience will stand, cheer and start chanting “u’v’ Now! u’v’ Now!” The business doesn’t work that way.
The WCG and HDR content is there and the consumer has a desire to see the content. Hopefully some large consumer electronics manufacturer will decide WCG and HDR will give them a competitive advantage over their rivals and will develop a system to deliver this content to viewers. After they develop the system, the CE manufacturer can submit it to the IEC or SMPTE as a proposed standard, and the standard can (eventually) be approved and adopted by other companies. It’s likely to take years, but that’s the way the business works. Unless Apple is the CE manufacturer that introduces WCG and HDR, of course. iColor anyone? Visible only if you have an AppleTV? –Matthew Brennesholtz
3 Responses to Wide Color Gamuts and High Dynamic Range
So, Matt, let’s take this a step further. Consumers like wide gamuts and saturated colors, but do they really care if those colors are non-standard? Given the way consumers used to oversaturate the colors on their CRT TVs — which had knobs for the purpose rather than today’s menu structures, which many consumers never use — I suspect that most of them don’t. The push here will come from the content creators, including big-name film directors, who care very much that viewers are seeing the same colors the creators have painstakingly produced.
This is not to say the creators are wrong. A scene with muted colors and a blue tinge will create a very different emotional response in most viewers than the same scene with vibrant colors and a yellow tinge, even if many viewers don’t understand why. In this context, it’s interesting that when Kodak and THX talk about their content and TV certification programs, they do not talk about giving the consumer what he wants. They talk about making sure the creator’s intentions are accurately transferred to the screen.
Kenneth Werner
September 24, 2014 at 3:27 pm
Log in to Reply
I rather think Matt is right on this one. Joe Public will not understand colour in the stated terms. He will only buy the expanded colour when it is sold to him/her as “better”. The recent history or TV “improvements” shows this pretty clearly.
Even now, most people do not understand the impact of the various control functions they may have in their TV’s. How many people can even comprehend the video control of contrast and brightness? I have found very few even in ght display business that properly comprehend what these things do. But – they can mostly all be sold on “better”!
John Thomas
September 24, 2014 at 4:40 pm
Log in to Reply
Dear Matt, I see some minor errors in your story.
I don’t know that Poynton will present anything at a next SMPTE meeting, I only know that a certain group within SMPTE wanted to know more from us, so I have updated our paper to the last status. I don’t know yet that I am free to distribute that, Poynton needs to decide that when he gets back.
For xvYCC I refer to the papers that Katoh and I presented at the 2007 HPA Tech Retreat. xvYCC should work well enough with DVD, and at no extra bit cost. Only if you expand the color space, like BT.2020 or DzDx, do you need more bits.
HDR costs more bits too, but then we will change the coding to a more perceptually uniform scale than the old gamma function, closer to a logarithm actually. 10 bits is considered an absolute minimum here.
We have proposed u’v’ instead of CbCr initially because it is more perceptually uniform and it contains no dynamic range. Thus u’v’ need fewer bits for HDR. The infinite color gamut is merely a side effect, u’v’ is a color space to end all other color spaces. It seems wasteful because of all the unusable colors, but CbCr waste far more space on dynamic range (they encode hue, saturation, and scale with brightness). For u’v’ 9 bits each should forever be enough, let’s give them 10.
Note that a color space only defines a coordinate system for encoding. A color gamut is what is actually used, and it is typically smaller. For example, BT.2020 color space is mostly used for encoding a DCI P3 gamut. Do not assume that the gamut will suddenly grow just because it can.
Then we discovered more benefits of u’v’ for HDR, and this is what the IBC paper was about. HDR comes with an even less linear OECF-EOCF than BT.1886. Combined with CbCr and 4:2:0 down-sampling this causes much larger color crosstalk errors, and amplification of noise near the edges of the RGB color space (the triangle of 100% saturated colors). With u’v’ it is easier to make these problems go away, once you have found the optimum order of operations.
Y”u’v’ is just a cleaner way of encoding video, even in 4:2:0. Constant Luminance comes standard too, and the sharpness stays close to 4:4:4. You can imagine our excitement that it turns out to work so well.
With new cameras and coding standards it is easy enough to transmit a high dynamic range and a very wide color gamut. The displays are the fundamental limitation here. If you transmit more than the display can handle then some tone mapping and gamut mapping must be done. Our proposal for Y”u’v’ nicely separates those problems: the high dynamic range is in the Y” signal, the wide color gamut is in the u’v’ signals. The mappings can be defined by 1D and 2D lookup functions. It is in our interest that a single signal can feed multiple kinds of displays, with very modest receiver side processing for the display adaptation.
Yes, we have some (former) display specialists on the team. It helps !
jeroen.stessen@philips.com
September 25, 2014 at 10:24 am
Log in to Reply
Stessen kindly provided me with the paper as presented at the recent IBC conference in Amsterdam, plus an updated and expanded version to be presented by Poynton at the upcoming SMPTE 2014 Technical Conference and Exhibition in Hollywood, October 20 – 23. He also provided email explanations of some of the fine points of the system.
The need for encoding wide color gamut information is obvious, as I discussed last week. Many colors content creators want to display on a consumer’s TV simply cannot be encoded in the current system. The flip side of the coin is also there – many consumer TV and mobile displays are wide color gamut, but there is no wide color gamut material available to show on them. These include not only OLED displays, but displays based on quantum dots from Nanosys or QD Vision and even displays based on conventional LEDs. The electronics driving these displays stretch normal color gamut material to fill the wide color gamut of the display, but this is not the same as having native material. While much has been made of the lack of native UltraHD content to show on UltraHD displays, I’ve heard little about the lack of WCG content to show on WCG displays.
xvYCC or Extended-gamut YCC (also called x.v.Color) is a color space that was proposed by Sony and approved as a standard in January 2006 as IEC 61966-2-4. This is a backward-compatible extension to HDTV color encoding that supposedly gives a color gamut of 1.8x the color gamut of Rec. 709. This standard has received only modest support, in part because it is not supported by DVDs. One problem with xvYCC encoding is it is backward compatible with an 8-bit encoding standard and 8 bits aren’t really enough to encode these extra colors. Blu-Ray can support xvYCC and in 2013 Sony announced “Mastered in 4K” Blu-ray disks that not only have native UltraHD content but have xvYCC colors as well.
One of the problems with xvYCC as a backward compatible system, is it sticks with the 8-bit encoding of HDTV or 24 bits per pixel for 4:4:4 encoding. Unfortunately, a wide color gamut display can display many additional color/brightness combinations at the pixel than a normal color gamut system and 24 bits is not really enough to encode these extra combinations. This is one of several of the justifications for HDR, or 10 bits per color, 30 bits per pixel with 4:4:4 encoding.
One of the problems with u’v’ encoding and other systems such as DCI and ACES that can encode all possible colors, is the reverse of a problem with Rec. 709 encoding. While Rec. 709 cannot encode some colors that should be displayed, u’v’ and these other systems can encode many colors that are physically impossible and cannot be encoded. These non-physical colors take up space in code tables and are part of the reason why 30 bits are needed as opposed to 24.
Of course, these extra code groups could be used for something other than color. Closed captions and multi-language subtitles? Multilingual sound? Metadata? Second screen information? Picture in picture? If I know the CE industry, once a channel is established, they will find data to fill it. With advertising, if nothing else.
Figure 7 from Poynton, et. al. Rec. 709 and Rec. 2020 are both color spaces and color gamuts. Only colors within these triangles can be encoded by these systems. P# is the digital cinema color gamut but the encoding DCI color space includes all possible colors, as defined by the CIE horseshoe curve. U’v’ encoding can encode all colors within the dotted line, including some outside the CIE gamut of real colors.
Figure 7 from Poynton, et. al. Rec. 709 and Rec. 2020 are both color spaces and color gamuts. Only colors within these triangles can be encoded by these systems. P# is the digital cinema color gamut but the encoding DCI color space includes all possible colors, as defined by the CIE horseshoe curve. U’v’ encoding can encode all colors within the dotted line, including some outside the CIE gamut of real colors.
One must distinguish between color space and color gamut. Color space is the range of colors that can be encoded into a video signal and color gamut typically refers to the colors a display is capable of reproducing. In CRT days, and the HDTV Rec. 709 standard was created in CRT days, color space and color gamut were identical. All CRTs used essentially the same red, green and blue phosphors and the color space of Rec. 709, NTSC, PAL and SECAM encoding were designed to encode the colors that could be created by these CRT phosphors and no other colors. Content editors in post-production used Sony Trinitron CRTs with these same phosphors so they saw, in terms of color, pretty much what the consumer saw.
Times have changed, to put it mildly. The Trinitron has pretty much vanished from post-production facilities, replaced by LCD displays. While many CRT TVs are still in the field (I have one!) no new ones are being sold, all new display sales are flat panel displays such as LCD, OLED, plasma or projection displays based on mercury lamps, xenon lamps, LEDs or lasers. “Laser” projectors is pretty much a misnomer, most of them are phosphor-based and sometimes have an even smaller color gamut than CRTs. Displays with three, four, or more primary colors are available to the consumer. There is a huge variation in the color gamut for all of these displays, but they are all limited by the colors of the input signal, which is intended to be displayed on a CRT.
The TV, smartphone and tablet manufacturers , of course, process this input video to make it look as colorful as possible on their display. Consumers like bright, saturated colors, even if they are not contained in the original content. There are some very sophisticated algorithms out there to expand the color space encoded at the input to match the color gamut of the display, most of them are proprietary and none of them use a common defined standard. Since this gamut expansion is not standardized, the content creator has no idea what his content will look like on a “typical” consumer display.
A question arises at this point. Why has the consumer, who has shown a definite preference for wide color gamuts and colorful content, been deprived of native WCG content? Most consumers are relatively indifferent to UltraHD content but there has been a strong push to provide them with native UltraHD content.
One reason is UltraHD content is available since most Hollywood films are made with UltraHD resolution, even if they are often shown in the local multiplex at 2K resolution. Of course, the Hollywood content is also made with a WCG and HDR, but these are lost when it is transferred to video.
Another of the reasons is standards. Existing standards, e.g. Blu-ray and HDMI, support UltraHD native content but existing standards for consumer content do a poor job of supporting WCG or HDR content. Standards in the consumer electronics business evolve slowly. No one expects that at the upcoming SMPTE technical conference, after the u’v’ WCG HDR system from Poynton, Stessen and Nijland is presented, the audience will stand, cheer and start chanting “u’v’ Now! u’v’ Now!” The business doesn’t work that way.
The WCG and HDR content is there and the consumer has a desire to see the content. Hopefully some large consumer electronics manufacturer will decide WCG and HDR will give them a competitive advantage over their rivals and will develop a system to deliver this content to viewers. After they develop the system, the CE manufacturer can submit it to the IEC or SMPTE as a proposed standard, and the standard can (eventually) be approved and adopted by other companies. It’s likely to take years, but that’s the way the business works. Unless Apple is the CE manufacturer that introduces WCG and HDR, of course. iColor anyone? Visible only if you have an AppleTV? –Matthew Brennesholtz
3 Responses to Wide Color Gamuts and High Dynamic Range
So, Matt, let’s take this a step further. Consumers like wide gamuts and saturated colors, but do they really care if those colors are non-standard? Given the way consumers used to oversaturate the colors on their CRT TVs — which had knobs for the purpose rather than today’s menu structures, which many consumers never use — I suspect that most of them don’t. The push here will come from the content creators, including big-name film directors, who care very much that viewers are seeing the same colors the creators have painstakingly produced.
This is not to say the creators are wrong. A scene with muted colors and a blue tinge will create a very different emotional response in most viewers than the same scene with vibrant colors and a yellow tinge, even if many viewers don’t understand why. In this context, it’s interesting that when Kodak and THX talk about their content and TV certification programs, they do not talk about giving the consumer what he wants. They talk about making sure the creator’s intentions are accurately transferred to the screen.
Kenneth Werner
September 24, 2014 at 3:27 pm
Log in to Reply
I rather think Matt is right on this one. Joe Public will not understand colour in the stated terms. He will only buy the expanded colour when it is sold to him/her as “better”. The recent history or TV “improvements” shows this pretty clearly.
Even now, most people do not understand the impact of the various control functions they may have in their TV’s. How many people can even comprehend the video control of contrast and brightness? I have found very few even in ght display business that properly comprehend what these things do. But – they can mostly all be sold on “better”!
John Thomas
September 24, 2014 at 4:40 pm
Log in to Reply
Dear Matt, I see some minor errors in your story.
I don’t know that Poynton will present anything at a next SMPTE meeting, I only know that a certain group within SMPTE wanted to know more from us, so I have updated our paper to the last status. I don’t know yet that I am free to distribute that, Poynton needs to decide that when he gets back.
For xvYCC I refer to the papers that Katoh and I presented at the 2007 HPA Tech Retreat. xvYCC should work well enough with DVD, and at no extra bit cost. Only if you expand the color space, like BT.2020 or DzDx, do you need more bits.
HDR costs more bits too, but then we will change the coding to a more perceptually uniform scale than the old gamma function, closer to a logarithm actually. 10 bits is considered an absolute minimum here.
We have proposed u’v’ instead of CbCr initially because it is more perceptually uniform and it contains no dynamic range. Thus u’v’ need fewer bits for HDR. The infinite color gamut is merely a side effect, u’v’ is a color space to end all other color spaces. It seems wasteful because of all the unusable colors, but CbCr waste far more space on dynamic range (they encode hue, saturation, and scale with brightness). For u’v’ 9 bits each should forever be enough, let’s give them 10.
Note that a color space only defines a coordinate system for encoding. A color gamut is what is actually used, and it is typically smaller. For example, BT.2020 color space is mostly used for encoding a DCI P3 gamut. Do not assume that the gamut will suddenly grow just because it can.
Then we discovered more benefits of u’v’ for HDR, and this is what the IBC paper was about. HDR comes with an even less linear OECF-EOCF than BT.1886. Combined with CbCr and 4:2:0 down-sampling this causes much larger color crosstalk errors, and amplification of noise near the edges of the RGB color space (the triangle of 100% saturated colors). With u’v’ it is easier to make these problems go away, once you have found the optimum order of operations.
Y”u’v’ is just a cleaner way of encoding video, even in 4:2:0. Constant Luminance comes standard too, and the sharpness stays close to 4:4:4. You can imagine our excitement that it turns out to work so well.
With new cameras and coding standards it is easy enough to transmit a high dynamic range and a very wide color gamut. The displays are the fundamental limitation here. If you transmit more than the display can handle then some tone mapping and gamut mapping must be done. Our proposal for Y”u’v’ nicely separates those problems: the high dynamic range is in the Y” signal, the wide color gamut is in the u’v’ signals. The mappings can be defined by 1D and 2D lookup functions. It is in our interest that a single signal can feed multiple kinds of displays, with very modest receiver side processing for the display adaptation.
Yes, we have some (former) display specialists on the team. It helps !
jeroen.stessen@philips.com
September 25, 2014 at 10:24 am
Log in to Reply