[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] toDataURL behavior when the WebGL drawing buffer contains invalid premultiplied alpha values



On Wed, Jul 3, 2013 at 2:53 PM, Olli Etuaho <oetuaho@nvidia.com> wrote:
> Okay, looks like there is more to consider than what I originally thought. I didn't think of the trouble for browsers that use third-party compositors - if compositing for superluminal values would be specified, they might have to do one extra blit for the whole framebuffer in GL before sending it for composition, or might not be able to conform at all. The question of whether to use re-normalization or clipping is also something that should be carefully considered.
>
> As for the 2D canvas case, I do think that storing 2D canvas contents internally as premultiplied is just an optimization, and it should not significantly affect how the canvas behaves, so normal draw operations should not result in superluminal values. Of course drawImage from WebGL is currently a corner case, that can probably result in superluminal values in some implementations.
>
> Anyway, I agree that the effort required to tighten up the spec is a bit much compared to the priority. For now we can just fix the broken on-screen display in the conformance tests by just setting premultipliedAlpha: false. This needs to be done to about 25 glsl/functions tests, so simply setting it for all of them would be a reasonable thing to do.

That sounds good. Would you mind creating a pull request on Github
with this change? Since it isn't a correctness fix to the tests I
don't think it's worth the risk of putting in the 1.0.1 suite, though
I wouldn't personally object to having it in the 1.0.2 suite.

-Ken


> -Olli
> ________________________________________
> From: Kenneth Russell [kbr@google.com]
> Sent: Wednesday, July 03, 2013 2:27 PM
> To: Olli Etuaho
> Cc: Jeff Gilbert; public webgl
> Subject: Re: [Public WebGL] toDataURL behavior when the WebGL drawing buffer contains invalid premultiplied alpha values
>
> It's heavily OS-dependent how such out-of-range color values are
> handled. As an example, it's possible to use OpenGL to feed colors
> with A < (R, G, B) to Mac OS' compositor, which show up on-screen very
> differently than on other operating systems.
>
> I'm wary of overspecifying the behavior in this area because toDataURL
> is not the only place in the WebGL pipeline where specification of
> clamping would be needed. Uploading HTML canvases to WebGL textures
> when the UNPACK_PREMULTIPLY_ALPHA_WEBGL pixel store parameter is 0
> also requires unmultiplication (in some browsers), and at least
> theoretically, can have out-of-range source colors in the HTML canvas.
> (I'm not sure whether all of CanvasRenderingContext2D's operations
> guarantee that all operations result in "valid" colors where R, G, and
> B <= A.) It's browser-dependent how CanvasRenderingContext2D stores
> its internal representation and it would be impossible to test any
> specified behavior.
>
> How these colors should be clamped is also ambiguous. Simply clamping
> the R, G, and B components to A will cause "clipping" and can cause
> the shade of the color to change dramatically. It might be better to
> test if R, G, or B is greater than A, and if so, re-normalize all
> color channels to make the greatest one equal to A.
>
> The conformance tests don't examine toDataURL's output; the pixel
> comparisons are done with glReadPixels' output. Therefore this
> behavior only affects the on-screen display. My personal opinion is
> that it isn't worth the effort to tighten up this area of the spec
> because there are higher-priority issues to deal with right now.
>
> -Ken
>
>
>
> On Wed, Jul 3, 2013 at 12:08 PM, Olli Etuaho <oetuaho@nvidia.com> wrote:
>>
>> I don't see a need to use the same clamping code path for webgl.texImage2D. I'd treat it as a different case from toDataURL, since the need to unpremultiply is specific to PNG encoding. canvas.drawImage is a slightly trickier case, but it could also benefit from the clamping, and if canvas is implemented on top of GL it shouldn't cause significant slowdown either, since shader math is hardly the bottleneck.
>>
>> I'd like to hear more opinions on this. I think it would be at least good if the behavior with these superluminal values would be specified somehow, so the results one gets in each of these cases wouldn't be completely implementation-dependent.
>>
>> -Olli
>> ________________________________________
>> From: Jeff Gilbert [jgilbert@mozilla.com]
>> Sent: Tuesday, July 02, 2013 9:43 PM
>> To: Olli Etuaho
>> Cc: public webgl
>> Subject: Re: [Public WebGL] toDataURL behavior when the WebGL drawing buffer contains invalid premultiplied alpha values
>>
>> The only issue I see with this is that it would then diverge from how we (want to be able to) do canvas.drawImage and webgl.texImage2D. If we wanted to preserve the speed of these operations, we can't clamp their values as in the way this proposal would clamp toDataURL data.
>>
>> Such 'superluminal' values would be troublesome anyways, since the browser's compositing engine may try to do compositing operations with this invalid data. The best thing to do is to simple limit yourself to valid values in the first place. (Or just use `premultiplyAlpha: false`)
>>
>> -Jeff
>>
>> ----- Original Message -----
>> From: "Olli Etuaho" <oetuaho@nvidia.com>
>> To: "public webgl" <public_webgl@khronos.org>
>> Sent: Tuesday, July 2, 2013 7:28:48 AM
>> Subject: [Public WebGL] toDataURL behavior when the WebGL drawing buffer contains invalid premultiplied alpha values
>>
>>
>> OpenGL does not enforce correctness of premultiplied alpha values in any way, so the red, green, and blue values can end up greater than the alpha value of the same pixel in a WebGL drawing buffer of a context specified with premultiplyAlpha: true. This causes odd behavior in current Firefox and Chrome, which seem to pass this invalid color data as-is to unpremultiplication if toDataURL is called. An example of this can be seen in https://www.khronos.org/registry/webgl/sdk/tests/conformance/glsl/functions/glsl-function-smoothstep-float.html - the fourth test actually renders a smooth gradient from the shader perspective, and this is what would show up if the WebGL canvas was displayed directly, but as the canvas contents get copied to a PNG image on the page the result ends up wildly different.
>>
>> I propose that this would be addressed in the WebGL spec so that when toDataURL is called on a WebGL canvas that has premultiplyAlpha: true, the red, green, and blue values of each pixel must be clamped to alpha before unpremultiplying and encoding the image. A test could be added to the conformance suite as well. This would clear up any ambiguity about this matter.
>>
>> -Olli
>> -----------------------------------------------------------
>> You are currently subscribed to public_webgl@khronos.org.
>> To unsubscribe, send an email to majordomo@khronos.org with
>> the following command in the body of your email:
>> unsubscribe public_webgl
>> -----------------------------------------------------------
>>
>>
>> -----------------------------------------------------------
>> You are currently subscribed to public_webgl@khronos.org.
>> To unsubscribe, send an email to majordomo@khronos.org with
>> the following command in the body of your email:
>> unsubscribe public_webgl
>> -----------------------------------------------------------
>>

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------