[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] y-orientation for texImage2D from HTML elements



copyTexImage2D, texImage2D, readPixels, scissors and viewport are defined in the ES 2.0 specification to be bottom up. When rendering to an FBO then left/bottom is UV 0,0. Internally the ES 2.0 specification is consistent. The correct choice would have been to upload bottom row first for all calls from <img>, <canvas>, <video> etc.

As everybody is using <img> for upload it's unfortunately not a choice for WebGL 1.0 to fix this since it would break everbodies code. I would be in favor of fixing it for WebGL 2.0.


On Fri, Dec 7, 2012 at 1:51 AM, Jeff Gilbert <jgilbert@mozilla.com> wrote:

It most certainly is a bug that it is not specified.
My point is we never specify that data is uploaded to texImage2D from HTML elements top-row-first. Just because we're 'used to' talking about image data top-row-first doesn't mean we shouldn't process this data into GL's coord system. The reasonable default, if we're doing the legwork of uploading HTML element contents, is to upload them (at least by default) in the orientation GL expects: bottom-row-first. If you want to specify everything top-row-first, we should use our UNPACK boolean for that.

As it stands, it works consistently only if you pretend that for some reason, OpenGL textures coords are 0,0=top-left. That's the way we upload HTML elements, and if you pass texImage TypedArray data in top-row-first order, it'll upload upside-down, but since you y-flip the texture coords (that is, use 0,0=top-left), now it's correctly oriented.

One of the key issues is there's no way to configure UNPACK such that texImage2D's upload orientation always matches GL's coord system.

CopyTexImage isn't backwards, it matches GL's coordinate system. CopyTexImage, ReadPixels, and TexImage all specify that they are bottom-row-first. Scissor and Viewport, being in window space, are also 0,0=bottom-left. The idea that textures are 0,0=top-left goes against OpenGL's 0,0=bottom-left coord system. (actually -1,-1=bottom-left, for clip space) CopyTexImage matches the behavior of calling texImage2D on data retrieved from readPixels. It would *ideally* match calling texImage2D on our canvas, but it doesn't. Instead, we readPixels data down, y-flip it, then reupload it. Ideally we would just call CopyTexImage instead of ReadPixels+TexImage, but it doesn't looks like we even can. Instead, we would have to CopyTexImage, create a new texture, wrap it in a framebuffer, and draw a full-screen quad with the original copyTexImage texture, but with y coords flipped.

TexImage2D with both TypedArrays and canvas/image/video elements is only consistent for u/v 0,0=top-left. But then it's no longer consistent with CopyTexImage or ReadPixels. Now we're in y-flip hell.

Proposal:
Since basically everyone uses texImage2D(<image>) for content delivery, I don't think we can afford to break compatibility there. What I propose is adding a new UNPACK_HTML_FLIP_Y_WEBGL, which defaults to `true`, which specifies that WebGL will upload HTML elements top-row-first into texImage2D, as we do today. However, with `false`, we would upload elements bottom-row-first, such that texImage2D(gl.canvas) has the same result as copyTexImage2D.

-Jeff

----- Original Message -----
From: "Gregg Tavares (社用)" <gman@google.com>
To: "Jeff Gilbert" <jgilbert@mozilla.com>
Cc: "public webgl" <public_webgl@khronos.org>
Sent: Thursday, December 6, 2012 12:35:14 AM
Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML elements



This isn't a bug. it's by design.


By default WebGL does the same as GL. It doesn't flip anything. The "data" pointer passed to texImage2D represents the bottom left corner of the texture. Whether that's an ArrayBuffer, an Image, a Canvas, a Video. For Image, Canvas, and Video that means by default they'be upside down (which is the same as GL).


CopyTexImage2D in GL has always been "effectively" backward relative to TexImage2D when talking about images because most (all?) image libraries load images with the first pixel in the top, left and GL expects the first pixel is the bottom, left. Since CopyTexImage is copying internally it appears to have different behavior because it's matching src data 0x0 = bottom, left, dst data 0x0 = bottom left where as TexImage2D, when passing standard image data, is src data 0x0 = top,left, dest = bottom, left


If you want WebGL to flip the data for you call gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);


There is no PACK_FLIP_Y so readPixels will always return the bottom,left as the first data in the array buffer.


As for matching implementations it's all tested in the conformance tests AFAIK.



On Thu, Dec 6, 2012 at 4:59 PM, Jeff Gilbert < jgilbert@mozilla.com > wrote:



It seems backwards (as implemented) and underdefined in the spec. It looks like we all upload HTML element texImage data with 0,0=top-left, making it effectively upside-down.

The following two lines yield different results:
[1] gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight, 0);
[2] gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, gl.canvas);

[1] matches this: [3]
var data = "" Uint8Array(gl.drawingBufferWidth * gl.drawingBufferHeight * 4);
gl.readPixels(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight, gl.RGBA, gl.UNSIGNED_BYTE, data);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.drawingBufferWidth, gl.drawingBufferHeight, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);

UNPACK_FLIP_Y_WEBGL=true makes [2] the same as [1], but now [3] is upside-down.

This is Mozilla bug 818810:
https://bugzilla.mozilla.org/show_bug.cgi?id=818810

Testcase showing inconsistent behavior:
https://bugzilla.mozilla.org/attachment.cgi?id=689110

As I mention in the bug, it doesn't look like y-orientation of texImage2D of HTML elements is well-specified in the spec. So far everyone appears to have implemented it the same, though.

My suggestion was adding an UNPACK_ pixelStore option that allows us to get OpenGL-style 0,0=bottom-left texImage2D uploads of HTML elements.

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org .
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------



-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------