[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] y-orientation for texImage2D from HTML elements



There is clearly developer confusion in this regard, since many think that texture coordinates start in the top-left. They don't. The GLES2 spec is clear about this, even. But people think they do because when they upload an image, it's upside down if you put 0,0 in the bottom-left. Then they try copyTexImage or readPixels, and everything appears upside-down. How is this behavior not confusing?

WebGL's UNPACK_FLIP_Y_WEBGL behavior appears consistent in theory, but falls down in practice. For an app that uses 0,0=bottom-left coords, the behavior of texImage(HTML element) is never consistent with texImage(TypedArray). Since internally, GL textures are 0,0=bottom-left, this makes this behavior confusing.

It's already come up in this thread how "copyTexImage is backwards", even though it isn't, so there's very clearly some developer confusion in this area. Don't misunderstand people getting something to work eventually with nothing being wrong. If people made websites for IE6, I'm pretty sure they'll eventually figure out how to make due with anything.

OpenGL internally uses 0,0=bottom-left, so why is it so hard to use WebGL this way. Why do texImage2D calls need to be interleaved with twiddling of UNPACK_FLIP_Y_WEBGL to get it to work.

While there is a very good reason to want to flip HTML element uploads, there is much less of a reason to want to flip TypedArray uploads. Regardless of whether your app is 0,0 at the top-left or bottom-left, upload the data in top-to-bottom or bottom-to-top respectively, set 0,0 to top-left or bottom-left respectively, and your data will be right-side up. (Both with UNPACK_FLIP_Y_WEBGL=false) However, for HTML elements, top-left apps need UNPACK_FLIP_Y_WEBGL=false, and bottom-left apps need UNPACK_FLIP_Y_WEBGL=true.
The only usecase for TypedArray uploads with UNPACK_FLIP_Y_WEBGL=true is if the data you have is backwards. (That is, UNPACK_FLIP_Y_WEBGL=true is only useful for uploading bottom-to-top data in a 0,0=top-left app, and vice-versa) Why your data is already wrong inside your own app is anyone's guess.

A form of UNPACK_FLIP_Y_WEBGL that only affected HTML element uploads is needed to provide a level of (texImage) consistency for 0,0=bottom-left apps that 0,0=top-left apps already enjoy. The choice to have UNPACK_FLIP_Y_WEBGL modify both texImage2D behaviors, while seemingly more consistent, actually makes things more convoluted than just having it affect HTML element uploads.

----- Original Message -----
From: "Kenneth Russell" <kbr@google.com>
To: "Gregg Tavares (社用)" <gman@google.com>
Cc: "Jeff Gilbert" <jgilbert@mozilla.com>, "public webgl" <public_webgl@khronos.org>
Sent: Thursday, December 6, 2012 6:06:04 PM
Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML elements

On Thu, Dec 6, 2012 at 5:44 PM, Gregg Tavares (社用) <gman@google.com> wrote:
>
>
>
> On Fri, Dec 7, 2012 at 10:27 AM, Jeff Gilbert <jgilbert@mozilla.com> wrote:
>>
>> I am not saying people should not have the option (or even the default) of
>> having texImage2D of an HTML element be y-flipped to match their upside-down
>> tex coords, but it should not make it harder for projects that use 0,0 as
>> the bottom-left, as GL does natively. It should at least be possible to get
>> consistent behavior such that an app can upload TypedArrays bottom-row first
>> while *also* having texImage on HTML elements get uploaded bottom-row-first.
>> This is not currently possible.
>>
>> It is also a little crazy that texImage2D(gl.canvas) does not reduce to
>> the same operation as copyTexImage2D.
>
>
> Why is this crazy? Canvas started as a 2D software implementation with
> top-row first. It's still top row first.
>
>>
>>
>> I understand that people are used to passing image data directly to
>> texImage2D as top-row-first, and just dealing with the flipped u/v
>> coords[1], but this choice should not be force on everyone as it is now.
>> UNPACK_FLIP_Y_WEBGL as it is today does not solve this, as to get consistent
>> bottom-row-first behavior, an app needs to use UNPACK_FLIP_Y_WEBGL=false for
>> TypedArray uploads, and UNPACK_FLIP_Y_WEBGL=true for HTML element uploads
>
>
> I don't see that as a problem. Most game devs flip their images in their
> asset pipeline since there's no UNPACK_FLIP_Y on real GL. In that case their
> ported to WebGL apps just work. They load their flipped at build time
> images, set no flags and get the same results they got on GL.
>
>
>>
>> .
>>
>> WebGL is *not* working the same way as OpenGL here, because OpenGL *only*
>> accepts raw bytes. If you upload data top-row-first, and use uv 0,0 at the
>> top-left, it's right-side-up. If you upload data bottom-row-first (like the
>> texImage2D spec says), and use uv 0,0 at the bottom-left, it's also
>> right-side-up.
>
>
> WebGL is providing 2 things. 1) GL. 2) image loaders. Both are behaving as
> most do. GL is doing bottom row first. The Image loaders are doing top-row
> first.
>
>>
>>
>> WebGL introduces loading from images directly. This means WebGL has to
>> make a choice over what order to upload the rows. Whereas texImage2D on
>> TypedArrays doesn't really matter, texImage2D on HTML elements can't
>> automatically handle both. We have half the solution in the form of
>> UNPACK_FLIP_Y_WEBGL, but it doesn't provide a solution to making WebGL
>> consistent for both choices of origin location.
>>
>> [1] This also tends to create the appearance that CopyTexImage and/or
>> ReadPixels are backwards, even though they're not.
>
>
> As has always been the case. I know of no OS who's windowing system is
> bottom first. Every OS provides ways to draw to a window and read from a
> window. AFAIK all of those are top-row first. GL is bottom-row first so this
> problem exists everywhere not just WebGL in browser. Browsers, like OSes are
> top-row first.

For historical reference, *all* image loading libraries for OpenGL
which have ever existed have an option to vertically flip the incoming
image, precisely because the first byte of the image in all modern
image formats is at the upper-left corner.

HTML's coordinate systems assume that (0, 0) is the upper left corner
of images; e.g. CanvasRenderingContext2D.drawImage. WebGL's
UNPACK_FLIP_Y parameter is both convenient and consistent as it
applies not only to the upload paths taking HTML elements but also
applies to those taking Typed Arrays. (It didn't always, in early
WebGL implementations.)

I don't believe there is any developer confusion in this area and do
not think any new tunable parameters should be added in this area.

-Ken


>>
>>
>> ----- Original Message -----
>> From: "Gregg Tavares (社用)" <gman@google.com>
>> To: "Jeff Gilbert" <jgilbert@mozilla.com>
>> Sent: Thursday, December 6, 2012 5:01:18 PM
>> Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
>> elements
>>
>>
>>
>>
>>
>>
>>
>>
>> On Fri, Dec 7, 2012 at 9:51 AM, Jeff Gilbert < jgilbert@mozilla.com >
>> wrote:
>>
>>
>> It most certainly is a bug that it is not specified.
>> My point is we never specify that data is uploaded to texImage2D from HTML
>> elements top-row-first. Just because we're 'used to' talking about image
>> data top-row-first doesn't mean we shouldn't process this data into GL's
>> coord system. The reasonable default, if we're doing the legwork of
>> uploading HTML element contents, is to upload them (at least by default) in
>> the orientation GL expects: bottom-row-first.
>>
>>
>> No that is not a reasonable default. A reasonable default is to upload the
>> same as GL programmers expect. GL programmers expect that image libraries
>> load top row first and GL expects bottom row first and adjust their code
>> accordingly. Many people are porting GL code. WebGL should not be different
>> here.
>>
>>
>> If you want to specify everything top-row-first, we should use our UNPACK
>> boolean for that.
>>
>> As it stands, it works consistently only if you pretend that for some
>> reason, OpenGL textures coords are 0,0=top-left. That's the way we upload
>> HTML elements, and if you pass texImage TypedArray data in top-row-first
>> order, it'll upload upside-down, but since you y-flip the texture coords
>> (that is, use 0,0=top-left), now it's correctly oriented.
>>
>> One of the key issues is there's no way to configure UNPACK such that
>> texImage2D's upload orientation always matches GL's coord system.
>>
>> CopyTexImage isn't backwards, it matches GL's coordinate system.
>> CopyTexImage, ReadPixels, and TexImage all specify that they are
>> bottom-row-first. Scissor and Viewport, being in window space, are also
>> 0,0=bottom-left. The idea that textures are 0,0=top-left goes against
>> OpenGL's 0,0=bottom-left coord system. (actually -1,-1=bottom-left, for clip
>> space) CopyTexImage matches the behavior of calling texImage2D on data
>> retrieved from readPixels. It would *ideally* match calling texImage2D on
>> our canvas, but it doesn't. Instead, we readPixels data down, y-flip it,
>> then reupload it. Ideally we would just call CopyTexImage instead of
>> ReadPixels+TexImage, but it doesn't looks like we even can. Instead, we
>> would have to CopyTexImage, create a new texture, wrap it in a framebuffer,
>> and draw a full-screen quad with the original copyTexImage texture, but with
>> y coords flipped.
>>
>> TexImage2D with both TypedArrays and canvas/image/video elements is only
>> consistent for u/v 0,0=top-left. But then it's no longer consistent with
>> CopyTexImage or ReadPixels. Now we're in y-flip hell.
>>
>> Proposal:
>> Since basically everyone uses texImage2D(<image>) for content delivery, I
>> don't think we can afford to break compatibility there. What I propose is
>> adding a new UNPACK_HTML_FLIP_Y_WEBGL, which defaults to `true`, which
>> specifies that WebGL will upload HTML elements top-row-first into
>> texImage2D, as we do today. However, with `false`, we would upload elements
>> bottom-row-first, such that texImage2D(gl.canvas) has the same result as
>> copyTexImage2D.
>>
>>
>>
>> Disagree. No change needed. WebGL works the same as OpenGL.
>> UNPACK_FLIP_Y_WEBGL is there as a convenience only because unlike GL we
>> don't expect web developers to supply their own image loaders (though some
>> do) so we've provided the option to tell to the browser to flip the image
>> before providing it to the driver.
>>
>>
>>
>>
>>
>> -Jeff
>>
>>
>>
>> ----- Original Message -----
>> From: "Gregg Tavares (社用)" < gman@google.com >
>> To: "Jeff Gilbert" < jgilbert@mozilla.com >
>> Cc: "public webgl" < public_webgl@khronos.org >
>> Sent: Thursday, December 6, 2012 12:35:14 AM
>> Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
>> elements
>>
>>
>>
>> This isn't a bug. it's by design.
>>
>>
>> By default WebGL does the same as GL. It doesn't flip anything. The "data"
>> pointer passed to texImage2D represents the bottom left corner of the
>> texture. Whether that's an ArrayBuffer, an Image, a Canvas, a Video. For
>> Image, Canvas, and Video that means by default they'be upside down (which is
>> the same as GL).
>>
>>
>> CopyTexImage2D in GL has always been "effectively" backward relative to
>> TexImage2D when talking about images because most (all?) image libraries
>> load images with the first pixel in the top, left and GL expects the first
>> pixel is the bottom, left. Since CopyTexImage is copying internally it
>> appears to have different behavior because it's matching src data 0x0 =
>> bottom, left, dst data 0x0 = bottom left where as TexImage2D, when passing
>> standard image data, is src data 0x0 = top,left, dest = bottom, left
>>
>>
>> If you want WebGL to flip the data for you call
>> gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);
>>
>>
>> There is no PACK_FLIP_Y so readPixels will always return the bottom,left
>> as the first data in the array buffer.
>>
>>
>> As for matching implementations it's all tested in the conformance tests
>> AFAIK.
>>
>>
>>
>> On Thu, Dec 6, 2012 at 4:59 PM, Jeff Gilbert < jgilbert@mozilla.com >
>> wrote:
>>
>>
>>
>> It seems backwards (as implemented) and underdefined in the spec. It looks
>> like we all upload HTML element texImage data with 0,0=top-left, making it
>> effectively upside-down.
>>
>> The following two lines yield different results:
>> [1] gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0,
>> gl.drawingBufferWidth, gl.drawingBufferHeight, 0);
>> [2] gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE,
>> gl.canvas);
>>
>> [1] matches this: [3]
>> var data = new Uint8Array(gl.drawingBufferWidth * gl.drawingBufferHeight *
>> 4);
>> gl.readPixels(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight,
>> gl.RGBA, gl.UNSIGNED_BYTE, data);
>> gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.drawingBufferWidth,
>> gl.drawingBufferHeight, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);
>>
>> UNPACK_FLIP_Y_WEBGL=true makes [2] the same as [1], but now [3] is
>> upside-down.
>>
>> This is Mozilla bug 818810:
>> https://bugzilla.mozilla.org/show_bug.cgi?id=818810
>>
>> Testcase showing inconsistent behavior:
>> https://bugzilla.mozilla.org/attachment.cgi?id=689110
>>
>> As I mention in the bug, it doesn't look like y-orientation of texImage2D
>> of HTML elements is well-specified in the spec. So far everyone appears to
>> have implemented it the same, though.
>>
>> My suggestion was adding an UNPACK_ pixelStore option that allows us to
>> get OpenGL-style 0,0=bottom-left texImage2D uploads of HTML elements.
>>
>> -----------------------------------------------------------
>> You are currently subscribed to public_webgl@khronos.org .
>> To unsubscribe, send an email to majordomo@khronos.org with
>> the following command in the body of your email:
>> unsubscribe public_webgl
>> -----------------------------------------------------------
>>
>>
>>
>

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------