[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] y-orientation for texImage2D from HTML elements

The orientation of the windowing system doesn't matter, because that's beyond the scope of GL. The orientation of decoded image data is not defined anywhere, at least for the purposes of texImage2D. Reading the spec of texImage2D in GLES2, since it says that data is uploaded bottom-row-first, it would be reasonable to assume that if the browser is offering to upload image data directly, it will take care of *all* conversions that need to happen in order to get it into a format which GL can accept. Since the TexImage2D spec is very clear that it's bottom-row-first, part of the automatic conversion to a GL-friendly format would be flipping the rows, unless otherwise countermanded by an UNPACK setting.

Data as decoded from images isn't always necessarily top-to-bottom anyways, just like it's not always in RGB(A) order, or 8 bit-per-channel. As part of the texImage2D upload, though, we convert it. GL specifies textures bottom-to-top, so why are we uploading textures top-to-bottom? Just because it works for some people doesn't mean it's the right way to do it. The way it functions currently flies in the face of how GL operates, but it 'works' because people like using the hack of uploading data upside-down and flipping their v-coords. When you hit a usecase that actually desires bottom-to-top textures, it falls down.

Regardless of which way may-or-may-not be the 'proper' way to do it, WebGL should be as agnostic as it can in this regard. WebGL shouldn't fall down in either case. It should be configurable to work with both. As it stands today, it's not possible to configure WebGL to always handle texture uploads as bottom-to-top.

Also this stuff all needs to be specified, whereas right now, it's not.

----- Original Message -----
From: "Gregg Tavares (社用)" <gman@google.com>
To: "Jeff Gilbert" <jgilbert@mozilla.com>
Cc: "public webgl" <public_webgl@khronos.org>
Sent: Thursday, December 6, 2012 5:44:11 PM
Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML elements

On Fri, Dec 7, 2012 at 10:27 AM, Jeff Gilbert < jgilbert@mozilla.com > wrote: 

I am not saying people should not have the option (or even the default) of having texImage2D of an HTML element be y-flipped to match their upside-down tex coords, but it should not make it harder for projects that use 0,0 as the bottom-left, as GL does natively. It should at least be possible to get consistent behavior such that an app can upload TypedArrays bottom-row first while *also* having texImage on HTML elements get uploaded bottom-row-first. This is not currently possible. 

It is also a little crazy that texImage2D(gl.canvas) does not reduce to the same operation as copyTexImage2D. 

Why is this crazy? Canvas started as a 2D software implementation with top-row first. It's still top row first. 

I understand that people are used to passing image data directly to texImage2D as top-row-first, and just dealing with the flipped u/v coords[1], but this choice should not be force on everyone as it is now. UNPACK_FLIP_Y_WEBGL as it is today does not solve this, as to get consistent bottom-row-first behavior, an app needs to use UNPACK_FLIP_Y_WEBGL=false for TypedArray uploads, and UNPACK_FLIP_Y_WEBGL=true for HTML element uploads 

I don't see that as a problem. Most game devs flip their images in their asset pipeline since there's no UNPACK_FLIP_Y on real GL. In that case their ported to WebGL apps just work. They load their flipped at build time images, set no flags and get the same results they got on GL. 


WebGL is *not* working the same way as OpenGL here, because OpenGL *only* accepts raw bytes. If you upload data top-row-first, and use uv 0,0 at the top-left, it's right-side-up. If you upload data bottom-row-first (like the texImage2D spec says), and use uv 0,0 at the bottom-left, it's also right-side-up. 

WebGL is providing 2 things. 1) GL. 2) image loaders. Both are behaving as most do. GL is doing bottom row first. The Image loaders are doing top-row first. 

WebGL introduces loading from images directly. This means WebGL has to make a choice over what order to upload the rows. Whereas texImage2D on TypedArrays doesn't really matter, texImage2D on HTML elements can't automatically handle both. We have half the solution in the form of UNPACK_FLIP_Y_WEBGL, but it doesn't provide a solution to making WebGL consistent for both choices of origin location. 

[1] This also tends to create the appearance that CopyTexImage and/or ReadPixels are backwards, even though they're not. 

As has always been the case. I know of no OS who's windowing system is bottom first. Every OS provides ways to draw to a window and read from a window. AFAIK all of those are top-row first. GL is bottom-row first so this problem exists everywhere not just WebGL in browser. Browsers, like OSes are top-row first. 

----- Original Message ----- 
From: "Gregg Tavares (社用)" < gman@google.com > 
To: "Jeff Gilbert" < jgilbert@mozilla.com > 

Sent: Thursday, December 6, 2012 5:01:18 PM 
Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML elements 

On Fri, Dec 7, 2012 at 9:51 AM, Jeff Gilbert < jgilbert@mozilla.com > wrote: 

It most certainly is a bug that it is not specified. 
My point is we never specify that data is uploaded to texImage2D from HTML elements top-row-first. Just because we're 'used to' talking about image data top-row-first doesn't mean we shouldn't process this data into GL's coord system. The reasonable default, if we're doing the legwork of uploading HTML element contents, is to upload them (at least by default) in the orientation GL expects: bottom-row-first. 

No that is not a reasonable default. A reasonable default is to upload the same as GL programmers expect. GL programmers expect that image libraries load top row first and GL expects bottom row first and adjust their code accordingly. Many people are porting GL code. WebGL should not be different here. 

If you want to specify everything top-row-first, we should use our UNPACK boolean for that. 

As it stands, it works consistently only if you pretend that for some reason, OpenGL textures coords are 0,0=top-left. That's the way we upload HTML elements, and if you pass texImage TypedArray data in top-row-first order, it'll upload upside-down, but since you y-flip the texture coords (that is, use 0,0=top-left), now it's correctly oriented. 

One of the key issues is there's no way to configure UNPACK such that texImage2D's upload orientation always matches GL's coord system. 

CopyTexImage isn't backwards, it matches GL's coordinate system. CopyTexImage, ReadPixels, and TexImage all specify that they are bottom-row-first. Scissor and Viewport, being in window space, are also 0,0=bottom-left. The idea that textures are 0,0=top-left goes against OpenGL's 0,0=bottom-left coord system. (actually -1,-1=bottom-left, for clip space) CopyTexImage matches the behavior of calling texImage2D on data retrieved from readPixels. It would *ideally* match calling texImage2D on our canvas, but it doesn't. Instead, we readPixels data down, y-flip it, then reupload it. Ideally we would just call CopyTexImage instead of ReadPixels+TexImage, but it doesn't looks like we even can. Instead, we would have to CopyTexImage, create a new texture, wrap it in a framebuffer, and draw a full-screen quad with the original copyTexImage texture, but with y coords flipped. 

TexImage2D with both TypedArrays and canvas/image/video elements is only consistent for u/v 0,0=top-left. But then it's no longer consistent with CopyTexImage or ReadPixels. Now we're in y-flip hell. 

Since basically everyone uses texImage2D(<image>) for content delivery, I don't think we can afford to break compatibility there. What I propose is adding a new UNPACK_HTML_FLIP_Y_WEBGL, which defaults to `true`, which specifies that WebGL will upload HTML elements top-row-first into texImage2D, as we do today. However, with `false`, we would upload elements bottom-row-first, such that texImage2D(gl.canvas) has the same result as copyTexImage2D. 

Disagree. No change needed. WebGL works the same as OpenGL. UNPACK_FLIP_Y_WEBGL is there as a convenience only because unlike GL we don't expect web developers to supply their own image loaders (though some do) so we've provided the option to tell to the browser to flip the image before providing it to the driver. 


----- Original Message ----- 
From: "Gregg Tavares (社用)" < gman@google.com > 
To: "Jeff Gilbert" < jgilbert@mozilla.com > 
Cc: "public webgl" < public_webgl@khronos.org > 
Sent: Thursday, December 6, 2012 12:35:14 AM 
Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML elements 

This isn't a bug. it's by design. 

By default WebGL does the same as GL. It doesn't flip anything. The "data" pointer passed to texImage2D represents the bottom left corner of the texture. Whether that's an ArrayBuffer, an Image, a Canvas, a Video. For Image, Canvas, and Video that means by default they'be upside down (which is the same as GL). 

CopyTexImage2D in GL has always been "effectively" backward relative to TexImage2D when talking about images because most (all?) image libraries load images with the first pixel in the top, left and GL expects the first pixel is the bottom, left. Since CopyTexImage is copying internally it appears to have different behavior because it's matching src data 0x0 = bottom, left, dst data 0x0 = bottom left where as TexImage2D, when passing standard image data, is src data 0x0 = top,left, dest = bottom, left 

If you want WebGL to flip the data for you call gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true); 

There is no PACK_FLIP_Y so readPixels will always return the bottom,left as the first data in the array buffer. 

As for matching implementations it's all tested in the conformance tests AFAIK. 

On Thu, Dec 6, 2012 at 4:59 PM, Jeff Gilbert < jgilbert@mozilla.com > wrote: 

It seems backwards (as implemented) and underdefined in the spec. It looks like we all upload HTML element texImage data with 0,0=top-left, making it effectively upside-down. 

The following two lines yield different results: 
[1] gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight, 0); 
[2] gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, gl.canvas); 

[1] matches this: [3] 
var data = new Uint8Array(gl.drawingBufferWidth * gl.drawingBufferHeight * 4); 
gl.readPixels(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight, gl.RGBA, gl.UNSIGNED_BYTE, data); 
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.drawingBufferWidth, gl.drawingBufferHeight, 0, gl.RGBA, gl.UNSIGNED_BYTE, data); 

UNPACK_FLIP_Y_WEBGL=true makes [2] the same as [1], but now [3] is upside-down. 

This is Mozilla bug 818810: 

Testcase showing inconsistent behavior: 

As I mention in the bug, it doesn't look like y-orientation of texImage2D of HTML elements is well-specified in the spec. So far everyone appears to have implemented it the same, though. 

My suggestion was adding an UNPACK_ pixelStore option that allows us to get OpenGL-style 0,0=bottom-left texImage2D uploads of HTML elements. 

You are currently subscribed to public_webgl@khronos.org . 
To unsubscribe, send an email to majordomo@khronos.org with 
the following command in the body of your email: 
unsubscribe public_webgl 

You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl