[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] y-orientation for texImage2D from HTML elements



On Thu, Dec 6, 2012 at 5:44 PM, Gregg Tavares (社用) <gman@google.com> wrote:
>
>
>
> On Fri, Dec 7, 2012 at 10:27 AM, Jeff Gilbert <jgilbert@mozilla.com> wrote:
>>
>> I am not saying people should not have the option (or even the default) of
>> having texImage2D of an HTML element be y-flipped to match their upside-down
>> tex coords, but it should not make it harder for projects that use 0,0 as
>> the bottom-left, as GL does natively. It should at least be possible to get
>> consistent behavior such that an app can upload TypedArrays bottom-row first
>> while *also* having texImage on HTML elements get uploaded bottom-row-first.
>> This is not currently possible.
>>
>> It is also a little crazy that texImage2D(gl.canvas) does not reduce to
>> the same operation as copyTexImage2D.
>
>
> Why is this crazy? Canvas started as a 2D software implementation with
> top-row first. It's still top row first.
>
>>
>>
>> I understand that people are used to passing image data directly to
>> texImage2D as top-row-first, and just dealing with the flipped u/v
>> coords[1], but this choice should not be force on everyone as it is now.
>> UNPACK_FLIP_Y_WEBGL as it is today does not solve this, as to get consistent
>> bottom-row-first behavior, an app needs to use UNPACK_FLIP_Y_WEBGL=false for
>> TypedArray uploads, and UNPACK_FLIP_Y_WEBGL=true for HTML element uploads
>
>
> I don't see that as a problem. Most game devs flip their images in their
> asset pipeline since there's no UNPACK_FLIP_Y on real GL. In that case their
> ported to WebGL apps just work. They load their flipped at build time
> images, set no flags and get the same results they got on GL.
>
>
>>
>> .
>>
>> WebGL is *not* working the same way as OpenGL here, because OpenGL *only*
>> accepts raw bytes. If you upload data top-row-first, and use uv 0,0 at the
>> top-left, it's right-side-up. If you upload data bottom-row-first (like the
>> texImage2D spec says), and use uv 0,0 at the bottom-left, it's also
>> right-side-up.
>
>
> WebGL is providing 2 things. 1) GL. 2) image loaders. Both are behaving as
> most do. GL is doing bottom row first. The Image loaders are doing top-row
> first.
>
>>
>>
>> WebGL introduces loading from images directly. This means WebGL has to
>> make a choice over what order to upload the rows. Whereas texImage2D on
>> TypedArrays doesn't really matter, texImage2D on HTML elements can't
>> automatically handle both. We have half the solution in the form of
>> UNPACK_FLIP_Y_WEBGL, but it doesn't provide a solution to making WebGL
>> consistent for both choices of origin location.
>>
>> [1] This also tends to create the appearance that CopyTexImage and/or
>> ReadPixels are backwards, even though they're not.
>
>
> As has always been the case. I know of no OS who's windowing system is
> bottom first. Every OS provides ways to draw to a window and read from a
> window. AFAIK all of those are top-row first. GL is bottom-row first so this
> problem exists everywhere not just WebGL in browser. Browsers, like OSes are
> top-row first.

For historical reference, *all* image loading libraries for OpenGL
which have ever existed have an option to vertically flip the incoming
image, precisely because the first byte of the image in all modern
image formats is at the upper-left corner.

HTML's coordinate systems assume that (0, 0) is the upper left corner
of images; e.g. CanvasRenderingContext2D.drawImage. WebGL's
UNPACK_FLIP_Y parameter is both convenient and consistent as it
applies not only to the upload paths taking HTML elements but also
applies to those taking Typed Arrays. (It didn't always, in early
WebGL implementations.)

I don't believe there is any developer confusion in this area and do
not think any new tunable parameters should be added in this area.

-Ken


>>
>>
>> ----- Original Message -----
>> From: "Gregg Tavares (社用)" <gman@google.com>
>> To: "Jeff Gilbert" <jgilbert@mozilla.com>
>> Sent: Thursday, December 6, 2012 5:01:18 PM
>> Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
>> elements
>>
>>
>>
>>
>>
>>
>>
>>
>> On Fri, Dec 7, 2012 at 9:51 AM, Jeff Gilbert < jgilbert@mozilla.com >
>> wrote:
>>
>>
>> It most certainly is a bug that it is not specified.
>> My point is we never specify that data is uploaded to texImage2D from HTML
>> elements top-row-first. Just because we're 'used to' talking about image
>> data top-row-first doesn't mean we shouldn't process this data into GL's
>> coord system. The reasonable default, if we're doing the legwork of
>> uploading HTML element contents, is to upload them (at least by default) in
>> the orientation GL expects: bottom-row-first.
>>
>>
>> No that is not a reasonable default. A reasonable default is to upload the
>> same as GL programmers expect. GL programmers expect that image libraries
>> load top row first and GL expects bottom row first and adjust their code
>> accordingly. Many people are porting GL code. WebGL should not be different
>> here.
>>
>>
>> If you want to specify everything top-row-first, we should use our UNPACK
>> boolean for that.
>>
>> As it stands, it works consistently only if you pretend that for some
>> reason, OpenGL textures coords are 0,0=top-left. That's the way we upload
>> HTML elements, and if you pass texImage TypedArray data in top-row-first
>> order, it'll upload upside-down, but since you y-flip the texture coords
>> (that is, use 0,0=top-left), now it's correctly oriented.
>>
>> One of the key issues is there's no way to configure UNPACK such that
>> texImage2D's upload orientation always matches GL's coord system.
>>
>> CopyTexImage isn't backwards, it matches GL's coordinate system.
>> CopyTexImage, ReadPixels, and TexImage all specify that they are
>> bottom-row-first. Scissor and Viewport, being in window space, are also
>> 0,0=bottom-left. The idea that textures are 0,0=top-left goes against
>> OpenGL's 0,0=bottom-left coord system. (actually -1,-1=bottom-left, for clip
>> space) CopyTexImage matches the behavior of calling texImage2D on data
>> retrieved from readPixels. It would *ideally* match calling texImage2D on
>> our canvas, but it doesn't. Instead, we readPixels data down, y-flip it,
>> then reupload it. Ideally we would just call CopyTexImage instead of
>> ReadPixels+TexImage, but it doesn't looks like we even can. Instead, we
>> would have to CopyTexImage, create a new texture, wrap it in a framebuffer,
>> and draw a full-screen quad with the original copyTexImage texture, but with
>> y coords flipped.
>>
>> TexImage2D with both TypedArrays and canvas/image/video elements is only
>> consistent for u/v 0,0=top-left. But then it's no longer consistent with
>> CopyTexImage or ReadPixels. Now we're in y-flip hell.
>>
>> Proposal:
>> Since basically everyone uses texImage2D(<image>) for content delivery, I
>> don't think we can afford to break compatibility there. What I propose is
>> adding a new UNPACK_HTML_FLIP_Y_WEBGL, which defaults to `true`, which
>> specifies that WebGL will upload HTML elements top-row-first into
>> texImage2D, as we do today. However, with `false`, we would upload elements
>> bottom-row-first, such that texImage2D(gl.canvas) has the same result as
>> copyTexImage2D.
>>
>>
>>
>> Disagree. No change needed. WebGL works the same as OpenGL.
>> UNPACK_FLIP_Y_WEBGL is there as a convenience only because unlike GL we
>> don't expect web developers to supply their own image loaders (though some
>> do) so we've provided the option to tell to the browser to flip the image
>> before providing it to the driver.
>>
>>
>>
>>
>>
>> -Jeff
>>
>>
>>
>> ----- Original Message -----
>> From: "Gregg Tavares (社用)" < gman@google.com >
>> To: "Jeff Gilbert" < jgilbert@mozilla.com >
>> Cc: "public webgl" < public_webgl@khronos.org >
>> Sent: Thursday, December 6, 2012 12:35:14 AM
>> Subject: Re: [Public WebGL] y-orientation for texImage2D from HTML
>> elements
>>
>>
>>
>> This isn't a bug. it's by design.
>>
>>
>> By default WebGL does the same as GL. It doesn't flip anything. The "data"
>> pointer passed to texImage2D represents the bottom left corner of the
>> texture. Whether that's an ArrayBuffer, an Image, a Canvas, a Video. For
>> Image, Canvas, and Video that means by default they'be upside down (which is
>> the same as GL).
>>
>>
>> CopyTexImage2D in GL has always been "effectively" backward relative to
>> TexImage2D when talking about images because most (all?) image libraries
>> load images with the first pixel in the top, left and GL expects the first
>> pixel is the bottom, left. Since CopyTexImage is copying internally it
>> appears to have different behavior because it's matching src data 0x0 =
>> bottom, left, dst data 0x0 = bottom left where as TexImage2D, when passing
>> standard image data, is src data 0x0 = top,left, dest = bottom, left
>>
>>
>> If you want WebGL to flip the data for you call
>> gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);
>>
>>
>> There is no PACK_FLIP_Y so readPixels will always return the bottom,left
>> as the first data in the array buffer.
>>
>>
>> As for matching implementations it's all tested in the conformance tests
>> AFAIK.
>>
>>
>>
>> On Thu, Dec 6, 2012 at 4:59 PM, Jeff Gilbert < jgilbert@mozilla.com >
>> wrote:
>>
>>
>>
>> It seems backwards (as implemented) and underdefined in the spec. It looks
>> like we all upload HTML element texImage data with 0,0=top-left, making it
>> effectively upside-down.
>>
>> The following two lines yield different results:
>> [1] gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0,
>> gl.drawingBufferWidth, gl.drawingBufferHeight, 0);
>> [2] gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE,
>> gl.canvas);
>>
>> [1] matches this: [3]
>> var data = new Uint8Array(gl.drawingBufferWidth * gl.drawingBufferHeight *
>> 4);
>> gl.readPixels(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight,
>> gl.RGBA, gl.UNSIGNED_BYTE, data);
>> gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.drawingBufferWidth,
>> gl.drawingBufferHeight, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);
>>
>> UNPACK_FLIP_Y_WEBGL=true makes [2] the same as [1], but now [3] is
>> upside-down.
>>
>> This is Mozilla bug 818810:
>> https://bugzilla.mozilla.org/show_bug.cgi?id=818810
>>
>> Testcase showing inconsistent behavior:
>> https://bugzilla.mozilla.org/attachment.cgi?id=689110
>>
>> As I mention in the bug, it doesn't look like y-orientation of texImage2D
>> of HTML elements is well-specified in the spec. So far everyone appears to
>> have implemented it the same, though.
>>
>> My suggestion was adding an UNPACK_ pixelStore option that allows us to
>> get OpenGL-style 0,0=bottom-left texImage2D uploads of HTML elements.
>>
>> -----------------------------------------------------------
>> You are currently subscribed to public_webgl@khronos.org .
>> To unsubscribe, send an email to majordomo@khronos.org with
>> the following command in the body of your email:
>> unsubscribe public_webgl
>> -----------------------------------------------------------
>>
>>
>>
>

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------