[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] Proposal: Ignore UNPACK_FLIP_Y and UNPACK_PREMULTIPLY_ALPHA for ArrayBufferViews in WebGL 2



AFAIK all other JS APIs deal with unpremultipied image data, for example canvas2d's getImageData() returns an ImageData with an ArrayBuffer based on unpremultiplied data. It is convenient for developers if this type of data can be passed directly to WebGL simply by specifying a flag. Additionally given this fact I would anticipate it is used a lot and this change would break a lot of existing content. Our own WebGL 2D game engine relies on PREMULTIPLY_ALPHA for this reason, and I suspect many others do too.

A bigger problem is PREMULTIPLY_ALPHA is also necessary in some cases when passing a HTMLImageElement. Converting HTMLImageElement to an ImageData for premultiplication, with current JS APIs, often forces a synchronous image decode on the main thread, responsible for 100ms+ jank and unable to use multiple CPU cores to increase throughput. Async image encoding/decoding is currently a weak point on the web platform, and I've drafted a spec to mitigate this, which so far I don't believe has any implementation work going on: http://wicg.github.io/img-conversion/
Forcing the marshaling to happen in JS code could exacerbate this weakness and significantly increase loading times and loading jank, so it could actually significantly degrade the performance situation.

I'm also not sure JS code can reach the same level of performance as the browser's native code in premultiplying large amounts of data. This is the kind of thing direct memory access, SIMD and parallelisation can presumably help optimise (which JS/WebAssembly is gradually approaching, but not there yet).

I would suggest maintaining the options for backwards compatibility and performance reasons, but perhaps specifying that they are only applied for the cases they are currently applied in WebGL 1, and none of the new cases that apply with WebGL 2 if the prove particularly difficult to implement. I don't think the functionality should be removed until it is both plausible to deploy a WebAssembly-compiled library that can do the same work at close to the same performance, and when JS APIs to asynchronously decode images are available. Otherwise loading a large WebGL app will be a slow and janky hell.




On 4 November 2015 at 06:36, Florian Bösch <pyalot@gmail.com> wrote:
If you take a WebGL1 application and run it in WebGL2, will it break because of that?

On Wed, Nov 4, 2015 at 12:17 AM, Jeff Gilbert <jgilbert@mozilla.com> wrote:

Unpack gets a lot trickier in WebGL 2 with the addition of settings
for unpacking a subrect from a buffer, and there are a ton of new
formats as well.

FLIP_Y and PREMULTIPLY_ALPHA for ArrayBufferViews do not provide new
functionality to users, since they can (and should) be marshaling
their data into the format they actually want before passing it to us.
They should not ask us to do costly transformations on their data
before actually doing the upload. These functionalities can be handled
completely with existing JS, and could even be retrofitted onto the
WebGL prototype via monkey-patching.

It's also not clear exactly how FLIP_Y interacts with the subrect
selection UNPACK_ options, which would need to be speced, and will
likely be confusing for users regardless.

Further, compressedTex(Sub)Image doesn't and won't support these, so
users are already forced to have their orientation and
alpha-premultiplied status pre-marshaled when using these APIs.

Lastly, these are slow paths, and as such are performance foot-guns,
even if we can (and do) warn when users use this functionality.

WebGL 2 gives us the opportunity to not-quite-maintain backwards
compatibility, and I believe we should leverage this here.

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------