[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] OES_texture_half_float doesn't allow Tex(Sub)Image2D to take ArrayBuffers as input



The extensions for half float as well as the ES 3.0 specification, specify half float as 1-bit sign, 5-bit exponent and 10-bit mantissa. The IEEE 754-2008 specification defines a half float as 1-bit sign, 5-bit exponent and 10-bit mantissa. It also defines the precise interpretation of the bits. While the specification only notes that the 32-bit floats are IEEE compliant, it would seem to me that the 16-bit variant is largely compliant, except perhaps for the allowance of interpretations that do not contain NaN or Infinity clauses. These would still largely work though on GPUs, it's just that Infinity would be interpreted by the GPU to be +- 2^(E-15) and NaN would be interpreted as +- 2^(E-15) * (1+ M/2^10).

 
On Thu, Feb 13, 2014 at 1:54 AM, Kenneth Russell <kbr@google.com> wrote:
C source code is available which can pack and unpack values into and
from 16-bit floats, and it might be possible to translate this code to
_javascript_. I'm not sure whether _javascript_ provides the necessary
math primitives. See
http://en.wikipedia.org/wiki/Half-precision_floating-point_format and
the references at the bottom.

If it were possible to do the packing in _javascript_, then plausibly
UNSIGNED_SHORT should be allowed as an upload format, and the user
would be responsible for packing their own data. We definitely should
not add a Float16 ArrayBuffer view type. That would add a lot of
complex code to _javascript_ VMs for a very narrow use case.

 
Here's some code in JS (http://codeflow.org/experiment/half-float/) that can convert numbers to half floats expressed as ushorts, and ushort expressed half floats to numbers. It tests the examples listed on wikipedia plus all their negatives, in both directions (half -> num, num -> half) and it also runs trough every short bit pattern to see if the back/forth comparison yields the same result: http://codeflow.org/experiment/half-float/

There's some corner cases this probably misses (what if you input numbers larger than expressable by half-float?) and it doesn't account for endianness. But I think it demonstrates that you can implement half-floats in JS. Now obviously, a Float16Array would be a *lot* more convenient and probably a bunch better performing.



On Thu, Feb 13, 2014 at 7:32 AM, Vladimir Vukicevic <vladimir@mozilla.com> wrote:



----- Original Message -----
> From: "Jeff Gilbert" <jgilbert@mozilla.com>
>
> I agree that trying to spec a Float16Array is out of scope, but it's
> completely possible to have a blob (or user-JS-packed data) that it would be
> desirable to upload. Just as we allow for types like UNSIGNED_SHORT_5_6_5 to
> be represented by Uint16 arrays, it seems reasonable to allow HALF_FLOAT to
> be, as well.
>
> Even with a lack of typed array, it should still be possible to submit a raw
> blob untyped ArrayBuffer of the right size.
> I'm going to submit a quick PR of this.

Huh, I've just assumed this was always in there!  Using a raw ArrayBuffer is problematic, because we don't have a way to refer to a sub-ArrayBuffer.  IMO, we should accept a Uint16Array, to match common the common storage data format in native OpenGL usage.

And on that note, I'm going to propose an extension soonish to do the ArrayBuffer case -- add entry points that take an ArrayBuffer, offset, length.  I've seen JS code that runs into this where it loads a blob of data from network/file/wherever and then wants to upload it needing to create a bunch of unnecessary views.  We're also running into this a lot with Emscripten/asm.js code, where the views end up being unnecessary GC objects created/destroyed where we have no need of any GC objects.

    - Vlad

> ----- Original Message -----
> From: "Kenneth Russell" <kbr@google.com>
> To: "Jeff Gilbert" <jgilbert@mozilla.com>
> Cc: "public webgl" <public_webgl@khronos.org>
> Sent: Wednesday, February 12, 2014 4:54:00 PM
> Subject: Re: [Public WebGL] OES_texture_half_float doesn't allow
> Tex(Sub)Image2D to take ArrayBuffers as input
>
> This limitation was deliberate, because there is no ArrayBuffer view
> type which supports working with half-float data. The main use case
> for half-float textures is memory reduction compared to 32-bit float
> textures when using them as render targets.
>
> C source code is available which can pack and unpack values into and
> from 16-bit floats, and it might be possible to translate this code to
> _javascript_. I'm not sure whether _javascript_ provides the necessary
> math primitives. See
> http://en.wikipedia.org/wiki/Half-precision_floating-point_format and
> the references at the bottom.
>
> If it were possible to do the packing in _javascript_, then plausibly
> UNSIGNED_SHORT should be allowed as an upload format, and the user
> would be responsible for packing their own data. We definitely should
> not add a Float16 ArrayBuffer view type. That would add a lot of
> complex code to _javascript_ VMs for a very narrow use case.
>
> I think it would be possible to re-ratify a new version of the
> extension, rather than creating a new one. At least, I'd argue for
> simply revising the extension, and would be willing to raise it with
> the Khronos board of promoters. However, I am not sure that focusing
> on this issue is the best investment of the working group's time.
> There are a lot of other outstanding issues that are more pressing in
> my opinion -- advancing the WebGL 2.0 spec and implementations in
> particular.
>
> -Ken
>
>
> On Wed, Feb 12, 2014 at 4:04 PM, Jeff Gilbert <jgilbert@mozilla.com> wrote:
> >
> > Surely it should?
> > Instead, it appears that the only way to upload data into them is through a
> > uselessly imprecise image/canvas/video source.
> >
> > We have since ratified this extension. If we want to accept a change, how
> > does that work? Do we have to make a new extension?
> >
> > -Jeff
> >
> > -----------------------------------------------------------
> > You are currently subscribed to public_webgl@khronos.org.
> > To unsubscribe, send an email to majordomo@khronos.org with
> > the following command in the body of your email:
> > unsubscribe public_webgl
> > -----------------------------------------------------------
> >
>
> -----------------------------------------------------------
> You are currently subscribed to public_webgl@khronos.org.
> To unsubscribe, send an email to majordomo@khronos.org with
> the following command in the body of your email:
> unsubscribe public_webgl
> -----------------------------------------------------------
>
>

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------