[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] TypedArray spec updates

On May 12, 2010, at 4:23 PM, Kenneth Russell wrote:

I still don't understand the DataArray API. I don't see why we need a separate view that is essentially an uberview, being able to view any offset as any type. That seems like overkill. It seems like it would be better if each view had a setter and getter that took an ArrayBuffer and a littleEndian flag. The flag would tell you the byte order of the data in the ArrayBuffer. Let's say I have some future version of XHR, which can give me data as an ArrayBuffer. Then I could do this:

       function XHRDataReady(request)
               var data = "" // New imaginary XHR API
               var buffer = new ArrayBuffer(10 * 2 + 10 * 4); // enough space for 10 shorts and 10 floats
               var shorts = new Int16Array(buffer, 0, 10);
               var floats = new FloatArray(buffer, 10 * 2, 10);
               shorts.setData(data, 0, 10, true); // Data I loaded with XHR is littleEndian
               floats.setData(data, 10 * 2, 10, true);

This would flip the order of the incoming data or not, as needed.

With your API, would I use it like this:

       function XHRDataReady(request)
               var data = "" // New imaginary XHR API
               var dataArray = new DataArray(data);

               var buffer = new ArrayBuffer(10 * 2 + 10 * 4); // enough space for 10 shorts and 10 floats
               var shorts = new Int16Array(buffer, 0, 10);
               var floats = new FloatArray(buffer, 10 * 2, 10);

               // Set the shorts
               for (var i = 0; i < 10; ++i)
                       shorts[i] = dataArray.getInt16(i * 2, true);

               // Set the floats
               for (var i = 0; i < 10; ++i)
                       floats[i] = dataArray.getFloat(i * 4 + 10 * 2, true);

First of all, I'm not sure passing 'true' is the right thing to do. I know my data is littleEndian, but don't I have to check to see if the data in the arrays needs to be littleEndian and then flip the flag if needed?

Second of all, it seems like having the uber-view is more complicated than just putting the API on the existing views.

The design constraints are:

- Support unaligned loads / stores. This is a requirement to handle
arbitrary file formats.

The setData() method would handle unaligned data since you pass it a byte offset to the start of the data.

- Support specified endianness rather than "native" endianness as in
the TypedArrays. Most file formats are defined in terms of big-endian
or little-endian values.

This is where I'm confused. Your current spec seems to indicate that the littleEndian param will place data into the destination buffer in little or big endian form. I think the source data can be in a specified endianness, but the data in the views should always be in native endianness.
- Do not slow down the fast path of the typed arrays.

I'm not sure what that means,  but certainly the setData() method collapses to memcpy if the src and dest endianness match.

The best way to achieve all of these goals is to provide an
alternative view, decoupled from the typed arrays. Doing so avoids any
conditionals in the fast path of the typed arrays, while still
providing reasonably fast access via the new view's getters and
setters, which can be implemented with a few machine instructions

I'm not sure why that is. Only the setData() method would have to do endianness tests. The rest of the existing API would not change, nor would the implementation have to do any extra work because of endianness. It's true that it would be possible to read data with the wrong endianness into an ArrayBuffer and then map that onto some multi-byte view that would  then have incorrect data. But that will be possible no matter what.

I don't think we need to generally deal with data in a view that is in the wrong endianness. So endian flipping wouldn't have to pervade the API calls. It's just setData() and getData() that have to worry about that, getting data into and out of the buffer.

Your proposal above doesn't handle the unaligned data case.

Since setData() is passed a byte offset it seems like it does handle the unaligned data case, doesn't it?

Perhaps an example of the best way to use DataArray would help?

I envision most users wrapping it into a stream-like interface,
fetching individual data elements of various types and incrementing
the byte offset of the stream. The proposal I originally sent out was
of that form, though I agree based on your earlier feedback that
lower-level functionality should be provided and this implemented in

Right, my comments now have to do with the API now that I've had a chance to see it. Maybe we went over some showstopper problems with my approach, but I don't remember them. And I think a separate uber-view feels too heavyweight.