[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] ArrayBuffer, Concatenation and String-Array Conversion, a proposal

On 10/06/2010 07:38 AM, Chris Marrin wrote:
On Oct 6, 2010, at 7:30 AM, alan@mechnicality.com wrote:

On 10/05/2010 05:34 PM, Chris Marrin wrote:
And if we were to do that, I would propose we use base64 as the serialization format. The overhead is small, there are many tools for it, and it's already supported in other parts of HTML in the data: URL format. I think it would be a mistake to support some sort of direct binary string format.
Base64 is fine - what encoding? and/or how would you define the encoding? (UTF-8, UTF-16, ASCII etc. etc)
Isn't that taken care of by XHR and/or the HTML encoding?

I spent all night thinking about this. The short answer is "not very reliably". ArrayBuffers are by definition arrays of bytes, and thus opaque to any of the server mechanisms for automatically managing content encoding. By default, some servers *deliver* data as ISO8259-1 and require some special configuration to properly handle other encodings - this sounds amazing, but its true. Also, it depends where the data for the AB is originated, for example, created by a desktop app and posted to a server as a file or generated (and cached) on-line.
But aren't we talking about transmitting base64, which is text not binary.
Hmm, I sort of think we are talking about different things. I was really commenting on Vlad's email:
In this case a two-file solution, one JSON and one raw binary, would probably be the most efficient.  I understand the desire to work with a single file, though.  One thing we could do is add something to the DataView class for reading/writing strings to/from an ArrayBuffer using a given format; that might be a fairly natural fit for that type of functionality (which has been requested in other contexts as well -- e.g. reading a binary file that has an embedded string that refers to a texture file or something).

I was talking about/am interested in/think is important to explore the idea of embedding strings *in* the array buffer data which is what I thought Vlad was referring to - especially the last sentence. The OP wanted to put arraybuffer data in a JSON file, and I believed that Vlad was aiming at the reverse - put the JSON file inside the arraybuffer and use something like (moz)ResponseArrayBuffer. I've spent some considering a number of ways of transferring data from the server to the client and this suggestion has some interesting aspects. If there was a TypedArray for strings,  for example Utf8TypedArray, as has been proposed elsewhere then it would be easy to do. In this example the issue of encoding is resolved by the specific type of the array.

Of course, it doesn't have to be JSON, it could be other textual metadata which might make it easier for an application to manage assets. As always, there are many tradeoffs and gotchas.