Typed Array Specification

Version 1.0, 08 February 2011

This version:
https://www.khronos.org/registry/typedarray/specs/1.0/
Web IDL: https://www.khronos.org/registry/typedarray/specs/1.0/typedarray.idl
Latest version:
https://www.khronos.org/registry/typedarray/specs/latest/
Web IDL: https://www.khronos.org/registry/typedarray/specs/latest/typedarray.idl
Previous version:
none
Editors:
Vladimir Vukicevic (Mozilla Corporation)
Kenneth Russell (Google, Inc.)

Abstract

This specification provides an API for interoperability with native binary data. It defines a generic fixed-length buffer type, as well as accessor types that allow access to the data stored within the buffer.

The functionality described here originated in the WebGL specification [WEBGL].

Status of this document

Public discussion of this specification is welcome on the (archived) WebGL mailing list public_webgl@khronos.org (see instructions).

Table of contents

Introduction

ECMAScript [ECMA-262] has traditionally been used in contexts where there is no access to binary data. Where binary data has needed to be manipulated, it is often stored as a String and accessed using charCodeAt(), or stored as an Array with conversion to and from base64 for transmission. Both of these methods are slow and error-prone. For example, reading binary data as 32-bit integers requires manual conversion of 4 source bytes to and from the target type. Reading floating-point data is even more expensive.

As web applications gain access to new functionality, working with binary data has become a much-demanded feature. Current specifications such as the File API [FILEAPI] and Web Sockets [WEBSOCKETS] would benefit from being able to read and write binary data directly in its native form. Specifications such as WebGL [WEBGL] require this functionality to meet acceptable performance characteristics.

This specification defines a minimal set of functionality for accessing binary data from ECMAScript.

Overview

This specification defines an ArrayBuffer type, representing a generic fixed-length binary buffer. It is not possible to manipulate the contents of an ArrayBuffer directly. Instead, a group of types are used to create views of the ArrayBuffer. For example, to access the buffer as an array of 32-bit signed integers, an Int32Array would be created that refers to the ArrayBuffer.

Multiple typed array views can refer to the same ArrayBuffer, of different types, lengths, and offsets. This allows for complex data structures to be built up in the ArrayBuffer. As an example, given the following code:

      // create an 8-byte ArrayBuffer
      var b = new ArrayBuffer(8);

      // create a view v1 referring to b, of type Int32, starting at
      // the default byte index (0) and extending until the end of the buffer
      var v1 = new Int32Array(b);

      // create a view v2 referring to b, of type Uint8, starting at
      // byte index 2 and extending until the end of the buffer
      var v2 = new Uint8Array(b, 2);

      // create a view v3 referring to b, of type Int16, starting at
      // byte index 2 and having a length of 2
      var v3 = new Int16Array(b, 2, 2);
    

The following buffer and view layout is created:

varindex
bytes (not indexable)
b = 01234567
indices
v1 = 01
v2 = 012345
v3 = 01

This defines an 8-byte buffer b, and three views of that buffer, v1, v2, and v3. Each of the views refers to the same buffer -- so v1[0] refers to bytes 0..3 as a signed 32-bit integer, v2[0] refers to byte 2 as a unsigned 8-bit integer, and v3[0] refers to bytes 2..3 as a signed 16-bit integer. Any modification to one view is immediately visible in the other: for example, after v2[0] = 0xff; v2[1] = 0xff; then v3[0] == -1 (where -1 is represented as 0xffff).

Type Conversion Rules

The specification below implicitly references certain type conversion rules; for example, conversion of floating-point values to integer values of various sizes. The Web IDL specification [WEBIDL] defines these rules, and references the ECMA-262 specification [ECMA-262] for conversion algorithms such as ToInt32.

The Web IDL specification does not currently define all of the numerical types referenced in this specification; for example, byte, which is a signed 8-bit integer type. For these types, the rules for the type of the closest size and signedness shall be extrapolated.

As a hint to implementors of the algorithms in the ECMA-262 specification, conversion of floating-point numbers to integer values uses the truncate, or round-to-zero, rounding mode.

Handling of Not-a-Number (NaN) Values

When the not-a-number (NaN) value is stored into a Float32Array or Float64Array, or into a DataView using the setFloat32 or setFloat64 methods, the bit pattern written into the underlying ArrayBuffer is not specified, but shall be one of the IEEE 754 bit patterns that represent NaN [IEEE-754].

When a bit pattern representing an IEEE 754 NaN is loaded from a Float32Array or Float64Array, or from a DataView using the getFloat32 or getFloat64 methods, the language binding (for example, ECMAScript) may use an alternative bit pattern to represent the NaN value.

The Web IDL [WEBIDL] and ECMA-262 [ECMA-262] specifications govern all other handling of NaN values, in particular the conversion to 0 when converting NaN to an integer value.

The ArrayBuffer Type

The ArrayBuffer type describes a buffer used to store data for the TypedArray interface and its subclasses. An ArrayBuffer has the following methods and properties:

[ Constructor(unsigned long length) ]
interface ArrayBuffer {
    readonly attribute unsigned long byteLength;
};
Constructors
ArrayBuffer(unsigned long length)

Creates a new ArrayBuffer of the given length in bytes. The contents of the ArrayBuffer are initialized to 0. If the requested number of bytes could not be allocated an exception is raised.

Properties
unsigned long byteLength

Read-only property.

The length of the ArrayBuffer in bytes, as fixed at construction time.

Methods
None

The ArrayBufferView Type

The ArrayBufferView type holds information shared among all of the types of views of ArrayBuffers. An ArrayBufferView has the following properties:

interface ArrayBufferView {
    readonly attribute ArrayBuffer buffer;
    readonly attribute unsigned long byteOffset;
    readonly attribute unsigned long byteLength;
};
Constructors
None
Properties
ArrayBuffer buffer

Read-only property.

The ArrayBuffer that this ArrayBufferView references.

unsigned long byteOffset

Read-only property.

The offset of this ArrayBufferView from the start of its ArrayBuffer, in bytes, as fixed at construction time.

unsigned long byteLength

Read-only property.

The length of the ArrayBufferView in bytes, as fixed at construction time.

Methods
None

The Typed Array View Types

The typed array view types represent a view of an ArrayBuffer that allows for indexing and manipulation. The length of each of these is fixed. Each of the typed array view types follows the same template.

The following typed array view types are defined by this specification. The size below is given in bytes, and corresponds to the BYTES_PER_ELEMENT constant for the given type.

TypeSizeDescriptionEquivalent C Type
Int8Array18-bit 2's complement signed integersigned char
Uint8Array18-bit unsigned integerunsigned char
Int16Array216-bit 2's complement signed integershort
Uint16Array216-bit unsigned integerunsigned short
Int32Array432-bit 2's complement signed integerint
Uint32Array432-bit unsigned integerunsigned int
Float32Array432-bit IEEE floating pointfloat
Float64Array864-bit IEEE floating pointdouble

Each of the typed array types has the following constructors, properties, constants and methods. In the descriptions below, the generic term TypedArray is used to indicate that any valid typed array view type is allowed.

[
    Constructor(unsigned long length),
    Constructor(TypedArray array),
    Constructor(type[] array),
    Constructor(ArrayBuffer buffer,
                optional unsigned long byteOffset, optional unsigned long length)
]
interface TypedArray : ArrayBufferView {
    const unsigned long BYTES_PER_ELEMENT = element size in bytes;

    readonly attribute unsigned long length;

    omittable getter type get(unsigned long index);
    omittable setter void set(unsigned long index, type value);
    void set(TypedArray array, optional unsigned long offset);
    void set(type[] array, optional unsigned long offset);
    TypedArray subarray(long begin, optional long end);
};
Constructors
TypedArray(unsigned long length)

Create a new ArrayBuffer with enough bytes to hold length elements of this typed array, then creates a typed array view referring to the full buffer. As with a directly constructed ArrayBuffer, the contents are initialized to 0. If the requested number of bytes could not be allocated an exception is raised.

TypedArray(TypeArray array)
TypedArray(type[] array)

Create a new ArrayBuffer with enough bytes to hold array.length elements of this typed array, then creates a typed array view referring to the full buffer. The contents of the new view are initialized to the contents of the given array or typed array, with each element converted to the appropriate typed array type.

TypedArray(ArrayBuffer buffer, optional unsigned long byteOffset, optional unsigned long length)

Create a new TypedArray object using the passed ArrayBuffer for its storage. Optional byteOffset and length can be used to limit the section of the buffer referenced. The byteOffset indicates the offset in bytes from the start of the ArrayBuffer, and the length is the count of elements from the offset that this TypedArray will reference. If both byteOffset and length are omitted, the TypedArray spans the entire ArrayBuffer range. If the length is omitted, the TypedArray extends from the given byteOffset until the end of the ArrayBuffer.

The given byteOffset must be a multiple of the element size of the specific type, otherwise an exception is raised.

If a given byteOffset and length references an area beyond the end of the ArrayBuffer an exception is raised.

If length is not explicitly specified, the length of the ArrayBuffer minus the byteOffset must be a multiple of the element size of the specific type, or an exception is raised.

Constants
unsigned long BYTES_PER_ELEMENT The size in bytes of each element in the array.
Properties
unsigned long length

Read-only property.

The length of the TypedArray in elements, as fixed at construction time.

Methods
omittable getter type get(unsigned long index)

This is an index getter.

Returns the element at the given numeric index.

See handling of NaN values for additional rules covering Float32Array and Float64Array.

omittable setter void set(unsigned long index, type value)

This is an index setter.

Sets the element at the given numeric index to the given value.

Conversions of values to type are defined by the type conversion rules and handling of NaN values.

void set(TypedArray array, optional unsigned long offset)
void set(type[] array, optional unsigned long offset)

Set multiple values, reading input values from the array. The optional offset value indicates the index in the current array where values are written. If omitted, it is assumed to be 0.

If the input array is a TypedArray, the two arrays may use the same underlying ArrayBuffer. In this situation, setting the values takes place as if all the data is first copied into a temporary buffer that does not overlap either of the arrays, and then the data from the temporary buffer is copied into the current array.

If the offset plus the length of the given array is out of range for the current TypedArray, an exception is raised.

TypedArray subarray(long begin, optional long end)

Returns a new TypedArray view of the ArrayBuffer store for this TypedArray, referencing the elements at begin, inclusive, up to end, exclusive. If either begin or end is negative, it refers to an index from the end of the array, as opposed to from the beginning.

If end is unspecified, the subarray contains all elements from begin to the end of the TypedArray.

The range specified by the begin and end values is clamped to the valid index range for the current array. If the computed length of the new TypedArray would be negative, it is clamped to zero.

The returned TypedArray will be of the same type as the array on which this method is invoked.

The DataView View Type

An ArrayBuffer is a useful object for representing an arbitrary chunk of data. In many cases, such data will be read from disk or from the network, and will not follow the alignment restrictions that are imposed on the typed array views described earlier. In addition, the data will often be heterogeneous in nature and have a defined byte order. The DataView view provides a low-level interface for reading such data from and writing it to an ArrayBuffer.

[
  Constructor(ArrayBuffer buffer,
              optional unsigned long byteOffset,
              optional unsigned long byteLength)
]
interface DataView : ArrayBufferView {
    // Gets the value of the given type at the specified byte offset
    // from the start of the view. There is no alignment constraint;
    // multi-byte values may be fetched from any offset.
    //
    // For multi-byte values, the optional littleEndian argument
    // indicates whether a big-endian or little-endian value should be
    // read. If false or undefined, a big-endian value is read.
    //
    // These methods raise an exception if they would read
    // beyond the end of the view.
    byte getInt8(unsigned long byteOffset);
    unsigned byte getUint8(unsigned long byteOffset);
    short getInt16(unsigned long byteOffset,
                   optional boolean littleEndian);
    unsigned short getUint16(unsigned long byteOffset,
                             optional boolean littleEndian);
    long getInt32(unsigned long byteOffset,
                  optional boolean littleEndian);
    unsigned long getUint32(unsigned long byteOffset,
                            optional boolean littleEndian);
    float getFloat32(unsigned long byteOffset,
                     optional boolean littleEndian);
    double getFloat64(unsigned long byteOffset,
                      optional boolean littleEndian);

    // Stores a value of the given type at the specified byte offset
    // from the start of the view. There is no alignment constraint;
    // multi-byte values may be stored at any offset.
    //
    // For multi-byte values, the optional littleEndian argument
    // indicates whether the value should be stored in big-endian or
    // little-endian byte order. If false or undefined, the value is
    // stored in big-endian byte order.
    //
    // These methods raise an exception if they would write
    // beyond the end of the view.
    void setInt8(unsigned long byteOffset,
                 byte value);
    void setUint8(unsigned long byteOffset,
                  unsigned byte value);
    void setInt16(unsigned long byteOffset,
                  short value,
                  optional boolean littleEndian);
    void setUint16(unsigned long byteOffset,
                   unsigned short value,
                   optional boolean littleEndian);
    void setInt32(unsigned long byteOffset,
                  long value,
                  optional boolean littleEndian);
    void setUint32(unsigned long byteOffset,
                   unsigned long value,
                   optional boolean littleEndian);
    void setFloat32(unsigned long byteOffset,
                    float value,
                    optional boolean littleEndian);
    void setFloat64(unsigned long byteOffset,
                    double value,
                    optional boolean littleEndian);
};

The following constructors, properties, and methods are available on a DataView:

Constructors
DataView(ArrayBuffer buffer, optional unsigned long byteOffset, optional unsigned long byteLength)

Create a new DataView object using the passed ArrayBuffer for its storage. Optional byteOffset and byteLength can be used to limit the section of the buffer referenced. The byteOffset indicates the offset in bytes from the start of the ArrayBuffer, and the byteLength is the number of bytes from the offset that this DataView will reference. If both byteOffset and byteLength are omitted, the DataView spans the entire ArrayBuffer range. If the byteLength is omitted, the DataView extends from the given byteOffset until the end of the ArrayBuffer.

If the given byteOffset and byteLength references an area beyond the end of the ArrayBuffer an exception is raised.

Properties
None
Methods
byte getInt8(unsigned long byteOffset);
unsigned byte getUint8(unsigned long byteOffset);
short getInt16(unsigned long byteOffset, optional boolean littleEndian);
unsigned short getUint16(unsigned long byteOffset, optional boolean littleEndian);
long getInt32(unsigned long byteOffset, optional boolean littleEndian);
unsigned long getUint32(unsigned long byteOffset, optional boolean littleEndian);
float getFloat32(unsigned long byteOffset, optional boolean littleEndian);
double getFloat64(unsigned long byteOffset, optional boolean littleEndian);

Gets the value of the given type at the specified byte offset from the start of the view. There is no alignment constraint; multi-byte values may be fetched from any offset.

For multi-byte values, the optional littleEndian argument indicates whether a big-endian or little-endian value should be read. If false or undefined, a big-endian value is read.

These methods raise an exception if they would read beyond the end of the view.

See handling of NaN values for additional rules covering getFloat32 and getFloat64.

void setInt8(unsigned long byteOffset, byte value);
void setUint8(unsigned long byteOffset, unsigned byte value);
void setInt16(unsigned long byteOffset, short value, optional boolean littleEndian);
void setUint16(unsigned long byteOffset, unsigned short value, optional boolean littleEndian);
void setInt32(unsigned long byteOffset, long value, optional boolean littleEndian);
void setUint32(unsigned long byteOffset, unsigned long value, optional boolean littleEndian);
void setFloat32(unsigned long byteOffset, float value, optional boolean littleEndian);
void setFloat64(unsigned long byteOffset, double value, optional boolean littleEndian);

Stores a value of the given type at the specified byte offset from the start of the view. There is no alignment constraint; multi-byte values may be stored at any offset.

For multi-byte values, the optional littleEndian argument indicates whether the value should be stored in big-endian or little-endian byte order. If false or undefined, the value is stored in big-endian byte order.

These methods raise an exception if they would write beyond the end of the view.

Conversions of values to the given type are defined by the type conversion rules and handling of NaN values.

Examples

Creating a simple array of 128 32-bit floats:
var f32s = new Float32Array(128);
Filling each 8 consecutive floats of the new array:
for (var i = 0; i < 128/8; ++i) {
  var sub_f32s = f32s.subarray(i, i+8);
  for (var j = 0; j < 8; ++j) {
    sub_f32s[j] = j;
  }
}
      

Note that this code uses subarray() to create a new Float32Array that references the same data as the original, so that it can always index the sub-array using 0..7.

Interleaved array types

Some APIs, in particular WebGL [WEBGL], can benefit from being able to use a single contiguous buffer, with interleaved data types. For example, a point might have coordinate data (3 Float32 values) followed by color data (4 Uint8 values).

For 4 points and their associated colors, this can be set up in the following way:

var elementSize = 3 * Float32Array.BYTES_PER_ELEMENT + 4 * Uint8Array.BYTES_PER_ELEMENT;
var buffer = new ArrayBuffer(4 * elementSize);
var coords = new Float32Array(buffer, 0);
var colors = new Uint8Array(buffer, 3 * Float32Array.BYTES_PER_ELEMENT);
      

However, typed arrays don't have a way to explicitly encode the desired per-point structure, so some manual arithmetic must be done to correctly index into the right values. Note that the colors Uint8Array view is created with an explicit offset (which is given in bytes), so that the [0] element points at the 13th byte in the underlying buffer.

In this example, each set of packed data is 16 bytes in size (3 4-byte floats followed by 4 bytes). 16 / Float32Array.BYTES_PER_ELEMENT is 4, so from any given Float32 element, to skip to the same Float32 element in the next point, 4 must be added to the index. Similarly, to skip from any given Uint8 element to the same in the next point, 16 must be added:

var coordOffset = elementSize / Float32Array.BYTES_PER_ELEMENT;
var colorOffset = elementSize / Uint8Array.BYTES_PER_ELEMENT;

coords[0] = coords[1] = coords[2] = 1.0; // The first point's three coordinate values
colors[0] = colors[1] = colors[2] = colors[3] = 255; // The first point's four colors

coords[0+N*coordOffset] = 5.0; // The Nth point's first coordinate value
colors[0+N*colorOffset] = 128; // The Nth point's first color value

coords[i+N*coordOffset] = 6.0; // The Nth point's i coordinate value;
colors[j+N*colorOffset] = 200; // The Nth point's j color value
      

In the above example, note that for keeping the data consistent, i must be one of 0, 1, or 2; and j must be one of 0, 1, 2, or 3. Any higher values will result in data segments that are reserved for 32-bit floats or for 8-bit integers being overwritten with incorrect data.

Slicing a large array into multiple regions

Another usage similar to the above is allocating one large buffer, and then using different regions of it for different purposes:

var buffer = new ArrayBuffer(1024);
      

Carve out 128 floats, 128*4 = 512 bytes in size:

var floats = new Float32Array(buffer, 0, 128);
      

Then 128 shorts, 128*2 = 256 bytes in size, immediately following the floats. Note that the 512 byte offset argument is equal to floats.byteOffset + floats.byteLength.

var shorts = new Uint16Array(buffer, 512, 128);
      

Finally, 256 unsigned bytes. We can write the byte offset in the form suggested above to simplify the chaining. We also let this array extend until the end of the ArrayBuffer by not explicitly specifying a length.

var bytes = new Uint8Array(buffer, shorts.byteOffset + shorts.byteLength);
      

If the data is no longer needed, the entire 1024-byte array can be repurposed without causing additional allocations simply by creating new views and discarding the old.

Resolved Issues

Should there be a way to copy or subset an ArrayBuffer?

Not at this time. A future version of this spec may introduce such functionality.

Should DataView have methods for reading arrays of data?

While such methods would be useful, they are not present in this version of the specification to reduce the API footprint. Such methods may be added in the future.

References

[ECMA-262]
ECMA-262 Specification
[IEEE-754]
IEEE Standard for Binary Floating-Point Arithmetic (ANSI/IEEE Std 754-1985). Institute of Electrical and Electronics Engineers, 1985.
[WEBIDL]
Web IDL
[WEBGL]
Khronos WebGL Specification
[FILEAPI]
File API
[WEBSOCKETS]
The Web Sockets API