> In the fragment shader the texture2D returns a float per component with a range greater than [0.. 1.0].
> I do not think this complies with the OpenGL specification.
A poor choice of words on my part. What I meant is that texture2D returns float components between [0.. 1.0]. However the float type can extend beyond the range [0.. 1.0] so the data type has sufficient size to handle a larger range. The point being that the ranges of the data types are different and don’t map perfectly.
I do not think this complies with the OpenGL specification. Section 2.1.2 of the OpenGL ES 2.0 describes a conversion that gives a float component with the range [0..1.0]
The texel component values go through type and range conversions between definition in glTexImage2D and when they are used in the fragment shader. At glTexImage2D time the components are typically ubyte, [0..255]. In the fragment shader the texture2D returns a float per component with a range greater than [0.. 1.0]. When the color fragment is written to the destination surface there’s another type and range conversion back to [0..255] per component.
On lower precision hardware, if the fragment shader is doing arithmetic on the texture values, then the result could fall into a range that ends up with an unexpected 1 bit lsb difference after conversion to the framebuffer format.
A simplified description would be something like [0..0xFF] gets converted to the range [0..0x100] and back to [0..0xFF]. The intermediate range has ‘holes’ that lose on the conversion to the final range.