[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] Compressed textures.



I'm very familiar with the distinctions here - I started out as a
graphics programmer in 1977 - so I've been around the block a few times!

It's all very well to have one "wire" format and a different "GPU"
format - IF you have an efficient mechanism to convert between them. 
That's true in conventional graphics systems.  But in the world of the
browser, I either have to use JavaScript to S-L-O-W-L-Y interconvert
them (I hope nobody advocates that position!) - or I need a feature
inside WebGL that can use efficient compiled code to convert them - or I
need the "wire" format and the "GPU" format to be identical so that no
conversion is needed.  Given the nature of lossy compression and how
GPU's work, I'd prefer the latter because that gives me control over how
the losses are taken (just try compressing normal maps with a regular
RGB photo compressor!)

The inability to do compression efficiently in JavaScript is the crux of
the problem here - and why this issue is more important than it is for
other OpenGL varients.

JPEG is utterly useless as a texture format - the math and
human-perceptual research that went into that format are predicated on
the image being displayed with one pixel in the image file being one
pixel on the screen - with the image being presented orthographically
and within a relatively small range of resolutions.  Critically, it also
presumes a simple  ("gamma only") relationship between the brightness of
the pixel in the file and the final presentation brightness on the
faceplate of the screen.  Most of it's compression comes from knowing
that the human eye has better spatial resolution for luminance - and
much less for chrominance.  It will do things like placing a green pixel
next to a magenta pixel - assuming that the eye will see that as grey. 
Which it will when you're looking at a photo of your cat in a <IMG
SRC="mycat.jpg"/> situation.  It does that extremely well.

But NONE of those assumptions are true when you're storing a normal map
which is wrapped around a lighted, fogged teapot!   JPEG is great for
what it was designed to do - but it's a total disaster area for texture
maps where the texels may not even represent "color".  It's common in
shader-based applications to store surface normal or height displacement
values - or to have funky one-off formats like: R=luminance, G=alpha and
a B="distance from nearest star" (which happens to be how I store my
sky/cloud maps).  The idea that "image" files are actually RGB values is
not even close to the true story for modern graphics applications in an
era of ubiquitous shaders.

PNG is a great file format as far as it goes.  You get non-lossy RGBA
with one byte per color component.  But the resulting files are a lot
bigger than either DDS or JPEG - so they are a poor "wire" format - and
you need a ton of expensive (and customized) processing to turn them
into compressed GPU data.  But when you need quality or four-band RGBA
data, PNG is the only option in the web right now.  That sucks because
it's neither a good wire format nor a good GPU format.

GIF...let's just no go there, OK?!  :-)

However, the real, underlying hardware uses DXTn compression - if you
want to use compressed textures, then as a practical matter, you WILL
finally end up in DXT.  DXT-compressed files are tiny - smaller even
than JPEG in many cases.  They also make a great "wire" format - and
they are the only game in town for GPU compression.

Expecting the driver to compress PNG (or god-help-us, JPEG) into DXT is
a bad idea - the algorithm for compressing actual RGB photos is
different from that needed to compress normal maps - and different yet
again from the one I use to compress my sky maps.  Expecting to do
anything in realtime that can be handled offline is a bad idea.   Hence,
we need a file format that we can generate offline - load through WebGL
and into the hardware.

DDS is just about perfect for that.  It's basically just exactly the
data that the GPU wants - with a tiny header.  You can store maps with
floating point texels, 3D maps, cube-maps, depth maps, shadow maps...you
name it.  You get to select between all of the commonly supported DXT
formats.

So: If .dds is unusable for IP/patent reasons then we should come up
with something very similar and support it in WebGL.  Ideally, that
would be in 1.0 - but I'll take "later" if that's all I can get.

  -- Steve

Cedric Vivier wrote:
> Hi Steve,
>
> I think you are mixing network file (wire) formats and texture
> formats, JPEG is a perfectly fine format to load high-res photo-like
> textures as PNG is fine for any non-photo like texture.
> In fact these formats are usually lighter in space ("transmission
> time") compared to DDS or other compressed texture formats, even in
> their most compressed form (e.g 4bpp) without sacrificing significiant
> quality.
>
> Of course they cannot be understood by GL as a texture format
> directly, so they are decompressed into one of the basic texture
> formats (e.g RGB888), considering compressed texture formats are very
> specific to hardware/drivers (interop issues) and most of them
> encumbered by IP issues of some sort, afaik it has been decided to
> postpone compressed textures support to post 1.0 (a thread discussing
> this issue: http://www.khronos.org/webgl/public-mailing-list/archives/1005/msg00022.html
> )
>
>
> Regards,
>
>
> On Sun, May 30, 2010 at 09:30, Steve Baker <steve@sjbaker.org> wrote:
>   
>> I guess we're more or less limited on texture file types to the set that
>> JavaScript generally supports: GIF/PNG/JPG ?  Since GIF and JPG are
>> totally hopeless as texture formats and PNG's compression isn't
>> agressive enough because it's not lossy, it would be truly spectacularly
>> handy if WebGL could also support some kind of compressed format - DDS
>> would be best.  (Although it's a bit 'microsofty/D3D-ish' - it's
>> actually a REALLY good  format for textures with tons of support out
>> there and a 1:1 mapping to the underlying hardware).
>>
>> Given that we'd like to run on low-end hardware, and even on high end
>> stuff, the transmission time from the server is not insignificant -
>> also, we don't want to use JavaScript to do compression on-the-fly.
>>
>> Thoughts?
>>
>>  -- Steve
>>
>> -----------------------------------------------------------
>> You are currently subscribed to public_webgl@khronos.org.
>> To unsubscribe, send an email to majordomo@khronos.org with
>> the following command in the body of your email:
>>
>>
>>     

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email: