Re: [Public WebGL] Support precompiled shaders as extensions

[Florian Bösch <pyalot@gmail.com>]

On Sat, Nov 12, 2016 at 9:54 PM, Mr F <arthur@playcanvas.com> wrote:

As there is more and more WebGL content around, shader compilation times start to annoy a lot (just go on shadertoy and see your tab hang for half a minute).

This is a known and not trivial problem. Slow shader compilation can stem from several factors:

1. ANGLE translation/unrolling, depending on circumstance it's possible to hit slow paths in the ANGLE translation, though that's been getting better.
2. Slow GL driver compilation, essentially unfixable.
3. HLSL compilation, usually better on newer versions of DX, except edge cases.

 

Browsers try to cache already compiled shaders, but that's it, it's hardly a solution IMHO.

No it isn't.

 

E.g. if you're on ANGLE, you can have a DX11 shader blob extension. Just feed it the binary, and it works. I don't think there's a security concern with it... you can make browsers compile almost any shader anyway, and they can't mess with unrelated user data.

I don't think DX shader blobs are an acceptable solution at all, in any scenario, for anybody.

On Mon, Nov 14, 2016 at 12:49 PM, Maksims Mihejevs <max@playcanvas.com> wrote:

Worth mentioning this topic on that:

http://toji.github.io/shader-perf/ by Brandon Jones.

Asynchronous compilation doesn't fix that at the time it's checked, it will block.

 

We already doing something similar, where we access parameters in delayed manner. This only works for assets loading scenarios, and does not work for runtime recompilation.

And even for loading scenario, this still takes ages, and in many cases shader compilation on medium devices can take more than half of loading times in extreme cases, that are seconds, for relatively simple demo cases.

These problems have to be solved:

1. How to signal shader compile completion without blocking (currently not possible)?
2. How to improve shader compilation speed drastically?
1. How to make sure ANGLE shader translation isn't compounding the problem?
2. How to improve slow GL driver compilation?
3. How to avoid HLSL slow compiles?

A relatively straightforward answer to this problem would be "Let's do WebVulkan", however we're a far way away from ever doing that. In the meantime, a solution would be nice.

#1: Nonblocking compile signaling (@khronos-wg)

Could we introduce a new getProgramParameter variable like gl.COMPILE_FINISHED or similar to produce nonblocking compile signaling? (there was some discussion of this in another thread). As there's no native support AFAIK, how do we avoid stalling the GPU process when checking the gl.COMPILE_STATUS and therefore prevent other shaders from continued compilation?

#2.1 Slow ANGLE translate phase (@google, @mozilla, @apple)

I'm not sure this is still a problem (though it might be with outdated ANGLE implementations), does this need more thorough testing? How to test this in isolation?

#2.2 Slow GL driver compiles (@nvidia, @amd, @intel)

There might not be anything that can be done to solve this (short of rewriting everybodies drivers). Is there some way to reduce compile times?

#2.3 Slow HLSL compiles (@microsoft)

This used to be a big problem in DX9 and apparently it got a bit better on DX11, though ANGLE based WebGL implementations still hit that problem occasionally, whereas it seems to me that Edge does less of that. Could you confirm that? If so, how does Edge avoid long compile times where ANGLE based HLSL backed still have it? Also how could HLSL compile times be improved in general?