[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] Support precompiled shaders as extensions



On Mon, Nov 14, 2016 at 11:04 AM, Mr F <arthur@playcanvas.com> wrote:
>Any sort of precompiled binary fed directly to the driver would bypass that, which means your shader would randomly break. 
Somehow people ship games (at least on D3D) with precompiled shaders, and I haven't seen too many problems with that (OK, I did in about 2007, but today things are way better).


Desktop game developers often have contacts at IHVs and can afford to test on multiple hardware configurations. This is not the case for WebGL developers which need WebGL to be portable so they can write on a platform and expect things to work fine on other platforms.

You might argue that PlayCanvas can afford to test like game developers do but:
  • The web can't expose a non-portable API. If a WebGL extension is present, then it needs to always work and not require extensive testing
  • New drivers and hardware will eventually come out that will break your precompiled shaders
  • Feeding DXIL directly to the driver is a big security hole
Here are some of the D3D-specific workarounds in ANGLE.
https://github.com/google/angle/blob/master/src/libANGLE/renderer/d3d/ShaderD3D.cpp#L43
https://github.com/google/angle/blob/master/src/libANGLE/renderer/d3d/WorkaroundsD3D.h#L27

Also ANGLE does dynamic compilation of D3D shaders to emulate some OpenGL ES behaviors and giving DXIL would prevent that.

For all of the reasons above, a precompiled shader extension can't happen. We should instead look at tackling the shader compilation problem in other ways. How about having a glFenceSync for GL_CPU_COMMANDS_BIT so that you can know when your shader has been compiled? Can we make shader compilation parallel? etc.

> Also the shader compile overhead of browsers is tiny compared to the driver compiling to hardware ISA
I do believe that translation is fast, but in my experience the actual hlsl->d3d bytecode part is the slowest, much slower than bytecode->hardware-specific. All the loop unrolling, function inlining, register counting and a load of optimizations happen here. And we'd better leave these optimizations enabled.

On 14 November 2016 at 18:49, Corentin Wallez <cwallez@google.com> wrote:
Also don't forget about the workarounds (35ish and counting) that are done in the ANGLE shader translator. Any sort of precompiled binary fed directly to the driver would bypass that, which means your shader would randomly break. Also the shader compile overhead of browsers is tiny compared to the driver compiling to hardware ISA, so what you need is more like asynchronous compiles.

Finally security-wise browsers will never trust drivers gracefully failing when given bad output, they want to own the frontend and handle errors there.

On Mon, Nov 14, 2016 at 10:03 AM, Florian Bösch <pyalot@gmail.com> wrote:
On Mon, Nov 14, 2016 at 3:11 PM, Maksims Mihejevs <max@playcanvas.com> wrote:
It does not feel that binary shaders is a complex feature. All background workings of WebGL already has functionality to deal with it I'm sure. It is only about exposing such functionality using extensions notation based on platforms.

Precompiled shaders are quite a huge problem. There's two main scenarios to consider:

D3D bytecode shaders

Supplying your shaders are precompiled D3D bytecode has a number of problems, these are:
  1. Depends on the D3D version (DX9 binaries are different than DX11 binaries)
  2. You need the D3D compiler to sign your binaries (DX will refuse to run unsigned binaries)
  3. It only works on windows
This presents a substantial problem for the web, we don't want to make "Windows only WebGL". We've fought long and hard to not end up exactly there. At one point Microsoft was noncomittal what GLSL format to support and also wanted to left the language wholly undefined. This would've harmed the web platform a lot.

Besides the above issues are also a considerable maintenance nightmare, as you now need to be aware against which backend you run your code, and which version of which backend, and cross-compile shaders for those, and supply those shaders, and so forth. Dragons live there.

OpenGL shader binary

The unsurmountable problem with this format is that it's GPU, driver and OS specific. You'd have to precompile your shaders for every conceivable combination of those. It's not intended to be used as a transport format, but rather a format to cache shaders on the local machine that that machine has compiled.


-----

Since both these options are extremely bad, that's why I said precompiled shaders (short of WebVulkan) are a complete nogo, for any developer, in any scenario, in WebGL.