[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] Support precompiled shaders as extensions

>every platform can implement those extensions
Well, not really.

>iOS: 12%
So we have WEBGL_compressed_texture_pvrtc just for these 12%. How 37% Win is less important?

On 15 November 2016 at 00:44, Florian Bösch <pyalot@gmail.com> wrote:
Every extension that exists in WebGL has been carefully vetted for cross-platform compatibility. It's not that every platform implements every extension to the same extent, but that every platform can implement those extensions (or are on the way to do it).

DX bytecode would be D3D backend only, and could never be supported by the majority of visits that do not use D3D backends. Windows is a minority platform on the web:
  • Windows: 37%
  • Non Windows: 63%
    • Android: 34%
    • iOS: 12%
    • OSX: 5%
    • Other: 12%
That makes DX bytecode an extremely bad fit for WebGL, and it should not be exposed, even if there where no other technical difficulties.

DX bytecode should also not be exposed because it would force WebGL authors to supply multiple different shader formats, which is widely recognized to be be a nightmare. That's why Vulkan did SPIR-V, a non platform dependent non hardware dependent bytecode.

If anything of the sort of bytecode to shortcut shader compile times is introduced, it should fulfill the following criteria:
  1. Every Platform (iOS, Android, Linux, OSX, Windows, etc.) should be able to implement it
  2. Authors should be able to supply a single platform, GPU, driver and backend independent precompiled shader (intermediary bytecode)
  3. The compile toolchain should not be platform specific (an extraordinary amount of Web authors do not use windows)
At the present time, this would only make ANGLE translation faster, it would not solve: HLSL -> d3d bytecode, GLSL -> GPU. Thus those would need to be addressed, I've pointed that out at the beginning of the thread.

On Mon, Nov 14, 2016 at 5:33 PM, Corentin Wallez <cwallez@google.com> wrote:
On Mon, Nov 14, 2016 at 11:04 AM, Mr F <arthur@playcanvas.com> wrote:
>Any sort of precompiled binary fed directly to the driver would bypass that, which means your shader would randomly break. 
Somehow people ship games (at least on D3D) with precompiled shaders, and I haven't seen too many problems with that (OK, I did in about 2007, but today things are way better).

Desktop game developers often have contacts at IHVs and can afford to test on multiple hardware configurations. This is not the case for WebGL developers which need WebGL to be portable so they can write on a platform and expect things to work fine on other platforms.

You might argue that PlayCanvas can afford to test like game developers do but:
  • The web can't expose a non-portable API. If a WebGL extension is present, then it needs to always work and not require extensive testing
  • New drivers and hardware will eventually come out that will break your precompiled shaders
  • Feeding DXIL directly to the driver is a big security hole
Here are some of the D3D-specific workarounds in ANGLE.

Also ANGLE does dynamic compilation of D3D shaders to emulate some OpenGL ES behaviors and giving DXIL would prevent that.

For all of the reasons above, a precompiled shader extension can't happen. We should instead look at tackling the shader compilation problem in other ways. How about having a glFenceSync for GL_CPU_COMMANDS_BIT so that you can know when your shader has been compiled? Can we make shader compilation parallel? etc.

> Also the shader compile overhead of browsers is tiny compared to the driver compiling to hardware ISA
I do believe that translation is fast, but in my experience the actual hlsl->d3d bytecode part is the slowest, much slower than bytecode->hardware-specific. All the loop unrolling, function inlining, register counting and a load of optimizations happen here. And we'd better leave these optimizations enabled.

On 14 November 2016 at 18:49, Corentin Wallez <cwallez@google.com> wrote:
Also don't forget about the workarounds (35ish and counting) that are done in the ANGLE shader translator. Any sort of precompiled binary fed directly to the driver would bypass that, which means your shader would randomly break. Also the shader compile overhead of browsers is tiny compared to the driver compiling to hardware ISA, so what you need is more like asynchronous compiles.

Finally security-wise browsers will never trust drivers gracefully failing when given bad output, they want to own the frontend and handle errors there.

On Mon, Nov 14, 2016 at 10:03 AM, Florian Bösch <pyalot@gmail.com> wrote:
On Mon, Nov 14, 2016 at 3:11 PM, Maksims Mihejevs <max@playcanvas.com> wrote:
It does not feel that binary shaders is a complex feature. All background workings of WebGL already has functionality to deal with it I'm sure. It is only about exposing such functionality using extensions notation based on platforms.

Precompiled shaders are quite a huge problem. There's two main scenarios to consider:

D3D bytecode shaders

Supplying your shaders are precompiled D3D bytecode has a number of problems, these are:
  1. Depends on the D3D version (DX9 binaries are different than DX11 binaries)
  2. You need the D3D compiler to sign your binaries (DX will refuse to run unsigned binaries)
  3. It only works on windows
This presents a substantial problem for the web, we don't want to make "Windows only WebGL". We've fought long and hard to not end up exactly there. At one point Microsoft was noncomittal what GLSL format to support and also wanted to left the language wholly undefined. This would've harmed the web platform a lot.

Besides the above issues are also a considerable maintenance nightmare, as you now need to be aware against which backend you run your code, and which version of which backend, and cross-compile shaders for those, and supply those shaders, and so forth. Dragons live there.

OpenGL shader binary

The unsurmountable problem with this format is that it's GPU, driver and OS specific. You'd have to precompile your shaders for every conceivable combination of those. It's not intended to be used as a transport format, but rather a format to cache shaders on the local machine that that machine has compiled.


Since both these options are extremely bad, that's why I said precompiled shaders (short of WebVulkan) are a complete nogo, for any developer, in any scenario, in WebGL.