[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] ESSL 3.5 Comments Compliance



Ignoring of course the wrapping newlines that were inserted into my
#error lines...

David

2011/1/15 David Sheets <kosmo.zb@gmail.com>:
> The ESSL spec is fairly explicit (3.3 step 3) that preprocessing
> occurs after comment removal and preprocessing tokenization.
>
> ESSL Sec 3.4 states that character constants are not supported and
> #if/#elif only supports literal integers and identifiers (Sec. 3.8
> specs their charset) consumed by the 'defined' operator. My
> understanding is that because "#error" has syntactic meaning, it must
> obey the restricted ESSL charset. Sec 3.4 defines #error's behavior as
> emitting the /tokens/ following #error until the EOL.
>
> Preprocessor directives are part of the macro language and do not
> enjoy the same syntactic immunity as comments. The preprocessor
> operates on the output of the lexical tokens from phase 2. No lexical
> tokens are generated from comments in phase 2. Lexical tokens are
> generated from preprocessor directives in phase 2. Therefore, #error
> cannot contain "$@'` etc. without specifying a full secondary
> browser-side lexer and preprocessor.
>
> Additionally, because there is no string support in the executed
> language and preprocessing occurs after decommenting, tiptoeing around
> the #error directive seems unnecessary. If I write:
>
> #define HAT
> #ifdef HAT
> #error No hats allowed! /* Just kidding! I think your hat is
> "snazzy".*/ Hats disrespect Our Lady GLSL
> #endif
>
> I expect one of:
> "No hats allowed!   Hats disrespect Our Lady GLSL"
> "No hats allowed !   Hats disrespect Our Lady GLSL"
> "No hats allowed! Hats disrespect Our Lady GLSL"
> "No hats allowed ! Hats disrespect Our Lady GLSL"
>
> Depending on the tokenizer and token emitter.
>
> If I write:
>
> #define CAT
> #ifdef CAT
> #error Fat cats get all the $$$ /* Teddy should bust those #$@!ing
> trusts! */ and railroads
> #endif
>
> I expect a lexical token generation error on the "$$$" as these are
> not allowed in the ESSL charset.
>
> Neither the shading language nor the shading language's macro language
> supports strings. The comments in compilation units support
> consumption of arbitrary character strings and nothing more (no
> assignment, equality, iteration, inspection, manipulation, or output)
> and thus can handle Unicode and the rest.
>
> David
>
> 2011/1/15 Gregg Tavares (wrk) <gman@google.com>:
>> One more thing. This really should go in the validator/translator and not be
>> a separate step
>> Why?
>> Because removing all the UTF8 before complication seems impossible without
>> real preprocessing. If we are allowing UTF8 then this is valid
>>
>> #define OPTION "二番"
>> #if OPTION == "二番"
>> ...
>> #else
>> ...
>> #endif
>>
>> Where as this is not
>>
>> #define OPTION "二番"
>> // #define OPTION "foo"
>> uniform vec4 OPTION;
>>
>> So the correct validation seems like it's
>> #1) Check that it's valid UTF8
>> #2) Preprocess
>> #3) Check the result is only the ES GLSL ASCII subset
>> #4) validate/translate
>> #5) call glShaderSource/glCompileShader
>>
>> On Sat, Jan 15, 2011 at 2:28 PM, Gregg Tavares (wrk) <gman@google.com>
>> wrote:
>>>
>>>
>>> On Sat, Jan 15, 2011 at 2:23 PM, Gregg Tavares (wrk) <gman@google.com>
>>> wrote:
>>>>
>>>> I don't see why shaderSource needs to care. Implementations have to hold
>>>> on to the complete source when shaderSource is called since what is actually
>>>> passed to GL is often something that's been translated but what is returned
>>>> by getShaderSource has to be what the user passed it. Errors aren't normally
>>>> reported until compileShader since they can't pass the shader to either the
>>>> validator or GL until compileShader time.
>>>> shaderSource doesn't care what the contents of the string is. Only
>>>> compileShader cares about that.
>>>
>>> Let me try to make this a little clearer. My implementation of
>>> shaderSource does not call glShaderSource, it just saves the string so there
>>> is no worry that the gl driver will barf on the contents of the string. My
>>> implementation of getShaderSource does not call glGetShaderSource it just
>>> returns the string the user previously passed in. My implementation of
>>> compileShader take the string it saved earlier, passes it to the
>>> validator/translator, passes that string to the glShaderSource, then calls
>>> glCompileShader.
>>>
>>> So, shaderSource doesn't need to do any validation and arguably it
>>> shouldn't. I has no way to report meaningful errors like compileShader does.
>>>>
>>>> ShaderSource should take a DOMString, GetShaderSource should return a
>>>> DOMString.
>>>> CompileShader should compile the shader in accordance with the OpenGL ES
>>>> 2.0 spec with the WebGL caveats on top and report errors.
>>>> How an implementation handles that would seem to be implementation
>>>> dependent. I'd expect most at CompileShader time to (1) convert DOMString to
>>>> UTF8, (2) strip UTF8 comments (3) validate the shader is OpenGL ES 2.0
>>>> compliant with no extensions (4) send the shader to the real GL.  Step (2)
>>>> can be skipped if you know your driver is happy with UTF8. On desktop you
>>>> can't know that but you can on embedded systems.
>>>>
>>>>
>>>> On Sat, Jan 15, 2011 at 12:55 PM, Mo, Zhenyao <zhenyao@gmail.com> wrote:
>>>>>
>>>>> It is one of the differences between WebGL and GLES.  See WebGL spec
>>>>> 6.18.
>>>>>
>>>>> 2011/1/15 Ben Vanik <ben.vanik@gmail.com>:
>>>>> > shaderSource isn't supposed to do anything, though. That would be a
>>>>> > big behavioral change and trip people up when they are trying to write
>>>>> > error handling/etc.
>>>>> > "The source code strings are not scanned or parsed at this time; they
>>>>> > are simply copied into the specified shader object."
>>>>> >
>>>>> > --
>>>>> > Ben Vanik
>>>>> > http://www.noxa.org
>>>>> >
>>>>> >
>>>>> >
>>>>> > 2011/1/15 Mo, Zhenyao <zhenyao@gmail.com>:
>>>>> >>
>>>>> >> I believe the validation also has to happen above ANGLE translator,
>>>>> >> as
>>>>> >> we should raise an error in shaderSource(), whereas ANGLE translator
>>>>> >> is only called in compileShader().
>>>>> >>
>>>>> >> 2011/1/14 Kenneth Russell <kbr@google.com>:
>>>>> >>>
>>>>> >>> On Fri, Jan 14, 2011 at 7:04 PM, Vladimir Vukicevic
>>>>> >>> <vladimir@mozilla.com> wrote:
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> ----- Original Message -----
>>>>> >>>>> On Thu, Jan 13, 2011 at 5:35 PM, Kenneth Russell <kbr@google.com>
>>>>> >>>>> wrote:
>>>>> >>>>> > I agree with you. With a little preprocessing at the browser
>>>>> >>>>> > layer
>>>>> >>>>> > before handing down a const char* to the OpenGL layer (be it the
>>>>> >>>>> > ANGLE
>>>>> >>>>> > shader translator or OpenGL directly), a WebGL implementation
>>>>> >>>>> > should
>>>>> >>>>> > be able to allow the entire Unicode character range to be used
>>>>> >>>>> > in
>>>>> >>>>> > comments by stripping them completely. I don't think we need to
>>>>> >>>>> > reference UTF-8, since DOMStrings come in in (not necessarily
>>>>> >>>>> > valid)
>>>>> >>>>> > UTF-16. See http://dev.w3.org/2006/webapi/WebIDL/#idl-DOMString
>>>>> >>>>> > .
>>>>> >>>>> >
>>>>> >>>>> > Can you either propose some spec text or just edit the spec
>>>>> >>>>> > directly?
>>>>> >>>>>
>>>>> >>>>> I've gone ahead and updated the WebGL spec to require that the
>>>>> >>>>> full
>>>>> >>>>> Unicode character set can be used in comments in shaders, and
>>>>> >>>>> updated
>>>>> >>>>> the glsl-conformance.html test to ensure both this and that
>>>>> >>>>> non-ASCII
>>>>> >>>>> characters in the actual shader text is rejected. Please send any
>>>>> >>>>> comments to the list.
>>>>> >>>>
>>>>> >>>> Sounds great to me -- Ken, is the plan to have ANGLE just strip
>>>>> >>>> comments and verify that the remainder is ASCII (or the limited subset
>>>>> >>>> thereof)?  Or would we have to do that at a level before the shader
>>>>> >>>> translator?
>>>>> >>>
>>>>> >>> I believe the removal of comments must be done at a level above the
>>>>> >>> shader translator. The browser takes in DOMStrings containing
>>>>> >>> potentially invalid UTF-16 data, and before the conversion to either
>>>>> >>> ASCII or UTF-8 it needs to strip the comments potentially containing
>>>>> >>> non-ASCII characters. We should modify ANGLE to enforce that the
>>>>> >>> incoming const char* is not only ASCII but also fully ESSL
>>>>> >>> compliant,
>>>>> >>> though in the WebKit implementation that is currently being done at
>>>>> >>> the browser level.
>>>>> >>>
>>>>> >>> I'm about to upload a patch for
>>>>> >>> https://bugs.webkit.org/show_bug.cgi?id=52390 implementing the
>>>>> >>> removal
>>>>> >>> of comments. The WebGL conformance suite has already been updated
>>>>> >>> for
>>>>> >>> the new rules.
>>>>> >>>
>>>>> >>> -Ken
>>>>> >>>
>>>>> >>>>    - Vlad
>>>>> >>>>
>>>>> >>>>> > On Thu, Jan 13, 2011 at 4:49 PM, Gregg Tavares (wrk)
>>>>> >>>>> > <gman@google.com> wrote:
>>>>> >>>>> >> I really want to emphasize that it would be really insensitive
>>>>> >>>>> >> to
>>>>> >>>>> >> make WebGL
>>>>> >>>>> >> only support ASCII in comments in shaders. It's called WebGL
>>>>> >>>>> >> because it's on
>>>>> >>>>> >> the WEB and the WEB is not ASCII. I know the OpenGL spec says
>>>>> >>>>> >> ASCII. I have
>>>>> >>>>> >> a feeling though few if any drivers enforce that and like I
>>>>> >>>>> >> mentioned
>>>>> >>>>> >> before, even if they do enforce it WebGL implementations can
>>>>> >>>>> >> work
>>>>> >>>>> >> around it
>>>>> >>>>> >> by stripping the comments.
>>>>> >>>>> >> Imagine someone told you couldn't document your code in a
>>>>> >>>>> >> language
>>>>> >>>>> >> you
>>>>> >>>>> >> understand. How would you feel? Imagine if you couldn't even
>>>>> >>>>> >> write
>>>>> >>>>> >> a blog
>>>>> >>>>> >> post that showed sample code with inline comments or that if
>>>>> >>>>> >> you
>>>>> >>>>> >> put inline
>>>>> >>>>> >> comments in users trying to copy and paste snippets from your
>>>>> >>>>> >> tutorial had
>>>>> >>>>> >> to manually remove all the inline comments.
>>>>> >>>>> >> Seriously, it seems morally wrong in 2011 to force programmers
>>>>> >>>>> >> the
>>>>> >>>>> >> world
>>>>> >>>>> >> over to ASCII only for their shader comments.
>>>>> >>>>> >> Here's a shader from this page
>>>>> >>>>> >> WebGL MQO Loader
>>>>> >>>>> >> Are we really going to tell that guy he has to learn English if
>>>>> >>>>> >> he
>>>>> >>>>> >> wants to
>>>>> >>>>> >> document his code?
>>>>> >>>>> >> I think we should diverge from the OpenGL ES spec here in the
>>>>> >>>>> >> interest of
>>>>> >>>>> >> the needs of the Web and the world at large. Checking for valid
>>>>> >>>>> >> UTF8 is easy
>>>>> >>>>> >> if we want to validate input and workarounds past that are
>>>>> >>>>> >> probably
>>>>> >>>>> >> not
>>>>> >>>>> >> needed and even if they are probably very easy. The ANGLE
>>>>> >>>>> >> translator already
>>>>> >>>>> >> strips comments as far as I know so just validating UTF8 and
>>>>> >>>>> >> calling the
>>>>> >>>>> >> translator is enough to allow WebGL to be an international
>>>>> >>>>> >> developer
>>>>> >>>>> >> friendly standard.
>>>>> >>>>> >>
>>>>> >>>>> >> ---------------------------
>>>>> >>>>> >> #ifdef GL_ES
>>>>> >>>>> >> precision highp float;
>>>>> >>>>> >> #endif
>>>>> >>>>> >> //VertexShader
>>>>> >>>>> >> //入力パラメータ
>>>>> >>>>> >> attribute vec3 aVertexPosition; //入力位置
>>>>> >>>>> >> attribute vec3 aVertexNormal; //入力法線
>>>>> >>>>> >> attribute vec3 aVertexTangent; //入力法線
>>>>> >>>>> >> attribute vec3 aVertexBinormal; //入力法線
>>>>> >>>>> >> attribute vec4 aVertexColor; //入力カラー
>>>>> >>>>> >> attribute vec2 aTextureCoord; //入力UV
>>>>> >>>>> >> //出力パラメータ
>>>>> >>>>> >> varying vec4 vColor; //出力カラー
>>>>> >>>>> >> varying vec2 vTextureCoord; //出力UV
>>>>> >>>>> >> varying vec3 vNormal; //出力法線
>>>>> >>>>> >> varying vec3 vTangent; //出力接線
>>>>> >>>>> >> varying vec3 vBinormal; //出力従法線
>>>>> >>>>> >> varying vec3 vLightDir; //出力ライト
>>>>> >>>>> >> varying vec3 vEyeDir; //出力視線
>>>>> >>>>> >> varying vec4 vShadowMap; //出力シャドウマップ位置
>>>>> >>>>> >> //固定パラメータ
>>>>> >>>>> >> uniform mat4 WorldMatrix; //ワールドマトリクス
>>>>> >>>>> >> uniform mat4 ViewMatrix; //ビューマトリクス
>>>>> >>>>> >> uniform mat4 ProjMatrix; //プロジェクションマトリクス
>>>>> >>>>> >> uniform mat4 SMapLightMatrix; //シャドウマップライトマトリクス
>>>>> >>>>> >> //
>>>>> >>>>> >> uniform bool use_dif_texture; //ディフューズテクスチャ使用
>>>>> >>>>> >> uniform bool use_alpha_texture; //アルファテクスチャ使用
>>>>> >>>>> >> uniform bool use_celsdif_texture; //ディフューズセルシェーダテクスチャ使用
>>>>> >>>>> >> uniform bool use_celsspc_texture; //スペキュラセルシェーダテクスチャ使用
>>>>> >>>>> >> uniform bool use_reflec_texture; //リフレクションテクスチャ使用
>>>>> >>>>> >> uniform bool use_bump_texture; //バンプテクスチャ使用
>>>>> >>>>> >> uniform bool use_smap_texture; //シャドウマップテクスチャ使用
>>>>> >>>>> >> uniform bool use_light; //ライティング使用
>>>>> >>>>> >> uniform bool use_edge; //エッジ使用
>>>>> >>>>> >> uniform bool draw_smap_flg; //シャドウマップ描画
>>>>> >>>>> >> //
>>>>> >>>>> >> uniform vec3 light_dir; //ライト方向
>>>>> >>>>> >> uniform vec3 g_ViewDir; //エッジ色
>>>>> >>>>> >> uniform float edge_scale; //エッジスケール
>>>>> >>>>> >> mat3 transpose_mat3(mat3 mt){
>>>>> >>>>> >> mat3 mt2;
>>>>> >>>>> >> mt2[0][0]=mt[0][0];
>>>>> >>>>> >> mt2[0][1]=mt[1][0];
>>>>> >>>>> >> mt2[0][2]=mt[2][0];
>>>>> >>>>> >> mt2[1][0]=mt[0][1];
>>>>> >>>>> >> mt2[1][1]=mt[1][1];
>>>>> >>>>> >> mt2[1][2]=mt[2][1];
>>>>> >>>>> >> mt2[2][0]=mt[0][2];
>>>>> >>>>> >> mt2[2][1]=mt[1][2];
>>>>> >>>>> >> mt2[2][2]=mt[2][2];
>>>>> >>>>> >> return mt2;
>>>>> >>>>> >> }
>>>>> >>>>> >> void main(void) {
>>>>> >>>>> >> vec3 now_pos=aVertexPosition;
>>>>> >>>>> >> if(use_edge){
>>>>> >>>>> >> now_pos+=(aVertexNormal*edge_scale);
>>>>> >>>>> >> }
>>>>> >>>>> >> mat4 WorldMatrixView=ViewMatrix * WorldMatrix;
>>>>> >>>>> >> vec3 dpos2 =(WorldMatrixView * vec4(now_pos,1.0)).xyz;
>>>>> >>>>> >> gl_Position = ProjMatrix * vec4(dpos2,1.0);
>>>>> >>>>> >> if(use_dif_texture || use_alpha_texture || use_bump_texture){
>>>>> >>>>> >> vTextureCoord = aTextureCoord;
>>>>> >>>>> >> }else{
>>>>> >>>>> >> vTextureCoord = vec2(0.0,0.0);
>>>>> >>>>> >> }
>>>>> >>>>> >> if(draw_smap_flg){
>>>>> >>>>> >> vShadowMap=vec4(gl_Position);
>>>>> >>>>> >> }else{
>>>>> >>>>> >> if(use_smap_texture){
>>>>> >>>>> >> vShadowMap=SMapLightMatrix * vec4(dpos2,1.0);
>>>>> >>>>> >> }
>>>>> >>>>> >> }
>>>>> >>>>> >> //
>>>>> >>>>> >> vColor = aVertexColor;
>>>>> >>>>> >> //
>>>>> >>>>> >> mat3 nmt=mat3(WorldMatrix);
>>>>> >>>>> >> vec3 norm=normalize(nmt*aVertexNormal);
>>>>> >>>>> >> vNormal=norm;
>>>>> >>>>> >> //
>>>>> >>>>> >> if(use_bump_texture){
>>>>> >>>>> >> vec3 tangent=normalize(nmt*aVertexTangent);
>>>>> >>>>> >> vec3 binorm =normalize(nmt*aVertexBinormal);
>>>>> >>>>> >> mat3 tan_mt=mat3(tangent,binorm,norm);
>>>>> >>>>> >> // tan_mt=transpose(tan_mt);
>>>>> >>>>> >> tan_mt=transpose_mat3(tan_mt);
>>>>> >>>>> >> vTangent =tangent;
>>>>> >>>>> >> vBinormal=binorm;
>>>>> >>>>> >> vLightDir=tan_mt*light_dir;
>>>>> >>>>> >> vEyeDir =tan_mt*g_ViewDir;
>>>>> >>>>> >> }else{
>>>>> >>>>> >> vLightDir=light_dir;
>>>>> >>>>> >> vEyeDir =g_ViewDir;
>>>>> >>>>> >> }
>>>>> >>>>> >> }
>>>>> >>>>> >>
>>>>> >>>>> >
>>>>> >>>>
>>>>> >>> -----------------------------------------------------------
>>>>> >>> You are currently subscribed to public_webgl@khronos.org.
>>>>> >>> To unsubscribe, send an email to majordomo@khronos.org with
>>>>> >>> the following command in the body of your email:
>>>>> >>> unsubscribe public_webgl
>>>>> >>> -----------------------------------------------------------
>>>>> >>>
>>>>> >>>
>>>>> >> -----------------------------------------------------------
>>>>> >> You are currently subscribed to public_webgl@khronos.org.
>>>>> >> To unsubscribe, send an email to majordomo@khronos.org with
>>>>> >> the following command in the body of your email:
>>>>> >> unsubscribe public_webgl
>>>>> >> -----------------------------------------------------------
>>>>> >>
>>>>> >>
>>>>> >
>>>>
>>>
>>
>>
>
-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------