[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] ESSL 3.5 Comments Compliance



One more thing. This really should go in the validator/translator and not be a separate step

Why?

Because removing all the UTF8 before complication seems impossible without real preprocessing. If we are allowing UTF8 then this is valid

#define OPTION "二番"

#if OPTION == "二番"
...
#else
...
#endif

Where as this is not

#define OPTION "二番"
// #define OPTION "foo"

uniform vec4 OPTION;


So the correct validation seems like it's

#1) Check that it's valid UTF8
#2) Preprocess
#3) Check the result is only the ES GLSL ASCII subset
#4) validate/translate
#5) call glShaderSource/glCompileShader


On Sat, Jan 15, 2011 at 2:28 PM, Gregg Tavares (wrk) <gman@google.com> wrote:


On Sat, Jan 15, 2011 at 2:23 PM, Gregg Tavares (wrk) <gman@google.com> wrote:
I don't see why shaderSource needs to care. Implementations have to hold on to the complete source when shaderSource is called since what is actually passed to GL is often something that's been translated but what is returned by getShaderSource has to be what the user passed it. Errors aren't normally reported until compileShader since they can't pass the shader to either the validator or GL until compileShader time.

shaderSource doesn't care what the contents of the string is. Only compileShader cares about that. 


Let me try to make this a little clearer. My implementation of shaderSource does not call glShaderSource, it just saves the string so there is no worry that the gl driver will barf on the contents of the string. My implementation of getShaderSource does not call glGetShaderSource it just returns the string the user previously passed in. My implementation of compileShader take the string it saved earlier, passes it to the validator/translator, passes that string to the glShaderSource, then calls glCompileShader.
 
So, shaderSource doesn't need to do any validation and arguably it shouldn't. I has no way to report meaningful errors like compileShader does.


ShaderSource should take a DOMString, GetShaderSource should return a DOMString.
CompileShader should compile the shader in accordance with the OpenGL ES 2.0 spec with the WebGL caveats on top and report errors. 

How an implementation handles that would seem to be implementation dependent. I'd expect most at CompileShader time to (1) convert DOMString to UTF8, (2) strip UTF8 comments (3) validate the shader is OpenGL ES 2.0 compliant with no extensions (4) send the shader to the real GL.  Step (2) can be skipped if you know your driver is happy with UTF8. On desktop you can't know that but you can on embedded systems.



On Sat, Jan 15, 2011 at 12:55 PM, Mo, Zhenyao <zhenyao@gmail.com> wrote:
It is one of the differences between WebGL and GLES.  See WebGL spec 6.18.

2011/1/15 Ben Vanik <ben.vanik@gmail.com>:
> shaderSource isn't supposed to do anything, though. That would be a
> big behavioral change and trip people up when they are trying to write
> error handling/etc.
> "The source code strings are not scanned or parsed at this time; they
> are simply copied into the specified shader object."
>
> --
> Ben Vanik
> http://www.noxa.org
>
>
>
> 2011/1/15 Mo, Zhenyao <zhenyao@gmail.com>:
>>
>> I believe the validation also has to happen above ANGLE translator, as
>> we should raise an error in shaderSource(), whereas ANGLE translator
>> is only called in compileShader().
>>
>> 2011/1/14 Kenneth Russell <kbr@google.com>:
>>>
>>> On Fri, Jan 14, 2011 at 7:04 PM, Vladimir Vukicevic
>>> <vladimir@mozilla.com> wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> On Thu, Jan 13, 2011 at 5:35 PM, Kenneth Russell <kbr@google.com>
>>>>> wrote:
>>>>> > I agree with you. With a little preprocessing at the browser layer
>>>>> > before handing down a const char* to the OpenGL layer (be it the
>>>>> > ANGLE
>>>>> > shader translator or OpenGL directly), a WebGL implementation should
>>>>> > be able to allow the entire Unicode character range to be used in
>>>>> > comments by stripping them completely. I don't think we need to
>>>>> > reference UTF-8, since DOMStrings come in in (not necessarily valid)
>>>>> > UTF-16. See http://dev.w3.org/2006/webapi/WebIDL/#idl-DOMString .
>>>>> >
>>>>> > Can you either propose some spec text or just edit the spec
>>>>> > directly?
>>>>>
>>>>> I've gone ahead and updated the WebGL spec to require that the full
>>>>> Unicode character set can be used in comments in shaders, and updated
>>>>> the glsl-conformance.html test to ensure both this and that non-ASCII
>>>>> characters in the actual shader text is rejected. Please send any
>>>>> comments to the list.
>>>>
>>>> Sounds great to me -- Ken, is the plan to have ANGLE just strip comments and verify that the remainder is ASCII (or the limited subset thereof)?  Or would we have to do that at a level before the shader translator?
>>>
>>> I believe the removal of comments must be done at a level above the
>>> shader translator. The browser takes in DOMStrings containing
>>> potentially invalid UTF-16 data, and before the conversion to either
>>> ASCII or UTF-8 it needs to strip the comments potentially containing
>>> non-ASCII characters. We should modify ANGLE to enforce that the
>>> incoming const char* is not only ASCII but also fully ESSL compliant,
>>> though in the WebKit implementation that is currently being done at
>>> the browser level.
>>>
>>> I'm about to upload a patch for
>>> https://bugs.webkit.org/show_bug.cgi?id=52390 implementing the removal
>>> of comments. The WebGL conformance suite has already been updated for
>>> the new rules.
>>>
>>> -Ken
>>>
>>>>    - Vlad
>>>>
>>>>> > On Thu, Jan 13, 2011 at 4:49 PM, Gregg Tavares (wrk)
>>>>> > <gman@google.com> wrote:
>>>>> >> I really want to emphasize that it would be really insensitive to
>>>>> >> make WebGL
>>>>> >> only support ASCII in comments in shaders. It's called WebGL
>>>>> >> because it's on
>>>>> >> the WEB and the WEB is not ASCII. I know the OpenGL spec says
>>>>> >> ASCII. I have
>>>>> >> a feeling though few if any drivers enforce that and like I
>>>>> >> mentioned
>>>>> >> before, even if they do enforce it WebGL implementations can work
>>>>> >> around it
>>>>> >> by stripping the comments.
>>>>> >> Imagine someone told you couldn't document your code in a language
>>>>> >> you
>>>>> >> understand. How would you feel? Imagine if you couldn't even write
>>>>> >> a blog
>>>>> >> post that showed sample code with inline comments or that if you
>>>>> >> put inline
>>>>> >> comments in users trying to copy and paste snippets from your
>>>>> >> tutorial had
>>>>> >> to manually remove all the inline comments.
>>>>> >> Seriously, it seems morally wrong in 2011 to force programmers the
>>>>> >> world
>>>>> >> over to ASCII only for their shader comments.
>>>>> >> Here's a shader from this page
>>>>> >> WebGL MQO Loader
>>>>> >> Are we really going to tell that guy he has to learn English if he
>>>>> >> wants to
>>>>> >> document his code?
>>>>> >> I think we should diverge from the OpenGL ES spec here in the
>>>>> >> interest of
>>>>> >> the needs of the Web and the world at large. Checking for valid
>>>>> >> UTF8 is easy
>>>>> >> if we want to validate input and workarounds past that are probably
>>>>> >> not
>>>>> >> needed and even if they are probably very easy. The ANGLE
>>>>> >> translator already
>>>>> >> strips comments as far as I know so just validating UTF8 and
>>>>> >> calling the
>>>>> >> translator is enough to allow WebGL to be an international
>>>>> >> developer
>>>>> >> friendly standard.
>>>>> >>
>>>>> >> ---------------------------
>>>>> >> #ifdef GL_ES
>>>>> >> precision highp float;
>>>>> >> #endif
>>>>> >> //VertexShader
>>>>> >> //入力パラメータ
>>>>> >> attribute vec3 aVertexPosition; //入力位置
>>>>> >> attribute vec3 aVertexNormal; //入力法線
>>>>> >> attribute vec3 aVertexTangent; //入力法線
>>>>> >> attribute vec3 aVertexBinormal; //入力法線
>>>>> >> attribute vec4 aVertexColor; //入力カラー
>>>>> >> attribute vec2 aTextureCoord; //入力UV
>>>>> >> //出力パラメータ
>>>>> >> varying vec4 vColor; //出力カラー
>>>>> >> varying vec2 vTextureCoord; //出力UV
>>>>> >> varying vec3 vNormal; //出力法線
>>>>> >> varying vec3 vTangent; //出力接線
>>>>> >> varying vec3 vBinormal; //出力従法線
>>>>> >> varying vec3 vLightDir; //出力ライト
>>>>> >> varying vec3 vEyeDir; //出力視線
>>>>> >> varying vec4 vShadowMap; //出力シャドウマップ位置
>>>>> >> //固定パラメータ
>>>>> >> uniform mat4 WorldMatrix; //ワールドマトリクス
>>>>> >> uniform mat4 ViewMatrix; //ビューマトリクス
>>>>> >> uniform mat4 ProjMatrix; //プロジェクションマトリクス
>>>>> >> uniform mat4 SMapLightMatrix; //シャドウマップライトマトリクス
>>>>> >> //
>>>>> >> uniform bool use_dif_texture; //ディフューズテクスチャ使用
>>>>> >> uniform bool use_alpha_texture; //アルファテクスチャ使用
>>>>> >> uniform bool use_celsdif_texture; //ディフューズセルシェーダテクスチャ使用
>>>>> >> uniform bool use_celsspc_texture; //スペキュラセルシェーダテクスチャ使用
>>>>> >> uniform bool use_reflec_texture; //リフレクションテクスチャ使用
>>>>> >> uniform bool use_bump_texture; //バンプテクスチャ使用
>>>>> >> uniform bool use_smap_texture; //シャドウマップテクスチャ使用
>>>>> >> uniform bool use_light; //ライティング使用
>>>>> >> uniform bool use_edge; //エッジ使用
>>>>> >> uniform bool draw_smap_flg; //シャドウマップ描画
>>>>> >> //
>>>>> >> uniform vec3 light_dir; //ライト方向
>>>>> >> uniform vec3 g_ViewDir; //エッジ色
>>>>> >> uniform float edge_scale; //エッジスケール
>>>>> >> mat3 transpose_mat3(mat3 mt){
>>>>> >> mat3 mt2;
>>>>> >> mt2[0][0]=mt[0][0];
>>>>> >> mt2[0][1]=mt[1][0];
>>>>> >> mt2[0][2]=mt[2][0];
>>>>> >> mt2[1][0]=mt[0][1];
>>>>> >> mt2[1][1]=mt[1][1];
>>>>> >> mt2[1][2]=mt[2][1];
>>>>> >> mt2[2][0]=mt[0][2];
>>>>> >> mt2[2][1]=mt[1][2];
>>>>> >> mt2[2][2]=mt[2][2];
>>>>> >> return mt2;
>>>>> >> }
>>>>> >> void main(void) {
>>>>> >> vec3 now_pos=aVertexPosition;
>>>>> >> if(use_edge){
>>>>> >> now_pos+=(aVertexNormal*edge_scale);
>>>>> >> }
>>>>> >> mat4 WorldMatrixView=ViewMatrix * WorldMatrix;
>>>>> >> vec3 dpos2 =(WorldMatrixView * vec4(now_pos,1.0)).xyz;
>>>>> >> gl_Position = ProjMatrix * vec4(dpos2,1.0);
>>>>> >> if(use_dif_texture || use_alpha_texture || use_bump_texture){
>>>>> >> vTextureCoord = aTextureCoord;
>>>>> >> }else{
>>>>> >> vTextureCoord = vec2(0.0,0.0);
>>>>> >> }
>>>>> >> if(draw_smap_flg){
>>>>> >> vShadowMap=vec4(gl_Position);
>>>>> >> }else{
>>>>> >> if(use_smap_texture){
>>>>> >> vShadowMap=SMapLightMatrix * vec4(dpos2,1.0);
>>>>> >> }
>>>>> >> }
>>>>> >> //
>>>>> >> vColor = aVertexColor;
>>>>> >> //
>>>>> >> mat3 nmt=mat3(WorldMatrix);
>>>>> >> vec3 norm=normalize(nmt*aVertexNormal);
>>>>> >> vNormal=norm;
>>>>> >> //
>>>>> >> if(use_bump_texture){
>>>>> >> vec3 tangent=normalize(nmt*aVertexTangent);
>>>>> >> vec3 binorm =normalize(nmt*aVertexBinormal);
>>>>> >> mat3 tan_mt=mat3(tangent,binorm,norm);
>>>>> >> // tan_mt=transpose(tan_mt);
>>>>> >> tan_mt=transpose_mat3(tan_mt);
>>>>> >> vTangent =tangent;
>>>>> >> vBinormal=binorm;
>>>>> >> vLightDir=tan_mt*light_dir;
>>>>> >> vEyeDir =tan_mt*g_ViewDir;
>>>>> >> }else{
>>>>> >> vLightDir=light_dir;
>>>>> >> vEyeDir =g_ViewDir;
>>>>> >> }
>>>>> >> }
>>>>> >>
>>>>> >
>>>>
>>> -----------------------------------------------------------
>>> You are currently subscribed to public_webgl@khronos.org.
>>> To unsubscribe, send an email to majordomo@khronos.org with
>>> the following command in the body of your email:
>>> unsubscribe public_webgl
>>> -----------------------------------------------------------
>>>
>>>
>> -----------------------------------------------------------
>> You are currently subscribed to public_webgl@khronos.org.
>> To unsubscribe, send an email to majordomo@khronos.org with
>> the following command in the body of your email:
>> unsubscribe public_webgl
>> -----------------------------------------------------------
>>
>>
>