Below is some feedback from Acorn Pooley originally posted to the
working group mailing list.|
Feedback on the WEBGL_dynamic_texture extension: GENERAL FEEDBACK: The extenstion is written based on the semantics of EGLImages. However there are a number of problems with the EGLImage semantics which EGLStream has addressed. I think it would benefit WebGL to adopt the EGLStream semantics rather than the EGLImage semantics. The WebGL extension could still be *implemented* using EGLImages (or even regular 2D textures using glTexImage2D), with the browser implementing the more EGLStream-like semantics. An example is the use of a texture that is acquired. With EGLImage semantics (and current WEBGL_dynamic_texture semantics) the texture can be modified asynchronously at any time while it is being used (this results in implementation dependent behavior which will vary from one implementation to another). With EGLStream semantics the acquired image can never be modified while it is acquired. This works because the EGLStream images are actually double (or triple or more) buffered. The producer is only allowed to modify images which are not currently acquired. This is easy to implement in the browser using EGLImages -- the browser just needs to create 2 or more EGLImages and implement the double buffering itself. This is especially critical for video sources where it makes no sense to allow the decoder to asynchronously modify the texture that is being used for rendering. But the EGLStream multi-buffered semantics would also be much easier to use and less error prone (and more similar between implementations) when applied to image and canvas sources. Another example is the use of a dynamic texture after dynamicTextureReleaseImage has been called. EGLStream semantics require that a TEXTURE_EXTERNAL texture that has no image acuired be treated as incomplete -- sampling it always returns black (0,0,0,1) pixels. The semantics of EGLImage which are in the current extension wording are basically "undefined results" which will make life difficult for developers and make content appear differently on different implementations. I have noted below in "SPECIFIC FEEDBACK" the details. Another area that much thought has gone into is how TEXTURE_EXTERNAL should work when the source is a video which is playing at a different rate than the WebGL rendering loop. EGLStream addresses this by stating that "Acquire" gets the image which should be displayed next. If the video is playing back very quickly relative to the rendering then this may result in frames being skipped (which is good because the user sees the frame they should be seeing "now" which is synced to the audio they are hearing "now"). If the video is playing back more slowly then 2 or more calls to Acquire may return the same frame (also good since again the user sees what they should be seeing "now"). The current WEBGL_dynamic_texture extension does not preclude this behavior, but in fact does not address it at all. I think it is a good idea to require it to work this way. Otherwise there is too much flexibility and different WebGL implementations may act differently. (There is also the GL_KHR_stream_fifo extension which allows the app to opt in to different semantics (never drop frames) but that is probably less useful -- you can decide if you need that option or not.) SPECIFIC FEEDBACK: - In EGL_KHR_stream_consumer_gltexture the acquire/release functions work like this: Before dynamicTextureAcquireImage is called the texture is incomplete - this is well defined and acts like a texture with all black pixels (i.e. every pixel is 0,0,0,1). After dynamicTextureReleaseImage is called the texture is again incomplete. Calling dynamicTextureReleaseImage is not required. A call to dynamicTextureAcquireImage implicitly first calls dynamicTextureReleaseImage if it has not been called since the last acquire. While an image is acquired (i.e after a successful call to dynamicTextureAcquireImage and until a call to dynamicTextureReleaseImage) the source is free to continue producing new frames. These changes do not affect the acquired image. Basically the producer is modifying a different buffer than the one that the texture is using. Having an image acquired has no affect on the producer. This is better defined and less error prone than the current WEBGL_dynamic_texture wording. I think it would benefit WebGL to work a little more like EGL_KHR_stream_consumer_gltexture. - In section 126.96.36.199 it says Dynamic textures are texture objects whose images change outside the control of the WebGL application and asynchronously with its rendering. I think this is a problem. The textures can be *acquired* from objects whose images chage outside and asynchronously. But the image displayed by the texture itself should only updated when the dynamicTextureAcquireImage() function is called. I think this is very important. Perhaps change the opening sentences to Dynamic textures are texture objects whose images can be acquired from sources outside the WebGL application. The image source changes outside the control of the WebGL application and asynchronously with its rendering, but acquisition of the image into the WebGL texture is synchronous. - In section 188.8.131.52 Dynamic textures: the description of the dynamicTextureAcquireImage() function states that it locks the image so it cannot change. Really the operation that we want to occur is that the image gets latched. The source of the images can continue to create new frames of image data and is not really affected by the acquire call. (Implementation note: when implemented with EGLImages this is accomplished by the browser creating 2 EGLImages and double buffering. When implemented as glTexImage2D it is even easier since each acquire corresponds to a call to glTex[Sub]Image2D.) - In section 184.108.40.206 Dynamic textures: the description of dynamicTextureReleaseImage says that the image may be modified asynchronously while being sampled. I think this is a bad idea. It will lead to differences in behavior between different implementations. The https://cvs.khronos.org/svn/repos/registry/trunk/public/egl/extensions/KHR/EGL_KHR_stream_consumer_gltexture.txt extension solves this by making the texture incomplete at times that it is not acquired. In other words, when no acquired sampling the texture always returns well defined black pixels. I think this is a better choice of semantics. - In section 220.127.116.11 Dynamic textures: It says If a texture object to which a dynamic source is bound is bound to a texture target other than TEXTURE_EXTERNAL the dynamic source will be ignored. Data will be sampled from the texture object's regular data store. I think it is a mistake to allow the same texture to be bound at one time to a TEXTURE_EXTERNAL target and at another time to a different target. In GLES2 this is an error (INVALID_OPERATION). Once a texture is bound to a TEXTURE_EXTERNAL target it may never be bound to any other target. Once a texture is bound to a target other than TEXTURE_EXTERNAL it may never be bound to the TEXTURE_EXTERNAL target. (Maybe it is not possible to have the same semantics in WebGL, but if it is possible I think this is preferable.) - Note that the GL_OES_EGL_image_external extension describes how the GL_TEXTURE_EXTERNAL_KHR texture target works with EGLImages. It is the GL_NV_EGL_stream_consumer_external extension which describes how GL_TEXTURE_EXTERNAL_KHR works with EGLStreams. It might make more sense to mention GL_NV_EGL_stream_consumer_external instead of (or in addition to) the GL_OES_EGL_image_external extension. The two extensions are very similar so this may not really matter. Actually it is the EGL_KHR_stream_consumer_gltexture which really describes how EGLStreams work with TEXTURE_EXTERNAL. It might be worth referring to this extension and/or borrowing some of the language about Acquire/Release functions. Thanks -Acorn
注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情報の使用を固く禁じております。エラー、手違いでこのメールを受け取られましたら削除を行い配信者にご連絡をお願いいたし ます。
NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies.