[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] WEBGL_texture_from_depth_video extension proposal

Hi Ken/Florian,

yep I think you make a really good point here.

There's some potential issues under the hood that we're looking into and we're also exploring a different option that's come out of our discussions with another WG.

So lets put this on hold and we'll get back to you as soon as we have a consolidated response.

Thanks again for all your time and constructive feedback.


On 13/11/14 9:07 AM, Kenneth Russell wrote:

Rob, Ningxin (in particular), all,

After more thought and offline chat with Florian, it does seem
unfortunate to introduce a WebGL extension that will have to be
supported forever, just to handle a potential brief time period when
the Media Capture Depth Stream Extensions are supported, but the WebGL
texture upload path isn't. (Hopefully, both pieces of support will
land at the same time).

Per Florian's suggestion, what if we simply add a new section 5.7
"WebGLRenderingContext Interface" to
http://w3c.github.io/mediacapture-depth/ stating:

  - An HTMLVideoElement containing a depth stream may be uploaded to a
WebGL texture with the same bit depth as the depth stream.
  -- In WebGL 1.0 [1], this includes RGB/UNSIGNED_SHORT_5_6_5 and
RGBA/UNSIGNED_SHORT_4_4_4_4 textures.
  -- In WebGL 2.0 [2], this includes RED_INTEGER/SHORT/R16I and

Then include the code snippet from WEBGL_texture_from_depth_video,
stating clearly that this is the code path for WebGL 1.0, and that
WebGL 2.0 will avoid the unpacking of the red, green and blue

[1] https://www.khronos.org/registry/webgl/specs/1.0/ (or
https://www.khronos.org/registry/webgl/specs/latest/1.0/ , your
[2] https://www.khronos.org/registry/webgl/specs/latest/2.0/

Then the WEBGL_texture_from_depth_video extension proposal isn't
needed. If the Media Capture Depth Stream extensions are supported at
all by the browser, then uploading depth streams to WebGL will be
implicitly supported.

What do you think?



On Wed, Nov 12, 2014 at 3:24 AM, Florian Bösch <pyalot@gmail.com> wrote:
I'd really rather not have an extension, and lengthy discussion on the
nonsensical nature of it for something that's essentially a few lines of
code ala pseudocode:

if obj.isVideo
   if internalFormat.bitDepth == video.bitDepth
     texImage2D obj.bytes
    throw "bit depths do not match"

Not supporting the video formats, that the browser implements isn't a
"capability". It's a bug.

On Wed, Nov 12, 2014 at 11:55 AM, Florian Bösch <pyalot@gmail.com> wrote:

In the right thread again, a bit more structured. I don't like this being an extension, I don't think it's an extension to begin with. It's basically just saying "a 16-bpp texture can be filled from an array of bytes with the matching size", and then tacks on a tutorial on how to interprete those bytes in a shader. It's an "extension" that regulates user behavior and external image formats, that's got nothing todo with WebGL to begin with.

I'd propse adding a very simple change to the WebGL section of

The per-pixel bit depth of a depth video stream is 16-bit. It can be
uploaded to any WebGL texture of matching per pixel bit depth. The depth is
an unsigned short value and it can be interpreted in a shader as follows...

On Wed, Nov 12, 2014 at 1:26 AM, Jeff Gilbert <jgilbert@mozilla.com>

It seems like it would already require a bunch of code, to me. I'm not sure how using packed 565 instead of something else requires less code.

The current proposal is an ugly hack, and I'd like to be sure there's no
better way before considering implementing it.
"We already have this working in prototype" is not a convincing argument
for the quality of a spec.

If all of the target devices support depth textures, but not GLES3, the
spec should use depth textures.

In GLES3, we get R16UI, though it isn't normalized. It has the same
capabilities as DEPTH_COMPONENT16, anyways. (sampleable and renderable, but
not filterable) It seems like you'd want normalization for something like
this, but maybe not? If renderability isn't important, R32F could give
normalization with sufficient precision.

Regardless of all of this, like GL extensions do, it would be better if
this extension proposal had a brief discussion of reasoning for why it makes
the tradeoffs it does.


----- Original Message -----
From: "Rob Manson" <roBman@buildAR.com>
To: "Kenneth Russell" <kbr@google.com>
Cc: "Daniel Koch" <dkoch@nvidia.com>, "Jeff Gilbert"
<jgilbert@mozilla.com>, "Florian Bösch" <pyalot@gmail.com>, "public webgl"
<public_webgl@khronos.org>, "Ningxin Hu" <ningxin.hu@intel.com>
Sent: Tuesday, November 11, 2014 4:11:06 PM
Subject: Re: [Public WebGL] WEBGL_texture_from_depth_video extension

+1 for this and to us not creating more work for implementers.

On 12/11/14 10:46 AM, Kenneth Russell wrote:
I still think it's reasonable to limit depth camera texture uploads to
just 5_6_5 textures in WebGL 1.0. WebGL 2.0 implementations are
actively being developed and spending excessive time on the 1.0 code
paths will just delay their release. Intel's already implemented the
depth stream upload path to 5_6_5 textures and shown some demos
publicly, so integrating this support will take very little time.

----------------------------------------------------------- You are currently subscribed to public_webgl@khronos.org. To unsubscribe, send an email to majordomo@khronos.org with the following command in the body of your email: unsubscribe public_webgl -----------------------------------------------------------

----------------------------------------------------------- You are currently subscribed to public_webgl@khronos.org. To unsubscribe, send an email to majordomo@khronos.org with the following command in the body of your email: unsubscribe public_webgl -----------------------------------------------------------