[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] WEBGL_texture_from_depth_video extension proposal



On Wed, Nov 5, 2014 at 1:11 PM, Florian Bösch <pyalot@gmail.com> wrote:
I've seen a new extension pop up in the proposals: https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_texture_from_depth_video/

I'd like to take issue with that extension in 3 points.

1) Personally I'd like extension proposals be announced on the public ML. It keeps everybody in the loop. I can't speak for the WG of course since I'm not a member, but it'd be nice.

I apologize for that -- right after pushing the extension proposal there were two days of back-to-back meetings. The intent was certainly to announce the new extension proposal on the mailing list.

 

2) I'm not sure an extension is needed in this case, since basically it just covers uploading a particular surface format to a texture, with WebGL being largely oblivious of the underlying format. It would appear to me to be an extension a bit like trying to add one to make the browser do a YUV conversion of video to RGB, which it already does as part of the course. I agree that the behavior of the browser upon encountering a different video surface format for upload to either WebGL or Canvas2D should be specified, but I'm not sure an extension is the right place to do that. 

Additionally, a separate WebGL extension would imply a vendor who would implement WebGL, and the Media Capture Depth Stream Extensions, would somehow not implement using that depth stream with WebGL. This doesn't make much sense, since it's not a hardware limitation, and so it'd probably be good if whichever vendor implements WebGL and media capture depth stream would implement it fully from the get-go.


The reason to expose an extension from my point of view is (a) to allow detection of support for uploading depth videos to WebGL and (b) to provide a single, efficient upload path that users can rely on.

 
3) The style of how to support the upload isn't very friendly to users because it requires manual conversion in the fragment shader. I'd much rather see float/half-float texture formats being satisfied, where the browser knows about the surface format of the depth texture, and converts it (on GPU, without blocking operation) to the specified floating point format for upload. This'd have the advantage of offering users a transparent support for depth images, regardless of their underlying format.

Yes, with WebGL 1.0 the conversion is a little painful. This will become trivial with WebGL 2.0 and the availability of R16UI / R16I textures. I'd like to focus on getting WebGL 2.0 out the door, but we'd appreciate any concrete suggestions you have for the Media Capture folks about better data upload formats to use with WebGL 1.0.

-Ken