[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Public WebGL] WEBGL_texture_from_depth_video extension proposal

I've seen a new extension pop up in the proposals: https://www.khronos.org/registry/webgl/extensions/proposals/WEBGL_texture_from_depth_video/

I'd like to take issue with that extension in 3 points.

1) Personally I'd like extension proposals be announced on the public ML. It keeps everybody in the loop. I can't speak for the WG of course since I'm not a member, but it'd be nice.

2) I'm not sure an extension is needed in this case, since basically it just covers uploading a particular surface format to a texture, with WebGL being largely oblivious of the underlying format. It would appear to me to be an extension a bit like trying to add one to make the browser do a YUV conversion of video to RGB, which it already does as part of the course. I agree that the behavior of the browser upon encountering a different video surface format for upload to either WebGL or Canvas2D should be specified, but I'm not sure an extension is the right place to do that. 

Additionally, a separate WebGL extension would imply a vendor who would implement WebGL, and the Media Capture Depth Stream Extensions, would somehow not implement using that depth stream with WebGL. This doesn't make much sense, since it's not a hardware limitation, and so it'd probably be good if whichever vendor implements WebGL and media capture depth stream would implement it fully from the get-go.

3) The style of how to support the upload isn't very friendly to users because it requires manual conversion in the fragment shader. I'd much rather see float/half-float texture formats being satisfied, where the browser knows about the surface format of the depth texture, and converts it (on GPU, without blocking operation) to the specified floating point format for upload. This'd have the advantage of offering users a transparent support for depth images, regardless of their underlying format.