[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Public WebGL] A Declarative node set for WebGL?



This post is attempting to continue a conversation that has been ongoing in the public-html mailing list. It started as my response to the Web3D group's request that X3D be added to browsers in the same way that SVG and MathML have been. I said that I didn't think that was a good idea and started to dish out my opinion in what the right way forward was for a declarative 3D node set. 

I'd like to move that conversation here. I think we have some great general browser expertise here, as well as 3D expertise. I avoided posting this to the X3D mailing list on purpose. I didn't want to start stomping around in their playground with unformed ideas that would seem off-putting to many there. But I do plan to post on that list, inviting anyone there to come over and participate in the conversation.

Let me also say that this has nothing to do with the current WebGL spec. There are related issues that may have spec implications (like the image format issue) but those are being handled in other threads. This is just some thoughts about adding 3D nodes to the browsers, as an alternative to the purely imperative approach of WebGL. But I relate the discussion to WebGL because I think it's an interesting idea to develop a node set that is directly related to WebGL - a declarative form of WebGL, if you will.

I first want to throw a few ideas out there. I also have an action to respond to a mail from Johannes Behr, which I will do in a response to this post. I won't go into a lot of motivational detail about this approach. I've covered all that in the public-html thread (search for "request for guidance: centralized extensibility for HTML5 using X3D graphics" in the public-html archives if you're interested). Suffice it to say that I'm of the opinion that X3D is both too big (even in it's proposed "HTML profile") and designed for a previous era of the web. I think we can do much better now.

I've been thinking that it would be possible to create a set of nodes for 3D embedded in a web page. These would be embedded in a <scene> element and would match closely with WebGL functionality. In fact, it should probably be possible to get to a WebGLRenderingContext object from the <scene> object. So <scene> would behave much like <canvas> in that it would represent the drawing buffer used for 3D rendering. In fact, you'd be able to make rendering calls into the <scene> element using the WebGLRenderingContext. But you'd also be able to embed elements in the <scene> object to build a declarative scene graph.

You'd start with a <group> node. This would establish the scene hierarchy and would respond to the CSS 'transform' property. With this you can build an animatable transformation hierarchy. Then you'd need a <mesh> node. This would embody the VBO and drawArrays/drawElements functionality of WebGL. The rendering hierarchy consist of as many nested <group> elements as needed with <mesh> elements at the leaves. Inside the <mesh> element would be a <program> element which would embody the WebGLProgram object. Inside that would be <shader> and <texture> elements. The <shader> elements would have the type (vertex or fragment) and would contain the GLSL scripts. The <texture> element would contain the <img>, <canvas> or <video> element, as well as texParameters and the name used by the shaders to access it. There would also be a <uniform> element, which would allow you to define uniform parameters, or associate a uniform parameter with a CSS property. You could use this to apply an animated CSS color to a 3D object, for instance. This is also how you would get the current composite transformation matrix into the shaders.

There's not a lot more that's needed to represent WebGL functionality. You'd probably want an element to represent an FBO. It would be like a <scene> element, but it would render to the FBO, which you could then use as a <texture>. The viewport, scissoring and other scene level state would simply be attributes on the <scene> element, as would the clear() function. A lot more of the state might or might not be represented in some of the other nodes. Some of the state might only be accessible via the WebGLRenderingContext. At runtime the <scene> node would decide when a render was needed (based on the same factors that cause the page to rerender). Then it would clear the drawing buffer, traverse the scene, accumulate transforms, make those transforms available to the shaders, do the uniform setup and render the meshes.

You could imagine that if you want things like multi-pass rendering with different stencil modes, you could accomplish those by hooking into the <scene> rendering logic and using the WebGLRenderingContext to play with the state before each rendering pass.

Doing things like hit testing and interaction would not really need additional nodes, just some API on <scene> to let you traverse and cast rays and such.

All told, I've described 7 or 8 nodes. This is a gross simplification of course, but I think it is interesting to think of the the 3D node set in this way. Some will say that it would be much too hard for an author to do 3D like this. They would have to do lots of low level shader/uniform/texture setup (much like they have to do today with WebGL).  But you can still use an X3DOM approach to create a much higher level node set which would construct a rendered scene using the nodes I described. The advantages over using WebGL directly include:

- Tight integration with the page DOM
- Availability of CSS properties (including animated properties)
- Logic to determine when a render cycle is needed.
- Faster scene rendering through native traversal, matrix computation, etc.
- Native hit testing through a <scene> API
- Higher video texture performance through native video binding

I'm sure there are others.

Anyway, this is all just blue sky hand waving. I just wanted to get the thoughts out there.

-----
~Chris
cmarrin@apple.com





-----------------------------------------------------------
You are currently subscribe to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email: