Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: Usefulness

  1. #1
    Member
    Join Date
    Aug 2004
    Location
    SCEJ - Tokyo, Japan
    Posts
    34

    Usefulness

    Great work. I'm glad someone is taking the initiative to make this happen.

    I'm not sure how other developers work but I'm used to specifying lots of game related and conversion related information inside my 3D editor (maya, max, lightwave, softimage)

    My impressions is the current standard of Collada doesn't address these types of issues.

    For example I don't force my artists to collapse models into one model before exporting even though i might want them to be one model at runtime. Forcing the artist to collapse them means they are no longer editable. They could keep two versions of the files but that means the possibility of not remembering which file is newest. It also means a manual collapsing activity everytime they export.

    Instead I have them mark where in the heirarchy a model starts and optionally where it ends, everything inbetween in that part of the hierarchy gets turned into one model by my tools

    Another example is I don't have them remove stuff they don't need. Instead I have them mark what they do need and ignore the rest. They often need lights, constraint objects, etc, in their scene. Asking them to remove that stuff everytime is asking for trouble.

    While it would be possible to write my own collada exporter to add this data it seems like the point should be to allow me to avoid having to write my own. Otherwise nothing is gained and I might as well use my own format since i'm going to have to write the backend anyway to use the data.

    A standard that defined how to export all this extra data (comments, notes, blind data, sets, that is attached to 3D data in a typical scene in max, maya etc) would mean that I could actually use Collada as my middle format

    Not just a way, but a defined standard that says which pieces of extra data in max/maya/xsi correspond to which extra pieces in the collada file. Comments, blind data, user attributes, etc all need to get exported from all packages in the same way or at least as close as possible.

    I guess, at least for me, I need Collada to not only define a file format but to specify, at least to some extent, how things get exported from the various packages. Otherwise I'll just end up having to write my own.

    For example, exporting everything, even hidden and frozen items. My artists often freeze or hide stuff that they still want exported. My exporters export everything the aritst marks for export. To use collada I'd need the exporters to either do the same (not feasable since it would most likely be game specific) or else to export EVERYTHING (including hidden and frozen objects) and let me choose how to use the info.

    Also,

    I know animation is not covered yet but for me it's not very useful without it. I know that's harsh but it's true. Why would I make my work flow support collada only to have to make some hack for animation and then re-write it when collada has animation added. I only say that to emphasise the importance of getting the animation spec finished.

    But, with that in mind there is often data I need there as well. I need not just the animation on models, I need the animation on user parameters, materials, etc,. My artists for example make a list of animations and frames (walk is frame 1 to 20, run is frames 25 to 40) for example. They even go so far as to set, for a particular animation, which bones need data. So for example to blend a guy waving his arm anim with a guy running anim I need to know not to blend the leg positions (in fact for compression not even to save them).

    Again, I don't need that data saved out as that. What I need is for Collada to specifiy how Maya, Max, XSI etc export the extra data so that on reading the Collada file into my own tools I can dig that info out and figure out what I really want to do with all the data.

    Note, that includes animation of user types. If I make float field in Maya called "translumosity" and set animation on it I need that field definition and data to make it down to the collada file.

    Comments?

  2. #2
    Senior Member
    Join Date
    Jul 2004
    Location
    Santa Clara
    Posts
    356
    Thanks a lot for this feedback. This is exactly why we need from the community to be able to make COLLADA evolve in the right direction.

    While it would be possible to write my own collada exporter to add this data it seems like the point should be to allow me to avoid having to write my own. Otherwise nothing is gained and I might as well use my own format since i'm going to have to write the backend anyway to use the data.
    I do not understand why you think adding a feature to COLLADA would require as much work as creating your own format, exporter and loader ?
    Since we are using XML we are relying on open source tools to parse the file ourselves, and we are constantly changing the format while testing our evolutions of COLLADA without habing to start from scratch each time?

    Not just a way, but a defined standard that says which pieces of extra data in max/maya/xsi correspond to which extra pieces in the collada file. Comments, blind data, user attributes, etc all need to get exported from all packages in the same way or at least as close as possible.
    That's right. We want to have this done for our next release. The major issue is to get everybody to agree on how to export/import such data the same way.
    What would be very useful for us, is if you would like to spend some time to put a straw man proposal, and maybe a prototype export/import for how you would like the extra data to be available in COLLADA. You do not have to think about how every tool would implement it, just to think about how you would like it.
    Note, that includes animation of user types. If I make float field in Maya called "translumosity" and set animation on it I need that field definition and data to make it down to the collada file.
    COLLADA 1.0 does not address animations. This is something we are working on for next release. This is a good time to make proposals regarding the animation system within COLLADA as well.

    Thanks for your help.

  3. #3
    Member
    Join Date
    Aug 2004
    Location
    SCEJ - Tokyo, Japan
    Posts
    34
    Quote Originally Posted by remi
    While it would be possible to write my own collada exporter to add this data it seems like the point should be to allow me to avoid having to write my own. Otherwise nothing is gained and I might as well use my own format since i'm going to have to write the backend anyway to use the data.
    I do not understand why you think adding a feature to COLLADA would require as much work as creating your own format, exporter and loader ?
    Since we are using XML we are relying on open source tools to parse the file ourselves, and we are constantly changing the format while testing our evolutions of COLLADA without habing to start from scratch each time?
    My point is unless collada exports everything I need then if I modifiy it to do so, everytime a new version comes out I have it re-insert my mods on every different exporter. Rather than do that I'd be more likely to just write my own and not have to worry about it.

    If on the other hand collada did export everything I needed including all that extra data then maybe there would be no reason to write my own.

    I will point out that I know some companies that would say a problem with XML and text based formats is they are too slow for million or 100 million polygon levels. I see their point. I suppose one suggestion there would be to allow storing the vertex data, color data, weight data, UV data and other large datasets as CDATA in well defined formats or optionally as filenames of binary data.

    As for the animation data I would personally want to see the data baked on export. That's not to say collada shouldn't also include function curves for those teams that want them but I'm of the camp that I'd prefer to let the 3D tool bake the data and then I'll curve fit it later. That means if collada only exported function curves I'd be out of luck and again have to write my own.

  4. #4
    Junior Member
    Join Date
    Aug 2004
    Posts
    6
    Just as a little background information, we've wrote our own XML exporter in 1999 and we're still using it. We didn't bother with schema for validation because our conversion tools would detect weird data anyway, we kept things as simple as possible.

    We've been very happy with XML as our intermediate format, we've redesigned the format a few times over the years to fit changing requirements and to reduce the file size. The files are still big, often in the 60-110MB range, and several of these are joined by our conversion tools to build a level.

    Still, conversion times are not limited by the current XML file sizes, other conversion tasks take up much more time. But if the next generation consoles bring 10x to 50x the triangle count we may have to change our pipeline to keep conversion times down.

    We wrote our own, very simple, XML parser to keep parsing quick and we designed the XML format such that we can do it all in one pass.

    I agree with greggman here in that the benefit Collada could give us is:

    1. We don't have to write exporters / importers anymore
    2. We gain flexibility because it is supported by several tools

    But, if I have to modify the exporters/importers anyway to add special features to the exporter, I may as well design my own data format and implement my own exporter/importer for whatever tools we use.

    As for animation we currently support exporting both the actual keys and baked animation data. Exporting just keys is problematic in a few situations, e.g. when animators apply IK since IK'ed bones do not get keys. Because of this we almost always use baked animation data and have our tools do their thing (curve fitting, compression, etc.). Baked data is nice too because you can use any weird plug-in to modify the animation and you'll still get the proper result.

    We also use note tracks along with animation data to mark foot-steps and other fun triggers in animations.

    Roar

  5. #5
    Senior Member
    Join Date
    Jul 2004
    Location
    Santa Clara
    Posts
    356
    Thanks for those comments.
    This is very important as we are currently designing animation support in COLLADA.

  6. #6
    Junior Member
    Join Date
    Sep 2004
    Location
    Bellevue, WA
    Posts
    11
    The files are still big, often in the 60-110MB range, and several of these are joined by our conversion tools to build a level.
    I just found this site, and it seems very cool. I haven't read the spec yet, but I have to ask something before I forget. On a cursory scan this doesn't appear to be part of it....

    Has any thought been given to specifying blocks of numerical data by referencing a chunk in a data file (ala fxcomposer)? That could help out a lot for the file sizes if that becomes an issue.

    Ok... I hope that wasn't completely pointless , off to read the spec...

    EDIT: Ok, this question was already asked, my bad... reading away...

    Adruab

  7. #7
    Senior Member
    Join Date
    Aug 2004
    Location
    California
    Posts
    771
    Quote Originally Posted by adruab
    Has any thought been given to specifying blocks of numerical data by referencing a chunk in a data file (ala fxcomposer)? That could help out a lot for the file sizes if that becomes an issue.

    EDIT: Ok, this question was already asked, my bad... reading away...
    Yes it has

    COLLADA uses URI to refer to external resources and that enables fairly fine grained referencing of data including document fragments (binary chunks even) and SQL queries.

  8. #8
    Member
    Join Date
    Aug 2004
    Location
    SCEJ - Tokyo, Japan
    Posts
    34
    Quote Originally Posted by marcus
    COLLADA uses URI to refer to external resources and that enables fairly fine grained referencing of data including document fragments (binary chunks even) and SQL queries.
    Can you clearify this statement?

    Are you saying that any collada standards conforming library out there that will load externally referenced binary formats and do SQL queries? From the spec I do not get that impression. In HTML you can reference a link a in <a href="http://somesite.com/someC++madeapp.exe"> but that does not mean HTML supports C++.

    My impression is your statement is misleading. I saw no standard for binary formats in the spec or SQL queries therefore collada does not support these. Please clearify if I'm wrong.

  9. #9
    Senior Member
    Join Date
    Aug 2004
    Location
    California
    Posts
    771
    Quote Originally Posted by greggman
    Can you clearify this statement?
    Sure, I said we are using URI technology to reference external resources. This is an enabling technology that satisifies the arguments being made here in regards to accessing external binaries.

    Quote Originally Posted by greggman
    Are you saying that any collada standards conforming library out there that will load externally referenced binary formats and do SQL queries?
    No I didn't say that. I am saying COLLADA implementors can create tools that have those capabilities because we have chosen XML and URI technologies for the COLLADA schema. As COLLADA is its early stages as a project, all of the partners can expound on the specification and implement features that they need in their tool chains that comform to the specification.

    Quote Originally Posted by greggman
    From the spec I do not get that impression. In HTML you can reference a link a in <a href="http://somesite.com/someC++madeapp.exe"> but that does not mean HTML supports C++.
    I think your example mixes the functionality of the HTML schema with some implementation of a web browser that may or may not understand how to handle a URL to an executable binary. HTML is not the web browser, nor is COLLADA a run-time library or application. I think if you can separate those two things in your mind, my statements will be more understandable.
    Quote Originally Posted by greggman
    My impression is your statement is misleading. I saw no standard for binary formats in the spec or SQL queries therefore collada does not support these. Please clearify if I'm wrong.
    I'm not here to say "You are wrong". I'm here to design an XML schema that will enable vendors to implement a normalized tool chain for the benefit of the video game industry and related media industries such as motion pictures. Your contributions to that effort are encouraged and appreciated.

    The collaborative aspect of COLLADA extends beyond schema design, reference implementation, and importers and exporters. The specification itself is open to more authors then just myself and my peers at SCEA. You are welcome to submit clarifications to the specification and ammendments that you think need to be covered. My capacity in that regard is Editor in Chief.

  10. #10
    Member
    Join Date
    Aug 2004
    Location
    SCEJ - Tokyo, Japan
    Posts
    34
    I agree with the goals I posted in another thread. Basically that collada should first and foremost be about getting data from any 3D tool into a game.

    It only approaches that goal if by using a collada format reading library I can load any collada file and access the data inside in a consistant manner.

    I don't understand the statements that collada can support external URIs to binary data and SQL queries if you happen to implement them in your own pipeline. Is there a difference between that statement and saying I can reference any file in Maya in my pipeline by putting the comment "file=myexternalfile.myformat" as an attached attribute on a node in Maya and then adjusting my pipeline to look at those comments.

    Which goals are you actually trying to achieve? The goal of being useful (ie, being able to load any collada file and reference **specific** data in a consistant way) and the goal of being flexible (ie, being ability to insert the data any way you want) are not in agreement.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •