Khronos Frequently Asked Questions
- How exactly is a conformance test run?
That’s an interesting side effect of the way that we’re doing conformance, which is quite different from the traditional OpenGL way of doing it. We do not have a mechanism for removing conformance certification from a device, and the rule is if you have in good faith designed a device to pass conformance, then you get to call yourself conformant. One of the reasons we’re excited about having the conformance test out in public, though, is that as it evolves, community members can run it. They can ... I don’t want to say name and shame, but they can name and perhaps register issues. We hope that will have multiple good effects. One is it will encourage all implementers to be much more aggressive about updating, so that they maintain conformance, and that conformance is visible to the community. The other, of course, is that the conformance tests can evolve more rapidly, and just make the API more reliable.
- In reading your Adopter's Agreement, we noticed that in section 2, "Confidential Information" that the test results fall under this heading. Depending on the results of the tests, we'd want to share those results (not the Adopter's Package) with the vendor. Is this use of the test results acceptable? The vendor is under NDA with us.
Test results may be disclosed under NDA between customer and vendor or the other way around as long as it is made clear to the party receiving that:
- This does not allow a vendor to claim conformance unless they have filed the results and conformance has been granted
- The results are not allowed to be publically disclosed as proof of conformance
- Conformance does not flow from one party to another (i.e. if your company files conformance on an OEM product, that doesn’t automatically give the silicon vendor or IP supplier conformance)
- You say that we must “successfully run the implementation with the Conformance Tests;” but these other conforming members are doing this on our platforms, aren’t we covered?
Conformance claims to not “pass through” from customers.
- You say that we must “obtain the required license from the Khronos Group by signing the appropriate License Agreement;” , but I believe we are covered as member company
Not in the case of conformance- you will have to adopt and submit test results for approval to claim conformance.
- I do not understand why we are being referred to the “adopter documents”.
An adopter may be a member or non-member, but all must adhere to the adopter process to make claims of conformance.
- Our company does not submit conformance tests, but we require our graphics IP provider to do so with our products. Under http://www.khronos.org/adopters/conformant-products/#topengles you can see that other companies have submitted multiple tests with our company’s products. What’s the problem with us doing that?
Claims of conformance do not “pass-through” from customers.
- We do not need to use the Khronos logos or trademarks in our marketing. Conformance submissions would hence not be needed. Is there a reduced fee for this kind of adopter?
Are you sure you do not need to use the name of the APIs in association with your product/platform? Any use of a Khronos trademark (e.g. using the API name) in association with a specific product requires the execution of the Adopters Agreement and passing the conformance tests through making a submission. Any company using Khronos trademarks in association with a specific product needs to be an Adopter - as the Adopters agreement contains the trademark license.
However - if a company is acting purely as a subcontractor to another Adopter and is willing to forgo any rights to use the marks in their own right then that company may apply for a fee waiver as a subcontractor.
- Is there a fee for becoming an Adopter, other than the conformance fees paid for each API?
The conformance fee per API is the only payment due. The conformance fee enables an unlimited number of submissions for that version of the specification.
- Does a company have to be an Adopter to get access to Khronos conformance test sources? If not, what are the requirements for getting the right to utilize the conformance tests?
Any company using the Khronos Conformance tests to test products have to become Adopters through executing the Adopters Agreement. This is necessary as the Adopters Agreement contains the source code license to the conformance tests for this purpose.
- If a company buys a conformant product from a Khronos Adopter to deploy in their own product(s), do they have to be a paid adopter and also have passed conformance tests in order to use the relevant Khronos marks and claims of conformance?
Even though a piece of IP passes conformance, an SoC product implementing that IP does not inherit the conformance. Alterations in CPU config, memory system, target resolution etc. are potential variables which could cause a previously conformant product to fail conformance. Therefore SoC’s designs containing IP, with or without previous conformance filings must still file for conformance for that SoC implementation.
- We plan on using a Conformance Test to test our phone. To do it, we would like to hand the Conformance Test source code to a subcontract company and let them make the test environment. Is this allowed and what is the procedure?
It is allowable for Member Adopter Company to provide a subcontractor company the conformance test source code so that subcontractor may test Adopter Member Company’s product(s) as long as the agreement between Member Adopter Company and the subcontractor preserves Khronos Group’s rights/NDA as provided in the Membership and Adopter documents.
- SGI still has the trademark rights of OpenGL. Is the Khronos Group allowed to use this trademark with the permission of SGI?
Yes. The OpenGL trademark agreement is informal. The Khronos Group are in discussions with SGI to allow Khronos run an OpenGL Conformance program.
- What is the relationship between SGI and Khronos Group?
SGI is not a member of the Khronos Group. Khronos licenses the OpenGL ES trademark from SGI. Khronos is authorized to run the OpenGL ES adopter program.
- Is an OpenGL ES adopter required to comply with OpenGL licensing when they have an OpenGL ES conformant product?
- How does Khronos Group set the Adopter Fees?
The adopter fee is set by the individual workgroups and then approved by the Khronos board of promoters at the time the conformance test becomes available. For most of the Khronos APIs the adopter fee is currently $15,000 with a $5000 discount for paid Khronos Members. (Promoter or Contributor level membership, price subject to change). The adopter fee is a one-time charge per version of the API and allows adopting company unlimited conformance test submissions of products for certification for the life of the API.
Additional information can be found on the Khronos Adopters page.
- We are looking forward to implementing OpenMAX compliance of codecs. Will the implementers get Khronos SDKs? If so please provide me the necessary directions.
OpenMAX has 3 levels of APIs, with separate specifications: OpenMAX AL, OpenMAX IL and OpenMAX DL. Referring to a codec implementation, I suspect the specification you would like to implement is OpenMAX IL.
There is currently no SDK available for OpenMAX IL. An open source implementation called Bellagio exist on SourceForge, but this implementation is not maintained by Khronos and does not provide you with any guarantee your components will be conformant with the OpenMAX IL specification. As the original work (Bellagio) has not been tested for conformance, any derivative work based on it, unless submitted for conformance testing as a complete work, could not be declared conformant either as you cannot guarantee that the interface for the codec was conformant in the first place.
The correct way forward to ensure conformance would be to sign the Khronos Adopters Agreement, pay the associated Conformance Testing Fees, then access and run the Conformance Test associated with the specification. You will find all the necessary information on the Khronos adopter page.
- The chip we want to certify as conformant uses a core that we know to be conformant: is it possible to submit a conformance test result package for the core itself and have all chips using that core be covered under the one test?
From section A8 of the Conformance Process Document, any company can submit results for any implementation they like, and that submission will cover any products that fall into the criteria below:
“Implementations claimed as Conformant Products must be similar to the Conformant Implementation in the following ways:
. the identical rendering pipeline - i.e. identical binaries and/or accelerator data path to the display, or if a JIT compiler is used to generate binaries then the use of the identical JIT compiler binary, or new versions of the binaries and/or accelerator data path or JIT compiler binary that do not cause any previously passing test to fail; . the same major version of the same OS that uses substantially similar display functionality or minor version updates to the OS that do not cause any previously passing test to fail; . the identical set, or a subset, of supported configs. Different display resolutions are permitted.”
So - Yes, the Adopter Company can submit results for a core, or a chip - and the use of the mark will cover the products that are similar according to the above rules.
- Can we get a membership which grants us access the conformance suite for OpenGL or any other API and allows us to make requests for software limitations or defects we find?
The Khronos contributor membership allows members to participate in all Khronos working groups including OpenGL. Members have the ability to participate and monitor progress of projects as well as the opportunity for input as related to limitations, bugs, defects, and the opportunity to participate in the ongoing evolution of the specification.
- How are your licenses distributed?
The Khronos Conformance Test Source License Agreement (CTSLA) enables members to cooperate in building conformance tests – the license states that any modifications to the tests would be passed back to Khronos.
- Can we get a membership to use code within our company, and not be involved in submitting code back to Khronos?
Khronos Ratified specifications are publicly available on the Khronos Web-site. If your company wishes to use Khronos trademarks for conformant implementations of APIs, then the Adopters’ Agreement, along with applicable payment, provides access to the testing procedure and provides a trademark license.
- We would like to know what membership will cost to obtain OpenVG and OpenGL, with the exact same type of membership?
The Khronos membership agreement includes a Conformance Test Source License Agreement (CTSLA). Members that have not excluded themselves from the CTSLA, have access to the conformance test source. Members wishing to claim conformance to OpenVG and OpenGL ES specifications would be required to execute the adopters agreement and pay the applicable fee for each API.
Current Adopter fees are available here: http://www.khronos.org/adopters/
- If we decide to enter the conformance test process, does it require our SDK, and our Demo Board?
You do not need to submit your SDK or demo board-only the conformance test results for peer review.
- If we have fail a conformance test result, how much is the re-test fee ?
Khronos conformance test submission fee is only paid once per API version. Therefore, if a test fails, you may re-submit unlimited conformance test results for that API with no additional fee.
- If we decide to enter the conformance test process, how long will it take for the test?
Typically, the conformance test result peer review is 30 days. Please see conformance test procedure documents for more information.
- There is one issue that our attorney raised with the section 4.3 of the Members Agreement that we would like to address.
The full question:
“The Source Package distributed under the Source License is subject to the confidentiality provisions of the Base Agreement. As already noted, that agreement appears to preclude the use of the Source Package to do conformance testing, which is an obvious oversight by Khronos. In addition, however, the same portion of the Base Agreement limits disclosure of confidential information to employees, on a need to know basis. This means that it is not permitted to show any of the Conformance Testing Source Package to any subcontractors doing software coding work on this project.”
Could you please confirm that it is NOT Khronos intent to preclude the use of the Source Package to do conformance testing as it might be construed from Section 4.3 of the Membership Agreement?
It is NOT Khronos’ intent to preclude the use of the Source Package to do conformance testing as it might be construed from Section 4.3 of the Membership Agreement.
Section 4.2 states: “Signatory may use Confidential Information in any Khronos-related activities.”
On the matter of subcontractors, subcontractor access is NOT automatically granted - but we do have a provision where a bona fide subcontractor can execute the Conformance Source License with no fee if approved by the Board.
- Can we access the Conformance Test Package (CTP) before the actual payment? In the Khronos Adopter's Agreement, it says the total due should be paid prior to getting the CTP.
Conformance test submission fee payment must be received before conformance test results may be posted for review. If you would like an invoice for Conformance Testing, please follow the directions on our Adopters page. We will return invoice promptly for your review.
- We are a Member of Khronos Group and have an Adopter Agreement for OpenGL ES. Our OpenVG team wants to access an OpenVG Conformance Package before the submission of actual Conformance Test. Is it possible to access the OpenVG Conformance Package and do we need an Adopter Agreement for OpenVG additionally?
Yes, as Khronos member you can get access to the Conformance Kit before paying for submission.
The only condition is signing of the Khronos Conformance Test Source License Agreement (CTSLA) by your company which is required only once, so quite likely it is in place already. If so, you should be able to get the test package from the Subversion repository right away.
This will give you access to the entire Conformance Test Suite (CTS). You can run all test cases internally and make sure that your implementation passes them. This isn’t equivalent to submission of your product for conformance.
- What is the policy about the payment for a Conformance Test? In the Khronos Adopter's Agreement, it says the total dues should be paid prior to getting the Conformance Test Package (CTP). Can we access the CTP before the actual payment?
Conformance test submission fee payment must be received before you may download the official conformance test package and before you can submit your results for review. If you would like an invoice to become an adopter for any of our APIs, please review the Adopter pages.
- We, are working with a Khronos Adopter as a subcontractor on for a Khronos Technology. This Adopter has used the sample source code, customized it and distributed to us as a source code and a binary code. We need to integrate this with our own software modules, and then distribute to our customers after we compile them together. What are the limitations on redistribution?
The Adopters Agreement for Khronos Technologies includes a license for the source to the tests.
The Test Source may ONLY be used to port and run the tests - this code may not be re-distributed.
Any Open Source sample source wrappers or implementations may be distributed freely as long as the copyright notices in the software are met.
- How many conformance tests can I submit for a particular technology?
Your conformance test fee lets you make an unlimited number of Submissions for any number of Conformant Products using any version of the technology up to and including the latest version of the Tests included in the current Adopters Package at the time of execution of the Adopters Agreement, at your selected Fee.
- What are the currently available conformance tests and fees?
- All memberships levels and pricing are available from our Become a Member Information page. Currently available conformance tests are available here.
- How much does it cost to become an Adopter Member and submit Conformance Tests for a particular Khronos technology?
- All memberships levels and pricing are available from our Become a Member Information page.
- How do I become a Khronos Adopting Member and submit my product to the Conformance Test Process
To understand the Adopters program and conformance testing, please review the Adopters web page. Be sure to review these documents:
- What are the Khronos Conformance Tests and why do I want to submit my product?
Conformance Testing is performed to ensure that a product complies with the Khronos technology standards. Products that pass the Khronos Conformance Tests are certified as compliant with the standard and may use the technology logo to advertise and promote their products. This insures consumers with a guaranteed level of performance and thus helps product developers with a way to differentiate and validate their offerings.
Conformance Test will be available for all of the Khronos technologies. More information is available on the Adopters web page.