[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Public WebGL] WebGL Conformance Testing Harness First Attempt



I put together a testing harness for WebGL.

My goals were as follows:


With those goals in mind here's a first attempt at a harness that meets those goals.

https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/tests/webgl-conformance-tests.html

I included all the tests that were already there and I added tests for the issues I felt most implementations were likely to still have problems with. Most of those problems have to do with the fact that all implementations are running on top of desktop OpenGL and are not rejecting enums and other parameters that are not valid for OpenGL ES 2.0.

The most important issue is with the differences with shaders. To get desktop OpenGL to behave like the OpenGL ES 2.0 spec will require implementations to both pre-process and parse the shaders themselves and munge them to remove stuff that works in GLES2 but not in DesktopGL, to enforce things that GLES2 requires but DesktopGL does not (precision specifications) and to disallow DesktopGL stuff that is not allowed in GLES2 (gl_ProjectionMatrix, gl_ClipVertex, ...).

It's easy to add new tests. If you use the framework there it just works.  You need to add your test to sdk/tests/fast/00_test_list.txt  or you can add to sdk/tests/test_list.txt.  The harness reads starts reading the later and adds the pages listed inside. If it sees another .txt file it will parse those as well for more tests.

Otherwise, if you prefer a different framework, to make a test page work with the harness the test page needs to call

    window.parent.webglTestHarness.reportResults(success, msg);

for each test passing true or false for success and a message about the test.

Once all the tests on the page have run the page must call

    window.parent.webglTestHarness.notifyFinished();

If notifyFinished is not called in a reasonable amount of time the harness will assume the page timed out and mark it as failed.

Both of these calls are easily wrapped with checks for their existence so that if the page is not run from the harness nothing happens. As an example, to make all the tests that were already checked in work all I had to do was add these two functions to sdk/tests/resources/js-test-pre.js

function reportTestResultsToHarness(success, msg) {
  if (window.parent.webglTestHarness) {
    window.parent.webglTestHarness.reportResults(success, msg);
  }
}

function notifyFinishedToHarness() {
  if (window.parent.webglTestHarness) {
    window.parent.webglTestHarness.notifyFinished();
  }
}
I then updated testPassed and testFailed as follows

function testPassed(msg)
{
reportTestResultsToHarness(true, msg);
debug('<span><span class="pass">PASS</span> ' + escapeHTML(msg) + '</span>');
}

function testFailed(msg)
{
reportTestResultsToHarness(false, msg);
debug('<span><span class="fail">FAIL</span> ' + escapeHTML(msg) + '</span>');
}
And I updated sdk/tests/resources/js-test-post.js as follows

shouldBeTrue("successfullyParsed");
debug('<br /><span class="pass">TEST COMPLETE</span>');
notifyFinishedToHarness();

I didn't make a lot of effort to style the harness. Everything having to do with UI and display is in sdk/tests/webgl-conformance-tests.html so if someone wants to make it prettier be my guest.

Otherwise it mostly seems to work. Some implementations need to synthesize GL errors instead of throwing exceptions.

Of course the tests represent my understanding of the spec. Please test your implementations and bring up any issues you find.