[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] WebGL Conformance Testing Harness First Attempt



On Feb 8, 2010, at 5:42 PM, Gregg Tavares wrote:
> ...Maybe it would be better to avoid the test_list.txt file and rather have a skip list like in WebKit. That still gives us the ability to omit tests, but easier to add them and not have to remember to add them to a separate list.
> 
> I'm not sure what you mean here. As far as I know it's not possible for JavaScript to read directories to discover which HTML files are available, hence the need for test_list.txt.  You can certainly auto generate test_list.txt in some build process though and I can also add a skip_list.txt as well.  I'm trying to keep the tests runnable with nothing but a browser.

Ah, right. I'm forgetting that on WebKit we have DumpRenderTest which is a native harness! Maybe a cgi-bin script? I'm not sure what our server side permissions are. But it would be easy to write a script that returns all the html files in a directory. I bet we could get James to turn on cgi-bin if we don't have it enabled now.

> 
>  
> 
> >
> > Otherwise, if you prefer a different framework, to make a test page work with the harness the test page needs to call
> >
> >     window.parent.webglTestHarness.reportResults(success, msg);
> >
> > for each test passing true or false for success and a message about the test.
> >
> > Once all the tests on the page have run the page must call
> >
> >     window.parent.webglTestHarness.notifyFinished();
> >
> > If notifyFinished is not called in a reasonable amount of time the harness will assume the page timed out and mark it as failed.
> >
> > Both of these calls are easily wrapped with checks for their existence so that if the page is not run from the harness nothing happens. As an example, to make all the tests that were already checked in work all I had to do was add these two functions to sdk/tests/resources/js-test-pre.js
> >
> > function reportTestResultsToHarness(success, msg) {
> >   if (window.parent.webglTestHarness) {
> >     window.parent.webglTestHarness.reportResults(success, msg);
> >   }
> > }
> >
> > function notifyFinishedToHarness() {
> >   if (window.parent.webglTestHarness) {
> >     window.parent.webglTestHarness.notifyFinished();
> >   }
> > }
> >
> > I then updated testPassed and testFailed as follows
> >
> > function testPassed(msg)
> > {
> >     reportTestResultsToHarness(true, msg);
> >     debug('<span><span class="pass">PASS</span> ' + escapeHTML(msg) + '</span>');
> >
> >
> >
> > }
> >
> >
> > function testFailed(msg)
> > {
> >     reportTestResultsToHarness(false, msg);
> >     debug('<span><span class="fail">FAIL</span> ' + escapeHTML(msg) + '</span>');
> >
> >
> >
> > }
> >
> > And I updated sdk/tests/resources/js-test-post.js as follows
> >
> > shouldBeTrue("successfullyParsed");
> > debug('<br /><span class="pass">TEST COMPLETE</span>');
> > notifyFinishedToHarness();
> >
> > I didn't make a lot of effort to style the harness. Everything having to do with UI and display is in sdk/tests/webgl-conformance-tests.html so if someone wants to make it prettier be my guest.
> 
> Yeah, making it pretty can come later.
> 
> > Otherwise it mostly seems to work. Some implementations need to synthesize GL errors instead of throwing exceptions.
> >
> > Of course the tests represent my understanding of the spec. Please test your implementations and bring up any issues you find.
> >
> 
> All looks good. Next I'd like to see if we can integrate some sort of pixel tests. I think we can come up with reasonable results with at least simple static scenes.
> 
> What is Ilmari's status? Can we integrate his tests into this harness?
> 
> -----
> ~Chris
> cmarrin@apple.com
> 
> 
> 
> 
> 
> -----------------------------------------------------------
> You are currently subscribe to public_webgl@khronos.org.
> To unsubscribe, send an email to majordomo@khronos.org with
> the following command in the body of your email:
> 
> 

-----
~Chris
cmarrin@apple.com




-----------------------------------------------------------
You are currently subscribe to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email: