[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] WebGL performance regression tests!



On Thu, Mar 29, 2012 at 4:03 PM, Benoit Jacob <bjacob@mozilla.com> wrote:
>
>
> ----- Original Message -----
>> On Thu, Mar 29, 2012 at 3:27 PM, Ilja Friedel <ihf@google.com> wrote:
>> > On Thu, Mar 29, 2012 at 2:21 PM, Benoit Jacob <bjacob@mozilla.com>
>> > wrote:
>> >>
>> >>
>> >> For what it's worth, the performance regression tests are now much
>> >> expanded, covering in particular the dreaded texture format
>> >> conversions.
>> >>
>> >> Still same URL:
>> >>
>> >> http://hg.mozilla.org/users/bjacob_mozilla.com/webgl-perf-tests/raw-file/tip/webgl-performance-tests.html
>> >
>> >
>> > I second the very nice. To run this in practice in a regression
>> > suite one
>> > would need an easy way to adjust the work done for each individual
>> > test to
>> > say 50ms or 100ms. Right now on my atom test runtimes are between
>> > 8ms and
>> > 517ms. 8ms is awfully hard to distinguish from zero and computing
>> > any sum
>> > (or even geometric mean) is dodgy. Maybe allow for some kind of
>> > configuration/ parameterization of the number of loops running for
>> > each
>> > test?
>>
>> Good point. The V8 benchmark suite generally runs each benchmark for
>> a
>> fixed period of time, say 2 seconds, and measures the number of
>> iterations of the test that can be performed in that time, rather
>> than
>> using a fixed number of iterations. This seems to be a pretty robust
>> methodology.
>
> The performance tests currently run until both conditions are met: they've either done 10 iterations, and run for at least 300 ms.
>
> See in WebGLPerformanceTest.js:
>
> WebGLPerformanceTest.prototype.hasRecordedEnoughFrames = function(time) {
>  // stop when we have recorded at least 10 frames and run for at least 300 ms
>  return this.timings.length > 10 &&
>         time - this.startTime > 300;
> }
>
> The final result is then computed as the median result among all iterations/frames.
>
> So when you get "8 ms", it doesn't mean that the test stopped after 8 ms. It means that 8 ms is the median frame time.

Sounds good. Thanks for the clarification.

-Ken

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------