[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] WebGL performance regression tests!





----- Original Message -----
> On Thu, Mar 29, 2012 at 3:27 PM, Ilja Friedel <ihf@google.com> wrote:
> > On Thu, Mar 29, 2012 at 2:21 PM, Benoit Jacob <bjacob@mozilla.com>
> > wrote:
> >>
> >>
> >> For what it's worth, the performance regression tests are now much
> >> expanded, covering in particular the dreaded texture format
> >> conversions.
> >>
> >> Still same URL:
> >>
> >> http://hg.mozilla.org/users/bjacob_mozilla.com/webgl-perf-tests/raw-file/tip/webgl-performance-tests.html
> >
> >
> > I second the very nice. To run this in practice in a regression
> > suite one
> > would need an easy way to adjust the work done for each individual
> > test to
> > say 50ms or 100ms. Right now on my atom test runtimes are between
> > 8ms and
> > 517ms. 8ms is awfully hard to distinguish from zero and computing
> > any sum
> > (or even geometric mean) is dodgy. Maybe allow for some kind of
> > configuration/ parameterization of the number of loops running for
> > each
> > test?
> 
> Good point. The V8 benchmark suite generally runs each benchmark for
> a
> fixed period of time, say 2 seconds, and measures the number of
> iterations of the test that can be performed in that time, rather
> than
> using a fixed number of iterations. This seems to be a pretty robust
> methodology.

The performance tests currently run until both conditions are met: they've either done 10 iterations, and run for at least 300 ms.

See in WebGLPerformanceTest.js:

WebGLPerformanceTest.prototype.hasRecordedEnoughFrames = function(time) {
  // stop when we have recorded at least 10 frames and run for at least 300 ms
  return this.timings.length > 10 && 
         time - this.startTime > 300;
}

The final result is then computed as the median result among all iterations/frames.

So when you get "8 ms", it doesn't mean that the test stopped after 8 ms. It means that 8 ms is the median frame time.

Benoit

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------