[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] webgl-bench now has uploads, show aggregate results

On Sat, Dec 4, 2010 at 1:39 AM, Steve Baker <steve@sjbaker.org> wrote:
> Wow!  That's a really useful service.  Many thanks!

Thanks for checking it out!

> I have some comments (I appreciate that you're not finished yet):
> 1) You can't come back and see your previous stats - the only option is
> to re-run the test AND re-upload the stats - which means that you're
> going to get a bunch of duplicates.  It would be nice (for example) for
> me to be able to tell what difference my shiney new graphics card made
> when I install it by comparing my previous stats with the new ones...you
> could leave that as a cookie on my machine and thereby avoid having to
> have people log in and have passwords and such and de-anonymizing the
> results.

Agreed.  A cookie is a good idea for a quick & dirty approach.  I'm
definitely going to make sure there's a way to re-run and re-plot
without re-uploading, it just turned out to be awkward initially since
I currently draw the charts server-side.  Also I have a rudimentary
de-duping cookie but the stats are not actually using it yet.

> 2) The page says "The blue lines are the percentiles at 1, 25, 50, 75
> and 99." - I see lines and boxes - but it's not really clear what they
> mean.  (Actually, I figured it out eventually - but not everyone
> would).  I know (for example) that no machines had a zero for
> caps/BLUE_BITS - so the left end of the line can't be the 1st
> percentile...which is confusing.

At least one report had all 0's for the caps values, including
BLUE_BITS.  Once there are enough outliers to anonymize a bit better,
I'll try to show more detail about the outliers (browser, os, etc), so
implementors can debug.

> 3) It would be most useful if there were a text display of the result so
> people can cut/paste it and use the results in software directly...that
> would also go some way to address the issue of not being able to re-show
> old stats when I re-visit the site because I could cut/paste the results
> into a spreadsheet or something.

There's a primitive copy-pastable display in the text box on the
right, but another TODO is CSV download for everything (aggregate
stats as well as your own results).

> 4) It would be nice to indicate the sample size somewhere.

You should see a little "n=###" on the right of each box plot.  I'm
currently using SVG to draw those plots so it's possible the layout is
messed up on certain browsers and you can't see it.  I'm going to
switch to canvas and re-work the charts.

> 5) All of the machines I tested under Minefield (both Linux and Windows)
> had no green circles for many of the statistics...those same machines
> produced believably-placed green circles under Chrome under both OS's.
> Are these things that Minefield doesn't yet have a working glGet for?

OK, that could be the caps benchmark throwing an exception partway
through, or maybe getParameter returning non-numeric results.  The
caps code is at http://webgl-bench.appspot.com/static/caps.js so it
shouldn't be hard to debug.

Aha, my Minefield has the same problem.  It looks like
gl.getParameter(gl.MAX_FRAGMENT_UNIFORM_VECTORS) is returning null.  I
fixed the caps benchmark so it doesn't abort the whole caps collection
if one metric is null.


>  -- Steve
> On 12/03/2010 09:23 AM, Thatcher Ulrich wrote:
>> I've been hacking away at webgl-bench and now it shows aggregate
>> performance results:
>> http://webgl-bench.appspot.com
>> You can upload your benchmark results (or run the benchmarks without uploading).
>> It's still very rough and likely full of bugs, but fun to look at.
>> Next TODO is to show breakdowns by browser, OS, GPU vendor, etc.
>> Then, more benchmarks covering more interesting stuff.
>> -T
>> -----------------------------------------------------------
>> You are currently subscribed to public_webgl@khronos.org.
>> To unsubscribe, send an email to majordomo@khronos.org with
>> the following command in the body of your email:

You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email: