[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] Issues with sharing resources across contexts



On Wed, Jul 18, 2012 at 1:53 PM, Ben Vanik <benvanik@google.com> wrote:
> o_o
> wait, so are we talking about the same thing, then? ;)

I believe so. You also propose explicitly being able to release a
resource and when all readers have released the resource, it is
available to a single thread for RW acquisition. This means every
acquisition needs to check a lock and could potentially fail or block.
On the flip side, no resource allocation is required.

We both want:
freeze : (1 RW, 0 R) -> (0 RW, n R) [immutable] where (1 RW, 0 R)
becomes (0 RW, 1 R) handle backed by the result (0 RW, n=1 R)

You would also like:
unfreeze : (0 RW, n R) -> (1 RW, 0 R) where unfreeze of (0 RW, n R)
with n > 0 would produce an error

I propose that unfreeze always produce a new JS resource handle and
never generate an error.

If it would otherwise block or fail because n R where n > 0, then a
copy should occur instead (and it should be easy for an author to see
that this is happening if they do not intend it).

If it would not block or fail (n=0), the underlying GL resource may be
reused in any context. If GL resource creation is hurting your
performance, you should always trigger this case.

Only resources of type (0 RW, n R) should be able to cross
thread/context boundaries. No lock need be checked to access resources
of type (0 RW, n R). To use a resource of (0 RW, n R):
gl.handle : (0 RW, n R) -> (0 RW, 1 R) where (0 RW, 1 R) is not
shareable just like (1 RW, 0 R)

gl.handle will never fail and will always return the same object for a
given context. If the (0 RW, 1 R) that was returned is the first for
this context (or has been released), (0 RW, n R) becomes (0 RW, (n+1)
R).

To make it easier to manage read handles, a per context function could
be provided to declare the resource no longer used:
gl.release : (0 RW, 1 R) -> void where backing (0 RW, n R) becomes (0
RW, (n-1) R) if this handle was active otherwise nothing occurs

I assume that when GC occurs on a (0 RW, 1 R), gl.release is implicit.

Does this make sense? Are there performance problems? Unexpected sharp
corners? Precluded use cases?

Additional statistical functions could be defined:
readers : (0 RW, n R) -> n

I believe this proposal reduces the number of exceptional cases (still
can't mutate a (0 RW, 1 R) without an exception) with the addition of
a single easily avoidable performance penalty (to serve authors'
intuitions).

Thoughts?

> Basically, as a dev, I don't care how it's implemented so long as:
> - I can read the same resource from multiple workers
> - I can write to a resource from a worker and (at some point in the future)
> have those changes read by other workers
> - I do not have to duplicate/copy/etc a resource in order to do any of the
> above
> - Doing any one of these things does not preclude me from doing another
> (e.g., I could read from multiple workers but then not be able to update it)

Some resources (e.g. shader programs) do not have meaningful/useful
mutation semantics. I don't believe you should be able to unfreeze or
release them and acquiring their handles should be implicit.

David

>
> On Wed, Jul 18, 2012 at 1:51 PM, David Sheets <kosmo.zb@gmail.com> wrote:
>>
>> On Wed, Jul 18, 2012 at 1:41 PM, Ben Vanik <benvanik@google.com> wrote:
>> > Oh, in case it wasn't clear: the objects are exclusive write - there can
>> > be
>> > no readers while there are writers, and vice versa.
>>
>> Excellent. I believe we are actually in agreement.
>>
>> > I'm not sure how you aren't seeing how immutable resources are bad.
>> > Almost
>> > all of the cases I (and many others) want WebGL in workers for require
>> > mutable resources. Resources being immutable makes them almost useless,
>> > except to offload what exists as single-threaded rendering today to a
>> > worker
>> > (which just adds additional latency and not much else).
>>
>> Your read lock that excludes write makes the resource immutable. Once
>> all readers are released, acquisition of your write lock is explicit
>> resource recovery. In no cases do you employ sharing and mutability
>> simultaneously.
>>
>> > On Wed, Jul 18, 2012 at 1:37 PM, David Sheets <kosmo.zb@gmail.com>
>> > wrote:
>> >>
>> >> On Wed, Jul 18, 2012 at 1:15 PM, Ben Vanik <benvanik@google.com> wrote:
>> >> > All of them ;)
>> >> > Almost every use case of WebGL on workers needs mutable resources.
>> >> >
>> >> > The primary ones I've used in the past (and would love to use in
>> >> > WebGL
>> >> > with
>> >> > workers) is modifying buffer data in various worker threads. You
>> >> > always
>> >> > want
>> >> > to pool the buffers and double (or triple) buffer them when
>> >> > rendering.
>> >> > They
>> >> > may be populated on one worker and drawn on another (or the main
>> >> > thread).
>> >> > Same with textures in the case of either a render-to-texture done on
>> >> > a
>> >> > worker or a plain old upload, or the reverse of reading back the
>> >> > framebuffer
>> >> > on another worker for JS processing.
>> >>
>> >> Except for subrange mutation, I do not see how these use cases benefit
>> >> from shared mutable resources over immutable resources.
>> >>
>> >> > These cases would be possible with ownership transfer, and
>> >> > transferable
>> >> > arrays has shown that it's an easy concept to wrap ones head around
>> >> > and
>> >> > works well.
>> >> >
>> >> > One thing that would suffer, though, is the case of multiple readers.
>> >> > I
>> >> > may
>> >> > want to do some readback of a texture for GPGPU data, and could
>> >> > imagine
>> >> > sending that texture to multiple workers - each worker would then
>> >> > readPixels
>> >> > a separate region of the texture.
>> >>
>> >> Immutable resources solves this problem.
>> >>
>> >> > The other thing that suffers with single ownership is reusing shared
>> >> > resources (shaders/buffers/textures) - I can imagine (and want) many
>> >> > cases
>> >> > where I'd have multiple workers rendering the same scene in different
>> >> > ways -
>> >> > whether it's a visualization tool or a game that does things like
>> >> > GPGPU
>> >> > occlusion culling in one worker and real rendering in another. Or
>> >> > megatexture processing using the same geometry buffers in one worker,
>> >> > readback of the framebuffer for processing in another, texture
>> >> > uploads
>> >> > in
>> >> > yet another, and final scene rendering on the main thread. If I had
>> >> > to
>> >> > duplicate all of my level geometry, rerun my CPU skinning, process my
>> >> > particle systems, and consume Nx as much GPU memory for each worker
>> >> > I'd
>> >> > be
>> >> > disappointed - especially on embedded devices.
>> >>
>> >> Immutable but shared resources solves this problem without using Nx GPU
>> >> memory.
>> >>
>> >> > Hmm... I feel like an acquisition model probably makes the most sense
>> >> > for
>> >> > these cases, although it's a bit more complex...
>> >> > - pass webgl resources to other workers like transferrable arrays (as
>> >> > additional args to postMessage)
>> >> > - a resource is not usable once received until acquired
>> >> > - some WebGLRenderingContext (or extension methods) to work with the
>> >> > objects:
>> >> >   - acquireResource(obj, mode [READ_ONLY, READ_WRITE]) --> returns
>> >> > whether
>> >> > acquired
>> >> >   - releaseResource(obj)
>> >> > - if you try to read or write an unacquired object, fail at API level
>> >> > - if you try to write to a read only object, fail at API level
>> >> > - multiple readers are allowed, only one writer
>> >>
>> >> What happens when I have 4 reader workers and 1 writer worker that
>> >> have all acquired the resource and then the writer mutates the
>> >> resource? Does it change underneath the readers?
>> >>
>> >> To be clear, I am proposing: (1 read-write context, no sharing) -> (0
>> >> read-write contexts, arbitrary sharing)
>> >>
>> >> > This allows multiple workers to render the same objects (all acquire
>> >> > as
>> >> > read
>> >> > only), removes the required implicit flush, and substitutes an
>> >> > explicit
>> >> > flush (an acquire for reading of a dirty object could flush, if that
>> >> > was
>> >> > tracked). Performance-wise most apps would likely acquire as read
>> >> > only
>> >> > once
>> >> > and never have to worry about it again (no synchronization across
>> >> > workers
>> >> > required), It's also not magic - meaning that it may require a few
>> >> > more
>> >> > calls, but it's very clear what's happening and why, and it's not
>> >> > possible
>> >> > to fall into some undiscoverable (or unavoidable) performance trap
>> >> > with
>> >> > implicit flushes.
>> >> >
>> >> >
>> >> > On Wed, Jul 18, 2012 at 12:52 PM, David Sheets <kosmo.zb@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> On Mon, Jul 16, 2012 at 6:03 PM, Gregg Tavares (社用)
>> >> >> <gman@google.com>
>> >> >> wrote:
>> >> >> > I agree that solution would work for 2+ contexts in a single
>> >> >> > webpage
>> >> >> > since
>> >> >> > webpages are single threaded. Is that enough for now? Do we care
>> >> >> > about
>> >> >> > WebWorkers where that solution will no longer be enough?
>> >> >> >
>> >> >> > Lemme throw out some ideas. They probably suck but maybe they'll
>> >> >> > jog
>> >> >> > some
>> >> >> > others.
>> >> >> >
>> >> >> > What if a resource could only be used in 1 context at a time and
>> >> >> > you
>> >> >> > had
>> >> >> > to
>> >> >> > transfer ownership to use it another context? That might solve
>> >> >> > this
>> >> >> > issue as
>> >> >> > well. Would that be too restrictive? What if an object could be
>> >> >> > used
>> >> >> > in
>> >> >> > any
>> >> >> > context but state only set in the context that "owns" it? Not sure
>> >> >> > that
>> >> >> > would help. In the case above it would mean any context could call
>> >> >> > UseProgram but only the context that owns it could call
>> >> >> > AttachShaders
>> >> >> > and
>> >> >> > LinkProgram since those change state. Of course that wouldn't
>> >> >> > solve
>> >> >> > every
>> >> >> > issue because UseProgram requires the results of LinkProgram.
>> >> >> >
>> >> >> > I'm just throwing that out there. The idea is probably not
>> >> >> > workable
>> >> >> > but
>> >> >> > it
>> >> >> > would be nice to try to design an WebGL API for WebWorkers that
>> >> >> > didn't
>> >> >> > have
>> >> >> > the ambiguity that OpenGL has as far as execution order. Whatever
>> >> >> > that
>> >> >> > solution is might also work for 2 contexts in the same page.
>> >> >>
>> >> >> What if a means is provided to make a given WebGL resource
>> >> >> immutable?
>> >> >> Once the resource is 'frozen', no further modification would be
>> >> >> allowed but the resource could be shared between contexts or
>> >> >> workers.
>> >> >>
>> >> >> What are the use cases for shared mutable WebGL resources?
>> >> >>
>> >> >> David
>> >> >>
>> >> >> > On Mon, Jul 16, 2012 at 5:36 PM, Ben Vanik <benvanik@google.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> In previous systems I've worked with I've used implicit flushes
>> >> >> >> when
>> >> >> >> swapping contexts and it made life much easier. Every call in the
>> >> >> >> graphics
>> >> >> >> layer basically had this:
>> >> >> >> void MyGraphicsAPI::SomeCall() {
>> >> >> >>   MakeCurrent();
>> >> >> >>   ...
>> >> >> >> };
>> >> >> >> void MyGraphicsAPI::MakeCurrent() {
>> >> >> >>   if (threadCurrentContext != this) {
>> >> >> >>     Flush();
>> >> >> >>   }
>> >> >> >>   // ... set global threadCurrentContext, etc
>> >> >> >> };
>> >> >> >>
>> >> >> >> Using some macro-fu it was made to be a single variable lookup
>> >> >> >> and
>> >> >> >> branch
>> >> >> >> that predicted well per call, and in the common case of
>> >> >> >> single-context
>> >> >> >> apps
>> >> >> >> had little impact on performance. In apps with multiple contexts
>> >> >> >> it
>> >> >> >> made
>> >> >> >> life much easier when dealing with the sometimes long and hairy
>> >> >> >> code
>> >> >> >> sequences that touched both (rare, but often unavoidable). Had
>> >> >> >> the
>> >> >> >> flush not
>> >> >> >> been implicit it would have required a significant amount of
>> >> >> >> bookkeeping
>> >> >> >> logic that every piece of code that touched a context would have
>> >> >> >> to
>> >> >> >> interact
>> >> >> >> with - yuck. It also meant that code could be moved between apps
>> >> >> >> that
>> >> >> >> used
>> >> >> >> single contexts and multiple contexts without having to change,
>> >> >> >> or
>> >> >> >> the
>> >> >> >> app
>> >> >> >> could even decide at runtime with no ill effect. Explicit flushes
>> >> >> >> would
>> >> >> >> have
>> >> >> >> made that a nightmare.
>> >> >> >>
>> >> >> >> The one downside of the implicit flushes is that it's easy to
>> >> >> >> start
>> >> >> >> flushing all over the place without knowing about it. A simple
>> >> >> >> counter
>> >> >> >> of
>> >> >> >> flushes/frame was enough to help warn that they were happening
>> >> >> >> though,
>> >> >> >> as
>> >> >> >> the target was usually <5 and if you mess things up you would see
>> >> >> >> dozens.
>> >> >> >> It's also often much trickier to find missing explicit flushes
>> >> >> >> that
>> >> >> >> cause
>> >> >> >> subtle and sometimes very hard to identify behavioral
>> >> >> >> differences.
>> >> >> >> Just
>> >> >> >> look
>> >> >> >> at the number of WebGL pages out there today that emit warnings
>> >> >> >> and
>> >> >> >> imagine
>> >> >> >> what it'd be like with this x_x
>> >> >> >>
>> >> >> >>
>> >> >> >> On Mon, Jul 16, 2012 at 5:18 PM, Gregg Tavares (社用)
>> >> >> >> <gman@google.com>
>> >> >> >> wrote:
>> >> >> >>>
>> >> >> >>> So ........... I've been playing around with sharing resources
>> >> >> >>> across
>> >> >> >>> WebGL contexts and I'm running into issues and looking for
>> >> >> >>> solutions.
>> >> >> >>>
>> >> >> >>> The biggest issue is that GL is command buffer driven and so
>> >> >> >>> calling
>> >> >> >>> some
>> >> >> >>> GL function doesn't mean it's actually been executed. You have
>> >> >> >>> to
>> >> >> >>> call
>> >> >> >>> glFlush. This raises lots of issues of where a program might
>> >> >> >>> work
>> >> >> >>> on
>> >> >> >>> some
>> >> >> >>> browser / driver / platform combo but not others if the users
>> >> >> >>> forgets
>> >> >> >>> to
>> >> >> >>> call flush.
>> >> >> >>>
>> >> >> >>> For example, assume I have 2 contexts sharing resources. gl1,
>> >> >> >>> and
>> >> >> >>> gl2
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>   var vs = gl1.createShader(gl1.VERTEX_SHADER);
>> >> >> >>>   var fs = gl1.createShader(gl1.FRAGMENT_SHADER);
>> >> >> >>>   //...
>> >> >> >>>   // assume shaders are validly compiled
>> >> >> >>>   // ...
>> >> >> >>>   var p = gl1.createProgram();
>> >> >> >>>   gl1.attachShader(p, vs);
>> >> >> >>>   gl1.attachShader(p, fs);
>> >> >> >>>   gl2.linkProgram(p);
>> >> >> >>>
>> >> >> >>> I attached on gl1 but linked on gl2. There's is no guarantee in
>> >> >> >>> GL
>> >> >> >>> that
>> >> >> >>> that link will succeed because the 2 attach commands may have
>> >> >> >>> only
>> >> >> >>> been
>> >> >> >>> queued and not excuted in which case the linkProgram call will
>> >> >> >>> fail
>> >> >> >>> with
>> >> >> >>> "missing shaders". The correct code is
>> >> >> >>>
>> >> >> >>>   var p = gl1.createProgram();
>> >> >> >>>   gl1.attachShader(p, vs);
>> >> >> >>>   gl1.attachShader(p, fs);
>> >> >> >>>   gl1.flush(); // <--- important
>> >> >> >>>   gl2.linkProgram(p);
>> >> >> >>>
>> >> >> >>> That seems unacceptable.
>> >> >> >>>
>> >> >> >>> 2 Approaches off the of my head
>> >> >> >>>
>> >> >> >>> 1) Try to make it just work without the flush
>> >> >> >>>
>> >> >> >>> One solution might be for the implementation to track which
>> >> >> >>> context
>> >> >> >>> last
>> >> >> >>> had a call. Any call to a different context causes an automatic
>> >> >> >>> implicit
>> >> >> >>> flush
>> >> >> >>>
>> >> >> >>> 2) Try to make it fail when not done correctly
>> >> >> >>>
>> >> >> >>> This solution would be to try to track of an object is "dirty"
>> >> >> >>> (state
>> >> >> >>> has
>> >> >> >>> been changed) and no flush has been issued since then for that
>> >> >> >>> object.
>> >> >> >>> When
>> >> >> >>> an object is used, if it's dirty then either a flush is issued
>> >> >> >>> (which
>> >> >> >>> is
>> >> >> >>> solution #1) or a error is generated. ("called function on dirty
>> >> >> >>> object, did
>> >> >> >>> you forget to call flush?")
>> >> >> >>>
>> >> >> >>> Neither solution seems ideal. Worse, whatever solution is chosen
>> >> >> >>> also
>> >> >> >>> has
>> >> >> >>> issues if we ever get WebGL in Workers.
>> >> >> >>>
>> >> >> >>> A 3rd solution is just to leave it with the flush required and
>> >> >> >>> forgetting
>> >> >> >>> means you get random success / failure. No such a great
>> >> >> >>> prospect.
>> >> >> >>> :-(
>> >> >> >>>
>> >> >> >>> Thoughts?
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>
>

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------