[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] texImage2D on unfinished images and video



On Thu, Nov 1, 2012 at 12:00 PM, Gregg Tavares (社用) <gman@google.com> wrote:
>
>
>
> On Thu, Nov 1, 2012 at 11:27 AM, Kenneth Russell <kbr@google.com> wrote:
>>
>> On Thu, Nov 1, 2012 at 9:05 AM, Chris Marrin <cmarrin@apple.com> wrote:
>> >
>> > On Oct 31, 2012, at 3:15 PM, Kenneth Russell <kbr@google.com> wrote:
>> >
>> >>
>> >>>
>> >>> Also, while an unloaded video/image could return 1x1, off the top of
>> >>> my
>> >>> head, what it returns while changing resolutions or queuing the next
>> >>> video
>> >>> could be implementation defined. Some browsers might return 1x1 while
>> >>> switching. Others might return the last valid frame until such time as
>> >>> a
>> >>> frame from the new source is ready. As long as there is no error that
>> >>> seems
>> >>> better to me.
>> >>
>> >> It seems that the consensus is to change the WebGL specification as you
>> >> suggest.
>> >>
>> >> The changes would essentially be:
>> >>
>> >>  - If an incomplete HTMLImageElement or HTMLVideoElement is passed to
>> >> either texImage2D or texSubImage2D, then the source is treated as a
>> >> 1x1 texture containing the RGBA value (0, 0, 0, 1). This means that
>> >> texImage2D will cause the level of the passed texture object to be
>> >> redefined as size 1x1; texSubImage2D will upload a single black pixel
>> >> at the given xoffset and yoffset. It would be defined that an OpenGL
>> >> error would never occur as a result of these calls.
>> >>
>> >> Are there any objections to these changes? If not, I'll update the
>> >> spec.
>> >
>> > It would be really nice if we could handle incremental image downloads.
>> > To do this, we'd need some way to know "enough of the image is downloaded
>> > that it's size is accurate and it has something (perhaps low res) to
>> > display", folllowed by "more of the image is available" events and then an
>> > "image complete" event.
>> >
>> > Maybe that's a separate issue, but it would be a really useful thing to
>> > support.
>>
>> Another idea which came up during offline conversations:
>>
>> It would be a backward-compatible change (from applications' point of
>> view) to change the return type of the texImage2D and texSubImage2D
>> overloads taking HTMLImageElement and HTMLVideoElement (and, I
>> suppose, HTMLCanvasElement) to return some sort of status code. We
>> could indicate whether the element was fully uploaded, whether it was
>> only partially uploaded due to progressive downloads, whether there
>> wasn't enough information to do any data uploading, etc.
>>
>> That way, applications could still determine what had happened
>> programmatically, without querying for any OpenGL errors.
>
>
> I like that idea.
>
> Another thought, separating the 2 issues.
>
> (1) what to do before the image is completely loaded
>
> sounds like no-op is the plan. Implementations can provide a console warning
> if they want. We can move forward with this change separate from issue 2
>
> (2) what to do, if anything, for progressive loading
>
> One idea, gl.pixelStorei(gl.UNPACK_ALLOW_PROGRESSIVE_IMAGE_UPLOADS, true);
>
> If you're worried that developers might get unexpected results otherwise. I
> don't know if that's needed or not if we ever decide to support progressive
> uploading. Note the canvas spec currently requires the image to be 100%
> downloaded. In other words it does NOT support progressive uploading.
>
> Unfortunately, like many specs, it's poorly worded making it semi ambiguous
> :-(
>
> 4.8.11.1.13 says
>
> If the image argument is an HTMLImageElement object that is not fully
> decodable,
>
>
> "fully decodable" says
>
> When an img element is in the completely available state and the user agent
> can decode the media data without errors, then the img element is said to be
> fully decodable.
>
>
> "completely available" says
>
> The user agent has obtained all of the image data and at least the image
> dimensions are available.
>
>
> "completely available" part seems ambiguous to me since the part "at least
> the image dimensions are available" makes no sense given that a corrupt
> image is covered by "fully decodable". Of course the image dimensions are
> available if all the data is downloaded so why even have that statement?
> Where as the image dimensions might be available if it is partially
> downloaded which suggests to me at least support for partially downloaded
> images was the original intent.
>
> ...sigh...

It looks to me like the 2D Canvas API was specifically not designed to
draw incomplete images. "completely available" covers the situation
where all of the data has come down from the server, but the file
itself was truncated or corrupt, and so the file is actually "broken".
Canvas 2D deals specifically with fully downloaded images not
containing any errors.

http://www.whatwg.org/specs/web-apps/current-work/multipage/embedded-content-1.html#update-the-image-data
seems designed to support incremental rendering of progressive images:
"If the resource is in a supported image format, then each task that
is queued by the networking task source while the image is being
fetched must update the presentation of the image appropriately".

Adding another pixel unpack parameter like
gl.UNPACK_ALLOW_PROGRESSIVE_IMAGE_UPLOADS_WEBGL sounds better than
automatically starting to support incremental texture uploads to
WebGL.

We need to think through all of the return codes from texImage2D and
texSubImage2D:

 - Complete success
 - Success of partial image upload
 - Failed, image dimensions not known
 - Failed, image data incomplete and progressive uploads disabled
 - Failed, image data corrupt
 - ???

Do we need to distinguish between the case where more data is expected
and the case where all the image's data is downloaded? Two status
codes, partially available/fully available and something indicating
success or failure due to having or not having image metadata and
image data? Argh.

-Ken

-----------------------------------------------------------
You are currently subscribed to public_webgl@khronos.org.
To unsubscribe, send an email to majordomo@khronos.org with
the following command in the body of your email:
unsubscribe public_webgl
-----------------------------------------------------------