[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Public WebGL] WEBGL_dynamic_texture redux



On 2012/11/22 11:15, David Sheets wrote:


  //   // readonly int freq; ? do videos get one per channel? only max
frequency of all media streams?
What would you use this for? Without knowing the use case, the obvious
answer is the frequency (framerate) of the producer video stream. However
several modern video formats do not have a fixed framerate.
I would use this for relating frame counts to
latencies/timeouts/clocks. Specifically, without knowing the inherent
maximum frequency of the producer, the consumer must guess its frame
budget initially and back-off causing A/V glitches.
I agree this would be useful for exactly the reasons you cited. Unfortunately web browsers do not make the frame-rate available to scripts even though most, if not all, container formats store this information. For example WebM supports the same FrameRate item as the Matroska container.

Ideally we would persuade browser vendors/WHATWG to add it as an attribute of the HTMLVideoElement. Alternatively, since there may not be any uses outside WebGL, we could add it to the WebGLDynamicTextureSource interface, if we go that route.

Are you planning on exposing the variable framerate of decoders to
stream consumers in the msc? Is it bad to insert "duplicate" frames
and increment the msc when the source framerate drops below max? I
think consistency is valuable here and simplifies the author's work.
If I test my generic video app with constant framerate videos but not
variable framerate videos, I might be disappointed with performance on
sophisticated codecs in future. Can this consistency abstraction leak
in some way I am missing?
I have come to the opinion that msc is not a useful concept for a variable-frame-rate video source. I'm leaning towards exposing the time at which the frame is expected to be displayed.

I'll come back to your other questions in a separate message. I want to keep this focused on  interface for hooking up producers.

I am thinking of the following way of setting up a dynamic texture modeled on your WebGLDynamicTextureSource proposal.

  // Use object for "global" variables to avoid polluting the global space.
  var g = {};

  // Array of files currently loading
  g.loadingFiles = [];

    gl.dte = gl.getExtension("WEBGL_dynamic_texture");
    g.videoTexture =
{
      texture: gl.createTexture(),
      video: createVideoElement(),
      stream: null
    };
    // gl.activeTexture(gl.TEXTURE0);
    gl.bindTexture(gl.TEXTURE_EXTERNAL_OES, g.videoTexture.texture);
    // Create stream with currently bound texture as consumer.
    g.videoTexture.stream = gl.dte.createStream();
    connectStreamToProducerAndSource(stream, video, "http://myserver/myvideo.mp4");

where createVideoElement() is

  // Create a hidden video element and append to the document
  // XXX Is it necessary to append the element?
  function createVideoElement() {
    var video = document.createElement('video');
    video.autoplay = true;
    video.controls = false;
    video.hidden = true;
    document.body.appendChild(video);
    return video;
  }

Note, creation of the video element was missing from the example previously.

connectStreamToProducerAndSource() is

  function connectStreamToProducerAndSource(stream, video, url) {
    video.>function() {
      g.loadingFiles.splice(g.loadingFiles.indexOf(video), 1);
    };
    video.>function() {
      g.loadingFiles.splice(g.loadingFiles.indexOf(video), 1);
      // Notify user? Abort application?
      // video.error gives the reason.
    };
    g.loadingFiles.push(video);
    video.wssConnectToSourceAsProducer(url, stream);
  }

You first create the stream with its consumer then pass that to wssConnectToSourceAsProducer method that is part of the interface added to HTMLVideoElement. I'd like to hear from the WebIDL and browser experts whether something like

interface WebGLStreamSource {
    void wssConnectToSourceAsProducer(DomString url, WebGLStream stream);
    void wssDisconnectFromStream();
    unsigned long wssFrameRate;
};
HTMLVideoElement implements WebGLStreamSource;

is acceptable.

Suggestions for better names are welcome.

Behind this are the concepts that

Having a standard interface for producers is attractive but I'm unsure how best to append this to existing elements as I'm lacking both _javascript_ and WebIDL expertise.

Opinions?

Regards

    -Mark

--
注意:この電子メールには、株式会社エイチアイの機密情報が含まれている場合が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情報の使用を固く禁じております。エラー、手違いでこのメールを受け取られましたら削除を行い配信者にご連絡をお願いいたし ます.

NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies.