Should XRWebGLLayers clear/invalidate their framebuffers prior to each frame · Issue #775 · immersive-web/webxr · GitHub
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should XRWebGLLayers clear/invalidate their framebuffers prior to each frame #775

Closed
toji opened this issue Jul 22, 2019 · 14 comments · Fixed by #866
Closed

Should XRWebGLLayers clear/invalidate their framebuffers prior to each frame #775

toji opened this issue Jul 22, 2019 · 14 comments · Fixed by #866
Labels
help wanted This is a good issue for anyone to pick up and work on filing a PR for.
Milestone

Comments

@toji
Copy link
Member

toji commented Jul 22, 2019

I'm relatively certain that we've had this conversation before and that the consensus was "yes, they should" but I can't find any record of it (which is a fail on my part) so let's get this done properly!

We should explicitly state whether or not framebuffers provided by XRWebGLLayer (and in the future similar layer sources) should be invalidated or cleared by the WebXR implementation prior to each frame. My recollection is that on some platforms the target buffers provided by the native API will be invalidated by default, and as such forcing web apps to do an explicit clear each frame "just in case" is a waste of fillrate in a situation where we have very little to spare. As such the spec should probably just specify that framebuffers are "ready to go" at the start of each frame so that developers can skip the clear step with confidence.

Any objections?

(I definitely recall that we made the explicit decision to leave the preserveDrawingBuffer option off of the XRWebGLLayer for similar reasons, so we don't ever need to worry about needing to preserve the content of the buffer for more than a frame.)

@toji
Copy link
Member Author

toji commented Jul 22, 2019

Oh, and to be clear: I don't see this as a backwards compatibility-breaking change. If we decided that yes, everything should be cleared then the worst that we do in introduce some unnecessary overhead for early apps that assumed a clear was necessary, but no content will break.

@klausw
Copy link
Contributor

klausw commented Jul 22, 2019

Somewhat related, if the spec adds new optional features that supply color or depth buffers to the app, i.e. pose-aligned AR camera frames or occlusion data, would this potentially use prepopulated attachments to the webxr framebuffer? That's a separate decision, but I think that requiring UA-provided ready-to-use framebuffers and discouraging apps from calling glClear would help add flexibility for this in the future.

For background (no pun intended), the experimental legacy-inline-ar implementation in Chromium had been using a color buffer prefilled with camera data instead of being blank, so a glClear from the app would break AR mode by erasing the camera image. The current immersive-ar implementation no longer does this since it composites the app-drawn image as a separate step, and adding camera frame access may be more useful as a separate texture anyway, but a hypothetical prepopulated depth buffer might be most convenient if it's directly available without copying.

@toji toji added this to the August 2019 milestone Jul 31, 2019
@toji toji added agenda Request discussion in the next telecon/FTF help wanted This is a good issue for anyone to pick up and work on filing a PR for. labels Jul 31, 2019
@asajeffrey
Copy link

For the Servo implementation of WebXR, it doesn't make a huge amount of difference whether user code calls clear or the device driver does. I'm not sure which clear colour would be used if it's the device driver though, are we going to say that it's the clear colour of the WebGL context for the WebXR content?

@toji
Copy link
Member Author

toji commented Aug 12, 2019

I think we'd want to specify it as transparent or opaque black, depending on whether or not the framebuffer has an alpha channel.

@asajeffrey
Copy link

That would certainly be the easiest to implement.

@RafaelCintron
Copy link

RafaelCintron commented Aug 26, 2019

I am supportive of the proposal: XRWebGLLayers should clear/invalidate their framebuffers prior to each frame.

@Artyom17
Copy link

Artyom17 commented Aug 27, 2019

But let separate clear from invalidation. What I think we should do is to INVALIDATE both color and depth buffers at the end of the frame, and this must be mandatory. Clear is another story and it could be a waste of GPU if I still need to re-clear it to something different from (0,0,0,0).

@kearwood
Copy link
Contributor

As per my comments in the last call, I would suggest that if we do define the buffers being cleared, we should also define what happens with the depth and stencil buffers.

For the depth in particular, there is no single value that will be ideal for all rendering engines, but the value that it is cleared to can be expected to be the same for all frames.

Many render engines will prefer an inverted depth buffer in order to reduce depth sorting artifacts, by interpreting "1" as the near plane and "0" as the far plane. Simpler WebGL samples are likely to use "0" as the near plane and "1" as the far plane, as this is the traditional OpenGL format.

For background on this particular optimization:

https://developer.nvidia.com/content/depth-precision-visualized

@kearwood
Copy link
Contributor

IMHO, it would be okay to let the depth and stencil be cleared by the platform; however, the value that it is cleared to should be configurable (at least allow it to be set once at XRSession initialization).

Please note that clearing the depth buffer redundantly has implications other than consuming fill rate. On tile-based-deferred rendering GPU's (most modern mobile GPU's), this operation also resets acceleration structures used to perform early culling operations. If done incorrectly or at the incorrect time during a frame, early culling can be effectively disabled, resulting in greater fragment shading costs and increased fill in subsequent render passes.

@RafaelCintron
Copy link

I agree with @Artyom17 that the browser should avoid needless clears by only clearing buffers when the developer chooses not to clear it themselves.

The only buffers the WebXR API should lazy clear are ones it specifically hands out to web developers. In the future, the plan is to have XR WebGL layers supply just a texture array to the web developer that they can use in any framebuffer of their choosing. So unless the depth buffer is supplied by the XR API for enhanced LSR purposes, I do not think we should get in the business of lazy clearing random buffers behind the developer's back.

We should use the clear values WebGL already defines as initial values. For depth, the lazy clear value is 1.0.

@cabanier
Copy link
Member

The only buffers the WebXR API should lazy clear are ones it specifically hands out to web developers. In the future, the plan is to have XR WebGL layers supply just a texture array to the web developer that they can use in any framebuffer of their choosing.

I have not heard about such a proposal. Is this for a new WebXR spec?

@RafaelCintron
Copy link

@cabanier , the issue I was referring to was the Use Texture Arrays with WebGL 2 one. It will be part of the layers module.

@toji toji removed the agenda Request discussion in the next telecon/FTF label Sep 10, 2019
@asajeffrey
Copy link

I'm going to back away from what I said earlier about it not making a difference for Servo... We're thinking of moving from an implementation that uses one texture which is locked during compositing, to an implementation that uses double-buffering (or triple-buffering, details still under construction). This is fine if the layer is cleared on each rAF loop, not so fine if it's required to keep state.

@toji
Copy link
Member Author

toji commented Oct 2, 2019

Put up a pull request to fix this at #866. Don't think it'll be controversial, but please take a look!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted This is a good issue for anyone to pick up and work on filing a PR for.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants