Change blur/focus to visibilitychange by toji · Pull Request #687 · immersive-web/webxr · GitHub
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change blur/focus to visibilitychange #687

Merged
merged 8 commits into from
Jun 13, 2019
Merged

Change blur/focus to visibilitychange #687

merged 8 commits into from
Jun 13, 2019

Conversation

toji
Copy link
Member

@toji toji commented Jun 6, 2019

A proposal, based on discussion at the F2F.

Fixes #488
Fixes #468

Changes the blur and focus events to visibilitychange, along with
a visibilityState attribute on the XRSession. The visibilityState
may be visible, hidden, or background. This is pretty close to the
document.visibilityState attribute and associated event, with the
exception of the background state.

@toji toji requested a review from NellWaliczek June 6, 2019 16:18
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated
@@ -1958,7 +1968,7 @@ Gaze Tracking {#gazetracking-security}

While the API does not yet expose eye tracking capabilities a lot can be inferred about where the user is looking by tracking the orientation of their head. This is especially true of XR devices that have limited input capabilities, such as Google Cardboard, which frequently require users to control a "gaze cursor" with their head orientation. This means that it may be possible for a malicious page to infer what a user is typing on a virtual keyboard or how they are interacting with a virtual UI based solely on monitoring their head movements. For example: if not prevented from doing so a page could estimate what URL a user is entering into the user agent's URL bar.

To prevent this risk the user agent MUST [=blur all sessions=] when the users is interacting with sensitive, trusted UI such as URL bars or system dialogs. Additionally, to prevent a malicious page from being able to monitor input on a other pages the user agent MUST [=blur all sessions=] on non-focused pages.
To prevent this risk the user agent MUST set the [=visibility state=] of all {{XRSession}}s to {{XRVisibilityState/hidden}} or {{XRVisibilityState/background}} with throttled frame production when the users is interacting with sensitive, trusted UI such as URL bars or system dialogs. Additionally, to prevent a malicious page from being able to monitor input on a other pages the user agent MUST set the [=visibility state=] of all {{XRSession}}s on non-focused pages to {{XRVisibilityState/hidden}}.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting - is the idea then that during background an app's frames are not being displayed, but it should keep its render loop going? Or the idea that during background the app is throttled to 5 fps, counting on reprojection to keep the view reasonable? Do we have a belief that UAs can throttle enough to eliminate the privacy risk without causing users to be uncomfortable?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or sometimes background is during a sensitive overlay (e.g. 5 fps) and sometimes background is during a non-sensitive overlay (e.g. 60/90 fps)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. This is to allow for UAs to have some flexibility in how they display sensitive content AND to let UAs take advantage of some platforms features where the content is still visible even when the input focus is elsewhere. Primary example of that I can think of is Steam VR, which continues to display the app (lower res and greyed out, but still live) when you pull up the Steam overlay.
2019-06-07-PM_04_11_46 This is a situation in which the app would be put in that state but still pump frames at ~full speed.

You could also picture a scenario where the app is placed in that state but rendering is severely throttled to the point where it may only render intermittently to patch up around reprojection or even just process one frame to make a cubemap. This could be either for privacy reasons OR simply because the device lacks the horsepower to render two realtime stereo scenes at once. I don't believe the API should differentiate between those scenarios to the developer. Whether or not the UA is comfortable with that being sufficient to preserve privacy is up to them, though the hidden state is always available as a last resort.

@toji
Copy link
Member Author

toji commented Jun 7, 2019

Updated to take into account feedback from Alex.

@darktears
Copy link
Contributor

LGTM

@toji toji added this to the June 2019 milestone Jun 10, 2019
Copy link
Member

@NellWaliczek NellWaliczek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the direction! A few smallish comments, but also... explainer updates? ;)

index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Show resolved Hide resolved
explainer.md Outdated Show resolved Hide resolved
explainer.md Outdated Show resolved Hide resolved
explainer.md Outdated Show resolved Hide resolved
@toji toji force-pushed the visibility-state branch from cf9a314 to a804fc4 Compare June 10, 2019 22:05
@toji
Copy link
Member Author

toji commented Jun 10, 2019

Done, done, and done. Thanks!

Copy link
Member

@NellWaliczek NellWaliczek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looking good! Two main comments are: a clarification question about throttling as the only acceptable mitigation and a desire to have normative text explaining the expected relationship between the page visibility and the session visibility on various hardware form factors.

index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
toji added 7 commits June 13, 2019 13:18
Fixes #488
Fixes #468

Changes the `blur` and `focus` events to `visibilitychange`, along with
a `visibilityState` attribute on the `XRSession`. The `visibilityState`
may be `visible`, `hidden`, or `background`. This is pretty close to the
`document.visibilityState` attribute and associated event, with the
exception of the `background` state.
@toji toji force-pushed the visibility-state branch from b1bc201 to 7fe3694 Compare June 13, 2019 21:37
Copy link
Member

@NellWaliczek NellWaliczek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@toji toji merged commit b88c9d2 into master Jun 13, 2019
@toji toji deleted the visibility-state branch June 13, 2019 23:04
@AdaRoseCannon AdaRoseCannon added ed:explainer Include in newsletter, explainer change ed:spec Include in newsletter, spec change labels Jun 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ed:explainer Include in newsletter, explainer change ed:spec Include in newsletter, spec change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"focus"/"blur" naming on XRSession may be confusing Document effects when session is blurred
6 participants