AF Manager class for dynamic animation frame request/cancel by phoenixbf · Pull Request #1407 · NASA-AMMOS/3DTilesRendererJS

phoenixbf · 2025-12-13T12:36:31Z

gkjohnson

Great, thanks! A couple comments but I think this is the right direction

gkjohnson · 2025-12-14T02:56:52Z

@@ -0,0 +1,43 @@
+class AFManager {


Lets call this FrameScheduler for now.

gkjohnson · 2025-12-14T03:04:26Z

+	setXRSession( xrsession ) {
+
+		this.xrsession = xrsession;
+
+	}
+
+	removeXRSession() {
+
+		this.xrsession = null;
+
+	}


When changing the system being used for callbacks we will have to cancel all previous callbacks and reregister them with the new function. Eg if the "LRUCache" schedules an unload event it will be waiting until that event fires before scheduling another one. So if the last event is scheduled using the "xrSession" callback and then the session closes, the scheduled callback will never be fired.

So we'll need to keep track of all scheduled callbacks and rAF handles to swap over the scheduled events.

gkjohnson · 2026-01-09T02:26:35Z

@phoenixbf - just wanted to check in to see if you had any more questions on the changes needed here.

phoenixbf · 2026-01-10T09:26:48Z

@phoenixbf - just wanted to check in to see if you had any more questions on the changes needed here.

Hi, performing some local testing in order to have a clean rAF tracking system and handling swaps

phoenixbf · 2026-01-10T21:37:42Z

I've added management of pending AF.
It basically keeps track of requested AFs via handle as suggested, providing methods to cancel or flush all pending AFs. When entering XR session, it internally flushes and completes all pending, switching to this.xrsession.requestAnimationFrame for new requests.

So far I tried the above logic in both local and online setups without any evident issues, also with mutiple tsets. If this is the right direction, it should be probably instantiated in TilesRendererBase and then referenced by LRUCache, PriorityQueue, etc. during their initializations

gkjohnson · 2026-01-11T00:50:04Z

Thanks the direction of this looks great. I think we can cut down on some code redundancy in the implementation but we can deal with that once everything is working.

If this is the right direction, it should be probably instantiated in TilesRendererBase and then referenced by LRUCache, PriorityQueue, etc. during their initializations

This sounds good to me. Looks like the "LRUCache", "PriorityQueue", the "throttle" function, and "QueryManager" are all places where rAF is used (though the throttle function can maybe be changed to "queueMicrotask")

…ryManager; updated Dingo Gap VR sample

gkjohnson

Looking good! Made a few comments -

gkjohnson · 2026-01-19T22:51:48Z


+	setXRSession( xrsession ) {
+
+		if ( ! this.framescheduler ) return;


this should never be "null" so we should be able to remove this check.

gkjohnson · 2026-01-19T22:52:28Z

 		this.isLoading = false;

+		// FrameScheduler referenced by LRUCache, PriorityQueue
+		this.framescheduler = new FrameScheduler();


lets use camel casing to align with the class naming:

this.framescheduler -> this.frameScheduler

gkjohnson · 2026-01-19T22:53:45Z


 		this._unloadPriorityCallback = null;

+		this.framescheduler = null;


Lets initialize these to new FrameScheduler() so the classes are in a usable state when constructed. Then we can remove the null checks, as well. This should help the tests pass, too.

- remove unused function - remove use of "window" - ensure "flushPending" clears all pending handles - code style update

gkjohnson · 2026-01-22T23:34:40Z

I've just made some of the remaining changes to fix tests, add new tests for FrameScheduler, and address some code styling and other issues. I think this should be ready to merge but if you could test it in a real XR environment that would be great.

phoenixbf · 2026-01-23T07:10:50Z

I've just made some of the remaining changes to fix tests, add new tests for FrameScheduler, and address some code styling and other issues. I think this should be ready to merge but if you could test it in a real XR environment that would be great.

Yep, sure. Today I'll carry out additional online tests with a few HMDs

h0ldemslav · 2026-03-17T11:09:38Z

The ImageOverlayPlugin creates a "PriorityQueue" internally so that would need to share a FrameScheduler, as well. The more of these changes that need to be made, though, the more brittle this solution seems. I'm wondering if we should just provide a global FrameScheduler that everything shares by default so these issues aren't encountered.

One other potential case that comes to mind: Is there ever a case where the XR Session is "paused" (as in not rendering or updating) when a separate window is open? It seems like there could be a case where the XR Session has initialized but the session "rAF" has stopped firing and the window "rAF" starts firing again, instead. Is this something you're able to reproduce? I'm wondering how to be resilient to this.

Not sure if I understand the last paragraph. I mean rAF is fired always, even if you have other windows in Meta quest open. What's interesting is that, both window.requestAnimationFrame and xrsession.requestAnimationFrame are fired by the frame scheduler, whether your session has visibilityState "visible" or "visible-blurred".

I made a video to illustrate the problem. Have a look at it: https://drive.google.com/file/d/12ZfhMkAjkJPxaYLlh30wFM-1iiOrX0Fw/view?usp=sharing (cannot attach to GitHub, size exceeds GitHub limits). I'm not good at it but I can make more videos, if you need, feel free to give me concrete instructions.

In the video, you can see how the imagery is not applied at all and how there are very few calls to both rAF, when I was in VR, or better say visibilityState was "visible". Once I press the meta button on the controller, the session visibilityState changes to "visible-blurred" and you can see how many rAF are fired (both window and xrsession). Once I toggle the state (from "visible" to "visible-again"), there are very few rAF calls.

gkjohnson · 2026-03-17T13:48:11Z

@phoenixbf

A FrameScheduler is indeed needed at the moment to handle and delegate to the correct rAF routine. Where do you imagine this happening and how? A singleton?

Yeah lets just make this a singleton for now. I'm open to suggestions but we can manage it like so:

import { FrameScheduler } from '3d-tiles-renderer';

FrameScheduler.requestAnimationFrame( ... );
FrameScehduler.setXRSession( ... )

@h0ldemslav

Not sure if I understand the last paragraph. I mean rAF is fired always, even if you have other windows in Meta quest open. What's interesting is that, both window.requestAnimationFrame and xrsession.requestAnimationFrame are fired by the frame scheduler, whether your session has visibilityState "visible" or "visible-blurred".

The window's "requestAnimationFrame" will have to fire when the "window" is visible so content in just the window, like DOM, can be updated. And the xr session rAF will have to be fired when the XR session is presenting so content can be updated for the XR view. If the XR rAF didn't fire while the background was rendering then it would just freeze and the camera view would not be able to update, either. So what we're seeing are the following situations when the presenting === true (the xr session is running):

	xr rAF fires	window rAF fires
immersive view	true	false
window is up with immersive background	true	true

What I'm wondering is if there is every a situation we can get into where the "presenting" is true (eg sessionstart has fired & sessionend has not) and the window rAF is firing while the xr session rAF is not (this would be a line with "false | true" on the above table). If we think there is or there may be in the future then we should make sure we account for this in our FrameSchduler implementation. Otherwise this will happen again and it's quite a hard issue to understand the track down.

Relatedly, I'd appreciate it if you could make any issue at the meta emulator and link it here so we can more easily understand and test this behavior:

I would also recommend making an issue for Meta's web emulator to request that the rAF callback behavior be made consistent with the real hardware (or better add a toggle so different device / browser behaviors can be emulated).

--

I made a video to illustrate the problem. Have a look at it: https://drive.google.com/file/d/12ZfhMkAjkJPxaYLlh30wFM-1iiOrX0Fw/view?usp=sharing (cannot attach to GitHub, size exceeds GitHub limits)

I can't read any of the text in this video with the compression, unfortunately, so I can't fully understand what its showing.

h0ldemslav · 2026-03-17T16:43:44Z

The window's "requestAnimationFrame" will have to fire when the "window" is visible so content in just the window, like DOM, can be updated. And the xr session rAF will have to be fired when the XR session is presenting so content can be updated for the XR view. If the XR rAF didn't fire while the background was rendering then it would just freeze and the camera view would not be able to update, either. So what we're seeing are the following situations when the presenting === true (the xr session is running):

I see, now that makes much more sense. Thanks a lot!! Your table basically summarized what I did in the video. Though some window.rAF are called even in the immersive view (perhaps this is some other part of the library). I don't know why Google drive still hasn't provided a better quality video.

Anyway, I'm not sure if one can reproduce a situation where the renderer.xr is presenting, the window.rAF is fired, while xrsession.rAF is not fired. (I can't really think of any good example) If you look at the WebXR specification, 4.3 (links to the highlighted part), they discourage to use window.rAF in the context of xrsession because "the timing of callbacks of window.rAF may not coincide with xrsession.rAF". Though there is some sort of the pattern they suggest using for "transition between these two types of animation loops ".

So, I would rule out the use case where you solely use window.rAF while renderer.xr presenting. I think referring to the WebXR specification is a good justification. However, I'm thinking that maybe the frame scheduler could also use a third option, setTimeout(cb, 16) as a wilcard option that would suffice anyway, though someone has already mentioned in the issues that tiles start flickering if settimeout is used. What do you think?

gkjohnson · 2026-03-18T00:40:08Z

However, I'm thinking that maybe the frame scheduler could also use a third option, setTimeout(cb, 16) as a wilcard option that would suffice anyway

I'm not sure what you mean by this? The classes using frame scheduler need to run every frame and setTimeout is not guaranteed to do so. It may run multiple times per frame or not at all depending on the platform and the framerate.

phoenixbf · 2026-04-01T11:57:35Z

Hi there,
I've committed a revised FrameScheduler that how behaves as singleton. If the current proposal sounds good, it should be now possible to use this anywhere:

import { FrameScheduler } from '../utilities/FrameScheduler.js';

let fs = new FrameScheduler(); // this now behave as singleton

// do stuff with fs
fs.setXRSession(...);
fs.requestAnimationFrame(...);

gkjohnson · 2026-04-02T07:21:49Z

Thanks @phoenixbf - I'll take a deeper look this weekend and make a few more changes I've had in the back of my mind.

It looks like one of the tests is failing, though. Would you be able to take a look?

gkjohnson · 2026-04-03T09:00:22Z

I've made a few changes - primarily changing the class to use all static functions and using it as a singleton everywhere. The class has also been renamed to "Scheduler" and I've removed the "setXRSession" function from "TilesRenderer" so you set it like so:

import { Scheduler } from '3d-tiles-renderer';
Scheduler.setXRSession( session );

I'll merge this but if you could test it and follow up if there are any issues that would be great! Thanks again for your help in tracking down the issue and coming up with a solution.

phoenixbf · 2026-04-03T13:58:26Z

I've made a few changes - primarily changing the class to use all static functions and using it as a singleton everywhere. The class has also been renamed to "Scheduler" and I've removed the "setXRSession" function from "TilesRenderer" so you set it like so:
import { Scheduler } from '3d-tiles-renderer';
Scheduler.setXRSession( session );
I'll merge this but if you could test it and follow up if there are any issues that would be great! Thanks again for your help in tracking down the issue and coming up with a solution.

Thanks, I'll test these ASAP on my XR equipment

phoenixbf · 2026-04-13T10:20:29Z

just to confirm I tested several online tilesets on different equipment (Meta Quest 2, Meta Quest Pro and Hololens 2) without issues

phoenixbf added 4 commits December 13, 2025 13:32

AFManager class for dynamic animation frame request/cancel

74aed37

Merge branch 'NASA-AMMOS:master' into master

59e1230

spacing

eafb6b7

spacing

e4b313f

gkjohnson reviewed Dec 14, 2025

View reviewed changes

rename class

ecffd60

phoenixbf added 3 commits January 10, 2026 20:56

Merge branch 'NASA-AMMOS:master' into master

a7fb47b

Add pending AF management

0cf8ae5

minor comment

3fa5aae

phoenixbf added 3 commits January 18, 2026 12:34

Merge branch 'NASA-AMMOS:master' into master

6079607

spacing

f6eb5eb

FrameScheduler referenced by LRUCache,PriorityQueue, throttle and Que…

d78ab44

…ryManager; updated Dingo Gap VR sample

gkjohnson reviewed Jan 19, 2026

View reviewed changes

gkjohnson added this to the v0.4.20 milestone Jan 19, 2026

phoenixbf and others added 7 commits January 20, 2026 08:44

camel casing; intialization in LRU, PriorityQueue and QueryManager

f8ab031

Merge branch 'NASA-AMMOS:master' into master

edeab4b

- replace "removeXRSession" with "setXRSession( null )"

743dc61

- remove unused function - remove use of "window" - ensure "flushPending" clears all pending handles - code style update

Add tests for FrameScheduler

c3c0051

small cleanup

72e0260

comment

ca0a22f

variable rename

c65fbc7

gkjohnson modified the milestones: v0.4.20, v0.4.21 Feb 1, 2026

Merge branch 'NASA-AMMOS:master' into master

8182160

gkjohnson modified the milestones: v0.4.23, v0.4.24 Mar 18, 2026

Merge branch 'NASA-AMMOS:master' into master

fabb9fb

h0ldemslav mentioned this pull request Mar 31, 2026

Inconsistency in xrSession.rAF behavior between the emulator and Meta Quest Pro meta-quest/immersive-web-emulator#78

Closed

phoenixbf added 2 commits April 1, 2026 00:20

initial commit for singleton framescheduler

88f31e5

fix spaces

f32931f

Merge branch 'NASA-AMMOS:master' into master

b90075d

gkjohnson mentioned this pull request Apr 2, 2026

Use "requestIdleCallback" for tile processing #1547

Open

gkjohnson modified the milestones: v0.4.24, v0.4.25 Apr 3, 2026

gkjohnson added 6 commits April 3, 2026 17:43

Make the FrameScheduler a singleton

45850e2

Fix tsts

45ec014

Update docs

4e89c4c

FrameScheduler -> Scheduler

47eb241

Update the vrDemo

db0422d

Add d.ts

0fece78

gkjohnson merged commit 9975c04 into NASA-AMMOS:master Apr 3, 2026
1 check passed

gkjohnson modified the milestones: v0.4.25, v0.4.24 Apr 3, 2026

gkjohnson mentioned this pull request Apr 3, 2026

3D tiles are not updated during VR session #1342

Closed

AnthonyGlt mentioned this pull request Apr 3, 2026

[PROPOSAL] Check the new VR implem of 3DTilesRenderer iTowns/itowns#2735

Open


		setXRSession( xrsession ) {

		if ( ! this.framescheduler ) return;


		this._unloadPriorityCallback = null;

		this.framescheduler = null;

Conversation

phoenixbf commented Dec 13, 2025

Uh oh!

gkjohnson left a comment

Choose a reason for hiding this comment

Uh oh!

gkjohnson Dec 14, 2025

Choose a reason for hiding this comment

Uh oh!

gkjohnson Dec 14, 2025

Choose a reason for hiding this comment

Uh oh!

gkjohnson commented Jan 9, 2026

Uh oh!

phoenixbf commented Jan 10, 2026

Uh oh!

phoenixbf commented Jan 10, 2026

Uh oh!

gkjohnson commented Jan 11, 2026

Uh oh!

gkjohnson left a comment

Choose a reason for hiding this comment

Uh oh!

gkjohnson Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

gkjohnson Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

gkjohnson Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gkjohnson commented Jan 22, 2026

Uh oh!

phoenixbf commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

h0ldemslav commented Mar 17, 2026

Uh oh!

gkjohnson commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

h0ldemslav commented Mar 17, 2026

Uh oh!

gkjohnson commented Mar 18, 2026

Uh oh!

phoenixbf commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gkjohnson commented Apr 2, 2026

Uh oh!

gkjohnson commented Apr 3, 2026

Uh oh!

Uh oh!

phoenixbf commented Apr 3, 2026

Uh oh!

phoenixbf commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gkjohnson Jan 19, 2026 •

edited

Loading

phoenixbf commented Jan 23, 2026 •

edited

Loading

gkjohnson commented Mar 17, 2026 •

edited

Loading

phoenixbf commented Apr 1, 2026 •

edited

Loading