Regression with missing download progress notification for PBS

[rorbech]

Expected results

Progress notifications for PSB seems to have changed when there is currently no known outstanding downloads.

In 14.5.1 and below it was possible to implement a quite simple "progress spinner"-experience with code like

val realm = Realm.open(...)
realm.syncSession.progressAsFlow(Direction.DOWNLOAD, ProgressMode.CURRENT_CHANGES)
    .onEach {
        // Display progress
    }
    .takeWhile { !it.isTransferComplete }

// Display downloaded content

If there were no remote updates since the app was opened last time, the progress notifier would fire an immediate event with transferableBytes == transferredBytes indicating that there was no immediate known ongoing download, and you could proceed to displaying the data.

The new behavior is not only a regression, but also seems to go against all other notifier registration points that actually posts an immediate callback of the initial state.

Actual Results

With v14.5.2-39-geef7fcf04 we are no longer getting a progress notification if all known things have been fully downloaded, so
neither

realm.syncSession.progressAsFlow(Direction.DOWNLOAD, ProgressMode.CURRENT_CHANGES)

nor

realm.syncSession.progressAsFlow(Direction.DOWNLOAD, ProgressMode.INDEFINITELY)

The actual tests showing the regression in Kotlin was https://github.com/realm/realm-kotlin/blob/main/packages/test-sync/src/commonTest/kotlin/io/realm/kotlin/test/mongodb/common/ProgressListenerTests.kt#L228.

Steps & Code to Reproduce

This Kotlin test is now failling with a timeout exception due to the missing callbacks.

Core version

Core version: v14.5.2-39-geef7fcf04

[sync-by-unito]

➤ PM Bot commented:

Jira ticket: RCORE-2099

[sync-by-unito]

➤ Jonathan Reams commented:

Want to clarify that while this may be a regression in PBS, for FLX sync the current behavior should be correct? We added the current behavior in RCORE-2013 because in that case if you open a realm you could get a download progress notification showing that the progress estimate was at 1.0 but that was really stale data for an earlier (usually empty) subscription set, and so non-streaming download notifications were pretty useless. I can see how that would be a regression for PBS, but is the behavior okay for FLX?

[kiburtse]

The issue description is a bit misleading. Although it is true that this aspect of behavior has changed after #7452 (RCORE-2013), it didn't change any documented behaviours or the way valid usecases behaved before.

What the real linked testcase was doing is roughly this:

val realm = Realm.open(...)

// attempt to sync and complete all known changes
realm.syncSession.uploadAllLocalChangesOrFail()
realm.syncSession.downloadAllServerChanges()

// invalid: expect to get one callback with last known values after full sync?
realm.syncSession.progressAsFlow(Direction.DOWNLOAD, ProgressMode.CURRENT_CHANGES).first()
realm.syncSession.progressAsFlow(Direction.DOWNLOAD, ProgressMode.INDEFINITELY).first()

The notion of CURRENT_CHANGES is only known for upload (since it's client side info), for download we always need to wait for the next DOWNLOAD message with sync progress info. There is no way to tell in advance what's the state of sync. That hadn't changed in any way.
There is no way registering download notifier after download complete is useful for the user facing spinner with the progress.

In any way if you want to get all known changes from the server before letting the user to see your app (and show some progress spinner in between), you should open the realm, register the notifier, and then wait for all downloads to complete.
Registering the notifier after the fact, would only yield (in previous implementation) one single value (where transferred == transferable, and that implies progress estimate of 1.0), which isn't useful for the simple "progress spinner"-experience.
That's why i think the mentioned testcase in kotlin wasn't really useful or valid.

[rorbech]

From the docs of realm_sync_session_register_progress_notifier it doesn't state that the notion of current changes is only for uploads.

I guess the old behavior was used to answer the question: "are there any ongoing downloads at the moment?" A callback with downloadable == downloaded indicated that there weren't, where as downloadable > downloaded indicates that there was an ongoing download. How would you answer the same questions with the new behavior?

There is no way registering download notifier after download complete is useful for the user facing spinner with the progress.

The problem is that I don't know if I register the notifier after a download is complete or during the actual download, so there is no way to know if I should expect a callback or not. So I need to have concurrent paths for registering the notifier and waiting for download and then cancelling the notifier when potential download is done.

When calling realm_sync_session_register_progress_notifier for uploads after uploadAllLocalChanges I get:
Progress(transferredBytes=2906, transferableBytes=2906)
But the documentation states

Register a callback that will be invoked every time the session reports progress

But there is no updates to the "session progress", so seems weird that we fire an event following the above argumentation.

Anyway, I just think that things are inconsistent and really difficult to reason about. If this is considered a bug in the old implementation, then at least make the docs for realm_sync_session_register_progress_notifier reflect the behavior and make it consistent on downloads and uploads.

[tgoyne]

This new behavior is pretty unambiguously a bug. A notification that there are no pending downloads is absolutely something that an app could be relying on, and the fact that we never explicitly spelled out that a notification would be sent in that scenario in the documentation is irrelevant.

[jbreams]

@tgoyne , agree on PBS notifications, but what about for FLX notifications?