[css-view-transitions-2] (feature) access view-transition snapshot as an image/ImageData

[noamr]

For view transitions, right now we take snapshots for the old & new state and they get rendered in the ::view-transition-{old|new} pseudo elements as the contents. But what if we could access those images elsewhere? e.g. paint them into 2D/WebGL canvas or display them in an image element?
This can open a world of interesting animations.

A way to express this can be something like a URL (e.g. view-transition:name#old) or an imperative API
viewTransition.getSnapshotImageData(name, "old") or viewTransition.createSnapshotURL(name, "old")

(* Not planning on implementing anything like this soon but wanted the idea documented)

[khushalsagar]

This has some intersection with #8830. The use-case there is being able to get a static snapshot of an element when there is no transition.

[SebastianZ]

API access to the image data surely enables a lot of new possibilities regarding transitions, though it also amplifies the security risks. UAs need to ensure that no passwords or other sensitive data is exposed in those snapshots.

Sebastian

[tabatkins]

Yeah, we have historically avoided the "snapshot the page into a canvas texture" issues because of the security concerns. (I think these are manageable, but it's been enough to stall previous efforts.)

[noamr]

In this case though we already snapshot the page, and it can be drawn into the canvas in a tainting way (meaning the result is not readable, like a cross-origin no-cors image).

[nt1m]

I'm not sure this is a great idea, nor if it's straightforward, especially for the new capture which isn't a snapshot given its live nature.

[nt1m]

I also wouldn't want people to suddenly start using View Transitions just to get snapshots (but not actually run a transition)

[noamr]

Yeah, we have historically avoided the "snapshot the page into a canvas texture" issues because of the security concerns. (I think these are manageable, but it's been enough to stall previous efforts.)

I think the main security issue would be WebGL timing attacks to read cross-origin non-isolated content. It would have to be enabled only in isolated environments and disallow reading iframes.

I think the security issues are solvable but I agree none of this is trivial (hence: not on any short term work list).

I'm not sure this is a great idea, nor if it's straightforward, especially for the new capture which isn't a snapshot given its live nature.

How is that conceptually different from painting from a video? You can get the live image with every requestAnimationFrame.
I agree that this idea is not straightforward, however the use case exists.

I also wouldn't want people to suddenly start using View Transitions just to get snapshots (but not actually run a transition)

Good point... We could perhaps do something more use-case specific, like allow a "big canvas mode" for the whole view transition that you can paint to, but you can't paint the images outside of that.

[nt1m]

How is that conceptually different from painting from a video? You can get the live image with every requestAnimationFrame.
I agree that this idea is not straightforward, however the use case exists.

Under the hood, the old snapshot is actually a snapshot while the new captures are just a bunch of reparented layers. Allowing getting the image data of the new capture could affect the performance of the live capture or create some unnecessary restrictions as to how live captures are implemented? It's obviously not impossible to do but I'm not keen to exposing something that's not just readily available when the counterpart could be reduced quality of the main use case.

[noamr]

How is that conceptually different from painting from a video? You can get the live image with every requestAnimationFrame.
I agree that this idea is not straightforward, however the use case exists.

Under the hood, the old snapshot is actually a snapshot while the new captures are just a bunch of reparented layers. Allowing getting the image data of the new capture could affect the performance of the live capture or create some unnecessary restrictions as to how live captures are implemented? It's obviously not impossible to do but I'm not keen to exposing something that's not just readily available when the counterpart could be reduced quality of the main use case.

Good point, I guess the new live snapshot thing makes this feature not different from any other "element capture" feature. Bummer.