How to enable client-side video upscaling¶
The Anbox Streaming SDK offers experimental video upscaling features for client side streaming, leveraging AMD FidelityFX Super Resolution 1.0. This feature allows video to be transmitted at lower resolutions, saving bandwidth without losing too much detail. The SDK sets up a WebGL context and utilises a fragment shader based on the FSR 1.0 algorithm for post-processing, producing clearer and sharper video frames before they are displayed on the screen.
Initiate the AnboxStream
object¶
To enable client-side video upscaling, initiate the AnboxStream
object as follows:
<!DOCTYPE html>
stream = new AnboxStream({
targetElement: "anbox-stream",
experimental: {
upscaling: {
enabled: true,
},
},
...
});
Use target frame rate¶
The option experimental.upscaling.useTargetFrameRate
allows canvas to use the refresh frame rate setting in the Anbox container for rendering video frames. This avoids relying on HTMLVideoElement.requestVideoFrameCallback
function, which can occasionally be fired one v-sync late. This option is off by default and requires explicit activation.
Note
On a WebView where the HTMLVideoElement.requestVideoFrameCallback
function is not supported, this option will be enabled by default.
Use custom fragment shader¶
The option options.experimental.upscaling.fragmentShader
allows the use of a custom fragment shader-based upscaling algorithm for video streaming. By providing the fragment shader source code in this option, the canvas can use the custom fragment shader for video upscaling instead of the default FSR 1.0 shader. Currently, the implementation supports only single-pass rendering, which uses a single fragment shader to perform the upscaling.
When using a custom fragment shader in WebGL, you need to manage the following pre-defined variables to interact with textures and control rendering:
uSampler Type: sampler2D Description: this uniform represents the texture sampler for your fragment shader. It allows the shader to sample from the texture bound to texture unit 0.
vTextureCoord Type: varying vec2 Description: This varying variable is passed from the vertex shader to the fragment shader and represents the texture coordinates for the current fragment. It is used to sample the texture at specific coordinates.
uResolution Type: vec2 Description: This uniform contains the dimensions of the rendering canvas, typically set to the width and height of the canvas in pixels. It helps in normalising texture coordinates for effects like scaling and post-processing.