As with AntOS and its applications, one can remotely access and edit server resources from browser in a desktop-like manner. However, sometime these web based applications are not enough for some specific tasks. For a long time, I've thinking of a web based API for controlling remote desktop right from the browser ( or AntOS application). The VNC protocol is a good starting point. After playing around with libvncserver/libvncclient, i came up with wvnc a web based protocol and API for accessing VNC servers using websocket.
As shown, WVNC consists of two parts: the server bridge and the client API:
The server bridge, which is implemented in C, is not a standalone application but is an Antd (my HTTP/HTTPS web server) plugin. This allows me to simplify the development since Antd handles everything related to the HTTP/HTTPs as well as the websocket protocol. I only need to implement the VNC-to-websocket bridge. To communicate with VNC servers, the plugin relies on the well-known libvncclient library. In addition to this, it defines the base protocol for communicating with web application via web socket.
To speed up data streaming between web applications and the plugin, some data compression options are available based on Zlib and libjpeg. Using the provided protocol, the web based client have 4 choices for data compression:
For the last option, one may ask what is the point of using both JPEG and Zlib for compression since compressing an already compressed data doesn't really produce any improvement. Well, this indeed does improve the data transmission if we look closely at how WVNC compresses and streams data. Let's take an example, suppose that the size of the raw frame buffer segment is 300 bytes and the client chooses JPEG for data compression, the plugin will try to compress the raw data to JPEG format. It then compares the size of the compressed data with the original one, if the size is smaller, it will stream the compressed data, otherwise, the raw data is sent instead. In this case, let's say the compressed data is around 600 bytes due to the added JPEG header, obviously, the plugin will send the raw data (300 bytes). Now if the client chooses both JPEG and Zlib as compression option, this 300 bytes raw data will then be compressed using Zlib producing the final data to be streamed of around 75 bytes. This does a lot of improvement in term of data transmission.
Decoding compressed data from server is a heavy task on the client side. To speed up this process, wvnc.js uses web worker to provide background data decoding support. Further more, the decoding algorithm is implemented using Web assembly (with the help of libjpeg and zLib) to achieve fast decoding and rendering.
Please refer to the two project pages mentioned earlier to find out how to build and use WVNC