5. DCMS Cache-B Data Access

Some datasets listed in the data offer are particularly huge sized, therefore they’re not immediately ready for visualization and need to go through a preliminary extraction process before they can be viewed inside a VIZLAB scene. Extraction jobs can be very slow and computationally expensive, depending on the amount of data that needs to be downloaded from the external data source and processed on VIZLAB’s back-end. Each extraction job is therefore limited to a small subset of the available data, in both spatial and temporal dimensions. The results of an extraction job is referred to as a data cube. When creating a new layer from a selected sub-dataset that requires data-extraction, users can choose to view one of the available data cubes, or to start a new data-extraction job that will generate a new data cube with user-specified parameters.

Fig. 5.1 List of data cubes from a tiff dataset on the right.

In the Data Offer section, the available data cubes appear on a list at the right side of the dataset info. During the creation of a new layer, access to the available data cubes is provided through a modal panel. The panel opens up when the addition to the layer is made through the data offer panel or directly in the layer section.

Fig. 5.2 Layer addition through data offer modal

Fig. 5.3 Data cube selection modal panel

If the list is empty or they wish to make their own custom selection, through the button “Create New Image Set”, user can open another panel to create a new data cube. Here they can specify the start and the end date, the south, west, north and east, the pixel resolution and the temporal resolution (temporal resolution not selectable in the current version).

Fig. 5.4 Data cube creation modal panel.

It’s possible to manually select the region and reset the region values to their default settings. Default region covers the entire globe for all data except volumetric ones (refer to the table in the “Getting Started” chapter for their limitations). Clicking the “Draw Region” button activates the region selection mode, hiding the modal panel and all of the UI, except for a bar displaying the instruction “Press SHIFT + click to select a region”, to perform the selection.

Fig. 5.5 Region selection message

Fig. 5.6 Region selection screen

The selected area is marked by red lines appearing on the globe and the coordinates. Once the selection is completed, the modal panel will be visible again, with the newly selected coordinates.

Fig. 5.7 Selected region

Fig. 5.8 Selected region after confirming

If you wish to manually change the selected region, you can use the draw region function again, or adjust the values is the text boxes. Note : as warned in the modal panel, the image processing may take hours depending on the specified parameters. Be wary about it when creating a new data cube.

Once the add layer button is clicked, a panel confirmation appears warning the amount of quota that is going to be used to store that data cube.

Fig. 5.9 Dataset loading from zarr store

Upon adding the data cube to the layer, in the layer section the dataset displays an additional circle with the percent value corresponding to the progress of the image processing and loading. Once completed, the value fades out and the dataset entry in the list returns to its usual appearance. If any error occurs and the loading fails, the progress circle turns red. The boundaries of the selected region are rendered independently from the result of the image loading, resulting in an empty rectangle on the surface of the 3D Earth in case of error or empty data selection.

Fig. 5.10 Dataset loading from zarr store

Fig. 5.11 Empty selected area on the globe due to an empty data selection

5.1. User quota management

As mentioned in the previous paragraphs, each registered user has a quota limitation for data that can be stored.

A registered user can request for generating data cubes from DCMS Cache-B, which are subsequently stored on Vizlab back-end to be available on need. Upon generation request a data cube is automatically locked by the requestor. A data cube locked by a user consumes their quota.

A locked data cube is retained by the system and can be used by both registered and unregistered users, without consuming their quota. Another registered user can lock the same data cube and, even if the system doesn’t duplicate it, this locking operation consumes user quota. A locking user can unlock a data cube and when there are no more users locking a given data cube, it can be deleted from the system. Since data cube creation is a time consuming process, the application is designed to actually erase the data cube only to free up some space when the disk is running out of memory.

User can load an unlocked data cube in a layer, but has no guarantee that the data cube won’t be deleted by the system.