The CVL shall allow data analysis and processing (run own code or code shared in the CVL) both locally (e.g., on the user’s own PC by transferring to the user only the data needed for processing at each processing step) and on-demand (cloud based), with the possibility to run the process on virtual machines with pre-installed software for access, processing and analysis of geospatial data. Initially the scientific use of CVL will be enabled by offering a data search and selection service on the landing page. The selected data products can then be visualised online using a visualization based on NGVOC (Norut Geo Visualization Open Core), and downloaded for further processing and analysis using own tools. Following this, CVL will offer VM configurations for local use, and VMs in the cloud, with a selection of software tool for data access, visualisation, processing, and analysis to ease the process of setting up a technical execution environment.
The scientific use of CVL under the various use cases in WP 300 will be enabled in Jupyter notebooks. The scientists develop new code or translate software developed in other high-level tools like MatLab or IDL or standalone Python-code into the Jupyter notebooks. For example, the use case on snow-phenology will take advantage of IDL-software that handles daily snow products (snow cover fraction) derived from MODIS and Sentinel-1 data. The following aggregation steps where each daily product is aggregated into annual products for the first day of snow and the last day of snow can with simple means be translated into Python. Similarly, regional and temporal averaging will result in plots that show the long-term development of the Svalbard snow cover over the last 20 years. In the initial phases the data will be stored on the computer (local or cloud based virtual machine; VM) running the Jupyter notebook. The next steps will be to produce output (geographical change-maps and plots of the temporal development) which can be shown in visualisations based on the NGVOC system within the notebook. When the maturity of the processing chain has been assured, it can be set up for batch operation on a VM hosted by CVL. Automatic data streaming from the SIOS server will be implemented, and the results can be uploaded for further sharing and use from a CVL data server accessed via the CVL-landing page. During development, all software code will be under GIT version control and available via online repository, e.g. hosted by GitHub or similar. This will allow both external scientists and CVL users to take advantage of the developments at a later stage.
We define two operation modes:
- interactive mode
- batch processing mode.
In the interactive mode, python commands are launched directly in the interpreter (e.g., IPython or Jupyter Notebook), whereas in the batch processing mode, operations are pre-registered in the system and executed by the users (with data as input). This could be an operation made available on the system by the user, by other users, or by the CVL team (e.g., the scientific use cases).
