ChronosServer

ChronosServer is the backend of Climate Wikience. It runs on the high performance computer cluster and manages large volumes of climate data. ChronosServer delivers data over Internet in near realtime to your PC for 3D visualization in Climate Wikience and analysis in R.
Rodriges Zalipynis R.A., ChronosServer: real-time access to “native” multi-terabyte retrospective data warehouse by thousands of concurrent clients. // Informatics, cybernetics and computer engineering, pp. 151–161. Vol. 14 (188), Donetsk, DonNTU, 2011. PDF
Abstract
ChronosServer runs on a cluster of commodity hardware and possesses scalability, high availability, and fault tolerance properties. It turns vast amounts of already existing data into actionable intelligence with no changes to the source data files. ChronosServer discovers files on cluster nodes, analyses their structure, and provides format independent SQL-like query model to access their contents. It is capable to read compressed data directly in various formats, including NetCDF, GeoTIFF, GRIB, HDF and many others. This entirely preserves original file metadata as is, vital for its correct interpretation and processing by other software. New data added to the system in a seamless plug-and-play fashion by simple copying it to a cluster node reducing administration overheads. This allows existing software like GIS or statistical packages to operate on files in use by ChronosServer as well as unmodified legacy code to generate data for it. ChronosServer preserves operational infrastructure intact avoiding painful, time-consuming and error-prone data conversions while offering additional opportunities for data analysis.

chronosserver_dataflow_en



Keywords

  • in-situ data processing
  • inherently distributed
  • raster data
  • raster time series data
  • large data volumes
  • NetCDF, HDF4, HDF5, Grib2

Operational facts

During the project funding by the US Civilian Research and Development Foundation (CRDF Global), ChronosServer had been running on the computer cluster of 6 nodes with 24 TB of total disk space.

Cluster photo

Experimental computer cluster was assembled in Donetsk National Technical University (Ukraine), was connected to the Internet. It was running ChronosServer which provided data over the Internet to Climate Wikience.

chronos_cluster

 

Currently it runs on the rented VPS and continue providing climate reanalysis, Earth remote sensing data and time series to demonstrate its functionality. However, the time period for each variable is reduced to 1 month due to limited VPS disk space.