Datacubes:
An Enabling Paradigm for Big Data Analytics
Datacubes constitute an enabling paradigm for user-friendly, flexible, and scalable handling of massive multi-dimensional arrays, such as spatio-temporal sensor, image (timeseries), simulation, and statistics data.
Such datacubes can be sparsely populated (such as in OLAP) or densely populated (such as in satellite imagery and weather forecasts).
Datacube technology has been pioneered by the rasdaman ("raster data manager")
array engine which constitutes the worldwide lead in terms of flexibility, scalability, performance,
and support for open standards. Key international standards are based on rasdaman as their blueprint:
- ISO SQL/MDA (Multi-Dimensional Arrays)
which enables maintaining data alongside with their metadata, in one and the same SQL database.
Final adoption is expected by mid-2017.
- OGC Web Coverage Service (WCS) Interface Standard
is the OGC "Big Earth Data" standards suite for accessing, subsetting, analytics, and processing
of massive spatio-temporal datacubse, such as 1-D sensor timeseries, 2-D satellite imagery, 3-D x/y/t image timeseries and x/y/z geophysical voxel data, 4-D x/y/z/t weather and climate data, etc;
official reference implementation is rasdaman.
- OGC Web Coverage Processing Service (WCPS) Interface Language, a spatio-temporal datacube query language with concise geo semantics; official reference implementation is rasdaman.
Ultimately, with rasdaman "any query, any time, on any size" becomes a Big Data commodity.
imprint; image: Shutterstock