Science Data Analytics Platform (SDAP)
The Earth and Space Science Informatics community has a long history of building many innovative solutions. With a quick search on the web we can find various tools that offer similar capabilities such as search, visualization, subsetting, analysis, etc. The community is very good at building domain-specific solution for specific applications. The lack of cohesiveness among these tools introduces technology gaps, which lead to even more stovepipe solutions. An Analytics Center Framework (ACF) is an architectural concept to encapsulate scalable computational and data infrastructures to harmonize data, tools and computational resources that enable scientific investigations. The Apache Science Data Analytics Platform (SDAP) is a professional open source implementation of an ACF. It is an ensemble of big data technologies for Earth science that is optimized to leverage the elastic cloud or on-premise computing clusters. SDAP has a growing collection of webservice capabilities including:
- Satellite and model data analysis
- Anomaly detection
- In situ data integration and matchup
- Fast data subsetting
- ML-Driven search and discovery
Distributed SDAP
The number of publicly hosted SDAP instances has grown over the years. By making their data and services available to the public, these discipline-specific SDAP instances are contributing to the global climate research effort. The federation of managed SDAP instances offers the following benefits:
- Reduces data replication and necessary egress (i.e. reduces the overall operation cost)
- Harmonizes data and services (i.e. reduces learning and adaptation curves)
- Independence from computing infrastructure (i.e. selects the best fit infrastructure)
- Offers programming language-agnostic application and system interface (i.e. researchers should be free to use their preferred programming language)