About the POLDER - Polar Data Search
The portal was created to aid researchers, stakeholders, and rights holders who are looking for polar meta(data). The Polar Data Search (Formerly POLDER Polar Federated Search) is an amalgamation of 23 (and growing) multidisciplinary repositories that host some form of polar data, which are then indexed into a single search interface. Research datasets that are available through the Polar Data Search encompass many different types of observations including but are not limited to weather station records, human written stories, organism counts, and instrument cruises. Observations include measures of almost any variable that exists in nature. For this reason, there is no realistic prospect of standardizing and aggregating the da a itself; therefore, federated metadata search is the only viable way to make these datasets easily discoverable, and maximize their value (Verhey, C., Minch, M., & Payne, K. 2023).
This search engine is a customizable federated search website, originally developed by the World Data System International Technology Office between 2021 and 2023, in response to needs identified by the POLDER Working Group. This system was created in response to a need identified by the polar research community, in particular Polar Data Discovery Enhancement Research (POLDER - https://polder.info/). POLDER is a collaboration between the Southern Ocean Observing System (SOOS), the Arctic Data Committee (ADC), and SCAR - Standing Committee on Antarctic Data Management (SCADM).
The POLDER Data Management community has been working towards a federated search system since 2016. The working group established the guidelines for the portal with the intentions to have something that came from, and is meant to serve, an international community of polar (Arctic and Antarctic) researchers. The portal has been built as an easy to use, fast, accessible site that is tailored to this specific community and responds to its needs.
The Polar Data Search uses open source software which has the added advantage of making it freely available to other communities to use in their own domain-specific search portals. The problem at hand is that the state of the data discovery is a complex landscape. The POLDER group alone consists of ~103 different data repositories. Globally, not all polar focused data repositories participate in the POLDER space; some repositories are more generalist which further exacerbates the data discoverability problem. Some aggregate other repositories such as the Polar Data Catalogue, but there is no set of standards or recommendations to organize these harvesting structures. For researchers who may be searching for data, this may mean that they are individually searching each repository and can result in a time-consuming hunt. A federated search helps reduce this dilemma by having repositories voluntarily complete the semantic mark-up in order to be included, have their participation publicly known for users to see, and it does not change their current harvesting structures that they may already have in place.
Find out more by reading the documentation book or exploring the GitHub repository.
The current strategy for the POLDER WG is to utilize the semantic alignment and to build on Schema.org (SDO, http://www.schema.org) successes, which are being replicated in other communities that are using SDO as a base ontology, and creating extensions to it that are relevant to their own communities. In particular, this strategy is being utilized by the Earth Science Information Partner's Science-on-Schema.org (SO–SO), the Ocean Data and Information System (ODIS), the ESIP soil cluster, and EarthCube via their P418 project, Magnetics Information Consortium (MagIC), and GeoCODES search engine (https://geocodes.earthcube.org). Far more impactful than data publishing and exposure, the long-term vision of this work is to support pathways for cross domain research and collaboration.
If you'd like to send us questions or comments directly, you are welcome to email us.
What to expect in the Search
Searching
The Polar Data Search indexes metadata records from the participating repositories (see below). The multifaceted search options allow you to search by: Author, Date Ranges, and Keywords.
Metadata
What is Metadata? Metadata is data about data - which means it is the information that has been submitted to describe a dataset, object, or thing. The Polar Data Search, does not HOST data, rather indexes the metadata for discovery level datasets
Once you search, all relevant metadata records will show up in a paged system, as well as a spatial map will display the metadata records spatial boundaries. Feel free to click on the map which will pop-out a box to show you the available metadata records for that region, and by selecting on the title of your preference, you will be redirected to the metadata record drop down box.
Data Access
To access any datasets associated with the metadata record that you are interested in, select the title and/or the DOI (if the record has one) in the top right corner. You will then be redirected to the original records landing page at the host repository. This ensures that any licensing or access restrictions are still in control and respected at the original source level.
Covered data repositories
There are two ways a repository is included in searches that are made from this application:
-
Repository is indexed by the app itself - through completing the schema.org mark-up and providing sitemap to the PDS team
-
Guidance documentation for individual repositories can be found at https://repository.oceanbestpractices.org/handle/11329/2301
-
The repository is indexed by a participating aggregator (examples: DataONE, Datacite, CCADI Metadata API) AND has data within the search parameters
Updated List of Participating Repositories
Special Thanks
The production and development environments are currently hosted by Cybera, who are graciously providing computing infrastructure and support.
