Accessing Magnetic Field Data from EarthScope Using MTH5

Ground-based magnetic field data for space physics researchers are often sourced from InterMagnet, or SuperMagnet. These data sources tend to be long term observatories and thus make reliable sources for data. However, over the past decade another source of magnetic field data has become publicly available. These constitute magnetotelluric observatories (or “stations” which are typicaly only installed for a short period of time (days to weeks). While they usually lack continuous temporal cooverage of magnetic field observations, they have a much higher spatial density than InterMagnet, and make possible new studies of magnetic phenomena.

One curator of this type is data is EarthScope (formerly IRIS and UNAVCO) which archives a significant volume of Electromagnetic (EM) data in addition to the seismological data which they are primarily known for. There are several tools available for exploring the available data at EarthScope, here are some of them:

1. Inteactive Map

2. Station Service (URL builder for exploring metadata)

3. Metadata Aggregator (MDA)

4. Availability Service (URL builder for exploring data holdings)

Additionally, an example of programatically querying EarthScope’s holdings for EM (specifically magnetotelluric) data can be found in the earthscope_tests branch of aurora . In this example notebook, the Interactive Map will be used to identify magnetometer data in a spatial region and time interval of interest.

This notebook was motivated by the desire to provide Earthscope data to space physics researchers via a plug-in to the popular “PySPEDAS” code used in that field. In this notebook an MTH5 workflow is shown for accessing Earthscope magnetic field data. There are numerous “asides” in this workflow that serve to give examples of interacting with mth5 and Earthscope. For the user who simply wants to request data in SI units, they are referred to a companion notebook where the workflows shown here are wrapped in a simple call that can be accessed from the “PySPEDAS” software favored by the space physics community.

It is divided into the following sections:

0. Getting Started

1. Exploring Data Availability

2. Application to a Case Study

3. Request DataFrame

4. Build MTH5 (metadata only)

5. Build MTH5 (with data)

6. Calibrating time series

7. Comparison of time series

0. Getting started

Install mth5, which can be done via the Python Package index using pip

pip install mth5

or via conda-forge: > conda install mth5

To run the companion notebook install pyspedas via pip. > pip install pyspedas

or to install a particular branch: > pip install git@github:/pyspedas/branch

Confirm installation

Note that installing mth5 will also install the mt_metadata package

import mt_metadata
import mth5
print(f"mt_metadata.__version__ {mt_metadata.__version__}")
print(f"mt_metadata.__file__ {mt_metadata.__file__}")
print(f"mth5.__version__ {mth5.__version__}")
print(f"mth5.__file__ {mth5.__file__}")

mt_metadata.__version__ 0.3.3
mt_metadata.__file__ /home/kkappler/software/irismt/mt_metadata/mt_metadata/__init__.py
mth5.__version__ 0.4.1
mth5.__file__ /home/kkappler/software/irismt/mth5/mth5/__init__.py

Import required packages and modules

from mt_metadata.timeseries.stationxml import XMLInventoryMTExperiment
from mth5.clients.fdsn import FDSN
from mth5.clients.make_mth5 import MakeMTH5
from mth5.mth5 import MTH5
from mth5.utils.helpers import initialize_mth5
import numpy as np
import pandas as pd
import pathlib

import warnings
warnings.filterwarnings("ignore")

1. Exploring the data at EarthScope

An interactive map can be found at “https://ds.iris.edu/gmap/”

Here is an image of the interactive map:

%%html
<iframe src="https://ds.iris.edu/gmap/" width="1200" height="1000"></iframe>

Figure 1: The EarthScope Geographic Interface

If you click the “Update Map” button in the upper left, the map will populate with station data. To avoid being overwhelmed with the map results, queries can be restricted both spatially, temporally, as well as in terms of data type, sample rate, and instrument orientation.

FDSN formats

Earthscope is one of several organizations that use standards defined by the International Federation of Digital Seismograph Networks (FDSN) which has specific conventions for specifying data time series.

The FDSN gives detailed information of their “SEED” convention in the SEED manual and the interested reader is referred to that document for details on conventions, including an Appendix on channel naming conventions.

For the example here, we need only know two things: 1. The Earthscope data collection is organized as a heirarchy, with the top level container being a “network”, which in turn contains “stations” (physical observatories), which in turn contain “channels” which correspond to individual data streams from instruments.

2. that a 3-character code is used to specify a data stream, where the first character specifies sample rate, the second corresponds to the field being measured, and the third character indicates the orientation of the field.

The main features of the interface will be illustrated for the purpose of magnetometer data. - Magnetic data is associated with the SEED Instrument Code F, - The SEED manual specifies that orientations for magnetometer data are [“N”, “E”, “Z”], however, in recent years the codes [“1”, “2”, “3”] have also been used. - Many sample rates for magnetic field data are used, so we will use the wildcard character for sample rate (’*’, ‘?’). - The ‘?’ wildcard replaces a single character, whereas the ‘*’ wildcard can replace mulitple values. We use ‘?’ here.

Notes: - Using wildcards in the third (field component) position is not advised as the “F” designator can also be associated with station health metrics such “VFP” as Packet Buffer Usage. - The “Channel” field in the map accepts lists. Thus, to get a sense of all the magnetometer data, one can place in the the channel field “?F1, ?F2, ?F3, ?FN, ?FE, ?FZ” as in the example below. - Most Magnetometer measurements are 3-component, so it is normally sufficient to query one numeric style and on letter style component, for example [“?F3, ?FZ”] to get a sense of data coverage.

Figure 2: Spatial distribution of the EarthScope magnetic dataset

Global data coverage returned from a query restricted only to magnetometer data.

Exploring Metadata Availability Programatically With Queries

Station Service

Below are some code examples for programatically generating URLs for exploring metadata availability. All the qureries use the same base URL “http://service.iris.edu/fdsnws”. For more detailed examples of generating URLs from queries see the Earthscope URL Builder website.

Here is an example of a URL that lists metadata for all stations and channels with magnetic field data:

# Specify the base URL
fdsn_URL = "http://service.iris.edu/fdsnws"

# magnetic query using seed conventions
channels=['?FE', '?FN', '?FZ','?F1','?F2', '?F3', ]
channels = ','.join(channels)


# To obtain a list of all stations available:
channel_list_URL=f"{fdsn_URL}/station/1/query?cha={channels}&level=channel&format=text&includecomments=true&nodata=204"
print(f"channel_list_URL: {channel_list_URL}")

channel_list_URL: http://service.iris.edu/fdsnws/station/1/query?cha=?FE,?FN,?FZ,?F1,?F2,?F3&level=channel&format=text&includecomments=true&nodata=204

Figure 3: Screengrab of station list associated with magnetic field data.

As of December 2023, this webpage has 13228 lines!

Availability Service

Note that each station is associated with a “Network” two-character code (first column in Figure above). We can specify a network and see detail about the stations associated with this network, if we want to explore data availability via tabular summaries instead of using the map.

Below is a code snippet for accessing a data coverage (“availability”) list for network “4P”.

network = "4P"
channels_str = f"{channels}"
print(channels_str.replace(" ",""))
channel_URL = f"{fdsn_URL}/availability/1/query?format=text&net={network}&cha={channels}&orderby=nslc_time_quality_samplerate&includerestricted=False&nodata=204"
print(f"channel_URL:\n {channel_URL}")

?FE,?FN,?FZ,?F1,?F2,?F3
channel_URL:
 http://service.iris.edu/fdsnws/availability/1/query?format=text&net=4P&cha=?FE,?FN,?FZ,?F1,?F2,?F3&orderby=nslc_time_quality_samplerate&includerestricted=False&nodata=204

Figure 4: Screengrab of availability URL for Network=”4P”

Metadata Aggregator (MDA)

Besides the tools described above, the Earthscope collection can be navigated heirarchically via the Metadata Aggregator (MDA)

Figure 5: Earthscope Metadata Aggregator (MDA) website landing page.

2. Case Study:

Comparison with Published Observation (Heyns et al 2020)

Here we will try to extract some magnetic field data and compare with published values.

Lets use this manuscript https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020SW002557

Figure 6: Title and Abstract from Heyns et al. 2020

Figure 1 from the above paper shows an image of a magnetic field time series.

Figure 7: Screengrab of Figure 1 from Heyns et al. 2020

Note that the inset in the lower panel shows 35 minutes of data from the USGS FRD observatory, component B_y. This is the data we will examine.

We may be curious if other nearby stations also recorded the same fluctuations in the ambient magnetic field, which we will call the “event”. We can query the Earthscope data availability to see of other stations were recoding during this time interval.

Identify relevant stations

We could download data from FRD by using intermagnet, or the MTH5 extension for global geomagnetic observatories but lets use an EarthScope station instead.

The USGS website gives us the coordinates for FRD (near Fredricksburg VA):

https://www.usgs.gov/programs/geomagnetism/science/fredericksburg-frd

Station Id: FRD

Location: Corbin, VA

Latitude: 38.2047°N

Longitude: 77.3729°W

• First, we get virginia on the map and center it (to the right because there are not many observatories in the Atlantic Ocean), then we enter a date range around that the day of interest: [2015-06-01, 2015-06-30]

• Several search methods are available to put a geographic boundary on that search such as using a bounding box or a radius. Below is an example using “bounding box”.

• Add ~5 degrees latitude and ~10 degees longitude around FRD to make a bounding box.

Note that Earthscope uses negative longitudes To learn about this convetion and conversions, see this webpage from NASA

Figure 8 Interactive map showing query results.

Stations are shown with markers within the bounding box. Stations are tabulated below the map (and the list is scrollable). Clicking on a station marker makes a pop-up box, such as that shown for 4P ALW49.

Select a single station

Let’s pick the first one in alphabetical order. Using the map we can readily see that there are metadata available in our time range, though we need to make sure that there is also data for the period. Clicking a station makes a pop-up info box appear with further detail about the stations operating time range, and location. Clicking on “more information” in this box takes us to the metadata aggregator (MDA) page, as shown below:

Figure 9 MDA info about station ALW49

# define the station
station = "ALW49"
# station = "TNU48"

# We can also query availability for a stion restrict availability by station:
station_availability_URL = f"{fdsn_URL}/availability/1/query?format=text&net={network}&sta={station}&cha={channels}&orderby=nslc_time_quality