Supported Instrument Data Sets¶
Currently, support is included for files from the following sources:
- SuperMAG (
ocbpy.instruments.supermag
) - SuperDARN Vorticity (
ocbpy.instruments.vort
) - pysat (
ocbpy.instruments.pysat_instruments
)
These routines may be used as a guide to write routines for other data sets.
A ocbpy.instruments.general
loading sub-module is also provided for
ASCII files. All the non-boundary data routines are part of the
ocbpy.instruments
module. Support for time-handling that may be useful
for specific data sets are provided in ocbpy.ocb_time
.
General Instrument Module¶
General loading routines for data files.
-
ocbpy.instruments.general.
load_ascii_data
(filename, hlines, gft_kwargs={}, hsplit=None, datetime_cols=None, datetime_fmt=None, int_cols=None, str_cols=None, max_str_length=50, header=None)[source]¶ Load an ASCII data file into a dict of numpy arrays.
Parameters: - filename (str) – data file name
- hlines (int) – number of lines in header. If zero, must include header.
- gft_kwargs (dict) – Dictionary holding optional keyword arguments for the numpy genfromtxt routine (default=dict())
- hsplit (str, NoneType) – character seperating data labels in header. None splits on all whitespace characters. (default=None)
- datetime_cols (list, NoneType) – If there are date strings or values that should be converted to a datetime object, list them in order here. Not processed as floats. NoneType produces an empty list. (default=None)
- datetime_fmt (str, NoneType) – Format needed to convert the datetime_cols entries into a datetime object. Special formats permitted are: ‘YEAR SOY’, ‘SOD’. ‘YEAR SOY’ must be used together; ‘SOD’ indicates seconds of day, and may be used with any date format (default=None)
- int_cols (list, NoneType) – Data that should be processed as integers, not floats. NoneType produces an empty list. (default=None)
- str_cols (list, NoneType) – Data that should be processed as strings, not floats. NoneType produces an empty list. (default=None)
- max_str_length (int) – Maximum allowed string length. (default=50)
- header (list, NoneType) – Header string(s) where the last line contains whitespace separated data names. NoneType produces an empty list. (default=None)
Returns: - header (list of strings) – Contains all specified header lines
- out (dict of numpy.arrays) – The dict keys are specified by the header data line, the data for each key are stored in the numpy array
Notes
Data is assumed to be float unless otherwise stated.
SuperMAG Instrument Module¶
Perform OCB gridding for SuperMAG data.
Notes
SuperMAG data available at: http://supermag.jhuapl.edu/
-
ocbpy.instruments.supermag.
load_supermag_ascii_data
(filename)[source]¶ Load a SuperMAG ASCII data file.
Parameters: filename (str) – SuperMAG ASCI data file name Returns: out – The dict keys are specified by the header data line, the data for each key are stored in the numpy array Return type: dict
-
ocbpy.instruments.supermag.
supermag2ascii_ocb
(smagfile, outfile, hemisphere=0, ocb=None, ocbfile='default', instrument='', max_sdiff=600, min_merit=None, max_merit=None, scale_func=<function normal_curl_evar>, **kwargs)[source]¶ Covert and scales the SuperMAG data into OCB coordinates.
Parameters: - smagfile (str) – File containing the required SuperMAG file sorted by time
- outfile (str) – Filename for the output data
- hemisphere (int) – Hemisphere to process (can only do one at a time). 1=Northern, -1=Southern, 0=Determine from data (default=0)
- ocb (ocbpy.OCBoundary, ocbpy.DualBoundary, or NoneType) – OCBoundary or DualBoundary object with data loaded already. If None, looks to ocbfile and creates an OCBoundary object. (default=None)
- ocbfile (str) – File containing the required OC Boundary data sorted by time, or ‘default’ to load default file for time and hemisphere. Only used if no OCBoundary object is supplied (default=’default’)
- instrument (str) – Instrument providing the OCBoundaries. Requires ‘image’ or ‘ampere’ if a file is provided. If using filename=’default’, also accepts ‘amp’, ‘si12’, ‘si13’, ‘wic’, and ‘’. (default=’’)
- max_sdiff (int) – Maximum seconds between OCB and data record in sec (default=60)
- min_merit (float or NoneType) – Minimum value for the default figure of merit or None to not apply a custom minimum (default=None)
- max_merit (float or NoneTye) – Maximum value for the default figure of merit or None to not apply a custom maximum (default=None)
- kwargs (dict) – Dict with optional selection criteria. The key should correspond to a data attribute and the value must be a tuple with the first value specifying ‘max’, ‘min’, ‘maxeq’, ‘mineq’, or ‘equal’ and the second value specifying the value to use in the comparison.
- min_sectors (int) – Minimum number of MLT sectors required for good OCB. Deprecated, will be removed in version 0.3.1+ (default=7).
- rcent_dev (float) – Maximum number of degrees between the new centre and the AACGM pole. Deprecated, will be removed in version 0.3.1+ (default=8.0)
- max_r (float) – Maximum radius for open-closed field line boundary in degrees/ Deprecated, will be removed in version 0.3.1+ (default=23.0)
- min_r (float) – Minimum radius for open-closed field line boundary in degrees. Deprecated, will be removed in version 0.3.1+ (default=10.0)
- scale_func (function or NoneType) – Scale the magnetic field observations unless None (default=ocbpy.ocb_scale.normal_curl_evar)
Raises: IOError
– If unable to open the input or output fileNotes
May only process one hemisphere at a time.
See also
ocbpy.ocb_scale.normal_curl_evar()
SuperDARN Vorticity Instrument Module¶
Perform OCB gridding for SuperDARN vorticity data.
Notes
Specialised SuperDARN data product, available from: gchi@bas.ac.uk
-
ocbpy.instruments.vort.
load_vorticity_ascii_data
(vortfile, save_all=False)[source]¶ Load SuperDARN vorticity data files.
Parameters: Returns: vdata – Dictionary of numpy arrays
Return type:
-
ocbpy.instruments.vort.
vort2ascii_ocb
(vortfile, outfile, hemisphere=0, ocb=None, ocbfile='default', instrument='', max_sdiff=600, save_all=False, min_merit=None, max_merit=None, scale_func=<function normal_curl_evar>, **kwargs)[source]¶ Covert the location of vorticity data from AACGM to OCB coordinates.
Parameters: - vortfile (str) – file containing the required vorticity file sorted by time
- outfile (str) – filename for the output data
- hemisphere (int) – Hemisphere to process (can only do one at a time). 1=Northern, -1=Southern, 0=Determine from data (default=0)
- ocb (ocbpy.ocboundary.OCBoundary, ocbpy.DualBoundary, or NoneType) – OCBoundary or DualBoundary object with data loaded already. If None, looks to ocbfile and creates an OCBoundary object. (default=None)
- ocbfile (str) – File containing the required OC boundary data sorted by time, or ‘default’ to load default file for time and hemisphere. Only used if no OCBoundary object is supplied (default=’default’)
- instrument (str) – Instrument providing the OCBoundaries. Requires ‘image’ or ‘ampere’ if a file is provided. If using filename=’default’, also accepts ‘amp’, ‘si12’, ‘si13’, ‘wic’, and ‘’. (default=’’)
- max_sdiff (int) – maximum seconds between OCB and data record in sec (default=600)
- save_all (bool) – Save all data (True), or only that needed to calcuate OCB and vorticity (False). (default=False)
- min_merit (float or NoneType) – Minimum value for the default figure of merit or None to not apply a custom minimum (default=None)
- max_merit (float or NoneTye) – Maximum value for the default figure of merit or None to not apply a custom maximum (default=None)
- kwargs (dict) – Dict with optional selection criteria. The key should correspond to a data attribute and the value must be a tuple with the first value specifying ‘max’, ‘min’, ‘maxeq’, ‘mineq’, or ‘equal’ and the second value specifying the value to use in the comparison.
- min_sectors (int) – Minimum number of MLT sectors required for good OCB. Deprecated, will be removed in version 0.3.1+ (default=7)
- rcent_dev (float) – Maximum number of degrees between the new centre and the AACGM pole. Deprecated, will be removed in version 0.3.1+ (default=8.0)
- max_r (float) – Maximum radius for open-closed field line boundary in degrees. Deprecated, will be removed in version 0.3.1+ (default=23.0)
- min_r (float) – Minimum radius for open-closed field line boundary in degrees. Deprecated, will be removed in version 0.3.1+ (default=10.0)
- scale_func (function or NoneType) – Scaling function for Vorticity data or None to not scale (default=ocbpy.ocb_scale.normal_curl_evar)
Raises: IOError
– If unable to open input or output fileValueError
– If unable to retrieve all necessary data from the input file
Notes
Input header or col_names must include the names in the default string.
pysat Instrument Module¶
Perform OCB gridding for appropriate instrument data loaded in pysat.
Notes
pysat is available at: http://github.com/pysat/pysat or pypi
-
ocbpy.instruments.pysat_instruments.
add_ocb_to_data
(pysat_inst, mlat_name='', mlt_name='', evar_names=None, curl_evar_names=None, vector_names=None, hemisphere=0, ocb=None, ocbfile='default', instrument='', max_sdiff=60, min_merit=None, max_merit=None, **kwargs)[source]¶ Covert the location of pysat data into OCB, EAB, or Dual coordinates.
Parameters: - pysat_inst (pysat.Instrument) – pysat.Instrument class object containing magnetic coordinates
- mlat_name (str) – Instrument data key or column for magnetic latitudes (default=’’)
- mlt_name (str) – Instrument data key or column for magnetic local times (default=’’)
- evar_names (list or NoneType) – List of Instrument data keys or columns pointing to measurements that are proportional to the electric field (E); e.g. ion drift (default=None)
- curl_evar_names (list or NoneType) – List of Instrument data keys or columns pointing to measurements that are proportional to the curl of E; e.g. ion vorticity (default=None)
- vector_names (dict or NoneType) – Dict of Instrument data keys or columns pointing to measurements that are vectors that are proportional to either E or the curl of E. The key should correspond to one of the values in the evar_names or curl_evar_names list. If this is not done, a scaling function must be provided. The value corresponding to each key must be a dict that indicates the names holding data needed to initialise the ocbpy.ocb_scaling.VectorData object (default=None)
- hemisphere (int) – Hemisphere to process (can only do one at a time). 1=Northern, -1=Southern, 0=Determine from data (default=0)
- ocb (ocbpy.OCBoundary, ocbpy.DualBoundary, or NoneType) – OCBoundary or DualBoundary object with data loaded already. If None, looks to ocbfile and creates an OCBoundary object. (default=None)
- ocbfile (str) – file containing the required OC boundary data sorted by time, ignorned if OCBoundary object supplied (default=’default’)
- instrument (str) – Instrument providing the OCBoundaries. Requires ‘image’ or ‘ampere’ if a file is provided. If using filename=’default’, also accepts ‘amp’, ‘si12’, ‘si13’, ‘wic’, and ‘’ (default=’’)
- max_sdiff (int) – maximum seconds between OCB and data record in sec (default=60)
- min_merit (float or NoneType) – Minimum value for the default figure of merit or None to not apply a custom minimum (default=None)
- max_merit (float or NoneTye) – Maximum value for the default figure of merit or None to not apply a custom maximum (default=None)
- kwargs (dict) – Dict with optional selection criteria. The key should correspond to a data attribute and the value must be a tuple with the first value specifying ‘max’, ‘min’, ‘maxeq’, ‘mineq’, or ‘equal’ and the second value specifying the value to use in the comparison.
- min_sectors (int) – Minimum number of MLT sectors required for good OCB. Deprecated, will be removed in version 0.3.1+ (default=7)
- rcent_dev (float) – Maximum number of degrees between the new centre and the AACGM pole. Deprecated, will be removed in version 0.3.1+ (default=8.0)
- max_r (float) – Maximum radius for open-closed field line boundary in degrees. Deprecated, will be removed in version 0.3.1+ (default=23.0)
- min_r (float) – Minimum radius for open-closed field line boundary in degrees. Deprecated, will be removed in version 0.3.1+ (default=10.0)
Raises: ValueError
– If the pysat Instrument doesn’t have the necessary data values or if the input provided is not a pysat Instrument.Notes
This may be run on a pysat instrument or as a custom function when loading pysat data.
Examples
# Example vector name input looks like: vector_names={'vel': {'aacgm_n': 'vel_n', 'aacgm_e': 'vel_e', 'dat_name': 'velocity', 'dat_units': 'm/s'}, 'dat': {'aacgm_n': 'dat_n', 'aacgm_e': 'dat_e', 'scale_func': local_scale_func}}
-
ocbpy.instruments.pysat_instruments.
add_ocb_to_metadata
(pysat_inst, ocb_name, pysat_name, overwrite=False, notes='', isvector=False)[source]¶ Update pysat metadata for OCB data.
Parameters: - pysat_inst (pysat.Instrument) – pysat.Instrument class object containing magnetic coordinates
- ocb_name (str) – Data column name for OCB data
- pysat_name (str) – Data column name for non-OCB version of this data
- overwrite (bool) – Overwrite existing metadata, if present (default=False)
- notes (str)) – Notes about this OCB data (default=’’)
- isvector (bool) – Is this vector data or not (default=False)
Raises: ValueError
– If input pysat Instrument object is the wrong class
Time Handling Module¶
Routines to convert from different file timekeeping methods to datetime.
-
ocbpy.ocb_time.
convert_time
(year=None, soy=None, yyddd=None, sod=None, date=None, tod=None, datetime_fmt='%Y-%m-%d %H:%M:%S')[source]¶ Convert to datetime from multiple time formats.
Parameters: - year (int or NoneType) – Year or None if not in year-soy format (default=None)
- soy (int or NoneType) – Seconds of year or None if not in year-soy format (default=None)
- yyddd (str or NoneType) – String containing years since 1900 and 3-digit day of year (default=None)
- sod (int, float, or NoneType) – Seconds of day or None if the time of day is not in this format (default=None)
- date (str or NoneType) – String containing date information or None if not in date-time format (default=None)
- tod (str or NoneType) – String containing time of day information or None if not in date-time format (default=None)
- datetime_fmt (str) – String with the date-time or date format (default=’%Y-%m-%d %H:%M:%S’)
Returns: dtime – Datetime object
Return type: dt.datetime
-
ocbpy.ocb_time.
datetime2hr
(dtime)[source]¶ Calculate hours of day from datetime.
Parameters: dtime (dt.datetime) – Universal time as a timestamp Returns: uth – Hours of day, includes fractional hours Return type: float
-
ocbpy.ocb_time.
deg2hr
(lon)[source]¶ Convert from degrees to hours.
Parameters: lon (float or array-like) – Longitude-like value in degrees Returns: lt – Local time-like value in hours Return type: float or array-like
-
ocbpy.ocb_time.
fix_range
(values, min_val, max_val, val_range=None)[source]¶ Ensure cyclic values lie below the maximum and at or above the mininum.
Parameters: Returns: fixed_vals – Values adjusted to lie min_val <= fixed_vals < max_val
Return type:
-
ocbpy.ocb_time.
get_datetime_fmt_len
(datetime_fmt)[source]¶ Get the lenght of a string line needed for a specific datetime format.
Parameters: datetime_fmt (str) – Formatting string used to convert between datetime and string object Returns: str_len – Minimum length of a string needed to hold the specified data Return type: int Notes
See the datetime documentation for meanings of the datetime directives
-
ocbpy.ocb_time.
glon2slt
(glon, dtime)[source]¶ Convert from geographic longitude to solar local time.
Parameters: - glon (float or array-like) – Geographic longitude in degrees
- dtime (dt.datetime) – Universal time as a timestamp
Returns: slt – Solar local time in hours
Return type: float or array-like
-
ocbpy.ocb_time.
hr2deg
(lt)[source]¶ Convert from degrees to hours.
Parameters: lt (float or array-like) – Local time-like value in hours Returns: lon – Longitude-like value in degrees Return type: float or array-like
-
ocbpy.ocb_time.
hr2rad
(lt)[source]¶ Convert from hours to radians.
Parameters: lt (float or array-like) – Local time-like value in hours Returns: lon – Longitude-like value in radians Return type: float or array-like
-
ocbpy.ocb_time.
rad2hr
(lon)[source]¶ Convert from radians to hours.
Parameters: lon (float or array-like) – Longitude-like value in radians Returns: lt – Local time-like value in hours Return type: float or array-like
-
ocbpy.ocb_time.
slt2glon
(slt, dtime)[source]¶ Convert from solar local time to geographic longitude.
Parameters: - slt (float or array-like) – Solar local time in hours
- dtime (dt.datetime) – Universal time as a timestamp
Returns: glon – Geographic longitude in degrees
Return type: float or array-like