Supported Instrument Data Sets

Currently, support is included for files from the following sources:

  1. SuperMAG (ocbpy.instruments.supermag)
  2. SuperDARN Vorticity (ocbpy.instruments.vort)
  3. pysat (ocbpy.instruments.pysat_instruments)

These routines may be used as a guide to write routines for other data sets. A ocbpy.instruments.general loading sub-module is also provided for ASCII files. All the non-boundary data routines are part of the ocbpy.instruments module. Support for time-handling that may be useful for specific data sets are provided in ocbpy.ocb_time.

General Instrument Module

General loading routines for data files

ocbpy.instruments.general.load_ascii_data(filename, hlines, gft_kwargs={}, hsplit=None, datetime_cols=None, datetime_fmt=None, int_cols=None, str_cols=None, max_str_length=50, header=None)[source]

Load an ascii data file into a dict of numpy array

Parameters:
  • filename (str) – data file name
  • hlines (int) – number of lines in header. If zero, must include header.
  • gft_kwargs (dict) – Dictionary holding optional keyword arguments for the numpy genfromtxt routine (default=dict())
  • hsplit (str, NoneType) – character seperating data labels in header. None splits on all whitespace characters. (default=None)
  • datetime_cols (list, NoneType) – If there are date strings or values that should be converted to a datetime object, list them in order here. Not processed as floats. NoneType produces an empty list. (default=None)
  • datetime_fmt (str, NoneType) – Format needed to convert the datetime_cols entries into a datetime object. Special formats permitted are: ‘YEAR SOY’, ‘SOD’. ‘YEAR SOY’ must be used together; ‘SOD’ indicates seconds of day, and may be used with any date format (default=None)
  • int_cols (list, NoneType) – Data that should be processed as integers, not floats. NoneType produces an empty list. (default=None)
  • str_cols (list, NoneType) – Data that should be processed as strings, not floats. NoneType produces an empty list. (default=None)
  • max_str_length (int) – Maximum allowed string length. (default=50)
  • header (list, NoneType) – Header string(s) where the last line contains whitespace separated data names. NoneType produces an empty list. (default=None)
Returns:

  • header (list of strings) – Contains all specified header lines
  • out (dict of numpy.arrays) – The dict keys are specified by the header data line, the data for each key are stored in the numpy array

Notes

Data is assumed to be float unless otherwise stated.

ocbpy.instruments.general.test_file(filename)[source]

Test to ensure the file is small enough to read in

Parameters:filename (str) – Filename to test
Returns:good_flag – True if good, bad if false
Return type:bool

Notes

Python can only allocate 2GB of data without crashing

SuperMAG Instrument Module

Perform OCB gridding for SuperMAG data

Notes

SuperMAG data available at: http://supermag.jhuapl.edu/

ocbpy.instruments.supermag.load_supermag_ascii_data(filename)[source]

Load a SuperMAG ASCII data file

Parameters:filename (str) – SuperMAG ASCI data file name
Returns:out – The dict keys are specified by the header data line, the data for each key are stored in the numpy array
Return type:dict
ocbpy.instruments.supermag.supermag2ascii_ocb(smagfile, outfile, hemisphere=0, ocb=None, ocbfile='default', instrument='', max_sdiff=600, min_sectors=7, rcent_dev=8.0, max_r=23.0, min_r=10.0)[source]

Coverts and scales the SuperMAG data into OCB coordinates

Parameters:
  • smagfile (str) – file containing the required SuperMAG file sorted by time
  • outfile (str) – filename for the output data
  • hemisphere (int) – Hemisphere to process (can only do one at a time). 1=Northern, -1=Southern, 0=Determine from data (default=0)
  • ocb (ocbpy.OCBoundary or NoneType) – OCBoundary object with data loaded from an OC boundary data file. If None, looks to ocbfile
  • ocbfile (str) – file containing the required OC boundary data sorted by time, or ‘default’ to load default file for time and hemisphere. Only used if no OCBoundary object is supplied (default=’default’)
  • instrument (str) – Instrument providing the OCBoundaries. Requires ‘image’ or ‘ampere’ if a file is provided. If using filename=’default’, also accepts ‘amp’, ‘si12’, ‘si13’, ‘wic’, and ‘’. (default=’’)
  • max_sdiff (int) – maximum seconds between OCB and data record in sec (default=600)
  • min_sectors (int) – Minimum number of MLT sectors required for good OCB (default=7).
  • rcent_dev (float) – Maximum number of degrees between the new centre and the AACGM pole (default=8.0)
  • max_r (float) – Maximum radius for open-closed field line boundary in degrees default=23.0)
  • min_r (float) – Minimum radius for open-closed field line boundary in degrees (default=10.0)
Raises:

IOError – If unable to open the input or output file

Notes

May only process one hemisphere at a time. Scales the magnetic field observations using ocbpy.ocb_scale.normal_curl_evar.

See also

ocbpy.ocb_scale.normal_curl_evar()

SuperDARN Vorticity Instrument Module

Perform OCB gridding for SuperDARN vorticity data

Notes

Specialised SuperDARN data product, available from: gchi@bas.ac.uk

ocbpy.instruments.vort.load_vorticity_ascii_data(vortfile, save_all=False)[source]

Load SuperDARN vorticity data files.

Parameters:
  • vortfile (str) – SuperDARN vorticity file in block format
  • save_all (bool) – Save all data from the file (True), or only data needed to calculate the OCB coordinates and normalised vorticity (False). (default=False)
Returns:

vdata – Dictionary of numpy arrays
Return type:

dict
ocbpy.instruments.vort.vort2ascii_ocb(vortfile, outfile, hemisphere=0, ocb=None, ocbfile='default', instrument='', max_sdiff=600, save_all=False, min_sectors=7, rcent_dev=8.0, max_r=23.0, min_r=10.0)[source]

Coverts the location of vorticity data from AACGM to OCB coordinates

Parameters:
  • vortfile (str) – file containing the required vorticity file sorted by time
  • outfile (str) – filename for the output data
  • hemisphere (int) – Hemisphere to process (can only do one at a time). 1=Northern, -1=Southern, 0=Determine from data (default=0)
  • ocb (ocbpy.ocboundary.OCBoundary or NoneType)) – Object containing open closed boundary data or None to load from file
  • ocbfile (str) – file containing the required OC boundary data sorted by time, ignorned if OCBoundary object supplied. (default=’default’)
  • instrument (str) – Instrument providing the OCBoundaries. Requires ‘image’ or ‘ampere’ if a file is provided. If using filename=’default’, also accepts ‘amp’, ‘si12’, ‘si13’, ‘wic’, and ‘’. (default=’’)
  • max_sdiff (int) – maximum seconds between OCB and data record in sec (default=600)
  • save_all (bool) – Save all data (True), or only that needed to calcuate OCB and vorticity (False). (default=False)
  • min_sectors (int) – Minimum number of MLT sectors required for good OCB. (default=7)
  • rcent_dev (float) – Maximum number of degrees between the new centre and the AACGM pole (default=8.0).
  • max_r (float) – Maximum radius for open-closed field line boundary in degrees. (default=23.0)
  • min_r (float) – Minimum radius for open-closed field line boundary in degrees (default=10.0)
Raises:
  • IOError – If unable to open input or output file
  • ValueError – If unable to retrieve all necessary data from the input file

Notes

Input header or col_names must include the names in the default string.

pysat Instrument Module

Time Handling Module

Routines to convert from different file timekeeping methods to datetime

ocbpy.ocb_time.convert_time(year=None, soy=None, yyddd=None, sod=None, date=None, tod=None, datetime_fmt='%Y-%m-%d %H:%M:%S')[source]

Convert to datetime from multiple time formats

Parameters:
  • year (int or NoneType) – Year or None if not in year-soy format (default=None)
  • soy (int or NoneType) – Seconds of year or None if not in year-soy format (default=None)
  • yyddd (str or NoneType) – String containing years since 1900 and 3-digit day of year (default=None)
  • sod (int, float, or NoneType) – Seconds of day or None if the time of day is not in this format (default=None)
  • date (str or NoneType) – String containing date information or None if not in date-time format (default=None)
  • tod (str or NoneType) – String containing time of day information or None if not in date-time format (default=None)
  • datetime_fmt (str) – String with the date-time or date format (default=’%Y-%m-%d %H:%M:%S’)
Returns:

dtime – Datetime object
Return type:

dt.datetime
ocbpy.ocb_time.datetime2hr(dtime)[source]

Calculate hours of day from datetime

Parameters:dtime (dt.datetime) – Universal time as a timestamp
Returns:uth – Hours of day, includes fractional hours
Return type:float
ocbpy.ocb_time.deg2hr(lon)[source]

Convert from degrees to hours

Parameters:lon (float or array-like) – Longitude-like value in degrees
Returns:lt – Local time-like value in hours
Return type:float or array-like
ocbpy.ocb_time.fix_range(values, min_val, max_val, val_range=None)[source]

Ensure cyclic values lie below the maximum and at or above the mininum

Parameters:
  • values (int, float, or array-like) – Values to adjust
  • min_val (int or float) – Maximum that values may not meet or exceed
  • max_val (int or float) – Minimum that values may not lie below
  • val_range (int, float, or NoneType) – Value range or None to calculate from min and max (default=None)
Returns:

fixed_vals – Values adjusted to lie min_val <= fixed_vals < max_val
Return type:

int, float, or array-like
ocbpy.ocb_time.get_datetime_fmt_len(datetime_fmt)[source]

Get the lenght of a string line needed for a specific datetime format

Parameters:datetime_fmt (str) – Formatting string used to convert between datetime and string object
Returns:str_len – Minimum length of a string needed to hold the specified data
Return type:int

Notes

See the datetime documentation for meanings of the datetime directives

ocbpy.ocb_time.glon2slt(glon, dtime)[source]

Convert from geographic longitude to solar local time

Parameters:
  • glon (float or array-like) – Geographic longitude in degrees
  • dtime (dt.datetime) – Universal time as a timestamp
Returns:

slt – Solar local time in hours
Return type:

float or array-like
ocbpy.ocb_time.hr2deg(lt)[source]

Convert from degrees to hours

Parameters:lt (float or array-like) – Local time-like value in hours
Returns:lon – Longitude-like value in degrees
Return type:float or array-like
ocbpy.ocb_time.hr2rad(lt)[source]

Convert from hours to radians

Parameters:lt (float or array-like) – Local time-like value in hours
Returns:lon – Longitude-like value in radians
Return type:float or array-like
ocbpy.ocb_time.rad2hr(lon)[source]

Convert from radians to hours

Parameters:lon (float or array-like) – Longitude-like value in radians
Returns:lt – Local time-like value in hours
Return type:float or array-like
ocbpy.ocb_time.slt2glon(slt, dtime)[source]

Convert from solar local time to geographic longitude

Parameters:
  • slt (float or array-like) – Solar local time in hours
  • dtime (dt.datetime) – Universal time as a timestamp
Returns:

glon – Geographic longitude in degrees
Return type:

float or array-like
ocbpy.ocb_time.year_soy_to_datetime(yyyy, soy)[source]

Converts year and soy to datetime

Parameters:
  • yyyy (int) – 4 digit year
  • soy (float) – seconds of year
Returns:

dtime – datetime object
Return type:

dt.datetime
ocbpy.ocb_time.yyddd_to_date(yyddd)[source]

Convert from years since 1900 and day of year to datetime

Parameters:yyddd (str) – String containing years since 1900 and day of year (e.g. 100126 = 2000-05-5).
Returns:dtime – Datetime object containing date information
Return type:dt.datetime