Tables, views and columns - Solar Orbiter Archive
Tables, VIEWS and COLUMNS
The Table Access Protocol (TAP) server allows read access to the SOAR database of metadata about the data products. The information is stored in various tables, with rows and columns, for different purposes or covering different types of product. The most useful columns of these tables are gathered into other tables known as 'views', whose names start with 'v_'.
While the tables are simple in that they consist of rows and columns, sometimes the meaning or contents of a column are not obvious and these descriptions are currently not available along with the tables/views. It should also be noted that these tables and views (tables beginning with 'v_'), while very useful, were not designed with the user in mind, and so some column names or view content may not be ideal for users. This will be corrected in the future, but for now, this page lists the tables and columns along with descriptions of their contents.
The Science (often referred to as SCI, and in the table names as 'sc') and Low Latency (often referred to as LL, and as 'll' in the table names) products each have a data_item_id, and this is the same for all versions of that product. Only the latest versions of the SCI and LL products are given in the two tables v_sc_data_item and v_ll_data_item, respectively. These are likely to be the most useful tables for SCI and LL data, and this is why a list of the most useful columns are described on the How-To page, but a complete list is provided below.
In the explanations below, the contents of the tables are the same for SCI and LL (where the title of the table/view explanations includes <ll/sc>), so are given together.
Product tables
Latest version product tables: v_<ll/sc>_data_item
(v_sc_data_item for science products, v_ll_data_item for low latency data products)
All versions of all products: v_<ll/sc>_repository_file
Remote sensing instrument tables (FITS):
There are four views for each remote sensing instrument, where <instrument> can be eui (EUI), met (Metis), phi (PHI), shi (SoloHI), spi (SPICE), or stx (STIX)
v_<instrument>_<ll/sc>_extension_fits
For example, for EUI:
v_eui_sc_fits
v_eui_ll_fits
v_eui_sc_extension_fits
v_eui_ll_extension_fits
In situ instrument tables (CDF):
v_cdf_dataset
v_cdf_plot_metadata
v_cdf_variable
SPICE kernels
v_auxiliary_file
Data processing level explanation
| LEVEL | SOURCE | DATA TYPE | FORMAT AND METADATA CONTENT |
|---|---|---|---|
| L0 | Instrument Team | 'Raw' data, unpacked and decompressed data | Data format preferable in FITS or CDF, but can be different if this is more appropriate. Metadata reflect the information that was available in the TM packets only. |
| L1 | Instrument Team | "Engineering" data, uncalibrated | FITS or CDF, metadata follows Solar Orbiter standard for L1. |
| L2 | Instrument Team | "Calibrated" data, science quality | FITS or CDF, metadata follows Solar Orbiter standard for L2 (see Section 3): full attitude information in WCS coordinate frame and time in UTC. |
| L3 | Instrument Team | Higher-level (derived) data | Data format as appropriate. The format of Level-3 data, calibration data and ancillary data can be chosen depending on the type of data product and the objectives. However, as much as possible standard formats should be used (MPEG, FITS, JPEG2000, CDF, PNG, ...). |
| CAL | Instrument Team | Calibration data | Data format as appropriate. Not all calibration data are necessarily open to the scientific community. |
| ANC | Instrument Team/SOC | Ancillary data | Data format as appropriate. |
| LL01 | SOC | LL engineering data, output of LL pipeline | FITS or CDF, metadata follows Solar Orbiter standard, with some specifics for LL01 data (see [LLFITSICD] and [LLCDFICD] : time in OBT, attitude in instrument detector reference frame. Private data. |
| LL02 | SOC | Operational LL data, enhanced with S/C HK | FITS or CDF, metadata follows Solar Orbiter standard, with some specifics for LL02 data (see [LLFITSICD] and [LLCDFICD]: time in UTC, attitude in WCS coordinate frame. |
| LL03 | SOC | Visualisation of operational LL data, in "quicklook" format | 'Quicklook' data in PNG or JPEG2000 (details TBC). This level is also used for LL data products derived from (multiple) LL02 products. |
v_<ll/sc>_data_item - Latest Versions:
The columns are the same for the SCI and LL, but the examples may not be. As stated above, the most useful columns are described on the How-To page, but here the complete list is given.
If there is only space below, try refreshing the page.
| COLUMN NAME | FITS KEYWORD | CDF KEYWORD | DESCRIPTION | EXAMPLE |
|---|---|---|---|---|
| begin_time | DATE-BEG | TIME_MIN | Start time of observation | '2022-03-21T00:00:00.0' |
| data_item_id | ID used for this data product, consisting of the filename up to the datetime of the observation, i.e., 'solo_' + "level" + "descriptor" + begin_time + (optionally) end_time | 'solo_L2_mag-rtn-normal-1-minute_20220321' | ||
| data_item_oid | Numerical ID used by the database for this data product. | 632982 | ||
| data_type | Science or low latency, so SCI for sc table and LL for ll table. Not the same as the CDF keyword Data_type (data_type in v_cdf_dataset) which refers to the processing level (e.g., L2 | 'SCI' | ||
| "descriptor" | Descriptor of the type of product, similar to the name of a dataset, to describe the generalised contents of the product. Note the quotes since this is a TAP reserved word. | 'mag-rtn-normal-1-minute' | ||
| end_time | DATE-END | TIME_MAX | Date and time of the end of the observation. | '2022-03-22T00:00:00.0' |
| file_format | File format of the product. At present, this can be BIN, CDF, FITS, H5, or JP2 | 'CDF' | ||
| filename | FILENAME | Logical_file_id | The full filename - this is the data_item_id plus the version, an optional section of free-field text and the extension. | 'solo_L2_mag-rtn-normal-1-minute_20220321_V01.cdf' |
| filesize | Size of the file in bytes | 32605 | ||
| generation_time | DATE | Generation_Date | Date and time the file was produced in ISO format | '2022-09-20T13:50:16.0' |
| icon | icon code | |||
| insertion_time | Date and time file was ingested into the archive in ISO format | '2022-09-20T15:19:07.451' | ||
| instrument | INSTRUME | Instrument | Main instrument name: one of EUI, RPW, EPD, SWA, MAG, SPICE, SOLOHI, METIS, PHI or STIX | 'MAG' |
| is_active | Flag for whether the file is active or not. Currently, data products in the SOAR are not deactivated. | 1 | ||
| is_public | Flag for whether the file is public or not. Private files can be reached by authorised members through authentication. | 1 | ||
| "level" | LEVEL | Processing level of the data. L0, L1, L2, or L3 for SCI; LL01 or LL02 for LL. Note that this is in quotes because it is a reserved word for TAP. | 'L2' | |
| ll_flag | Flag for whether this product is Low Latency or not. | 0 | ||
| postcard_item_oid | Database identification code for the postcard file, if it exists. | |||
| prop_end | Date and time of end of proprietary period. Currently not used. | '1900-01-01T00:00:00' | ||
| sc_begin_time | OBT_BEG | Start of observation in spacecraft time | may be NULL | |
| sc_end_time | OBT_END | End of observation in spacecraft time | may be NULL | |
| sensor | Sensor used, e.g., SWA has PAS, HIS and EAS | may be NULL |
Lists all versions of the files received into the SOAR.
| COLUMN NAME | FITS KEYWORD | CDF KEYWORD | DESCRIPTION | EXAMPLE |
|---|---|---|---|---|
| active | 1 | |||
| backup | ||||
| creation date | DATE | File generation time, so same as generation_time in v_[sc/ll]_data_item | ||
| data_item_id | Same as in v_[sc/ll]_data_item | |||
| data_item_oid | ||||
| filename | FILENAME | |||
| filepath | ||||
| filesize | ||||
| icon | ||||
| item_version | VERSION | |||
| modification_date | Ingestion time, so same as insertion_time in v_[sc/ll]_data_item | |||
| postcard_item_oid | ||||
| repository_file_oid | ||||
| soop_name | SOOPNAME (FITS), SOOP_NAME (CDF) | Long name of the Solar Orbiter Observing Plan, see SOOP page | ||
| soop_type | SOOPTYPE (FITS), SOOP_TYPE (CDF) | Short name/code of the Solar Orbiter Observing Plan, see SOOP page |
Note that multi-dimensional data will have multiple entries in the table, one for each dimension (see dimension_index).
As stated above, these tables were not necessarily designed with the user in mind, and therefore some of these columns will have no value to the user, but are included here for completeness.
If there is only space below, try refreshing the page.
| COLUMN NAME | FITS KEYWORD | DESCRIPTION | EXAMPLE |
|---|---|---|---|
| apid | APID | APID number of associated TM | |
| binning_factor | NBINn | Specifies the binning factor in the nth dimension, i.e., the actual number of pixels binned in that direction. Default value (no binning) is 0 | 2 |
| bit_pixel | BITPIX | Number of bits per pixel, and can only take values 8, 16, 32, 64 for binary integers and -32 or -64 for single, resp. double precision floating points | 16 |
| blank | BLANK | For integer data (positive values of BIXPIX/bit_pixel), the integer value used to mark undefined pixels. | 2147483648 |
| bscale | BSCALE | Ratio of physical to array value at 0 offset | 1.0 |
| btype | BTYPE | Type of data | 'Intensity' |
| bunit | BUNIT | Units of physical value, see bscale and bzero, 'DN' for 'data number' if unitless | 'DN' |
| bzero | BZERO | Physical value for the array value 0, see bscale. | 0.0 |
| c_delt[n] | CDELT[n] | Pixel scale along axis n [arcsec] | 3.17 |
| checksum | CHECKSUM | ||
| complete_flag | COMPLETE | Whether the file is complete or not - this only applies to LL data. | 1 |
| compressed_ratio | COMP_RAT | Ratio of size uncompressed vs compressed image | 4 |
| creator | CREATOR | Name of the software that produced the FITS file | 'MyFitsSoftware' |
| cr_pix[n] | CRPIX[n] | Reference pixel location along axis n | |
| cr_val[n] | CRVAL[n] | Value of reference pixel along n axis, where n will be 1, 2 or 3. [arcsec] | 0.0 |
| ctype1 | CTYPE1 | Helioprojective longitude (Solar X) | ’HPLN-TAN’ |
| ctype2 | CTYPE2 | Helioprojective latitude (Solar Y) | ’HPLT-TAN’ |
| ctype3 | CTYPE3 | ||
| cunit[n] | CUNIT[n] | Units along axis n, where n will be 1, 2 or 3 | ’arcsec’ |
| data_item_oid | Data item ID identifier | ||
| data_max | DATAMAX | Maximum valid physical value | 3588.0 |
| data_min | DATAMIN | Minimum valid physical value | 10.0 |
| datasum | DATASUM | ||
| date_average | DATE-AVG | Average time of observation (midpoint between start and end times) | '2020-01-22T14:00:30.000' |
| date_begin | DATE-BEG | Start time of observation | '2020-01-22T14:00:00.000' |
| date_end | DATE-END | End time of observation | '2020-01-22T14:01:00.000' |
| date_obsv | DATE-OBS | Time of observation. Same as date_begin (DATE-BEG) | '2020-01-22T14:00:00.000' |
| detector | DETECTOR | Instrument subunit or sensor | 'FSI' |
| dimension_index | n, the dimension that this row is specifying | 1 | |
| distance_sun_obsv | DSUN_OBS | Distance of the spacecraft from the center of the Sun, P L2+ in metres. | 150707994180.746 |
| elapse_time | TELAPSE | ||
| eui_fits_oid | instr_fits_oid - index | ||
| eui_parent_id | PARENT | Name of file that was processed to create this file | 'solo_L1_eui-fsi304-image_20200512T085922556_V04.fits' |
| eui_primary_naxis_oid | |||
| eui_wcs_attitude_oid | Index of WCSAXES | ||
| exposure | XPOSURE | Total effective exposure time (s) | 10.0 |
| extend | EXTEND | Boolean specifying whether the FITS file contains extensions - see the extension tables for information about the extensions. | F |
| filename | FILENAME | FITS filename | 'solo_L2_eui-fsi304-image_20200512T085922556_V04.fits' |
| file_raw | FILE_RAW | Raw file (telemetry) that the FITS file was derived from | 'BatchRequest.PktTmRaw. SOL.0.2018.262.13.26.30.860.jphr @2018.262.13.26.31.239.1.xml' |
| filter | FILTER | Specify the filter in multi-filter instruments | <filter name> |
| fits_compress | COMPRESS | Data compression quality. Can be "None", "Lossless", "Lossy-high quality", "Lossy-strong" or "Lossy-extreme" | 'Lossless' |
| fits_dataset_oid | |||
| fits_date | DATE | Date and time of FITS file creation | '2020-01-27T14:23:57.167' |
| fits_instrument | INSTRUME | Instrument name, one of: 'EUI', 'Metis', 'PHI', 'STIX', 'SoloHI', 'EPD', 'MAG', 'SPICE', 'RPW', 'SWA' | 'PHI' |
| hc_inertial_x_obsv | HCIX_OBS | X axis position of the spacecraft in Heliocentric Inertial coordinates, in metres. | 109043378382. |
| hc_inertial_y_obsv | HCIY_OBS | Y axis position of the spacecraft in Heliocentric Inertial coordinates, in metres. | 94332806329.1 |
| hc_inertial_z_obsv | HCIZ_OBS | Z axis position of the spacecraft in Heliocentric Inertial coordinates, in metres. | -12321790987.1 |
| h_earth_ex_obsv | HEEX_OBS | X axis position of the spacecraft in Heliocentric Earth Ecliptic coordinates, in metres. | 150707994015.506 |
| h_earth_ey_obsv | HEEY_OBS | Y axis position of the spacecraft in Heliocentric Earth Ecliptic coordinates, in metres. | 4059466.73665196 |
| h_earth_ez_obsv | HEEZ_OBS | Z axis position of the spacecraft in Heliocentric Earth Ecliptic coordinates, in metres. | 5772938.06008995 |
| hg_latitude_obsv | HGLT_OBS | Stonyhurst heliographic latitude of the spacecraft, in degrees. | -4.18794601814401 |
| hg_longitud_obsv | HGLN_OBS | Stonyhurst heliographic longitude of the spacecraft, in degrees. | 0.00131206248030214 |
| history | HISTORY | ||
| index | |||
| "level" | LEVEL | Data processing level of file, see note below. | 'L2' |
| logical_source | |||
| num_dimensions | NAXIS | The number of dimensions in the data (0 indicates header only). | 2 |
| num_pixels_per_axis | NAXISn | The number of pixels along this axis, see dimension_index | 1024 |
| observation_mode | OBS_MODE | Observation mode used to acquire the image | 'EUV_CH' |
| observatory | OBSRVTRY | Satellite name | 'Solar Orbiter' |
| obt_beginning | OBT_BEG | Start acquisition time in onboard time (OBT). Note that this is not a decimal number - see note below. | 636564608.5277252 |
| obt_end | OBT_END | End acquisition time in onboard time (OBT). Note that this is not a decimal number - see note below. | 636564618.5293884 |
| origin | ORIGIN | Location where FITS file was generated | 'Solar Orbiter SOC, ESAC' |
| pc[i]_[j] | PC[i]_[j] | WCS coordinate transformation matrix. PCi_j co-ordinate transformation matrix element used to express dependence of an intermediate world co-ordinate i on data dimension j, where i, j = 1, 2, or 3 | 0.8660254 |
| pixel_begin | PXBEGn | First read-out pixel in dimension n | 1 |
| pixel_end | PXENDn | Last read-out pixel in dimension n | 512 |
| rsun_arc | RSUN_ARC | Apparent photospheric solar radius in arc seconds. | 952.565823288377 |
| simple | SIMPLE | Boolean for whether the file conforms to FITS standard, and so can only be 'T' | T |
| spectral_system | SPECSYS | Reference frame for spectral axes | 'TOPOCENT' |
| telescope | TELESCOP | Telescope/sensor name | 'SOLO/EUI/FSI' |
| time_system | TIMESYS | System used for time keywords, i.e., UTC or OBT | 'UTC' |
| total_binning_factor | NBIN | Total binning factor, product of all NBINn values. | 4 |
| velocity_system | VELOSYS | Velocity correction applied to spectral data [m/s] | 0.0 |
| version_cal | VERS_CAL | Version number of the full package and calibration dataand products applied to the data. | 2.4 |
| version_item | VERSION | Version of this data item, padded to two digits | '03' |
| version_sw | VERS_SW | Version of the software that produced the FITS file, specified in creator (CREATOR) | 2.5 |
| waveband | WAVEBAND | Bandpass description | 'ENERGY_L' |
| wavelength | WAVELNTH | Characteristic wavelength at which observations was taken, in angstroms | 174.0 |
| wavemax | WAVEMAX | Max wavelength where response > 0.05 of max | 170.1 |
| wavemin | WAVEMIN | Min wavelength where response > 0.05 of max | 176.0 |
| wcs_name | WCSNAME | Name of coordinate system | 'Helioprojective- cartesian’ |
v_<instr>_<sc/ll>_extension_fits
The extension details go into here:
| COLUMN NAME | FITS KEYWORD | DESCRIPTION | EXAMPLE |
|---|---|---|---|
| binning_factor | NBINn | ||
| bin_table_fields | |||
| bit_pix | BITPIX | ||
| column_number | |||
| data_dimensions | |||
| data_format | |||
| data_type | |||
| data_units | |||
| dimension_index | n | ||
| eui_bin_column_oid | |||
| eui_fits_oid | |||
| eui_hdu_extension_oid | |||
| eui_hdu_naxis_oid | |||
| extension | |||
| ext_name | |||
| group_count | |||
| num_axis | |||
| num_order | |||
| num_pixels_per_axis | |||
| param_count | |||
| pixel_begin | |||
| pixel_end |
|
OBT explanation
The OnBoard Time is a double-precision floating-point number in onboard time units which is not actually decimal. It consists of a ‘coarse’ time and, after the decimal point, a ‘fine’ time for sub-seconds which is from a counter from 0-65535.
OBT = coarse + fine/65536
So for a given OBT of 637551003.4117279, expressed as a decimal in the metadata, 637551003 is the coarse part, and 4117279 is the fine -> round(0. 4117279 * 65536) = round(26982.9996544) = 26983
The value fed into the spice time conversion routine is then 637551003:26983
The archive will store the floating-point number given in the metadata so that the information is retained.