Frequently Asked Questions (FAQ) - Cluster Science Archive
FREQUENTLY ASKED QUESTIONS (FAQ)
Q2. The CSA web GUI won't load
Q3. On Windows: WinZip may truncate filenames or corrupt files, please use 7zip
Q4. Mac: why clicking on .tar.gz, sometimes, it does not create a folder with the datasets requested?
Q5. Mac: Version 1.18 of wget refuses to accept the certificate?
Q6. Cookie generation: hash (#) and ampersand (&) characters in your password can cause problems
Q7. The pipe character is reserved and cannot be used in passwords
Q8. IDL command line scripts: bug and workaround found for IDL version less than 8.4
Q9. How do I know the dataset content in a Cluster CDF file with Matlab?
Q10. How do I update my scripts to work with HTTPS?
Q11. How I installed CEFLIB for Python on Catalina
Q1. How can I stay logged into the CSA GUI?
Unfortunately, you aren't able to stay logged in, however, your browser can remember your username and password.
Q2. The CSA web GUI won't load
It could be that there has been an new release since the page was last loaded. A hard refresh may work, or you may need to delete history and cookies, depending on your browser.
Q3. On Windows: WinZip may truncate filenames or corrupt files, please use 7zip
7zip is an open source software able to open the CSA files: http://www.7-zip.org/
WinZip version 17.0 on Windows 64 bit is known to truncate filenames, with extension .ce or .cd instead of .cef and .cdf
Possible file corruption has also been reported in the past. A default setting must be changed on the WinZip compression utility. The default setting may corrupt data, while data is being unzipped. Solution: Open WinZip Classic. Select Options > Configuration, and click the Miscellaneous tab. Uncheck the 'TAR file smart CR/LF conversion' box. The box will remain unchecked until it is checked again, or WinZip is reinstalled. Unzip and import the data.
Q4. Mac: why clicking on .tar.gz sometimes does not create a folder with the datasets requested?
Any bunch of files requested on the CSA GUI will be compacted and downloaded as one *.tar.gz file. By default, Mac is opening a *.tar.gz file with a native program called Archive Utility. This program will be called when clicking on such a file and will create a folder with a tree of folder(s) with each type of file inside each folder. If among the files you request, you request text files (*.txt) such as FGM calibration files. These files will be compacted as *.tar.gz files inside the overall single .tar.gz file delivered. This is not properly handled with Archive Utility which will only gunzip the file but will not create the overall folder. Either use another archive utility software or use a terminal window and untar it with the command: tar -xvf *.tar
Q5. Mac: Version 1.18 of wget refuses to accept the certificate?
On installing GNU wget-1.18-0 on Mac OS X El Capitan using Rudix (as this gives an installable package), the CSA security certificate was no longer accepted and wget advised that the only way to allow the data transfer is to add the flag --no-check-certificate to the wget command.
Re-install wget from GNU using homebrew; install Homebrew using:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”
then install wget with
brew install wget --with-libressl
and the requests using wget now work as expected.
Q6. Cookie generation: hash (#) and ampersand (&) characters in your password can cause problems
If you use the # or & signs in your password, the generation of a new cookie for a command line request via https or wget may fail unless you replace them with the URL encoded equivalent (# = %23 and & = %26). You can also change your password to not include the # or & characters.
Q7. The pipe character is reserved and cannot be used in passwords
Using the | (pipe) symbol in your password causes problems in the system and so cannot be used in passwords. If you need to, you can reset your password.
Q8. IDL command line scripts: bug and workaround found for IDL version less than 8.4
Two IDL scripts are available to login and download data directly from IDL, please follow this link. If you have IDL with an older version than 8.4, these programs may not work and display an error message as follows
% Loaded DLM: URL.
% IDLNETURL::GET: CCurlException: Error: Http Get Request Failed. Error = SSL certificate problem: self signed certificate in certificate chain, Curl Error Code = 60..
% Execution halted at: CSA_LOGIN
To quickly solve this issue:
in the login script (csa_login.pro), please add
csa_login_obj->SetProperty, ssl_verify_peer = 0
in the product script (csa_product.pro), please add
csa_product_obj->SetProperty, ssl_verify_peer = 0
Alternatively, please have a look at the NASA SPDF web services page.
Q9. How do I look into the dataset content of a Cluster CDF file with Matlab?
Let's assume, you have downloaded a
DC magnetic field file recorded in 2004 May 16 between 22:30 and 22:34 by FGM on C1 with a time resolution of 0.2s (i.e. 5 Vectors Per Second or 5VPS),
dataset id: C1_CP_FGM_5VPS
Full filename: C1_CP_FGM_5VPS__20040516_223000_20040516_223400_V140304.cdf
Step 1: Open the FGM file just downloaded
>> [filename, pathname]=uigetfile('*.cdf','Select Data');
>> full_filename=[pathname filename];
Step 2: create a structure containing the information of your CDF, let’s call it infocdf
>> infocdf=cdfinfo(full_filename)
infocdf =
struct with fields:
Filename: 'C1_CP_FGM_5VPS__20040516_223000_20040516_223400_V140304.cdf'
FileModDate: '18-Apr-2017 16:03:12'
FileSize: 98999
Format: 'CDF'
FormatVersion: '3.3.1'
FileSettings: [1×1 struct]
Subfiles: {}
Variables: {11×6 cell}
GlobalAttributes: [1×1 struct]
VariableAttributes: [1×1 struct]
Step 3: let’s look at the content of this file
>> infocdf.Variables
ans =
11×6 cell array
'time_tags__C1_CP_…' [1×2 double] [1200] 'epoch' 'T/' 'Full'
'half_interval__C1…' [1×2 double] [1200] 'single' 'T/' 'Full'
'B_vec_xyz_gse__C1…' [1×2 double] [1200] 'single' 'T/T' 'Full'
'B_mag__C1_CP_FGM_…' [1×2 double] [1200] 'single' 'T/' 'Full'
'sc_pos_xyz_gse__C…' [1×2 double] [1200] 'single' 'T/T' 'Full'
'range__C1_CP_FGM_…' [1×2 double] [1200] 'int32' 'T/' 'Full'
'tm__C1_CP_FGM_5VPS' [1×2 double] [1200] 'int32' 'T/' 'Full'
'B_vec_xyz_gse__C1…' [1×2 double] [ 1] 'char' 'F/T' 'Full'
'B_vec_xyz_gse__C1…' [1×2 double] [ 1] 'char' 'F/T' 'Full'
'sc_pos_xyz_gse__C…' [1×2 double] [ 1] 'char' 'F/T' 'Full'
'sc_pos_xyz_gse__C…' [1×2 double] [ 1] 'char' 'F/T' ‘Full'
Step 4: let’s extract the variable names and their corresponding number of columns in the file (i.e. components)
>> nb_of_columns=cell2mat(infocdf.Variables(1:7,2));components=nb_of_columns(:,1);
>> reduced_infocdf=infocdf.Variables(1:7,1:2);
>> for i=1:7, reduced_infocdf(i,2)={components(i)};end
>> reduced_infocdf
reduced_infocdf =
7×2 cell array
'time_tags__C1_CP_FGM_5VPS' [1]
'half_interval__C1_CP_FGM_5VPS' [1]
'B_vec_xyz_gse__C1_CP_FGM_5VPS' [3]
'B_mag__C1_CP_FGM_5VPS' [1]
'sc_pos_xyz_gse__C1_CP_FGM_5VPS' [3]
'range__C1_CP_FGM_5VPS' [1]
'tm__C1_CP_FGM_5VPS' [1]
Q10. How do I update my scripts to work with HTTPS?
Q11. How I installed CEFLIB for Python on Catalina
See the CEFLIB installation instruction page on http://ceflib.irap.omp.eu/page=install
Prerequisites:
* Numpy (from Anaconda with other science data libraries/modules)
* Xcode (and probably Xcode Command Line Tools): Xcode installed from App Store (needs to be opened and installed even after App Store says ‘Open’), and then Command Line Tools for that version (11.5 in this case) from https://developer.apple.com/download/more/
After uncompressing and extracting the source, then choosing a place to install (Steps 1 and 2), for Step 2.3, you can find the working Numpy path by running Python (or ipython here), then:
In [1]: import sys
In [2]: print('\n'.join(sys.path))
Choose the one ending in site-packages and add /numpy
E.g., mine is /opt/anaconda3/lib/python3.7/site-packages/numpy
As I only wanted the installation for Python, I could go straight to Step 4 and follow those instructions. When I did this, the error told me that from within LIB/inc/COMPAT.h, it can’t find values.h. I edited LIB/inc/COMPAT.h, changing values.h to limits.h, and then Step 4 worked and I moved to Step 7.
In my installation, the .so file created was called ceflib.cpython-37m-darwin.so and not libcef.so as detailed in the instructions. I duplicated this file and called it libcef.so just in case…