Updating scripts for HTTPS

For improved security, the CSA now uses HTTPS instead of HTTP. We are also removing the username and password parameters from the https requests used to access the CSA through the CAIO.

The CAIO website as a whole, but especially the CAIO How-To page, has been updated to reflect these changes, but if you already have established code, these instructions describe the specific simple steps you will need to adapt your code to the new protocol.

For MATLAB and IDL scripts, the only change needed is Step 1.

STEP 1

All instances of 'http' need to be replaced by 'https'.

For example, in most text editors, you could use 'find and replace' to find all instances of 'http' and replace with 'https'.

STEP 2

The parameters 'USERNAME' and 'PASSWORD' need to be taken out.

If you have find and replace, find 'USERNAME=<username>&PASSWORD=<password>' (using your own username in place of <username> and your password in place of <password>), and replace with '' (nothing).

STEP 3

Add the parameter 'CSACOOKIE' to the https request. You can create your own CSACOOKIE by using your username and password to log in on the Login page. The code printed in bold in the centre of the 'successful login' page is your cookie - specific to you.

This cookie could be quite long and so it is probably preferable to add it to the end of the https request, instead of at the beginning where you might have put your username and password before.

Once you have the cookie value from the login page, you could use find and replace again to find the end of the request, likely to be 'NON_BROWSER', and replace it with 'NON_BROWSER&CSACOOKIE=<cookie>', where you use your own cookie value in place of <cookie>. For example:

https://csa.esac.esa.int/csa/aio/streaming-action?DATASET_ID=C4_CP_PEA_LARL_PSD&START_DATE=2004-09-18T10:00:00Z&END_DATE=2004-10-20T22:00:00Z&GZIP=1&NON_BROWSER&CSACOOKIE=<cookie>

STEP 3 WGET ALTERNATIVE

If you use wget, there is an alternative to Step 3. Instead of storing your cookie value in each https request (i.e. in each wget call) you can store your cookie in a local file and give the name of that file in each request instead. 

To get the cookie file, use the following with your own username (<username>), password (<pwd>) and chosen filename for your cookie file (<cookie_file>).

$ wget --keep-session-cookies --save-cookies <cookie_file>.txt --content-disposition 'https://csa.esac.esa.int/csa/aio/login-action?USERNAME=<username>&PASSWORD=<pwd>&SUBMIT=LOGIN&NON_BROWSER'

You will receive two files, your cookie file and a login success file.

Then, in each wget command, use --load-cookies with the name of your cookie file in place of <cookie_file> like in the example below:

$ wget --load-cookies <cookie_file>.txt --content-disposition "https://csa.esac.esa.int/csa/aio/streaming-action?DATASET_ID=C4_CP_PEA_LERL_PSD&START_DATE=2004-09-18T10:00:00Z&END_DATE=2004-10-20T22:00:00Z&GZIP=1&NON_BROWSER"

STEP 3 WGET ALTERNATIVE (without --content-disposition)

In case --content-disposition does not work for you, replace it with -O LoginRequest.txt, otherwise your login success file, which is downloaded with the cookie file, will have the same filename as your request, including your username and password.

For example, with the name of your cookie file in place of <cookie_file>:

$ wget --keep-session-cookies --save-cookies <cookie_file>.txt -O LoginRequest.txt 'https://csa.esac.esa.int/csa/aio/login-action?USERNAME=<username>&PASSWORD=<pwd>&SUBMIT=LOGIN&NON_BROWSER'