RT Guide: Chapter 2, Initial Setup and Validation

Owned by Jeff Ussing
Last updated: 02 Aug 2023 by Pallavi Mehta
6 min read

Initial Setup and Validation

This chapter provides guidance on the initial setup and validation of the user into the SDC system.

Accessing Secure Data Commons Portal

Users can access the SDC web portal by navigating to https://portal.sdc.dot.gov

Select Register/Login from the top menu to display the Access Request Form and Privacy Policy links, as well as the email address sign-in prompt.

The Access Request Form link takes you to a (downloadable) .docx form in which you can provide the support team with contact info, a rationale for your access request, your preferred workstation type, and signoff for the SDC data use agreements.

Users with a DOT email address (@dot.gov)

If you have a DOT email address (@dot.gov), enter it into Email Address and then select Sign In
to access the portal. If you do not have a DOT email address, continue to the next section-https://securedatacommons.atlassian.net/wiki/spaces/DESK/pages/2224128024/RT+Guide+Chapter+2+Initial+Setup+and+Validation#Users-without-a-DOT-email-address-(%40dot.gov)
Notes:

  • If you are logged into your government laptop with your PIV badge inserted and
    connected to VPN or from your Virtual Desktop Infrastructure (VDI), your credentials
    are already in your browser so you will be automatically logged in to the form shown in
    Figure 3 below and skip the ADFS Sign in prompt.

  • If you attempt to log in from your personal machine or not connected to VPN, you will be
    prompted for your full ADFS DOT credentials.

    • Username: First.Last@ad.dot.gov

On the following form, select your DOT account from the list.

You may be redirected to a one-time sign-in form on the portal if this is the first time syncing
your DOT account with your SDC account. Enter your SDC (example: username: bsmith)
credentials and then select Sign in.

If you are accessing the portal for the first time, you will be prompted to change your password after entering the credentials provided in the welcome email.

After signing in, you will be redirected to the Secure Data Commons landing page.

Users without a DOT email address (@dot.gov)

If you do not have a DOT email address, you will need to create a new Login.gov account or log
in with your existing one so that it can be linked with your SDC credentials. Linking a Login.gov
account with your SDC credentials provides extra security by adding protection you can
configure as two layers of authentication. Authentication methods will consist of phone text or
call; an app on your phone, tablet, or computer; a security key; a government employee ID; and
pre-generated backup codes.


Enter the email address you would like to use as your Login.gov account into the portal’s Email
Address sign in (Figure 2 above) and then select Sign In to be redirected to the Login.gov
website, where you can create a new account. If you already have an existing Login.gov account,
enter your credentials and then select Sign in.


If you do not have an existing Login.gov account, select “Create an account” on the page you are
redirected to:

For further instructions on setting up your new Login.gov account and configuring its secure
authentication methods, refer to https://www.login.gov/help/creating-an-account/how-to-create-an-account/


After you have finished setting up all authentication methods or signed in with an existing
Login.gov account, you will be redirected to a one-time sign-in form on the portal (refer to
previous Figure 4). Enter your SDC ADFS credentials (provided in your welcome email) and
then select Sign in.


If you are accessing the portal for the first time, you will be prompted to change your password
after entering the credentials provided in the welcome email.

SDC Portal Landing Page

Upon successfully logging in, you will be redirected to the landing page, which provides an
overview of Secure Data Commons and the different actions you can perform from the web
portal:

  1. Request access to curated and published datasets

  2. Access to workstations with programming tools

  3. Bring your own datasets / algorithms

  4. Publish your datasets / algorithms

Request Access to Datasets

Users can request access to the datasets that are available within the SDC platform as published /
enabled by the SDC team or published by other users.

Once you are logged in, go to Datasets in the top menu.

Expand the SDC Datasets. You will be able to see all available datasets in the SDC platform. To access a dataset, click on Request.

Complete the SDC Data Access Request form that appears. Once completed, click on Send Request.

The request will be sent to the support team and access to the requested dataset will be given upon validation and approval of the information in the form.

Upload User Data to S3 Bucket through Portal

Users who want to share data with other users from their project team can upload their own data
to their assigned team/individual buckets through the portal.

  1. Click on Datasets from the home page.

  2. Click on Upload Files under “My Datasets / Algorithm.”

  3. A pop-up window appears prompting you to choose one or more files for upload to the
    assigned bucket. (The assigned bucket name will be displayed on the upload pop-up
    window.)

  4. A success message will be displayed upon a successful upload.

  5. Files that are uploaded from the web portal will be saved in the folder on your SDC workstation in your Team bucket which you can access from CyberDuck or AWS command line username/uploaded_files

  6. To make these files available to members of the project team, users then need to move
    files from username/uploaded_files to the project team S3 bucket using command-line
    tools (see AWS S3 CLI Commands for an overview).

  7. Users would be able to access only the files that are under the uploaded_files and export_requests folders.

Download User Data from S3 Bucket through Portal

Users can download their data from their assigned team/individual buckets through the portal.

  1. Click on Datasets from the home page.

  2. All the available files under username/uploaded_files in the assigned bucket will be
    displayed along with the assigned bucket name under My Datasets / Algorithm.

  3. Select the files that you want to download and then click on Download Files.

  4. Users should go through the export request workflow to download files that are uploaded
    under export_requests folder. Export requests workflow can be found in Chapter 4.

Notes:

  • Not all the files are downloaded directly. Files with extensions such as .txt, .png, or .pdf
    will be opened in a separate tab from where they can be downloaded. All other files with
    extensions like .csv, .zip, etc. can be downloaded directly.

  • Files are downloaded individually.

  • The Filename box allows searches for partial filenames. This can be used to download all
    the contents of a sub-folder in an S3 bucket by searching for the sub-folder name and
    then clicking the box next to Filename to select all objects.

  • Files must be copied to and from S3 buckets using the SDC workstation.

    • NOTE: Files stored in S3 buckets are not visible through Windows Explorer but can be copied to your
      SDC workstation.

    • For example: To copy the file shown above in Figure 13 that is hosted in an S3
      bucket to your SDC workstation, use the command:

      aws s3 cp s3://prod-sdc-cveval-911061262852-us-east-1- bucket/duser/uploaded_files/dot_sdc_historical_2017_ra w_and_curated_counts.zip dot_sdc_historical_2017_raw_and_curated_counts.zip