How to use Data, delete files, push, pull

How to use your Toolkit data home

How to Pull Certain Files

See examples in eai data pull --help

eai data pull acme.account.data ./remote .

How to Delete Directories in a Data?

Although toolkit does not let you delete but deactivate data objects ( see Can I delete an account, a data or a job?), you can also delete its contents.

You can clear the contents of a data object using the cli, as shown below:

# List the contents
eai data content ls snow.some_account.some_data
    name                                           status
    .DS_Store                                      new
    AAA                                            new
    AAA/Accelerometer Data 2019-06-28 14-00-01.txt new
    AAA/Accelerometer Data 2019-06-28 14-37-42.txt new
    BBB                                            new
    BBB/.DS_Store                                  new
    BBB/BBB-1                                      new
    BBB/BBB-1/.DS_Store                            new
    ...

# You can selectively delete a directory inside it
eai data content rm snow.some_account.some_data AAA

# You can see directory AAA is now deleted
eai data content ls snow.some_account.some_data
    name                     status
    .DS_Store                new
    BBB                      new
    BBB/.DS_Store            new

# You can also clear out its contents entirely
eai data content rm snow.some_account.some_data .

# If you try to list the contents now, the data object will be empty
eai data content ls snow.some_account.some_data

You can also deactivate the data object, as shown below:

# Mount the data while submitting the job
eai job submit --data dataID:/data -- sleep 720

# Now you can execute commands to delete the contents of the data object from inside the job
# Here <expr_id> can be `*` or `some_folder` or `some_folder/inner_folder` or `some_folder/some_file` etc.
eai job exec jobID -- bash -c 'rm -rf /data/<exp_id>'

How to move a data to another account ?

Since the cli release (0.9.66), a new command is available: eai data chown ...

Before being able to move a data to another account, note that you need to have a few different policies:

Write access to the data: for the duration of the moving process, the data needs to be locked to avoid being modified by any users or jobs during the move
Write access to the source account: moving a data implies modifying an account
Write access to the destination account: moving a data implies modifying an account

See Permissions table to know the required actions.

Also see How to Share section.

Once all your permissions are set, run the following command:

$ eai data chown DATA_ID DESTINATION_ACCOUNT