hsclient HydroShare Python Client Resource File Operation Examples

The following code snippets show examples for how to use the hsclient HydroShare Python Client to manipulate files within a HydroShare Resource.

Install the hsclient Python Client

The hsclient Python Client for HydroShare may not be installed by default in your Python environment, so it has to be installed first before you can work with it. Use the following command to install hsclient via the Python Package Index (PyPi).

!pip install hsclient

Authenticating with HydroShare

Before you start interacting with resources in HydroShare you will need to authenticate.

from hsclient import HydroShare

hs = HydroShare()
hs.sign_in()

Create a New Empty Resource

A "resource" is a container for your content in HydroShare. Think of it as a "working directory" into which you are going to organize the code and/or data you are using and want to share. The following code can be used to create a new, empty resource within which you can create content and metadata.

This code creates a new resource in HydroShare. It also creates an in-memory object representation of that resource in your local environmment that you can then manipulate with further code.

# Create the new, empty resource
new_resource = hs.create()

# Get the HydroShare identifier for the new resource
res_identifier = new_resource.resource_id
print(f'The HydroShare Identifier for your new resource is: {res_identifier}')

# Construct a hyperlink for the new resource
print(f'Your new resource is available at: {new_resource.metadata.url}')

Resource File Handling

HydroShare resources can have any number of files within them organized within a file/directory structure. File handing operations allow you to manage the content files within a resource.

First, show the list of files within the resource, which is initially empty. The search_aggregations argument tells the client whether you want to look at all of the files in the resource (search_aggregations=True) or if you want to want to only look at files that do not belong to a content aggregation (search_aggregations=False).

# Print the title of the resource and the list of files it contains
print(f'Working on: {new_resource.metadata.title}')
print('File list:')
for file in new_resource.files(search_aggregations=True):
    print(file.name)

Adding Files to a Resource

You may need to add content files to your resource. The examples here upload files from the Example_Files folder that is included in the same folder that contains these Jupyter Notebook examples. If you are running in your own local Python environment and want to load files from your local machine, you would specify the path to the file(s) on your hard drive. If you want to upload multiple files at once, you can pass multiple file paths separated by commas to the upload() function.

Note that if you upload files that already exist, those files will be overwritten.

# Upload one or more files to your resource
new_resource.file_upload('Example_Files/Data_File1.txt', 'Example_Files/Data_File2.txt')

# Print the names of the files in the resource
print('Updated file list after adding a file: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

HydroShare also allows you to create a folder hierarchy within your resource. You can use this functionality to keep your content organized, just as you would on your own computer. You can upload files to specific folders within the resource. Paths to folders are specified relative to the "content" directory of the resource.

# First create a new folder
new_resource.folder_create('New_Folder')

# Upload one or more files to a specific folder within a resource
new_resource.file_upload('Example_Files/Data_File2.txt', destination_path='New_Folder')

# Print the names of the files in the resource
print('Updated file list after adding a file: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

Searching for Files within a Resource

If you need to find/get one or more files within a resource so you can download or remove it from the resource, there are several filters available that allow you to return a list of files that meet your search criteria or a single file.

Get a List of Files

Execute a filter to return a list of files within the resource that meet the search critera.

# Get a list of the files in the resource that are not part of an aggregation
file_list = new_resource.files()
print('All files that are not part of an aggregation:')
print(*file_list, sep='\n')
print('\n')

# Get a list of the files in the resource inclusive of files that are inside
# content type aggregations
file_list = new_resource.files(search_aggregations=True)
print('All files in the resource:')
print(*file_list, sep='\n')
print('\n')

# Get a list of the files within a folder in the resource
# Note that you have to pass the full relative path to the folder you are searching
# because there may be multiple folders within a resource with the same name.
# To get files in the root folder, pass an empty string (folder="")
file_list = new_resource.files(folder='New_Folder')
print('All files within a specific folder:')
print(*file_list, sep='\n')
print('\n')

# Get a list of all files that have a specific extension. This searches all folders
file_list = new_resource.files(extension='.txt')
print('All files with a .txt file extension:')
print(*file_list, sep='\n')
print('\n')

# Filters can be combined
# Get a list of all files in a particular folder that have a specific extension
file_list = new_resource.files(folder='New_Folder', extension='.txt')
print('All files with a .txt file extension in a particular folder:')
print(*file_list, sep='\n')

Search for a Single File

Execute a filter to look for a single file in the resource that meets the search criteria.

# Get a single file using its path relative to the resource content directory
file = new_resource.file(path='New_Folder/Data_File2.txt')
print('File retrieved using path:')
print(file)
print('\n')

# Get a single file using its name
# Note that if you have multiple files in your resource with the same name, but in different
# folders, you should search for a particular file using the path parameter to ensure that
# you get the right file
file = new_resource.file(name='Data_File2.txt')
print('File retrieved using name:')
print(file)

Get the Properties of a File

When you use the filters to return a file from a resource, you get back a file object that holds properties of the file.

# Search for a file within a resource
file = new_resource.file(path='New_Folder/Data_File2.txt')

# Print the properties of the file
print(f'File name: {file.name}')
print(f'File extension: {file.extension}')
print(f'File folder name: {file.folder}')
print(f'File path: {file.path}')
print(f'File url_path: {file.url}')
print(f'File checksum: {file.checksum}')

Renaming and Moving Files

You may need to rename or move files once they have been added to a resource. First get the file object and then rename or move it.

# Get a file to rename - use the relative path to the file to make sure you have the right one
file = new_resource.file(path='Data_File2.txt')

# Rename the file to whatever you want
new_resource.file_rename(file, 'Data_File2_Renamed.txt')

# Print the names of the files in the resource
print('Updated file list after renaming a file: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

Moving files is similar to renaming. Instead of just changing the file name, change the relative path of the file to move it to the new location within the resource.

# Get a file to move
file = new_resource.file(path='Data_File1.txt')

# Move the file to a different folder
new_resource.file_rename(file, 'New_Folder/Data_File1.txt')

# Print the names of the files in the resource
print('Updated file list after renaming a file: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

Downloading Files from a Resource

You can download individual files from an existing HydroShare resource. You can use the filters shown above to specify which file(s) you want to download.

When you call the file_download() function on an individual file, you can pass a path where you want to save the file as a string. Leaving the path blank downloads the files to the same directory as your Jupyter Notebook.

# Download a single file from a resource
# Note that if you have multiple files within the same resource that have the same name,
# and you want a particular file, you need to specify the relative path to the specific file
file = new_resource.file(path='New_Folder/Data_File1.txt')
new_resource.file_download(file)

If you want to, you can clean up the file that was just downloaded by deleting it using a terminal command.

!rm 'Data_File1.txt'

Unzipping a File in a Resource

You can unzip a file in a HydroShare resource.

# Upload a zipped file to the resource
new_resource.file_upload('Example_Files/test.zip')

# Specify the file you want to unzip
new_resource.file_unzip(path='test.zip')

# Print the names of the files in the resource
print('Updated file list after unzipping a file: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

Removing Files from a Resource

You can also delete files from a resource. In this example, I remove one of the files I added to the resource above. You have to delete each individual file. Make sure you call delete using the path parameter to make sure you are deleting the right file.

# Specify the file you want to delete
file = new_resource.file(path='New_Folder/Data_File2.txt')

new_resource.file_delete(file)

# Print the names of the files in the resource
print('Updated file list after removing file: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

Deleting Folders from a Resource

You can delete folders from a HydroShare resource. You can delete a folder and all of the files within it.

# Specify the folder you want to delete
new_resource.folder_delete('New_Folder')

# Print the names of the files in the resource
print('Updated file list after deleting folder: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

Moving a Folder in a Resource

You can move a folder in a HydroShare resource. You can move a folder and all of the files within it to a different folder in that resource. Moving folders is similar to renaming. Instead of just changing the folder name, change the relative path of the folder to move it to the new location within the resource.

# First create a new folder
new_resource.folder_create('New_Folder_1')
# Upload one or more files to a specific folder within a resource
new_resource.file_upload('Example_Files/Data_File1.txt', destination_path='New_Folder_1')

# Print the names of the files in the resource
print('Updated file list after uploading file to folder: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

# Create a 2nd folder
new_resource.folder_create('New_Folder_2')
# Upload one or more files to a specific folder within a resource
new_resource.file_upload('Example_Files/Data_File2.txt', destination_path='New_Folder_2')

# Print the names of the files in the resource
print('Updated file list after uploading file to folder: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

# Now move the folder 'New_Folder_2' to 'New_Folder_1' - this will make 'New_Folder_2' a subfolder of 'New_Folder_1'
new_resource.folder_rename('New_Folder_2', 'New_Folder_1/New_Folder_2')

# Print the names of the files in the resource
print('Updated file list after moving folder: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

Renaming a Folder in a Resource

You can rename a folder in a HydroShare resource.

# Specify the folder you want to rename
new_resource.folder_rename('New_Folder_1', 'New_Folder_1_Renamed')

# Print the names of the files in the resource
print('Updated file list after renaming folder: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

Zipping a Folder in a Resource

You can zip a folder in a HydroShare resource. By default, the name of the generated zip file will be based on the name of the folder being zipped. You can optionally provide a name for the generated zip file using the 'zip_name' parameter to the file_zip() function. The zip file will be created at the same location where the folder being zipped exists.

# Specify the folder you want to zip - optionally provide a name for the zip file using the parameter 'zip_name'
new_resource.file_zip('New_Folder_1_Renamed/New_Folder_2')

# Print the names of the files in the resource
print('Updated file list after zipping folder: ')
for file in new_resource.files(search_aggregations=True):
    print(file.path)

Downloading a Folder in a Resource

You can download a folder in a HydroShare resource as a zipped file. You can pass a path using the 'save_path' parameter where you want to save the folder zipped file as a string to the file_download() function. Leaving the path blank downloads the folder to the same directory as your Jupyter Notebook.

# Specify the folder you want to download - optionally use the parameter 'save_path` to specify where you want to save the zipped file
new_resource.file_download('New_Folder_1_Renamed', zipped=True)