snakemake.remote package¶
Submodules¶
snakemake.remote.EGA module¶
-
class
snakemake.remote.EGA.
EGAFile
(dataset, path)¶ Bases:
tuple
-
dataset
¶ Alias for field number 0
-
path
¶ Alias for field number 1
-
-
class
snakemake.remote.EGA.
EGAFileInfo
(size, status, id, checksum)¶ Bases:
tuple
-
checksum
¶ Alias for field number 3
-
id
¶ Alias for field number 2
-
size
¶ Alias for field number 0
-
status
¶ Alias for field number 1
-
-
class
snakemake.remote.EGA.
RemoteObject
(*args, protocol=None, keep_local=False, stay_on_remote=False, provider=None, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteObject
-
property
parts
¶
-
property
-
class
snakemake.remote.EGA.
RemoteProvider
(*args, keep_local=False, stay_on_remote=False, is_default=False, retry=5, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteProvider
-
api_request
(url_suffix, url_prefix='https://ega.ebi.ac.uk:8051/elixir/', json=True, post=False, **params)[source]¶ Make an API request.
- Parameters
url_suffix (str) – Part of REST API URL right of https://ega.ebi.ac.uk:8051/elixir/
params (dict) – Parameters to pass, except session
-
property
available_protocols
¶ List of valid protocols for this remote provider.
-
property
default_protocol
¶ The protocol that is prepended to the path when no protocol is specified.
-
property
token
¶
-
snakemake.remote.FTP module¶
snakemake.remote.GS module¶
snakemake.remote.HTTP module¶
-
class
snakemake.remote.HTTP.
RemoteObject
(*args, keep_local=False, provider=None, additional_request_string='', allow_redirects=True, **kwargs)[source]¶ Bases:
snakemake.remote.DomainObject
This is a class to interact with an HTTP server.
-
get_header_item
(httpr, header_name, default)[source]¶ Since HTTP header capitalization may differ, this returns a header value regardless of case
-
property
list
¶
-
-
class
snakemake.remote.HTTP.
RemoteProvider
(*args, keep_local=False, stay_on_remote=False, is_default=False, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteProvider
-
property
available_protocols
¶ List of valid protocols for this remote provider.
-
property
default_protocol
¶ The protocol that is prepended to the path when no protocol is specified.
-
property
snakemake.remote.NCBI module¶
-
class
snakemake.remote.NCBI.
NCBIHelper
(*args, email=None, **kwargs)[source]¶ Bases:
object
-
fetch_from_ncbi
(accession_list, destination_dir, force_overwrite=False, rettype='fasta', retmode='text', file_ext=None, combined_file_prefix=None, remove_separate_files=False, chunk_size=1, db='nuccore', **kwargs)[source]¶ This function downloads and saves files from NCBI. Adapted in part from the BSD-licensed code here:
-
property
valid_dbs
¶
-
property
valid_extensions
¶
-
-
class
snakemake.remote.NCBI.
RemoteObject
(*args, keep_local=False, stay_on_remote=False, provider=None, email=None, db=None, rettype=None, retmode=None, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteObject
This is a class to interact with NCBI / GenBank.
-
property
accession
¶
-
property
file_ext
¶
-
property
list
¶
-
property
version
¶
-
property
-
class
snakemake.remote.NCBI.
RemoteProvider
(*args, keep_local=False, stay_on_remote=False, is_default=False, email=None, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteProvider
-
property
available_protocols
¶ List of valid protocols for this remote provider.
-
property
default_protocol
¶ The protocol that is prepended to the path when no protocol is specified.
-
property
snakemake.remote.S3 module¶
-
class
snakemake.remote.S3.
RemoteObject
(*args, keep_local=False, provider=None, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteObject
This is a class to interact with the AWS S3 object store.
-
property
list
¶
-
property
name
¶
-
property
s3_bucket
¶
-
property
s3_key
¶
-
property
-
class
snakemake.remote.S3.
RemoteProvider
(*args, keep_local=False, stay_on_remote=False, is_default=False, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteProvider
-
property
available_protocols
¶ List of valid protocols for this remote provider.
-
property
default_protocol
¶ The protocol that is prepended to the path when no protocol is specified.
-
supports_default
= True¶
-
property
-
class
snakemake.remote.S3.
S3Helper
(*args, **kwargs)[source]¶ Bases:
object
-
delete_from_bucket
(bucket_name, key)[source]¶ Delete a file from s3
This function deletes an object from a specified AWS S3 bucket.
- Parameters
bucket_name – the name of the S3 bucket to use (bucket name only, not ARN)
key – the key of the object to delete from the bucket
- Returns
The name of the object deleted
-
download_from_s3
(bucket_name, key, destination_path=None, expandKeyIntoDirs=True, make_dest_dirs=True, create_stub_only=False)[source]¶ Download a file from s3
This function downloads an object from a specified AWS S3 bucket.
- Parameters
bucket_name – the name of the S3 bucket to use (bucket name only, not ARN)
destination_path – If specified, the file will be saved to this path, otherwise cwd.
expandKeyIntoDirs – Since S3 keys can include slashes, if this is True (defult) then S3 keys with slashes are expanded into directories on the receiving end. If it is False, the key is passed to os.path.basename() to get the substring following the last slash.
make_dest_dirs – If this is True (default) and the destination path includes directories that do not exist, they will be created.
- Returns
The destination path of the downloaded file on the receiving end, or None if the destination_path could not be downloaded
-
exists_in_bucket
(bucket_name, key)[source]¶ Returns whether the key exists in the bucket
- Parameters
bucket_name – the name of the S3 bucket to use (bucket name only, not ARN)
key – the key of the object to delete from the bucket
- Returns
True | False
-
key_last_modified
(bucket_name, key)[source]¶ Returns a timestamp of a key based on a HEAD request
- Parameters
bucket_name – the name of the S3 bucket to use (bucket name only, not ARN)
key – the key of the object to delete from the bucket
- Returns
timestamp
-
key_size
(bucket_name, key)[source]¶ Returns the size of a key based on a HEAD request
- Parameters
bucket_name – the name of the S3 bucket to use (bucket name only, not ARN)
key – the key of the object to delete from the bucket
- Returns
Size in kb
-
upload_to_s3
(bucket_name, file_path, key=None, use_relative_path_for_key=True, relative_start_dir=None, extra_args=None, config=None)[source]¶ Upload a file to S3
This function uploads a file to an AWS S3 bucket.
- Parameters
bucket_name – the name of the S3 bucket to use (bucket name only, not ARN)
file_path – The path to the file to upload.
key – The key to set for the file on S3. If not specified, this will default to the name of the file.
use_relative_path_for_key – If set to True (default), and key is None, the S3 key will include slashes representing the path of the file relative to the CWD. If False only the file basename will be used for the key.
relative_start_dir – The start dir to use for use_relative_path_for_key. No effect if key is set.
Returns: The key of the file on S3 if written, None otherwise
-
snakemake.remote.S3Mocked module¶
snakemake.remote.SFTP module¶
snakemake.remote.XRootD module¶
snakemake.remote.dropbox module¶
-
class
snakemake.remote.dropbox.
RemoteObject
(*args, keep_local=False, provider=None, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteObject
This is a class to interact with the Dropbox API.
-
property
list
¶
-
property
name
¶
-
property
-
class
snakemake.remote.dropbox.
RemoteProvider
(*args, keep_local=False, stay_on_remote=False, is_default=False, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteProvider
-
property
available_protocols
¶ List of valid protocols for this remote provider.
-
property
default_protocol
¶ The protocol that is prepended to the path when no protocol is specified.
-
property
snakemake.remote.gfal module¶
snakemake.remote.gridftp module¶
snakemake.remote.iRODS module¶
snakemake.remote.webdav module¶
Module contents¶
-
class
snakemake.remote.
AbstractRemoteObject
(*args, protocol=None, keep_local=False, stay_on_remote=False, provider=None, **kwargs)[source]¶ Bases:
object
This is an abstract class to be used to derive remote object classes for different cloud storage providers. For example, there could be classes for interacting with Amazon AWS S3 and Google Cloud Storage, both derived from this common base class.
-
inventory
(cache: snakemake.io.IOCache)[source]¶ From this file, try to find as much existence and modification date information as possible.
-
-
class
snakemake.remote.
AbstractRemoteProvider
(*args, keep_local=False, stay_on_remote=False, is_default=False, **kwargs)[source]¶ Bases:
object
This is an abstract class to be used to derive remote provider classes. These might be used to hold common credentials, and are then passed to RemoteObjects.
-
allows_directories
= False¶
-
abstract
default_protocol
()[source]¶ The protocol that is prepended to the path when no protocol is specified.
-
supports_default
= False¶
-
-
class
snakemake.remote.
DomainObject
(*args, **kwargs)[source]¶ Bases:
snakemake.remote.AbstractRemoteObject
This is a mixin related to parsing components out of a location path specified as (host|IP):port/remote/location
-
property
host
¶
-
property
local_path
¶
-
property
name
¶
-
property
path_prefix
¶
-
property
path_remainder
¶
-
property
port
¶
-
property
remote_path
¶
-
property