disco.ddfs – Client interface for Disco Distributed Filesystem¶
See also: Disco Distributed Filesystem.
Note
Parameters below which are indicated as tags can be specified as a tag:// URL, or the name of the tag.
- class disco.ddfs.DDFS(master=None, proxy=None, settings=None)¶
Opens and encapsulates a connection to a DDFS master.
Parameters: master – address of the master, for instance disco://localhost. - attrs(tag, token=None)¶
Get a list of the attributes of the tag tag and their values.
- blobs(tag, ignore_missing=True, token=None)¶
Walks the tag graph starting at tag.
Yields only the terminal nodes of the graph (blobs).
Parameters: ignore_missing (bool) – Whether or not missing tags will raise a disco.error.CommError.
- chunk(tag, urls, replicas=None, forceon=, []retries=10, delayed=False, update=False, token=None, chunk_size=67108864, max_record_size=1048576, **kwargs)¶
Chunks the contents of urls, pushes the chunks to ddfs and tags them with tag.
- delattr(tag, attr, token=None)¶
Delete the attribute attr of the tag tag.
- delete(tag, token=None)¶
Delete tag.
- exists(tag)¶
Returns whether or not tag exists.
- get(tag, token=None)¶
Return the tag object stored at tag.
- getattr(tag, attr, token=None)¶
Return the value of the attribute attr of the tag tag.
- list(prefix='')¶
Return a list of all tags starting with prefix.
- pull(tag, blobfilter=<function <lambda> at 0x7f92526f6b18>, token=None)¶
Iterate over the blobs in a tag after optionally applying a blobfilter over the blob names.
- push(tag, files, replicas=None, forceon=, []retries=10, delayed=False, update=False, token=None)¶
Pushes a bunch of files to ddfs and tags them with tag.
Parameters: files (a list of paths, (path, name)-tuples, or (fileobject, name)-tuples.) – the files to push as blobs to DDFS. If names are provided, they will be used as prefixes by DDFS for the blobnames. Names may only contain chars in r'[^A-Za-z0-9_\-@:]'.
- put(tag, urls, token=None)¶
Put the list of urls to the tag tag.
Warning
Generally speaking, concurrent applications should use DDFS.tag() instead.
- setattr(tag, attr, val, token=None)¶
Set the value of the attribute attr of the tag tag.
- tag(tag, urls, token=None, **kwargs)¶
Append the list of urls to the tag.
- urls(tag, token=None)¶
Return the urls in the tag.
- walk(tag, ignore_missing=True, tagpath=(), token=None)¶
Walks the tag graph starting at tag.
Yields a 3-tuple (tagpath, tags, blobs).
Parameters: ignore_missing (bool) – Whether or not missing tags will raise a disco.error.CommError.