disco.util – Helper functions

This module provides utility functions that are mostly used by Disco internally.

Deprecated since version 0.4: disco.util.data_err(), disco.util.err(), and disco.util.msg() will be removed completely in the next release, in favor of using normal Python raise and print statements.

disco.util.data_err(message, url)

Deprecated since version 0.4: raise disco.error.DataError instead.

Raises a disco.error.DataError. A data error should only be raised if it is likely that the error is transient. Typically this function is used by map readers to signal a temporary failure in accessing an input file.


Deprecated since version 0.4: raise disco.error.DiscoError instead.

Raises a disco.error.DiscoError. This terminates the job.


Extracts the job name from url.

This function is particularly useful for using the methods in disco.core.Disco given only the results of a job. A typical case is that you no longer need the results. You can tell Disco to delete the unneeded data as follows:

from disco.core import Disco
from disco.util import jobname


Group the values of consecutive keys which compare equal.

Takes an iterator over k, v pairs, and returns an iterator over k, vs. Does not sort the input first.


Deprecated since version 0.4: use print instead.

Sends the string message to the master for logging. The message is shown on the web interface. To prevent a rogue job from overwhelming the master, the maximum message size is set to 255 characters and job is allowed to send at most 10 messages per second.

disco.util.parse_dir(dir, partition=None)

Translates a directory URL (dir://...) to a list of normal URLs.

This function might be useful for other programs that need to parse results returned by disco.core.Disco.wait(), for instance.

Parameters:dir – a directory url, such as dir://nx02/test_simple@12243344
Read the Docs v: 0.4.5
On Read the Docs
Project Home

Free document hosting provided by Read the Docs.