This module provides utility functions that are mostly used by Disco internally.
Deprecated since version 0.4: disco.util.data_err(), disco.util.err(), and disco.util.msg() will be removed completely in the next release, in favor of using normal Python raise and print statements.
Deprecated since version 0.4: raise disco.error.DataError instead.
Raises a disco.error.DataError. A data error should only be raised if it is likely that the error is transient. Typically this function is used by map readers to signal a temporary failure in accessing an input file.
Deprecated since version 0.4: raise disco.error.DiscoError instead.
Raises a disco.error.DiscoError. This terminates the job.
Extracts the job name from url.
This function is particularly useful for using the methods in disco.core.Disco given only the results of a job. A typical case is that you no longer need the results. You can tell Disco to delete the unneeded data as follows:
from disco.core import Disco
from disco.util import jobname
Disco().purge(jobname(results[0]))
Group the values of consecutive keys which compare equal.
Takes an iterator over k, v pairs, and returns an iterator over k, vs. Does not sort the input first.
Deprecated since version 0.4: use print instead.
Sends the string message to the master for logging. The message is shown on the web interface. To prevent a rogue job from overwhelming the master, the maximum message size is set to 255 characters and job is allowed to send at most 10 messages per second.
Translates a directory URL (dir://...) to a list of normal URLs.
This function might be useful for other programs that need to parse results returned by disco.core.Disco.wait(), for instance.
| Parameters: | dir – a directory url, such as dir://nx02/test_simple@12243344 |
|---|