The disco.core module provides a high-level interface for communication with the Disco master. It provides functions for submitting new jobs, querying the status of the system, and getting results of jobs.
The Disco object provides an interface to the Disco master. It can be used to query the status of the system, or to submit a new job.
Parameters: | master (url) – address of the Disco master, e.g. disco://localhost. |
---|
See also
Disco.new_job() and disco.job for more on creating jobs.
Blacklists node so that tasks are no longer run on it.
New in version 0.2.4.
Deletes job metadata.
Deprecated since version 0.4: Use Disco.purge() to delete job results, deleting job metadata only is strongly discouraged.
Note
After the job has been cleaned, there is no way to obtain the result urls from the master. However, no data is actually deleted by Disco.clean(), in contrast to Disco.purge().
Returns an iterator that iterates over job events, ordered by time. It is safe to call this function while the job is running, thus it provides an efficient way to monitor job events continuously. The iterator yields tuples offset, event.
Parameters: | offset (int) – skip events that occurred before this offset |
---|
New in version 0.2.3.
See also
DISCO_EVENTS for information on how to enable the console output of job events.
Returns a dict containing information about the job.
Returns a list of jobs and their statuses.
Return the disco.job.JobPack submitted for the job.
Kills the job.
Submits a new job request to the master using disco.job.Job.
Returns a dictionary describing status of the nodes that are managed by this Disco master.
Returns an out-of-band value assigned to key for the job.
OOB data can be stored and retrieved for job tasks using disco.task.Task.get() and disco.task.Task.put().
Returns all out-of-band keys for the job.
See also
Returns results of job profiling. The Job Dict must have had the profile flag enabled.
Parameters: |
|
---|
The function returns a pstats.Stats object. For instance, you can print out results as follows:
job.profile_stats().sort_stats('cumulative').print_stats()
New in version 0.2.1.
Deletes all metadata and data related to the job.
Returns a list of results for a single job or for many concurrently running jobs, depending on the type of jobspec.
Parameters: |
|
---|
Using a list of jobs is a more efficient way to wait for multiple jobs to finish. Consider the following example that prints out results as soon as the jobs (initially active) finish:
while active:
inactive, active = disco.results(jobs)
for jobname, (status, results) in inactive:
if status == 'ready':
for k, v in result_iterator(results):
print(k, v)
disco.purge(jobname)
Note how the list of active jobs, active, returned by Disco.results(), can be used as the input to this function as well.
Block until the job has finished. Returns a list of the result urls.
Parameters: |
|
---|
Whitelists node so that the master may submit tasks to it.
New in version 0.2.4.
An iterator over records as seen by the classic map interface.
Parameters: |
|
---|
Backwards compatible alias for classic_iterator()