disco.schemes – Default input streams for URL schemes

By default, Disco looks at an input URL and extracts its scheme in order to figure out which input stream to use.

When Disco determines the URL scheme, it tries to import the name input_stream from disco.schemes.scheme_[SCHEME], where [SCHEME] is replaced by the scheme identified. For instance, an input URL of http://discoproject.org would import and use disco.schemes.scheme_http.input_stream().

disco.schemes.scheme_disco.input_stream(fd, size, url, params)

Opens the path on host locally if it is local, otherwise over http.

disco.schemes.scheme_file.input_stream(fd, size, url, params)

Opens the url locally on the node.

disco.schemes.scheme_http.input_stream(fd, sze, url, params)

Opens the specified url using an http client.

disco.schemes.scheme_raw.input_stream(fd, sze, url, params)

Opens a StringIO whose data is everything after the url scheme.

For example, raw://hello_world would return hello_world when read by the task.