furiosa.common.transport package

Submodules

furiosa.common.transport.base module

class furiosa.common.transport.base.Transport

Bases: ABC

Base transport class to read/download from various sources.

property cache_directory: str: Cache directory to save downloaded files.

abstract async download(uri: str) → str

Download a directory into local destination.

Parameters:: uri (str) – URI to download the data.
Returns:: Destination directory name. This directory will be located in cache directory.
Return type:: str

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

read: Dispatch methods based on type signature

See also

Dispatcher

furiosa.common.transport.error module

exception furiosa.common.transport.error.TransportError(msg): Bases: Exception

exception furiosa.common.transport.error.TransportNotFound(uri: str, transports: List[Transport]): Bases: TransportError

furiosa.common.transport.file module

class furiosa.common.transport.file.FileTransport

Bases: Transport

Transport for local file path.

This transport check specified URI has valid local file scheme (e.g. file://).

async download(uri: str) → str

Download a directory into local destination.

FileTransport download is just to copy local directory into the cache.

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

read: Dispatch methods based on type signature

See also

Dispatcher

scheme = 'file://'

furiosa.common.transport.github module

class furiosa.common.transport.github.GithubTransport(headers: Dict = {}, params: Dict = {})

Bases: HTTPTransport

Transport for Github repository.

This transport check specified URI has valid Github repository URL. (e.g. https://github.com/)

async download(uri: str)

Download a directory into local destination.

Parameters:: uri (str) – URI to download the data.
Returns:: Destination directory name. This directory will be located in cache directory.
Return type:: str

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

parse(uri: str) → Tuple[str, str, str]

Parse URI following Github repository scheme.

e.g. https://github.com/furiosa-ai/furiosa-models:main to (furiosa-ai, furiosa-models, main)

read: Dispatch methods based on type signature

See also

Dispatcher

schemes = ('https://github.com/', 'http://github.com/')

furiosa.common.transport.http module

class furiosa.common.transport.http.HTTPTransport(headers: Dict = {}, params: Dict = {})

Bases: Transport

Transport for HTTP.

This transport check specified URI has valid http scheme (e.g. https://, http://)

async download(uri: str) → str

Download a directory into local destination.

Parameters:: uri (str) – URI to download the data.
Returns:: Destination directory name. This directory will be located in cache directory.
Return type:: str

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

read: Dispatch methods based on type signature

See also

Dispatcher

schemes = ('https://', 'http://')

furiosa.common.transport.s3 module

class furiosa.common.transport.s3.S3Transport

Bases: Transport

Transport for S3.

This transport check specified URI has valid S3 scheme (e.g. s3://).

async download(uri: str) → str

Download a directory into local destination.

Parameters:: uri (str) – URI to download the data.
Returns:: Destination directory name. This directory will be located in cache directory.
Return type:: str

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

static parse(uri: str) → Tuple[str, str]: Parse URI to network location and path.

read: Dispatch methods based on type signature

See also

Dispatcher

scheme = 's3://'

Module contents

Transport utilities to fetch files from various sources.

class furiosa.common.transport.FileTransport

Bases: Transport

Transport for local file path.

This transport check specified URI has valid local file scheme (e.g. file://).

async download(uri: str) → str

Download a directory into local destination.

FileTransport download is just to copy local directory into the cache.

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

read: Dispatch methods based on type signature

See also

Dispatcher

scheme = 'file://'

class furiosa.common.transport.GithubTransport(headers: Dict = {}, params: Dict = {})

Bases: HTTPTransport

Transport for Github repository.

This transport check specified URI has valid Github repository URL. (e.g. https://github.com/)

async download(uri: str)

Download a directory into local destination.

Parameters:: uri (str) – URI to download the data.
Returns:: Destination directory name. This directory will be located in cache directory.
Return type:: str

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

parse(uri: str) → Tuple[str, str, str]

Parse URI following Github repository scheme.

e.g. https://github.com/furiosa-ai/furiosa-models:main to (furiosa-ai, furiosa-models, main)

read: Dispatch methods based on type signature

See also

Dispatcher

schemes = ('https://github.com/', 'http://github.com/')

class furiosa.common.transport.HTTPTransport(headers: Dict = {}, params: Dict = {})

Bases: Transport

Transport for HTTP.

This transport check specified URI has valid http scheme (e.g. https://, http://)

async download(uri: str) → str

Download a directory into local destination.

Parameters:: uri (str) – URI to download the data.
Returns:: Destination directory name. This directory will be located in cache directory.
Return type:: str

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

read: Dispatch methods based on type signature

See also

Dispatcher

schemes = ('https://', 'http://')

class furiosa.common.transport.S3Transport

Bases: Transport

Transport for S3.

This transport check specified URI has valid S3 scheme (e.g. s3://).

async download(uri: str) → str

Download a directory into local destination.

Parameters:: uri (str) – URI to download the data.
Returns:: Destination directory name. This directory will be located in cache directory.
Return type:: str

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

static parse(uri: str) → Tuple[str, str]: Parse URI to network location and path.

read: Dispatch methods based on type signature

See also

Dispatcher

scheme = 's3://'

class furiosa.common.transport.Transport

Bases: ABC

Base transport class to read/download from various sources.

property cache_directory: str: Cache directory to save downloaded files.

abstract async download(uri: str) → str

Download a directory into local destination.

Parameters:: uri (str) – URI to download the data.
Returns:: Destination directory name. This directory will be located in cache directory.
Return type:: str

static is_supported(uri: str) → bool

Decide whether this transport supports the specified URI.

Parameters:: uri (str) – URI to locate the models.
Returns:: This transport supports the URI or not.
Return type:: bool

read: Dispatch methods based on type signature

See also

Dispatcher

async furiosa.common.transport.download(uri: str) → str

Download a directory into local destination with a transport which supports the URI.

Parameters:: uri (str) – URI to download the data.
Returns:: Destination directory name. This directory will be located in cache directory.
Return type:: str
Raises:: TransportNotFound – If all the available transports are not supporting the URI.

furiosa.common.transport.is_relative(path: str) → bool

Is this path relative path?

If all the available transports are not supporting the path, we assume that it’s a relative path. You should find the path from the URI if it’s a relative path.

furiosa.common.transport.supported(uri: str) → Iterator[Transport]: Supported transport for the URI.