furiosa.common.transport package

Submodules

furiosa.common.transport.base module

class furiosa.common.transport.base.Transport

Bases: ABC

Base transport class to read/download from various sources.

property cache_directory: str

Cache directory to save downloaded files.

abstract async download(uri: str) str

Download a directory into local destination.

Parameters:

uri (str) – URI to download the data.

Returns:

Destination directory name. This directory will be located in cache directory.

Return type:

str

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

read

Dispatch methods based on type signature

See also

Dispatcher

furiosa.common.transport.error module

exception furiosa.common.transport.error.TransportError(msg)

Bases: Exception

exception furiosa.common.transport.error.TransportNotFound(uri: str, transports: List[Transport])

Bases: TransportError

furiosa.common.transport.file module

class furiosa.common.transport.file.FileTransport

Bases: Transport

Transport for local file path.

This transport check specified URI has valid local file scheme (e.g. file://).

async download(uri: str) str

Download a directory into local destination.

FileTransport download is just to copy local directory into the cache.

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

read

Dispatch methods based on type signature

See also

Dispatcher

scheme = 'file://'

furiosa.common.transport.github module

class furiosa.common.transport.github.GithubTransport(headers: Dict = {}, params: Dict = {})

Bases: HTTPTransport

Transport for Github repository.

This transport check specified URI has valid Github repository URL. (e.g. https://github.com/)

async download(uri: str)

Download a directory into local destination.

Parameters:

uri (str) – URI to download the data.

Returns:

Destination directory name. This directory will be located in cache directory.

Return type:

str

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

parse(uri: str) Tuple[str, str, str]

Parse URI following Github repository scheme.

e.g. https://github.com/furiosa-ai/furiosa-models:main to (furiosa-ai, furiosa-models, main)

read

Dispatch methods based on type signature

See also

Dispatcher

schemes = ('https://github.com/', 'http://github.com/')

furiosa.common.transport.http module

class furiosa.common.transport.http.HTTPTransport(headers: Dict = {}, params: Dict = {})

Bases: Transport

Transport for HTTP.

This transport check specified URI has valid http scheme (e.g. https://, http://)

async download(uri: str) str

Download a directory into local destination.

Parameters:

uri (str) – URI to download the data.

Returns:

Destination directory name. This directory will be located in cache directory.

Return type:

str

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

read

Dispatch methods based on type signature

See also

Dispatcher

schemes = ('https://', 'http://')

furiosa.common.transport.s3 module

class furiosa.common.transport.s3.S3Transport

Bases: Transport

Transport for S3.

This transport check specified URI has valid S3 scheme (e.g. s3://).

async download(uri: str) str

Download a directory into local destination.

Parameters:

uri (str) – URI to download the data.

Returns:

Destination directory name. This directory will be located in cache directory.

Return type:

str

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

static parse(uri: str) Tuple[str, str]

Parse URI to network location and path.

read

Dispatch methods based on type signature

See also

Dispatcher

scheme = 's3://'

Module contents

Transport utilities to fetch files from various sources.

class furiosa.common.transport.FileTransport

Bases: Transport

Transport for local file path.

This transport check specified URI has valid local file scheme (e.g. file://).

async download(uri: str) str

Download a directory into local destination.

FileTransport download is just to copy local directory into the cache.

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

read

Dispatch methods based on type signature

See also

Dispatcher

scheme = 'file://'
class furiosa.common.transport.GithubTransport(headers: Dict = {}, params: Dict = {})

Bases: HTTPTransport

Transport for Github repository.

This transport check specified URI has valid Github repository URL. (e.g. https://github.com/)

async download(uri: str)

Download a directory into local destination.

Parameters:

uri (str) – URI to download the data.

Returns:

Destination directory name. This directory will be located in cache directory.

Return type:

str

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

parse(uri: str) Tuple[str, str, str]

Parse URI following Github repository scheme.

e.g. https://github.com/furiosa-ai/furiosa-models:main to (furiosa-ai, furiosa-models, main)

read

Dispatch methods based on type signature

See also

Dispatcher

schemes = ('https://github.com/', 'http://github.com/')
class furiosa.common.transport.HTTPTransport(headers: Dict = {}, params: Dict = {})

Bases: Transport

Transport for HTTP.

This transport check specified URI has valid http scheme (e.g. https://, http://)

async download(uri: str) str

Download a directory into local destination.

Parameters:

uri (str) – URI to download the data.

Returns:

Destination directory name. This directory will be located in cache directory.

Return type:

str

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

read

Dispatch methods based on type signature

See also

Dispatcher

schemes = ('https://', 'http://')
class furiosa.common.transport.S3Transport

Bases: Transport

Transport for S3.

This transport check specified URI has valid S3 scheme (e.g. s3://).

async download(uri: str) str

Download a directory into local destination.

Parameters:

uri (str) – URI to download the data.

Returns:

Destination directory name. This directory will be located in cache directory.

Return type:

str

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

static parse(uri: str) Tuple[str, str]

Parse URI to network location and path.

read

Dispatch methods based on type signature

See also

Dispatcher

scheme = 's3://'
class furiosa.common.transport.Transport

Bases: ABC

Base transport class to read/download from various sources.

property cache_directory: str

Cache directory to save downloaded files.

abstract async download(uri: str) str

Download a directory into local destination.

Parameters:

uri (str) – URI to download the data.

Returns:

Destination directory name. This directory will be located in cache directory.

Return type:

str

static is_supported(uri: str) bool

Decide whether this transport supports the specified URI.

Parameters:

uri (str) – URI to locate the models.

Returns:

This transport supports the URI or not.

Return type:

bool

read

Dispatch methods based on type signature

See also

Dispatcher

async furiosa.common.transport.download(uri: str) str

Download a directory into local destination with a transport which supports the URI.

Parameters:

uri (str) – URI to download the data.

Returns:

Destination directory name. This directory will be located in cache directory.

Return type:

str

Raises:

TransportNotFound – If all the available transports are not supporting the URI.

furiosa.common.transport.is_relative(path: str) bool

Is this path relative path?

If all the available transports are not supporting the path, we assume that it’s a relative path. You should find the path from the URI if it’s a relative path.

furiosa.common.transport.supported(uri: str) Iterator[Transport]

Supported transport for the URI.