Python SDK Installation & User Guide

Using the Furiosa Python SDK library, you can easily write Python application programs that facilitate the NPU. Python SDK makes application programmers to exploit various toolkits, libraries, and frameworks widely used in Python ecosystem. Pythoon SDK consists of several modules and provides inference API, quantization API, command line interface tools, and inference serving.

Requirements for installation

In order to use Python SDK, kernel driver, firmware, and runtime library must be installed (refer to FuriosaAI SDK Required Packages for installation)

Setting up Python Environment

Python SDK requires Python 3.7 or higher version. This section introduces how to set up Python environment.

Note

Please skip this chapter either FuriosaAI Python SDK is not required or you are used to Python environment.

You may check the version of Python on system by the following command:

python --version
Python 3.8.10

If Python command is not installed yet or Python version is lower than 3.8, Python environment can be setup by choosing one of options below:

  • Setting up Python Environment with Conda (recommended): Conda provides an isolated Python environment for certain applications. Conda resolves some problems such as package dependency conflicts and version conflicts when installing Python applications.

  • Setting up Python Environment on Linux - setting up Python environment directly on Linux: Recommended only when there are no conflicts with other Python applications.

Setting up Python Environment with Conda

Conda provides an isolated Python environment for certain applications. Please refer to Conda for details.

Installation requires download of Conda program. sh ./Miniconda3-latest-Linux-x86_64.sh choose yes for all questions.

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh ./Miniconda3-latest-Linux-x86_64.sh
source ~/.bashrc
conda --version

Creating and Activating Isolated Python Environment

An isolated Python environment can be created and activated after installing Anaconda.

  1. Create an isolated Python environment named furiosa-3.8 which uses Python version 3.8.

conda create -n furiosa-3.8 python=3.8

  1. Activate the created Python 3.8 environment with command activate.

conda activate furiosa-3.8
# check the version of Python
python --version

  1. Install Python SDK with instructions on Installation of Python SDK Package.

  2. Deactivate the Python environment with command deactivate.

$ conda deactivate

The created Python environment can be activated again with command activate. The installed Python SDK remains on the Python environment hence the Python SDK does not need to be installed again.

Setting up Python Environment on Linux

  1. Install the required packages for Python environment.

sudo apt install -y python3 python3-pip python-is-python3

  1. Check the version of Python.

python --version
Python 3.8.10

Installation of Python SDK Package

FuriosaAI Python SDK package is uploaded on PyPi storage pypi, hence it can be easily installed with command pip.

pip install furiosa-sdk

Package contains compiler command line tools and inference API. Please refer to furiosa compile and Tutorial and Code Examples for details.

Extra Python packages contains other extra functions, please refer to Extra Packages. Furiosa litmus can check whether user provided models are compatible with Furiosa NPU.

pip install 'furiosa-sdk[litmus]'

Extra Packages

FuriosaAI Models

FuriosaAI Models contain pre-trained DNN models which are optimized on Furiosa NPU.

pip install 'furiosa-sdk[models]'

Model Server

Model Server provides GRPC and Restful API of model inference accelerated on Furiosa NPU. Please refer to Model Server (Serving Framework) for details.

pip install 'furiosa-sdk[server]'

Litmus

Litmus checks that the user-provided model is compatible with FuriosaAI SDK. The user-provided models are quickly quantized and compiled to binary for checking compatibility.

pip install 'furiosa-sdk[litmus]'

Quantizer

Quantizer package provides quantization API. Please refer to Model Quantization for details.

pip install 'furiosa-sdk[quantizer]'