Nux is the C API library of the NPU Engine. Nux provides the two kinds of execution ways to run inference tasks through NPU. The followings demonstrate the examples of two execution ways. If you want to see more details about API types and functions, please refer to nux.h.
Sync Model
The sync model provides APIs to run a single inference task with a single input batch at a time.
Example of Sync Model
const char *enf_path = "<a path of ENF file>";
struct stat st;
stat(enf_path, &st);
size_t enf_size = st.st_size;
void *enf_buf = malloc(enf_size);
FILE *enf_file = fopen(enf_path, "rb");
size_t read_item = fread(enf_buf, enf_size, 1, enf_file);
assert(read_item == 1);
fclose(enf_file);
uint8_t input_buf [784] = {[0 ... 783] = 1};
uint8_t *output_buf = NULL;
uintptr_t output_len = NULL;
enum nux_error_t nux_create_sync_model(nux_handle_t nux, const uint8_t *buffer, nux_buffer_len_t len, nux_sync_model_t *model)
Create an instance of a synchronous model.
void destroy_sync_model(nux_sync_model_t model)
Destroy the sync model and release its resources.
struct Model * nux_sync_model_t
Handle of a synchronous model.
Definition: nux.h:240
enum nux_error_t tensor_set_buffer(nux_tensor_t tensor, nux_buffer_t buffer, nux_buffer_len_t len, void(*free)(uint8_t *, uintptr_t))
Specify a pointer to a data buffer to a specified input tensor.
@ nux_error_t_success
Definition: nux.h:56
enum nux_error_t model_input_tensor(nux_sync_model_t model, uint32_t index, nux_tensor_t *tensor)
Get the handle of a specified input tensor from a given sync model.
struct Nux * nux_handle_t
Nux handle.
Definition: nux.h:228
struct OpaqueTensor * nux_tensor_t
Handle of tensor.
Definition: nux.h:247
enum nux_error_t tensor_get_buffer(nux_tensor_t tensor, nux_buffer_t *buffer, nux_buffer_len_t *len)
Get a pointer to the data buffer of a given tensor.
enum nux_error_t create_nux(nux_handle_t *nux)
Creates a new nux handle.
enum nux_error_t model_run(nux_sync_model_t model)
Run a single inference task.
void destroy_nux(nux_handle_t nux)
Destroy the Nux handle and release its resources.
enum nux_error_t model_output_tensor(nux_sync_model_t model, uint32_t index, nux_tensor_t *tensor)
Get the handle of a specified output tensor from a given sync model.
Task Model
The task model provides APIs to run multiple inference tasks asynchronously and simultaneously. The callback functions defined by users will be called when each task is completed. When a user creates a task model, the user can configure the concurrency of running tasks depending on HW capacity.
Example of Task Model
}
}
}
void run() {
const char *enf_path = "<a path of ENF file>";
struct stat st;
stat(enf_path, &st);
size_t enf_size = st.st_size;
void *enf_buf = malloc(enf_size);
FILE *enf_file = fopen(enf_path, "rb");
size_t read_item = fread(enf_buf, enf_size, 1, enf_file);
assert(read_item == 1);
fclose(enf_file);
const uint32_t max_concurrency = 3;
const uintptr_t INPUT_SIZE = 784;
uint8_t input_buf [784] = {[0 ... 783] = 1};
memcpy(buf, input_buf, INPUT_SIZE);
sleep(1);
}
free(enf_buf);
}
enum nux_error_t nux_create_task_model(nux_handle_t nux, const uint8_t *buffer, nux_buffer_len_t len, uint32_t max_batch, nux_output_callback_t output_callback, nux_error_callback_t error_callback, nux_finish_callback_t finish_callback, nux_task_model_t *task_model)
Create an instance of a task model.
enum nux_error_t task_model_get_task(nux_task_model_t task_model, nux_task_t *task)
Get a task handle from a specified task model.
OutputIndex nux_output_index_t
Index of an output tensor, used in task model.
Definition: nux.h:269
struct TaskModel * nux_task_model_t
Handle of a task model.
Definition: nux.h:298
bool task_model_is_all_task_done(nux_task_model_t task_model)
Return true if there's no running tasks, or false if any task is still running.
nux_buffer_len_t task_input_size(nux_task_t task, nux_input_index_t index)
Return the buffer length in bytes of the specified input tensor.
nux_error_t
Represent a return status of a function that can fail.
Definition: nux.h:52
void destroy_task_model(nux_task_model_t task_model)
Destroy the task model and release its resources.
uintptr_t RequestId
Definition: nux.h:257
RequestId nux_request_id_t
Identifier to distinguish tasks in a task model.
Definition: nux.h:262
nux_buffer_t task_input(nux_task_t task, nux_input_index_t index)
Return a mutable pointer to the buffer of the specified input tensor.
struct Task * nux_task_t
Handle of a task of a task model.
Definition: nux.h:305
uintptr_t nux_buffer_len_t
Byte length of a data buffer.
Definition: nux.h:233
uint8_t * nux_buffer_t
Pointer to a data buffer.
Definition: nux.h:255
enum nux_error_t task_execute(nux_task_t task, nux_request_id_t request_id)
Request one asynchronous inference task.
How to build source codes with Nux
To build your source code with Nux, please follow:
export NPU_TOOLS_DIR="<npu-tools src dir>"
export NUX_DIR=${NPU_TOOLS_DIR}/crates/nux
cargo build --release
cc sync_example.c -I${NUX_DIR}/include -L${NPU_TOOLS_DIR}/target/x86_64-unknown-linux-gnu/release -lnux
How to generate Doxygen
The following commands will generate a Doxygen document at $NUX_TOOLS_DIR/target/doxygen/html
.
cd <nux src root>
doxygen crates/nux/Doxyfile