Nux C API  0.0.1
C API of NPU Engine
C API of NPU Engine

Nux is the C API library of the NPU Engine. Nux provides the two kinds of execution ways to run inference tasks through NPU. The followings demonstrate the examples of two execution ways. If you want to see more details about API types and functions, please refer to nux.h.

Sync Model

The sync model provides APIs to run a single inference task with a single input batch at a time.

Example of Sync Model

// Get the size of ENF file
const char *enf_path = "<a path of ENF file>";
struct stat st;
stat(enf_path, &st);
size_t enf_size = st.st_size;
// Read the ENF binary from the ENF file
void *enf_buf = malloc(enf_size);
FILE *enf_file = fopen(enf_path, "rb");
size_t read_item = fread(enf_buf, enf_size, 1, enf_file);
assert(read_item == 1);
fclose(enf_file);
// Initialize nux handle
nux_handle_t nux = NULL;
// Create a sync model
nux_sync_model_t sync_model = NULL;
assert(nux_create_sync_model(nux, binary_buf, enf_size, &sync_model) == nux_error_t_success);
// Get the 0th input tensor from this model, and
// A model can have more than one input tensor depending on its model design.
// You can specify an index of input tensor to get your desired input tensor.
nux_tensor_t input_tensor = NULL;
assert(model_input_tensor(sync_model, 0, &input_tensor) == nux_error_t_success);
// Fill the buffer of the input tensor with your data
// Here, the input shape of the 0th input tensor is 1x28x28x1 in NxHxWxC,
// and it has totally 784 elements. In this example, the buffer is filled with a fake data.
uint8_t input_buf [784] = {[0 ... 783] = 1};
tensor_set_buffer(input_tensor, input_buf, 784);
// Get the 0th output tensor from the model
nux_tensor_t output_tensor = NULL;
assert(model_output_tensor(sync_model, 0, &output_tensor) == nux_error_t_success);
// Run an inference task with the input data filled as above.
assert(model_run(sync_model) == nux_error_t_success);
// Get the buffer of the 0th's output tensor.
// output_buf will receive a pointer to the output tensor's buffer.
// output_len will receive the length of the output buffer.
uint8_t *output_buf = NULL;
uintptr_t output_len = NULL;
tensor_get_buffer(output_tensor, &output_buf, &output_len);
// Release resources
destroy_sync_model(sync_model);
enum nux_error_t nux_create_sync_model(nux_handle_t nux, const uint8_t *buffer, nux_buffer_len_t len, nux_sync_model_t *model)
Create an instance of a synchronous model.
void destroy_sync_model(nux_sync_model_t model)
Destroy the sync model and release its resources.
struct Model * nux_sync_model_t
Handle of a synchronous model.
Definition: nux.h:240
enum nux_error_t tensor_set_buffer(nux_tensor_t tensor, nux_buffer_t buffer, nux_buffer_len_t len, void(*free)(uint8_t *, uintptr_t))
Specify a pointer to a data buffer to a specified input tensor.
@ nux_error_t_success
Definition: nux.h:56
enum nux_error_t model_input_tensor(nux_sync_model_t model, uint32_t index, nux_tensor_t *tensor)
Get the handle of a specified input tensor from a given sync model.
struct Nux * nux_handle_t
Nux handle.
Definition: nux.h:228
struct OpaqueTensor * nux_tensor_t
Handle of tensor.
Definition: nux.h:247
enum nux_error_t tensor_get_buffer(nux_tensor_t tensor, nux_buffer_t *buffer, nux_buffer_len_t *len)
Get a pointer to the data buffer of a given tensor.
enum nux_error_t create_nux(nux_handle_t *nux)
Creates a new nux handle.
enum nux_error_t model_run(nux_sync_model_t model)
Run a single inference task.
void destroy_nux(nux_handle_t nux)
Destroy the Nux handle and release its resources.
enum nux_error_t model_output_tensor(nux_sync_model_t model, uint32_t index, nux_tensor_t *tensor)
Get the handle of a specified output tensor from a given sync model.

Task Model

The task model provides APIs to run multiple inference tasks asynchronously and simultaneously. The callback functions defined by users will be called when each task is completed. When a user creates a task model, the user can configure the concurrency of running tasks depending on HW capacity.

Example of Task Model

// Define three callback functions for a task model
void my_nux_output_cb(nux_request_id_t id,
nux_buffer_len_t buf_len) {
// fill your logic
}
void my_error_cb(nux_request_id_t id, nux_error_t err) {
// fill your logic
}
void my_finish_cb(nux_request_id_t id) {
// fill your logic
}
void run() {
// Get the size of ENF file
const char *enf_path = "<a path of ENF file>";
struct stat st;
stat(enf_path, &st);
size_t enf_size = st.st_size;
// Read the ENF binary from the ENF file
void *enf_buf = malloc(enf_size);
FILE *enf_file = fopen(enf_path, "rb");
size_t read_item = fread(enf_buf, enf_size, 1, enf_file);
assert(read_item == 1);
fclose(enf_file);
// Create a nux handle
nux_handle_t nux = NULL;
assert(create_nux(&nux) == nux_error_t_success);
// Create a task model with 3 max concurrency
nux_task_model_t task_model = NULL;
const uint32_t max_concurrency = 3;
assert(nux_create_task_model(nux, enf_buf, enf_size, max_concurrency,
my_nux_output_cb, my_error_cb, my_finish_cb, &task_model) == nux_error_t_success);
// Get a single task which allows your code to request a single asynchronous inference task.
// Depending on the concurrency, you can get multiple tasks at the same time.
// In this case, you can request multiple tasks simultaneously.
// If there's no available task slot, calling task_model_get_task will be blocked
// until new available task comes out.
nux_task_t task1 = NULL;
assert(task_model_get_task(task_model, &task1) == nux_error_t_success);
// Fill the buffer of the 0th input tensor with your data.
// Here, the input shape of the 0th input tensor is 1x28x28x1 in NxHxWxC,
// and it has totally 784 elements.
const uintptr_t INPUT_SIZE = 784;
assert(task_input_size(task1, 0) == INPUT_SIZE);
// It's a fake data. Please fill this buffer with yours.
uint8_t input_buf [784] = {[0 ... 783] = 1};
// Get a mutable buffer of the 0th input tensor from task1
uint8_t* buf = task_input(task1, 0);
// Copy your data to the buffer
memcpy(buf, input_buf, INPUT_SIZE);
// Submit the task1 with an arbitrary req_id,
// which will be passed to callback functions to identify some context.
RequestId req_id = 1;
task_execute(task1, req_id);
// Wait for until all submitted tasks are completed
while(!task_model_is_all_task_done(task_model)) {
sleep(1);
}
// Release resources
destroy_task_model(task_model);
free(enf_buf);
}
enum nux_error_t nux_create_task_model(nux_handle_t nux, const uint8_t *buffer, nux_buffer_len_t len, uint32_t max_batch, nux_output_callback_t output_callback, nux_error_callback_t error_callback, nux_finish_callback_t finish_callback, nux_task_model_t *task_model)
Create an instance of a task model.
enum nux_error_t task_model_get_task(nux_task_model_t task_model, nux_task_t *task)
Get a task handle from a specified task model.
OutputIndex nux_output_index_t
Index of an output tensor, used in task model.
Definition: nux.h:269
struct TaskModel * nux_task_model_t
Handle of a task model.
Definition: nux.h:298
bool task_model_is_all_task_done(nux_task_model_t task_model)
Return true if there's no running tasks, or false if any task is still running.
nux_buffer_len_t task_input_size(nux_task_t task, nux_input_index_t index)
Return the buffer length in bytes of the specified input tensor.
nux_error_t
Represent a return status of a function that can fail.
Definition: nux.h:52
void destroy_task_model(nux_task_model_t task_model)
Destroy the task model and release its resources.
uintptr_t RequestId
Definition: nux.h:257
RequestId nux_request_id_t
Identifier to distinguish tasks in a task model.
Definition: nux.h:262
nux_buffer_t task_input(nux_task_t task, nux_input_index_t index)
Return a mutable pointer to the buffer of the specified input tensor.
struct Task * nux_task_t
Handle of a task of a task model.
Definition: nux.h:305
uintptr_t nux_buffer_len_t
Byte length of a data buffer.
Definition: nux.h:233
uint8_t * nux_buffer_t
Pointer to a data buffer.
Definition: nux.h:255
enum nux_error_t task_execute(nux_task_t task, nux_request_id_t request_id)
Request one asynchronous inference task.

How to build source codes with Nux

To build your source code with Nux, please follow:

export NPU_TOOLS_DIR="<npu-tools src dir>"
export NUX_DIR=${NPU_TOOLS_DIR}/crates/nux
cargo build --release
cc sync_example.c -I${NUX_DIR}/include -L${NPU_TOOLS_DIR}/target/x86_64-unknown-linux-gnu/release -lnux

How to generate Doxygen

The following commands will generate a Doxygen document at $NUX_TOOLS_DIR/target/doxygen/html.

cd <nux src root>
doxygen crates/nux/Doxyfile