Rspress

# #Model Inference

## #hbDNNInferV2

`int32_t hbDNNInferV2(hbUCPTaskHandle_t *taskHandle, hbDNNTensor *output, hbDNNTensor const *input, hbDNNHandle_t dnnHandle);`

Create synchronous/asynchronous inference tasks based on input parameters. For the asynchronous inference task, the caller can use the returned `taskHandle` across functions and threads.

- Parameter
  - [out] `taskHandle` Task handle pointer.
  - [in/out] `output` Output of the inference task.
  - [in] `input` Input of the inference task.
  - [in] `dnnHandle` DNN handle pointer.
- Return Value
  - Return `0` means the API was successfully executed, otherwise the execution failed.

Note

1. If `taskHandle` is set to `nullptr`, a synchronous task will be created, as returned by the interface, and the interface will be complete. Synchronous tasks do not support multi-core model inference.
2. If `*taskHandle` is set to `nullptr`, an asynchronous task will be created, and the `taskHandle` returned by the interface can be used for subsequent blocking or callbacking.
3. If `*taskHandle` is not null and points to a previously created but uncommitted task, a new task will be created and added to it. Multi-core tasks do not support task addition operations.

Up to 32 coexisting model tasks are supported.

## #hbDNNGetTaskOutputTensorProperties

`int32_t hbDNNGetTaskOutputTensorProperties(hbDNNTensorProperties *properties, hbUCPTaskHandle_t taskHandle, int32_t subTaskIndex, int32_t outputIndex);`

Get dynamic output property information of the task that `taskHandle` points to.

- Parameter
  - [out] `properties` Info of the input tensor.
  - [in] `taskHandle` Task handle pointer.
  - [in] `subTaskIndex` Subtask index.
  - [in] `outputIndex` Index of the output tensor of the subtask.
- Return Value
  - Return `0` means the API was successfully executed, otherwise the execution failed.

Note

1. If the model has dynamic output, you need to use this interface to obtain the dynamic attribute information of the task output for subsequent processing. The non-dynamic output tensor information is the same as that obtained by `hbDNNGetOutputTensorProperties`.

2. If the task pointed to by `taskHandle` has multiple subtasks, `subTaskIndex` is used to specify the subtask number.

3. If the model contains dynamic output, please do not use the synchronous task execution method during inference, because the synchronously executed task cannot obtain the taskHandle.

On This Page