Control flow

Starting from Android 11, the NNAPI includes two control flow operations, IF and WHILE, that take other models as arguments and execute them conditionally (IF) or repeatedly (WHILE). This allows for constructing models that execute different operations based on the input values or execute operations multiple times without unrolling. This is important for use cases such as dynamic RNN and seq2seq.

In NN HAL 1.3, the model incorporates multiple subgraphs, including the main subgraph that's used for specifying inputs and outputs of an execution. A subgraph can reference other subgraphs using operands of type SUBGRAPH. The framework can send a control flow operation to an accelerator only if the accelerator supports all of the operations in all subgraphs referenced by that control flow operation.

HAL interfaces

In NN HAL 1.3, the definitions related to control flow are in types.hal.

• IF and WHILE operation types
• SUBGRAPH operand type and the corresponding SUBGRAPH operand lifetime
• Model structure that contains the main subgraph and a list of referenced subgraphs
• Capabilities structure that contains ifPerformance and whilePerformance

IDevice.hal contains IDevice, whose method getSupportedOperations_1_3() must treat IF and WHILE differently from other operations.

IPreparedModel.hal contains IPreparedModel, whose methods execute_1_3(), executeSynchronously_1_3(), and executeFenced() take an optional loopTimeoutDuration argument.

Driver implementation

For a sample operation implementation, see CpuExecutor::executeIfOperation and CpuExecutor::executeWhileOperation. For sample operation validation logic, see validateIfOperation() and validateWhileOperation().

Note that it's important to support arithmetic and comparison operations on TENSOR_INT32 operands of shape 1, as these can be used as loop counters. Similarly, operations producing TENSOR_BOOL8 operands of shape 1 should be used with IF and WHILE conditions.

WHILE loop execution timeout

To prevent infinite loops, execution must be aborted if a WHILE loop takes longer than the loopTimeoutDuration value passed to a call of IPreparedModel::execute_1_3(), IPreparedModel::executeSynchronously_1_3(), or IPreparedModel::executeFenced() (or the default value if omitted).

Validation

Control flow tests are part of the CTS and VTS test suites. For more information, see Validation.