Issues with RTP params

[GiuseppeSorrentino99]

Hello everyone,
I am trying to write a very basic kernel taking as input a float RTP, using bfloat16 or classic float.

This is the kernel Code:

void test(uint8_t* img1, uint8_t* img2, float* out, float size) {

    out[0] = size;
}

I am using the following behaviouralFX

def test_behaviour(invobj):
    invobj.out.array = np.ndarray(shape = (1,1,4), dtype = np.float32)

test.behavioralfx = test_behaviour

And the following application

class TESTApplication(AppBuilder):

    def __init__(self, kernel:Kernel):
        self.kernel = kernel
        super().__init__()

    def callgraph(self, img1: np.ndarray, img2: np.ndarray, out: np.ndarray) -> None:
        out[0] = self.kernel(img1,img2,size)
        
        
appBuilder = TESTApplication(kernel=test)

To set the size, I use

app.test_0.size.value = 2.5

However, when looking at the output, I have the following results,

Using Float
Input 0 is 0
Input 1 is 1.40129846e-45
Input 2 is 2.8025969e-45

It looks like "Input * 1.4e-45"

Using bfloat16 ( hence changing the type of the kernel function size param)
Input 0 is 0
Input 1 is 9.1835496e-41
Input 2 is 1.8367099e-40

It may probably be an error in my code, but the same code works with uint32_t

[mariodruiz]

Hi @GiuseppeSorrentino99,

Can you please provide the full code?
Are these output after run on the NPU or just the behavioral run?

[GiuseppeSorrentino99]

Sure, this is the full code:

import npu

%%kernel

#define vector_size 32 // Size of the vector unit, used for reading from input stream
#define read_type uint8 // type of the input stream
#define func_type int // type used for doing the computations
#define write_type float // type of the output stream
#define max_pix_val 255 // max pixel val used for psnr

inline aie::vector<func_type,vector_size> convert_to_func_type(aie::vector<read_type,vector_size> vec) {
    aie::accum<acc32, vector_size> acc;
    acc.from_vector(vec, 0);
    return acc.to_vector<func_type>();
}

void test(uint8_t* img1, uint8_t* img2, float* out, bfloat16 size) {

    out[0] = size;
}


def test_behaviour(invobj):
    invobj.out.array = np.ndarray(shape = (1,1,4), dtype = np.float32)

test.behavioralfx = test_behaviour

import numpy as np
from npu.build.kernel import Kernel
from npu.build.appbuilder import AppBuilder

class TESTApplication(AppBuilder):

    def __init__(self, kernel:Kernel):
        self.kernel = kernel
        super().__init__()

    def callgraph(self, img1: np.ndarray, img2: np.ndarray, out: np.ndarray) -> None:
        out[0] = self.kernel(img1,img2,size)
        
        
appBuilder = TESTApplication(kernel=test)

vol = 1
dim = 64
img1 = np.zeros(shape=(dim,dim,vol),dtype=np.uint8)
img2 = np.zeros(shape=(dim,dim,vol),dtype=np.uint8)
out = np.zeros(shape=(1,1,4),dtype=np.float32)
size = dim*dim*vol

appBuilder.build(img1,img2,out)
image1 = np.full(size,255, dtype=np.uint8)
image2 = np.full(size,1, dtype=np.uint8)
image1p = np.uint32(image1)
image2p = np.uint32(image2)

from npu.runtime import AppRunner
app = AppRunner('TESTApplication.xclbin')
app.metadata

app.test_0.size.value = 2.0

input_image1 = app.allocate(shape=(dim,dim,vol), dtype=np.uint8)
input_image2 = app.allocate(shape=(dim,dim,vol), dtype=np.uint8)

input_image1[:] = image1.reshape(dim,dim,vol)
input_image1.sync_to_npu()

input_image2[:] = image2.reshape(dim,dim,vol)
input_image2.sync_to_npu()

output_val = app.allocate(shape=(1,1,4), dtype=np.float32)
app.call(input_image1,input_image2, output_val)
output_val.sync_from_npu()
print(output_val)

If my understanding is correct, this is the output after running on the NPU

[mariodruiz]

I think you have found a limitation. Only integers seems to be supported for run time parameters

[GiuseppeSorrentino99]

Oh, got it.
Thanks again for your support and the fast reply!

[mariodruiz]

@GiuseppeSorrentino99 I found a work around, the bfloat16 value needs to be interpreted as corresponding 16-bit integer.

a = bfloat16(3.14)

app.test_0.size.value = int.from_bytes(a.tobytes(), "little")

This should also work for float numbers.

This could be a nice contribution to Riallto, if you would like to collaborate the code to interpret the values should go here https://github.com/AMDResearch/Riallto/blob/main/npu/runtime/sequence.py#L138 you would probably want to track the dtype of the RTP to be able to interpret it correctly