Missing input quantization

[CIPop]

Thank you for the sample!

I'm not 100% sure, but I believe that the processor is missing input quantization as described at https://www.tensorflow.org/lite/performance/post_training_integer_quant#run_the_tensorflow_lite_models

Something like

Edited with corrected code based on this discussion:

# 1. Read image as 300x300x3 uint8
res_im = im.resize((300, 300))
np_res_im = np.array(res_im)

#2 Transform from input uint8 RGB [0..255] to float [-1, 1] 
np_res_im = (np_res_im / 255) * 2 - 1

#3 Apply quantization based on https://www.tensorflow.org/lite/performance/post_training_integer_quant#run_the_tensorflow_lite_models
# Check if the input type is quantized, then rescale input data to uint8
if input_details['dtype'] == np.uint8:
    input_scale, input_zero_point = input_details["quantization"]
    np_res_im = np_res_im / input_scale + input_zero_point
 
np_res_im = np.expand_dims(np_res_im, axis=0).astype(input_details["dtype"])

# Quantized input [0..255] is still 300x300x3 but the quantized values are different from the original image:
print(np_res_im)

This is based on this output from the first cell:

[{'name': 'normalized_input_image_tensor',
  'index': 175,
  'shape': array([  1, 300, 300,   3], dtype=int32),
  'shape_signature': array([  1, 300, 300,   3], dtype=int32),
  'dtype': numpy.uint8,
  'quantization': (0.0078125, 128),
  'quantization_parameters': {'scales': array([0.0078125], dtype=float32),
   'zero_points': array([128], dtype=int32),
   'quantized_dimension': 0},
  'sparsity_parameters': {}}]

The dtype is uint8 and quantization and zero-point information is available.

[CIPop]

I take this back: doing above will result in really bad detection.
(As reference, I was trying to port this model to WAMR, which internally does the quantization whenever this information is available: https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/libraries/wasi-nn/src/wasi_nn_tensorflowlite.cpp#L313)

[CIPop]

Re-opening based on the discussion/explanation from WAMR.

The pre-processing should be:

1. Obtain input - coincidentally, uint8_t `300x300x3, values [0..255]
2. Transform input to float 300x300x3, values [-1..1]
3. Based on the model input type (uint8_t), apply quantization parameters. This transforms the input back to uint8_t 300x300x3, values [0..255].

The important part is that, while the tensors in steps 1 and 3 have the same shape and type, the values are clearly different.