Ytukyg

From this google cloud doc and this one,
and the stackoverflow answer in this post by
rhaertel80, I think the recommended format of a json request to send images to a model for prediction on Google cloud is:

{'instances': {'image_bytes': {'b64': base64.b64encode(jpeg_data)}}, {'image_bytes':...}}

The next step is to create the serving_input_fn() (described in the google cloud docs and this GCP tutorial), which can cope with the nested dictionary that the request will send.

To do this I need to create 'features' and a 'receiver_tensor' to pass into the ServingInputReciever function which the serving_input_fn() needs to return.

However, I do not see how the requirement of the receiver_tensor to be a dictionary with keys and tensors as values can fit the nested format of the json request. (As I understand it, the receiver_tensors are placeholders for the request).

If the request does not contain nested dictionaries the approach seems to be fairly simple as shown in the tutorials and this answer.

Question

So, how can the serving_input_fn() be formatted to receive the image request in the described form and create features and receiver_tensors which fill the requirements of the ServingInputReceiver function?

Part of the difficulty may be that I do not understand what the serving_input_fn() will need to process. Will it be the entire request in one go? Or will each instance be passed one at a time? Or is there some other way to understand what the function will be processing

More details

For more context, I am using tf.Estimator and the train_and_evaluate function to train a model and deploy it to google cloud. The input to the model is a tf.dataset containing tuples of ({'spectrogram': image}, label) where image is a tensor.

My attempt to create the input_fn assumes one element of the instances list is passed at a time:

def serving_input_fn():

    feature_placeholders = {'image_bytes': {'b64': tf.placeholder(dtype=tf.string,

                                                                  shape=[None],

                                                                  name='source')}}

    input_images = convert_bytestrings_to_images(feature_placeholders['image_bytes']['b64'])

    features = {'spectrogram': image for image in input_images}

    return tf.estimator.export.ServingInputReceiver(features, feature_placeholders)

Which leads to the error

ValueError: receiver_tensor image_bytes must be a Tensor.

And I am not sure if features would be in the correct form either.

The b64 is handled transparently by ML Engine. So, while the input JSON needs to have the b64, your serving input function will just receive the raw bytes. So your serving input fn needs to be:

def serving_input_fn():

    feature_placeholders = {'image_bytes': 

                            tf.placeholder(tf.string, shape=())}

    input_images = convert_bytestrings_to_images(feature_placeholders['image_bytes']))

    features = {'spectrogram': image for image in input_images}

    return tf.estimator.export.ServingInputReceiver(features, feature_placeholders)