TF hub training image classifier

reference resources Training image classifier


The image classification model has millions of parameters. Training from scratch requires a lot of labeled training data and a lot of computing power. Transfer learning is a skill that greatly simplifies the process by adopting a model that has been trained on related tasks and reusing it in a new model.

This paper demonstrates how to extract five kinds of features from the generic image set of kingsflow through the pre training of kingsflow. Alternatively, the feature extractor can be trained ("fine tuned") with the newly added classifier.

Looking for tools?

This is a TensorFlow coding tutorial. If you want a tool that can only be used to build TensorFlow or TF Lite models for it, please check the make tool installed by the PIP package TensorFlow hub [make_image_classifier]_ image_ Classifier command line tool, or view it in this TF Lite collaboration lab.

Setup settings

import itertools
import os

import matplotlib.pylab as plt
import numpy as np

import tensorflow as tf
import tensorflow_hub as hub

print("TF version:", tf.__version__)
print("Hub version:", hub.__version__)
print("GPU is", "available" if tf.test.is_gpu_available() else "NOT AVAILABLE")
TF version: 2.4.1
Hub version: 0.11.0
WARNING:tensorflow:From <ipython-input-1-0831fa394ed3>:12: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
GPU is available

Select the TF2 SavedModel module to use

For beginners, please use . You can use the same URL in your code to identify the SavedModel and use the same URL in your browser to display its documents. (note that the model of TF1 Hub format does not work here.)

You can here Find more TF2 models that generate image feature vectors.

There are many possible models to try. All you need to do is select a different in the cells below and continue using the notebook.

model_name = "mobilenet_v3_small_100_224" # @param ['bit_s-r50x1', 'efficientnet_b0', 'efficientnet_b1', 'efficientnet_b2', 'efficientnet_b3', 'efficientnet_b4', 'efficientnet_b5', 'efficientnet_b6', 'efficientnet_b7', 'inception_v3', 'inception_resnet_v2', 'mobilenet_v2_100_224', 'mobilenet_v2_130_224', 'mobilenet_v2_140_224', 'mobilenet_v3_large_100_224', 'mobilenet_v3_large_075_224', 'mobilenet_v3_small_100_224', 'mobilenet_v3_small_075_224', 'nasnet_large', 'nasnet_mobile', 'pnasnet_large', 'resnet_v1_50', 'resnet_v1_101', 'resnet_v1_152', 'resnet_v2_50', 'resnet_v2_101', 'resnet_v2_152']

model_handle_map = {
  "efficientnet_b0": "",
  "efficientnet_b1": "",
  "efficientnet_b2": "",
  "efficientnet_b3": "",
  "efficientnet_b4": "",
  "efficientnet_b5": "",
  "efficientnet_b6": "",
  "efficientnet_b7": "",
  "bit_s-r50x1": "",
  "inception_v3": "",
  "inception_resnet_v2": "",
  "resnet_v1_50": "",
  "resnet_v1_101": "",
  "resnet_v1_152": "",
  "resnet_v2_50": "",
  "resnet_v2_101": "",
  "resnet_v2_152": "",
  "nasnet_large": "",
  "nasnet_mobile": "",
  "pnasnet_large": "",
  "mobilenet_v2_100_224": "",
  "mobilenet_v2_130_224": "",
  "mobilenet_v2_140_224": "",
  "mobilenet_v3_small_100_224": "",
  "mobilenet_v3_small_075_224": "",
  "mobilenet_v3_large_100_224": "",
  "mobilenet_v3_large_075_224": "",

model_image_size_map = {
  "efficientnet_b0": 224,
  "efficientnet_b1": 240,
  "efficientnet_b2": 260,
  "efficientnet_b3": 300,
  "efficientnet_b4": 380,
  "efficientnet_b5": 456,
  "efficientnet_b6": 528,
  "efficientnet_b7": 600,
  "inception_v3": 299,
  "inception_resnet_v2": 299,
  "nasnet_large": 331,
  "pnasnet_large": 331,

model_handle = model_handle_map.get(model_name)
pixels = model_image_size_map.get(model_name, 224)

print(f"Selected model: {model_name} : {model_handle}")

IMAGE_SIZE = (pixels, pixels)
print(f"Input size {IMAGE_SIZE}")

Selected model: mobilenet_v3_small_100_224 :
Input size (224, 224)

Set up flower dataset

The input will be adjusted according to the size of the selected module. Data set enhancement (that is, random distortions occur every time the image is read) can improve the training effect, especially. When fine tuning.

data_dir = tf.keras.utils.get_file(
Downloading data from
228818944/228813984 [==============================] - 14s 0us/step	
datagen_kwargs = dict(rescale=1./255, validation_split=.20)
dataflow_kwargs = dict(target_size=IMAGE_SIZE, batch_size=BATCH_SIZE,

valid_datagen = tf.keras.preprocessing.image.ImageDataGenerator(
valid_generator = valid_datagen.flow_from_directory(
    data_dir, subset="validation", shuffle=False, **dataflow_kwargs)

do_data_augmentation = False
if do_data_augmentation:
  train_datagen = tf.keras.preprocessing.image.ImageDataGenerator(
      width_shift_range=0.2, height_shift_range=0.2,
      shear_range=0.2, zoom_range=0.2,
  train_datagen = valid_datagen
train_generator = train_datagen.flow_from_directory(
    data_dir, subset="training", shuffle=True, **dataflow_kwargs)
Found 731 images belonging to 5 classes.
Found 2939 images belonging to 5 classes.

Define model

All you have to do is feature_extractor_layer places a linear classifier on the top of the Hub module.

In order to improve speed, we never train features_ extractor_ Layer starts, but you can also enable fine tuning to improve accuracy.

do_fine_tuning = False

Do not use fine_tuning

model structure 
print("Building model with", model_handle)
model = tf.keras.Sequential([
    # Explicitly define the input shape so the model can be properly
    #Explicitly define the input shape so that the model can run correctly
    # loaded by the TFLiteConverter
    tf.keras.layers.InputLayer(input_shape=IMAGE_SIZE + (3,)),
    hub.KerasLayer(model_handle, trainable=do_fine_tuning),
Building model with
Model: "sequential"
Layer (type)                 Output Shape              Param #   
keras_layer (KerasLayer)     (None, 1024)              1529968   
dropout (Dropout)            (None, 1024)              0         
dense (Dense)                (None, 5)                 5125      
Total params: 1,535,093
Trainable params: 5,125
Non-trainable params: 1,529,968

Training model

  optimizer=tf.keras.optimizers.SGD(lr=0.005, momentum=0.9), 
  loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.1),
steps_per_epoch = train_generator.samples // train_generator.batch_size
validation_steps = valid_generator.samples // valid_generator.batch_size
hist =
    epochs=5, steps_per_epoch=steps_per_epoch,

For each round of training of the generator, how many steps, that is, how many batches_ size
steps_per_epoch = samples divided by batch_size

Epoch 1/5
91/91 [==============================] - 30s 174ms/step - loss: 1.1899 - accuracy: 0.5899 - val_loss: 0.7321 - val_accuracy: 0.8423
Epoch 2/5
91/91 [==============================] - 15s 160ms/step - loss: 0.6630 - accuracy: 0.8935 - val_loss: 0.7036 - val_accuracy: 0.8651
Epoch 3/5
91/91 [==============================] - 14s 158ms/step - loss: 0.6405 - accuracy: 0.9095 - val_loss: 0.6973 - val_accuracy: 0.8580
Epoch 4/5
91/91 [==============================] - 15s 160ms/step - loss: 0.6143 - accuracy: 0.9156 - val_loss: 0.6817 - val_accuracy: 0.8722
Epoch 5/5
91/91 [==============================] - 15s 160ms/step - loss: 0.5917 - accuracy: 0.9323 - val_loss: 0.6795 - val_accuracy: 0.8778

Draw a picture

plt.ylabel("Loss (training and validation)")
plt.xlabel("Training Steps")

plt.ylabel("Accuracy (training and validation)")
plt.xlabel("Training Steps")
[<matplotlib.lines.Line2D at 0x7f0f2030b198>]

Test the model on the image from the validation data:

Perform steps

Define string and index Converted function

 next ,from valid_Generator Get a sample from x,y
x,y Remove dimension
 display plt.imshow()

model.predict forecast(image To expand a dimension)

Get maximum label index =np.argmax

Carry string string and index Conversion of
# This is an incoming index, that is, the maximum label index of the prediction vector, to obtain the class_ Function of index string
# valid_generator.class_indices.items() is a dictionary with correspondence between string and index
def get_class_string_from_index(index):
   for class_string, class_index in valid_generator.class_indices.items():
      if class_index == index:
         return class_string

x, y = next(valid_generator)
image = x[0, :, :, :]
true_index = np.argmax(y[0])

# Expand the validation image to (1, 224, 224, 3) before predicting the label
prediction_scores = model.predict(np.expand_dims(image, axis=0))
predicted_index = np.argmax(prediction_scores)
print("True label: " + get_class_string_from_index(true_index))
print("Predicted label: " + get_class_string_from_index(predicted_index))

True label: daisy
Predicted label: daisy

Finally, the trained model can be saved for deployment to TF Serving or TF Lite (on mobile devices), as shown below.

saved_model_path = f"/tmp/saved_flowers_model_{model_name}", saved_model_path)
INFO:tensorflow:Assets written to: /tmp/saved_flowers_model_mobilenet_v3_small_100_224/assets
INFO:tensorflow:Assets written to: /tmp/saved_flowers_model_mobilenet_v3_small_100_224/assets

Optional: deploy to TensorFlow Lite

TensorFlow Lite Enables you to deploy TensorFlow models to mobile and IoT devices. The following code shows how to convert the trained model into TF Lite and apply it TensorFlow Model Optimization Toolkit Post training tools in. Finally, it runs in the TF Lite interpreter to check the final quality

  1. The conversion can be carried out without optimization, and the result is the same as before (maximum rounding error).
  2. When the neural network is optimized, it will still be used to convert the weight of 8-bit to floating-point data, but it will not be used to calculate the weight. This reduces the model size by nearly four times and improves CPU latency on mobile devices.
  3. Most importantly, if a small reference data set is provided to calibrate the quantization range, the calculation of neural network activation can also be quantified as an 8-bit integer. On mobile devices, this can further speed up reasoning and enable it to run on accelerators such as EdgeTPU.

Optimize settings

optimize_lite_model = False 
num_calibration_examples = 60 
representative_dataset = None
if optimize_lite_model and num_calibration_examples:
  # Use a bounded number of training examples without labels for calibration.
  # TFLiteConverter expects a list of input tensors, each with batch size 1.
  representative_dataset = lambda: itertools.islice(
      ([image[None, ...]] for batch, _ in train_generator for image in batch),

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_path)
if optimize_lite_model:
  converter.optimizations = [tf.lite.Optimize.DEFAULT]
  if representative_dataset:  # This is optional, see above.
    converter.representative_dataset = representative_dataset
lite_model_content = converter.convert()

with open(f"/tmp/lite_flowers_model_{model_name}.tflite", "wb") as f:
print("Wrote %sTFLite model of %d bytes." %
      ("optimized " if optimize_lite_model else "", len(lite_model_content)))
Wrote TFLite model of 6097236 bytes.
interpreter = tf.lite.Interpreter(model_content=lite_model_content)
# This little helper wraps the TF Lite interpreter as a numpy-to-numpy function.
def lite_model(images):
  interpreter.set_tensor(interpreter.get_input_details()[0]['index'], images)
  return interpreter.get_tensor(interpreter.get_output_details()[0]['index'])
num_eval_examples = 50 
eval_dataset = ((image, label)  # TFLite expects batch size 1.
                for batch in train_generator
                for (image, label) in zip(*batch))
count = 0
count_lite_tf_agree = 0
count_lite_correct = 0
for image, label in eval_dataset:
  probs_lite = lite_model(image[None, ...])[0]
  probs_tf = model(image[None, ...]).numpy()[0]
  y_lite = np.argmax(probs_lite)
  y_tf = np.argmax(probs_tf)
  y_true = np.argmax(label)
  count +=1
  if y_lite == y_tf: count_lite_tf_agree += 1
  if y_lite == y_true: count_lite_correct += 1
  if count >= num_eval_examples: break
print("TF Lite model agrees with original model on %d of %d examples (%g%%)." %
      (count_lite_tf_agree, count, 100.0 * count_lite_tf_agree / count))
print("TF Lite model is accurate on %d of %d examples (%g%%)." %
      (count_lite_correct, count, 100.0 * count_lite_correct / count))
TF Lite model agrees with original model on 50 of 50 examples (100%).
TF Lite model is accurate on 47 of 50 examples (94%).


From two dictionaries
model_handle_map :get model of hub address
model_image_size_map:Get the input corresponding to the model image_size of pixels

get ready IMAGE_SIZE= (pixels, pixels) ,BATCH_SIZE = 32

Create parameter transfer dictionary datagen_kwargs ,dataflow_kwargs 
use**datagen_kwargs,**dataflow_kwargs Pass it in
 Get data  		tf.keras.utils.get_file()		
Read picture			 tf.keras.preprocessing.image.ImageDataGenerator(**datagen_kwargs)


Picture builder has properties num_class

steps_per_epoch = train_generator.samples // train_generator.batch_size
validation_steps = valid_generator.samples // valid_generator.batch_size

hist =	steps_per_epoch #How many steps per round History directly gets the list of history records

Save model,saved_model_path)

Keywords: TensorFlow Deep Learning

Added by g-force2k2 on Wed, 09 Feb 2022 04:11:09 +0200