Catalog

 

Code annotations

tf.random_normal()

tf.Variable()

ops.get_collection(ops.GraphKeys.GLOBAL_VARIABLES)

tf.variables_initializer

Why do variables have to be initialized?

Complete code

Code annotations

tf.random_normal()

Used to extract a specified number of values from a value that obeys a specified orthodox distribution.

tf.random_normal(
    shape,                #shape: shape of output tensor, required
    mean=0.0,             #mean: mean of normal distribution, default 0
    stddev=1.0,           #stddev: Standard deviation of normal distribution, default 1.0
    dtype=tf.float32,     #dtype: Type of output, default to tf.float32
    seed=None,            #Seed: The seed of a random number is an integer. When set, the random number generated is the same for each time.
    name=None             #Name: The name of the operation
)

tf.Variable()

Define variables.

weights = tf.Variable(tf.random_normal([2, 3], stddev=0.1),name="weights")
biases = tf.Variable(tf.zeros([3]), name="biases")
custom_variable = tf.Variable(tf.zeros([3]), name="custom")

 

ops.get_collection(ops.GraphKeys.GLOBAL_VARIABLES)

Put all three variables defined above into a list.

all_variables_list = ops.get_collection(ops.GraphKeys.GLOBAL_VARIABLES)

Put two variables together into a list.

variable_list_custom = [weights, custom_variable]

 

tf.variables_initializer

Returns an operation to initialize the list of variables. After starting the graph in session, you can run the returned operation to initialize all variables in var_list. This operation runs all initializers of variables in var_list parallels in parallel. Calling initialize_variables() is equivalent to passing the list that initializes. Pass it to Group(). However, if the var_list is empty, the tf.variables_initializer function still returns a runnable operation. The operation is just ineffective.

init_custom_op = tf.variables_initializer(var_list=variable_list_custom )

Of course, there is another way to initialize all variables directly:

init_all_op = tf.global_variables_initializer()

 

Initialize weights in variable definitions:

WeightsNew = tf.Variable(weights.initialized_value(), name="WeightsNew")

init_WeightsNew_op = tf.variables_initializer(var_list=[WeightsNew])

 

Why do variables have to be initialized?

We construct a variable by tf.Variable and add it to the graph. The Variable() constructor requires the initial value of the variable (a tensor of any type and shape), which specifies the type and shape of the variable. However, tensorflow actually runs in the form of graph graph graph structure, and calculates the value of a node in the graph only when sess.run is executed (passing in the node that needs to be valued). Before that, all the operations were to construct the structure of the graph and not to assign the actual value. When variable (1) is executed, it simply defines the structure (type is variable, initial value is 1). The value defined by the variable initialization method is assigned only when it is executed.

Define variables (put values in the corresponding variable space) - > initialize variables (place variables in the specified location of the graph) - > sess. run (start computing)

Complete code

## This code create some arbitrary variables and initialize them ###
# The goal is to show how to define and initialize variables from scratch.

import tensorflow as tf
from tensorflow.python.framework import ops

#######################################
######## Defining Variables ###########
#######################################

# Create three variables with some default values.
weights = tf.Variable(tf.random_normal([2, 3], stddev=0.1),name="weights")
biases = tf.Variable(tf.zeros([3]), name="biases")
custom_variable = tf.Variable(tf.zeros([3]), name="custom")

# Get all the variables' tensors and store them in a list.
all_variables_list = ops.get_collection(ops.GraphKeys.GLOBAL_VARIABLES)


############################################
######## Customized initializer ############
############################################

## Initialation of some custom variables.
## In this part we choose some variables and only initialize them rather than initializing all variables.

# "variable_list_custom" is the list of variables that we want to initialize.
variable_list_custom = [weights, custom_variable]

# The initializer
init_custom_op = tf.variables_initializer(var_list=variable_list_custom )


########################################
######## Global initializer ############
########################################

# Method-1
# Add an op to initialize the variables.
init_all_op = tf.global_variables_initializer()

# Method-2
init_all_op = tf.variables_initializer(var_list=all_variables_list)



##########################################################
######## Initialization using other variables ############
##########################################################

# Create another variable with the same value as 'weights'.
WeightsNew = tf.Variable(weights.initialized_value(), name="WeightsNew")

# Now, the variable must be initialized.
init_WeightsNew_op = tf.variables_initializer(var_list=[WeightsNew])

######################################
####### Running the session ##########
######################################
with tf.Session() as sess:
    # Run the initializer operation.
    sess.run(init_all_op)
    sess.run(init_custom_op)
    sess.run(init_WeightsNew_op)