Tensorflow : Tensor | Shape | Dtype | Graph | Session

What is a Tensor?

Tensorflow gets its name from word Tensor. Every computations in tensorflow is done on tensors. A tensor is simple terms are like a vector or a multi-dimensional matrix. They are used to represents all types of data in tensorflow. A tensor can either be the input data or the output of a computation.

In TensorFlow we also have graph. It is a series of all the tensorflow operations or computations that to be performed. It is tensorflow’s way of representing the flow of computations. They are hence also know as computational graphs. Each operation in the graph is called an op node. These nodes are connected to each other, based on the flow of the operations.

The graphs don’t display the values. We populate values to the operations in the graph through the tensors. Tensors are edge of the nodes.

In Machine Learning, to create a trained models we feed them with feature vectors. Input tensors are populated with feature vector. The tensor will pass the feature vector values into an op node and as a result of this operation/computation a new tensor is returned that will be passed to the next new operation.

In this tutorial, you will learn-

• Representation of a Tensor
• Types & Shape of Tensor
• Type of data
• Creating operator
• Variables & Placeholder
• Session & Graph

Representation of a Tensor

A tensor is nothing but n-dimensions. For instance, if we have a 2×3 matrix with values from 1 to 6, we write:

TensorFlow represents above matrix as:

[[1, 2, 3], 
 [4, 5, 6]] 

If we create a three-dimensional matrix we have:

[[['#', '#', '#'], ['#', '#', '#'], ['#', '#', '#']],  
 [['#', '#', '#'], ['#', '#', '#'], ['#', '#', '#']],  
 [['#', '#', '#'], ['#', '#', '#'], ['#', '#', '#']]]  

Note: A tensor can be a scalar or multi-dimensional vector.

Types of Tensor

Three types of tensors:

1. A unique label (represents a name)
2. A dimension (represents a shape)
3. A data type (represents a data type)

Each operation we perform are manipulation of these tensor. Four main tensors are:

1. tf.Variable
2. tf.constant
3. tf.placeholder
4. tf.SparseTensor

Let us start by importing tensorflow

# Import tf
import tensorflow as tf			

Create a tensor of n-dimension

Lets start with a tensor of one dimension, a scalar.

To create a scalar tensor, we can use the function tf.constant()

tf.constant(value, dtype, name = "")

Arguments
- "value": Value of n dimension to define the tensor. Optional
- "dtype": Define the type of data:    
    - `tf.string`: String variable    
    - `tf.float32`: Float variable    
    - `tf.int16`: Integer variable
- "name": Name of the tensor. Optional. By default, `Const_1:0`     

Following command will create a scalar tensor:

s1 = tf.constant(1, tf.int16) 
print(s1) 

Output:
Tensor(“Const:0”, shape=(), dtype=int16)

# Named my_scalar
r2 = tf.constant(1, tf.int16, name = "my_scalar") 
print(r2)			

Output:
Tensor(“my_scalar:0”, shape=(), dtype=int16)

Each tensor object has a unique label (name), a dimension (shape) and a data type (dtype).

To create a tensor of type float or a string you need to changing the type of data.

# Decimal
r1_decimal = tf.constant(1.12345, tf.float32)
print(r1_decimal)
# String
r1_string = tf.constant("Guru99", tf.string)
print(r1_string)			

Output:
Tensor("Const_1:0", shape=(), dtype=float32)
Tensor("Const_2:0", shape=(), dtype=string) 

To create a 1-dimension tensor:

## Rank 
1r1_vector = tf.constant([1,3,5], tf.int16)
print(r1_vector)
r2_boolean = tf.constant([True, True, False], tf.bool)
print(r2_boolean) 

Output:
Tensor("Const_3:0", shape=(3,), dtype=int16)
Tensor("Const_4:0", shape=(3,), dtype=bool) 

As we observe above, shape has 1 column.

To create 2 dimensions tensor, we need to close the brackets after each row, as done below

## Rank 2
r2_matrix = tf.constant([ [1, 2],
                          [3, 4] ],tf.int16)
print(r2_matrix)			

Output:
Tensor("Const_5:0", shape=(2, 2), dtype=int16) 

The matrix contains 2 rows & 2 columns with values 1, 2, 3, 4.

A tensor of 3 dimensions is defined by adding one more level with the brackets.

## Rank 3
r3_matrix = tf.constant([ [[1, 2],
                           [3, 4], 
                           [5, 6]] ], tf.int16)
print(r3_matrix)			

Output:
Tensor("Const_6:0", shape=(1, 3, 2), dtype=int16) 

Shape of tensor

You can get shape of a tensor with its default shape property.

Example:

# Shape of tensor
m_shape = tf.constant([ [10, 11],
                        [12, 13],
                        [14, 15] ]) 
m_shape.shape 

Output:
TensorShape([Dimension(3), Dimension(2)])

The matrix has 3 rows and 2 columns.

TensorFlow has useful commands that creates a vector or matrix pre-filled with values0 or 1.

Let us create a 1-D tensor of shape 10, filled with 0.

# Create a vector of 0
print(tf.zeros(10))			

Output

Tensor("zeros:0", shape=(10,), dtype=float32)			

The property works for matrix as well. Here, you create a 10×10 matrix filled with 1

# Create a vector of 1
print(tf.ones([10, 10]))			

Output:
Tensor(“ones:0”, shape=(10, 10), dtype=float32)

We can use shape tensor of a matrix to create a vector of ones.

# Create a vector of ones with the same number of rows as m_shape
print(tf.ones(m_shape.shape[0]))			

Output:
Tensor(“ones_1:0”, shape=(3,), dtype=float32)

If you pass the value 1 into the bracket, you can construct a vector of ones equals to the number of columns in the matrix m_shape.

# Create a vector of ones with the same number of column as m_shape
print(tf.ones(m_shape.shape[1]))			

Output:
Tensor("ones_2:0", shape=(2,), dtype=float32) 

Finally, you can create a matrix 3×2 with only one’s

print(tf.ones(m_shape.shape))			

Output:
Tensor("ones_3:0", shape=(3, 2), dtype=float32) 

Type of data

Another property of a tensor is type of data. A tensor can only have one type of data at a time. To know the type of a tensor we can use its property dtype.

print(m_shape.dtype)			

Output:
<dtype: 'int32'> 

tf.cast function can be used to change data type of a tensor.

Example

Below, a float tensor is converted to integer using you use the method cast.

# Change type of data
type_float = tf.constant(3.123456789, tf.float32)
type_int = tf.cast(type_float, dtype=tf.int32)
print(type_float.dtype)
print(type_int.dtype)			

Output

<dtype: 'float32'>
<dtype: 'int32'>			

Creating operator

Some Useful TensorFlow operators

Now let us perform some mathematical operations.

Let us perform some basic operations.

The square of a number is constructed with tf.sqrt(x) with x as a floating number.

x = tf.constant([2.0], dtype = tf.float32)
print(tf.sqrt(x))			

Output

Tensor("Sqrt:0", shape=(1,), dtype=float32)			

Following are the commonly operations available in tensorflow.

• tf.add(a, b)
• tf.substract(a, b)
• tf.multiply(a, b)
• tf.div(a, b)
• tf.pow(a, b)
• tf.exp(a)
• tf.sqrt(a)

Example

# Add
tensor_a = tf.constant([[1,2]], dtype = tf.int32)
tensor_b = tf.constant([[3, 4]], dtype = tf.int32)

tensor_add = tf.add(tensor_a, tensor_b)print(tensor_add)			

Output

Tensor("Add:0", shape=(1, 2), dtype=int32)			

Code Explanation

Create two tensors:

• one tensor with 1 and 2
• one tensor with 3 and 4

You add up both tensors.

# Multiply
tensor_multiply = tf.multiply(tensor_a, tensor_b)
print(tensor_multiply)			

Output

Tensor("Mul:0", shape=(1, 2), dtype=int32)			

Variables

Let us now see how we can declare variables in tensorflow.

To create a variable, you can use tf.get_variable() method

tf.get_variable(name = "", values, dtype, initializer)
argument
- name = "" : Name of the variable
- values : Dimension of the tensor
- dtype: Type of data. Optional
- initializer : How to initialize the tensor. Optional
If initializer is specified, there is no need to include the values as the shape of `initializer` is used. 

Example:

# Create a Variable
## Create 2 Randomized values
var = tf.get_variable("var", [1, 2])
print(var.shape)			

Output

(1, 2)			

We can also pass the values of a constant tensor to a variable.

The first values of the variable are 10, 20, 30 and 40. The new tensor will have a shape of 2×2.

# Create a 2x2 matrixtensor_const = tf.constant([[10, 20],
[30, 40]])
# Initialize the first value of the tensor equals to tensor_const
var_init_2 = tf.get_variable( "var_init_2", dtype=tf.int32,  initializer=tensor_const )
print( var_init_2.shape ) 

Output

(2, 2)			

Placeholder

Placeholder is used to initialize the data to flow inside the tensors. It is a value that we input when we run a computation. 

Lets create a placeholder.

The syntax is:

tf.placeholder(dtype,shape=None,name=None )
arguments:
- dtype: Type of data
- shape: dimension of the placeholder. Optional. By default, shape of the data
- name: Name of the placeholder. Optional 
data_placeholder_a = tf.placeholder(tf.float32, name = "data_placeholder_a")
print(data_placeholder_a) 

Output

Tensor("data_placeholder_a:0", dtype=float32)			

Session

TensorFlow works around 3 main components:

• Graph
• Tensor
• Session

Graph

The graph is very important in TensorFlow. All of the mathematical operations (ops) are performed inside a graph.

Tensor

A tensor represents the data that progress between operations.

Session

A session will execute all the operation in the graph. We feed the graph with tensors, through a session. Inside a session, you must run an operator to create an output.

Graphs and sessions are independent. You can run a session and get the values to use later for further computations.

In the example below, you will:

• Create two tensors
• Create an operation
• Open a session
• Print the result

Step 1) You create two tensors x and y

## Create, run  and evaluate a session
x = tf.constant([2])
y = tf.constant([4])			

Step 2) You create the operator by multiplying x and y

## Create operator
multiply = tf.multiply(x, y)			

Step 3) You open a session. All the computations will happen within the session. When you are done, you need to close the session.

## Create a session to run the code
sess = tf.Session()
result_1 = sess.run(multiply)
print(result_1)
sess.close() 

Output

[8]			

Code explanation

• tf.Session(): Open a session. All the operations will flow within the sessions
• run(multiply): execute the operation created in step 2.
• print(result_1): Finally, you can print the result
• close(): Close the session

The result shows 8, which is the multiplication of x and y.

Another way to create a session is inside a block. The advantage is it automatically closes the session.

with tf.Session() as sess:    
result_2 = multiply.eval()
print(result_2) 			

Output

[8]			

In a context of the session, you can use the eval() method to execute the operation. It is equivalent to run(). It makes the code more readable.

You can create a session and see the values inside the tensors you created so far.

## Check the tensors created before
sess = tf.Session()
print(sess.run(r1))
print(sess.run(r2_matrix))
print(sess.run(r3_matrix))			

Output

1
[[1 2] 
 [3 4]]
[[[1 2]  
  [3 4]  
  [5 6]]]			

Variables are empty by default, even after you create a tensor. We initialize the variable to use it. The object tf.global_variables_initializer() initializes a variable. The previous function will explicitly initialize all the variables. This is helpful before you train a model.

You can check the values of the variables you created before. Note that you need to use run to evaluate the tensor

sess.run(tf.global_variables_initializer())
print(sess.run(var))
print(sess.run(var_init_1))
print(sess.run(var_init_2))			

Output

[[-0.05356491  0.75867283]]
[[0 0]]
[[10 20] 
 [30 40]]			

You can use the placeholder you created before and feed it with actual value. You need to pass the data into the method feed_dict.

For example, you will take the power of 2 of the placeholder data_placeholder_a.

import numpy as np
power_a = tf.pow(data_placeholder_a, 2)
with tf.Session() as sess:  
data = np.random.rand(1, 10)  
print(sess.run(power_a, feed_dict={data_placeholder_a: data}))  # Will succeed.			

Code Explanation

• import numpy as np: Import numpy library to create the data
• tf.pow(data_placeholder_a, 2): Create the ops
• np.random.rand(1, 10): Create a random array of data
• feed_dict={data_placeholder_a: data}: Feed the placeholder with data

Output

[[0.05478134 0.27213147 0.8803037  0.0398424  0.21172127 0.01444725  0.02584014 0.3763949  0.66022706 0.7565559 ]]			

Graph

As we expained at the start, in TensorFlow we have something called graph. It is a series of all the tensorflow operations or computations that to be performed. It is tensorflow’s way of representing the flow of computations. They are hence also know as computational graphs. Each operation in the graph is called an op node. These nodes are connected to each other, based on the flow of the operations. The graphs don’t display the values. We populate values to the operations in the graph through the tensors. Tensors are edge of the nodes.

The graph does not display the output of the operations, it only helps us in visualizing the connection among individual operations.

---

That’s all for this post, thank you for reading till the end. You can also go through our post on Tensorflow Js Basics and Basic concepts in Tensorflow Js