Keras is an Open Source Neural Network library written in Python that runs on top of Theano, CNTK or Tensorflow. It is design to be fast and easy to use. It was develop by Francois Chollet, Google engineer. Keras does not handle low-level computation. So, it make uses another Backend library to resolve it.

It is high-level Application Program Interface wrapper for the low-level Application Program Interface capable of running on top of TensorFlow, Theano or CNTK (Microsoft Cognitive Toolkit). Keras High-Level API handles the way we make models, defining layers, or set up multiple input-output models.

Keras Library Function
logo

Keras Library Functions

  • relu function
  • sigmoid function
  • softmax function
  • softplus function
  • softsign sunction
  • tanh function
  • selu function
  • elu function
  • exponential function

Relu function

Relu function is stand for Rectified Linear Activation function. It is the most common choice of activation function. Relu function provides state of the art result and computationally very efficient at the same time is knows as Relu function.

Syntax

tf.keras.activations.relu(x, alpha=0.0, max_value=None, threshold=0.0)

The pseudo code for Relu function are follows:

if input > 0:
return input
else:
return 0

Implementing Relu function in python

def relu(x):
return max(0.0, x)

x = 1.0
print(‘Applying Relu on (%.1f) gives %.1f’ % (x, relu(x)))
x = -10.0
print(‘Applying Relu on (%.1f) gives %.1f’ % (x, relu(x)))
x = 0.0
print(‘Applying Relu on (%.1f) gives %.1f’ % (x, relu(x)))
x = 15.0
print(‘Applying Relu on (%.1f) gives %.1f’ % (x, relu(x)))
x = -20.0
print(‘Applying Relu on (%.1f) gives %.1f’ % (x, relu(x)))

Output

Applying Relu on (1.0) gives 1.0
Applying Relu on (-10.0) gives 0.0
Applying Relu on (0.0) gives 0.0
Applying Relu on (15.0) gives 15.0
Applying Relu on (-20.0) gives 0.0

Sigmoid function

The sigmoid function always returns an output between 0 and 1. An Sigmoid activation function is a mathematical function that controls the output of a neural network is knows as Sigmoid Function.

Syntax

tf.keras.activations.sigmoid(x)

Implementing Sigmoid function in Python

mport numpy as np
def sig(x):
return 1/(1 + np.exp(-x))

x = 1.0
print(‘Applying Sigmoid Activation on (%.1f) gives %.1f’ % (x, sig(x)))

x = -10.0
print(‘Applying Sigmoid Activation on (%.1f) gives %.1f’ % (x, sig(x)))

x = 0.0
print(‘Applying Sigmoid Activation on (%.1f) gives %.1f’ % (x, sig(x)))

x = 15.0
print(‘Applying Sigmoid Activation on (%.1f) gives %.1f’ % (x, sig(x)))

x = -2.0
print(‘Applying Sigmoid Activation on (%.1f) gives %.1f’ % (x, sig(x)))

Output

Applying Sigmoid Activation on (1.0) gives 0.7
Applying Sigmoid Activation on (-10.0) gives 0.0
Applying Sigmoid Activation on (0.0) gives 0.5
Applying Sigmoid Activation on (15.0) gives 1.0
Applying Sigmoid Activation on (-2.0) gives 0.1

Softmax function

Softmax converts a vector of values to a probability is knows as Softmax function. The output vector are in range (0, 1) and sum to 1.

Syntax

tf.keras.activations.softplus(x)

Example of Softmax Function

a = tf.constant([-20, -1.0, 0.0, 1.0, 20], dtype = tf.float32)
b = tf.keras.activations.softplus(a)
b.numpy()
array([2.0611537e-09, 3.1326166e-01, 6.9314718e-01, 1.3132616e+00,
         2.0000000e+01], dtype=float32)

Arguments

x : Input tensor.

Returns

The softplus activationlog(exp(x) + 1).

Softplus function

 It is use to find softplus of the stated input tensor is knows as Softplus Function.

Syntax

tf.keras.layers.Softmax(axis=-1, **kwargs)

Example of Softplus function

a = tf.constant([-20, -1.0, 0.0, 1.0, 20], dtype = tf.float32)
b = tf.keras.activations.softplus(a)
b.numpy()
array([2.0611537e-09, 3.1326166e-01, 6.9314718e-01, 1.3132616e+00,
         2.0000000e+01], dtype=float32)

Arguments

x: Input tensor.

Returns

The softplus activation: log(exp(x) + 1).

Softsign sunction

syntax

tf.keras.activations.softsign(x)

Example of softsign function

a = tf.constant([-1.0, 0.0, 1.0], dtype = tf.float32)
b = tf.keras.activations.softsign(a)
b.numpy()
array([-0.5,  0. ,  0.5], dtype=float32)

Arguments

x: Input tensor.

Returns

The softsign activation: x / (abs(x) + 1).

Tanh function

It is a Hyperbolic tangent activation function is knows as tanh function.

Syntax

tf.keras.activations.tanh(x)

Example of tanh function

a = tf.constant([-3.0,-1.0, 0.0,1.0,3.0], dtype = tf.float32)
b = tf.keras.activations.tanh(a)
b.numpy()
array([-0.9950547, -0.7615942,  0.,  0.7615942,  0.9950547], dtype=float32)

Arguments

x: Input tensor.

Returns

Tensor of same shape and dtype of input x, with tanh activation: tanh(x) = sinh(x)/cosh(x) = ((exp(x) - exp(-x))/(exp(x) + exp(-x))).

Selu function

It is stand for Scaled Exponential Linear Unit

Syntax

tf.keras.activations.selu(x)

Example of selu function

num_classes = 10  # 10-class problem
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(64, kernel_initializer='lecun_normal',
                                 activation='selu'))
model.add(tf.keras.layers.Dense(32, kernel_initializer='lecun_normal',
                                 activation='selu'))
model.add(tf.keras.layers.Dense(16, kernel_initializer='lecun_normal',
                                 activation='selu'))
model.add(tf.keras.layers.Dense(num_classes, activation='softmax'))

Arguments

x : A tensor or variable to compute the activation function for.

Returns

The scaled exponential unit activation: scale * elu(x, alpha).

elu function

It is stand for Exponential Linear Unit

Example of elu function

import tensorflow as tf
model = tf.keras.Sequential()
model.add(tf.keras.layers.Conv2D(32, (3, 3), activation='elu',
          input_shape=(28, 28, 1)))
model.add(tf.keras.layers.MaxPooling2D((2, 2)))
model.add(tf.keras.layers.Conv2D(64, (3, 3), activation='elu'))
model.add(tf.keras.layers.MaxPooling2D((2, 2)))
model.add(tf.keras.layers.Conv2D(64, (3, 3), activation='elu'))

Arguments

  • x: Input tensor.
  • alpha: A scalar, slope of negative section. alpha controls the value to which an ELU saturates for negative net inputs.

Returns

The exponential linear unit activation function: x if x > 0 and alpha * (exp(x) - 1) if x < 0.

Exponential function

Syntax

tf.keras.activations.exponential(x)

Example of Exponential function

a = tf.constant([-3.0,-1.0, 0.0,1.0,3.0], dtype = tf.float32)
b = tf.keras.activations.exponential(a)
b.numpy()
array([0.04978707,  0.36787945,  1.,  2.7182817 , 20.085537], dtype=float32)

Arguments

  • x: Input tensor.

Returns

Tensor with exponential activation: exp(x).

If you have any queries regarding this article or if I have missed something on this topic, please feel free to add in the comment down below for the audience. See you guys in another article.

To know more about keras please Wikipedia Click here


Basic Engineer

Hey Readers! We have more than fifteen years of experience in Software Development, IoT, Telecom, Banking, Finance and Embedded domain. Currently we are actively working on Data Science, ML and AI with multiple market leaders worldwide. Happy Reading. Cheers!

0 Comments

Leave a Reply

Avatar placeholder

Your email address will not be published. Required fields are marked *