Avoid for loops with Vectorization
Vectorization in Python
In this article, we will discuss about how to speed up your code by using NumPy’s feature of Vectorization.
Python’s NumPy arrays enable us to express batch operations on data without writing any explicit for loops, referred as Vectorization. This is one of the key features of optimization using NumPy apart from broadcasting.
Let’s dig deep into vectorization!
Vectorized array operations execute faster than their python equivalents and thus have a high impact in numerical computations. The reason why NumPy is efficient because it’s operations are mapped to highly optimized C codes.
In deep learning practice, we often train on relatively large data sets, so it’s important that our code should be optimized to execute fast. In these scenarios vectorization is quite useful, it enables us to take much better advantage of parallelism and to do our computations much faster on CPUs and GPUs as well.
Examples to showcase impact of Vectorization
Example 1 — Dot product operation
Approach 1 — Using for Loop
# Import libraries
import numpy as np
import time
import math#1. Using for loop
n = 1000000
arr1 = np.random.rand(n)
arr2 = np.random.rand(n)# dot product using for loop
start_time_1 = time.time()
output=0
for i in range(n):
output += arr1[i]*arr2[i]print("time taken using for loop approach ",\
time.time()- start_time_1,'ms')
print('Output',output)
Approach 2 — Using NumPy
## 2. using vectorization
start_time_2 = time.time()
output1 = np.dot(arr1,arr2)
print("time taken using vectorization",\
time.time() - start_time_2,'ms')
print('Output',output1)
In the above example, dot product operation has been performed and it is clearly visible that for the same operation ‘for’ loop takes 0.6ms while vectorization takes merely 0.003ms.
Example 2 : Exponent operation
n = 100000000
arr1 = np.random.rand(n)
output = np.zeros((n,1))## Approach 1
print("Using for loop")
start_time = time.time()
for i in range(n):
output[i] = math.exp(arr1[i])print("time taken ",time.time()- start_time,'ms')
print('Output - first 3 values',output[:3])## Approach 2
print("Numpy implementation")
start_time = time.time()
output1 = np.exp(arr1)
print("time taken ",time.time()- start_time,'ms')
print('Output - first 3 values',output1[:3])
From the above example it may feel like 70 ms is way small even using ‘for’ loop but vectorization showcases its core strength while working with big datasets and creates magic by executing computations at a much faster speed!
