Concatenate Images with NumPy

Introduction

Concatenation is a process of combining two things from the end. In formal language theory and computer programming, string concatenation is the operation of joining character strings end-to-end. To precisely explain this, we can perform concatenation for two words snow and ball resulting in snowball. It need not be from the end.

Credits of Cover Image - Photo by Bia Andrade on Unsplash

Concatenation of images is simply concatenating multiple matrices. But in Python, we need to understand that concatenation of lists is different from the concatenation of NumPy arrays.

Concatenating lists

For concatenating two or multiple lists, we can use the + operation that will combine two lists or multiple lists into one big list. Below is an example that can be understood easily.

>>> l1 = [1, 2, 3, 4]
>>> l2 = [4, 5, 6, 7]
>>> 
>>> # concatenation
>>> l3 = l1 + l2
>>> l3
[1, 2, 3, 4, 4, 5, 6, 7]
>>> 
>>> # or
>>> l3 = l1.extend(l2)
>>> l3
[1, 2, 3, 4, 4, 5, 6, 7]

Concatenating NumPy arrays

The scenario would be completely different for NumPy arrays if we were to perform the same + operation as we did for lists (above). NumPy automatically performs linear addition (broadcasting technique) considering the shape of each array is similar. Below is an example that can be understood easily.

For 1D arrays

>>> import numpy as np
>>> 
>>> l1 = np.array([1, 2, 3, 4])
>>> l2 = np.array([4, 5, 6, 7])
>>> 
>>> # using `+` operation
>>> l3 = l1 + l2
>>> print(l3)
[ 5  7  9 11]
>>>

For 2D arrays

>>> import numpy as np
>>> 
>>> l1 = np.array([[1, 2], [3, 4]])
>>> l2 = np.array([[4, 5], [6, 7]])
>>> 
>>> # using `+` operation
>>> l3 = l1 + l2
>>> print(l3)
[[ 5  7]
 [ 9 11]]
>>>

For both cases, the + operation does not work. Rather, we can use the concatenate() method of NumPy's module to combine the arrays. The only criteria here is that we should have all the arrays in the same dimension.

With the help of this method, we can either concatenate horizontally or vertically. Below is an example that can be understood easily.

>>> import numpy as np
>>> 
>>> l1 = np.array([[1, 2], [3, 4]])
>>> l2 = np.array([[4, 5], [6, 7]])
>>> 
>>> # concatenation
>>> # axis = 0 → concatenates vertically
>>> l3 = np.concatenate((l1, l2), axis=0)
>>> print(l3)
[[1 2]
 [3 4]
 [4 5]
 [6 7]]
>>> 
>>> # axis = 1 → concatenates horizontally
>>> l4 = np.concatenate((l1, l2), axis=1)
>>> print(l4)
[[1 2 4 5]
 [3 4 6 7]]
>>>

From the above example, we are clear that concatenation can be easily done and the same thing is performed on the images to combine two or multiple (different) images.

We are basically replicating the methods hconcat() (horizontal concatenation) and vconcat() (vertical concatenation) of the module cv2 in NumPy.

Time to Code

The packages that we mainly use are:

NumPy
Matplotlib
OpenCV → It is only used for reading the image (in this article).

`import` the Packages

import numpy as np
import cv2
import json
from matplotlib import pyplot as plt

Read the Image

def read_this(image_file, gray_scale=False):
    image_src = cv2.imread(image_file)
    if gray_scale:
        image_src = cv2.cvtColor(image_src, cv2.COLOR_BGR2GRAY)
    else:
        image_src = cv2.cvtColor(image_src, cv2.COLOR_BGR2RGB)
    return image_src

The above function reads the image either in grayscale or RGB and returns the image matrix.

Image Re-sizing

While concatenating two different images, there can be a difference in the dimensions of the image matrices. We need to make sure that the dimensions are the same. There is a need to resize the image matrix (refer to this article for understanding image resizing). And, hence the below function.

def resize_image(image_matrix, nh, nw):
    image_size = image_matrix.shape
    oh = image_size[0]
    ow = image_size[1]   

    re_image_matrix = np.array([
        np.array([image_matrix[(oh*h // nh)][(ow*w // nw)] for w in range(nw)]) 
        for h in range(nh)
    ])

    return re_image_matrix

For this, we will need to take two different images.

Image 1

Image 2

Note - The dimensions of the above images are different.

Image Matrices

# grayscale mode is false by default

image1 = read_this(image_file='lena_original.png')
image2 = read_this(image_file='pinktree.jpg')

Code Implementation with Library

The function takes totally three arguments -

image_set → a list of image matrices.
how → concatenating on what basis, vertically or horizontally.
default arg with_plot → to plot the results or not.

def concat_lib(image_set, how, with_plot=False):
    # dimension of each matrix in image_set
    shape_vals = [imat.shape for imat in image_set]

    # length of dimension of each matrix in image_set
    shape_lens = [len(ishp) for ishp in shape_vals]

    # if all the images in image_set are read in same mode
    channel_flag = True if len(set(shape_lens)) == 1 else False

    if channel_flag:
        ideal_shape = max(shape_vals)
        images_resized = [
            # function call to resize the image
            resize_image(image_matrix=imat, nh=ideal_shape[0], nw=ideal_shape[1]) 
            if imat.shape != ideal_shape else imat for imat in image_set
        ]
    else:
        return False

    # cv2 library code to concatenate the image matrices
    # we use methods like
    #     - vconcat() → vertical concat
    #     - hconcat() → horizontal concat
    if (how == 0) or (how == 'vertical'):
        concats = cv2.vconcat(images_resized)
    elif (how == 1) or (how == 'horizontal'):
        concats = cv2.hconcat(images_resized)
    else:
        concats = cv2.hconcat(images_resized)

    if with_plot:
        cmap_val = None if len(concats.shape) == 3 else 'gray'
        plt.figure(figsize=(10, 6))
        plt.axis("off")
        plt.imshow(concats, cmap=cmap_val)
        return True
    return concats

In the case of image matrices with different dimensions, they are handled carefully by taking the ideal dimension whose matrix dimension is maximum. We are using the methods hconcat() and vconcat() based on the value of how that is passed.

Let's test the above function -

concat_lib(
    image_set=[image1, image2, image1, image1, image2], 
    how='horizontal', 
    with_plot=True
)

image2 is automatically resized to the size of image1 and the same is converted into a giant matrix.

Code Implementation from Scratch