🐶
Machine Vision

OpenCV Image Similarity Comparison with Python

By Jan on 03/04/2025

Learn how to leverage the power of OpenCV in Python to compare the similarity between images for tasks like duplicate detection or image search.

OpenCV Image Similarity Comparison with Python

Table of Contents

Introduction

Image similarity comparison is a fundamental task in computer vision, often used in various applications like object recognition, image retrieval, and quality assessment. This article provides a step-by-step guide on how to compare the similarity between two images using Python and OpenCV. We will explore two common metrics: Mean Squared Error (MSE) and Structural Similarity Index (SSIM). We will start by loading the necessary libraries and images, then delve into the calculation and interpretation of each metric. Additionally, we will cover optional steps for visualizing the differences between the images.

Step-by-Step Guide

  1. Import necessary libraries:
import cv2
import numpy as np
  1. Load the images:
image1 = cv2.imread('image1.jpg')
image2 = cv2.imread('image2.jpg')
  1. Convert images to grayscale:
gray1 = cv2.cvtColor(image1, cv2.COLOR_BGR2GRAY)
gray2 = cv2.cvtColor(image2, cv2.COLOR_BGR2GRAY)
  1. Calculate Mean Squared Error (MSE):
error = np.sum((gray1.astype("float") - gray2.astype("float")) ** 2)
error /= float(gray1.shape[0] * gray1.shape[1])

  1. Interpret MSE:

    • Lower MSE values indicate higher similarity.
    • Higher MSE values indicate lower similarity.

  2. (Optional) Calculate Structural Similarity Index (SSIM):

(score, diff) = cv2.compareSSIM(gray1, gray2, full=True)
diff = (diff * 255).astype("uint8")

  1. Interpret SSIM:

    • SSIM values range from -1 to 1.
    • Values closer to 1 indicate higher similarity.

  2. (Optional) Visualize differences:

thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
  1. (Optional) Display results:
print("MSE:", error)
print("SSIM:", score)
cv2.imshow("Diff", diff)
cv2.waitKey(0)

Code Example

The Python code compares the similarity between two images using Mean Squared Error (MSE) and Structural Similarity Index (SSIM). It loads the images, converts them to grayscale, calculates MSE and SSIM, and visualizes the differences. Lower MSE and higher SSIM values indicate greater similarity. The code displays the original images, a difference map, and a thresholded difference map to highlight areas of dissimilarity.

import cv2
import numpy as np

# Load the images
image1 = cv2.imread('image1.jpg')
image2 = cv2.imread('image2.jpg')

# Convert images to grayscale
gray1 = cv2.cvtColor(image1, cv2.COLOR_BGR2GRAY)
gray2 = cv2.cvtColor(image2, cv2.COLOR_BGR2GRAY)

# Calculate Mean Squared Error (MSE)
error = np.sum((gray1.astype("float") - gray2.astype("float")) ** 2)
error /= float(gray1.shape[0] * gray1.shape[1])

# Interpret MSE
print("MSE:", error)  # Lower values indicate higher similarity

# Calculate Structural Similarity Index (SSIM)
(score, diff) = cv2.compareSSIM(gray1, gray2, full=True)
diff = (diff * 255).astype("uint8")

# Interpret SSIM
print("SSIM:", score)  # Values closer to 1 indicate higher similarity

# Visualize differences
thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]

# Display results
cv2.imshow("Image 1", image1)
cv2.imshow("Image 2", image2)
cv2.imshow("Diff", diff)
cv2.imshow("Thresh", thresh)
cv2.waitKey(0)

Explanation:

  1. Import Libraries: Import cv2 for image processing and numpy for numerical operations.
  2. Load Images: Load the two images using cv2.imread().
  3. Convert to Grayscale: Convert both images to grayscale using cv2.cvtColor(). This is often necessary for accurate similarity comparisons.
  4. Calculate MSE: Calculate the Mean Squared Error between the grayscale images. MSE measures the average squared difference between corresponding pixels.
  5. Interpret MSE: Print the MSE value. Lower MSE indicates higher similarity.
  6. Calculate SSIM: Calculate the Structural Similarity Index (SSIM) using cv2.compareSSIM(). SSIM is a more advanced metric that considers structural information in the images.
  7. Interpret SSIM: Print the SSIM score. Values closer to 1 indicate higher similarity.
  8. Visualize Differences:
    • Multiply the diff output from cv2.compareSSIM() by 255 and convert it to an 8-bit unsigned integer to visualize the differences.
    • Apply thresholding using cv2.threshold() to highlight the most significant differences.
  9. Display Results: Display the original images, the difference map (diff), and the thresholded difference map (thresh) using cv2.imshow().

To use this code:

  1. Replace 'image1.jpg' and 'image2.jpg' with the actual file names of your images.
  2. Run the code. It will print the MSE and SSIM values and display the images and difference maps.

Additional Notes

General Considerations:

  • Image Preprocessing: Before comparison, consider resizing images to a common size and normalizing their histograms to account for lighting variations. This can improve the accuracy of similarity metrics.
  • Color vs. Grayscale: While converting to grayscale is common, preserving color information might be crucial for certain applications. Explore color-based similarity metrics if needed.
  • Metric Selection: The choice between MSE and SSIM depends on the specific application. MSE is simple but sensitive to noise and distortions. SSIM is more robust but computationally expensive.

MSE:

  • Sensitivity: MSE is sensitive to even small pixel-level differences. A high MSE value doesn't necessarily mean the images are completely dissimilar; they might just have minor variations.
  • Normalization: Consider normalizing the MSE value by the image size or the maximum possible error to obtain a relative measure of similarity.

SSIM:

  • Perceptual Quality: SSIM is designed to align better with human perception of similarity, considering luminance, contrast, and structure.
  • Channel-wise Comparison: SSIM can be calculated for individual color channels (R, G, B) and then averaged to get a comprehensive similarity score.

Visualizing Differences:

  • Heatmaps: Instead of simple thresholding, consider generating heatmaps to visualize the magnitude of differences between images.
  • Contour Detection: Use contour detection algorithms (e.g., cv2.findContours()) on the difference map to highlight regions of dissimilarity.

Applications:

  • Object Tracking: Track objects across video frames by comparing the similarity of regions of interest.
  • Image Retrieval: Build image search engines that retrieve visually similar images from a database.
  • Quality Assessment: Evaluate the quality of compressed or reconstructed images by comparing them to their original versions.

Beyond MSE and SSIM:

  • Explore other similarity metrics like:
    • Feature-based comparison (SIFT, SURF, ORB): Extract and compare distinctive features between images.
    • Deep learning-based approaches: Utilize pre-trained convolutional neural networks (CNNs) for image similarity tasks.

Summary

This code snippet demonstrates how to compare the similarity between two images using Python and the OpenCV library.

Steps:

  1. Load Images: Load the two images to be compared using cv2.imread().
  2. Convert to Grayscale: Convert both images to grayscale using cv2.cvtColor() for efficient comparison.
  3. Calculate MSE: Compute the Mean Squared Error (MSE) between the grayscale images. Lower MSE values indicate higher similarity.
  4. (Optional) Calculate SSIM: Calculate the Structural Similarity Index (SSIM) for a more comprehensive similarity measure. SSIM values closer to 1 indicate higher similarity.
  5. (Optional) Visualize Differences: Highlight the differences between the images using thresholding on the SSIM difference map.
  6. (Optional) Display Results: Print the calculated MSE and SSIM values and display the difference image.

Key Takeaways:

  • MSE: Provides a simple numerical measure of image difference.
  • SSIM: Offers a more perceptually aligned similarity metric considering structural information.
  • Visualization: Helps identify specific regions of difference between the images.

This code provides a basic framework for image similarity comparison. You can adapt and extend it for various applications like image search, object tracking, and change detection.

Conclusion

This article explored image similarity comparison using Python and OpenCV, focusing on MSE and SSIM metrics. We learned to calculate these metrics, interpret their results, and visualize differences between images. While MSE offers a simple difference measure, SSIM provides a perceptually aligned approach by considering structural information. Choosing the appropriate metric depends on the specific application requirements. This guide serves as a starting point for incorporating image similarity comparison into various computer vision tasks. Remember to consider image preprocessing, explore color-based metrics when needed, and investigate advanced techniques like feature-based comparison or deep learning for more complex scenarios.

References

Were You Able to Follow the Instructions?

😍Love it!
😊Yes
😐Meh-gical
😞No
🤮Clickbait

Related posts

Square Detection in Images: Algorithms and Techniques

Learn how to accurately detect squares in images using computer vision techniques and algorithms for various applications.

Find Max Value in OpenCV cv::Mat (C++)

Learn how to efficiently extract the maximum value from a cv::Mat in OpenCV using C++, along with code examples and explanations.

PyTorch CUDA Out of Memory Error: Despite Free Memory

Learn how to troubleshoot and fix the frustrating "CUDA out of memory" error in PyTorch, even when your GPU seems to have plenty of free memory available.