Image Similarity Search using Vector embeddings and Cosine similarity

 Image embeddings capture visual features (shapes, colors, objects, textures)

Image embedding models are typically CNN or vision transformer based

Image Embedding Generation:

• Instead of embedding text, we embed images using a pre-trained vision model
• Popular models include Vision Transformers (ViT), ResNet, EfficientNet, or CLIP
• The model extracts features from images and converts them to dense vector representations

Core Similarities with Text Search:

• Both methods convert unstructured data (text/images) into vector representations
• Both use similarity metrics (typically cosine similarity) to find closest matches
• Both can be stored in vector databases like AstraDB for efficient retrieval

Applications:

• Product recommendations (visually similar products)
• Reverse image search
• Finding duplicate or near-duplicate images
• Content moderation (finding similar inappropriate content)
• Medical image analysis (finding similar cases)

Python code -

import os

import pandas as pd

import numpy as np

from sklearn.metrics.pairwise import cosine_similarity

from PIL import Image

import torch

from transformers import AutoImageProcessor, AutoModel

# Set device

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Initialize image embedding model - using ViT (Vision Transformer)

model_name = "google/vit-base-patch16-224"

processor = AutoImageProcessor.from_pretrained(model_name)

model = AutoModel.from_pretrained(model_name).to(device)

# Function to get image embedding

def get_image_embedding(image_path):

    image = Image.open(image_path)

    inputs = processor(images=image, return_tensors="pt").to(device)

    with torch.no_grad():

        outputs = model(**inputs)

    # Use CLS token as the image embedding

    embedding = outputs.last_hidden_state[:, 0, :].cpu().numpy().flatten()

    return embedding

# List of image paths

image_paths = [

    "Indigo Nation Men Plain Orange Shirts.jpg",

    "ADIDAS Men Navy Blue Shirts.jpg",

    "Indigo Nation Men Price catch Blue Shirts .jpg",

    "Puma Men's Foundation Grey Polo T-shirt.jpg",

    "Indigo Nation Men  Bling Pink Shirts.jpg"

]

# Query image

query_image = "Indigo Nation Men Plain Orange Shirts.jpg"

# Get embedding for the query image

query_embedding = get_image_embedding(query_image)

# Store all embeddings and their corresponding images

all_embeddings = []

for image_path in image_paths:

    embedding = get_image_embedding(image_path)

    all_embeddings.append(embedding)

# Convert to numpy arrays for similarity calculation

query_embedding_np = np.array(query_embedding).reshape(1, -1)

all_embeddings_np = np.array(all_embeddings)

# Calculate cosine similarity between query and all images

similarities = cosine_similarity(query_embedding_np, all_embeddings_np).flatten()

# Create a DataFrame to display results

results = pd.DataFrame({

    'Image': image_paths,

    'Similarity Score': similarities

})

# Sort by similarity score in descending order

results = results.sort_values('Similarity Score', ascending=False)

print(f"Query image: {query_image}")

print("\nSimilarity Search Results:")

print(results)

# Find the most similar image

most_similar_idx = np.argmax(similarities)

print(f"\nMost similar image: \"{image_paths[most_similar_idx]}\" with similarity score: {similarities[most_similar_idx]:.4f}")

Text Similarity Search using Vector embeddings and Cosine similarity

This code shows how to find semantically similar text using vector embeddings and cosine similarity.

HuggingFaceEmbedding from llama_index is imported to generate text embeddings

BAAI/bge-small-en-v1.5 -  lightweight  embedding model used to convert each text in the weather_descs  list  to vector embeddings.

Cosine similarity helps identify texts with similar meaning by measuring how "aligned" their vector representations are

Python Code -

import os

import pandas as pd

import numpy as np

from sklearn.metrics.pairwise import cosine_similarity

from llama_index.embeddings.huggingface import HuggingFaceEmbedding

# Set TensorFlow environment variable to suppress warnings

os.environ["TF_ENABLE_ONEDNN_OPTS"] = "0"

# Initialize embedding model

embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

# List of Texts

weather_descs = [

    "The weather cold today",

    "Today is Monday",

    "Today is first Sunday of winter",  

    "Today is Sunday"

]

# Query text

query = "It is freezing today"

# Get embedding for the query

query_embedding = embed_model.get_text_embedding(query)

# Store all embeddings and their corresponding texts

all_embeddings = []

for weather_desc in weather_descs:

    # Get embeddings for the current text

    embedding = embed_model.get_text_embedding(weather_desc)

    all_embeddings.append(embedding)

# Convert to numpy arrays for similarity calculation

query_embedding_np = np.array(query_embedding).reshape(1, -1)

all_embeddings_np = np.array(all_embeddings)

# Calculate cosine similarity between query and all texts

similarities = cosine_similarity(query_embedding_np, all_embeddings_np).flatten()

# Create a DataFrame to display results

results = pd.DataFrame({

    'Text': weather_descs,

    'Similarity Score': similarities

})

# Sort by similarity score in descending order

results = results.sort_values('Similarity Score', ascending=False)

print("Query:", query)

print("\nSimilarity Search Results:")

print(results)

# Find the most similar text

most_similar_idx = np.argmax(similarities)

print(f"\nMost similar text: \"{weather_descs[most_similar_idx]}\" with similarity score: {similarities[most_similar_idx]:.4f}")

Output :