Compare Models

You can compare two or more models on multiple images using the compare function.

This function is ideal if you want to evaluate how different models perform on a single image or multiple images.

The following example shows how to compare OWLv2 and Grounding DINO on a single image:

from autodistill_grounding_dino import GroundingDINO
from autodistill_owlv2 import OWLv2

from autodistill.detection import CaptionOntology
from autodistill.utils import compare

ontology = CaptionOntology(
    {
        "solar panel": "solar panel",
    }
)

models = [
    GroundingDINO(ontology=ontology),
    OWLv2(ontology=ontology),
]

images = [
    "./solar.jpg"
]

compare(
    models=models,
    images=images
)

Here are the results:

Above, we can see predictions from Grounding DINO and OWLv2.

Code Reference

Compare the predictions of multiple models on multiple images.

Parameters:

Name	Type	Description	Default
models	list	The models to compare	required
images	List[str]	The images to compare	required

Returns:

Type	Description
	A grid of images with the predictions of each model on each image.

Source code in autodistill/utils.py

def compare(models: list, images: List[str]):
    """
    Compare the predictions of multiple models on multiple images.

    Args:
        models: The models to compare
        images: The images to compare

    Returns:
        A grid of images with the predictions of each model on each image.
    """
    image_results = []
    model_results = []

    for model in models:
        # get model class name
        model_name = model.__class__.__name__

        for image in images:
            results = model.predict(image)

            image_data = cv2.imread(image)

            image_result = plot(
                image_data, results, classes=model.ontology.prompts(), raw=True
            )

            image_results.append(image_result)

            model_results.append(model_name)

    sv.plot_images_grid(
        image_results,
        grid_size=(len(models), len(images)),
        titles=model_results,
        size=(16, 16),
    )