Moderating Memes with Qwen2.5-VL: Zero-Shot Hateful Content Detection in Transformers 5.12.1

TL;DR

Qwen2.5-VL is a practical way to try multimodal content moderation without training a custom classifier first.
In transformers==5.12.1, the clean path is AutoProcessor plus Qwen2_5_VLForConditionalGeneration.
A robust zero-shot workflow needs a few fixes that notebook exports often skip: no hard-coded cache paths, no GPU-only assumptions, and no brittle "Yes." string matching.

Abstract

Zero-shot moderation is appealing because it lets you test a multimodal model on a real safety task before you commit to fine-tuning, dataset curation, or model serving work. The original notebook draft here had the right core idea, but it mixed blog prose with Colab scaffolding, assumed a CUDA-only runtime, and used a fragile evaluation rule that only counted the exact string "Yes." as positive. This rewrite turns that export into a cleaner technical post: it explains the task, updates the code for transformers==5.12.1, and shows how to evaluate Qwen2.5-VL on the Hateful Memes validation split with a reproducible zero-shot script. By the end, you will have a current baseline you can adapt for your own moderation experiments.

Requirements

Use this setup if you want to run the example as written:

python>=3.10
transformers==5.12.1
torch==2.12.1+cu130 or a current CPU-compatible PyTorch build
kagglehub
Pillow
scikit-learn
matplotlib for visualization

Prerequisites

Basic familiarity with Hugging Face model loading
Comfort reading classification metrics such as precision, recall, and F1
Enough local disk space for the model and the Hateful Memes dataset
A GPU is helpful, but the script below falls back to CPU

Problem
Background
Approach
Example
Failure Modes & Caveats
Practical Guidance
Summary
Next Steps
Resources

Problem

Content moderation is usually discussed as a text classification problem, but real moderation work is often multimodal. Memes mix image context, embedded text, sarcasm, and cultural cues in a way that breaks simple OCR-plus-classifier pipelines.

That is why the Hateful Memes dataset is still a useful stress test. It is not just asking whether a model can read text inside an image. It is asking whether the model can combine image content and language well enough to decide when the meme targets a protected group.

If you want a fast baseline, zero-shot inference is a good place to start. It will not replace a carefully tuned moderation stack, but it tells you quickly whether a general-purpose vision-language model is even in the right neighborhood.

Background

Qwen2.5-VL is a vision-language model, so the workflow is different from a plain text moderation model:

You pass both an image and a prompt.
The processor converts the multimodal input into tensors.
The model generates text, which you normalize into a binary label.

That makes zero-shot moderation simple to prototype, but the details matter. A few small implementation choices can quietly make the result less trustworthy:

Hard-coding a Colab cache path makes the script non-portable.
Forcing tensors onto "cuda" makes the script fail on CPU-only machines.
Treating only one exact output string as positive undercounts the model when it answers with variants like "yes" or "Yes, hateful".

The goal of this rewrite is not to make the task look easy. It is to make the baseline honest and reproducible.

Approach

The updated workflow is deliberately boring:

Download the dataset through kagglehub.
Resolve the validation split without assuming a fixed cache directory.
Load Qwen/Qwen2.5-VL-3B-Instruct with AutoProcessor and Qwen2_5_VLForConditionalGeneration.
Prompt the model to return exactly yes or no.
Normalize the generated text before scoring.
Report accuracy, precision, recall, and F1.

Two practical changes make the example more reusable than the original notebook:

The prompt includes the dataset’s meme text as explicit context, instead of relying only on the model to read the overlaid text from pixels.
The processor handles the image directly, so you do not need an extra qwen-vl-utils dependency for this single-image use case.

Example

The script below keeps the original idea but removes the notebook-only assumptions. It was updated against transformers==5.12.1, and the processor/template path was smoke-tested in the local fiftyone environment with torch==2.12.1+cu130.

from __future__ import annotations

import json
import random
import re
from pathlib import Path
from typing import Final

import kagglehub
import matplotlib.pyplot as plt
import torch
from PIL import Image
from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration


DATASET_REF: Final[str] = "parthplc/facebook-hateful-meme-dataset"
MODEL_NAME: Final[str] = "Qwen/Qwen2.5-VL-3B-Instruct"
EVAL_LIMIT: Final[int] = 64
SEED: Final[int] = 42


def resolve_dataset_dir() -> Path:
    """Download the dataset and return the directory containing the split files."""
    root = Path(kagglehub.dataset_download(DATASET_REF))

    for candidate in (root / "data", root):
        if (candidate / "dev.jsonl").exists():
            return candidate

    raise FileNotFoundError("Could not locate dev.jsonl in the downloaded dataset.")


def load_validation_examples(limit: int | None = EVAL_LIMIT, seed: int = SEED) -> list[dict]:
    """Load a shuffled subset of the validation split."""
    dataset_dir = resolve_dataset_dir()
    dev_path = dataset_dir / "dev.jsonl"

    with dev_path.open() as file_obj:
        records = [json.loads(line) for line in file_obj]

    random.Random(seed).shuffle(records)
    if limit is not None:
        records = records[:limit]

    for record in records:
        record["image_path"] = dataset_dir / record["img"]

    return records


def build_prompt(meme_text: str) -> str:
    """Ask for a strict binary moderation decision."""
    return (
        "You are reviewing a meme for hateful content.\n"
        "Decide whether it attacks, demeans, or expresses hate toward a protected group.\n"
        "Answer with exactly one word: yes or no.\n\n"
        f"Meme text: {meme_text}"
    )


def normalize_yes_no(text: str) -> str:
    """Extract the first yes/no decision from generated text."""
    match = re.search(r"\b(yes|no)\b", text.strip().lower())
    if match is None:
        raise ValueError(f"Model output did not contain a yes/no answer: {text!r}")
    return match.group(1)


def move_to_model_device(batch: dict[str, torch.Tensor], model) -> dict[str, torch.Tensor]:
    """Move processor outputs onto the model's starting device."""
    return {key: value.to(model.device) for key, value in batch.items()}


def predict_label(
    image: Image.Image,
    meme_text: str,
    model: Qwen2_5_VLForConditionalGeneration,
    processor: AutoProcessor,
) -> str:
    """Run one zero-shot moderation prediction."""
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image"},
                {"type": "text", "text": build_prompt(meme_text)},
            ],
        }
    ]

    chat_text = processor.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    inputs = processor(
        text=[chat_text],
        images=[image],
        padding=True,
        return_tensors="pt",
    )
    inputs = move_to_model_device(inputs, model)

    with torch.inference_mode():
        generated_ids = model.generate(
            **inputs,
            max_new_tokens=4,
            do_sample=False,
        )

    trimmed_ids = generated_ids[:, inputs["input_ids"].shape[1] :]
    output_text = processor.batch_decode(
        trimmed_ids,
        skip_special_tokens=True,
        clean_up_tokenization_spaces=False,
    )[0]

    return normalize_yes_no(output_text)


def evaluate_model(
    records: list[dict],
    model: Qwen2_5_VLForConditionalGeneration,
    processor: AutoProcessor,
) -> tuple[list[str], list[str]]:
    """Generate predictions and collect ground-truth labels."""
    predictions: list[str] = []
    answers: list[str] = []

    for index, record in enumerate(records, start=1):
        image = Image.open(record["image_path"]).convert("RGB")
        prediction = predict_label(
            image=image,
            meme_text=record["text"],
            model=model,
            processor=processor,
        )

        predictions.append(prediction)
        answers.append("yes" if record["label"] == 1 else "no")
        print(f"Processed {index}/{len(records)} examples")

    return predictions, answers


def report_metrics(predictions: list[str], answers: list[str]) -> None:
    """Print binary classification metrics."""
    y_true = [1 if answer == "yes" else 0 for answer in answers]
    y_pred = [1 if prediction == "yes" else 0 for prediction in predictions]

    print(f"Accuracy : {accuracy_score(y_true, y_pred):.3f}")
    print(f"Precision: {precision_score(y_true, y_pred):.3f}")
    print(f"Recall   : {recall_score(y_true, y_pred):.3f}")
    print(f"F1 Score : {f1_score(y_true, y_pred):.3f}")


def show_examples(records: list[dict], predictions: list[str], num_images: int = 12) -> None:
    """Visualize a small prediction grid."""
    count = min(num_images, len(records))
    cols = 3
    rows = (count + cols - 1) // cols

    fig, axes = plt.subplots(rows, cols, figsize=(12, 4 * rows))
    axes = axes.flatten() if hasattr(axes, "flatten") else [axes]

    for index in range(count):
        image = Image.open(records[index]["image_path"]).convert("RGB")
        label = "yes" if records[index]["label"] == 1 else "no"

        axes[index].imshow(image)
        axes[index].axis("off")
        axes[index].set_title(f"Label: {label} | Predicted: {predictions[index]}")

    for index in range(count, len(axes)):
        axes[index].axis("off")

    plt.tight_layout()
    plt.show()


def main() -> None:
    device = "cuda" if torch.cuda.is_available() else "cpu"
    dtype = torch.bfloat16 if device == "cuda" else torch.float32

    records = load_validation_examples(limit=EVAL_LIMIT, seed=SEED)
    processor = AutoProcessor.from_pretrained(MODEL_NAME)
    model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
        MODEL_NAME,
        torch_dtype=dtype,
        device_map="auto" if device == "cuda" else None,
    )

    if device == "cpu":
        model = model.to(device)

    model.eval()

    predictions, answers = evaluate_model(records, model, processor)
    report_metrics(predictions, answers)
    show_examples(records, predictions, num_images=12)


if __name__ == "__main__":
    main()

There are four details worth noticing in this version:

resolve_dataset_dir() makes the script portable across machines and cache layouts.
normalize_yes_no() prevents small formatting variations from corrupting the metrics.
device_map="auto" is used only when CUDA is available, which avoids pretending a CPU-only machine can run the same way.
EVAL_LIMIT defaults to 64 so you can get a quick signal before committing to a full validation run.

Failure Modes & Caveats

Zero-shot moderation is useful, but it is not a production policy engine.

Watch for these limits:

The Hateful Memes task is sensitive to context, irony, and cultural knowledge, so false positives and false negatives are both plausible.
A model that answers yes or no correctly most of the time can still be unreliable on borderline cases that matter most in moderation.
CPU inference on a 3B vision-language model is possible in principle but slow in practice.
Including the dataset text field helps reproducibility, but it also changes the task slightly compared with a pure image-only prompt.
Accuracy alone can hide important behavior on an imbalanced moderation dataset, so always inspect precision, recall, and F1 together.

Practical Guidance

If you want to build from this baseline instead of just running it once, start here:

Keep the prompt strict and binary before experimenting with richer moderation labels.
Run a small validation slice first and inspect wrong answers manually.
Save raw model outputs, not just normalized labels, so you can see how often the model hedges.
Compare image-only prompting against image-plus-text prompting if you want to understand how much the transcribed meme text is helping.
Treat zero-shot results as triage evidence, not as a reason to skip a task-specific evaluation plan.

Sanity check

Before trusting the metrics, verify that:

the dataset download resolves a directory containing dev.jsonl,
the processor loads under transformers==5.12.1,
the model returns decodable text for at least a few examples,
and the normalization function handles yes, yes., and short yes/no phrases consistently.

Summary

The real value of this example is not that it turns moderation into a solved problem. It is that it gives you a current, reproducible baseline for testing whether a modern vision-language model can do something useful on a multimodal safety task.

Qwen2.5-VL is a sensible model for that experiment, and transformers==5.12.1 gives you a straightforward API for running it. Once the surrounding notebook clutter is removed, the core workflow is surprisingly small: load the validation split, prompt the model carefully, normalize the answer, and inspect the metrics with skepticism.

Next Steps

Increase EVAL_LIMIT and compare the small-slice behavior with a larger validation pass.
Try alternative prompts that define hateful content more narrowly or more conservatively.
Add per-example error logging so you can audit the hardest false positives and false negatives.