How to Know If Generative AI Fits Your Problem

TL;DR

• Generative AI is powerful, but it is not a universal solvent.
• Start by naming the task: generate, retrieve, classify, rank, transform, extract, summarize, or decide.
• The right approach may be prompting, RAG, fine-tuning, a vision-language model, traditional ML, rules, or no automation.
• AI fit is a design decision, not a vibe.

Abstract

Many AI projects begin with the wrong level of specificity.

The team says:

We want to use generative AI.

That is not enough. Generative AI can support many kinds of work, and each kind of work needs a different architecture, evaluation method, and risk model.

Some problems need a prompt. Some need retrieval. Some need fine-tuning. Some need a vision-language model. Some need a small classifier. Some need rules. Some need a better workflow and no AI at all.

This post gives a practical way to decide.

Table of Contents

• Start With the Task
• Map Task to Approach
• The Decision Tree
• Common Wrong Turns
• Examples
• Summary
• Next Steps

Start With The Task

Before choosing a model, name the task.

Useful task categories include:

• generate
• retrieve
• classify
• rank
• transform
• extract
• summarize
• decide

Each category points to a different solution shape.

If the task is “generate,” you may need an LLM, image model, or multimodal model.

If the task is “retrieve,” you may need search, embeddings, metadata filters, reranking, or RAG.

If the task is “classify,” you may need a small classifier, an LLM with structured output, or a human review queue.

If the task is “rank,” you may need embeddings, cross-encoders, preference data, or scoring rules. The pattern behind caption by consensus is a good example: the model helps choose between candidates rather than generate from scratch.

If the task is “decide,” slow down. Decision workflows need risk analysis, review, accountability, and evaluation.

Map Task To Approach

Once the task is clear, choose the simplest useful approach.

Prompting

Use prompting when the task is general, context is small, and the model already has enough capability.

Good for:

• drafting
• rewriting
• summarizing short inputs
• generating structured first passes
• brainstorming

RAG

Use RAG when the model needs private, current, or source-specific knowledge.

Good for:

• documentation assistants
• internal knowledge search
• policy Q&A
• customer support
• research assistants

Fine-Tuning

Use fine-tuning when you need the model to learn a pattern of behavior, style, format, or domain-specific task that prompting and retrieval cannot reliably provide.

Good for:

• instruction following in a narrow domain
• structured outputs
• domain style
• repeated task behavior

See from prompts to practice for a practical instruction-tuning workflow.

Vision-Language Models

Use VLMs when the task requires understanding images, documents, diagrams, screenshots, charts, or video frames.

Good for:

• visual question answering
• inspection
• content moderation
• document understanding
• visual grounding

Traditional ML

Use traditional ML when the task is structured, measurable, and does not require generation.

Good for:

• prediction
• classification
• anomaly detection
• recommendations

Rules

Use rules when the workflow is deterministic and stable.

Good for:

• known categories
• exact policies
• validation
• routing
• compliance checks with clear logic

The Decision Tree

Ask these questions in order.

1. Is generation needed?

If the system needs to create a new artifact, generative AI may fit. If it only needs to choose from known options, consider rules, classifiers, or ranking.

2. Is private or changing knowledge needed?

If yes, consider retrieval before fine-tuning. A model does not magically know your current docs, policies, tickets, or customer data.

3. Is domain behavior needed?

If the model must follow a specialized format or repeated behavior, fine-tuning may help after a baseline exists.

4. Is accuracy measurable?

If you cannot evaluate the output, the project is not ready. You need a rubric, test set, human review process, or measurable proxy.

5. Is human review available?

If the task is high-risk, human review may be the difference between a useful assistant and an unsafe product.

6. Is latency important?

If the workflow requires fast responses, a large model or multi-step agent may be too slow. Smaller models, caching, routing, or traditional automation may be better.

7. Would a rules-based system work?

If rules work, use rules. You can always add AI where the rules break down.

Want help choosing the right AI approach? We help teams pick the smallest system that can do the real job.

Common Wrong Turns

The first wrong turn is fine-tuning when RAG is enough.

If the model needs access to changing documents, fine-tuning is usually the wrong first move. Retrieval keeps knowledge updateable.

The second wrong turn is RAG when data quality is the issue.

Retrieval cannot fix stale docs, contradictory policies, missing metadata, or unclear ownership.

The third wrong turn is using LLMs when rules are enough.

If the workflow is deterministic, a model may add cost and uncertainty without adding value.

The fourth wrong turn is using multimodal models when OCR plus rules would work.

Vision-language models are powerful, but not every document workflow needs one.

The fifth wrong turn is building agents when a workflow would do.

Agents are useful when the system must plan and act across tools. They are unnecessary when the steps are known.

Examples

A legal document Q&A system likely needs RAG, citations, access controls, human review, and strong evals.

A support triage system may need classification, retrieval, draft generation, and agent review.

An image inspection workflow may need a vision-language model, region grounding, review UI, and domain-specific eval cases.

An internal research assistant may need retrieval, summarization, citations, and feedback capture.

A content moderation workflow may need classification, policy grounding, human escalation, and audit logs. For a multimodal example, see moderating memes with Qwen2.5-VL.

Summary

Generative AI fit starts with task fit.

Name the task first. Then choose the approach. The right answer might be prompting, RAG, fine-tuning, VLMs, traditional ML, rules, or no automation.

AI fit is a design decision, not a vibe.

Next Steps

Write your AI idea as a task:

We need to [generate/retrieve/classify/rank/transform/extract/summarize/decide] for [user] using [context] with [quality bar].

If you cannot fill that in, the problem is not ready for architecture.

Need help turning an AI idea into a working system? We build, break, and explain AI. Then we hand it back working.