How to Fix Inaccurate AI Product Recommendations
Quick answer
Fix inaccurate AI product recommendations by tracing each bad result to its source. Capture the exact shopper question, define the acceptable product set, compare it with the actual answer, classify the failure, correct the smallest source that caused it, and retest the same decision.
- Start with evidence: use a real session, not a hypothetical prompt.
- Classify the failure: separate product facts, availability, relevance, instructions, knowledge, and handoff.
- Fix the source: update the catalog, PDP, knowledge, custom response, recommendation instruction, or support path.
- Retest nearby cases: check the original question, a paraphrase, an unavailable variant, and one edge case.
- Monitor the pattern: track recurring questions and unresolved gaps separately from recommendation quality.
The operating rule: correct the source of the bad recommendation, then rerun the shopper decision that exposed it.
Why this matters
A shopper asks for a fragrance-free moisturizer under $50. The assistant returns a fragranced product that costs $62. Tightening the prompt might hide the symptom, but it will not repair stale prices, incomplete product attributes, weak recommendation logic, or missing policy context.
A useful QA process shows the team exactly where to make the correction. It also creates a record of what the shopper wanted, what the assistant returned, who owns the source, and whether the fix survived retesting.
1. Start with a real bad recommendation
Begin with the conversation that exposed the problem. Copy the shopper's exact wording and capture the products or answer the assistant returned. Summaries such as “recommendations seem off” are too vague to diagnose.
Write the expected result before changing anything. The expected result can be one product, a short acceptable set, or a safe handoff. A merchandising team rarely needs one mathematically perfect SKU; it needs results that respect the shopper's real constraints.
| QA field | Example |
|---|---|
| Shopper question | “I need a fragrance-free moisturizer under $50 for sensitive skin.” |
| Expected result | Only fragrance-free moisturizers priced below $50 with clear sensitive-skin context. |
| Actual result | A $62 fragranced moisturizer appeared first. |
| Open question | Is the product data wrong, did retrieval ignore the constraints, or did the assistant infer an unsupported match? |
Preserve the session or message reference with the record. If the same issue appears again, the team can compare the two failures instead of starting a fresh investigation.
2. Classify the failure before you edit
Most inaccurate recommendations fall into six categories. The category determines the source and owner of the fix.
| Failure type | What the shopper sees | What to verify first | Likely owner |
|---|---|---|---|
| Product fact | Wrong material, ingredient, compatibility, size detail, or price | Catalog fields, variant data, PDP copy, and product-specific knowledge | Ecommerce or merchandising |
| Availability | An unavailable product or variant is presented as buyable | Inventory tracking, variant availability, and storefront state | Ecommerce operations |
| Relevance | The product is real, but it does not satisfy the shopper's intent | Seed product, shopper constraints, recommendation mode, and candidate quality | Merchandising |
| Merchant instruction | The answer conflicts with a current assortment or merchandising rule | Recommendation instructions and whether they are broad, current, and compatible | Merchandising |
| Knowledge or policy | The product may fit, but the surrounding answer about shipping, returns, care, or use is wrong | Approved knowledge files, URLs, custom responses, and source freshness | Support or ecommerce |
| Handoff | The assistant continues guessing when a person should take over | Support boundaries, contact destination, and handoff behavior | Support |
One session can contain more than one failure. A recommended product might be relevant, while the answer gives an old price and promises a delivery date the assistant cannot verify. Record each failure separately so one correction does not create a false pass.
3. Trace the problem to the right source
Use a source ladder. Check the most authoritative product source first, then move outward to recommendation behavior and support policy.
Catalog, PDP, and variant facts
Price, options, product attributes, imagery, and availability should come from current product data. If the source says a garment is linen when it is cotton, an assistant-level correction leaves the bad fact available to every other surface.
Approved knowledge and custom responses
Policies, size guidance, product-use instructions, and recurring questions may live outside the core catalog. Add the approved source as a knowledge file or URL. For a repeatable question that needs a controlled answer, use a custom response and scope it to the relevant products when appropriate.
Keep ownership visible. A returns answer belongs to support or operations. An ingredient claim belongs to product or merchandising. The team that approves the fact should approve the correction.
Recommendation engine and merchant instructions
Recommendation behavior is the right layer when the product facts are correct but the candidate set is weak. Check which engine powers the suggestions, which recommendation mode is being used, and whether merchant instructions are helping or competing with the shopper's constraints.
Use instructions for durable merchandising guidance, such as prioritizing relevant bestsellers or matching accessories. A growing list of SKU exceptions becomes hard to reason about and can mask a catalog or taxonomy problem.
Support boundaries
Some requests need a person: an unusual compatibility question, an account-specific exception, a safety concern, or a policy decision outside the approved knowledge. Define the handoff path and test it as part of recommendation QA.
4. Make the smallest supported correction
A correction should be narrow enough to explain. Use these rules to keep the change reviewable:
- Repair facts where they originate. Update the catalog, variant, PDP, or approved knowledge source that contains the incorrect information.
- Use a custom response for a stable question-and-answer pattern. Write the approved answer in plain language and apply product scope when the answer is product-specific.
- Keep recommendation instructions short and general. State the merchandising priority without trying to encode the entire catalog in one instruction field.
- Remove expired guidance. Seasonal priorities, promotions, and assortment rules need an owner and review date.
- Escalate unsupported decisions. Give the shopper a clear support path instead of asking the assistant to improvise.
Change note: “Updated the product source to mark the item fragrance-free; added a product-scoped approved answer for sensitive-skin questions; no recommendation instruction changed.”
That note gives the reviewer a testable hypothesis. If the result remains wrong, the team knows which layer has already been cleared.
5. Retest the recommendation like a merchandiser
Rerun the decision, not only the original sentence. A useful retest checks whether the fix holds when the shopper changes wording or when an important constraint changes.
| Retest | Example | Pass signal |
|---|---|---|
| Original question | Repeat the exact shopper wording. | The result stays inside the approved product set and factual answer. |
| Natural paraphrase | “Sensitive skin moisturizer without fragrance, max $50.” | The same constraints are respected without relying on one phrase. |
| Availability case | Make the preferred variant unavailable or test a known unavailable option. | The assistant does not present the unavailable choice as ready to buy. |
| Boundary case | Ask for an item just outside the price, size, use, or compatibility rule. | The assistant explains the limitation or offers a valid alternative. |
| Support case | Ask for an exception that requires human approval. | The handoff path activates without an invented answer. |
Record the retest date, storefront, source version, result, and reviewer. A manual record is enough to start. The important part is preserving the link between the failure, the change, and the observed result.
6. Monitor knowledge gaps and hand off what the agent should not answer
Recommendation quality and knowledge coverage need separate checks. An assistant can confidently recommend the wrong product without creating a knowledge gap. It can also correctly report that it lacks an answer, which is a knowledge gap but safer than a fabricated recommendation.
Use three views together:
- Question themes show which topics repeat across shopper conversations.
- Session transcripts show the exact question, answer, product cards, and follow-up context.
- Knowledge Gaps collect questions the agent could not answer and let the team add a resolution.
Review unresolved gaps on a regular cadence, then sample sessions for questions that produced an answer but may still have produced a poor recommendation. Track support handoffs separately so a safe escalation is not counted as a recommendation failure.
7. Use this question-to-source audit template
Copy this template into a spreadsheet, QA document, or merchandising review. One row should describe one shopper decision and one corrective cycle.
| Field | What to record |
|---|---|
| Shopper question | Exact wording from the session |
| Intent | Discovery, comparison, compatibility, availability, policy, or support |
| Expected result | Approved product, acceptable set, answer, or handoff |
| Actual result | Products and answer the shopper received |
| Failure type | Fact, availability, relevance, instruction, knowledge, or handoff |
| Source to fix | Catalog/PDP, knowledge URL or file, custom response, recommendation instruction, or support path |
| Owner | Merchandising, ecommerce, support, or product |
| Correction | Smallest approved source change |
| Retest set | Original, paraphrase, availability case, edge case, and handoff case |
| Status | Open, fixed, retested, escalated, or monitoring |
Keep the expected result explicit. “Looks better” is not a pass criterion. “Returns only fragrance-free products under $50, avoids unavailable variants, and hands unsupported skin-safety questions to support” is.
8. How Tolstoy supports the recommendation QA loop
Tolstoy AI Shopper gives ecommerce teams the product and conversation surfaces needed to run this workflow:
- review question themes and individual shopper session transcripts;
- find unresolved knowledge gaps and add approved resolutions;
- add knowledge from supported files or URLs;
- save approved custom responses, including product-specific answers;
- choose recommendation behavior and add merchant guidance when using the Smart engine;
- send requests to a contact destination or storefront support event when a person should take over.
The operator still owns the judgment. Tolstoy makes the evidence, source controls, and retest surface easier to connect; merchandising and support teams decide what is correct for the catalog and customer.
9. Final takeaway
Inaccurate AI product recommendations become manageable when every failure has a question, expected result, source, owner, correction, and retest. That record turns a vague quality complaint into a merchandising task the team can verify.
Start with five real shopper sessions. Classify each failure, repair the smallest authoritative source, rerun the decision, and keep the result. The first goal is a QA loop your team trusts enough to repeat.
Ready to review product recommendations with real conversation context? Start free with Tolstoy.
Frequently asked questions
What causes inaccurate AI product recommendations?
Common causes include incomplete product data, stale availability, weak recommendation relevance, missing brand or policy knowledge, merchant instructions that conflict with shopper intent, and requests the assistant should hand to human support. Classify the failure before changing the source.
How should an ecommerce team test AI product recommendations?
Start with a real shopper question, record the expected and actual result, identify the source that needs correction, then retest the original wording, a paraphrase, an availability case, and one edge case. Store the evidence so the same failure can be recognized later.
What data improves ecommerce AI recommendations?
Accurate catalog and variant data, clear PDP details, current inventory, approved product and policy knowledge, and real shopper conversation evidence all help. The right fix depends on whether the failure is factual, relevance-based, instructional, or a support issue.
Does Tolstoy automatically fix a bad recommendation?
No. Tolstoy gives teams product and conversation surfaces for reviewing sessions, finding knowledge gaps, adding approved knowledge or custom responses, configuring recommendation behavior, and setting support handoff. A merchandiser or operator should still approve the source change and retest it.