Do AI tour guides make things up? How Strolio checks its stories

Short answer: yes, sometimes. And the places where an AI guide is most likely to be wrong are the ones a walking guide spends most of its time on: the ordinary building on the corner, not the cathedral.

We build Strolio, an AI walking tour app, so we have a stake in this question. Below is what has gone wrong for real travellers, why it happens, what we measured in our own app, and what still isn't solved.

Two places that don't exist

In September 2025, BBC Travel reported on a trek operator in rural Peru who met two tourists planning to hike alone to the "Sacred Canyon of Humantay". An AI chatbot had recommended it. There is no such canyon. "They showed me the screenshot, confidently written and full of vivid adjectives, but it was not true," the operator, Miguel Angel Gongora Meza, told the BBC. According to the report, one of the tourists had paid nearly $160 to reach a rural road with no guide and no destination.

The same article describes a couple stranded at the top of Mount Misen in Japan after ChatGPT gave them the wrong closing time for the ropeway down.

In January 2026, ABC News in Australia reported that a tour company's website had published an AI-written post sending visitors to "Weldborough Hot Springs" in north-east Tasmania. The river there is, in the words of the local publican, "freezing cold, it's definitely, definitely not a hot spring". Visitors kept turning up anyway. The owner's verdict: "Our AI has messed up completely."

Neither case involved a walking tour app. But a walking guide uses the same technology, and it talks to you when you're standing in the street with no reason to doubt it.

Why AI guides get things wrong

A large language model, the kind of AI behind chatbots, doesn't look facts up. It writes the most likely next words. MIT Sloan's teaching guide puts it plainly: these models "function like advanced autocomplete tools", and "their goal is to generate plausible content, not to verify its truth."

That's why a wrong answer sounds exactly like a right one. As Carnegie Mellon's Rayid Ghani told the BBC, AI programs present their made-up answers and their factual ones the same way, so it's hard for users to tell them apart.

Why your street is riskier than the Sagrada Família

Here is the part that matters most for walking tours. Models are much better at famous things than obscure ones.

A 2023 study by Mallen and colleagues, presented at ACL, tested ten models on 14,000 questions and found that they "struggle with less popular factual knowledge", and that making models bigger doesn't really fix it. A second 2023 study, by Kandpal and colleagues at ICML, found that a model's ability to answer a factual question tracks how many documents about it appeared in its training data. Rare facts would need models "many orders of magnitude" larger.

A world-famous basilica appears in millions of documents. The farmhouse that became your local library appears in a handful. A walking tour is mostly made of the second kind of place.

What we measured

We saw the same pattern in Strolio. We took 12 places in Barcelona and generated stories for each with two versions of our pipeline, keeping everything else the same. The first was our earlier approach: give the model the place and let it write. The second collected documented source passages about each place first and wrote from those. We then checked every claim in every story against the source passages.

Test run on 12 places in Barcelona, comparing our earlier pipeline with the evidence-first pipeline. Details on request.

The mistakes were not spread evenly. Across the four well-known landmarks in the test, the earlier pipeline made 1 contradiction. Across the eight ordinary neighbourhood buildings, it made 6. That is the research above, showing up in a real app.

The "precise claims the sources don't cover" row matters too. Those are exact dates, names and numbers the model added that the sources neither confirm nor deny. They may be true. We just can't show that they are, so we'd rather not say them.

How Strolio checks a story

1. Facts first, then the story. Before writing about a place, Strolio collects documented facts from archives, local publishers and open data such as OpenStreetMap and Wikidata. Each source is stored with its publisher, licence and the date we retrieved it.
2. Write from the evidence. The story is written from those facts, not from what the model happens to remember.
3. Label what kind of story it is. Every story is marked as a fact, a local story or a legend. A legend can be a great story. It just shouldn't be told as history.
4. Show the sources. When a story has sources, the app lists them, so you can check for yourself.
5. Stay short when evidence is thin. If we can't find a reliable source, the story stays short instead of getting inventive.

What checked sources don't fix

We don't want to overclaim, so here are the limits.

• Grounding reduces errors; it doesn't end them. A 2024 Stanford study of AI legal research tools that look up sources found they made fewer errors than a general chatbot, but still hallucinated "between 17% and 33% of the time". Looking things up helps a lot. It is not magic.
• Our test is small. Twelve places in one city is enough to show a clear difference, not enough to promise a rate for every city.
• Sources can be wrong. We can only be as accurate as the archives and databases we rely on.
• Some places have almost nothing written about them. There, you'll hear a short story or none, rather than a long confident one.

Strolio is in beta. If a story gets something wrong, the feedback form on our homepage goes straight to the team, and that's exactly the report we want.

How to check any AI guide

Whichever app you use, a few habits help.

• Ask where a fact comes from. A guide that can't point to a source is guessing.
• Be wary of precise details about obscure places. An exact year for a small chapel is where invention hides.
• Check anything practical on the official site. Opening hours, last trains and ropeway times change, and getting them wrong can leave you stuck.
• Notice whether legends are labelled. Good guides tell you when a story is a story.

Sources

1. Lynn Brown, "The perils of letting AI plan your next trip", BBC Travel, 29 September 2025. bbc.com
2. Josh Duggan, "Tour website's AI sends visitors to Tasmanian sites that do not exist", ABC News, 22 January 2026. abc.net.au
3. "When AI Gets It Wrong: Addressing AI Hallucinations and Bias", MIT Sloan Teaching & Learning Technologies. mitsloanedtech.mit.edu
4. Alex Mallen et al., "When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories", ACL 2023. arXiv:2212.10511
5. Nikhil Kandpal et al., "Large Language Models Struggle to Learn Long-Tail Knowledge", ICML 2023. arXiv:2211.08411
6. Patrick Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", NeurIPS 2020. arXiv:2005.11401
7. Varun Magesh et al., "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools", Stanford, 2024. arXiv:2405.20362
8. Strolio internal evaluation, 12 places in Barcelona, 2026. Details on request.