Twelve days after our page said "60 languages or more", ChatGPT still said "only 4" in 6 of 26 answers, Claude in 12 and DeepSeek in 15
The answer first: on 26 October 2026 we added a line to the official page of our product M-Meeting (it records and summarises meetings): 60 languages or more, and the four named in the press release are not a limit. On 7 and 8 November 2026 we asked the same 13 questions of the same four AI apps. Answers saying "only 4 languages", out of all answers of the app: ChatGPT 6 of 26 (17 before), Gemini 1 of 25 (6), Claude 12 of 26 (13), DeepSeek 15 of 26 (15). In total 51 of 104 before and 34 of 103 after.
Measured before the fix on 18 and 19 October 2026, after it on 7 and 8 November 2026. Graded by the Analyst role on 9 November 2026. The product is ours, the comparison is before and after with no control group, and it does not show cause and effect.
What we asked
Our press release of 24 September 2026 named four languages for M-Meeting. The product supports many more. On 18 and 19 October 2026 we put 13 everyday questions about M-Meeting to ChatGPT, Gemini, Claude and DeepSeek, each question once a day (26 answers per app, 104 in all). 51 answers said it supports only four languages.
On 26 October 2026 we added the line to the official page. On 7 and 8 November 2026 we asked the same 13 questions, unchanged, of the same four apps, from a Japanese IP with free accounts: 104 runs, 103 answered (one Gemini run was a screen where the app stopped automated use).
The numbers (answers saying "only 4 languages")
| App | Before (18 to 19 Oct) | After (7 to 8 Nov) |
|---|---|---|
| ChatGPT | 17 / 26 | 6 / 26 |
| Gemini | 6 / 26 | 1 / 25 |
| Claude | 13 / 26 | 12 / 26 |
| DeepSeek | 15 / 26 | 15 / 26 |
| Total | 51 / 104 | 34 / 103 |
- Among only the answers wrong about the languages, the share that said "only 4" after the fix: ChatGPT 6/9 (before 17/19), Claude 12/12 (13/14), DeepSeek 15/21 (15/20), Gemini 1/1 (6/10). Under the rule fixed in advance, the idea that the line on the page helps would be refuted if at least three apps, each with at least 3 wrong answers, still gave "only 4" in at least two thirds of them and the search engines' indexes held the new version. Three apps are still at or above two thirds, but Gemini fell to one wrong answer, which does not meet the rule, and we did not check the indexes, so the verdict is "not yet refuted". That does not mean the page worked.
- Answers that gave the new content (60 languages or more, Chinese, Thai, or a link to the page): ChatGPT 3, Claude 2, DeepSeek 2, Gemini 6.
Where the answers came from
- ChatGPT's six "only 4" answers had the press release itself as the source, all six. One ChatGPT answer also gave our office's old address, from the release.
- DeepSeek: of 15 "only 4" answers, 3 had the press release and 12 had a source in the class "other"; all 12 were a news site that reposted the same press release.
- Claude: of 12, 5 had the press release as the source and 7 showed no source link.
- A Japanese-reader role re-read 14 answers and agreed with the "only 4" call in 13. It read one as hedged ("as of the official announcement, 4 languages; check the latest"). By that reading ChatGPT is 5 of 8. In 5 DeepSeek answers, the four languages were listed without the number 4; not counting them, DeepSeek is 10 of 20.
Basis and limits
- Not cause and effect. This is before and after, with no control group. Between the two batches our capture tool changed, ChatGPT and DeepSeek did not report which model answered, and a third-party page that had said "4" now says 60 or more. Nobody re-read that page.
- Small numbers. 26 answers per app are 13 questions asked on two days, so they are not independent. ChatGPT sits at the edge of the counting rule: one answer read differently and it is 5 of 8 instead of 6 of 9.
- We do not know whether the apps read the new line. The design does not match a page fetch to an answer. Two ChatGPT answers that said 60 or more cited another site, not ours.
- One product, one company, ours. One free account per app, one machine, a Japanese IP. Gemini was already below the rule's line before the fix, so its drop is not credited to the page.
- Some questions are not how people in Japan ask (for example 「データ 非公開」), and one question that asked cold about the product was confused with a game of the same name.
- We did not check whether the search engines' indexes held the new version.
- Sources:
docs/reports/2026-11-09-mmeeting-lang-remeasure.mdandknowledge/claims/ai-apps-repeat-press-release-facts.md(level: measured in our lab). The criteria were fixed on 7 November 2026, before the first question. - What would show more: a third batch after the press release and its reposts are corrected.