An AI writing system you can trust: start from facts, check, and let a person decide

The first two chapters said Google looks at value and accuracy, and that AI can be confidently wrong. This chapter joins the two into a system: if you use AI to write regularly, how do you design it so what you publish is accurate and useful? The examples come from the system this site actually runs.

3.1 Start from facts, not from the AI’s memory

The most common mistake is telling the AI “write an article about X”, so it writes from what it remembers, which may be old or wrong.

A safer approach has three steps:

1. Collect the facts from primary sources, each with its link and a quote from that page.
2. Decide which facts can be used, with rules set in advance.
3. Have the AI write from that table only, then check that every claim in the article traces back to it.

On this site, the decision rules in step 2 are fixed code, not an AI, deliberately. If an AI made the decision, the layer meant to be the referee would be as unreliable as the sources it is refereeing. An AI is used only to place sites the system does not know into a trust tier, and if that fails the site goes into the lowest tier. Our rules:

Rule	Why
A primary source — the company in the story, a paper, an official document — beats everything else	It knows the matter first-hand
Without a primary source, a figure needs at least 2 independent sources	The sources must be on different sites (this does not yet catch several sites copying one story)
Sources with no editorial layer may be leads, never cited alone	Nobody knows where they got it
Conflicting values from lower-tier sources are recorded but not used	So a reviewer can trace them later

A small site does not need a system this size. A spreadsheet with “fact, source, quote” filled in before the AI writes gets you most of the benefit.

3.2 Checks: let machines check what machines can check for certain

Some mistakes a machine can catch for certain without any AI; for others it can only flag candidates, and the meaning has to be read. Check before every publish:

Check	What it looks at	Can a machine decide alone?
Numbers	Every number in the article must be in the fact table	Yes
Dates	Does the weekday match the date? Is there a date that doesn’t exist, such as 30 February?	Yes
Links	Every link opens	Yes
Self-contradiction	The same figure should not differ between two paragraphs, or between the Thai and English versions	No: it flags candidates, then someone must read whether they describe the same thing (we use an AI to help read)
Duplicate topics	A new topic should not repeat an existing article, so you don’t end up with several pages about the same thing	No: it finds similar topics, then someone must decide whether they really repeat (we use an AI to help)

Our date check exists because dates were the error that kept reaching readers. The duplicate-topic check keeps us out of chapter 1’s scaled content abuse trap of several pages on one subject.

A check should report a problem, not quietly fix it. A check that corrects things automatically can turn something right into something wrong without anyone noticing.

3.3 The reviewer must not be the writer

Writers miss their own mistakes, and so does AI. What we do is have another person, or another AI that did not write the draft, open the primary pages and check claim by claim — told to find errors, not to confirm it is right.

What a reviewer should look at:

• does every number and quote match the source page;
• does any sentence claim more than the source says, such as “will” where the source says “may”;
• do the two language versions say the same things;
• is anything unfair to a person or company mentioned.

3.4 A lesson from KoishiAI: three times our own system got it wrong

Revising made it worse. We used to have the AI revise drafts that failed a check several times, telling it to “fix only these issues”. The quality score fell every round, because each revision rewrote the article and broke unrelated parts. Now a repair is kept only if it scores better; otherwise the original draft stays.

Our own checker hid an error. While comparing baht-text libraries, we wrote a checker that treated any difference between “เอ็ด” and “หนึ่ง” as a matter of style. It hid a real bug — “เอ็ดสตางค์” — until we re-examined the checker’s rule and found it. Checkers need their own tests.

A check based on arithmetic alone did not work. We tried to catch contradicting figures with purely arithmetic rules. Every finding it reported was wrong, because two figures that look contradictory usually describe different things. Now arithmetic only proposes, and reading the meaning decides.

For drafts that fail a check, the system fixes only certain kinds itself: it renames a duplicate URL slug, and revises a draft whose quality score is below the bar, keeping that revision only if it scores better. Anything still failing is held for the site owner to publish, rewrite or discard.

3.5 Tell readers the truth

Google recommends saying how content was made (chapter 1). What matters is that it is true. If your system mainly checks by machine, do not write “reviewed by our editors”; write that only if a person really reads every article. A good disclosure says three things:

• what AI does — drafting, translating, summarising;
• what is checked — against primary sources, numbers;
• who is responsible, and where to report errors.

We got this wrong ourselves. While writing this chapter we found that three disclosures on our site disagreed: one said checking was automated, two said an editor fact-checked before publishing. The truth is that an automated pipeline checks facts against the sources, and the site owner reads every article for clarity. We corrected all three to say so.

3.6 After this chapter

• Before the AI writes your next piece, fill in a “fact, source, quote” table first.
• Pick at least two checks from table 3.2 you can start now, such as numbers and links.
• Have someone who did not write it review it, told to find errors.
• Read the AI disclosure on your site and ask whether every word matches what you actually do.

The next chapter turns to the other side: how AI search chooses which sites to show, and what you can control.

Sources

• Google Search Central — Creating helpful, reliable, people-first content
• Google Search Central — Using generative AI content
• Google Search Central — Spam policies for Google web search