Dan Luu has published an extensive comparison of six search engines and ChatGPT, testing them on a set of common queries including “download youtube videos,” “ad blocker,” and “vancouver snow forecast winter 2023.” The results are stark: Marginalia, a search engine built by a single person, does better than Google and Bing on several queries, while ChatGPT also does well on some queries. Google and Bing frequently return spam, scams, and low-quality SEO content, while Marginalia does better on several queries, although it sometimes lacks coverage. Mwmbl, which lets users edit result rankings, is rated Bad on every query in Luu’s table.

Luu notes that the ad-based business model creates misaligned incentives, citing Google’s own founders’ concerns from their original PageRank paper. He demonstrates how scam sites and ads for gambling appear on addiction management searches, and how search engines prioritize recent YouTube videos of low quality. The post argues that traditional search engines are now so full of spam that users have developed workarounds without realizing it, and that small projects like Marginalia show that better results are possible with sufficient curation.