DuckDuckGo dabbles with AI search

Privacy-focused search engine DuckDuckGo has followed Microsoft and Google to become the latest veteran search player to dip its mouth into the generative AI trend. searches for users.

DDG says it uses natural language technology from ChatGPT creator OpenAI and Anthropic, an AI startup founded by ex-OpenAI employees, to enable natural language summarization capability, combined with its own active indexing of Wikipedia and other reference sites it uses to find sources answers (the encyclopedia Britannia is another source it mentions).

Founder Gabe Weinberg tells TechCrunch that the sources it uses for DuckAssist are currently “99%+ Wikipedia.” But he notes that the company is “experimenting how incorporating other resources might work and when they should be used” — suggesting it could try to tailor sourcing to the context of the query (i.e. a current news related search can be better answered by letting DuckAssist pull information from trusted news media). So it remains to be seen how DDG will develop the feature – and whether it could try to forge partnerships with reference sites, for example.

At launch, DuckAssist is only available through DDG’s apps and browser extensions, but the company says it plans to roll it out to all search users in the coming weeks. The beta feature is free to use and does not require the user to be logged in to access it. It is only available in English for now.

According to Weinberg, the AI ​​models that DDG (“currently”) uses to enable the natural language summary are: OpenAI’s Davinci model and Anthropic’s Claude model. He also notes that DDG is “experimenting” with the new Turbo model OpenAI recently announced.

It’s worth noting that DDG’s search engine already has an Instant Answers feature that activates for certain types of searches and also displays answers right above the usual list of links. (Example scenarios include if you ask the search engine to add up basic calculations, display a calendar for the current month, or ask for factual snippets of information.)

However, DDG says adding Generative AI Summarization has made it possible to expand how many questions can be directly answered this way – calling the addition of Generative AI into the mix a “fully integrated direct answer” here.

“The two main advantages over other direct answers are that DuckAssist answers respond more directly to user queries and that DuckAssist can answer significantly more questions,” Weinberg says. “The generative AI behind DuckAssist generates new text for a given query, with standard instant answers generally being quotes. This way, DuckAssist can respond more directly to the query, quickly surface information buried in articles, and synthesize information from multiple Wikipedia excerpts. As a result, it can answer a wider range of questions.”

DuckAssist is intended to help search engine users find factual information faster – that’s why it only appears as an option when the technology assesses that it can help with a specific query.

“If you search for a question in a DuckDuckGo app or browser extension, and DuckAssist thinks it can find an answer from Wikipedia, you may see a magic wand icon and an ‘Ask Me’ button at the top of your search results,” explains DDG.

If another DDG user has asked for an answer before, the company says it will appear automatically, but also notes that users can choose to disable Instant Answers (including DuckAssist) in the settings if they prefer not be exposed to AI-generated summaries.

Weinberg says the feature works by using AI to generate new natural language responses “based on specific/relevant sections of Wikipedia articles” that DDG provides through its own scan of sources. (He points out that DDG uses its own indexing technology “to identify the relevant bits of text from Wikipedia and then ask the models to format answers in a way that directly responds to the question”.)

Accuracy is one of the main concerns in generative AI applications – as the technology can tend to fabricate information and yet automated output presented in a natural language wrapper can sound very authoritative despite not being fact-checked.

On this, Weinberg says DuckAssist is designed to increase the likelihood that it will give a correct answer, while also informing users that the answer is automated – pointing them to the reference sources where they can do their own fact-checking (i.e. if it turns out that DuckAssist is more of a ‘donkey’ than an assistant).

“The task of a search engine is to quickly retrieve reliable information. We’ve designed DuckAssist in a way that takes advantage of what natural language technology does right, while trying to increase the chances of it giving a correct answer when it appears in search results. We did that by deliberately limiting the sources from which DuckAssist summarizes,” he says. “For now, DuckAssist is only pulling answers from Wikipedia and a handful of related resources, such as Britannica. This greatly reduces the chances of DuckAssist generating incorrect information or ‘hallucinating’, where the AI ​​tool makes up random information.

“Nevertheless, we know it won’t be right 100% of the time – if we don’t deliver the most relevant text [to the AI] to summarize, for example, or if Wikipedia itself contains errors. In any case, we mark each answer as not independently verified for accuracy and provide a link to the most relevant Wikipedia article for more information.”

On the privacy front, DDG promises that this AI-enabled search will be anonymous – further emphasizing, in line with its main privacy pledge, that no data will be shared with any of the third parties it works with to leverage the generative AI capability. his search engine integrate. (In the blog post, Weinberg also notes that users’ anonymous search queries are not used to train its vendors’ AI models.)

It asks users to provide feedback on the quality of the DuckAssist summaries – via a feedback link that appears next to all DuckAssist answers, as part of its approach to the generative AI accuracy problem – and says that this feedback is also anonymous , with user reports also only sent to DDG itself, not third parties.

While the launch of DuckAssist means more automated answers will inevitably pop up in response to user queries, DDG notes that the feature will still only be available for a minority of searches, as it’s only intended to help with relatively simple to ask. Phrasing a query as a question makes the feature more likely to appear in search results, it adds.

“Generative AI is hitting the world of search and browsing at scale,” Weinberg writes in a blog post announcing what he says is “the first in a series of generative AI-assisted features we hope to roll out in the coming months.” “At DuckDuckGo we tried to understand the difference between what it could do well in the future and what it can do well straight away. But however we decide to use this new technology, we want it to add significant value to our personalized search and browsing experience.”

More AI-powered search and browsing features are also in the works from DDG, with additional AI-related news planned for the coming months. (Although he won’t be attracted to whatever else it’s cooking – he’s just saying “stay tuned!”.)

Here’s a clip of the DuckAssist feature in action for a query that asks “is antarctica a country” – showing the user being promoted to activate DuckAssist (“ask”) and then a summarized response in natural language will appear above the source (Wikipedia) and a reference to the section of the article from which it originates:

In its blog post, DDG explains that it chose Wikipedia as the primary source for DuckAssist, as the crowdsourced encyclopedia is already the primary source for its existing Instant Answers feature and, while not foolproof, rates it as “relatively reliable across a wide range to subjects”. ”.

It also points out that Wikipedia has the added benefit of being a public source “with a transparent editorial process that cites all sources used in an article, you can easily trace exactly where the information came from”.

Plus, of course, Wikipedia is constantly being updated – expanding the questions DuckAssist can meaningfully respond to. That said, there’s still a lag in the knowledge graph – as DDG notes that “at this point” the DuckAssist Wikipedia index may be up to “a few weeks old.” (But it says it has plans to “make it even more recent”, in addition to adding more resources “soon”.)

It’s worth noting that DDG’s current generation of Instant Answers isn’t always right either.

At the time of writing, a DDG search for “people in space” generated a neat stack of ten cards of astronauts suggested to be currently in orbit – but a photo of the American astronaut was found twice. Kayla Barron shown; once on her own map and once (incorrectly) linked to the map of the German astronaut Matthias Maurer. So error-prone tech shortcuts are nothing new.

Still, the power of generative AI to automate many more interactions – and in this case respond to many more types of queries – could create greater skews in the information landscape by significantly expanding the ability of platforms to apply such shortcuts, which boosts the likelihood of their users encountering technology-generated errors.

Leave a Comment