Siri doesn’t have all the answers, but Apple’s digital assistant continues to get smarter.
Loup Ventures is throwing 800 questions to key digital assistants for its biannual IQ test to measure the practical use of voice as a means of computer input on smartphones. They are all improving, but not at the same rate.
Google Assistant continues to outperform the Big Three. He understood all the questions and answered 92.9% correctly. Siri missed two questions for full comprehension and answered 83.1% correctly. Alexa made the most dramatic improvement. He understood 799 questions and answered 79.8 percent.
Cortana was taken out of the competition because Microsoft turned to integrating the digital assistant with the operating systems of its former competitors.
The IQ test measures “within reason”
“With our test scores quickly approaching 100%, it may appear that PDAs will soon be able to answer any questions you ask them, but we are warning that this is not the case.” Wolf Ventures‘Gene Munster and Will Thompson wrote. “Today, they’re able to understand, within reason, anything you tell them, and the main use cases are well constructed, but they’re generally not smart. “
The test scores digital assistants on five question categories: local, commerce, navigation, information and ordering.
Siri IQ test: win, lose
Siri has beaten its competition on ordering matters, going bold with calls, texts, emails, calendar and music requests. In fact, Siri still tops this category. This time, Siri answered correctly 93% of the time, compared to 86% for Google and 69% for Alexa.
As the Wolf Ventures chart shows, Google Assistant tops the other four categories, even commerce (92%), a category that should belong to Amazon’s Alexa (71%).
Smartphone users who regularly use digital assistants appear to use voice to purchase items at lower rates. Google Assistant performed better than Alexa in answering more research-based business questions, such as product specifications and places to buy.
Since this is a competition, Alexa would win the Most Improved Player award. Its percentage of correct answers increased by 18 points, compared to seven for Google and five for Siri.
Wolf Ventures is performing a separate test for voice assistants on smart speakers due to variations in products and applications.