Baidu tops Microsoft and Google in teaching AI to understand human language, thanks to differences between Chinese and English
- Google’s model hides 15 per cent of the words in each sequence and then tries to predict them based on context
Chinese search giant Baidu has beaten Microsoft and Google in an ongoing natural language processing competition, thanks to the linguistic differences between Chinese and English.
Baidu’s model, called ERNIE (Enhanced Representation through kNowledge IntEgration), recorded the highest score of 90.1 – just ahead of Microsoft and Google – on the General Language Understanding Evaluation (GLUE) benchmark and analysis platform for natural language understanding. Baidu’s model was first developed to understand Chinese language but researchers soon found it was able to understand English better too.
Baidu’s ERNIE was inspired by Google’s BERT, a “masked” language model used by the US tech giant to train AI to understand human language. Google’s model hides 15 per cent of the words in each sequence and then tries to predict them based on the context.
However, given that many Chinese characters do not have an inherent meaning until they are strung together with other characters – a key linguistic difference from English – the team at Baidu needed to train its AI model to understand how to hide a string of meaningful characters and predict the masked ones.
The team at Baidu illustrated the technique on its Github page, taking “Harry Potter is a series of fantasy novels written by J. K. Rowling”, as an example.
Baidu CEO more confident after better-than-expected quarterly results
BERT was able to identify the “K” through the local co-occurring words J, K, and Rowling, but was not able to learn any knowledge related to the word “J. K. Rowling”. However, ERNIE was able to extrapolate the relationship between Harry Potter and J. K. Rowling by analysing implicit knowledge of words and entities, to infer that Harry Potter was a novel written by J. K. Rowling.
As the Baidu algorithm started to understand meaningful words instead of individual characters, it performed better in both English and Chinese. The company has since adopted ERNIE for real-world applications by using the AI model to deliver better search results. A paper describing the model has been accepted by the Association for the Advancement of Artificial Intelligence (AAAI) for its annual conference in February.
Baidu has a stranglehold on search in China, amassing a 70 per cent market share after Google exited China in 2010, but a shift in internet usage patterns has chipped away at that dominance. The rise of self-contained, super-app ecosystems like Tencent’s WeChat means that people can watch videos, read news, shop online, and order takeaway food without having to use a traditional search engine.
Separately, new competition in search has emerged from ByteDance, operator of the globally popular TikTok short video app, which said it is building a “general search engine for a more ideal user experience.”
For more insights into China tech, sign up for our tech newsletters, subscribe to our award-winning Inside China Tech podcast, and download the comprehensive 2019 China Internet Report. Also roam China Tech City, an award-winning interactive digital map at our sister site Abacus.
