Is Google SyntaxNet Really the World’s Most Accurate Parser?

Google is a giant and its marketing is more than powerful.  While the whole world was stunned at their exciting claim in Natural Language Parsing and Understanding, while we respect Google research and congratulate their breakthrough in statistical parsing space, we have to point out that their claim in their recently released blog that that SyntaxNet is the “world’s most accurate parser” is simply not true.  In fact, far from truth.

The point is that they have totally ignored the other school of NLU, which is based on linguistic rules, as if it were non-existent.  While it is true that for various reasons, the other school is hardly presented any more in academia today due to the  mainstream’s dominance by machine learning (which is unhealthy but admittedly a reality, see Church’s long article for a historical background of this inbalance in AI and NLU:  K. Church: “A Pendulum Swung Too Far”), any serious researcher knows that it has never vanished from the world, and it actually has been well developed in industry’s real life applications for many years, including ours.

In the same blog, Google mentioned that Parsey McParseface is the “most accurate such model in the world“,  with model referring to “powerful machine learning algorithms”.  This statement seems to be true based on their cited literature review, but the equating this to the “world’s most accurate parser” publicized in the same blog news and almost instantly disseminated all over the media and Internet is simply irresponsible, and misleading at the very least.

In the next blog of mine, I will present an apples-to-apples comparison of Google’s SyntaxNet with the NetBase deep parser to prove and illustrate the misleading nature of Google’s recent announcement.

Stay tuned.

 

[Related]

It is untrue that Google SyntaxNet is the “world’s most accurate parser”

Announcing SyntaxNet: The World’s Most Accurate Parser Goes Open

K. Church: “A Pendulum Swung Too Far”, Linguistics issues in Language Technology, 2011; 6(5)

Pros and Cons of Two Approaches: Machine Learning vs Grammar Engineering

Introduction of Netbase NLP Core Engine

Overview of Natural Language Processing

Dr. Wei Li’s English Blog on NLP

 

 

发布者

liweinlp

立委博士,自然语言处理(NLP)资深架构师,Research Director, Beyond AI.前 Principle Scientist, jd-valley, 主攻深度解析和知识图谱及其应用。Netbase前首席科学家,期间指挥研发了18种语言的理解和应用系统。特别是汉语和英语,具有世界一流的解析(parsing)精度,并且做到鲁棒、线速,scale up to 大数据,语义落地到数据挖掘和问答产品。Cymfony前研发副总,曾荣获第一届问答系统第一名(TREC-8 QA Track),并赢得联邦政府17个小企业创新研究的信息抽取项目(PI for 17 SBIRs)。立委NLP工作的应用方向包括大数据舆情挖掘、客户情报、信息抽取、知识图谱、问答系统、智能助理、语义搜索等等。

发表评论