白:
“天就是这样被聊死的。”

李：
说谁呢 ？

梁:
@wei ，I also admired your "层次纠缠“ comment.

李：
哦 那是刘少奇主义, 吃小亏占大便宜。真的，反单层parsing的传统潮流，悄悄地不知道占了多少便宜了，不吃点亏都觉得不好意思了

白:
“的”可以“买一送一”或者“卖一送一”。比如“卖火柴的小女孩”中，“小女孩”既可以作为整个定中结构的代表正常对外填坑，也可以“无偿”填定语从句内部“卖”的坑；“这本书的出版”既可以作为整个定中结构的代表正常对外填坑，也“无偿”对内接受定语部分“这本书”的填坑。“无偿”的意思是，一个括号配一个反方向括号后，对方消失了自己还在。多“饶”了一次匹配机会。现在从坑的角度看，最不情愿的一个处理就是把形容词的使动用法处理成一可选的坑。如果坑可以强制出来就好了。

李：
多一次匹配机会，就是一个儿子可以有两个老子，类似一个是生父，一个养父。对外填坑是句法的显性 dependency，对内无偿就是逻辑语义的隐性 dependency，中外皆然。“小女孩”就是如此，“小女孩”做“卖火柴”的【逻辑主语】。而“卖火柴”其实是“小女孩”的【定语】：让自己的显性的儿子去做自己的隐性的逻辑老子。到了谓词指称化就不同了，对外填坑不变，爱啥啥。对内的逻辑关系则反过来，自己的显性的句法定语儿子（adjunct），被用来填充为自己的隐性的逻辑儿子（argument）。“这本书” 做 “出版” 的逻辑宾语。与英语平行。

this book‘s publication；girl selling matches
? the sell-matches girl

白:
这两件事，被我统一成一件事了。

李：
统一的好处是？

白:
词典化，没有规则，只有词典和原则，一条原则管两头。到原则层面，只需要解决什么条件下谁提供bonus。

李：
一边是 NP ， 一边是 V 或 VP。如果是 NP 的 V，V有坑，尽管 V 指称化了。那么 NP 就去填坑（宾语，或主语），如果是 VP 的 N，那么 VP 有主语的坑，N正好填。

白:
根本不看POS，只有买单和卖单。一对多的订单、一对一的订单、可以提供bonus的一对一订单。

李：
POS 也没啥，不过就是填坑中几百上千个可能的约束条件的一个而已。我要 Human 来填坑，与我要名词来填坑，对系统不过就是条件的宽窄不同而已。这是从填坑的角度看 POS。对于设坑的一方，当然无所谓 POS，V 也好 N 也好 A 也好，他们有没有坑，有几个坑，都可以认为是词典化的 subcat 规定的。都直接量（词）驱动了，自然就不谈 POS 了，因为 literal 永远比任何抽象信息量更足。

据说当年一个叫 Small 的人首创专家词典（Expert Lexicon），里面全部是词例化的规则，没有任何抽象规则，可以想象这样的系统在一个狭窄的 domain 里面的可行性。譬如 在天气预报的 domain 里。词条 “下雨”里面规定 要到左边去找 “老天”，到右边去找 “很猛、很急”，等等。

白:
肯定不是这样的。

李：
在一个小词汇表中 是可行的的 而且准确 有点组合爆炸而已。这是没有任何抽象的本义。一旦有抽象，哪怕是词例化，也必须引入 features，而不是直接量之间的交易了。lexical-POS 就是最简单的一个 feature。

白:
原则不看POS，matcher要看。但原则不放水，matcher就没有bonus给。

“张三这两条建议是关于修宿舍的。”

这里面，“建议”有两个坑，“张三”填了一个，然后这个词的“母体”填给“是”了，剩下一个坑成了没娘的孩子。后面当“关于修宿舍的”作为一个整体与“修宿舍”剩下的没娘孩子（human）不匹配的时候，匹配点会迁移到前一个没娘孩子（info，建议的内容）进行匹配。

李：

白:
这不是说，建议的内容就一定是“关于修宿舍”，万一“是”换成了“不是”呢？只是说，这一萝卜一坑，存在着语义类型上的关联。至于肯定否定，那是由围绕着谓词的修饰语决定的。

李：
肯定否定是另一层次的东西，无需牵扯进来。说到底就是 “建议” 有 Subcat，里面有 human agent，和 “关于 content” 的 PP 的规定。human S 连上了，【关于】的坑暂时没连上，但也不难。

白:
建议谁修宿舍，文本中找不到，作罢

“他死去多年的战友就埋葬在这里。”

“他”要憋着不参加后面的VP，直到被定语从句修饰的中心语露面，填入中心语所带的坑，才算了结。什么样的“过程性”控制策略能给出这个选择？

宋:
他死去多年，儿女都已经长大的战友埋葬在这里。

白:
嗯，其实谓词部分是收束的，只有谓词部分对外需要消解。所以，需要栈，但栈不必很深。栈和RNN是不矛盾的。栈顶元素可以作为输入的一部分，对栈的操作可以作为本轮输出的一部分。

李：
查一下后条件不就解决了？在主谓规则中 ，一个 np 与 一个 vp 不着急结为 s，往后查一下条件再决定。

没问后条件，错了。可以加上：

白:
往前看一个，只能做等待与否的决策，不能做结合与否的决策。等待就意味着要记忆某种东西。

李：
等待与否与决策与否，这里不是一个简单的答案。因为涉及两个问题：一个是“他死” 的主谓问题，一个是“死”做定语（兼“战友”的逻辑谓语）的问题。如果不考虑二者相交，第一个问题当成一个独立的问题，当然可以决策，不过是问合适的条件包括后条件而已。这样“他死”本来的主谓错误可以避免，但还是需要有人（“埋葬”）接盘。从相交的角度看，关键是定从句型的处置安放在何处合适的问题，定从解决好了，顺带也就解决了“他死”要不要就近连主谓的问题。涉及的句型也不那么复杂：

NP+VP+de+N

就是一个四元组。把上述句型在做主谓之前 fine-tune 到正好涵盖【定从】，问题就解决了。宋老师的句子是难一些，难在那个 VP 复杂化了，VP 实际是两个 VP 用逗号并列了（其实应该用顿号的，可国人把逗号当万金油，没办法）。这倒也罢，第二个谓语本身也是一个主谓结构：“儿女都已经长大”。“儿女长大” 与 “身体健康” 类似，都是那种句型紧凑的典型的【主谓谓语】。这类主谓只能有限扩展，跟通常主谓的自由度无法比，也因此可以考虑先行解决，给个标签，作为整体，它有一个逻辑主语的坑（通常是其前的 Topic 去填）：实质上是对付一层的中心递归（center recursion）。总之是有些难缠，但并非无迹可寻，要做也可以做，考验的是细活。等低枝果实都摘差不多了，再去磨这个细活好了，现在不必。

白:

他那些杀红了眼，刺刀上沾满血的战友们可管不了那些了。

“儿女”有坑，把“战友”捎带上还可以解释。“刺刀”的坑是“枪”，“枪”的主人是human，这弯儿拐的。句法非标配的坑，靠语义中间件凌空凿开一个坑，才能把定语从句的钩子钩上。第一个“那些”如果去掉：

？他杀红了眼，刺刀上沾满血的战友们可管不了那些了。

好像不通了。或者说链条断开了。所以凌空开凿的坑无法填装远距离的“友元”。

李：
看样子这个 “那些” 是个关键的小词，应该善加利用：

human+那些+[human action] + 的+human

麻烦的是 human action 的谓语的扩充性。如果这个句型足够的频繁（感觉上是的），那么一个策略是，对于那个【定从谓语】的界定可以放的很宽，一路扫描下去，直到发现 【的+human】，就把这个 【定从】 的性质卡住了。定语定性以后，再慢慢对付里面的monsters，这个策略可能管用。

他的那些blahblah的朋友们

管它 blah 有多长、多复杂。一个 token* 就卡住了。还有一个策略就是 patching，对上面的那颗“循规蹈矩”而出错了的树做修补:

S1[X那些] + Pred1 + Conj + Mod(Pred2)+S2+Pred3

要问五个链条才能修补全，也不知道能概括多少现象，值得费这么大力气，会不会弄巧成拙。道理上是可行，问了这五个链条了，然后

（1）离婚 S1 和 Pred1
（2）结合 S1 与 S2，让 S1 zuo S2 的定语
（3） 切断 Conj
（4） 用新的 Conj 链接 Pred1 和 Pred2

可以做个实验玩玩，看这条路可行不。
MY GOD 值不值得做先放在一边，可的确做成了！

这个太tm牛了。我都不得不崇拜自己了。

还是那句话，没有翻不了的案子，毛太祖钦定的文化大革命都彻底否定了。这样的翻案 patching 应该没有副作用，因为都是 word driven 和非常 restricted 的现象。

同一条规则略加微调（没有“那些”但原主语是 human），就把宋老师的难题一并解决了。休眠唤醒术好使，以后要多使，这比条件不成熟的时候霸王硬上弓轻松多了。

白:
不对呀……
怎么是“他”死去？应该是“战友”死去才对。另外，“战友”并没有“长大”，“长大”的是战友的“儿女”。

李：
鸡蛋里挑骨头啊。明明“儿女”是“长大” 的 S。长大的战友，不过是一个边界不合适的 XP 懒得在 patching 的时候再动手术去重新修理边界而已。

白:
就是说，定语从句的两个分句，第一个“死去”的坑被提取出来，由中心语“战友”反填；第二个“长大”的坑由“儿女”填充，同时“儿女”挖了一个新的human类的坑，由中心语“战友”反填。

李：
真要修理也不是不可以，但已经没有啥意义，因为逻辑语义上已经阻止了 “战友” 做 “长大” 的主语。对，“他” 不该是 “死去” 的 S，这个我去查查 code

白:
要简化也是“儿女长大的 战友”，而不是“长大的 战友”

李：
那是因为偷懒 共享了 “那些”的规则。得，我分开来一步步来。
目前的机制可以改关系，暂时不可改边界。有空了写个 specs 让工程师增加边界调整的功能。不该有的 S 没有删去，是个 bug，规则是对的。对数据结构做手术，要做干净、不拖泥带水、不留后遗症，还需要磨一阵子。不过苗头是好的。

白:

“目前尚未毕业、导师已经超过六十三岁且不是院士的博士研究生要来教务处登记。”

谁不是院士？导师，还是博士研究生？如何确定？两个conj并列，是一股势力；“导师”辖域延展，是另一股势力。

李：
不用 common sense 出场？

白:
后一股势力因为“导师”自带的坑得到“博士研究生”在右侧遥相呼应而得到加强。当然“博士研究生”自身也因距离更近参与“不是院士”的逻辑主语坑的争夺。定性分析这是打个平手。common sense之外似乎看不到一锤定音的结构性要素。或者换个说法，大数据里，“导师”和“院士”共现的频度，与“博士研究生”和“院士”共现的频度比起来，谁高？

一提common sense就有把问题搞复杂的嫌疑，提大数据则明显把问题简化了。

李：
不错。现在的问题是，应该怎么挖掘和表达大数据的这些隐含了常识的知识，使得需要用的时候，够得着。人手工费那么大劲精心构建的 ontology 和常识，目前用起来还是不能得心应手，挖掘的东西应该呈现怎样的形态才好用呢。

白:
词向量可直接反映共现。

李：
在两个词抢夺同一个词的时候，最简单的办法就是看他们的两两共现来决定力量对比。这个听起来简单，但这种三角争夺是 on-the-fly 的，共现数据可以预先计算并 index，三角计算必须是 at run time，感觉上有一个不小的 overhead

白:
现场直接变成算距离了，index出来是向量，向量的距离直接反映共现。而且是“应该的”共现而不是“现实的”共现，中间加上了互通有无。互通有无是数学模型帮我们做的。

李：
大数据出来的统计不都是“应该的”么？都只是一种趋向。增加一个砝码，不是铁定。（一定有违反大数据统计的反例在。）

白:
不是这个意思，是说很多数据是稀疏的

宋:

（1）应该做大数据挖掘，与专家的规则结合起来。白硕建议比较两对共现频次，我觉得比常识知识库靠谱。

（2）这种大数据中的知识挖掘应当是实时的。应该有某种大数据存放的中间形式，支持快速的实时统计。这种中间形式会比原始的线性字符串对于统计更高效，同时应当有一定的通用性。

白:
在降维中稠密化了，原来语料中直接没搭上钩的，经过降维处理也可以搭上钩了。

宋:
（3）恐怕会有一些问题不是单纯的词语共现所能解决的。

白:
算距离，复杂性主要跟维度有关。维度降下来了，不仅数据稠密了，而且计算开销也下来了。@宋 老师的（3）完全赞同。共现的数学模型，build和run的确是分离的。李：

李：
synonym 或 clustering 就是 降维 和 数据稠密化吧，但同时也抹平了。不知道目前有没有哪个系统真地在歧义判别时候用到大数据统计的。

白:
word embedding并不严格抹平，但可以拉近，而且如果只为了比较大小，距离算到平方和足矣，没必要再开方。

李：
对，根本不需要精确计算，只需要相对的结论，谁强谁弱，或打平。

康：【首届语言与智能高峰论坛会议邀请函】 ...

白:
这种会怎么不请伟哥啊……

阮:
第一届会议重在推动，伟老师估计会泼凉水。

白:
我们大家还在混圈子，伟哥已经高处不胜寒了。

李：
一觉醒来 左眼发跳 原来是白老师。冷不丁开个涮 由头却是啥高峰会议。
认真滴说 休眠唤醒是正道 开始尝甜头了 。感觉以前syntax下力可能太大太苦 ，不如把负担更多转嫁给语义唤醒。

【相关】

【立委科普：结构歧义的休眠唤醒演义】

【李白对话录系列】

中文处理

Parsing

【置顶：立委NLP博文一览】

《朝华午拾》总目录

白:
“一切都在变，只有变本身永远不变。”
前后两个“变”动词特征明显，中间那个“变”怎么说？

我:
有了“本身”来构成chunk，那就是动名化的意思。
x 本身 --》 NP

白:
“这个学校就爱拿本身的那点光荣历史说事儿。”
好像这规则有副作用
在贴这个例子的时候，伟哥说的规则已经在脑子里转了N转，但还是感觉不那么简单。似乎一堆爹在抢儿子，儿子归谁跟爹们的场上表现有关。

我:
那就 fine-tune 呗：

“本身” 可以独立成 NP，“x+本身” 也可以成 NP，怎么区分呢？
tricky，不过磨细活总是可以磨。只说 x 是动词的情形：一种是限定 x 是什么样的 v；一种是限定x不能是怎样的 v。后者的话，第一个条件可以是 这个 v 不能是可以做 prep 的 v，于是“拿”踢出去了。当然这感觉还是 underkill。

白:
糊弄老板可以，如果自己是老板，不情愿这么干。

我:
还有一种就是拓宽 context：不过那个法子也 tricky 因为每拓宽一个 token，又增加很多可能性要 include or exclude，但所有的歧义区分如果需要看上下文条件手工调教，都是这么个事儿：precontext and/or post-context and/or include-or-exclude conditions on self, 手工系统让人抓狂就在这里。

白:
我是这么考虑的：作名词的“本身”一定有先行词，因此先行词的匹配特征一定会继承到“本身”上来。以“这个学校就爱拿本身的那点光荣历史说事儿。”为例，“学校”与“历史”的匹配特征，会被“本身”继承下来。于是，从“历史”反推，可以给先行词候选“学校”加分，而另一个先行词候选“拿”却得不到这样的加分。“爹们”的角力，就这样决定了“本身”的命运。

我:
这个有理。把 chunking 的边界问题 转化为 self 的 binding 的问题。不过上面的那套思路实现起来 也不是那么直接。听上去是一个 procedure, 而不是一个 pattern

白:
不需要procedure，确定一个标配的先行词，匹配特征差再唤醒非标配的。
比如，最近出现的名词作为标配的先行词。如果名词太远或者名词匹配特征得分太低，再启用左侧邻近词作为先行词。当然与“本身”呼应的也可能不是先行词而是后继词，比如“以本身的实力而论，张三是考不进清华的。”

我:
这个机制不是通常的 pattern matching 可以实现的 吧。通常 FSA 的 runner，运行的时候可以加一些 config。暂不考虑“本身”的 binding，句法捋顺倒也不难：

白:
后继词也可以设定标配。这就显出坑论的好处了。“本身”挖个坑，左侧名词优先填坑，左侧邻近词次优（如果不是名词还要给它变性）。由于是共指关系填坑，不耗费萝卜指标（还可以填其他坑）。
不做pattern matching
btw，昨天讨论的闭环填坑也不耗费萝卜指标。用填坑来取代pattern matching，规则按词典化的方式组织更方便。

我:
有意思。

白:
“那些质疑凯文·凯利的朋友，我真心怀疑是否认真读过他的书”

我:
他？
一共就俩 candidates，都联上算了，爱谁谁。到语义落地，大不了生成两个 insights，至少 recall 是保全了。

白:
（1）怀疑谁？（2）他是谁？（3）前一小句本可不作定语从句解，如何排除的？
这些问题是连带的，一环扣一环。

“三十里有几个五？-六个。”
即使数量结构这么铁，也有不该在一起的时候。
“三十里有多远？-不远，十来分钟就到。”
等判据出来，不知道要几个词。
“因强烈不认可挪威国家石油终止合同 中海油服提起民事诉讼”
谁不认可？
“因不满老师虐待儿童口出狂言张三愤然辍学。”
谁口出狂言？
理由？

我：
先 parse 看看：

最后一句掉链子情有可原，貌似人理解也有困难。原句似有语病，总之不对劲儿。

【相关】

【李白对话录系列】

中文处理

Parsing

【置顶：立委NLP博文一览】

《朝华午拾》总目录

白:
hownet坑的供给比较充分，但是也很难说一定不会超出上限。（董老师可以试试“这场火多亏消防队来得及时”）

有很多其他类型，比如，“穿着拖鞋就走出来了”。不知道该贴啥标签。还有“撸起袖子加油干”。这两个例子中，在时间上，伴随行为本身在前，伴随行为的遗留状态持续时段包含核心行为持续时段。比如，撸袖子的动作在前，然后遗留状态持续，然后加油干持续，然后加油干结束（或有），然后放下袖子（或有）。

李：
with sentiment:

at least for semantic landing to sentiment, the parse gives decent support。

宋:
O和ObjV是什么关系？

李：
宾语和宾语补足语

白:
比如：知道你来了。你是O，来了是objV

李：
not really

宋:
@wei 你的论元关系分析，相当炉火纯青了。

李：
宋老师果酱。

白:
这俩的区别？“消防队来得及时”为啥不可以是objClause？

李：
“多亏” 直接 link “消防队”，“知道” 不直接 link “你”，only linking to the event

白:
我是问，是否直接link，准则是什么？有什么必要区分这两者？

李：
语言学家的多数有共识。词典subcats 印象也是区分的，落地时候也发现有益。

白:
但很多研究汉语语法的人并不认为汉语有“宾语补足语”的。准则是什么呢？一词一议？
sentiment传递？
“我讨厌他总出差”
这里的“总出差”成了objV，“他”是O。“讨厌”的负面属性可以传递给O，是吗？如果这样，O必须是objV的逻辑S，否则，“我讨厌花摆在阳台上（，掉下来砸到人怎么办？）”
里面的“花”就要受牵连了。

李：
对。

我讨厌ipod
我讨厌iPod老死机
我讨厌花儿
我讨厌花儿老养不活

这事儿真细究的话，属于 subcats 同形打架：带 clause 的句型与带宾补的句型。

白:
“我喜欢厕所门朝北开”
问题是要给宾补一个存在的理由。理由不存在，打架就不存在了。

李：
几乎所有的汉语文法（英语文法亦大同小异）都区别下面三种句型：（1）动宾式；（2）兼语式；（3）宾语从句式。动+宾+宾补的 representation 反映的是兼语式句型。兼语是一种简省的说法，不是一个单纯的（atomic）的关系成分标签。兼语说的是既当V1的宾语又当V2的主语。表面上看，只要我们连上了 O，也连上了 S，所谓的宾补V2 也就间接连上了 V1，因此 把 V2 直接联系上 V1 作为宾补似乎没有必要。问题是，那样的 representation 不能表达 V2 在结构configuration上是低于 V1 的。事实上，这是一个典型的 right branching recursion，V2 是一个 VP（V-bar according to X-bar theory），V1 只是一个 lexical V (V no bar)，V2 代表的 VP 整体都是包含在 V1 所辖的边界内。帮助 V1 构成一个更大的 VP。V2 的 VP 也可以是一个兼语式 (递归)。

白:
对。兼语句前后句的坑共享没有疑问。有疑问的是后句不必是前句的objV。在能清晰表达坑共享的前提下，V2有必要成为V1的直接成分吗？如果是，那就做小句宾。如果不是，就是松散的两个谓词做兼语式。为什么要弄出个第三条道路？

李：
小句宾 与 兼语式 有相同的地方，有不同的地方，也有灰色地带。

白:
我知道你来。我喜欢你来。
这是一样的结构。

李：
不能拿灰色说事儿。“得悉”，只能带小句，不能带兼语，这是黑白分明的 case。

得悉你来
* 得悉你

灰色地带最突出的案例在英语的 minimal pair：

I demand him to leave immediately.
I demand that he leave immediately.

白:
这是生成时才用得到的区别，分析时可忽略。

李：
英语句法上泾渭分明的两个句型，在逻辑语义深层其实是大同小异的，很灰色。

白:
汉语里没必要弄出第三条道路

李：
不一定只有生成才需要区分。语义落地也可能得益。如果没有宾补的直接联系，我问 O，就不能区分单纯的 O 和带宾补的 O，如果所有的兼语都变成 宾语小句。

白:
我派你去，就是“我派你”“你去”不需要让“去”从属于“派”。
或者说，“派”不需要给“去”预留一个坑。

李：
那就切断了动词 V1 与兼语的直接联系。必须通过 V2 才能间接联系到兼语。

白:
错，萝卜共享就是联系

李：
有时候我们不在乎是 宾语 还是 兼语，我们就没办法用同一个子树句型做抽取。譬如，“多亏”：

多亏老李。
多亏老李送我回来。

这两句要找感谢的对象，就无需区分 宾语和兼语。

白:
这没什么不好。多亏我处理成副词。“多亏了”也是一个副词

李：
如果第一句是 VO 第二句是 V+Clause，就必须用两个句型才能捕捉。

白:
多亏老李，是副词升格为动词。

李：
关键是，有时候我们需要区分宾语和兼语，有时候我们不要区分，为了同时照顾这两种需要，把兼语处理成子句是不妥的。

白:
可以不从句法上区分，而从词典来区分。

李：
退一万步，多一个宾补的直接联系，只有好处，没有坏处。

白:
我的处理是要么包进来要么切断。多了歧义打架需要处理，而这本来可以是伪歧义。

李：
是真歧义，那就该处理。是伪歧义，也可以硬走一线，系统内是可以保持一致性的。你这里所谓伪歧义的情形实际是灰色地带，或两可地带，系统内部可以规定走哪一线，内部协调即可。伪歧义的真正困扰是系统无法区分，随机走路径。如果有办法确定性地走一条路径，理论上的伪歧义在实践中就化解了。传统 parser 的伪歧义困扰是，伪歧义与真歧义混淆在一起，使得真歧义无法 stand out (being  identified)。这里的情形不是。

白:
中间没有N的，也有助动词和补语两种可能性。助动词带的谓宾真包进来，补语我处理成共享萝卜的对等合并，与所谓核心动词并无隶属关系。只不过形式上把根让渡给前面核心动词而已。
看看前面的例子，“我喜欢厕所朝北开门”，什么特征决定了“厕所朝北开门”是小句宾？

李：
不好说。
假如有一家厕所公司专门建厕所，就好像苹果公司造 iPhone 一样，“喜欢厕所朝北开门” 与 “喜欢iPhone照相清晰” 有什么根本区别？再如，“喜欢厕所清洁卫生”。

与其花力气区分灰色地带的兼语 from 子句，不如一条路走到黑，对灰色的那些词规定一条路径。到落地的时候，如果需要，再进一步 fine-tune 来决定。如果是 sentiment 落地，就可以这样来 fine-tune：“喜欢”的兼语必须是产品或其他实体专名，才可以让其得到 positive sentiment，这是从严的 fine tuning。从宽的 fine-tuning 可以要求兼语不能是带有贬义色彩的名词，assuming “厕所”是这种类别。但是这种 fine-tuning 的拿捏，只对做“兼语”的名词需要，如果“喜欢”所带的不是兼语，而是纯粹的名词宾语，那么不管宾语是王八蛋还是杀人犯，喜欢的 positive sentiment 还是落在宾语身上。

“当年的德国，很多人喜欢甚至崇拜希特勒。” 对于希特勒，这是 positive sentiment。但是，cf：

“我喜欢希特勒被处以绞刑，而不是自杀身亡，逃避人民的审判。”

这句中的“喜欢”，对于 希特勒 就不是 positive sentiment，因为 parser 把希特勒parse成有宾补的O（即兼语），而不是纯粹的 O

白:
喜欢厕所朝北开门 与 喜欢iPhone照相清晰 有什么根本区别？---这话也是我想说的，但我认为它们都是小句宾句式，与兼语无关。

李：
我要说的是，对于这样的 cases，要不一律处理成兼语。要不一律处理成小句宾语，只要 consistent 即可。

白:
希特勒那句，也是小句宾，没兼语什么事儿
什么情况下sentiment可以穿透到下面一层的成分，不是由句法决定，而是由另外因素决定。

李：
“我喜欢少年希特勒努力发奋。” 这句呢，也是小句？

白:
是

李：
我要说的是，对于有灰色地带的那些词和句型，可以人为规定一条路径。
区分：“我喜欢少年希特勒努力发奋” 和 “我喜欢老年希特勒被处绞刑”，一个是对“希特勒”是褒义，一个不是。

白:
褒义针对整体，是否传导到部分，it depends, 针对整体都是褒义。

李：
说的是部分。我喜欢【human】和 我喜欢【human】VP，对于这个 human，默认是褒义的。

白:
要区分的只是能传导到部分的褒义和不能传导到部分的褒义。我喜欢【human】vp 是天经地义的，喜不喜欢其中的【human】，不由句法决定。
在我这里不默认。而且默认sentiment设定，和搞出一个句法类别，也是两回事。

李：
默认有很多便利。偷懒的时候，默认在统计上对数据质量有好处。默认这东西，不用白不用，尤其是从工程和实用上考量。我目前的 “喜欢” 的 sentiment 规则，不论中文英文，都是走默认的道路：管它后面的 NP 带不带 VP，只要 NP 是 entity，就默认是 positive 落地了。这个 practice 在实践中有莫大的好处。 “喜欢希特勒 VP” 这样的可能的例外极其罕见，以致于根本不必在默认之外再费力气去区分。而 “喜欢厕所VP”不算例外，无害：depending on 厕所是否看成是 entity，或者被排除在语义落地的雷达之外；或者落地了，也是正确的。

白:
充其量是个heuristic

我喜欢张老师讲古代史，不喜欢张老师讲现代史

李：
这个不是反例，前半句是 褒，后半句是 贬，都落地到 张老师身上。

白:
加分减分都没问题。问题是逻辑推论有没有。喜欢张老师和不喜欢张老师如果都是逻辑推论就有问题。讲逻辑的人是分得很清楚的，喜欢，对事不对人。sentiment也是很清楚的，一个给张老师加分，一个给张老师减分。但是确实没有“喜欢张老师”和“不喜欢张老师”这两个逻辑推论。

李：
回到原来的 arguments，如果 （1）我喜欢张老师；与 （2）我喜欢张老师讲古代史，是两个不同的 parses，sentiment 落地的时候，就必须有两个规则来对付这两个不同的结构

白:
这个很难吗？我不觉得。爱屋及乌，爱小句及小句逻辑主。一句话的事情.

李：
天下无难事。
但是，对于如此基本而普遍的 sentiment 的表达（love类 和 hate 类），本来可以直接搞定，非要绕一个弯儿再搞定。本来一组规则，非要增加一倍的工作量去写规则，去维护，这是很不合算的。

即便不谈 sentiment 这个具体的落地应用，抽象来说策略，这个争论的本质就是：两个句型有相同的部分，有不同的部分，如何处理？其实没有普世标准，就是系统内的一致性和方便性，系统内协调。

白:
不是这样的，就是一个sentiment penetration属性赋值yes还是no的问题，直接写在相关动词的词典里。其他都不受影响。标配赋值no，需要设yes的明示。

李：
我喜欢他。
我喜欢他老实。
我喜欢他做好事。

换一个角度看，第一句和第二、三句在“我喜欢他”这个部分是一样的。后面的 AP 和 VP 也可以看成是喜欢的某个部分或方面（aspect）（或理由），这个角度是先对人后对事儿。与你的“喜欢”应该对事不对人，或由事及人，具有等效的逻辑性。即便我说“我喜欢X”，常识告诉我们，世界上没有完美的X。“喜欢X” 后面隐藏着后面的 likable aspects，X 与 aspects 是整体与部分的关系。

白:
我喜欢他，我喜欢他老实，我喜欢他的老实。VS
我喜欢这本书，我喜欢这本书出版，我喜欢这本书的出版。

也就我们这种领域，讨论一个语言学概念的立和废，也要扯上系统甚至应用。

李：
是从部分的角度来推及整体，还是从整体的角度推及部分，不过是角度不同。

白:
如果一个arg，是这样。角度不同。两个坑就未必了。

李：
就好比半瓶水，一个角度看到了半瓶水，另一个角度看到了半瓶空气，都是有效的逻辑。

白:
我喜欢这本书出版，和我喜欢这本书，不是延展后者的关系。我们还可以说，我喜欢这本书春节后再出版。喜欢的重点既不在书也不在出版，而是在出版的时间。可以既不喜欢书，也不喜欢出版，只是对那个出版时间情有独钟。一个坏人干了件好事，我们总可以喜欢那件好事吧。不需要扯上喜欢坏人。

一个出版商可能对某本书恶心到吐，但是那本书的出版可以帮他赚大钱。他喜欢“那本书出版”但不喜欢“那本书”也是情理之中的。heuristic可以进入系统，但进入语言学是否合适要画个问号。进入系统也有“诛心”之嫌。喜欢坏人干的某件好事，被当成支持坏人的证据，这样的系统大家都要当心了。

李：
喜欢坏人干的某件好事，被当成支持坏人的证据，在情感分析上没有错。在大数据分析的时候，点点滴滴皆证据。坏人干了好事儿，只要被提到，就给这个坏人加了一分（褒）。这一点儿也不影响对于这个坏人的舆情分析全貌。恰恰相反，这才是真实的舆情。如果坏人干了好事儿被提及 m 次，坏人干了坏事儿提到了 n 次，纯粹厌恶坏人的情绪表达提到了 o 次，纯粹喜欢坏人的情绪表达提到了 p 次（p 通常接近零），那么这个坏人的褒贬指数，就是 （m+p） 与 （n+o）的比例。请放心，p 基本是 0，而 m 也会远远小于 n，这个大众舆情不仅靠谱，而且真实，bias free。

宋：
我喜欢希特勒自杀。

李：
宋老师的这个例子前面已经论及。回顾一下就是：情感分析中，“喜欢”的默认规则的确可能把它算成是对“希特勒”的正面评价。因为这个默认规则是把宾语和兼语同等看待，不去问后面的宾补 VP。理论上，这个结构根本就不是【宾语+宾补】的结构，而是【宾语子句】的结构，但是由于区分二者对于“喜欢”这样的词，有一定的难度，所以我们可以在 parsing 阶段一律当成兼语结构处理和表达。这样一来，默认的sentiment规则就会犯错。

犯错怎么办？如果这种案例极其罕见，不值得做，那就算了，因为默认的sentiment规则在绝大多数的场合是保证了数据质量的。如果觉得还是值得做，那就在默认sentiment规则之上再做一条规则去 override 默认。这条规则不需要改变 parser，而是利用 parsing 在这类结构上的 consistency（错也错得可以预测） ，将错就错，矫枉过正，把这个错纠正过来。换句话说，这个策略等于是休眠唤醒，不过这个休眠唤醒不是在 parsing 的后期进行，而是在 sentiment 语义落地的时候进行，其效果等价于把【兼语式】重新分析为【宾语小句】，切断“喜欢”与“希特勒”的语义直接联系。

不知道我说清了没有。可能有人会问：既然在语义落地时候要做类似休眠唤醒的工作，为什么不索性在parsing里面（parsing的后期，语义模块）里面做休眠唤醒呢？理论上，提前做休眠唤醒，使得parsing更精准，可以benefit不仅仅这个 sentiment 的语义落地，还可以 benefit 其他的语义落地和应用的场合。但是，实践中在哪个阶段做合算，不是那么简单。因为休眠唤醒这些事儿大多是长尾现象，鸡零狗碎，做不胜做。在还不能确认到底有多少好处前，往往顾不过来在 parsing 中统一处理好。而且很多时候，做了也白做，你需要用到那个落地或那批落地也许根本就用不到。

Anyway，point is，落地的时候是可以包容 parsing 的错误和不足做包容性产品开发（adaptive dev），只要 parsing 是可预测的数据结构，是有内部直通车的，而不是 offshelf  拿来的 parser，缺乏足够的内部支持和协调。Having said that，如果确实事先可以 identify 一些带有普遍性的休眠唤醒现象，以及可以惠及很多可能的语义落地应用，而且也有足够的时间和资源去做这些细线条的深度语义工作，那么不要等到落地就在提前在 deep parser 的语义模块里面做好，当然是更理想的情况。

白:
对坏人的褒贬判断，对事不对人自然左右不了大局，没什么可担心的。问题是拿对事不对人的表述做以坏人为参照的站队分析，这就很可怕了。

李：
可怕啥？或者就是大海里的一颗老鼠屎，丝毫没有影响。或者就是，用休眠唤醒杜绝它（它=“拿对事不对人的表述做以坏人为参照的站队分析”）。休眠唤醒之所以可以进行，是因为 parser，已经准备好了结构基础：要啥有啥，哪怕这个 parse 是不完善的。要 O 有 O，要 宾补 有 ObjV，要逻辑 S 有对于 V2 的逻辑 S，四通八达的路径都在。driving word “喜欢”也在，可以词驱动。所有的结构的节点词的信息，包括 ontology 都在，包括 “希特勒” 是个独裁者、通常是坏人这样的信息也都在。有了直通车，啥事儿做不成？什么案子不能翻？什么深度休眠不能唤醒？什么具有统计性的“可怕”的后果不可以杜绝？

白:
分析成小句宾，诛心或误伤的责任就不在分析器的开发者。而在后道工序。否则人家会说，都是你分析的我喜欢希特勒，我哪儿喜欢了？一颗老鼠屎，也那个。对于小句宾来说，没分析出我喜欢希特勒，这大不一样。但是小句宾结构与“喜欢”结合，大概率被穿透，这句话可以说，之后也是要啥有啥。谁用了谁负责，但分析器的开发者也没说错话。

李：
话说到这份上了，其实也没有多少进一步需要辩护各自做法的 arguments 了。选择这个兼语的表达，肯定不是因为明明知道处理成宾语小句更好，而不为之。一定是自有苦衷和盘算。

白:
我若干年前也用过这样的表达。最典型就是“我是县长派来的”，把其中的“来”也作为“派”的一个arg。后来发现，其实去掉它没什么损失。因为“派”和“来”共享了一个萝卜，所有联系都在里面了。

李：
隐约的感觉是，去掉它在逻辑语义深层没有损失，可能的损失是句法表层的痕迹（语法学界叫做 configuration info，就是 X-bar 理论里面的 bar 的信息）。

白:
这是有很多其他途径可以记录的

李：
留住它就是留下了这种 bar 的痕迹，就好比我们利用小词解构了实词之间的句法语义关系以后，我们并不把小词当敲门砖丢弃，而是用 X 挂上一样。虽然 理论上，这些小词对于深层语义已经没有意义。另一个例子是主动被动，到了逻辑语义的核心，这个 voice 的不同色彩，没有地位，可是语言分析中，留下表层的 voice 信息，还是可能对语义落地和应用有好处的。

【相关】

【李白对话录系列】

中文处理

Parsing

【置顶：立委NLP博文一览】

《朝华午拾》总目录

白:
他写字好
他人品好

多么平行呀，为什么两个“好”待遇这么悬殊

李：
好 是一个百搭词。因为百搭 所以赶上谁谁：他好；人品好；写字好。

白:
其实就是有一个pos为wildcard的坑，我这里标记是X

李：
人品和写字 都有一个 “人” 坑：一个是【所有】，一个是【施事】。

白：

李：
没看清 “得”怎么带的“好”。

白：
箭头方向是坑，箭尾方向是萝卜。得的输入是谓词，输出是体词。好的输入既可以是谓词，也可以是体词。语义上是把动作转化为为对象，然后以对象的身份填坑。

李：
“好” 没疑问。说 “得” 输出 N， N 被“好”吸收，这个说法有些不好理解，也不大看得出好处。传统说法是 “得字结构” 做谓语的【补语】。不一致的地方在于得字结构怎么构成、谁先谁后。得字结构的构成是“得”有坑，而不是“得”先与谓语结合，然后去填坑。你这里parsing 的先后正好与传统做法次序相反。

白：
结合以后做主语，让“好”做谓语。不是“好”或者“得好”做补语。

李：
这个句法分析与众不同。“得字结构”构成很容易：

【得 ➕ 评价类形容词】 是一种； 干得漂亮。
【得➕vp】：跑得快（这时候 vp 的逻辑主语沿用前一个谓语的主语）。
【得 ➕ s】 是另一种: 气得他哭了。

白:
没问题，“他”反填回“气”不占名额，可以处理成一个。无非就是约定什么情况下一个萝卜填俩坑不占名额的问题。跑得快，和跑得上气不接下气，结构上没有本质不同。病得很严重，和病得起不来床，也是一个结构。并不因是否使用了评价类形容词而不同。传统所谓程度补语，都可以用这个套路。

但是可能补语好像不一样。搬得动，睡得着，考得上一类。跑得快，跑快了，跑快些，跑赢了 填坑结果一样吗？

李：
可能补语是词典扩展 lexical rule，算是词典一级的延伸扩展，不难处理。

白:

因循守旧最简单了，我之前就是这么玩的。垂直方向是填坑关系，水平方向是修饰关系。但是不知道谁填“严重”的坑。root也和语感不符。这样的结构也是可以有的：
“他的病很严重”和“他病得很严重”不应该在“严重”的填坑方面有大的不同才对。要么这样：

“得”的填入体“很严重”留下了一个待填的坑X。作为root的“病”，向自己修饰语的体内回填S，不占萝卜指标。这就成环了。

同理，“他病得起不来床”可以处理成：这次轮到体外的“他”向“起不来床”回填N。在体外不占名额，所以“病”照填不误；体内“起”“来”为合并关系，共享除了“床”之外的一个坑，也只用一个名额。所以从“他”出发的三个萝卜，竟有两个是不占名额的。

这么处理，如果不耽误回填，也挺好。反正再怎么折腾都只折腾“得”一个词条，跟别人无关。补语还是补语，但需要回填。的[N+/X]、地[S+/X]、得[+S/S]都有了。它们都是单进单出，去掉确实不影响拓扑，但是放在那里可以揭示语义类型转换的逻辑。

李：

关键是 “病”与“很严重”具有直接联系。另外，【human】作为 N 的 Mod 与 【human】作为 V 的 S，具有相似性。这个甚至在 X-bar theory 中就有所揭示（specifier and subj are both external args）。进入深层逻辑，可以认为这是同一个关系的两种变式。

白:
可以再推广：
这本书的出版，这套房子的装修，这款软件的开发。

不是S，不是human，也有同样的暗通款曲。

李：

白:
看不到“这本书”是如何填“出版”的坑的。

李：
还没做呢，自然看不到。都是 Mod 作为句法桥梁，语义中间件目前还没全到位。加上这个逻辑 O 不难，只要与逻辑 S 区别就好：“出版”的宾语坑，需要一个【publication】的萝卜，放松一点也起码是一个非【human】或非【organization】的萝卜。

白：
这一部分完全是探讨，与formalism无关，只是在formalism框架内探讨小词“得”负载结构的不同表示方法的优劣。

【相关】

【李白对话录系列】

中文处理

Parsing

【置顶：立委NLP博文一览】

《朝华午拾》总目录

白:
伟哥对主谓vs.述补，️“得”字的作用有何评述？

李：
题目大了点儿
小半部汉语语法了 都。

白:
动作敏捷，行动迅速，打仗内行，排兵布阵有章法，运行平稳……
执行命令坚决，干家务不情愿

root感觉上都是在后面，是谓词在填谓词的坑。为什么被填坑的反而成了陪衬，这说不通。第一个“动作”是名词，我拿来跟后面对比的。说大不大，就是上面这些表达，head是谁，谁填谁的坑。如果跟主流是反着的，有什么后果。

李：
拿 practice 说话吧。

他努力工作
他工作努力

一般来说 “工作” 只有一个坑，“努力” 有两个坑： 一个人，一个是人的动作。这就是句法 subcat 与 逻辑 subcat 不尽相同之处。句法上，老子的 subcat 里面有儿子 args 的坑， 没有 mods 的坑。譬如 动词有 主宾等 但没有状语的坑。名词有补足语的坑 但没有修饰语的坑。但是逻辑上不同，逻辑上 任何 mods 定语啊 状语啊 对于与它直接联系的老子，都有一个语义相谐的要求。这种要求可以写进做修饰语的那个词的词典去。但对于语义搭配稍强的情况 也不仿写进做老子的词条去。关系是mutual 的 选择谁去 encode 这种细线条信息 其实没有一定之规。

问白老师一个问题，这样的借助小词接力，一步一步、一对 tokens 一对 tokens 的两两向前推进 parsing，会不会造成过多的假 parses (所谓伪歧义的困扰)？
DG中，语言单位虽然都可以 binary 关系表示，但是 parsing 的时候却不宜两两推进，因为那样的话，条件往往失于过宽。还是我理解有误？

白:
不会啊。（1）小词对距离很敏感，太远了肯定要靠实词自身的互联。（2）小词对结合方向很敏感。实词语序乱了不要紧，坑可以把它们找回来，但是小词语序乱了一定不知所云。什么语法不借助中间件都必然失之于宽。所以，WSD也好，二元关系也好，树也好，最后都要靠中间件摆平。

李：
对着例子谈吧：譬如 是AP的
这个汉语表达 affirmative 的 pattern，就是这么简单的一个 pattern，
用 “是。。。的” 把形成的 AP 包裹一层。因为这种小词没有啥意义，最多不过是给这个谓语加一个 affirmative 的 feature 的信息，其他的关系连接，还是要直接通向 AP：

她是漂亮的 == 她漂亮

白:
AP有两个表达方式，一个是紧靠名词的，一个是松散的（作谓语，作补语，通过“的”作定语）。是……的，是后一种。我们先解决句法成分“一个都不能少”的问题，再解决语义关系的抽取问题。

李：
可是看白老师的图，感觉在小词里面绕。

白:
没有无缘无故的绕。

李：
包裹一层的话，就不需要绕，就当成 AP 的前缀后缀了。说的是 pattern 包裹，结构根本就不用包裹，不过加一点信息而已，甚至丢掉那信息也无大碍。

白:
问题是，不光“是……的”，还有“有……的”还有其他“V……的”。我们可以统一处理。到了语义关系抽取阶段，衣服会脱下来的。二元关系进行到底，应该看不见pattern。pattern体现为判断二元关系成立与否的前后条件。

李：
的字负载结构 统一处理的好处何在？

白:
一视同仁

李：
这个好处貌似不实惠，也不必需。

白:
所有词、所有二元关系一视同仁，用同样的formalism处理，不排除神经。

李：
哦，有无缝挂靠RNN的便利？
一视同仁本身最多是显得 elegant，这种二元推进一视同仁的处理与用长短不同、随机决定的 patterns 处理，除了所用的机制不同，还有什么特别的说法？

白:
没有规则，没有pattern，bug都在词典里，要de，就改词典。语义构造最大限度地平行于句法提供的二元关系。

李：
patterns 甭管怎么写，说到底也是统一的机制，某种有限状态及其延伸罢了。

白:
pattern写了也不矛盾。不改变核心机制，只改变前后条件。

李：
只改词典，不改规则的例子见过，那就是 HPSG，但那是建立在词典结构无限复杂化的基础上。现如今用的是简单得多的词典结构。Categorial grammar 那种吃了吐、吐了吃在 cat 基础上的填坑挖坑，要实现只改词典就能 parsing 的开发，感觉哪里缺了什么。

白:
但是把pattern固化，就会引进来不robust的问题。很多情况下，系统自己找对象比用pattern拉郎配更聪明。

李：
同意，这是一个问题，不好拿捏。但换一个角度，用 patterns 直观易读，与人对语言现象的捕捉非常近似，而且 patterns 其实也还是立足于自己找对象的 subcats 的词典化信息基础。问题不在 patterns 上，而是在 patterns 的条件拿捏上。

白:
对，我就是在扬弃范畴语法复杂结构的方面跨出了一大步，坑和萝卜，都是“单体化”的。直到目前还没有发现什么语法现象必须引入复杂结构的。
都一样，吃了吐的路线，关键也在拿捏。

李：
又要简单统一，又要避免伪歧义，感觉是一个矛盾（当然 结构复杂本身也 adds to 伪歧义 ，那是另一个话题）。

白:
伪歧义在外面，中间件来搞。但是中间件面对二元关系搞非常清晰，中间件面向pattern搞就累了。二元关系是最简单的结构。就是一个词典词一个义项，其pos定义只有一层。只有一个“/”号。

李：
这就是我还没有理解的地方，感觉机制太简陋。机制简单统一，词典信息也简单，pos 只有一层，不过是 encode 了一些坑的信息，用的也是简单的 x/y，规定了输入（挖谁的坑）和输出（填什么坑）。如此简单统一，对付自然语言的窍门在哪里呢？

中间件通过二元关系搞定原则上没有问题。所谓语义中间件，在我这里，不过是把已经成串的珍珠链，经过某个子链，把一些语义相谐的珍珠挑出来，让间接关系变成直接的二元语义关系。假如初始的二元图是： 1--》2 --》3 --》4 --》5，语义中间件可以做到：1 --》3； 2 --》5，揭示诸如此类的hidden的逻辑语义关系。白老师的中间件有所不同，用的也是语义相谐（通过某种无监督训练而来），但目的是确保parsing不受伪歧义的羁绊。

白:
复杂的地方是什么时候有免费额度。

荀:
白老师是把这些简单的范畴放到RNN中，这个Rnn中间件性能决定了分析器性能

白:
句法“是什么”在这种机制下确实不复杂但管用。“怎么达到”是另一个问题。

李：
免费额度怎么讲？

白:
比如定语从句里面的坑，就是不占萝卜名额的。填了里面还可以再填外面。还有“NP1的NP2”，如果NP2有坑且与NP1语义相谐或统计意义上搭配，则NP1填入NP2也不占萝卜指标。比如“张三的弟弟”，“这本书的封面”。

李：
好，作为 syntax 表达，这些都不是问题。说说 ”怎么达到“ 吧。

白:
这一部分是parser最核心的地方了。

荀:
就是白老师的“毛毛”，一种利用大数据无监督的subcat嵌入算法。

李：
不是说只要 debug 词典 就可以达到吗？词典也没太多 debug 的余地，假设挖坑填坑都基本在词一级标注清楚的话。然后就灌输大数据？无监督的大数据在这些词典信息的基础上，学习出来的结果是什么形式呢？应该是词典 subcat 的语义相谐的条件。这些条件一旦学出来，就成为 parsing 的伪歧义的克星。

荀:
借助subcat嵌入的分析器要是突破了，短语和pattern也就嵌入在网中了

李：
换句话说，词典的每个词的粗线条的挖坑填坑先由人工敲定，而这些坑的语义条件 让大数据来填，从而粗线条变成细线条。从而伪歧义急剧减少。我是这么个理解。

白:
WSD和Matcher学出来的东西不一样的。

荀:
原来的做法是借助词的语言模型或词的Rnn消解伪歧义，白老师是把Subcat嵌入了网中。但一直担心前期中间件的可信赖程度，中间件错了，休眠唤醒，那么唤醒时是否用了patten？

李：
利用大数据无监督的subcat嵌入算法，这个方向没有问题，词（直接量）不够，一定要加subcat。唤醒可以看作二阶中间件。唤醒的都是局部的、个别的现象（子图 pattern），至少词驱动这一级唤醒，机制上与中间件利用语义相谐没区别，是靠谱的事儿。唤醒在我的实现中，一定是用 pattern （子树）为基。

WSD和Matcher 各自怎么学？

困了，回头好好学习，明儿天天向上。

荀:
这种唤醒的词典知识，主要是利用词搭配信息

李：
那当然。

荀:
李老师休息吧，明天搬着板凳听你们聊，到了关键地方了

白:
为了保证WSD和Matcher之间在大概率情形下都是串行的，被WSD压抑的候选都休眠。此时流程是线状的。为了对付有迹象可察的小概率情形，休眠的候选可以唤醒。此时流程是闭环。

李：
前半段懂了，后半段，闭环是什么？

白:
唤醒

李：
线状是说的 deterministic 吗？

荀:
如果被唤醒，wsd也会跟着改变

李：
唤醒在我看来就是局部重组。nondeterministic 是从一开始就留多条路径，唤醒是在一开始不留多条的情况下，activate 另一条子路径，摧毁现下的子路径。

荀:
如果是局部的，就是打补丁的机制。白老师的闭环是指唤醒和wsd联动？

李：
唤醒都是打补丁。
既然一开始不留多条路径，那么怎么唤醒另一条路径了？

白:
局部局到什么程度是不以主观意志为转移的，可以理解为一种lazy的、保相谐性的subcat传播。

李：
我这里的诀窍就在，开始不在实现的 parse 中留，但词典里面的潜在路径的种子还留着，因此词驱动可以唤醒它。

白:
唤醒的不是路径，而是词典的pos候选

李：
唤醒了不同的萝卜

荀:
唤醒了另外一个候选，也就是到引导不同的subcat路径，可以这样理解吗？

李：
不就等价于唤醒了不同的路径吗？因为萝卜换了，原来的萝卜所填的路径就废了，新的萝卜要填的坑也变了，新的路径也就唤醒（新生）了。

荀:
填好的萝卜重新拔出来

李：
对呀。
这是摧毁无效路径 为建立更好的路径做准备。

荀:
如果wsd需要频繁被唤醒，唤醒的中间件压力山大。

李：
wsd 常常被夸大 实际中没那么大。太细的也不用做。

白:
所以三省吾身。

@荀 是这样子：休眠唤醒的动态工作空间专属wsd，它的范围是受控的，但之间逻辑较为复杂。中间件依据ontology和大数据统计结果，体现了subcat嵌入，是相对静态的，一组二元关系一个查询，彼此逻辑独立，既为wsd服务也为matcher服务，因此根本不在乎频繁程度。忙了就并发，不会惹到谁。wsd局限到一个句子，最多加上定宽滑动窗口的语境变量，工作负载是可控的。wsd和matcher一切照章行事，没有任何语言知识和常识，只有“过程性的、机械化的”程序。语言知识和常识都在词典和subcat嵌入里。

荀:
明白了，就是根据需要，随时被调用的“语言模型”，这个语言模型嵌入了ontology和subccat信息的RNN网络。

白:
今天说多了，打住。

荀:
开启了一个大话题

白:
过去统计意义上的“语言模型”是一勺烩的，所以难以精准地把有结构潜质的二元关系筛选出来。一旦仅针对这样的二元关系上统计手段，前途就一片光明了。萝卜是第一性的，路径是第二性的。

王：
@白硕 wsd和matcher一切照章行事，没有任何语言知识和常识，只有“过程性的、机械化的”程序。语言知识和常识都在词典和subcat嵌入里。

白:
怎么

王:
白老师，要走”小程序，大词典”路线？

白:
对，小程序、大词典、特大ontology、数据被ontology揉碎消化。前面再加个“零规则”。

王:
不知是否每个词做为Agent？然后多个Agent之间相互自适应？

白:
没那么自主

王:
请教白老师，何为“零规则”？是预留的待扩展的规则？还是压根就不用规则？

白:
是根本不要规则。

我们先解决“谁和谁发生关系”而不必具体明确“是何种关系”，只笼统地分成：“a是b的直接成分”、“a是b的修饰成分”以及“a是b的合并成分”三种情况。

现在还都没说定性，只说定位，谁跟谁有关系。结论是，就这么糙的事儿，也得动用ontology。

李：
句法不必要太细。语义可以细，但那个活儿可以悠着点，做多少算多少。

白:
关系不对，上标签何用。标签可以是句法的，也可以是逻辑语义的

李：
句法的本身就模糊一些。很多语言的主语与谓语是有一致关系的。这就给“主语”这个标签一个独立的句法层面的支持。虽然细究起来，这个所谓的主语，可能是 human agent，也可能是 instrument。

白:
粗粒度不等于错位。位置对上了不知什么标签这叫粗粒度。
位置不对叫啥

李：
位置不对就是父子认错了。这是最大的错，皮之不存，句法或逻辑语义标签也就谈不上。

看看这个： 这些纸我能写很多字

“这些纸”无从着落了。那就用 Topic 或 Next 耍个流氓：它们的句法意义与句首状语（全句状语）差不多，至于是什么状语（时间、地点、让步、工具、结果、原因、。。。），这是逻辑语义的标签。想做的话，让后面的语义模块去做：这些纸 Next 能写字。

其实拿目前的 parser parse 一下大数据，对于 Next 前后的词做一个统计，基本肯定可以挖掘出不少强搭配或弱搭配来。Next 虽然标签模糊，它把有关系的 tokens 的距离拉近了，虽然句法不知道是何种关系。

白:
暂不确定逻辑语义我赞同，但没有坑就不填坑，没有加号就不修饰也是铁律。于是需要一个节点做这个对接。大数据中这个节点有实例支持，引进就更理直气壮而已。
在这个阶段根本没有人去做逻辑语义标记。

李：
逻辑语义是 semantic parsing 的目标；syntactic parsing 可以不管。乔老爷说句法独立于语义肯定是有瑕疵的，但句法确实有相当地独立于语义的部分。这个独立性在形态语言中表现很充分，到了裸奔的汉语表现就差一些。但也不是一点独立性也没有。换句话说，总可以把一部分现象看成是纯粹句法的现象，不牵扯语义，也可以一路走下去。

白:
现在我是在定义syntax，自己定义的东东，自己要认账。
说好了不耍流氓的地方，就是坚决不耍，可以耍的地方也一定当仁不让。

李：
系统内部怎么协调，没法说对错优劣。我是要说，耍流氓也有其耍的道理。不耍，又不想牵扯太多语言外知识，那就只有断链。耍流氓比断链强。

白:
引入虚节点，有大数据背书，挺好。同样达到不断链的效果。

李：
还有一个更重要的特点是：句法模块与语义模块分开，有开发和维护的便利。比一锅炒感觉有优势。

白:
相谐问中间件可以，补虚节点问中间件当然也可以，毕竟大多数情况不需要补。
wsd和matcher现在连一点语言学知识都没有，是最不一锅炒的架构了。
内事不决问词典，外事不决问数据

李：
开发一个模块有两个模式，一个是轻装粗线条，knowledge-poor。另一种是细线条，knowledge intensive，前者的好处不仅在轻装，不仅在覆盖面好，而且在鲁棒性好。后者则是精准度好，而且可以聚焦去做，一步一步 peace-meal 地去做。很多人做了前者，但是带来了一个巨大的伪歧义泛滥的问题。我们做到了前者，而且基本对伪歧义免疫，这算是一个成就。至于后者，那是一张无边无际的网，不急，慢慢做。

白:
大部分不鲁棒都是伪语序造成的。让萝卜和坑自由恋爱，是鲁棒性的最好体现。

李：
白老师主张先不利用语序作为句法的制约，而是立足于词典的对萝卜的预期，以及查与周边 candidates 在中间件表达出来的语义相谐度。这样做自然是增加了鲁棒性（我以前提过，汉语实词之间的语序灵活到了超出想象），但同时也隐隐觉得，不问语序也可能是自废武功的不必要的损失。其实是可以把语序作为一个option加入坑的预期的。

【相关】

【李白之15：白老师的秘密武器探幽】

【李白对话录系列】

中文处理

Parsing

【置顶：立委NLP博文一览】

《朝华午拾》总目录

李：
圣诞醒来 心里突然冒出一句貌似开涮海龟海鸥的打油：

我的人回来了
可心还在路上

问一下： 谁回来了？“我”还是“我的人”？
谁的心在路上？“我”的心还是“我的人”的心？
或者 either？

我的身子虽然回家 可【我的】心还在路上
我的【心上】人虽然回家 可【他的】心还在路上

白:
人，是身的借喻，我的人=我的身。心=我的心
伟哥这个浪漫语境，更需要一个带优先机制的逻辑门了。

“我的人吃饱了，可细胞还饿着。”
“我的人吃饱了，但马还饿着。”

前者“人”指身体，后者“人”指将士。

开学了。孩子们人回到了学校，心还在假期。
某某，人虽然死了，思想还在影响着我们。

所以，“人”成为一个有坑名词，至少在述及灵肉分离的语境里，还是需要的。
是一个body-part，不知道这个义项知网收了没？

李：
除去“的”就不浪漫了 诗意了无：

我人回心没回

其实 身心分离 或 人心分裂 乃是诗歌、心理学和宗教哲学的永恒话题。形神合一反而难得，需要修为或运气。

白：
“人走了，香味还在房间里飘荡”

李：
那是仙女 不是人。
换成仙女 还可以说义项问题吗，更像是是临时延伸或活用。

白:
给“人“老老实实加个body-part义项是其一，语境驱动的逻辑门是其二，一个都不能少。在人与body-part对举的语境下，这个body-part义项被激活，可以颠覆标配。

李：
人 的默认（标配）是 body 而不是 heart，不说 body 就是 body，但 heart 永远不可省去：

仙女走了 香味还在
== 仙女 body 走了

某某同志与我们永别了
== 某某同志 body 永别了
他的精神长存
说明其灵魂没有走

滑稽的是:

某同志与我们永别了
但他的body永存
-- 在水晶棺里
红场或什么场。
此 body 非彼 body

顺便一提，功过不论，留存 body 的做法感觉是违背伟人的意愿。为了留存 body，还要掏心掏肺 去除五脏 想想都让人悚然。

白:
楼歪了……

李：
歪了；也没歪。
“掏心掏肺” 语义计算要休眠唤醒，政治敏感性先放一边。
这种语义计算需要进入成语词典吗？一般认为不要进词典 但是不进入词典先休眠起来，想唤醒都没有依据。所谓词驱动休眠唤醒 就是把休眠的可能性预先隐藏在词典深处。细想起来这是一个巨大的虽然是可行的工程。起码 每个成语都要增加一个或数个字面意义的种子 。没有种子，context 再温床 另一种计算也难发芽。context 有 五脏 可以激发埋在种子里面的字面意义的心和肺，从而唤醒了 “掏” 的原义 而不是标配义。但这类激发不是驱动 驱动应该从成语本身发端 而不是从 context 发端。否则太发散 难以想象有实现唤醒的可行性。

【相关】

【李白对话录系列】

Parsing

【置顶：立委NLP博文一览】

《朝华午拾》总目录

我:
domain 很有意思，今天 study 客服的数据，发现我们现系统 tag “loyal customer” as positive，但实际上几乎总是 negative 抱怨。因为很多人（包括我自己，譬如跟电话公司打交道的时候）的抱怨总是这样开始：
I am your loyal customer for n years, blah blah [complaints: how can u treat me like that]

还有一些有意思的发现：new 这个词并不是啥褒义词，第一即便原义有一丝褒，但太弱，用得也太多。更重要的是，这个词最常出现在 promotion 的广告里面。客户情报里面很少用它表示褒义。那么 brand-new 呢？似乎稍微褒一些，但也很灰色，放过它可能更好。还有一个词 叫 available，以前以为是好话，其实用起来很 tricky：说 no Pepsi available， 不是说的 Pepsi 的坏话，很可能是说的好话 ，抱怨的是这么好的东西怎么没提供呢。诸如此类，不看 data 不知道。

白:
这么好的数据，不神经，糟蹋了

我:
大而言之，语义是泥坑；具体到 sentiment，几乎就是粪坑了。跳进去不仅可能被淹死，还会被呛死。

我等着神经在shentiment上来一个绝活。

目前为止，对手的 sentiment 全部用的学习，用没用 神经 就不知道了，但他们的 data quality 实在不敢恭维。

还有就是： I would kill for Pepsi，这样的说法是强烈的褒义：NND 为了 Pepsi 让我杀人都可以。这个倒不难逮住，无论是 pattern 去拿它，还是数据够了去训练出来它。

白:
可怜的米国人

我:
记得还有这么一句： Hell no man, pepsi or die. 这是对 pepsi 的极度褒扬：没 Pepsi 毋宁死（与自由同价： 不自由 毋宁死）宋柔:

宋:
@wei 可见，要做情感分析，对于所处理的语言必须有相当强的语感。国内的人做英语的情感分析恐怕不容易。

我：
昨天说了 sentiment data 的貌似诡异的事儿，明明说的是正面的词“loyal customer”，却几乎总是抱怨，至少是客服领域：原来人在表达情绪的时候，不仅会正话反说，而且还会先退一步。

今天再说一个案例：care about（关心） 一般认为是正面的动词，甚至关心钱财，从正面角度也可以说明这个企业懂得为 stake holders 创造价值，可能是一个兢兢业业的好企业吧。但是在客服领域，绝大多数场合，这却是抱怨的开始。

"All AT&T cares about is money. Worst service ever."
AT&T is one of the most GREEDY companies I have ever saw. All they care about is the $$$$$.
"there aledged customer service is beyond ridiculous, they seem to care more about being paid than helping there customers"

等于说：你他妈只认钱，贪得无厌。

顺便一提，我也是 AT&T 很多年的“loyal”customer，完全认同上面的抱怨。看这家公司的账单，那真是跟天书一样，变着法儿跟你要钱。昨天来账单，我的电视涨了近30元，原来是 promotion 到期了，我就 call 他们，说，我们基本不看电视，时间和视屏都耗在网上了，不过是多年的习惯而已，你怎么一个月 charge 我 80 多刀，比互联网的 70 多刀的 charge 还大？你不给 discount，我就掐了电视算了。不过真要掐电视，领导怕不同意，如今的乐视盒子小米盒子啥的，节目虽多，还是不那么灵光，有时连不上。

结果客服说，现下没有新的 promotion 可以提供 discount，不过不久会有。说你要是电视少看，那就降一级吧，从 TV-family 降级为 TV-basic，那个才 19 块钱，来 20多个频道，你们也该够了。于是，我就降级了，然后一查看，说是TV 19 块，其实是 50 多块。什么 HD 费10快，receiver 费 15 块，录像费，等等等等。这种企业真该死。可是美国电信企业，好人不多，也就懒得挪动了。互联网+++ 再发达一些，这 cable TV 就该自生自灭了。

【相关】

舆情挖掘

【置顶：立委NLP博文一览】

《朝华午拾》总目录

我:
哥仨老革命在去 IBM 的 traffic 中 去大名鼎鼎的沃森（Watson）系统探秘

洪:
讲者是这位印度籍女士：http://researcher.watson.ibm.com/researcher/view.php?person=us-vibha.sinha:

郭:
比较有意思的是她后面讲的三点:

1. LSTM based intent recognition and entity extraction

2. "tone" recognition
这里tone指的是从一句话（书面语）反应出的说话人的喜怒哀乐和处事方式等

3. personality recognition
主要基于心理学的分类，用200到2000条tweets训练

她重点强调的是，通过增加tone和personality的识别，人机对话可以有更高的可接受度。

我:
唐老师 诸位 汇报一下昨天的听闻。上面 郭老师也总结了几条，很好。我再说几点。
话说三位老革命慕名而去，这个 meet-up 一共才来了20几位听众吧 大概湾区此类活动甚多 marketing 不够的话 也难。据说北京的 AI 沙龙，弄个花哨一点的题目 往往门庭若市。

1. 没有什么 surprises 但参加沙龙的好处是可以问问题和可以听别人问问题，而主讲人常常在回答的时候给出一些书面没有的数据和细节。否则的话，各种资料都在网上（最后的 slide 给了链接），要写利人似的调研报告，只要不怕苦，有的是资料。

听讲的另一个好处是，主讲人事先已经组织好材料讲解，可以快速了解一个项目的概貌。

2. 特地替唐老师问了他钟情的 Prolog，问你们有用吗，在什么模块用。主讲人说，没有用。我说有报道说有用到。她说，她没听说，至少在她主讲的已经产品化的这个沃森 chatbot 的组建 toolkit 里面没有 Prolog。当然她不排除某个小组或个人 在沃森的某个项目或模块用到。IBM 对 AI 的投入增大，在沃森的名号下的各种研究项目和小组很多。

马:
我问过了IBM中国的，在沃森参加电视节目版本中没有用prolog，但是后续的版本中，确实用到了prolog

陈:
它是很多services构成，用不会奇怪，尤其是某些既有系统

我:

3. 现在不少巨头都在 offer 这样的 toolkit，问微软 offer 的 cortana 的 toolkit 与你们沃森的这套有啥不同。回答是，非常类似，不过她自认为沃森质量更好。亚马逊也有类似的 offer。

所以回来路上，我们就谈到这个 bots 遍地开花的场景。郭老师说，现如今谁要想做一个领域内的 bot，或自己的 app 做一个 bot 接口，根本就不需要编程。只要准备好领域的 experts，把数据准备好，用这些巨头的工具箱就可以构建一个出来。也一样可以 deploy 到 messenger 或嵌入其他场景，这几乎是一条龙的云服务。

当然 用这些服务是要交钱的，但主讲人说很便宜很便宜的，郭兄说，真用上了，其实也不便宜。便宜与否放一边，至少现如今，bots 的门槛很低，需要的不是软件人才，而是领域数据的人。于是，我看到一种前景，以前毕业即失业的语言学家、图书馆业人士，将来可能成为 AI 的主力，只有对数据和细节敏感的人，最终才是 AI 接口的血肉构筑者，反正架构是现成通用的。这个细想想是有道理的。这是沃森 API calls 的价格。

我:
这就回到我们以前议论过的话题。AI 创业，如果做平台或工具箱，初创公司怎么敌得过巨头呢？我觉得几乎是死路。

大而言之 做平台和工具箱创业的，历史上就没见过什么成功案例（不排除做了被收购那种，那也是“成功”，如果你的技术有幸被巨头看中：其实昨晚介绍的沃森系统的一个重要组件 AlchomyLanguage 就是收购的，洪爷知道收购的来路和细节）。

白:
麦当劳玩法，方便，质量可控，但绝非美食，虽然是“美”食。

我:
不错，这些巨头的 offerring 都是麦当劳式的流程。创业的空间，从工具角度，可以是中华料理的配方辅助工具之类。不过，还是那句话，最好绕过平台本身创业的思维，而是用巨头的工具或者自家建造匕首去做领域的 AI，这样的创业应该具有更大的空间和更多的可能性。

对于 NLP（AI之一种） 我写过 n 篇博文强调，所有的 offshelf 的平台和toolkit（譬如 历史悠久的GATE），甚至一个小插件（譬如 Brill Tagger or some Chinese word segmenter）都不好用。可以 prototyping 但如果稍微有点长期观点 要建一个大规模的NLP的应用，还是一切自家建造为好。当然，自家建造的门槛很高，多数人造不起，也没这个 architect 来指挥。但最终是，自家建造的胜出，从质量上说（质量包括速度、鲁棒性、精度广度、领域的可适应性等关键综合指标）。

巨头的工具箱的产品 offers 一开始也不赚钱，但他们的研发积累已经做了，且还在不断投入，不产品化成工具箱不是傻瓜吗，赚多少算多少。如果真到了AI bots 遍地开花的时候，他们凭借巨大的平台优势，赚钱也是可能的。小公司这条路没门吧。如果你的 offer 的确 unique，譬如是中华料理，譬如是伟哥的 parsing，你可能会吸引一批使用者。但想赚钱必须有规模，而 component tech 或平台工具之类，在小公司的环境中，是成不了规模的。所以不要想赚钱的事儿。

赚钱靠的是产品，而不是工具，这是AI创业铁律。

当然，通过平台或工具打出影响，做 marketing，曲线救国创业，另当别论。
回到 meet-up：

4. bots 构建的核心当然是 conversations 的训练工具。IBM沃森的工具用的是深度神经。

对于 bots，input 是确定的，就是用 bots 的人的输入。自然语言的语音也好 文字也好，语音反正也要转化为文字 所以我们面对的就是人机接口中的“人话”，理论上无止境 千变万化。

bots 的 output 呢？

在目前的框架里，在绝大多数实际场景，这个 output 都是以极为有限的集合
最典型的案例是为 apps（天气、股票、时间之类） 做 bots 作为 apps 的人机接口，
其 output 就是 app 里面的 commands 集合。于是 bot 产品定义为从无限到有限的映射，这是一个典型的分类场景。于是沃森提供这个深度学习为基础的工具帮助你训练你所需要的 classifiers，这是标准做法 无甚新意。

数据越多，分类质量越好。千变万化的死敌是稀疏数据。好在对于 bots，数据的收集会是一个边使用边加强的过程。如果你的 bots 开始有用户，你就形成了正循环，数据源源而来，你不断打磨、训练，这些都是可以 streamline 的流水作业，就越来越好。Siri 如此，Echo 也如此。

白:
分类本身是不带参数的，而bots的应对必须是带参数的，这是硬伤。
拿分类来做对话是看得到天花板的。

我:
I cannot agree more :=)

这里其实是有历史渊源的。IBM 做问答，一直是把问题简化为分类。18 年前我们在第一次 QA 竞赛（TREC-8）中交流 就是如此，这么多年这个核心做法一直不变。当时我们的QA成绩最好，得分66%，沃森的系统印象是40%左右，他们的组长就追在后面问，我们思路差不多呀，都是 question intents（我们叫 asking points，比多数 intents 其实更聚焦），外加 Named Entity 的support。我说我们还用到了语言结构啊。

直到今天他们仍然是没有句法分析，更甭提深度分析。他们当年的 QA 就是基于两点：
1. 问句分类：试图了解 intents；2. NE。有了这两条，通过 keywords 检索作为 context，在大数据中寻找答案，对于 factoid questions 是不难的（见【立委科普：问答系统的前生今世】）。这就是沃森打败人类的基本原理，一点也不奥秘，从来没有根本改变。现在这一套继续体现在其 bots 工具箱 offering 里面。

洪:

昨晚Watson讲座听，
今早广告已跟进。
IBM可真下本，
今天我试Bluemix云。

我:
2.  因此 conversations 训练，其核心就是两条：一个是 intents classification （这个 intents 是根据 output 的需求来定义的），一个 NE，不过 NE 是他们已经训练好的模块（NE有一定的domain独立性），用户只是做一些微调和增强而已。

顺便插一句，这几天一直在想，AI 现在的主打就是深度神经，所有的希望都寄托在神经上。但无论怎么神经，都不改 supervised learning 的本性：所以，我的问题是：你怎么克服缺乏带标大数据的知识瓶颈？

ok 你把机器翻译玩转了。因为 MT 有几乎无限的 “自然” 带标数据（其实也不是自然了，也是人工，幸运的是这些人力是历史的积累，是人类翻译活动的副产品，是不需要开发者花钱的 free ride）。可其他的 ai 和 nlp 应用呢，你还可以像 MT 这样幸运 这样享用免费午餐吗？

现在想，紧接着 MT 的具有大数据的热门应用是什么？非 bots 莫属。
对于 bots，数据已经有一定的积累了，其最大的特点在于，bots 的使用过程，数据就会源源而来。问题是 这些数据是对路的，real life data from the field，但还是不带标啊。所以，bots 的前景就是玩的跟数据打仗：可以雇佣人去没完没了地给数据做标注。这是一个很像卓别林的【摩登时代】的AI工厂的场景，或者是列宁同志攻打冬宫的人海战术。看上去很笨，但可以确定的是，bots 会越来越“智能”，应对的场景也越来越多。应了那句老话，有多少人工，就有多少智能。然而，这不是、也不应该是 唯一的克服知识瓶颈的做法。

毛:
嗯，有多少人工，就有多少智能。这话说得好。

我:
但这个景象成为常规 也不错 至少是帮助解决了一些白领就业。是用高级的专家知识去编写规则来提高系统质量，还是利用普罗标注去提高质量，从帮助就业和维稳角度看，几乎蛮力似的深度神经对于标注大数据的无休止的渴望和胃口，对于社会似乎更为有利。为了社会稳定和世界和平，我们该看好这种蛮力。我们做深度分析和理解的专家，试图尽可能逼真地去模拟人的智能过程，但对蛮力也应该起一份敬意。

将来的AI，什么人都可做：1. 你发现一个领域的 AI 需求； 2. 你雇佣一个对这个需求可以形式化定义的设计家； 3. 你调用巨头的一个通用的 AI 工具箱（譬如 TensorFlow） 或面向专项产品的工具箱（譬如 bot 的沃森工具箱） 4 你雇佣一批失业但受过教育的普罗，像富士康一样训练他们在流水线上去根据设计家的定义去标注数据、测试系统，你于是通过 AI 创造了价值，不排除你的产品会火。因为产品火不火已经不是技术了，而是你满足需求的产品角度。

3. 但是 正如白老师说的 这种用分类来简化问题的 AI 产品化，走不远。它可能满足一些特定领域的特定的需求 但是后劲不足是显然的。其中一个痛点或挑战就是，这种东西走不出三步，三步以上就抓瞎。如果你的应用可以在三步之内就基本满足需求，没问题。

bots 最显然的有利可图的应用场景是客服。一般而言，bots 取代和补充客服是大势所趋，因为客服的知识资源和记忆，根本没法与我们可以灌输给 bots 的知识来相比。利用知识去回答客户疑问，人不如机，是可以想见的。但是 观察一个好的客服与客户的交互 可以发现，三步的交流模型是远远无法满足稍微复杂一点的场景的。三步的说法是一个比喻，总之是目前的工具箱，对于较长时期的对话，还是束手无策。

bots 对用户话语的理解简化为 classification，以此为基础对用户的回答就不是那么简单了。目前提供的做法是：因为 intents 是有限的集合，是 classification 的结果，那么对于每一个 intent 可以预知答案（存在数据库的 hand-crafted text snippet）或回应（譬如展示一个图，譬如天气app的今日天气图表)。 这些预制的答案，听上去非常自然、生动甚至诙谐，它们都是领域专家的作品。且不说这些预制的 snippets，如何根据classification hierarchy 本身需要做不同组装，在存于数据库里面的核心应答的预制以外，还可以加上情感的维度，还可以加上 personalized 的维度，这些都可以使得对话更加人性化、自然化，但每加一个维度就意味着我们开始接近组装式策略的组合爆炸后果。三步、三维以上就无法收拾。

我问主讲人，你的这些预先制定好的应答片段，按照你的工具的组装方式，不就是一个 decision tree 吗？回答是，的确，就是一个 decision tree 的做法。然后她说，有不少研究想突破这种应答模式，但都是在探索，没有到可以产品化工具化的阶段。

郭老师说，谁要是有本事把人机的 “自然对话”能够延长到 20 分钟，换句话说 就是突破图灵测试，谁就是 AI bots 的真正破局者。如果你证明你能做到，巨头会抢着来高价收购你的。这是所有做 bots 的所面临的共同挑战。

据说小冰最高记录是与单一的人谈了九个小时的心。但那不是真正的突破，那是遇到了一个异常人类。正常的人，我的体会是两分钟定律，你与小冰谈话 超不过两分钟。我试过多次，到了两分钟，它所露出来的破绽就让你无法忍受，除非自己铁心要自我折磨。其实 工业界要求的连续对话，不是小冰这种闲扯。而是针对一个稍微复杂一点的任务场景（譬如订票）如何用自然对话的假象去把相关的信息收集全，来最大限度地满足客户需求。

累了，先笔记和评论如上。其余还有一些有趣的点儿可以讨论，以后再说。这是交给我们唐老师的作业。

郭:
Amazon’s $2.5M ‘Alexa Prize’ seeks chatbot that can converse intelligently for 20 minutes

洪:
亚马逊正设大奖，
chatbot赛悬赏。
对话若超廿分长，
两半米粒到手上。// 2.5M

【相关】

【立委科普：问答系统的前生今世】

Amazon’s $2.5M ‘Alexa Prize’ seeks chatbot that can converse intelligently for 20 minutes

《微软小冰，两分钟定律》

中文处理

Parsing

【置顶：立委NLP博文一览】

《朝华午拾》总目录

白:
“他分分钟就可以教那些不讲道理的人做人的道理。”

我:

一路通，直到最后的滑铁卢。
定语从句谓语是“做人”而不是“可以教”，可是定语从句【【可以教。。。的】道理】与 vp定语【【做人的】道理】，这账人是怎么算的？

白：
还记得“那个小集合”吗？sb 教 sb sth，坑已经齐活儿了
“道理”是一般性的，定语是谓词的话一定要隐含全称陈述，不能是所有坑都有萝卜的。当然这也是软性的。只是在比较中不占优而已。单独使用不参与比较就没事：“张三打李四的道理你不懂”就可以，这时意味着“张三打李四背后的逻辑你不懂”。
“他分分钟就可以把一个活人打趴下的道理我实在是琢磨不透。”这似乎可以。

我:
教 至少两个 subcats：
教 sb sth
教 sb todo sth

白:
这个可以有
刚刚看到一个标题起：没有一滴雨会认为自己制造了洪灾。
这个句法关系分析的再清楚，也解释不了标题的语义。

宋:
有意思。

我:
教他
教他做人
教他道理
教他做人的道理
教他的道理
教他做人的往事儿

这个 “道理” 和 “往事”，是属于同一个集合的，我们以前讨论过的那个集合，不参与定语从句成分的 head n。

白:
对

我:
这个集合里面有子集 是关于 info 的，包括 道理 新闻 公告 往事。。。

白：
但是于“道理”而言，坑不满更显得有抽象度。是没“提取”，但坑不满更顺更优先，因为隐含了全称量词。

我:
就是说 这个集合里面还有 nuances 需要照顾。滑铁卢就在 “教他做人的往事儿” 上，照顾了它 就照顾不了 “做人的道理”。
就事论事 我可以词典化 “做人的道理”，后者有大数据的支持。

白:
这可是能产的语言现象。
试试这个：“你们懂不懂做人要低调的道理？”

我:
我试试 人在外 但电脑带了 只好拍照了

你们懂不懂道理，这是主干
什么道理？
要低调的道理。
谁要低调？
你们。
懂什么类型的道理？
做人的道理。
谁做人？
你们。
小小的语义计算图谱 ，能回答这么多问题 ，这机器是不是有点牛叉？

白:
图上看，“要低调”是“懂道理”的状语而不是“道理”的定语？

我:
这个是对的，by design。但我们设计vn合成词的时候，我们要求把分离词合成起来。如果 n 带有定语，合成以后就指向 合成词整体。这时候 为了留下一些痕迹，有意在系统内部 保留定语的标签，以区别于其他的动词的状语修饰语。否则，“懂【要低调的】道理” 与 “【要低调的】懂道理”，就无法区分了。这样处理 语义落地有好处 完全是系统内部的对这种现象的约定和协调 system internal。定语 状语 都是修饰语 大类无异。

白:
“做人要低调”是一个整体，被拆解了。逻辑似乎不对。
拆解的问题还没解决：不管x是谁，如果x做人，x就要低调。
两个x是受全称量词管辖的同一个约束变元。
@宋 早上您似乎对“没有一滴雨会认为自己制造了洪灾”这个例子有话要说？

宋:
@白硕 主要是觉得这句话的意思有意思。从语义分析看应该不难，因为这是一种模式：没有NP V。即任何x，若x属于NP，则否定V(x)。

白:
首先这是一个隐喻，雨滴是不会“认为”如何如何的，既然这样用，就要提炼套路，看把雨滴代换成什么：雨滴和洪水的关系，是天上的部分和地上的整体的关系，是无害无责任的个体和有害有责任的整体的关系。

“美国网约车判决给北上广深的启示”

洪:
中土NLP全家福，
烟台开会倾巢出。
语言架桥机辅助，
兵强马壮数据足。

中国nlp全家福啊@wei

白: 哈
李白无暇混贵圈，一擎核弹一拨弦。精灵解语多奇智，不是冤家不上船。

洪:
冤家全都上贼船，李白有事别处赶。天宫迄今无甚关，Alien语言亟需练。

我:
白老师也没去啊 敢情。
黑压压一片 吾道不孤勒。

【相关】

【李白对话录：RNN 与语言学算法】

【李白对话录：如何学习和处置“打了一拳”】

【李白对话录：你波你的波，我粒我的粒】

【李白对话录- 从“把手”谈起】

【李白对话录：如何学习和处置“打了一拳”】 

【李白对话录之六：NLP 的Components 及其关系】

乔姆斯基批判

[转载]【白硕 – 穿越乔家大院寻找“毛毛虫”】

泥沙龙笔记：parsing 是引擎的核武器，再论NLP与搜索

泥沙龙笔记：从 sparse data 再论parsing乃是NLP应用的核武器

【立委科普：NLP核武器的奥秘】

【立委科普：语法结构树之美】

【立委科普：语法结构树之美（之二）】

中文处理

Parsing

【置顶：立委NLP博文一览】

《朝华午拾》总目录

1. A pre-existence

The traditional question answering (QA) system is an application of Artificial Intelligence (AI).  It is usually confined to a very narrow and specialized domain, which is basically made up of a hand-crafted knowledge base with a natural language interface. As the field is narrow, the vocabulary is very limited, and its pragmatic ambiguity can be effectively under control. Questions are highly predictable, or close to a closed set, the rules for the corresponding answers are fairly straightforward. Well-known projects in the 1960s include LUNAR, a QA system specializing in answering questions about the geological analysis on the lunar samples collected from the Apollo's landing on the Moon.  SHRDLE is another famous QA expert system in AI history, it simulates the operation of a robot in the toy building world. The robot can answer the question of the geometric state of a toy and listen to the language instruction for its operation.

These early AI explorations seemed promising, revealing a fairy-tale world of scientific fantasy, greatly stimulating our curiosity and imagination. Nevertheless, in essence, these are just toy systems that are confined to the laboratory and are not of much practical value. As the field of artificial intelligence was getting narrower and narrower (although some expert systems have reached a practical level, majority AI work based on common sense and knowledge reasoning could not get out beyond lab), the corresponding QA systems failed to render meaningful results. There were some conversational systems (chatterbots) that had been developed thus far and became children's popular online toys (I remember at one time when my daughter was young, she was very fond of surfing the Internet to find various chatbots, sometimes deliberately asking tricky questions for fun.  Recent years have seen a revival of this tradition by industrial giants, with some flavor seen in Siri, and greatly emphasized in Microsoft's Little Ice).

2. Rebirth

Industrial open-domain QA systems are another story, it came into existence with the development of the Internet boom and the popularity of search engines. Specifically, the open QA system was born in 1999, when the TREC-8 (Eighth Text Retrieval Conference) decided to add a natural language QA track of competition, funded by the US Department of Defense's DARPA program, administrated by the United States National Institute of Standards and Technology (NIST), thus giving birth to this emerging QA community.  Its opening remarks when calling for the participation of the competition are very impressive, to this effect:

Users have questions, they need answers. Search engines claim that they are doing information retrieval, yet the information is not an answer to their questions but links to thousands of possibly related files. Answers may or may not be in the returned documents. In any case, people are compelled to read the documents in order to find answers. A QA system in our vision is to solve this key problem of information need. For QA, the input is a natural language question, the output is the answer, it is that simple.

It seems of benefit to introduce some background for academia as well as the industry when the open QA was born.

From the academic point of view, the traditional sense of artificial intelligence is no longer popular, replaced by the large-scale corpus-based machine learning and statistical research. Linguistic rules still play a role in the field of natural language, but only as a complement to the mainstream machine learning. The so-called intelligent knowledge systems based purely on knowledge or common sense reasoning are largely put on hold by academic scholars (except for a few, such as Dr. Douglas Lenat with his Cyc). In the academic community before the birth of open-domain question and answering, there was a very important development, i.e. the birth and popularity of a new area called Information Extraction (IE), again a child of DARPA. The traditional natural language understanding (NLU) faces the entire language ocean, trying to analyze each sentence seeking a complete semantic representation of all its parts. IE is different, it is task-driven, aiming at only the defined target of information, leaving the rest aside.  For example, the IE template of a conference may be defined to fill in the information of the conference [name], [time], [location], [sponsors], [registration] and such. It is very similar to filling in the blank in a student's reading comprehension test. The idea of task-driven semantics for IE shortens the distance between the language technology and practicality, allowing researchers to focus on optimizing tasks according to the tasks, rather than trying to swallow the language monster at one bite. By 1999, the IE community competitions had been held for seven annual sessions (MUC-7: Seventh Message Understanding Conference), the tasks of this area, approaches and the then limitations were all relatively clear. The most mature part of information extraction technology is the so-called Named Entity (NE tagging), including identification of names for human, location, and organization as well as tagging time, percentage, etc. The state-of-the-art systems, whether using machine learning or hand-crafted rules, reached a precision-recall combined score (F-measures) of 90+%, close to the quality of human performance. This first-of-its-kind technological advancement in a young field turned out to play a key role in the new generation of open-domain QA.

In industry, by 1999, search engines had grown rapidly with the popularity of the Internet, and search algorithms based on keyword matching and page ranking were quite mature. Unless there was a methodological revolution, the keyword search field seemed to almost have reached its limit. There was an increasing call for going beyond basic keyword search. Users were dissatisfied with search results in the form of links, and they needed more granular results, at least in paragraphs (snippets) instead of URLs, preferably in the form of direct short answers to the questions in mind.  Although the direct answer was a dream yet to come true waiting for the timing of open-domain QA era, the full-text search more and more frequently adopted paragraph retrieval instead of simple document URLs as a common practice in the industry, the search results changed from the simple links to web pages to the highlighting of the keywords in snippets.

In such a favorable environment in industry and academia, the open-domain question answering came onto the stage of history. NIST organized its first competition, requiring participating QA systems to provide the exact answer to each question, with a short answer of no more than 50 bytes in length and a long answer no more than 250 bytes. Here are the sample questions for the first QA track:

Who was the first American in space?
Where is the Taj Mahal?
In what year did Joe DiMaggio compile his 56-game hitting streak?

3. Short-lived prosperity

What are the results and significance of this first open domain QA competition? It should be said that the results are impressive, a milestone of significance in the QA history. The best systems (including ours) achieve more than 60% correct rate, that is, for every three questions, the system can search the given corpus and is able to return two correct answers. This is a very encouraging result as a first attempt at an open domain system. At the time of dot.com's heyday, the IT industry was eager to move this latest research into information products and revolutionize the search. There were a lot of interesting stories after that (see my related blog post in Chinese: "the road to entrepreneurship"), eventually leading to the historical AI event of IBM Watson QA beating humans in Jeopardy.

The timing and everything prepared by then from the organizers, the search industry, and academia, have all contributed to the QA systems' seemingly miraculous results. The NIST emphasizes well-formed natural language questions as appropriate input (i.e. English questions, see above), rather than traditional simple and short keyword queries.  These questions tend to be long, well suited for paragraph searches as a leverage. For competition's sake, they have ensured that each question asked indeed has an answer in the given corpus. As a result, the text archive contains similar statements corresponding to the designed questions, having increased the odds of sentence matching in paragraph retrieval (Watson's later practice shows that from the big data perspective, similar statements containing answers are bound to appear in text as long as a question is naturally long). Imagine if there are only one or two keywords, it will be extremely difficult to identify relevant paragraphs and statements that contain answers. Of course, finding the relevant paragraphs or statements is not sufficient for this task, but it effectively narrows the scope of the search, creating a good condition for pinpointing the short answers required.  At this time, the relatively mature technology of named entity tagging from the information extraction community kicked in.  In order to achieve the objectivity and consistency in administrating the QA competition, the organizers deliberately select only those questions which are relatively simple and straightforward, questions about names, time or location (so-called factoid questions).  This practice naturally agrees with the named entity task closely, making the first step into open domain QA a smooth process, returning very encouraging results as well as a shining prospect to the world. For example, for the question "In what year did Joe DiMaggio compile his 56-game hitting streak?", the paragraph or sentence search could easily find text statements similar to the following: "Joe DiMaggio's 56 game hitting streak was between May 15, 1941 and July 16".  An NE system tags 1941 as time with no problem and the asking point for time in parsing the wh-phrase "in what year" is also not difficult to decode. Therefore, an exact answer to the exact question seems magically retrieved from the sea of documents to satisfy the user, like a needle found in the haystack. Following roughly the same approach, equipped with gigantic computing power for parallel processing of big data, 11 years later, IBM Watson QA beat humans in the Jeopardy live show in front of the nationwide TV audience, stimulating the entire nation's imagination with awe for this technology advance.  From QA research perspective, the IBM's victory in the show is, in fact, an expected natural outcome, more of an engineering scale-up showcase rather than research breakthrough as the basic approach of snippet + NE + asking-point has long been proven.

A retrospect shows that adequate QA systems for factoid questions are invariably combined with a solid Named Entity module and a question parser for identifying asking points.  As long as there is an IE-indexed big data behind, with information redundancy as its nature, factoid QA is a very tractable task .

4. State of the art

The year 1999 witnessed the academic community's initial success of the first open-domain QA track as a new frontier of the retrieval world.  We also benefited from that event as a winner, having soon secured a venture capital injection of $10 million from the Wall Street. It was an exciting time shortly after AskJeeves' initial success in presenting a natural language interface online (but they did not have the QA technology for handling the huge archive for retrieving exact answers automatically, instead they used human editors behind the scene to update the answers database).  A number of QA start-ups were funded.  We were all expecting to create a new era in the information revolution. Unfortunately, the good times are not long, the Internet bubble soon burst, and the IT industry fell into the abyss of depression.  Investors tightened their monetary operations, the QA heat soon declined to freezing point and almost disappeared from the industry (except for giants' labs such as IBM Watson; in our case, we shifted from QA to mining online brand intelligence for enterprise clients). No one in the mainstream believes in this technology anymore. Compared with traditional keyword indexing and searching, the open domain QA  is not as robust and is yet to scale up to really big data for showing its power. The focus of the search industry is shifting from depth back to breadth, focusing on the indexing coverage, including the so-called deep web. As the development of QA systems is almost extinct from the industry, this emerging field stays deeply rooted in the academic community, developed into an important branch, with increasing natural language research from universities and research labs. IBM later solves the scale-up challenge, as a precursor of the current big data architectural breakthrough.

At the same time, scholars begin to summarize the various types of questions that challenge QA. A common classification is based on identifying the type of questions for their asking points.  Many of us still remember our high school language classes, where the teacher stressed the 6 WHs for reading comprehension: who / what / when / where / how / why. (Who did what when, where, how and why?)  Once answers to these questions are clear , the central stories of an article are in hands. As a simulation of human reading comprehension, the QA system is designed to answer these key WH questions as well. It is worth noting that these WH questions are of different difficulty levels, depending on the types of asking points (one major goal for question parsing is to identify the key need from a question, what we call asking point identification, usually based on question parsing of wh-phrases and other question clues). Those asking points corresponding to an entity as an appropriate answer, such as who / when / where, are relatively easy questions to answer (i.e. factoid questions). Another type of question is not simply answerable by an entity, such as what-is / how / why, there is consensus that answering such questions is a much more challenging task than factors questions.  A brief introduction to these three types of "tough" questions and their solutions are presented below as a showcase of the on-going state to conclude this overview of the QA journey.

What/who is X? This type of questions is the so-called definition question, such as What is iPad II? Who is Bill Clinton? This type of question is typically very short, after the wh-word and the stop word "is" are stripped in question parsing, what is left is just a name or a term as input to the QA system.  Such an input is detrimental to the traditional keyword retrieval system as it ends up with too many hits from which the system can only pick the documents with the most keyword density or page rank as returns.  But from QA perspective, the minimal requirement to answer this question is a definition statement in the forms of "X is a ...".  Since any entity or object is in multiple relationships with other entities and involved in various events as described in the corpus, a better answer to the definition question involves a summary of the entity with all the links to its key associated relations and events, giving a profile of the entity.  Such technology is in existence, and, in fact, has been partly deployed today. It is called knowledge graph, supported by underlying information extraction and fusion. The state-of-the-art solution for this type of questions is best illustrated in the Google deployment of its knowledge graph in handling queries of a short search for movie stars or other VIP.

The next challenge is how-questions, asking about a solution for solving a problem or doing something, e.g. How can we increase bone density? How to treat a heart attack?  This type of question calls for a summary of all types of solutions such as medicine, experts, procedures, or recipe.  A simple phrase is usually not a good answer and is bound to miss varieties of possible solutions to satisfy the information need of the users (often product designers, scientists or patent lawyers) who typically are in the stage of prior art research and literature review for a conceived solution in mind.  We have developed such a powerful system based on deep parsing and information extraction to answer open-domain how-questions comprehensively in the product called Illumin8, as deployed by Elsevier for quite some years.  (Powerful as it is, unfortunately, it did not end up as a commercial success in the market from revenue perspective.)

The third difficult question is why.  People ask why-questions to find the cause or motive of a phenomenon, whether an event or an opinion.  For example, why people like or dislike our product Xyz?  There might be thousands of different reasons behind a sentiment or opinion.   Some reasons are explicitly expressed (I love the new iPhone 7 because of its greatly enhanced camera) and more reasons are actually in some implicit expressions (just replaced my iPhone , it sucks in battery life).  An adequate QA system should be equipped with the ability to mine the corpus and summarize and rank the key reasons for the user.  In the last 5 years, we have developed a customer insight product that can answer why questions behind the public opinions and sentiments for any topics by mining the entire social media space.

Since I came to the Silicon Valley 9 years ago, I have been lucky, with pride, in having had a chance to design and develop QA systems for answering the widely acknowledged challenging questions.  Two products for answering the open-domain how questions and why-questions in addition to deep sentiment analysis have been developed and deployed to global customers.  Our deep parsing and IE platform is also equipped with the capability to construct deep knowledge graph to help answer definition questions, but unlike Google with its huge platform for the search needs, we have not identified a commercial opportunity to deploy that capability for a market yet.

This  piece of writing first appeared in 2011 in my personal blog, with only limited revisions since. Thanks to Google Translate at https://translate.google.com/ for providing a quick basis, which was post-edited by myself.  

[Related]

Http://en.wikipedia.org/wiki/Question_answering

The Anti-Eliza Effect, New Concept in AI

"Knowledge map and open-domain QA (1)" (in Chinese)

"knowledge map and how-question QA (2)"  (in Chinese)

【Ask Jeeves and its million-dollar idea for human interface in 】(in Chinese)

Dr Li’s NLP Blog in English

微信最近疯传人工智能新进展：谷歌翻译实现重大突破！值得关注和庆贺。mt 几乎无限量的自然带标数据在新技术下，似乎开始发力。报道说：

十年前，我们发布了 Google Translate（谷歌翻译），这项服务背后的核心算法是基于短语的机器翻译（PBMT:Phrase-Based Machine Translation）。

自那时起，机器智能的快速发展已经给我们的语音识别和图像识别能力带来了巨大的提升，但改进机器翻译仍然是一个高难度的目标。

今天，我们宣布发布谷歌神经机器翻译（GNMT：Google Neural Machine Translation）系统，该系统使用了当前最先进的训练技术，能够实现到目前为止机器翻译质量的最大提升。我们的全部研究结果详情请参阅我们的论文《Google`s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation》。

几年前，我们开始使用循环神经网络（RNN：Recurrent Neural Networks）来直接学习一个输入序列（如一种语言的一个句子）到一个输出序列（另一种语言的同一个句子）的映射。其中基于短语的机器学习（PBMT）将输入句子分解成词和短语，然后在很大程度上对它们进行独立的翻译，而神经机器翻译（NMT）则将输入的整个句子视作翻译的基本单元。

这种方法的优点是：相比于之前的基于短语的翻译系统，这种方法所需的工程设计更少。当其首次被提出时，NMT 在中等规模的公共基准数据集上的准确度，就达到了可与基于短语的翻译系统媲美的程度。

自那以后，研究者已经提出了很多改进 NMT 的技术，其中包括模拟外部对准模型（external alignment model）来处理罕见词，使用注意（attention）来对准输入词和输出词 ，以及将词分解成更小的单元应对罕见词。尽管有这些进步，但 NMT 的速度和准确度还没能达到成为 Google Translate 这样的生产系统的要求。

我们的新论文描述了怎样克服让 NMT 在非常大型的数据集上工作的许多挑战、如何打造一个在速度和准确度上都足够能为谷歌 用户和服务带来更好的翻译体验的系统。

来自对比评估的数据，其中人类评估者对给定源句子的翻译质量进行比较评分。得分范围是 0 到 6，其中 0 表示「完全没有意义的翻译」，6 表示「完美的翻译」。

............

使用人类评估的并排比较作为一项标准，GNMT 系统得出的翻译相比于之前基于短语的生产系统有了极大提升。

在双语人类评估者的帮助下，我们在来自维基百科和新闻网站的样本句子上测定发现：GNMT 在多个主要语言对的翻译中将翻译误差降低了 55%-85% 甚至更多。

今天除了发布这份研究论文之外，我们还宣布将 GNMT 投入到了一个非常困难的语言（汉语-英语）的翻译的生产中。

现在，移动版和网页版的 Google Translate 的汉英翻译已经在 100% 使用 GNMT 机器翻译了——每天大约 1800 万条翻译。GNMT 的生产部署是使用我们公开开放的机器学习工具套件 TensorFlow 和我们的张量处理单元（TPU：Tensor Processing Units），它们为部署这些强大的 GNMT 模型提供了足够的计算算力，同时也满足了 Google Translate 产品的严格的延迟要求。

汉语到英语的翻译是 Google Translate 所支持的超过 10000 种语言对中的一种，在未来几个月，我们还将继续将我们的 GNMT 扩展到远远更多的语言对上。

from 谷歌翻译实现重大突破！

作为老机译，不能不被吸引。准备小试一下这最新版的谷歌神经翻译。
此前试过谷歌在线翻译，总体不如百度，可现如今说汉语mt已经很神经了：深度神经，接近人类。我有几百篇待译 正好一试，先试为快。期待谷歌的神译。

董:
@wei 但愿不致让你失望。我曾半开玩笑地说：规则机译是傻子，统计机译是疯子，现在我继续调侃：神经机译是“骗子”（我绝不是指研发者）。语言可不是猫脸或马克杯之类的，仅仅表面像不行，内容也要像！

我：现在是见证奇迹的时刻：

The automatic speech generation of this science blog of mine is attached here, it is amazingly clear and understandable. If you are an NLP student, you can listen to it as a lecture note from a seasoned NLPer (definitely clearer than if I were giving this lecture myself with my strong accent).   More amazingly, the original blog was in Chinese and I used the newest Google Translate claimed to be based on deep learning using sentence-based translation as well as character-based techniques.  My original blog in Chinese is here, you can compare:【立委科普：自然语言系统架构简说】。

董老师，您是知道我的背景和怀疑的。但是，面对这样的进步，这种远远超出我们当初入行的时候可以想象的极限的自动翻译质量和鲁棒性，我们不得不，不得不，不得不叹服。

董:
用他们的术语叫“less adequate，but more fluent”。机译已经历了三次paradigm shift，当人们发现无论如何它只能是一种很好的信息处理工具，而无法取代人类翻译时，那就选取代价较少的。

我:
无论如何，这个小小的测试让我这个老机译有点瞠目结舌。还没有从这种冲击回过味来。当然，赶巧我测试的是正规文体，谈的又是电脑和NLP话题，肯定在语料库的涵盖范围内，撞上枪眼了。可比起此前我使用的前神经时代的谷歌SMT和百度SMT，这个飞跃式突破还是让人惊异的。向我们的神经同行致敬。这是一批绝顶聪明的疯子。

毛老，这是我对谷歌最近的 claim 的一个反馈。上次对他们的 parser 嘲笑了一通，这次对他们用同样技术带来的MT的突破，表达一下深深的敬佩。这种 contrast 不是我神经了，或分裂了，而是在 parsing，他们苦于没有自然带标数据，巧妇难为无米之炊，因此无法跟符号逻辑派比试。可是 MT 不同，几乎有无穷无尽的自然带标数据（人的翻译从来没有终止过，留下的对照译文浩如烟海）。

毛: @wei 这就是说，你服了基于神经元的MT，改变了自己的见解和主张？

我: 我服了，但没真地改变。

毛: 怎么说？

我:
无论多少门户之见，基本事实总应该看清吧。听一下上面列出的他们的 SMT 翻译，其流利程度和对我原文的忠实程度，已经超过了一个普通的人做翻译。因为一个口译如果不懂我这一行，我如果拿着这个讲稿讲课，让这样的 average interpreter 做现场翻译，他是比不过机器的，无论信还是达。（翻译高手不论。）这一点不得不服。另一方面，我以前说的，神经再深度，我看不出它在今后几年内可以赶上我的深度 parser，这一点还没改变。尤其是应对不同 domains 和 genres 的能力，他们不可能做到。因为在自然世界里面，没有带标的句法树，有的只是线性句子。而迄今看到的突破都是有监督的深度学习（supervised learning），离开海量带标数据（labeled data）就抓瞎了。

毛: 我被你弄糊涂了。你究竟是说哪一派强哪？@wei 究竟谁是世界第0？

我: parsing 我是第0，谷歌赶不上。MT 谷歌有重大突破，估计符号逻辑派的 MT 的日子不好过。

毛: 我问的是，MT谁是第0，不管用什么方法。

我: 这不是说，MT 规则系统就没有活路了，但是总体而言，SMT（statistical MT）占据上风的 trends 还在增强。

云: THKS. 我来试试能不能翻译我写的公司白皮书？

我:
你要是加一点译后人工编辑的话，我估计会很好的。再不要傻傻地从头请人工做翻译了。翻译公司如果不使用 MT 做底，将会被淘汰，成本上看很难存活。

董:
学习上，初二是一个分水岭，学科数量明显增多，学习方法也有所改变，一些学生能及时调整适应变化，进步很快，由成绩中等上升为优秀。但也有一部分学生存在畏难情绪，将心思用在学习之外，成绩迅速下降，对学习失去兴趣，自暴自弃，从此一蹶不振，这样的同学到了初三往往很难有所突破，中考的失利难以避免。
Learning, the second is a watershed, the number of subjects increased significantly, learning methods have also changed, some students can adjust to adapt to changes in progress, progress quickly, from the middle to rise to outstanding. But there are some students there is fear of hard feelings, the mind used in the study, the rapid decline in performance, loss of interest in learning, self-abandonment, since the devastated, so the students often difficult to break through the third day,

毛: 这翻译没什么好呀？

我:
要的就是这句话 🙂 @毛，需要一个对比，才能回答你的问题。

毛: 那就拿你的出来比比？

我: 我早就不做 MT 了，我是逃兵。近20年前，我就转移到信息抽取 IE（以及sentiment）挖掘了，这方面我有底气，不怕比。

刘：转：谷歌新版翻译有多神？英文教授亲证后告诉你...

我: thanks 似乎评论得比较中肯。对于口语现在肯定还不行，它的训练集一直对口语 cover 的很不够。以前测过，有些常用的简单的口语表达也弄错。不知这次这方面加强多少。

董老师上面给的那段谷歌翻译的段落，毛老说翻译不咋样。不过我做过MT多年，知道达到这一步其实是有很大进步的。以前的汉译英的不可读，到现在读起来大体能听得懂。这里面有很多的进步值得留意。

刘: @wei 转一个: 话说大数据干的一些事属于人工智能操练（不能用“研究”这词了）的范畴吗，那本来不就是传统计算机系的人干的？动不动嘲笑每开掉一个语言学家就往前多走几步这眼界太浅了

马: 在数据充足的领域，这几年DL方法都突飞猛进，我认识的好几个以前对DL有偏见的都多少改变了看法。在IR领域DL还基本不见功效，但也在慢慢渗透中。

毛: 不认同“传统计算机系”这个话。计算机系应该跟着实践走，而不是反过来。

董:
NMT的关键是个“像”。于是出了一个有时不懂原文的人以为翻的很顺溜。没了信的翻译，不就成骗子了吗？如何知道自己的翻译是满拧了?这也是NMT的死穴。

马: 董老师，我觉得统计方法都有这个死穴

我:
寸有所长尺有所短，这也不奇怪。我今天自己听这个对我的blog的翻译已经三篇了，一步一叹。NND 咋这么溜呢。找碴子 找翻译错 总是有的。可是人也有译错啊。从可懂度和流畅程度看，我反正是服了。而这发生在没有亲属关系的两个语言之间。

董:
想当年有的领导干部对我说，“其实机器翻译只有百分之50的正确性，也不要紧，问题是你能不能把那个一半告诉我，我可以找人专翻那部分。”我回答说我做不到。从那时起我一直在关注这个问题。直到如今很多人在叫嚷要取代人工翻译了。这真有点像有了麦当劳就说不要法式大餐了一样。何况机译还做不到麦当劳。计算机、以致机译是上帝给人类玩的，上帝没有给人类那种可以复制自己的本领。

洪:

我的观点很简单：
影子不能三维变。
人若二维非压扁，
自叹弗如影子前。

人工智能影子般，
随人活动数据攒。
深度学习模型建，
类似皮影戏好玩。

董:
是的。我曾对照过10多本英国名著，曾经发现有一本译著明显的是译者故意大段大段地漏译的，那里面有太多的花草等，估计大师懒得查。就不译了。

为什么GNMT首先选择的语言对是汉英，而不是英汉？这是非常精明的。人工翻译即使错了或漏了，译文通常会是顺溜的，至少绝不会像传统的机译那样有傻又疯的，诘屈聱牙的，而这正是NMT的特点，它挑选的是译文中最大相似的。那样一来广大的英语读者，多数不懂中文，就容易被它“唬住”了。

我：
对。仔细看来，这次“突破”是达有余而信不足，矫枉过正了。
但一切才刚开始。我可以理解做NMT的人面对突破的欣喜心情。

洪:
伟爷久玩nlp，
一直孤傲头不低。
今朝服膺叹奇迹，
深度神经已皈依！

我:
皈依还不至于，也不够格。赞佩是由衷的，希望今后有合作的机会，取长补短，达成共赢。人家要是看不上咱呢，咱就单干。deep parsing 是 NLP 的皇冠。神经 parsing 何时全方位超过在下，咱就退休。现在仍然觉得，照这个标准，估计这辈子也退休不了。但愿自己错了，可以提早周游世界。

【相关】

Wei’s Introduction to NLP Architecture

谷歌翻译实现重大突破

谷歌新版翻译有多神？英文教授亲证后告诉你...

【立委科普：NLP 联络图】（姐妹篇）

机器翻译

Introduction to NLP Architecture
by Dr. Wei Li
(fully automatically translated by Google Translate)

The automatic speech generation of this science blog of mine is attached here, it is amazingly clear and understandable, if you are an NLP student, you can listen to it as a lecture note from a seasoned NLPer (definitely clearer than if I were giving this lecture myself with my strong accent):

To preserve the original translation, nothing is edited below.  I will write another blog to post-edit it to make this an "official" NLP architecture introduction to the audiences perused and honored by myself, the original writer.  But for time being, it is completely unedited, thanks to the newly launched Google Translate service from Chinese into English at https://translate.google.com/ 

[Legislature science: natural language system architecture brief]

For the natural language processing (NLP) and its application, the system architecture is the core issue, I blog [the legislature of science: NLP contact diagram] which gave four NLP system architecture diagram, now one by one to be a brief .
I put the NLP system from the core engine to the application, is divided into four stages, corresponding to the four frame diagram. At the bottom of the core is deep parsing, is the natural language of the bottom-up layer of automatic analyzer, this work is the most difficult, but it is the vast majority of NLP system based technology.

The purpose of parsing is to structure unstructured languages. The face of the ever-changing language, only structured, and patterns can be easily seized, the information we go to extract semantics to solve. This principle began to be the consensus of (linguistics) when Chomsky proposed the transition from superficial structure to deep structure after the linguistic revolution of 1957. A tree is not only the arcs that express syntactic relationships, but also the nodes of words or phrases that carry various information. Although the importance of the tree, but generally can not directly support the product, it is only the internal expression of the system, as a language analysis and understanding of the carrier and semantic landing for the application of the core support.

The next layer is the extraction layer (extraction), as shown above. Its input is the tree, the output is filled in the content of the templates, similar to fill in the form: is the information needed for the application, pre-defined a table out, so that the extraction system to fill in the blank, the statement related words or phrases caught out Sent to the table in the pre-defined columns (fields) to go. This layer has gone from the original domain-independent parser into the face-to-face, application-oriented and product-demanding tasks.
It is worth emphasizing that the extraction layer is domain-oriented semantic focus, while the previous analysis layer is domain-independent. Therefore, a good framework is to do a very thorough analysis of logic, in order to reduce the burden of extraction. In the depth analysis of the logical semantic structure to do the extraction, a rule is equivalent to the extraction of thousands of surface rules of language. This creates the conditions for the transfer of the domain.
There are two types of extraction, one is the traditional information extraction (IE), the extraction of fact or objective information: the relationship between entities, entities involved in different entities, such as events, can answer who dis what when and where When and where to do what) and the like. This extraction of objective information is the core technology and foundation of the knowledge graph which can not be renewed nowadays. After completion of IE, the next layer of information fusion (IF) can be used to construct the knowledge map. Another type of extraction is about subjective information, public opinion mining is based on this kind of extraction. What I have done over the past five years is this piece of fine line of public opinion to extract (not just praise classification, but also to explore the reasons behind the public opinion to provide the basis for decision-making). This is one of the hardest tasks in NLP, much more difficult than IE in objective information. Extracted information is usually stored in a database. This provides fragmentation information for the underlying excavation layer.
Many people confuse information extraction and text mining, but in fact this is two levels of the task. Extraction is the face of a language tree, from a sentence inside to find the information you want. The mining face is a corpus, or data source as a whole, from the language of the forest inside the excavation of statistical value information. In the information age, the biggest challenge we face is information overload, we have no way to exhaust the information ocean, therefore, must use the computer to dig out the information from the ocean of critical intelligence to meet different applications. Therefore, mining rely on natural statistics, there is no statistics, the information is still out of the chaos of the debris, there is a lot of redundancy, mining can integrate them.

Many systems do not dig deep, but simply to express the information needs of the query as an entrance, real-time (real time) to extract the relevant information from the fragmentation of the database, the top n results simply combined, and then provide products and user. This is actually a mining, but is a way to achieve a simple search mining directly support the application.
In fact, in order to do a good job of mining, there are a lot of work to do, not only can improve the quality of existing information. Moreover, in-depth, you can also tap the hidden information, that is not explicitly expressed in the metadata information, such as the causal relationship between information found, or other statistical trends. This type of mining was first done in traditional data mining because the traditional mining was aimed at structural data such as transaction records, making it easy to mine implicit associations (eg, people who buy diapers often buy beer , The original is the father of the new people's usual behavior, such information can be excavated to optimize the display and sale of goods). Nowadays, natural language is also structured to extract fragments of intelligence in the database, of course, can also do implicit association intelligence mining to enhance the value of intelligence.
The fourth architectural diagram is the NLP application layer. In this layer, analysis, extraction, mining out of the various information can support different NLP products and services. From the Q & A system to the dynamic mapping of the knowledge map (Google search search star has been able to see this application), from automatic polling to customer intelligence, from intelligent assistants to automatic digest and so on.

This is my overall understanding of the basic architecture of NLP. Based on nearly 20 years in the industry to do NLP product experience. 18 years ago, I was using a NLP structure diagram to the first venture to flicker, investors themselves told us that this is million dollar slide. Today's explanation is to extend from that map to expand from.
Days unchanged Road is also unchanged.

Where previously mentioned the million-dollar slide story. Clinton said that during the reign of 2000, the United States to a great leap forward in Internet technology, known as. Com bubble, a time of hot money rolling, all kinds of Internet startups are sprang up. In such a situation, the boss decided to hot to find venture capital, told me to achieve our prototype of the language system to do an introduction. I then draw the following three-tier structure of a NLP system diagram, the bottom is the parser, from shallow to deep, the middle is built on parsing based on information extraction, the top of the main categories are several types of applications, including Q & A system. Connection applications and the following two language processing is the database, used to store the results of information extraction, these results can be applied at any time to provide information. This architecture has not changed much since I made it 15 years ago, although the details and icons have been rewritten no less than 100 times. The architecture diagram in this article is about one of the first 20 editions. Off the core engine (background), does not include the application (front). Saying that early in the morning by my boss sent to Wall Street angel investors, by noon to get his reply, said he was very interested. Less than two weeks, we got the first $ 1 million angel investment check. Investors say that this is a million dollar slide, which not only shows the threshold of technology, but also shows the great potential of the technology.

Pre - Knowledge Mapping: The Structure of Information Extraction Engine

【Related】
[Legislature science: NLP contact map (one)]
Pre - Knowledge Mapping: The Architecture of Information Extraction Engine
[Legislature science: natural language parsers is to reveal the mystery of the language LIGO-type detector]
【Essay contest: a dream come true】
"OVERVIEW OF NATURAL LANGUAGE PROCESSING"

"NLP White Paper: Overview of Our NLP Core Engine"

White Paper of NLP Engine

"Zhaohua afternoon pick up" directory

[Top: Legislative Science Network blog NLP blog at a glance (regularly updated version)]

retrieved 10/1/2016 from https://translate.google.com/

translated from http://blog.sciencenet.cn/blog-362400-981742.html

我:
系统调试也上瘾。今夜无眠，调着调着，parser 貌似发疯了，大概是嫌我啥都喂给它，闹情绪了？？

仔细瞅瞅，好像也没啥大错，没疯。与鲁爷【狂人日记】不同，我怀疑得没理。

自然语言的任何并列（Conj）结构，到了逻辑层，都必须分列。赶上遇到好几个并列就热闹了，关系有组合爆炸的趋向。都是汉语的顿号惹的祸。用恁多顿号做啥，多写几个小句能死吗？纯句法parsing不管这些，图面倒是显得干净。可是 deep parsing 的语义计算是逻辑的，就不能不管。

白:
“或”的结合能力弱于“与”，顿号在被“或”绑架不成情况下标配解释为“与”。

我:
这几天净出怪，不知是机器走火入魔了，还是玩机器的走火入魔，总之，出来一些奇奇怪怪的 graphs，远远不是教科书上展示的句法树形图给人留下的印象。教科书都是这样的，太过优雅：

前两天出了一个葫芦形的图，昨天又出了双伞形的，今天是发飙，明天还不知会咋样。

这是昨天的两把伞。瞅了一瞅，好像也没错：

白:
吗的位置不对。两把伞那个，能……吗，才是一对。

我:
对，“吗“”应该更上一层楼。如果没有上一层，“吗”疑似就对了。为个小词爬楼不值当了，不是不可以爬 (patching). 当然这里面其实牵涉到决定 yes-no question 的所属问题，最终可能还是要上。

如果说 “电子签证是什么吗。”那就是活用。表面上用疑问，实际是应该是感叹？不是“吗”的标准用法。因为“吗”的本性是一般疑问句，而“什么”是特殊疑问句的疑问词（wh-word），不相谐。

白:
那个是“嘛”，不是“吗”

我:
肯定这里不可以用 “吗” 吗？

白:
他知道电子签证是什么

我:
感觉上可以，好像也不等同于“嘛”。

是那个什么吗。
真地忘了是那个什么了。

白:
你说的感叹义，应该用“嘛”。遗忘义，可以用“吗”
不过现在白字用的，早乱套了。

我:
这是前天的葫芦，白老师的名句。就是“与之”没挂上arg，差强人意，但总体逻辑语义的计算还都对。“你”（S）与“女人”（S）结了婚，而且这事儿修饰的（Mod-S：定语从句）是“女人”。

你说机器神不神，parser 好玩不好玩，这算不算对人类语言的机器理解的敲门砖：芝麻开门！芝麻芝麻快开门。

【相关】

【立委科普：语法结构树之美】

【立委科普：语法结构树之美（之二）】

中文处理

Parsing

【置顶：立委NLP博文一览】

《朝华午拾》总目录

W. Li. 1997. Chart Parsing Chinese Character Strings. In
Proceedings of the Ninth North American Conference on Chinese
Linguistics (NACCL-9). Victoria, Canada.

Chart Parsing Chinese Character Strings [1]

Wei  LI

Simon Fraser University
Burnaby B.C. V5A 1S6 CANADA ([email protected]) 

ABSTRACT

This paper examines problems in word identification for a Chinese natural language processing system and presents our solution to these problems. In conventional systems, written Chinese parsing takes two steps: (1) a segmentation preprocessor for word identification (segmenter); (2) a grammar parsing the string of identified words. Morphological analysis, when required, as in the case of productive word formation, has to be incorporated in the segmenter. This matches the conventional morphology-before-syntax architecture. We will demonstrate the theoretical defect of this architecture when applied to Chinese. This leads to the conclusion that segmentational approach, despite its being the mainstream in Chinese computational morphology, is in general not adequate for the task of Chinese word identification. To solve this problem, a full grammar should be made available. Therefore, we take an alternative one-step approach. We have implemented an integrated grammar of morphology and syntax for directly parsing a string of Chinese characters, building both morphological and syntactic structures. Compared with the conventional two-step approach, our strategy has advantages in resolving ambiguity in word identification and in handling productive word formation.

1. Introduction

A written Chinese sentence is a string of characters with no blanks to mark word boundaries. In conventional systems, Chinese parsing takes two steps as shown in the following Figure 1: (1) a segmentation preprocessor (called segmenter) for word identification; (2) a word based parsing grammar, building syntactic structures (Feng 1996, Chen & Liu (1992).

In contrast, we take an alternative one-step approach, as shown in Figure 2 below. We have implemented a grammar named W‑CPSG (for Wei's Chinese Phrase Structure Grammar). W‑CPSG integrates morphology and syntax for character based parsing, building both morphological and syntactic structures.

In the two-step architecture, the purpose for the segmenter is to properly identify a string of words to feed syntax. This is not an easy task due to the possible involvement of the segmentation ambiguity. For example, given a string of 4 Chinese characters 研究生命, the segmentation ambiguity is shown in (1.a) and (1.b) below.

(1.)  研究生命

(a)        研究生               | 命
graduate student         | life or destiny

(b)        研究    | 生命
study   | life

The resolution of the above ambiguity in the segmenter is a hopeless job because such ambiguity is syntactically conditioned. For sentences like 研究生命金贵 (life for graduate students is precious), (1.a) is the right identification. For the phrase 研究生命起源 (to study the origin of life), (1.b) is right. So far there are no segmenters which can handle this properly and guarantee right word segmentation (Feng 1996). In fact, there can never be such segmenters as long as a grammar is not brought in. This is a theoretical defect of all Chinese analysis systems in the conventional architecture. We have solved this problem in our morphology-syntax integrated W‑CPSG. Word identification in our design becomes a by-product of parsing instead of a pre-condition for parsing.

In the text below, Section 2 investigates problems with the conventional two-step approach. In Section 3, we will present W‑CPSG one-step approach and demonstrate how W‑CPSG parsing solves these problems. The following is a list for abbreviations used in this paper.

A (Adjective); AF (Affix); BM (Bound Morpheme);
CLA (Classifier); CLAP (Classifier Phrase);
DE (Chinese particle introducing a modifier of noun); DEP (DE Phrase);
DE3 (Chinese particle introducing a modifier of result or capability);
DET (Determiner); LE (Chinese perfective aspect marker);
N (Noun); NP (Noun Phrase); P (Preposition); PP (Prepositional Phrase);
S (Sentence); V (Verb); VP (Verb Phrase); Vt (Transitive Verb)

2. Problems Challenging Segmenters

In general, there are two basic problems for segmenters, namely, segmentation ambiguity and productive word formation.

2.1. segmentation ambiguity

This sub-section studies the segmentation ambiguity for Chinese word identification. We indicate that this ambiguity is structural in nature. Therefore it should be captured by structural trees via parsing. We conclude that a parsing grammar is indispensable in the resolution of the segmentation ambiguity.

Behind all segmenters are procedure based segmentation algorithms. Most proposals are some modified versions of large-lexicon based matching algorithms. As an underlying hypothesis, a longer match overrides a shorter match, hence the name maximum match. Decided by the  direction of the  procedure, i.e. whether  the segmentation proceeds from left (the beginning of a string) to right (the end of the string) or from right to left, we have two general types of maximum match: (1) FMM (Forward Maximum Match) algorithm; (2) BMM (Backward Maximum Match) algorithm (Feng 1996).

According to Liang 1987, segmenters have trouble with cases involving the segmentation ambiguity. There are two types of segmentation ambiguity: the cross ambiguity (AB|C vs. A|BC) and the embedded ambiguity (AB vs. A|B).

To detect possible ambiguity, many researchers use the technique of combining the FMM algorithm and the BMM algorithm. When the output of FMM and BMM are different, there must be some ambiguity involved. The following table lists the cases associated with the FMM and BMM combined approach.[2]

The following 3 examples all contain a cross ambiguity sub-string 研究生命 with 2 segmentation possibilities: 研究生|命 and 研究|生命. Example (4.) is a genuinely ambiguous case. Genuinely ambiguous sentences cannot be disambiguated within the sentence boundary, rendering multiple readings.

(2.) case 1:      研究生命金贵。

(a)        研究生                | 命     | 金贵                  (FMM: correct)
graduate student         | life   | precious
Life for graduate students is precious.

(b) * 研究 | 生命    |起源                                   (BMM: incorrect)
study        | life     | precious

(3.) case 2:       研究生命起源。

(a) *     研究生              | 命     | 起源                       (FMM: incorrect)
graduate-student       | life   | origin

(b)        研究     | 生命    | 起源                                (BMM: correct)
study   | life     | origin
to study the origin of life

(4.) case 3:       研究生命不好。

(a)        研究生                   | 命                 | 不       | 好     (FMM: correct)
graduate student         | destiny        | not     | good
The destiny of graduate students is not good.

(b) 研究 | 生命   | 不      | 好                                      (BMM: correct)
study    | life     |  not    | good
It is not good to study life.

The following example is a complicated case of cross ambiguity, involving more than 2 ways of segmentation. Both the FMM segmentation 出现|在世|界 and the BMM segmentation 出|现在|世界 are wrong. A third segmentation 出现|在|世界 is right.

(5.)  case 4:      出现在世界东方。

(a) * 出现 | 在世          | 界     | 东方                       (FMM: incorrect)
appear     | be-alive   | BM   | east

(b) * 出  | 现在  | 世界    | 东方                               (BMM: incorrect)
out        | now   | world | east

(c)  出现  | 在    | 世界     | 东方                               (correct)
appear    | at    | world  | east
to appear in the east of the world

In the following examples (6.) through (8.), ¿¾°×Êí involves embedded ambiguity. As separate words, the verb ¿¾ (bake) and the NP °×Êí (sweet potato) form a VP. As a whole, it is a compound noun ¿¾°×Êí (baked sweet potato). In cases of the embedded ambiguity, FMM and BMM always make the same segmentation, namely AB instead of A|B. It may be the only right choice, as seen in (6.). It may be wrong as shown in (7.). It may only be half right, as in the case of genuine ambiguity shown in (8.).

(6.) case 5:       他吃烤白薯。

(a)        他       | 吃      | 烤白薯                                 (FMM&BMM: correct)
he       | eat     | baked sweet potato
He eats baked sweet potatoes.

(b) *     他       | 吃      | 烤      | 白薯                        (incorrect)
he       | eat     | bake | sweet potato

(7.) case 6:       他会烤白薯。

(a) *     他       | 会      | 烤白薯                                 (FMM&BMM: incorrect)
he       | can    | baked sweet potato

(b)        他      | 会      | 烤      | 白薯                         (correct)
he      | can   | bake | sweet potato
He can bake sweet potatoes.

(8.) case 7:       他喜欢烤白薯。

(a)       他       | 喜欢 | 烤白薯                                  (FMM&BMM: correct)
he      | like  | baked sweet potato
He likes baked sweet potatoes.

(b)        他       | 喜欢   | 烤      | 白薯                       (correct)
he      | like     | bake | sweet potato
He likes baking sweet potatoes.

Compare the above examples, we see that there are severe limitations for the FMM-BMM combined approach. First, it only serves the purpose of ambiguity detection (when the results of FMM and BMM do not match), and contributes nothing to its resolution. It has no way to tell which segmentation is right (compare case 1 and case 2), and, worse still, whether both are right (case 3) or wrong (case 4). Second, even when the results of FMM and BMM do match, it by no means guarantees right segmentation (case 6). Third, as far as detection is concerned, it is only limited to the problems for the cross ambiguity. The existence of the embedded ambiguity defines a blind area for this way of detection (case 6 and case 7). This is because the underlying maximum match hypothesis assumed in the FMM and BMM segmentation algorithms is directly contradictory to the phenomena of the embedded ambiguity.

In face of ambiguity, how do people judge which segmentation is right in the first place? It really depends on whether we can understand the sentence or phrase based on the segmentation. In computational linguistics, this is equivalent to whether the segmented string can be parsed by a grammar. The segmentation ambiguity is one type of structural ambiguity, not in essence different from typical structural ambiguity like, say, PP attachment ambiguity. In fact, PP attachment problem is a counterpart of the cross ambiguity in English syntax, as shown below.

(9.)       Cross ambiguity in PP attachment: V NP PP

(a) [V NP] [PP]
(b) [V] [NP PP]

Therefore, like English PP attachment, Chinese word segmentation ambiguity should also be captured by a parsing grammar. A parser resolves the ambiguity if it can, or detects the ambiguity in the form of multiple parses when it cannot. As shall be demonstrated in Section 3, wrong segmentation will not lead to a parse. Right segmentation results in at least one successful parse. In any case, at least a parser (hence a grammar on which the parser is based) is required for proper word identification.

The important thing is that the ambiguity in word identification is a grammatical problem. The attempt to solve this problem without a grammar is bound to be crippled. Since traditional segmentation algorithms are non-grammatical in nature, they are theoretically not equipped for handling such ambiguity. A successive model of segmentater-before-grammar attempts to do what it is not yet able to do. This is the theoretical defect for almost all existing segmentation approaches.

(10.)     Conclusion for 2.1.

The segmentation ambiguity in word identification is one type of structural ambiguity. In order to solve this problem, a parsing grammar is indispensable.

2.2. productive word formation

Unless morphological analysis is incorporated, lexicon match based segmenters will have trouble with new words produced by Chinese productive word formation, including reduplication, derivation and the formation of proper names. When the morphology component is incorporated in the segmenter, the two-step design becomes a variant of the conventional morphology-before-syntax architecture. But this architecture is not effective when the segmentation ambiguity is at issue.

In the following, we investigate reduplication, derivation and proper names one by one. In each case, we find that there is always a possible involvement of the segmentation ambiguity. This problem cannot be solved by a morphology component independent of syntax. We therefore propose a  grammar incorporating both morphology and syntax.

2.2.1. reduplication

Reduplication in Chinese serves various grammatical and/or lexical functions. Not all reduplications pose challenges to segmentation algorithms. Assume that a word consists of 2 characters AB, reduplication of the type AB --> ABAB is no problem. What becomes a problem for word segmentation is the reduplication of the type AB --> AABB or its variants like AB --> AAB. For example, a two-morpheme verb with verb-object relation at the level of morphology has the following way of reduplication.

(11.) Verb Reduplication: AB --> AAB  (for diminutive use)

分心 (get distracted) --> 分分心 (get distracted a bit)

让他分分心。

让       | 他     | 分分心
let       | he    | get distracted a bit
Let him relax a while.

It seems that reduplication is a simple process which can be handled by incorporating some procedure-based function calls in the segmentation algorithm. If a 3-character string, say 分分心, cannot be found in the lexicon, the reduplication procedure will check whether the first 2 characters are the same, and if yes, delete one of them and consult the lexicon again. But, such expansion of the segmentation algorithm is powerless when the segmentation ambiguity is involved. For example, it is wrong to regard 分分心 as of reduplication in the following sentence.

(12.)   这件事十分分心。

(a) *     这       | 件     | 事        | 十      | 分分心
this      | CLA  | thing  | ten    | get distracted a bit

(b)        这       | 件      | 事        | 十分    | 分心
this      | CLA  | thing  | very   | distracting
This thing is very distracting.

2.2.2. derivation

In Contemporary Mandarin, there have come to be a few morphemes functioning similarly to English affixes, e.g. 可 (-able) turns a transitive verb into an adjective.

(13.)     可 (-able) + Vt --> A

可 (-able) + 读 (Vt: read) -->   可读 (A:readable)

这本书非常可读。

这       | 本     | 书       | 非常   | 可读
this    | CLA  | book  | very  | readable
This book is very readable.

The suffix 性 works just like '-ness',  changing an adjective into an abstract noun.  The derived noun 可读性 (readability) in the following example, similar to its English counterpart, involves a process of double affixation.

(14.)     A + 性 (-ness)  --> N
可 (-able) + 读 (Vt: read) -->   可读 (A:readable)
可读 (A:readable) + 性 (-ness) --> 可读性 (N:readability)

这本书的可读性

这       | 本      | 书       | 的     | 可读性
this    | CLA  | book  | DE    | readability
this book's readability

The suffix Í· can change a transitive verb into an abstract noun adding to it the meaning "worth-of".

(15.) Vt + 头 (AF:worth of) --> N

吃 (Vt:eat) + 头 (AF:worth of) --> 吃头 (N:worth of eating)

这道菜没有吃头

这       | 道     | 菜      | 没有             | 吃头
this    | CLA  | dish  | not-have    | worth-of-eating
This dish is not worth eating.

It is not difficult to incorporate in the segmenter these derivation rules for the morphological analysis. But, as in the case of reduplication, there is always a danger of wrongly applying the rules due to possible ambiguity involved. For example, 吃头 is a sub-string of embedded ambiguity. It can be both a derived noun 'worth of eating' or two separate words as seen in the following example.

(16.)  他饿得能吃头牛。

(a) *     他      | 饿             | 得      | 能     | 吃头·                       | 牛
             he     | hungry    | DE3  | can  | worth-of-eating   | ox

(b)        他      | 饿              | 得     | 能     | 吃      | 头      | 牛
              he     | hungry    | DE3  | can  | eat    | CLA  | ox
He is so hungry that he can eat an ox.

2.2.3. proper name

Proper names are of 2 major types: (1) Chinese names; (2) transliterated foreign names. In this paper, we only target the identification of Chinese names and leave the problem of transliterated foreign names for further research (Li, 1997b).

A Chinese human name usually consists of a family name followed by a given name. Chinese family names form a clear-cut closed set. A given name is usually either one character or two characters. For example, the late Chinese chairman 毛泽东 (Mao Zedong) used to have another name 李得胜 (Li Desheng). In the lexicon, 李 is a registered family name. Both 得胜 and 胜 mean 'win'. This may lead to 3 ways of word segmentation: (1) 李得胜; (2) 李|得胜; (3) 李得|胜, as seen in the following examples.

(17.)    李得胜了

(a)  李    | 得胜 | 了.
       Li    | win  | LE
Li won.

(b)   李得   | 胜     | 了
        Li De | win  | LE
Li De won.

(c) *  李得胜          | 了.
          Li Desheng | LE

(18.)   李得胜胜了 。

(a) *  李 | 得胜 | 胜    | 了.
         Li  | win | win | LE

(b) *  李得   | 胜     | 胜     | 了
          Li De | win  | win  | LE

(c)   李得胜            | 胜     | 了
Li Desheng   | win  | LE
Li Desheng won.

Since the given name like µÃʤ is an arbitrary string of 1 or 2 characters, the morphological analysis of the full name should start with family name which can optionally combine with any 1 or 2 characters to form candidate proper names Àî, ÀîµÃ and ÀîµÃʤ. In other words, family name serves as the left boundary of a full name and the length is used to determine candidates. The right segmentation can only be made via sentence analysis as shown in the above examples.

Most Chinese place proper names are made of 1 to 3 characters, for example, 武汉市(Wuhu City), 南陵县 (Nanling County). The arbitrariness of these names makes any sub-strings of n characters (0<n<4) in the sentence a suspect. Fortunately, in most cases we may find boundary indicators of these names, like 省 (province), 市 (city), 县 (county), etc. Once the boundary indicator is located, the similar technique in using Chinese family name to identify the given name can be applied to select candidates of place proper names for verification through grammatical analysis.

In general, there is always a possibility of ambiguity involvement in the formation of all types of proper names.

(19.)     Conclusion for 2.2.

Due to the possible involvement of ambiguity, a parsing grammar for morphological analysis as well as for sentence analysis is required for the proper identification of the words produced by Chinese productive word formation.

3. W‑CPSG Grammatical Approach

This section presents W‑CPSG approach to Chinese word identification and morphological analysis. We will demonstrate how a parser based on W‑CPSG solves the problems of the word identification ambiguity and productive word formation.

3.1. rationale of W‑CPSG approach

There have been a number of word identification algorithms based on both morphological and syntactic information (see survey in Feng 1996 and Sun & Huang 1996). Most such approaches do not use a self-contained grammar to parse the complete sentence. They are confined to the conventional two-step process of the segmentation-before-grammar design. As long as the word identification procedure is independent of a parsing grammar, it is extremely difficult to make full use of grammatical information to resolve ambiguity in word identification. Careful tuning up and sophisticated design improves the precision but will not change the theoretical defect of all such approaches. Chen & Liu acknowledges the limitation of their approach due to the lack of a grammar.  “However”, they say,  “it is almost impossible to apply real world knowledge nor to check the grammatical validity at this stage”. (Chen & Liu 1992, p.105) Why impossible at this stage? Because these segmentation systems are based on the concept of  two-step architecture and the grammar is not yet available! As we have demonstrated, the final judgment for proper word identification can hardly be made until the whole sentence is parsed, hence the requirement of a full grammar. Therefore, we are forced to make a compromise in involving how much of grammatical information depending on how much word identification precision we can afford to sacrifice. Needless to say, there is significant double-labor between such a word segmentation procedure and the following stage of parsing. As more and more grammatical information is used to achieve better precision, the overhead of this double labor becomes more serious. We consider the double labor as one strong argument against the two-step approach. If enough grammatical information is incorporated, it is essentially equivalent to a grammar. And  the segmenter will be equivalent to a parser.  Then why two grammars, one for word identification, and one for sentence parsing? Why not combine them? That is exactly what we are proposing in W‑CPSG - one-step approach based on an integrated grammar,  eliminating the necessity of a segmentation preprocessor.

3.2. W‑CPSG character-based parsing

W‑CPSG (Li. 1997a, 1997b) is a lexicalized Chinese unification grammar. The work on W‑CPSG is taken in the spirit of the modern linguistic theory Head-driven Phrase Structure Grammar (Pollard & Sag 1994). W‑CPSG consists of two parts: a minimized general grammar and an enriched lexicon. The general grammar only contains a handful of PS (phrase structure) rules, covering complement structure, modifier structure, conjunctive structure and morphological structure. This is the nature of lexicalized grammars. PS rules in such grammars are very abstract. Essentially, all they say is one thing, that is, 2 signs can combine so long as the lexicon so indicates. The lexicon houses lexical entries with their linguistic description in feature structures. Potential morphological structures as well as potential syntactic structures are lexically encoded. In syntax, a word expects another sign to form a phrase. In morphology, a morpheme expects another sign to form a word. For example, the prefix 可 (-able) expects a transitive verb to form an adjective. The morphological PS rule will build the morphological structure when a transitive verb does appear after the prefix 可 (-able) in the input string.

We now illustrate how W‑CPSG parses a string of Chinese characters by a sample parsing chart. The prototype of W‑CPSG was written in ALE, a grammar compiler developed on top of Prolog by Carpenter & Penn (1994). ALE compiles W‑CPSG into a Chinese parser, a Prolog program ready to accept a string of characters for analysis. W‑CPSG parse tree embodies both morphological analysis and syntactic analysis, as shown below.

This is so-called bottom-up parsing. It starts with lexicon look-up. Edges 1 through 7 are lexical edges. Other edges are phrasal edges. Each edge represents a sign, i.e. a character (morpheme), a word, a phrase or a sentence. Lexical edges result from a successful match between the signs in the input string and the entries in the lexicon during lexicon look-up. For example, 可 (-able), 读 (read) and 性 (-ness) are all registered entries in the lexicon, so they get matched and shown by edge 5, edge 6 and edge 7. Words produced by productive word formation present themselves as phrasal edges, e.g. edge ((5+6)+7) for 可读性 (readability). For the sake of concise illustration, we only show two pieces of information for the signs in the chart, namely category and interpretation with a delimiting colon (lexical edges are only labeled for either category or interpretation). The parser attempts to combine the signs according to PS rules in the grammar until parses are found. A parse is an edge which ranges over the whole string. The parse ((((1+2)+3)+4)+((5+6)+7)) for (20.) represents a binary structural tree based on the W‑CPSG analysis, as shown below.

3.3. ambiguity resolution in word identification

Given the resources of a phrase structure grammar like W‑CPSG, a parser based on standard chart parsing algorithms can handle both the cross ambiguity and the embedded ambiguity provided that a match algorithm based on exhaustive lookup instead of maximum match is adopted for lexicon lookup. All candidate words in the input string are presented to the parser for judgment. Ambiguous segmentation becomes a natural part of parsing: different ways of segmentation add different edges, a successful parse always embodies right identification. In other words, word identification in our design becomes a by-product of parsing instead of a pre-condition for parsing. The following example of the complicated cross ambiguity illustrates how the W‑CPSG parser resolves ambiguity. As seen, both the FMM segmentation (represented by the edge sequence 8-9-5-10) and the BMM segmentation (represented by 1-11-12-10) are in the chart as a result of exhaustive lexicon lookup. They are proved to be wrong because they do not lead to a successful parse according to the grammar. As a by-product, the final parse (8+(3+(12+10))) automatically embodies rightly identified word sequence 8-3-12-10, i.e. 出现  (appear) |在  (at) |世界 (world) |东方 (east).

Exhaustive lookup also makes an embedded ambiguity sub-string like 烤红薯 no longer a blind area for word identification, as shown in (22.) below. All the candidate words in the sub-string including 烤 (bake), 红薯 (sweet potato), 烤红薯 (baked sweet potato) are added to the chart as lexical edges (edge 4, edge 8 and edge 10). This is a case of genuine ambiguity, resulting in 2 parses corresponding to 2 readings. The first parse (1+(7+10)) identifies the word sequence 他|喜欢|烤红薯, and the second parse (1+(9+(4+8))) a different sequence 他|喜欢|烤|红薯. Edge 7 and edge 9 represent two lexical entries for the verb 喜欢 (like), with different syntactic expectation (categorization). One expects an NP object, notated in the chart by like<NP>, and the other expects a VP complement, notated by like<VP>.

We now illustrate how Chinese proper names are identified in W‑CPSG parsing. In the W‑CPSG lexicon, Chinese family name is encoded to optionally expect the given name. Due to the arbitrariness of given names, no other constraint except for the length (either 1 character or 2 characters) is specified in the expectation. Therefore, we have three candidates for proper names in the following example, namely 李 (Li), 李得 (Li De), 李得胜 (Li Desheng), represented respectively by edge 1, edge (1+2) and the NP edge (1+5).[3] The first two candidates contribute to two valid parses while the third does not, hence the identification of the word sequences 李|得胜|了 and 李得|胜|了.

Now we add one more character 胜 (win) to form a new sentence, as shown in (24.) below.

The first two candidate proper names 李 (Li) and 李得 (Li De) no longer lead to parses. But the third candidate 李得胜 (Li Desheng) becomes part of the parse as a subject NP. The parse (((1+6)+4)+5) corresponds to the identification of the only valid word sequence 李得胜|胜|了.

Finally, we give an example to demonstrate how W‑CPSG handles reduplication in parsing and word identification. The sample sentence to be processed by the parser is 让他分分心 (Let him relax a while), involving the AB-->AAB type verb reduplication for diminutive use.

In most lexicons, 分心 (distract-heart: get distracted) is a registered 2-morpheme verb with internal morphological verb-object relation. Therefore, the reduplication is considered morphological. But in Chinese syntax, we also have a  general verb reduplication rule of the type A-->AA for diminutive use, for example, 看(look) --> 看看(have a look). This morphological verb reduplication rule AB-->AAB and the syntactic verb reduplication rule A-->AA are essentially the same rule in Chinese grammar. 分心 sits in the gray area between morphology and syntax. It looks both like a word (verb) and a phrase (VP). Lexically, it corresponds to one generalized sense (concept) and the internal combination is idiomatic, i.e. 分 (distract) must combine with 心 (heart) to mean 'get distracted'. But, structurally, the combination of 分 and 心 is not fundamentally different from a VP consisting of Vt and NP, as in the phrase 看电影 (see a film). In fact, there is no clear-cut boundary between Chinese morphology and syntax. This morphology-syntax isomorphic fact serves as a further argument to support the W‑CPSG design of integrating morphology and syntax in one grammar module. Although the boundary between Chinese morphology and syntax is fuzzy, hence no universal definition of basic notions like word and phrase, the division can be easily defined system internally in an integrated grammar. In W‑CPSG,  分心 is treated as a phrase (VP) instead of a word (verb). The lexical entry 分 (distract) is coded to obligatorily expect the literal 心 (heart) as its syntactic object, shown in the following chart by the notation V<心>. This approach has the advantage of eliminating the doubling of the reduplication rule for diminutive use in both syntax and morphology, making the grammar more elegant. The verb reduplication rule is implemented as a lexical rule in W‑CPSG.[4] This lexical rule creates a reduplicated verb with added diminutive sense, shown by edge 8 (a lexical edge).  The whole parsing process is illustrated below.

REFERENCES

Carpenter, B. & Penn, G. (1994): ALE, The Attribute Logic Engine, User's Guide, Carnegie Mellon University

Chen, K-J., & S-H. Liu (1992): "Word identification for mandarin Chinese sentences". Proceedings of the 15th International Conference on Computational Linguistics, Nantes, 101-107.

Feng, Z-W. (1996): "COLIPS lecture series - Chinese natural language processing",  Communications of COLIPS, Vol.6, No.1 1996, Singapore

Li, W. (1997a): "Outline of an HPSG-style Chinese reversible grammar", Proceedings of The Northwest Linguistics Conference-97 (NWLC-97, forthcoming), UBC, Vancouver, Canada

Li, W. (1997b): W‑CPSG: A Lexicalized Chinese Unification Grammar And Its Application, Doctoral dissertation (on-going), Simon Fraser University, Canada

Liang, N. (1987): "Shumian Hanyu Zidong Fenci Xitong - CDWS" (Automatic word segmentation system for written Chinese - CDWS), Journal of Chinese Information Processing, No.2 1987, pp 44-52, Beijing

Pollard, C.  & I. Sag (1994): Head-Driven Phrase Structure Grammar,  Centre for the Study of Language and Information, Stanford University, CA

Sun, M-S. & C-N. Huang  (1996): "Word segmentation and part of speech tagging for unrestricted Chinese texts" (Tutorial Notes for International Conference on Chinese Computing ICCC'96), Singapore

~~~~~~~~~~~~~~~~~~~

[1] The author benefited from the insightful discussion with Dr. Dekang Lin on the feasibility of parsing Chinese character strings instead of word strings. Thanks also go to Paul McFetridge and Fred Popowich for their supervision and encouragement.

[2] This table is adapted from the following table in Sun & Huang (1996).

The 4 cases which they listed are not logically exhaustive in terms of sentence based processing (i.e. when discourse is not involved in a system). In particular, there is another case when the output of FMM and BMM are different, and both are correct. We call this a case of genuine cross ambiguity.

[3] Note that there is another S edge (1+5) in the chart. These two edges are structurally different, created via different PS rules. The NP edge (1+5) is formed through the morphological PS rule, combining the family name (edge 1) and its expected given name (edge 5). In the S edge (1+5). however, it is the subject rule (one of the complement PS rules) that decides the combination of the predicate (edge 5) and its expected subject NP (edge 1).

[4] Lexical rules are favored by many linguists to capture redundancy in the lexicon instead of the conventional approach of syntactic transformation. Lexical rules are applied at compile time to form an expanded lexicon before parsing starts.

[Related]

Interaction of syntax and semantics in parsing Chinese transitive verb patterns 

Handling Chinese NP predicate in HPSG 

Notes for An HPSG-style Chinese Reversible Grammar

Outline of an HPSG-style Chinese reversible grammar

PhD Thesis: Morpho-syntactic Interface in CPSG (cover page)

PhD Thesis: Chapter I Introduction

PhD Thesis: Chapter VII Concluding Remarks

Overview of Natural Language Processing

Dr. Wei Li’s English Blog on NLP

2.0. Introduction

This chapter examines the role of grammar in handling the three major types of morpho-syntactic interface problems.  This investigation  justifies the mono-stratal design of CPSG95 which contains feature structures of both morphology and syntax.

The major observation from this study is:  (i) grammatical analysis, including both morphology and syntax, plays the fundamental role in contributing to the solutions of the morpho-syntactic problems;  (ii)  when grammar alone is not sufficient to reach the final solution, knowledge beyond morphology and syntax may come into play and serve as “filters” based on the grammatical analysis results.[1]  Based on this observation, a study in the direction of interleaving morphology and syntax will be pursued in the grammatical analysis.  Knowledge beyond morphology and syntax is left to future research.

Section 2.1 investigates the relationship between grammatical analysis and  the resolution of segmentation ambiguity.  Section 2.2 studies the role of syntax in handling Chinese productive word formation.  The borderline cases and their relationship with grammar are explored in 2.3.  Section 2.4 examines the relevance of knowledge beyond syntax to segmentation disambiguation.  Finally, a summary of the presented arguments and discoveries is given in 2.5.

2.1. Segmentation Ambiguity and Syntax

Segmentation ambiguity is one major problem which challenges the traditional word segmenter or an independent morphology.  The following study shows that this ambiguity is structural in nature, not fundamentally different from other structural ambiguity in grammar.  It will be demonstrated that sentential structural analysis is the key to this problem.

A huge amount of research effort in the last decade has been made on resolving segmentation ambiguity (e.g. Chen and Liu 1992; Gan 1995; He, Xu and Sun 1991; Liang 1987; Lua 1994; Sproat, Shih, Gale and Chang 1996; Sun and T’sou 1995; Sun and Huang 1996; X. Wang 1989; Wu and Su 1993; Yao, Zhang and Wu 1990; Yeh and Lee 1991; Zhang, Chen and Chen 1991; Guo 1997b).  Many (e.g. Sun and Huang 1996; Guo 1997b) agree that this is still an unsolved problem.  The major difficulty with most approaches reported in the literature lies in the lack of support from sufficient grammar knowledge.  To ultimately solve this problem, grammatical analysis is vital, a point to be elaborated in the subsequent sections.

2.1.1. Resolution of Hidden Ambiguity

The topic of this section is the treatment of hidden ambiguity.   The conclusion out of the investigation below is that the structural analysis of the entire input string provides a sound basis for handling this problem.

The following sample sentences illustrate a typical case involving the hidden ambiguity string 烤白薯 kao bai shu.

(2-1.) (a)      他吃烤白薯
ta         | chi  | kao-bai-shu
he      | eat  | baked-sweet-potato
[S [NP ta] [VP [V chi] [NP kao-bai-shu]]]
He eats the baked sweet potato.

(b) * ta       | chi  | kao          | bai-shu
he      | eat  | bake         | sweet-potato

(2-2.) (a) *    他会烤白薯
ta         | hui  | kao-bai-shu.
he      | can | baked-sweet-potato

(b)     ta       | hui  | kao          | bai-shu.
he      | can | bake         | sweet-potato
[S [NP ta] [VP [V hui] [VP [V kao] [NP bai-shu]]]]
He can bake sweet potatoes.

Sentences (2-1) and (2-2) are a minimal pair;  the only difference is the choice of the predicate verb, namely chi (eat) versus hui (can, be capable of).  But they have very different structures and assume different word identification.  This is because verbs like chi expect an NP object but verbs like hui require a VP complement.  The two segmentations of the string kao bai shu provide two possibilities, one as an NP kao-bai-shu and the other as a VP kao | bai-shu.  When the provided unit matches the expectation, it leads to a successful syntactic analysis, as illustrated by the parse trees in (2‑1a) and (2-2b).  When the expectation constraint is not satisfied, as in (2-1b) and (2-2a), the analysis fails.  These examples show that all candidate words in the input string should be considered for grammatical analysis.  The disambiguation choice can be made via the analysis, as seen in the examples above with the sample parse trees.  Correct segmentation results in at least one successful parse.

He, Xu and Sun (1991) indicate that a hidden ambiguity string requires a larger context for disambiguation.  But they did not define what the 'larger context' should be.  The following discussion attempts to answer this question.

The input string to the parser constitutes a basic context as well as the object for sentential analysis.[2]  It will be argued that this input string is the proper context for handling the hidden ambiguity problem.  The point to be made is, context smaller than the input string is not reliable for the hidden ambiguity resolution.  This point is illustrated by the following examples of the hidden ambiguity string ge ren in (2-3).[3]  In each successive case, the context is expanded to form a new input string.   As a result, the analysis and the associated interpretation of ‘person’ versus ‘individual’ change accordingly.

(2-3.)  input string                            reading(s)

(a)      人  ren                                       person (or man, human)
[N ren]

(b)      个人  ge ren                               individual
[N ge-ren]

(c)      三个人  san ge ren                               three persons
[NP [CLAP [NUM san] [CLA ge]] [N ren]]

(d)      人的力量  ren de li liang                      the human power
[NP [DEP [NP ren] [DE de]] [N li-liang]]

(e)      个人的力量  ge ren de li liang                        the power of an individual
[NP [DEP [NP ge-ren] [DE de]] [N li-liang]]

(f)       三个人的力量  san ge ren de li liang              the power of three persons
[NP [DEP [NP [CLAP [NUM san] [CLA ge]] [N ren]] [DE de]] [N li-liang]]

(g)      他不是个人  ta bu shi ge ren.
           (1)    He is not a man. (He is a pig.)
[S [NP ta] [VP [ADV bu] [VP [V shi] [NP [CLAP ge] [N ren]]]]]
(2)  He is not an individual. (He represents a social organization.)
[S [NP ta] [VP [ADV bu] [VP [V shi] [NP  ge-ren]]]]

Comparing (a), (b) with (c), and (d), (e) with (f), one can see the associated change of readings when each successively expanded input string leads to a different grammatical analysis.  Accordingly, one segmentation is chosen over the other on the condition that the grammatical analysis of the full string can be established based on the segmentation.  In (b), the ambiguous string is all that is input to the parser, therefore the local context becomes full context.  It then acquires the lexical reading individual as the other possible segmentation ge | ren does not form a legitimate combination.  This reading may be retained, as in (e), or changed to the other reading person, as in (c) and (f), or reduced to one of the possible interpretations, as in (g), when the input string is further lengthened.  All these changes depend on the sentential analysis of the entire input string, as shown by the associated structural trees above.  It demonstrates that the full context is required for the adequate treatment of the hidden ambiguity phenomena.  Full context here refers to the entire input string to the parser.

It is necessary to explain some of the analyses as shown in the sample parses  above.  In Contemporary Mandarin, a numeral cannot  combine with a noun without a classifier in between.[4]  Therefore, the segmentation san (three) | ge-ren (individual) is excluded in (c) and (f), and the correct segmentation san (three) | ge (CLA) | ren (person) leads to the NP analysis.  In general, a classifier alone cannot combine with the following noun either, hence the interpretation of ge ren as one word ge-ren (individual) in (b) and (e).  A classifier usually combines with a preceding numeral or determiner before it can combine with the noun.  But things are more complicated.  In fact, the Chinese numeral yi (one) can be omitted when the NP is in object position.  In other words, the classifier alone can combine with a noun in a very restricted syntactic environment.  That explains the two readings in (g).[5]

The following is a summary of the arguments presented above.   These arguments have been shown to account for the hidden ambiguity phenomena.  The next section will further demonstrate the validity of these arguments for overlapping ambiguity as well.

(2-4.) Conclusion
The grammatical analysis of the entire input string is required for the adequate treatment of the hidden ambiguity problem in word identification.

2.1.2. Resolution of Overlapping Ambiguity

This section investigates overlapping ambiguity and its resolution.  A previous influential theory is examined, which claims that the overlapping ambiguity string can be locally disambiguated.   However, this theory is found to be unable to account for a significant amount of data.  The conclusion is that both overlapping ambiguity and hidden ambiguity require a context of the entire input string and a grammar for disambiguation.

For overlapping ambiguity, comparing different critical tokenizations will be able to detect it, but such a technique cannot guarantee a correct choice without introducing other knowledge.  Guo (1997) pointed out:

As all critical tokenizations hold the property of minimal elements on the word string cover relationship, the existence of critical ambiguity in tokenization implies that the “most powerful and commonly used” (Chen and Liu 1992, page 104) principle of maximum tokenization would not be effective in resolving critical ambiguity in tokenization and implies that other means such as statistical inferencing or grammatical reasoning have to be introduced.

However, He, Xu and Sun (1991) claim that overlapping ambiguity can be resolved within the local context of the ambiguous string.  They classify the overlapping ambiguity string into nine types.  The classification is based on the categories of the assumably correctly segmented words in the ambiguous strings, described below.

Suppose there is an overlapping ambiguous string consisting of ABC;  both AB and BC are entries listed in the lexicon.  There are two possible cases.  In case one, the category of A and the category of BC define the classification of the ambiguous string.  This is the case when the segmentation A|BC is considered correct.  For example, in the  ambiguous string 白天鹅 bai tian e, the word AB is  bai-tian (day-time) and the word BC is tian-e (swan).  The correct segmentation for this string is assumed to be A|BC, i.e. bai (A: white) | tian-e (N: swan) (in fact, this cannot be taken for granted as shall be shown shortly), therefore, it belongs to the A-N type.  In case two, i.e. when the segmentation AB|C is considered correct, the category of AB and the category C define the classification of the ambiguous string.   For example, in the ambiguous string 需求和 xu qiu he, the word AB is  xu-qiu (requirement) and the word BC qiu-he (sue for peace).  The correct segmentation for this string is AB|C, i.e. xu-qiu (N: requirement) | he (CONJ: and) (again, this should not be taken for granted), therefore, it belongs to the N-CONJ type.

After classifying the overlapping ambiguous strings into one of nine types, using the two different cases described above, they claim to have discovered a rule.[6]  That is, the category of the correctly segmented word BC in case one (or AB in case two) is predictable from AB (or BC in case two) within the local ambiguous string.  For example, the category of tian-e (swan) in bai | tian-e (white swan) is a noun.  This information is predictable from bai tian within the respective local string bai tian e.  The idea is, if ever an overlapping ambiguity string is formed of bai tian and C, the judgment of bai | tian-C as the correct segmentation entails that the word tian-C  must be a noun.  Otherwise, the segmentation A|BC is wrong and the other segmentation AB|C is right.  For illustration, it is noted that tian-shi (angel) in the ambiguous string 白天使 bai | tian-shi (white angel) is, as expected, a noun.  This predictability of the category information from within the local overlapping ambiguous string is seen as an important discovery (Feng 1996).  Based on this assumed feature of the overlapping ambiguous strings, He,  Xu and Sun (1991) developed their theory that an overlapping ambiguity string can be disambiguated within the local string itself.

The proposed disambiguation process within the overlapping ambiguous string proceeds as follows.  In order to correctly segment an overlapping ambiguous string, say, bai tian e or bai tian shi, the following information needs to be given under the entry bai-tian (day-time) in the tokenization lexicon:  (i) an ambiguity label, to indicate the necessity to call a disambiguation rule;  (ii) the ambiguity type A-N, to indicate that it should call the rule corresponding to this type.  Then the following disambiguation rule can be formulated.

(2-5.) A-N type rule       (He,  Xu and Sun 1991)
In the overlapping ambiguous string A(1)...A(i) B(1)...B(j) C(1)...C(k),
if        B(1)...B(j) and C(1)...C(k) form a noun,
then  the correct segmentation is A(1)...A(i) | B(1)...B(j)-C(1)...C(k),
else    the correct segmentation is A(1)...A(i)-B(1)...B(j) | C(1)...C(k).

This way, bai tian e and bai tian shi will always be segmented as bai (white) | tian-e (swan) and bai (white) | tian-shi (angel) instead of bai-tian (daytime) | e (goose) and bai-tian (daytime) | shi (make).  This can be easily accommodated in a segmentation algorithm provided the above  information is added to the lexicon and the disambiguation rules are implemented.  The whole procedure is running within the local context of the overlapping ambiguous string and uses only lexical information.  So they also name the overlapping ambiguity disambiguation morphology-based disambiguation, with no need to consult syntax, semantics or discourse.

Feng (1996) emphasizes that He, Xu and Sun's view on the overlapping ambiguous string constitutes a valuable contribution to the theory of Chinese word identification.  Indeed, this overlapping ambiguous string theory, if it were right, would be a breakthrough in this field.  It in effect suggests that the majority of the segmentation ambiguity is resolvable without and before a grammar module.  A handful of simple rules, like the A-N type rule formulated above, plus a lexicon would solve most ambiguity problems in word identification.[7]

Feng (1996) provides examples for all the nine types of overlapping ambiguous strings as evidence to support He, Xu and Sun (1991)'s theory.   In the case of the A-N type ambiguous string bai tian e, the correct segmentation is supposed to be bai | tian-e in this theory.  However, even with his own cited example, Feng ignores a perfect second reading (parse) when the time NP bai-tian (daytime) directly acts as a modifier for the sentence with no need for a preposition, as shown in (2‑6b) below.

(2-6.)           白天鹅游过来了
bai tian e you guo lai le       (Feng 1996)

(a)      bai     | tian-e       | you          | guo-lai      | le.
          white | swan        | swim        | over-here  | LE
[S [NP bai tian-e] [VP you guo-lai le]]
The white swan swam over here.

(b)      bai-tian       | e              | you          | guo-lai      | le.
          day-time      | goose        | swim        | over-here  | LE
[S [NP+mod bai-tian] [S [NP e] [VP you guo-lai le]]]
In the day time the geese swam over here.

In addition, one only needs to add a preposition zai (in) to the beginning of the sentence to make the abandoned segmentation bai-tian | e the only right one in the changed context.  The presumably correct segmentation, namely bai | tian-e, now turns out to be wrong, as shown in (2-7a) below.

(2-7.)           在白天鹅游过来了
zai bai tian e you guo lai le

(a) *   zai     | bai           | tian-e       | you          | guo-lai      | le.
          in      | white        | swan        | swim        | over-here  | LE

(b)      zai     | bai-tian    | e              | you          | guo-lai      | le.
          in      | day-time   | goose        | swim        | over-here  | LE
[S [PP+mod zai bai-tian] [S [NP e] [VP you guo-lai le]]]
In the day time the geese swam over here.

The above counter-example is by no means accidental.  In fact, for each cited ambiguous string in the examples given by Feng, there exist counter-examples.  It is not difficult to construct a different context where the preferred segmentation within the local string, i.e. the segmentation chosen according to one of the rules, is proven to be wrong.[8]  In the pairs of sample sentences (2‑8) through (2-10), (a) is an example which Feng (1996) cited to support the view that the local ambiguous string itself is enough for disambiguation.  Sentences in (b) are counter-examples to this theory.  It is a notable fact that the listed local string is often properly contained in a more complicated ambiguous string in an expanded context, seen in (2-9b) and (2-10b).  Therefore, even when the abandoned segmentation can never be linguistically correct in any context, as shown for tu-xing (graph) | shi (BM) in (2-9) where a bound morpheme still exists after the segmentation, it does not entail the correctness of the other segmentation in all contexts.  These data show that all possible segmentations should be retained for the grammatical analysis to judge.

(2-8.)  V-N type of overlapping ambiguous string

研究生命
          yan jiu sheng ming:
          yan-jiu (V:study) | sheng-ming (N:life)
yang-jiu-sheng (N:graduate student) | ming (life/destiny)

(a)      研究生命的本质
          yan-jiu    sheng-ming de      ben-zhi
          study          life               DE     essence
Study the essence of life.

(b)      研究生命金贵
           yan-jiu-sheng      ming  jin-gui
          graduate-student  life     precious
Life for graduate students is precious.

(2-9.)  CONJ-N type of overlapping ambiguous string
和平等 he ping deng:
          he (CONJ:and) | ping-deng (N:equality)
he-ping (N:peace) | deng (V:wait)?

(a)      独立自主和平等互利的原则
           du-li-zi-zhu           he      ping-deng-hu-li               de      yuan-ze
          independence       and    equal-reciprocal-benefit  DE     principle
the principle of independence and equal reciprocal benefit

(b)      和平等于胜利 he-ping       deng-yu       sheng-li
          peace           equal           victory
Peace is equal to victory.

(2-10.)  V-P type of overlapping ambiguous string
看中和 kan zhong he:
          kan-zhong (V:target) | he (P:with)
kan (V:see) | zhong-he (V:neutralize)

(a)      他们看中和日本人生意的机会
ta-men    kan-zhong   he      ri-ben          ren              zuo     sheng-yi      de      ji-hui    
they         target           with    Japan          person          do      business     DE   opportunity
They have targeted the opportunity to do business with the Japanese.

(b)      这要看中和作用的效果
zhe          yao    kan    zhong-he-zuo-yong                   de          xiao-guo
this    need  see     neutralization                DE     effect
This will depend on the effects of the neutralization.

The data in (b) above directly contradict the claim that an overlapping ambiguous string can be disambiguated within the local string itself.  While this approach is shown to be inappropriate in practice, the following comment attempts to reveal its theoretical motivation.

As reviewed in the previous text, He, Xu and Sun (1991)'s overlapping ambiguity theory is established on the classification of the overlapping ambiguous strings.  A careful examination of their proposed nine types of the overlapping ambiguous strings reveals an underlying assumption on which the classification is based.  That is, the correctly segmented words within the overlapping ambiguous string will automatically remain correct in a sentence containing the local string.   This is in general untrue, as shown by the counter-examples above.[9]   The following analysis reveals why.

Within the local context of the overlapping ambiguous string, the chosen segmentation often leads to a syntactically legitimate structure while the abandoned segmentation does not.  For example,  bai (white) | tian-e (swan) combines into a valid syntactic unit while there is no structure which can span bai-tian (daytime) | e (goose).  For another example,  yan-jiu (study) | sheng-ming (life) can be combined into a legitimate verb phrase [VP [V yan-jiu] [NP sheng-ming]], but  yan-jiu-sheng (graduate student) | ming (life/destiny) cannot.  But that legitimacy only stands locally within the boundary of the ambiguous string.  It does not necessarily hold true in a larger context containing the string.  As shown previously in (2-7a),  the locally legitimate structure bai | tian-e (white swan) does not lead to a successful parse for the sentence.  In contrast, the locally abandoned segmentation bai-tian (daytime) | e (goose) has turned out to be right with the parse in (2-7b).   Therefore, the full context instead of the local context of the ambiguous string is required for the final judgment on which segmentation can be safely abandoned.  Context smaller than the entire input string is not reliable for the overlapping ambiguity resolution.  Note that exactly the same conclusion has been reached for the hidden ambiguous strings in the previous section.

The following data in (2-11) further illustrate the point of the full context requirement for the overlapping ambiguity resolution, similar to what has been presented for the hidden ambiguity phenomena in (2-3).  In each successive case, the context is expanded to form a new input string.  As a result, the interpretation of ‘goose’ versus ‘swan’ changes accordingly.

(2-11.)  input string                reading(s)

(a)      鹅 e                                goose
[N e]

(b)      天鹅 tian e                                swan
[N tian-e]

(c)      白天鹅 bai tian e                       white swan
[N [A bai] [N tian-e]]

(d)      鹅游过来了 e you guo lai le.
The geese swam over here.
[S [NP e] [VP you guo-lai le]]

(e)      天鹅游过来了 tian e you guo lai le.
The swans swam over here.
[S [NP tian-e] [VP you guo-lai le]]

(f)      白天鹅游过来了 bai tian e you guo lai le.
          (i)       The white swan swam over here.
[S [NP bai tian-e] [VP you guo-lai le]]
          (ii)      In the daytime, the geese swam over here.
S [NP+mod bai-tian] [S [NP e] [VP you guo-lai le]]]

(g)       在白天鹅游过来了 zai bai tian e you guo lai le.
            In the daytime, the geese swam over here.
[S [PP zai bai-tian] [S [NP e] [VP you guo-lai le]]]

(h)      三只白天鹅游过来了 san zhi bai tian e you guo lai le.
           Three white swans swam over here.
[S [NP san zhi bai tian-e] [VP you guo-lai le]]

It is interesting to compare (c) with (f), (g) and (h) to see their associated change of readings based on different ways of  segmentation.  In (c), the overlapping ambiguous string is all that is input to the parser, therefore the local context becomes full context.  It then acquires the reading white swan corresponding to the segmentation bai | tian-e.  This reading may be retained, or changed, or reduced to one of the possible interpretations when the input string is lengthened.  That is respectively the case in (h), (g) and (f).  All these changes depend on the grammatical analysis of the entire input string.  It shows that the full context and a grammar are required for the resolution of most ambiguities;  and when sentential analysis cannot disambiguate - in cases of ‘genuine’ segmentation ambiguity like (f), the structural analysis can make explicit the ambiguity in the form of multiple parses (readings).

In the light of the inquiry in this section, the theoretical significance of the distinction between overlapping ambiguity and hidden ambiguity seems to have diminished.[10]  They are both structural in nature.  They both require full context and a grammar for proper treatment.

(2-12.) Conclusion

(i)  It is not necessarily true that an overlapping ambiguous string can be disambiguated within the local string.

(ii) The grammatical analysis of the entire input string is required for the adequate treatment of the overlapping ambiguity problem as well as the hidden ambiguity problem.

2.2. Productive Word Formation and Syntax

This section examines the connection of productive word formation and segmentation ambiguity.  The observation is that there is always a possible involvement of ambiguity with each type of word formation.  The point to be made is that no independent morphology systems can resolve this ambiguity when syntax is unavailable.  This is because words formed via morphology, just like words looked up from lexicon, only provide syntactic ‘candidate’ constituents for the sentential analysis.  The choice is decided by the structural analysis of the entire sentence.

Derivation is a major type of productive word formation in Chinese.   Section 1.2.2 has given an example of the involvement of hidden ambiguity in derivation, repeated below.

(2-13.)         这道菜没有吃头  zhe dao cai mei you chi tou.

(a)      zhe    | dao          | cai            | mei-you    | chi-tou
          this    | CLA                   | dish         | not-have   | worth-of-eating
[S [NP zhe dao cai] [VP [V mei-you] [NP chi-tou]]]
This dish is not worth eating.

(b) ?   zhe    | dao          | cai            | mei-you    | chi  | tou
          this    | CLA                   | dish         | not have   | eat  | head
[S [NP zhe dao cai] [VP [ADV mei-you] [VP [V chi] [NP tou]]]]
This dish did not eat the head.

(2-14.)         他饿得能吃头牛 ta e de neng chi tou niu.

(a) *   ta       | e              | de            | neng         | chi-tou               | niu
he      | hungry     | DE3         | can           | worth-of-eating  | ox

(b)      ta       | e              | de            | neng         | chi  | tou            | niu
he      | hungry     | DE3         | can           | eat  | CLA          | ox
[…[VP [V e] [DE3P [DE3 de] [VP [V neng] [VP [V chi] [NP tou niu]]]]]]
He is so hungry that he can eat an ox.

Some derivation rule like the one in (2-15) is responsible for combining the transitive verb stem and the suffix –tou (worth-of) into a derived noun for (2-13a) and (2-14a).

(2-15.)         X (transitive verb) + tou --> X-tou (noun, semantics: worth-of-X)

However, when syntax is not available, there is always a danger of wrongly applying this morphological rule due to possible ambiguity involved, as shown in (2-14a).  In other words, morphological rules only provide candidate words;  they cannot make the decision whether these words are legitimate in the context.

Reduplication is another method for productive word formation in Chinese.  An outstanding problem is the AB --> AABB reduplication or AB --> AAB reduplication if AB is a listed word.   In these cases, some reduplication rules or procedures need to be involved to recognize AABB or AAB.  If reduplication is a simple process confined to a local small context, it may be possible to handle it by incorporating some procedure-based function calls during the lexical lookup.  For example, when a three-character string, say 分分心 fen fen xin, cannot be found in the lexicon, the reduplication function will check whether the first two characters are the same, and if yes, delete one of them and consult the lexicon again.  This method is expected to handle the AAB type reduplication, e.g. fen-xin (divide-heart: distract) --> fen-fen-xin (distract a bit).

But, segmentation ambiguity can be involved in reduplication as well.  Compare the following examples in (2-16) and (2-17) containing the sub-string fen fen xin, the first is ambiguity free but the second is ambiguous.  In fact, (2‑17) involves an overlapping ambiguous string  shi fen fen xin:  shi (ten) | fen-fen-xin (distract a bit) and shi-fen (very) | fen-xin (distract).  Based on the conclusion presented in 2.1, it requires grammatical analysis to resolve the segmentation ambiguity.  This is illustrated in (2‑17).

(2-16.)         让他分分心

rang     | ta    | fen-fen-xin
let      | he   | distracted-a-bit
Let him relax a while.

(2-17.)         这件事十分分心

zhe jian shi shi fen fen xin.

(a) *   zhe    | jian          | shi           | shi  | fen-fen-xin
          this    | CLA         | thing       | ten  | distracted a bit

(b)      zhe    | jian          | shi            | shi-fen     | fen-xin
           this    | CLA         | thing       | very         | distract
[S [NP zhe jian shi] [VP [ADV shi-fen] [V fen-xin]]]
This thing is very distracting.

Finally, there is also possible ambiguity involvement in the proper name formation.  Proper names for persons, locations, etc. that are not listed in the lexicon are recognized as another major problem in word identification (Sun and Huang 1996).[11]  This problem is complicated when ambiguity is involved.

For example, a Chinese person name usually consists of a family name followed by a given name of one or two characters.  For example, the late Chinese chairman mao-ze-dong (Mao Zedong) used to have another name li-de-sheng (Li Desheng).  In the lexicon, li is a listed family name.  Both de-sheng and sheng mean ‘win’.  This may lead to three ways of word segmentation, a complicated case involving both overlapping ambiguity and hidden ambiguity:  (i) li | de-sheng;  (ii) li-de | sheng;  (iii) li-de-sheng, as shown in (2-18) below.

(2-18.)         李得胜了 li de sheng le.

(a)      li        | de-sheng  | le
           Li       | win          | LE
[S [NP li] [VP de-sheng le]]
Li won.

(b)      li-de   | sheng       | le
           Li De | win          | LE
[S [NP li de] [VP sheng le]]
Li De won.

(c) *    li-de-sheng  | le
           Li Desheng  | LE

For this particular type of compounding, the family name serves as the left boundary of a potential compound name of person and the length can be used to determine candidates.[12]  Again, the choice is decided by the grammatical analysis of the entire sentence, as illustrated in (2-18).

(2-19.) Conclusion

Due to the possible ambiguity involvement in productive word formation, a grammar containing both morphology and syntax is required for an adequate treatment.  An independent morphology system or separate word segmenter cannot solve ambiguity problems.

2.3. Borderline Cases and Grammar

This section reviews some outstanding morpho-syntactic borderline phenomena.  The points to be made are:  (i) each proposed morphological or syntactic analysis should be justified in terms of capturing the linguistic generality;  (ii) the design of a grammar should facilitate the access to the knowledge from both morphology and syntax in analysis.

The nature of the borderline phenomena calls for the coordination of morphology and syntax in a grammar.  The phenomena of Chinese separable verbs are one typical example.  The co-existence of their contiguous use and separate use leads to the confusion whether they belong to the lexicon and morphology, or whether they are syntactic phenomena.  In fact, as will be discussed in Chapter V, there are different degrees of ‘separability’ for different types of Chinese separable verbs;  there is no uniform analysis which can handle all separable verbs properly.  Different types of separable verbs may justify different approaches to the problems.  In terms of capturing linguistic generality, a good analysis should account for the demonstrated variety of separated uses and link the separated use and the contiguous use.

‘Quasi-affixation’ is another outstanding interface problem.  This problem requires careful morpho-syntactic coordination.  As presented in Chapter I, structurally, ‘quasi-affixes’ and ‘true’ affixes demonstrate very similar word formation potential, but ‘quasi-affixes’ often retain some ‘solid’ meaning while the meaning of ‘true’ affixes are functionalized.  Therefore, how to coordinate the semantic contribution of the derived words via ‘quasi-affixation’ in the context of the building of the semantics for the entire sentence is the key.  This coordination requires flexible information flow between data structures for morphology, syntax and semantics during the morpho-syntactic analysis.

In short, the proper treatment of the morpho-syntactic borderline phenomena requires inquiry into each individual problem in order to reach a morphological or syntactic analysis which maximally captures linguistic generality.  It also calls for the design of a grammar where information between morphology and syntax can be effectively coordinated.

2.4. Knowledge beyond Syntax

This section examines the roles of knowledge beyond syntax in the resolution of segmentation ambiguity.  Despite the fact that further information beyond syntax may be necessary for a thorough solution to segmentation ambiguity,[13] it will be argued that syntax is the appropriate place for initiating this process due to the structural nature of segmentation ambiguity.

Depending on which type of information is essential for the disambiguation, disambiguation can be classified as structure-oriented, semantics-oriented and pragmatics-oriented.  This classification hierarchy is modified from that in He, Xu and Sun (1991).  They have classified the hidden ambiguity disambiguation into three categories:  syntax-based, semantics-based and pragmatics-based.  Together with the morphology-based disambiguation which is equivalent to the overlapping ambiguity resolution in their theory, they have built a hierarchy from morphology up to pragmatics.

A note on the technical details is called for here.  The term X‑oriented (where X is either syntax, semantics or pragmatics) is selected here instead of X-based in order to avoid the potential misunderstanding that X is the basis for the relevant disambiguation.  It will be shown that while information from X is required for the ambiguity resolution, the basis is always syntax.

Based on the study in 2.1, it is believed that there is no morphology-based (or morphology-oriented) disambiguation independent of syntax.  This is because the context of morphology is a local context, too small for resolving structural ambiguity.  There is little doubt that the morphological analysis is a necessary part of word identification in terms of handling productive word formation.  But this analysis cannot by itself resolve ambiguity, as argued in 2.2.  The notion 'structure' in structure-oriented disambiguation includes both syntax and morphology.

He, Xu and Sun (1991) exclude the overlapping ambiguity resolution in the classification beyond morphology.  This exclusion is found to be not appropriate.  In fact, both the resolution of hidden ambiguity and overlapping ambiguity can be classified into this hierarchy.   In order to illustrate this point, for each such class, I will give examples from both hidden ambiguity and overlapping ambiguity.

Sentences in (2-20) and (2-21) which contain the hidden ambiguity string 阵风zhen feng  are examples for the structure-oriented disambiguation.  This type of disambiguation relying on a grammar constitutes the bulk of the disambiguation task required for word identification.

(2-20.)         一阵风吹过来了
yi zhen feng chui guo lai le.          (Feng 1996)

(a)      yi       | zhen         | feng         | chui          | guo-lai      | le
          one    | CLA          | wind        | blow         | over-here  | LE
[S [NP [CLAP yi zhen] [N feng]] [VP chui guo-lai le]]
A gust of wind blew here

(b) *   yi       | zhen-feng                    | chui                   | guo-lai      | le
          one    | gusts-of-wind    | blow         | over-here  | LE

(2-21.)         阵风会很快来临 zhen feng hui hen kuai lai lin.

(a)      zhen-feng              | hui  | hen          | kuai         | lai-lin
          gusts-of-wind       | will | very                   | soon         | come
[S [NP zhen-feng] [VP hui hen kuai lai-lin]]]
Gusts of wind will come very soon.

(b) *   zhen  | feng                   | hui  | hen          | kuai         | lai-lin
          CLA   | wind        | will | very                   | soon         | come

Compare (2-20a) where the ambiguity string is identified as two words zhen (CLA) feng (wind) and (2-21a) where the string is taken as one word zhen-feng (gusts-of-wind).  Chinese syntax defines that a numeral cannot directly combine with a noun, neither can a classifier alone when it is in non-object position.  The numeral and the classifier must combine together before they can combine with a noun.  So (2-20b) and (2‑21b) are both ruled out while (2-20a) and (2-21a) are structurally well-formed.

For the structure-oriented overlapping ambiguity resolution,  numerous examples have been cited before, and one typical example is repeated below.

(2-22.)         研究生命金贵 yan jiu sheng ming jin gui

(a)      yan-jiu-sheng       | ming         | jin-gui
graduate student | life            | precious
[S [NP yan-jiu-sheng] [S [NP ming] [AP jin-gui]]]
Life for graduate students is precious.

(b) *   yan-jiu        | sheng-ming        | jin-gui
study          | life                     | precious

As a predicate, the adjective jin-gui (precious) syntactically expects an NP as its subject, which is saturated by the second NP ming (life) in (2-22a).   The first NP serves as a topic of the sentence and is semantically linked to the subject ming (life) as its possessive entity.[14]  But there is no parse for (2-22b) despite the fact that the sub-string yan-jiu sheng-ming (to study life) forms a verb phrase [VP [V yan-jiu] [NP sheng-ming]] and the sub-string sheng-ming jin-gui (life is precious) forms a sentence [S [NP sheng-ming] [AP jin-gui]].  On one hand, the VP in the subject position does not satisfy the syntactic constraint (the category NP) expected by the adjective jin-gui (precious) - although other adjectives, say zhong-yao 'important', may expect a VP subject.  On the other hand, the transitive verb yan-jiu (study) expects an NP object.  It cannot take an S object (embedded object clause) as do other verbs, say ren-wei (think).

The resolution of the following hidden ambiguity belongs to the semantics-oriented disambiguation.

(2-23.)         请把手抬高一点儿 qing ba shou tai gao yi dian er            (Feng 1996)

(a1)    qing             | ba   | shou         | tai            | gao | yi-dian-er
          please          | BA  | hand        | hold         | high| a-little
[VP [ADV qing] [VP ba shou tai gao yi-dian-er]]
Please raise your hand a little higher.

(a2) * qing   | ba   | shou         | tai            | gao           | yi-dian-er
          invite | BA  | hand        | hold         | high         | a-little

(b1) * qing             | ba-shou    | tai            | gao           | yi-dian-er
          please          | N:handle  | hold         | high         | a-little

(b2) ? qing   | ba-shou    | tai            | gao           | yi-dian-er
          invite | N:handle  | hold         | high         | a-little
[VP [VG [V qing] [NP ba-shou]] [VP tai gao yi-dian-er]]
Invite the handle to hold a little higher.

This is an interesting example.  The same character qing is both an adverb ‘please’ and a verb ‘invite’.  (2-23b2) is syntactically valid, but violates the semantic constraint or semantic selection restriction.  The logical object of qing (invite) should be human but ba-shou (handle)  is not human.  The two syntactically valid parses (2-23a1) and (2-23b2), which correspond to two ways of segmentation, are expected to be somehow disambiguated on the above semantic grounds.

The following case is an example of semantics-oriented resolution of the overlapping ambiguity.

(2-24.)         茶点心吃了 cha dian xin chi le.

(a1)    cha    | dian-xin   | chi  | le
tea     | dim sum  | eat  | LE
[S [NP+object cha dian-xin] [VP chi le]]
The tea dim sum was eaten.

(a2) ? cha    | dian-xin   | chi  | le
tea     | dim sum  | eat  | LE
[S [NP+agent cha dian-xin] [VP chi le]]
The tea dim sum ate (something).

(a3) ? cha    | dian-xin   | chi  | le
tea     | dim sum  | eat  | LE
[S [NP+object cha ] [S [NP+agent dian-xin] [VP chi le]]]
Tea, the dim sum ate.

(a4) ? cha    | dian-xin   | chi  | le
tea     | dim sum  | eat  | LE
[S [NP+agent cha ] [VP [NP+object dian-xin] [VP chi le]]]
The tea ate the dim sum.

(b1) ? cha-dian               | xin           | chi  | le
tea dim sum         | heart       | eat  | LE
[S [NP+object cha-dian] [S [NP+agent xin] [VP chi le]]]
The tea dim sum, the heart ate.

(b2) ? cha-dian               | xin           | chi  | le
tea dim sum         | heart        | eat  | LE
[S [NP+agent cha-dian] [VP [NP+object xin] [VP chi le]]]
The tea dim sum ate the heart.

Most Chinese dictionaries contain the listed compound noun cha-dian (tea-dim-sum), but not cha dian-xin which stands for the same thing, namely the snacks served with the tea.  As shown above, there are four analyses for one segmentation and two analyses for the other segmentation.  These are all syntactically legitimate, corresponding to six different readings.  But there is only one analysis which makes sense, namely the implicit passive construction with the compound noun cha dian-xin as the preceding (logical) object in (a1).  All the other five analyses are nonsense and can be disambiguated if the semantic selection restriction that animate being eats (i.e. chi) food is enforced.   Syntactically, (a2) is an active construction with the optional object omitted.  The constructions for (a3) and (b1) are of long distance dependency where the object is topicalized and placed at the beginning.   The SOV (Subject Object Verb) pattern for (a4) and (b2) is a very  restrictive construction in Chinese.[15]

The pragmatics-oriented disambiguation is required for the case where ambiguity remains after the application of both structural and semantic constraints.[16]  The sentences containing this type of ambiguity are genuinely ambiguous within the sentence boundary, as shown with the multiple parses in (2-25) for the hidden ambiguity and (2-26) for the overlapping ambiguity below.

(2-25.)         他喜欢烤白薯 ta xi huan kao bai shu.

(a)      ta       | xi-huan    | kao          | bai-shu.
          he      | like           | bake         | sweet-potato
[S [NP ta] [VP [V xi-huan] [VP [V kao] [NP bai-shu]]]]
He likes baking sweet potatoes.

(b)      ta       | xi-huan    | kao-bai-shu.
          he      | like           | baked-sweet-potato
[S [NP ta] [VP [V xi-huan] [NP kao-bai-shu]]]
He likes the baked sweet potatoes.

(2-26.)         研究生命不好 yan jiu sheng ming bu hao

(a)      yan-jiu-sheng       | ming         | bu   | hao.
          graduate student | destiny     | not | good
[S [NP yan-jiu-sheng] [S [NP ming] [AP bu hao]]]
The destiny of graduate students is not good.

(b)      yan-jiu        | sheng-ming        | bu   | hao.
          study          | life                     | not | good
[S [VP yan-jiu sheng-ming] [AP bu hao]]
It is not good to study life.

An important distinction should be made among these classes of disambiguation.  Some ambiguity must be solved in order to get a reading during analysis.  Other ambiguity can be retained in the form of multiple parses, corresponding to multiple readings.  In either case, it demonstrates that at least a grammar (syntax and morphology) is required.  The structure-oriented ambiguity belongs to the former, and can be handled by the appropriate structural analysis.  The semantics-oriented ambiguity and the pragmatics-oriented ambiguity belong to the latter, so multiple parses are a way out.  The examples for different classes of ambiguity show that the structural analysis is the foundation for handling ambiguity problems in word identification.  It provides possible structures for the semantic constraints or pragmatic constraints to work on.

In fact, the resolution of segmentation ambiguity in Chinese word identification is but a special case of the resolution of structural ambiguity for NLP in general.  As a matter of fact, the grammatical analysis has been routinely used to resolve, and/or prepare the basis for resolving, the structural ambiguity like the PP attachment.[17]

2.5. Summary

The most important discovery in the field of Chinese word identification presented in this chapter is that the resolution of both types of segmentation ambiguity involves the analysis of the entire input string.  This means that the availability of a grammar is the key to the solution of this problem.

This chapter has also examined the ambiguity involvement in productive word formation and reached the following conclusion.  A grammar for morphological analysis as well as for sentential analysis is required for an adequate treatment of this problem.  This establishes the foundation for the general design of CPSG95 as consisting of morphology and syntax in one grammar formalism. [18]

The study of the morpho-syntactic borderline problems shows that  the sophisticated design of a grammar is called for so that information between morphology and syntax can be effectively coordinated.  This is the work to be presented in Chapter III and Chapter IV.  It also demonstrates that each individual borderline problem should be studied carefully in order to reach a morphological or syntactic analysis which maximally captures linguistic generality.  This study will be pursued in Chapter V and Chapter VI.

----------------------------------------------------------

[1]  Constraints beyond morphology and syntax can be implemented as subsequent modules, or “filters”, in order to select the correct analysis when morpho-syntactic analysis leads to multiple results (parses).  Alternatively, such constraints can also be integrated into CPSG95 as components parallel to, and interacting with, morphology and syntax.  W. Li (1996) illustrates how semantic selection restriction can be integrated into syntactic constraints in CPSG95 to support Chinese parsing.

[2] In theory, if discourse is integrated in the underlying grammar, the input can be a unit larger than sentence, say, a paragraph or even a full text.  But this will depend on the further development in discourse theory and its formalization.  Most grammars in current use assume sentential analysis.

[3]  Similar examples for the overlapping ambiguity string will be shown in 2.1.2.

[4]  But in Ancient Chinese, a numeral can freely combine with countable nouns.

[5] These two readings in written Chinese correspond to an obvious difference in Spoken Chinese:  ge (CLA) in (g1) is weakened in pronunciation, marked by the dropping of the tone, while in (g2) it reads with the original 4^th tone emphatically.

[6] It is likely that what they have found corresponds to Guo’s discovery of “one tokenization per source” (Guo 1998).  Guo’s finding is based on his experimental study involving domain (“source”) evidence and seems to account for the phenomena better.  In addition, Guo’s strategy in his proposal is also more effective, reported to be one of the best strategies for disambiguation in word segmenters.

[7] According to He, Xu and Sun (1991)'s statistics on a corpus of 50833 Chinese characters, the overlapping ambiguous strings make up 84.10%, and the hidden ambiguous strings 15.90%, of all ambiguous strings.

[8] Guo (1997b) goes to the other extreme to hypothesize that “every tokenization is possible”.   Although this seems to be a statement too strong, the investigation in this chapter shows that at least domain independently, local context is very unreliable for making tokenization decision one way or the other.

[9] However, this assumption may become statistically valid within a specific domain or source, as examined in Guo (1998).  But Guo did not give an operational definition of source/domain.  Without such a definition, it is difficult to decide where to collect the domain-specific information required for disambiguation based on the principle one tokenization per source, as proposed by Guo (1998).

[10] This distinction is crucial in the theories of Liang (1987) and He,  Xu and Sun (1991).

[11] This work is now defined as one fundamental task, called Named Entity tagging, in the world of information extraction (MUC-7 1998).  There has been great advance in developing Named Entity taggers both for Chinese (e.g. Yu et al 1997; Chen et al 1997) and for other languages.

[12] That is what was actually done with the CPSG95 implementation.  More precisely, the family name expects a special sign with hanzi-length of 1 or 2 to form a full name candidate.

[13] A typical, sophisticated word segmenter making reference to knowledge beyond syntax is presented in Gan (1995).

[14] This is in fact one very common construction in Chinese in the form of NP1 NP2 Predicate.  Other examples include ta (he) tou (head) tong (ache): ‘he has a head-ache’ and ta (he) shen-ti (body) hao (good): 'he is good in health'.

[15] For the detailed analysis of these constructions, see W. Li (1996).

[16] It seems that it may be more appropriate to use terms like global disambiguation or discourse-oriented disambiguation instead of the term pragmatics-oriented disambiguation for the relevant phenomena.

[17] It seems that some PP attachment problems can be resolved via grammatical analysis alone.  For example, put something on the table; found the key to that door.  Others require information beyond syntax (semantics, discourse, etc.) for a proper solution.  For example, see somebody with telescope. In either case, the structural analysis provides a basis.  The same thing happens to the disambiguation in Chinese word identification.

[18] In fact, once morphology is incorporated in the grammar, the identification of both vocabulary words and non-listable words becomes a by-product during the integrated morpho-syntactic analysis.  Most ambiguity is resolved automatically and the remaining ambiguity will be embodied in the multiple syntactic trees as the results of the analysis.  This has been shown to be true and viable by W. Li (1997, 2000) and Wu and Jiang (1998).

[Related]

PhD Thesis: Morpho-syntactic Interface in CPSG (cover page)

PhD Thesis: Chapter I Introduction

PhD Thesis: Chapter II Role of Grammar

PhD Thesis: Chapter III Design of CPSG95

PhD Thesis: Chapter IV Defining the Chinese Word

PhD Thesis: Chapter V Chinese Separable Verbs

PhD Thesis: Chapter VI Morpho-syntactic Interface Involving Derivation

PhD Thesis: Chapter VII Concluding Remarks

Overview of Natural Language Processing

Dr. Wei Li’s English Blog on NLP

1.0. Foreword

This thesis addresses the issue of the Chinese morpho-syntactic interface.  This study is motivated by the need for a solution to a series of long-standing problems at the interface.  These problems pose challenges to an independent morphology system or a separate word segmenter as there is a need to bring in syntactic information in handling these problems.

The key is to develop a Chinese grammar which is capable of representing sufficient information from both morphology and syntax.  On the basis of the theory of Head-Driven Phrase Structure Grammar (Pollard and Sag 1987, 1994), the thesis will present the design of a Chinese grammar, named CPSG95 (for Chinese Phrase Structure Grammar).  The interface between morphology and syntax is defined system internally in CPSG95.  For each problem, arguments will be presented for the linguistic analysis involved.  A solution to the problem will then be formulated based on the analysis.  The proposed solutions are formalized and implementable;  most of the proposals have been tested in the implementation of CPSG95.

In what follows, Section 1.1 reviews some important developments in the field of Chinese NLP (Natural Language Processing).  This serves as the background for this study.  Section 1.2 presents a series of long-standing problems related to the Chinese morpho-syntactic interface.  These problems are the focus of this thesis.  Section 1.3 introduces CPSG95 and sketches its morpho-syntactic interface by illustrating an example of the proposed morpho-syntactic analysis.

1.1. Background

This section presents the background for the work on the interface between morphology and syntax in CPSG95.  Major development on Chinese tokenization and parsing, the two areas which are related to this study, will be reviewed.

1.1.1. Principle of Maximum Tokenization and Critical Tokenization

This section reviews the influential Theory of Critical Tokenization (Guo 1997a) and its implications.  The point to be made is that the results of Guo’s study can help us to select the tokenization scheme used in the lexical lookup phase in order to create the basis for morpho-syntactic parsing.

Guo (1997a,b,c) has conducted a comprehensive formal study on tokenization schemes in the framework of formal languages, including deterministic tokenization such as FT (Forward Maximum Tokenization) and BT (Backward Maximum Tokenization), and non-deterministic tokenization such as CT (Critical Tokenization), ST (Shortest Tokenization) and ET (Exhaustive Tokenization).  In particular, Guo has focused on the study of the rich family of tokenization strategies following the general Principle of Maximum Tokenization, or “PMT”.  Except for ET, all the tokenization schemes mentioned above are PMT-based.

In terms of lexical lookup, PMT can be understood as a heuristic by which a longer match overrides all shorter matches.  PMT has been widely adopted (e.g. Webster and Kit 1992; Guo 1997b) and is believed to be “the most powerful and commonly used disambiguation rule” (Chen and Liu 1992:104).

Shortest Tokenization, or “ST”, first proposed by X. Wang (1989), is a non-deterministic tokenization scheme following the Principle of Maximum Tokenization.  A segmented token string is shortest if it contains the minimum number of vocabulary words possible - “short” in the sense of the shortest word string length.

Exhaustive Tokenization, or “ET”, does not follow PMT.  As its name suggests, the ET set is the universe of all possible segmentations consisting of all candidate vocabulary words.  The mathematical definition of ET is contained in Definition 4 for “the character string tokenization operation”  in Guo (1997a).

The most important concept in Guo’s theory is Critical Tokenization, or “CT”.  Guo’s definition is based on the partially ordered set, or ‘poset’, theory in discrete mathematics (Kolman and Busby 1987).  Guo has found that different segmentations can be linked by the cover relationship to form a poset.   For example, abc|d and ab|cd both cover ab|c|d, but they do not cover each other.

Critical tokenization is defined as the set of minimal elements, i.e. tokenizations which are not covered by other tokenizations, in the tokenization poset.  Guo has given proof for a number of mathematical properties involving critical tokenization.  The major ones are listed below.

• Every tokenization is a subtokenization of (i.e. covered by) a critical tokenization, but no critical tokenization has a true supertokenization;
• The tokenization variations following the Principle of Maximum Tokenization proposed in the literature, such as FT, BT, FT+BT and ST, are all true sub-classes of CT.

Based on these properties, Guo concludes that CT is the precise mathematical description of the widely adopted Principle of Maximum Tokenization.

Guo (1997c) further reports his experimental studies on relative merits of these tokenization schemes in terms of three quality indicators, namely, perplexity, precision and recall.  The perplexity of a tokenization scheme gives the expected number of tokenized strings generated for average ambiguous fragments.  The precision score is the percentage of correctly tokenized strings among all possible tokenized strings while the recall rate is the percentage of correctly tokenized strings generated by the system among all correctly tokenized strings.  The main results are:

• Both FT and BT can achieve perfect unity perplexity but have the worst precision and recall;
• ET achieves perfect recall but has the lowest precision and highest perplexity;
• ST and CT are simple with good computational properties.  Between the two, ST has lower perplexity but CT has better recall.

Guo (1997c) concludes, “for applications with moderate performance requirement, ST is the choice;  otherwise, CT is the solution.”

In addition to the above theoretical and experimental study, Guo (1997b) also develops a series of optimized algorithms for the implementation of these generation schemes.

The relevance and significance of Guo’s achievement to the research in this thesis lie in the following aspect.  The research on Chinese morpho-syntactic interface is conducted with the goal of  supporting Chinese morpho-syntactic parsing.  The input to a Chinese morpho-syntactic parser comes directly from the lexical lookup of the input string based on some non-deterministic tokenization scheme (W. Li 1997, 2000; Wu and Jiang 1998).  Guo’s research and algorithm development can help us to decide which tokenization schemes to use depending on the tradeoff between precision, recall and perplexity or the balance between reducing the search space and minimizing premature commitment.

1.1.2. Monotonicity Principle and Task-driven Segmentation

This section reviews the recent development on Chinese analysis systems involving the interface between morphology and syntax.  The research on the Chinese morpho-syntactic interface in this thesis echoes this new development in the field of Chinese NLP.

In the last few years, projects have been proposed for implementing a Chinese analysis system which integrates word identification and parsing.  Both rule-based systems and statistical models have been attempted with good results.

Wu (1998) has addressed the drawbacks of the conventional practice on the development of Chinese word segmenters, in particular, the problem of premature commitment in handling segmentation ambiguity.  In his A Position Statement on Chinese Segmentation, Wu proposed a general principle:

Monotonicity Principle for segmentation:

A valid basic segmentation unit (segment or token) is a substring that no processing stage after the segmenter needs to decompose.

The rationale behind this principle is to prevent premature commitment and to avoid repetition of work between modules.   In fact, traditional word segmenters are modules independent of subsequent applications (e.g. parsing).  Due to the lack of means for accessing sufficient grammar knowledge, they suffer from premature commitment and repetition of work, hence violating this principle.

Wu’s proposal of the monotonicity principle is a challenge to the Principle of Maximum Tokenization.  These two principles are not always compatible.  Due to the existence of hidden ambiguity (see 1.2.1), the PMT-based segmenters by definition are susceptible to premature commitment leading to “too-long segments”.  If the target application is designed to solve the hidden ambiguity problem in the segments, “decomposition” of some segments is unavoidable.

In line with the Monotonicity Principle, Wu (1998) proposes an alternative approach which he claims “eliminates the danger of premature commitment”, namely task-driven segmentation.  Wu (1998) points out, “Task-driven segmentation is performed in tandem with the application (parsing, translating, named-entity labeling, etc.) rather than as a preprocessing stage.  To optimize accuracy, modern systems make use of integrated statistically-based scores to make simultaneous decisions about segmentation and parsing/translation.”  The HKUST parser, developed by Wu’s group, is such a statistical system employing the task-driven segmentation.

As for rule-based systems, similar practice of integrating word identification and parsing has also been explored.  W. Li (1997, 2000) proposed that the results of an ET-based lexical lookup directly feed the parser for the hanzi-based parsing.  More concretely, morphological rules are designed to build word internal structure for productive morphology and non-productive morphology is lexicalized via entry enumeration.[1]  This approach is the background for conducting the research on Chinese morpho-syntactic interface for CPSG95 in this dissertation.

The Chinese parser on the platform of multilingual NLPWin developed by Microsoft Research also integrates word identification and parsing (Wu and Jiang 1998).  They also use a hand-coded grammar for word identification as well as for sentential parsing.  The unique part of this system is the use of a certain lexical constraint on ET in the lexical lookup phase.  This effectively reduces the parsing search space as well as the number of syntactic trees produced by the parser, with minimal sacrifice in the recall of tokenization.  This tokenization strategy provides a viable alternative to the PMT-based tokenization schemes like CT or ST in terms of the overall balance between precision, recall and perplexity.

The practice of simultaneous word identification and parsing in implementing a Chinese analysis system calls for the support of a grammar (or statistical model) which contains sufficient information from both morphology and syntax.  The research on Chinese morpho-syntactic interface in this dissertation aims at providing this support.

1.2. Morpho-syntactic Interface Problems

This section presents a series of outstanding problems in Chinese NLP which are related to the morpho-syntactic interface.  One major goal of this dissertation is to argue for the proposed analyses of the problems and to provide solutions to them based on the analyses.

Sun and Huang (1996) have reviewed numerous cases which challenge the existing word segmenters.  As many of these cases call for an exchange of information between morphology and syntax, an appropriate solution can hardly be reached within the module of a separate word segmenter.  Three major problems at issue are presented below.

1.2.1. Segmentation ambiguity

This section presents the long-standing problem in Chinese tokenization, i.e. the resolution of the segmentation ambiguity.  Within a separate word segmenter, resolving the segmentation ambiguity is a difficult, sometimes hopeless job.  However, the majority of ambiguity can be resolved when a grammar is available.

Segmentation ambiguity has been the focus of extensive study in Chinese NLP for the last decade (e.g. Chen and Liu 1992; Liang 1987;  Sproat, Shih, Gale and Chang 1996; Sun and Huang 1996; Guo 1997b).  There are two types of segmentation ambiguities (Liang 1987; Guo 1997b):  (i) overlapping ambiguity:  e.g. da-xue | sheng-huo vs. da-xue-sheng | huo as shown in (1-1) and (1-2);  and (ii) hidden ambiguity:  ge-ren vs. ge | ren, as shown in (1-3) and (1-4).

(1-1.) 大学生活很有趣
da-xue         | sheng-huo          | hen          | you-qu
university    | life                     | very          | interesting
The university life is very interesting.

(1-2.)  大学生活不下去了
da-xue-sheng                 | huo          | bu | xia-qu      | le
university student          | live           | not | down        | LEs
University students can no longer make a living.

(1-3.)  个人的力量
ge-ren         | de   | li-liang
individual   | DE  | power
the power of an individual

(1-4.) 三个人的力量
san    |  ge            | ren           | de   | li-liang
three  | CLA          | person      |DE   | power
the power of three persons

These examples show that the resolution of segmentation ambiguity requires larger syntactic context and grammatical analysis.   There will be further arguments and evidence in Chapter II (2.1) for the following conclusion:  both types of segmentation ambiguity are structural by nature and require sentential analysis for the resolution.  Without access to a grammar, no matter how sophisticated a tokenization algorithm is designed, a word segmenter is bound to face an upper bound for the precision of word identification.  However, in an integrated system, word identification becomes a natural by-product of parsing (W. Li 1997, 2000;  Wu and Jiang 1998).  More precisely, the majority of ambiguity can be resolved automatically during morpho-syntactic parsing;  the remaining ambiguity can be made explicit in the form of  multiple syntactic trees.[2]  But in order to make this happen, the parser requires reliable support from a grammar which contains both morphology and syntax.

1.2.2. Productive Word Formation

Non-listable words created via productive morphology pose another challenge (Sun and Huang 1996).  There are two major problems involved in this issue:  (i) problem in identifying lexicon-unlisted words;  (ii) problem of possible segmentation ambiguity.

One important method of productive word formation is derivation.  For example, the derived word 可读性 ke-du-xing (-able-read-ness: readability) is created via morphology rules, informally formulated below

(1-5.) derivation rules

ke + X (transitive verb) --> ke-X (adjective, semantics: X-able)

Y (adjective or verb) + xing --> Y-xing (abstract noun, semantics: Y-ness)

Rules like the above have to be incorporated properly in order to correctly identify such non-listable words.  However, there has been little research in the literature on what formalism should be adopted for Chinese morphology  and how it should be interfaced to syntax.

To make the case more complicated, ambiguity may also be involved in productive word formation.  When the segmentation ambiguity is involved in word formation, there is always a danger of wrongly applying morphological rules.  For example, 吃头 chi-tou (worth of eating) is a derived word (transitive verb + suffix tou);   however, it can also be segmented as two separate tokens chi (eat) | tou (CLA), as shown in (1-6) and (1-7) below.

(1-6.)  这道菜没有吃头
zhe    | dao           | cai            | mei-you    | chi-tou
this    | CLA          | dish         | not have   | worth-of-eating
This dish is not worth eating.

(1-7.) 他饿得能吃头牛
ta       | e               | de             | neng        | chi  | tou           | niu
he      | hungry     | DE3         | can           | eat  | CLA                   | ox
He is so hungry that he can eat an ox.

To resolve this segmentation ambiguity, as indicated before in 1.2.1, the structural analysis of the complete sentences is required.  An independent morphology system or a separate word segmenter cannot handle this problem without accessing syntactic knowledge.

1.2.3. Borderline Cases between Morphology and Syntax

It is widely acknowledged that there is a remarkable gray area between Chinese morphology and Chinese syntax (L. Li 1990; Sun and Huang 1996).  Two typical cases are described below.  The first is the phenomena of Chinese separable verbs.  The second case involves interfacing derivation and syntax.

Chinese separable verbs are usually in the form of V+N and V+V or V+A.  These idiomatic combinations are long-standing problems at the interface between compounding and syntax in Chinese grammar (L. Wang 1955; Z. Lu 1957; Lü 1989; Lin 1983; Q. Li 1983; L. Li 1990; Shi 1992; Zhao and Zhang 1996).

The separable verb 洗澡 xi zao (wash‑bath: take a bath) is a typical example.  Many native speakers regard xi zao as one word (verb), but the two morphemes are separable.  In fact, xi+zao shares the syntactic behavior and the pattern variations with the syntactic transitive combination V+NP:  not only can aspect markers appear between xi and zao,  but this structure can be passivized and topicalized as well.  The following is an example of topicalization (of long distance dependency) for xi zao.

(1-8.)(a)       我认为他应该洗澡
wo     ren-wei        ta       ying-gai       xi zao.
I         think           he      should        wash-bath
I think that he should take a bath.

(b)      澡我认为他应该洗
zao    wo     ren-wei        ta       ying-gai       xi.
bath  I         think           he      should        wash
The bath I think that he should take.

Although xi zao behaves like a syntactic phrase, it is a vocabulary word in the lexicon due to its idiomatic nature.  As a result, almost all word segmenters output xi-zao in (1-8a) as one word while treating the two signs[3] in (1-8b) as two words.  Thus the relationship between the separated use of the idiom and the non-separated use is lost.

The second case represents a considerable number of borderline cases often referred to as  ‘quasi-affixes’.  These are morphemes like 前 qian (former, ex-) in words like 前夫 qian-fu (ex-husband), 前领导 qian-[ling-dao] (former boss) and -盲 mang (person who has little knowledge of) in words like 计算机盲 [ji-suan-ji]-mang (computer layman), 法盲 fa-mang (person who has no knowledge of laws).

It is observed that 'quasi-affixes' are structurally not different from other affixes.  The major difference between 'quasi-affixes' and the few generally honored ('genuine') affixes like the nominalizer 性 -xing (-ness) lies mainly in the following aspect.  The former retain some 'solid' meaning while the latter are more functionalized.  Therefore, the key to this problem seems to lie in the appropriate way of coordinating the semantic contribution of the derived words using 'quasi-affixes' to the building of the semantics for the entire sentence.  This is an area which has not received enough investigation in the field of Chinese NLP.  While many word segmenters have included some type of derivational processing for a few typical affixes, few systems demonstrate where and how to handle these 'quasi-affixes'.

1.3. CPSG95:  HPSG-style Chinese Grammar in ALE

To investigate the interaction between morphological and syntactic information, it is important to develop a Chinese grammar which incorporates morphology and syntax in the same formalism.  This section gives a brief presentation on the design and background of CPSG95 (including lexicon).

1.3.1. Background and Overview of CPSG95

Shieber (1986) distinguishes two types of grammar formalism:  (i) theory-oriented formalism;  (ii) tool-oriented formalism.  In general, a language-specific grammar turns to a theory-oriented formalism for its foundation and a tool-oriented formalism for its implementation.  The work on CPSG95 is developed in the spirit of the theory-oriented formalism Head-driven Phrase Structure Grammar (HPSG, proposed by Pollard and Sag 1987).  The tool-oriented formalism used to implement CPSG95 is the Attribute Logic Engine (ALE, developed by Carpenter and Penn 1994).

The unique feature of CPSG95 is its incorporation of Chinese morphology in the HPSG framework.[4]  Like other HPSG grammars, CPSG95 is a heavily lexicalized unification grammar.  It consists of two parts:  a minimized general grammar and an information-enriched lexicon.  The general grammar contains a small number of Phrase Structure (PS) rules, roughly corresponding to the HPSG schemata tuned to the Chinese language.[5]  The syntactic PS rules capture the subject-predicate structure, complement structure, modifier structure, conjunctive structure and long-distance dependency.  The morphological PS rules cover morphological structures for productive word formation.  In one version of CPSG95 (its source code is  shown in APPENDIX I), there are nine PS rules:  seven syntactic rules and two morphological rules.

In CPSG95, potential morphological structures and potential syntactic structures are both lexically encoded.  In syntax, a word can expect (subcat-for or mod in HPSG terms) another sign to form a phrase.   Likewise, in Chinese morphology, a morpheme can expect another sign to form a word.[6]

One important modification of HPSG in designing CPSG95 is to use an atomic approach with separate features for each complement to replace the list design of obliqueness hierarchy among complements.  The rationale and arguments for this modification are presented in Section 3.2.3 in Chapter III.

1.3.2. Illustration

The example shown in (1-9) demonstrates the morpho-syntactic analysis  in CPSG95.

(1-9.) 这本书的可读性
zhe    ben    shu    de      ke               du      xing
this    CLA   book  DE     AF:-able      read   AF:-ness
this book’s readability
(Note: CLA for classifier; DE for particle de; AF for affix.)

Figure 1 illustrates the tree structure built by the morphological PS rules and the syntactic PS rules in CPSG95

Figure 1. Sample Tree Structure for CPSG95 Analysis

As shown, the tree embodies both morphological analysis (the sub-tree for ke-du-xing) and syntactic analysis (the NP structure).  The results of the morphological analysis (the category change from V to A and to N and the building of semantics, etc.) are readily accessible in building syntactic structures.

1.4. Organization of the Dissertation

The remainder of this dissertation is divided into six chapters.

Chapter II presents arguments for the need to involve syntactic analysis for a proper solution to the targeted morpho-syntactic problems.   This establishes the foundation on which CPSG95 is based.

Chapter III presents the design of CPSG95.  In particular, the expectation feature structures will be defined.  They are used to encode the lexical expectation of both morphological and syntactic structures.  This design provides the necessary means for formally defining Chinese word and the interface of morphology, syntax and semantics.

Chapter IV is on defining the Chinese word.  This is generally recognized as a basic issue in discussing Chinese morpho-syntactic interface.  The investigation leads to a way of the wordhood formalization and a coherent, system-internal definition of the work division between morphology and syntax.

Chapter V studies Chinese separable verbs.  It discusses  wordhood judgment for each type of separable verbs based on their distribution.   The corresponding morphological or syntactic solutions will then be presented.

Chapter VI investigates some outstanding problems of Chinese derivation and its interface with syntax.  It will be demonstrated that the general approach to Chinese derivation in CPSG95 works both for typical cases of derivation and the two special problems, namely 'quasi-affix' phenomena and zhe-affixation.

The last chapter, Chapter VII, concludes this dissertation.  In addition to a concise retrospect for what has been achieved, it also gives an account of the limitations of the present research and future research directions.

Finally, the three appendices give the source code of one version of the implemented CPSG95 and some tested results.[7]

--------------------------------------------------

[1] In line with the requirements by Chinese NLP, this thesis places emphasis on the analysis of productive morphology:  phenomena which are listable in the lexicon are not the major concern.  This is different from many previous works on Chinese morphology (e.g. Z. Lu 1957; Dai 1993) where the bulk of discussions is on unproductive morphemes (affixes or ‘bound stems’).

[2] Ambiguity which remains after sentential parsing may be resolved by using further semantic, discourse or pragmatic knowledge, or ‘filters’.

[3] In CPSG95 and other HPSG-style grammars, a ‘sign’ usually stands for the generalized notion of grammatical units such as morpheme, word, phrase, etc.

[4] Researchers have looked at the incorporation of morphology of other natural languages in the HPSG framework (e.g. Type-based Derivation Morphology by Riehemann 1998).  Arguments for  the inclusion of morphological features in the definition of sign will be presented in detail in Chapter III

[5] Note that ‘phrase structure’ in terms like Phrase Structure Grammar (PSG) or Phrase Structure rules (PS rules) does not necessarily refer to structures of (syntactic) phrases. It stands for surface-based constituency structure, in contrast to, say, dependency structure in Dependency Grammar.  In CPSG95, some productive morphological structures are also captured by PS rules.

[6] Note that in this dissertation, the term expect is used as a more generalized notion than the terms subcat-for (subcategorize for) and mod (modify).  ‘Expect’ is intended to be applied to morphology as well as to syntax.

[7]  There are differences in technical details between the proposed grammar in this dissertation and the implemented version.  This is because any implemented version was tested at a given time while this thesis evolved over a long period of time.  It is the author’s belief that it best benefits readers (including those who want to follow the CPSG practice) when a version was actually tested and given as was.

[Related]

PhD Thesis: Morpho-syntactic Interface in CPSG (cover page)

PhD Thesis: Chapter I Introduction

PhD Thesis: Chapter II Role of Grammar

PhD Thesis: Chapter III Design of CPSG95

PhD Thesis: Chapter IV Defining the Chinese Word

PhD Thesis: Chapter V Chinese Separable Verbs

PhD Thesis: Chapter VI Morpho-syntactic Interface Involving Derivation

PhD Thesis: Chapter VII Concluding Remarks

Overview of Natural Language Processing

Dr. Wei Li’s English Blog on NLP

看一组例子：
“洗了一个痛快的澡”
“痛快地洗了一个澡”
“洗澡洗得痛快”

好在我们把动宾离合词“洗澡”的搭配问题解决了，定语（Mod）、状语（Adv）同是附加语（adjunct），都挂到了同样的动词“洗澡”身上了，加上部分补语（Buyu）也是附加语，可谓世界大同了。原先较真的话，要问“痛快”的是“澡”，还是“洗”，还是“洗澡”, who cares？其实都是一个意思。类似的，英语也有：
live a happy life
live (a life) happily

白:
do了一个痛快的“洗澡”
程序还是要care的。

我:
如果程序在此类情形下 只选一个路径，或不做规约也是可以的。到语义落地的时候 只要系统适应性鲁棒即可：Adv:happily OR Mod:happy。

白:
借定语的壳装状语的瓤，总要有一个地方碰上的。
“开了一个无聊的会”
工程师可以不 care，架构师必须给说法。
我要说的是，伪定语伪状语在formalism层面就是可以解决的，并不带来额外负担。

我:
do + Adjunct + core pred
已经做了相当努力去规约这些本质上相同的说法了，如前面图中的“洗澡”：Mod 也好 Adv 也好 Buyu 也好，大体属于同样性质的附加语：
adjunct 痛快 ----》 pred 洗澡

白:
“张三做出了一个追悔莫及的决定。”
“张三遇上了这个倒霉的天气。”
“倒霉的”修饰“天气”，但倒霉的不是天气。
同理，“追悔莫及的”修饰“天气”，但追悔莫及的不是天气。
修饰关系和修饰语内置的填坑关系是脱钩的。

我:
“追悔莫及” 本义 有一个 human 的坑
“做出决定” 也有一个 human 的坑
现在 human （张三） 与 “做出决定” 发生了直接联系（S） 与 “追悔莫及” 发生了间接关系（通过“做出决定”）。离开让 human （张三） 与 需要 human 坑的 “追悔莫及”直接联系 只有一步之遥了。

白:
由此可见，有了的字结构，就由“的”统一应对被修饰语。至于修饰语内部的坑由谁填，被修饰语不过只是一个普通的候选而已。选不上不勉强，有更好的候选完全可以进来。所以我对把“的”这种重要的词仅仅处理成x，是有保留看法的。

我:
“的” 是敲门砖。句法树出来了， x它意思意思，比扔掉它也许好一些。

白:
我有更好的处理办法，绝非仅是敲门砖。

我:
关键是，第一个句子是一步之遥，第二个句子是两步之遥，几乎不可能超过两步。也就是说 从ngram角度看 也不过是dag中的 bigram 或 trigram 的语义规则，如果真想做的话。只要证明从间接联系到直接联系 在语义中间件做 对应用有益处 这个工作是非常 tractable 的。
一个有语义的坑 一个正好符合语义可以填坑 近在咫尺 有何难处？给我五分钟 我两条线都可以勾搭上，而且保证不是权宜之计 不引起副作用。其所以这些语义中间件的细活 虽然不难 但并没去全做 是因为不很确定做了 到底能带来多大好处，虽然理论上是有好处的。

白:
这些后缀，几乎每个case都是一样的。

我:
要的是这个结果吗？

白:
一点不错，就是它

我:
我做一下 regression testing 看看有无副作用，没有的话，这个 trigram 的语义填坑规则就留下来。

我:
trigram
具体到这个 case 是从线性 5-gram 缩小成 graph 的 trigram
5 与 3 在组合爆炸的考量中是天壤之别
何况完全可以造出比 5 更加远距离的同样合适的例子来 这就是句法的威力。
更主要的是，即便一个线性系统用得起 5-gram
没有结构支撑，也不敢乱用

白:
5-gram配得上的不稀疏的数据哪里来?

我:
说的是一回事儿 5gram 必然是稀疏数据 不足以支撑远距离选取。不能因为一个token需要human 另一个token恰好是human 中间隔了四个词，就可以填坑了。总之是，没有结构，这事儿就做不成。

【相关】

【置顶：立委NLP博文一览】

《朝华午拾》总目录

【立委科普：信息抽取】

《朝华午拾：信息抽取笔记》

泥沙龙笔记：搜索和知识图谱的话题

《知识图谱的先行：从Julian Hill 说起》

《有了deep parsing，信息抽取就是个玩儿》

【立委科普：实体关系到知识图谱，从“同学”谈起】

泥沙龙笔记： parsing vs. classification and IE

前知识图谱钩沉: 信息抽取引擎的架构 2015-11-01

前知识图谱钩沉: 信息体理论 2015-10-31

前知识图谱钩沉，信息抽取任务由浅至深的定义 2015-10-30

前知识图谱钩沉，关于事件的抽取

钩沉：SVO as General Events

Pre-Knowledge-Graph Profile Extraction Research via SBIR (1)

Pre-Knowledge-Graph Profile Extraction Research via SBIR (2)

Coarse-grained vs. fine-grained sentiment extraction

【立委科普：基于关键词的舆情分类系统面临挑战】

【“剩女”的去向和出路】

SBIR Grants

【关于 parsing】

【关于 NLP 以及杂谈】

【关于人工智能】

【关于NLP体系和设计哲学】

【关于NLP方法论以及两条路线之争】

《朝华午拾》总目录

【置顶：立委NLP博文一览（定期更新版）】

【立委NLP频道】

“快叫爸爸小视频” 这样的东西 有社会语言学的味道 随着时代和潮流翻滚。在微信朋友圈及其提供的小视频功能风靡之前 小视频不是术语 不是合成词 也没有动词的引申用法。它就是一个定中结构的 NP，在句型中等价于说”把爸爸叫做小视频”，虽然常识是 “人（爸爸）不可以等价于物（视频）”。在语言的强制性subcat结构（叫NP1NP2）里面，常识是没有位置的。句法不需要顾及常识 正如 “鸡把我吃了”的违反常识一样 也正如乔姆斯基千古名句的 green ideas。
可是 社会语言学登场了 语言被置于流动的社会背景之下，小视频成了 technical term，然后又从术语融入了语言共同体的动词用法，正如谷歌从术语（专名）变成动词一样： “我还是先谷歌一下再回应吧”，“快小视频呀”，“一定要小视频这个精彩时刻”。
白:
“一下”强制“谷歌”为动词。半个括号已经有了 另半个没有也得有。
我:
于是 subcats 开始 compete，有了 competition，有了结构歧义 就有了常识出场的理由。顺应常识者于是推翻了句法的第一个 reading。
白:
你是我的小苹果，怎解？
我：
“你是我的小苹果”是强制性的句法啊，无论怎么理解这个苹果（到现在我也没有理解为什么把爱人或意中人叫做小苹果，是因为拿高大上的苹果比喻珍贵吗？）都与常识无关：你是我的 x，就是强行的句法等价关系。
“一下”强制“谷歌”为动词 这一类看似临时的强制 在语言共同体中逐渐从临时变成常态后就侵入了词汇。换句话说，“谷歌”在以前的词典里面是没有也无需“潜在动词”的标注（lexical candidate POS feature），因为几乎所有的动词用法都是零星的 句法强制的 无需词典 support 的。但是随着语言的发展 “谷歌”的动词用法逐渐变成了语言共同体司空见惯的表达方式（其动词用法的流行显得简洁、时髦甚至俏皮），这时候 语言的用法被反映在语言共同体的集体词汇表中，我们模型这个共同体的语言能力的时候 就开始标注其动词的可能性了。
金:
厉害，这抠的!金融语义在一边看热闹
我:
或问：这词典里面标注了（反映的是共同体集体意识到这种用法的流行）和不标注 有什么区别？
当然有区别。标注了 就意味着其动词用法作为一个合理的路径 参与 parsing 的正常竞争；不标注 虽然也不能排除临时的动词用法 但是因为缺乏了底部的词典支持 其动词用法的路径是默认不合法，除非句法（包括词法）的context逼迫它成为动词，这就是 “一哈”的所谓强盗句法： 不仅词典是绑架的天堂，句法也可以绑架。
白老师说：“兼语理解（叫某人做某事）有谓词性的坑不饱和，双宾理解（叫某人某称呼）有体词性的坑不饱和。如果拘泥于结构，二者半斤八两。但如果结合语境，非兼语理解是颠覆性的，兼语理解是常识性的。放着常识性的理解不选选择颠覆性的理解，说明心头的阴云不是一天两天了。冰冻三尺。”
重温一下白老师 作为对比，字字玑珠，而且妙趣啊。“冰冻三尺”就是社会语言学。
白
也可以说，冰冻三尺就是大数据
我:
我们学习语言学 模型句法 绝大多数都是针对现时的 把语言看成是一个静态的剖面 来研究它 模型它。这个也没大错 而且简化了问题。但是语言是流动的 社会语言学强调的就是这个流动性。流动自然反映在大数据中。因此对于静态的语言模型 需要不断的更新 如果有大数据 那就定时地 check 它。
白：
有个动态更新的中间件就够了
我:
陈原是个大家。他写的社会语言学很有趣味。在世界语场合 有幸聆听过陈原先生的世界语演讲：那个才华四射、感染力和个性特色 让人高山仰止。人家做语言学是业余 本职工作是出版商。据说是中国最权威的出版家，也是个左派社会活动家。
洪:
虽然解放初才入党，但应该早就是中共地下党员，三十年代初就在三联当编辑，胡愈之邹韬奋的部下，以前《读书》上一直有陈原的《在语词的密林里》
我:
陈原的那次演讲 与 黄华(我做翻译的那次)的演讲 都有一个共同的特点，就是表情丰富、富于感染力，能感受到人的 personality，都是“大家”。

【相关】

《我的世界语国》

《朝华午拾：欧洲之行》

【置顶：立委科学网博客NLP博文一览（定期更新版）】

《朝华午拾》总目录

承蒙高博协助，立委牌NLP博客频道今天开张大吉，广告一下，尤其对后学：https://liweinlp.com/

其前身是【立委科学网博客】的NLP科普相关博文，所谓 NLP University： http://blog.sciencenet.cn/blog-362400-902391.html。我将逐渐把原NLP博客转移至此，新的博客会同步在此发布。非 NLP 博文仍然以科学网为基地。

本大学有网无墙，有教无类，对公众无条件全天候开放。学分以研读立委教授博文为单元，从下列清单任选100篇博文，计100分，急用先学，学有所得，学以致用，是为有效学分，学员自我判分，过60可毕业也。门槛说高不高，说低不低，师傅领进门，修行靠个人，能否诚实毕业，就看造化了。

不知道多少次电脑输入 NLP（自然语言处理），出来的都是“你老婆”。难怪 NLP 跟了我一辈子，or 我跟了 NLP 一辈子。不离不弃。

开篇词： 余致力自然语言处理凡30年，其目的在求交流之通畅，信息之自由，语言之归一，世界之大同。积30年之经验，深知欲达此目的，必须启蒙后进，普及科学，同心协力，共建通天之塔，因作文鼓而吹之。处理尚未成功，同志仍需努力。

分八章。

第一章：体系和方法论，关键是这一篇【NLP 联络图 】。除了体系和术语联络图，也谈方法论及其两条路线的斗争。

第二章 Parsing，包括 shallow parsing 和 deep parsing 的方方面面。要强调的一点是，deep parsing 是 NLP 的核武器。当自然语言的 unstructured text 被精准分析成 structures 以后，语言因为有了有限的 patterns 而变得有迹可循，NLP 应用的很多难题就迎刃而解了。

第三章 抽取，进入NLP语用。虽然学界绝大多数抽取都是不用parsing的，或者只用 stemming，最多是 shallow parsing，这里更注重的是在 deep parsing 基础上的抽取。可以看成是针对知识图谱的全自动最终解决方案。

科学网【NLP University 】