【李白66:“青春期父母指南”的语义计算】

李:
朋友圈看到一本书书名是:【青春期父母指南】(副标题:如何和谜一般的青春期孩子相处呢?)定语青春期不是指向父母的,而是指向父母的儿女。

有意思的是,定语“青春期”不是指向“父母”的,而是指向父母的孩子。书名的结构是 NP:N1+N2+N3:N1 is neither semantic Mod of N2 nor of N3, but of something only implied yet not mentioned in text.  Grammatically we might still label N1 as Mod of N2, not of N3, N2 is grammatical Mod of N3, and semantic Obj of N3.

当然也不能完全排除这个指南是给处于青春期的父母(teenager moms or dads)

白:
青春期有坑,父母有坑,指南有坑。没有“的”的填坑关系和修饰关系互斥。父母填青春期的坑,是一解。青春期自己升格(处于青春期的某某human子类型),再填父母的坑,是另一解。

李:
后一条路不好掌控。凭什么升格,什么时候不升格?

白:
后面没有相谐的被修饰语,又有人需要这个被修饰语。升格也是有准入条件和优先顺序的。“是什么”是语言学,“选什么”是NLP。讨论语言学和讨论NLP,不是一股劲儿。当父母填青春期的坑的时候,父母自身的坑也只能借助升格来处理。不在这里升格,就在那里升格。要么父母处于青春期,谁的父母没交代;要么父母的孩子处于青春期,孩子是谁没交代。那个没交代的对象就是升格升出来的。

李:
没交代是常态。记得当年学句法课,教授花了很长时间讲解 complement 与 adjunct 的区别,说前者是需要交代结构(及其语义)才算完整,后者是可有可无(可多可少)的附加语。但其实在语言中,需要交代的常常不交代 不出现;可有可无的 往往具有语义的重要性。这道线很难划。语言不是按照逻辑设计的。

白:
语言也不是按照树设计的。多爹、非树是常态。

李:
几乎所有的及物动词都能省略宾语。省了以后,那个没出现的宾语,或者需要根据常识去理解,或者是根据上下文去理解,或者你知我知地通过心灵感应去理解,或者误解。也或者根本就不去(充分)理解:糊涂说 糊涂听 也一样交流。人类交流中 unspecified 的信息太普遍了 大家熟视无睹了。貌似交流也没受到阻碍。

白:
允许名词有坑,这都不是事儿。像上面例子,好像就逼着听话人选择哪个是没交代的。而不能是两个都没交代。

李:
青春期的坑是【human】,父母是【human】,这就算对上了。交代了。至于【父母】的坑,标题上下文里没见着,只好拉倒,stay unspecified。没有升格的理由。

白:
你这是从左到右思想作怪。那我问:“多动症父母指南”呢?

李:
从右到左试试?

“父母”的坑是【human】,左边的“青春期”不是【human】,但是青春期的坑里有一个尚未出现的【human】。如果要让正解出现,不仅需要从右向左,而且还要从别人的坑里面挖出一个根本没见影子的萝卜来。就算挖出来了,填进去的是谁还是一笔糊涂账,最多只能以常识默认推测之。

白:
把“青春期”处理成N+,不存在挖别人坑里萝卜的问题,但前面一些不严谨的表述需要修改。

对前面一些表述的修改:1、“青春期”是修饰语,无坑。2、“父母”取N输出,不在乎/N坑是否饱和,不是“升格”,而是正常的萝卜输出。图上两个方案,(2)的饱和度更高

 

【相关】

【语义计算:李白对话录系列】

中文处理

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

【泥沙龙笔记:科学哲学宗教与人生】

尼:
罗素说科学够不着的哲学,哲学够不着的宗教。科学和宗教有交集,如生死,意识,自由意志等。我有篇未完长文 “当我们谈论生死时我们在谈论什么” , 准备作为我《人工智能简史》最后一章。再长的话,得问洪爷了。
@洪 邓总问哲学和宗教啥关系。我昨晚没回复。你接茬说。

邓:
原始问题是宗教、哲学和科学的关系是啥?

白:
科学尽头是哲学,哲学尽头是八卦。

邓:
宗教呢?

白:
宗教尽头是科学啊

洪:
好比有个大鱼缸,
鱼游随便哲学想。
科学只限缸内忙,
缸外有宗教对象。

尼:
@洪 缸能自动变大或变小吗?

邓:
@白 @洪 你俩意见貌似相反

洪:
缸是人所能感知的;如果人类感知进步了人,例如开了天眼啥的,边界就能扩大

邓:
我等白老师和洪爷打起来

邓:
罗素原话怎么说的?
白老师和洪爷彼此惺惺相惜,不打……

白:
缸外有缸

邓:
请白老师明示。解说解说。

马:
探索哲学的小孩

尼:
这小屁孩把想哲学的鱼都给捞走了

邓:
你们都是禅宗的

洪:
身心灵各有其缸,
有界有墙可以撞。
科学自觉不骑墙,
宗教墙外寄希望。

李:
除了红包与痛苦为真 一切皆假。宗教貌似也远不能普度众生,最多是抚慰一小片。至于科学 则是鸦片。让所谓科学家在虚假的高高在上中,求得半片玩积木的兴奋和满足。

邓:
立委登场,讨论正式开始,

彭:
已有的宗教如果不能普渡众生,就只能创新教,当教主了。@wei

李:
当教主肯定更惨。做个追随者至少还可以把负担转嫁 教主的重负转谁呢?

白:
不同领域,宗教尽头到来的时点不同。有的早有的晚。

洪:
假设鱼缸水不浑,
科学/哲学可显灵。
鱼缸壁若玻璃弄,
宗教可以很理性。

李:
面对终极问题 一切那么苍白。高人不高 哲人不哲 敬畏之心难存。

邓:
白老师本质上是认为科学发展会终结宗教?

张:
还有一种可能就是科学发展证实了宗教

邓:
洪爷的说法是两者之间有无法逾越的边界?

李:
都不能解决人类的痛苦问题。

白:
说的就是人类的痛苦问题,科学一定会有办法。

邓:
刺激神经中枢吗?

白:
不是人类整体的痛苦,是一个个具体个体的痛苦。

张:
具体个体的身体痛苦通过科学一个个在解决了。但是精神层面的痛苦无法解决的。就像科学对我们身体的自愈能力一无所知一样。

白:
精神痛苦需要举例,然后看搞定它的途径和时间表

洪:

鱼跃出水有可能,
科学宗教一时懵。
落回或许先知成,
天慧法佛眼开睁。

白:
@张 现阶段科学不等于科学。

邓:
@白 终极的科学是不是等价为绝对真理?

白:
不是。搞科学一般都怕谈“真理”。

张:
精神痛苦的例子太多了,幼年丧母、青年丧父、中年丧妻、老年丧子、失恋,被甩。。。

邓:
如果科学认为不存在真理,宗教又把自己定义为真理,那么是不是可以理解为完全两件事?

张:
估计在地球上找到绝对真理的时候我们要移民外太空了。科学的真理好像是相对的,而宗教的真理是绝对的。

白:
失恋跟记忆关系很密切。如果可以干预和改变记忆,对待失恋的态度也会有所不同。

邓:
原来@尼 大师提到“自我”的容器问题,就是那个换头术悖论,跟白老师这个观点探讨的方向一致吧。

鸣:
精神痛苦,多与欲望相关

白:
执着都是有物质基础的。现在不敢动这个物质基础,主要是怕误伤。如果定点改变,确保不误伤,改了又何妨。离开要解决的问题,真理的绝对还是相对就是个伪问题。有了要解决的问题,大家就直接比疗效好了,不必涉及那些伪问题。

张:
失恋不要被忘记。那也是一种情感需要被回忆的。有了痛苦才有反差,否则人生又有什么意思。

白:
想保留反差就谁也别找,不想反差太大就定点微剂量清除记忆,在这点上宗教活儿太糙,论精准还得寄希望于科学。

邓:
@白 “基督教是现代科学的接生婆”跟您的说法一致吗?

白:
也不尽然吧……

邓:
宗教的尽头是科学

张:
或者科学的尽头是宗教

邓:
现在是张老师力战白老师

张:
来搅浑水。
白老师是严谨的科学家,我们是来搅搅乱。

邓:
必须有人捣乱才能激发白老师讲话的欲望。

白:
娱乐而已。

邓:
@尼 大师该你了。向白老师开炮。咱们那天关于这个主题聚聚。顺便听白老师弹琴。
@白 《those were the days》这歌用曼陀铃应该不错。

尼:
好多讨论的词汇没法定义。例如,”人文”在文艺复兴时的意思是为了和“神”唱反调,科学也算人文,但几经周转,人文语义迁移了。在中国,就成文科生的意思了。

邓:
请大师先定义词汇

尼:
哲学也一样。一种方便的定义是哲学就是哲学系教的那些玩意。那恐怕现在的哲学和100年前哲学不一样了。再过10年,逻辑就不算哲学了。我得开会去,晚上喝两口再聊。

阮:
对于一个生命有穷的个体来说,科学能解决的问题少之又少,当科学不能解决时,就赖宗教获得个体幸福了。因此,科学的终端是宗教。

白:
@阮 宗教就是昨天的科学

李:
一个显而易见的事实是 科学技术的进步带来了物质繁荣 但人类的精神问题没有减少 貌似日趋严重,至少是停滞的感觉 与科技的一日千里 无法比拟。

白:
人类不好伺候啊。
穷也矫情,富也矫情。

阮:
科学伺候的是客观世界,宗教伺候的是人类精神世界,目的不同。

张:
白老师,宗教比科学早啊

白:
对啊,宗教的明天是科学。一个意思。

李:
宗教要真伺候得好 为什么还有那么多迷途羔羊?
譬如 执着心,佛教 甚至fl功 都有很多放下执着心的教义,道理是深刻的,几乎无懈可击,可是怎么那么违背人性 难以接受?

张:
其实科学和宗教确实很难放在一起,科学也许永远找不到最终的答案。

阮:
这世上有这么多无知的人,为什么不说科学伺候得不好?

白:
@阮 不一定啊。你以为高保真音响只是伺候耳朵的?不是的,是伺候听觉欣赏的。

马:
@阮 看你怎么定义终端。科学和技术要分开。享受属于技术。

阮:
@白 科学求真,让人类获得享受是副产品。

李:
好吧 就说死亡不可抗拒 这个我等可以接受 (虽然也是经历很多才接受的)。那么 在我们死亡之前的这些日子里,我们每个人都想减免痛苦,可痛苦依然挥之不去。

马:
所谓死亡只是换了一个躯体而已。

白:
说不定吃一种药就不怕。又减少痛苦又不上瘾。到时候再痛苦就矫情了。

dl:
人之所以痛苦,在于欲望,而其中相当部分欲望,与个体独立性反相关,社会科技越发达,社会越富有,反而大家都落不着好了,这就是现代社会的一个毛病。典型的是权力的欲望。

白:
藏族人对死的态度很豁达

李:
痛苦二字也许太抽象 但我们多少人没感受过一个 “累” 字。人生真累。活着真累。

dl:
越是发达的社会,越是号称平等自由的社会,人们从权力方面获得的满足感越少。越是专制,反而每层都能找到权力的感觉。

阮:
人类麻烦事太多,除了生死,还有一堆其他的。科学帮助解决问题,不能解决的就用宗教。

白:
能量极小化,就是懒;物质极大化,就是贪。

李:
其实这种累的感觉,不是简单的欲望不得满足。

白:
都想出人头地才累。

dl:
归根结底,出人头地也是权力欲望。没有雄心壮志,就比较容易快乐。

熊:
虚其心,实其腹,弱其志,强其骨,美国政府在做的。

白:
权力欲望也一定有物质基础的。吃药可医。早晚的事。

李:
在古代 流行的是简单的快乐。进入文明社会 简单的快乐的能力 现代人丧失了。
我们的远亲 猿猴就没有人类这么多这么深这么厚的痛苦。

dl:
以前皇帝的物质条件未必比现在普通人好啊,但是快乐太多了。现代社会另外一块问题,就是虚伪性,其中最大的就是婚姻制度,也是导致痛苦的主要原因。本来古代制度性解决的问题,现在需要每个聪明人耗尽自己的才智,其实也解决不了问题。@wei 主要是一夫一妻制度的推广造成的。

李:
那就废除这个制度好了。可群婚也不会幸福。

dl:
太多成功学,个人奋斗案例,给大家打鸡血,也是导致大家辛苦的原因。美国鼓吹的那一套人人奋斗也是大家痛苦的原因。以前不成功还可以把原因推到制度上,现在只能埋怨自己了,能不痛苦吗。

邓:
看高手过招真心快乐。@白 将来我们都喝老爸茶、弹琴、斗嘴。

李:
过度紧张 过度竞争 恶性竞争 肯定是罪魁之一。

dl:
人之所以快乐,无外乎比别人优越那么一点点。也许现在宗教存在的意义,就是我失败了,能让我找到一点点借口。否则就是赤裸裸的自我解剖,太痛苦,没几个人受得了。

李:
找不到工作 社会上混得不好 肯定不快乐。
但有一个不错工作的人 也往往快乐不起来

白:
这点借口,科学也会给的。将来都ai了,找不到工作是常态。

dl:
很重要的一点,和身边的人比较起来,是不是优越。最谦虚的人,内心也是希望得到别人的恭维的。最不在乎的人,他在乎的东西,超乎我们的想象,除非这个人完全破罐破摔。

白:
把自己贬低到常人难以接受的水平,来恭维别人的人,一定极其阴暗,破坏力不可低估。自尊心是守恒的。这里按下去,就意味着一定会在其他地方冒出来。

李:
回想起来 人生中比较真切的快乐的确有 就是助人为乐 远比自己得到好处快乐得多。这个助人为乐 首先是亲友。看到自己爱的人 亲友 由于自己而改善了境遇 那种满足幸福感相当不赖。如果觉得自己可以帮助全世界,快乐就源源而来。

dl:
@wei 这也算一个麻醉剂。中国人这个达则兼济天下的思想,本来就是高高在上的。
明白嘛,高高在上。帮助他人,也是体现自己优越感的好地方,当然客观上是有好处的。仔细分析,从内心来讲,并不比我要出人头地高尚多少。

白:
精英已经失去了代表人类平均感受的资格。

dl:
@白 现代社会的痛苦,就是精英阶层弱化的痛苦。

白:
去精英化

李:
暴发户回家乡办学 就是这种幸福的展示。

dl:
@wei 你说的这种东西,并非精英阶层追求的东西,而是把精英阶层平民化的过程。这里有一种不可调和的东西存在。

李:
雷锋的快乐就是,做好事不留名,记在日记里自我欣赏的感觉超级棒。

高:
雷锋不识字,何故多记事

dl:
@wei 雷锋这种现象,主要还是愚民策略的一个证明

李:
一介武夫 无权无钱 不富不贵 但没人否认,雷锋是幸福的 每一天活得那么充实满足。

dl:
反正在这个群里,我得到的快乐,肯定没有我的粉丝群里得到的多,这是肯定的。原因是肯定的,这个群里大家都很自我。没有人太屌谁,这就是以后社会发展的一个趋势。

白:
反过来说,精英扎堆儿的地方,不适合精英自我表扬。

桂:
李白是计算语言学界活雷锋。

dl:
所以每个人权力获得感会大幅下降,这是必然的。预测以后的世界,每个人的存在感问题会更大。这会是一个主要问题。

白:
唱戏当皇上也是爽的。
以后ai发达了,nlp发达了,慕容复何至于那么孤单,可以乱真的奴才臣子还不是要多少有多少?权力欲真那么难满足吗

dl:
@白 反正我不会和机器人谈恋爱。
这个每个人细细体察内心就可以知道

李:
不要说那么绝对。没人会拒绝快乐,快乐来自人和机器 不重要。

白:
来自药物和现实乃至虚拟现实,也不重要,关键是不要有副作用

dl:
精英阶层的人士,快乐在于控制和影响力,不在于太物质的东西。而世界趋势在背道而驰。庸俗化正在席卷全球。
@白 吃药也许可以。五石散。魏晋南北朝,其实也是一个世家没落的时代,和现在有点像。

白:
所以科学宗教哲学,说到底都是solution,是骡子是马,最后都要在problem面前遛遛。

dl:
@白 同意。问题是这里面存在不可调和性。发展趋势和人快乐的基础之间有不可调和性。这是现代社会的一个重要问题。庸俗化引起的权力满足感丢失,可能是问题的核心。

顾:
科学和宗教类同,只不过科学适用面宽些,预测能力强些。

白:
人太多了,逆选择一下也是必要的。这么辛辛苦苦伺候都快乐不起来的人,还是哪儿凉快哪儿呆着去吧。

dl:
目前解决方案,就是创立一个公司,然后去当土皇帝。这才是正道。所以要创业。这才是创业的终极目标啊。公司目前是满足权力感最好的形式。

白:
权力自由但财务不自由的创业,好不到哪儿去。

dl:
@白 这就是你说的唱戏当皇帝也快乐啊。

白:
自己印钱啊,虚拟货币。
想象力太受现实束缚了。

dl:
@白 我只是调侃一下而已

白:
我调侃两下行不

dl:
当然可以。以后都去参加拜公司教就好了。这就是宗教。solution直面最核心的问题,精英阶层存活的意义就在于此。

白:
ai让你唱戏当皇上比真皇上还爽,连个不爽的理由都找不出来。

dl:
@白 现在很多人沉醉于虚拟网络,可能也和这个有关系。找到了存在感。但是如果获得太容易,就没优越感了。存在感其实在某些意义上等同于优越感。

白:
那容易啊,工作量证明,挖矿挖到了的当皇上。

dl:
@白 其实发明一种机制,让人去做梦,这样人的一生其实不需要活动。可以拍一个电影,以后少数精英人士操纵社会,大多数人生下来,就被装在器皿里培养做梦,大家觉得如何?都很快乐。其实社会本质未尝不是这样?

邓:
今天讨论这么热闹@尼 得发个红包。

 

【李白之44:“明确”是老子还是儿子,需要明确】

白:
“北京大学率先明确入雄安路线图”。“明确”是副词还是动词,需要先明确。

李:
我觉得这不是问题的关键。pos 在抽象层面只规定非常宽泛的框框。关键还是语义之间的相谐。正确的问题不是pos,而是 “明确” 在这里是状语(儿子),还是谓词(老子)。在【明确+ vp】里面,pos 上它没有什么限制,所谓跨类词。跨类就是两可。 对于跨类 这是一个没啥意义的问题 或者说 这是一个 circular 的问题。

“明确”的最常见的 subcat 带的是 np,但是 在 “明确v” 出现时,这个 v 可能是 vp,因此 “明确”可能做其状语。也可能这个 v 是np内部的修饰语。

(1) 明确规定游泳违规
(2) 明确规定 严明纪律 这两条是整顿金融秩序的关键所在
(3) 我们要明确规定条例

大数据可以帮到 (1);(2) 是排比的力量 压住了大数据,把潜在的动宾可能激发出来。 但排比因素的形式化实现还需要探究(虽然形式特征在人看来是蛮明显的)。(3) 仍然歧义:“明确(地)规定(相关)条例”,还是 “明确(一下)(有关)规定(方面的)条例”?大数据似乎选择前一个解读,为什么我们仍然感觉得到后一种解读的可能性。设想 如果大数据中 “规定条例” 比例开始高到可以与 “明确规定” 相匹敌的时候,这就是大数据也搞不定的边界之争。

白:
“北京大学率先明确入雄安路线图”这个例句中,“明确”只有一种走得通的词性,就是动词S/N,X。如果是副词,“入”就会过饱和。虽然过饱和也是能处理的,但放着正常饱和的不用却去用过饱和的,终究不妥。因为“入雄安”只剩下一个坑了。副词“S+”不会改变萝卜数和坑数。

李:
“率先明确入雄安的是习大大吗?当年是毛主席率先明确入京的。虽然有人提议仍在南京建都。” 什么叫 入 的过饱和?

看样子 “明确” 的subcat 也有带 vp 的,虽然不如带np的 subcat 用得广。

另。假设 native speaker 的语感与大数据(作为语言共同体语言表现的代表)吻合(实际上当然不会总是吻合), 那么大数据搞不定的歧义就应该视为真歧义。大数据能搞定的就是伪歧义 尽管这个伪歧义以前被认为是句法真歧义。“以前”指的是没有大数据语义中间间或没有语义或常识消歧手段的句法分析器的那个时期。

 

【相关】

《李白之零:NLP 骨灰级砖家一席谈,关于伪歧义》

【语义计算:李白对话录系列】

《朝华午拾》总目录

 

【立委兄:温哥华都市游 2/3】

没办法只好返回Max家去吃晚饭,刚到Max家没多久,雨停了,太阳出来了,时不我待,我们饭也不吃,立马背起相机,直接再去与SFU大学连成一片的本那比山公园照落日晚霞去了。功夫不负有心人,到本那比山公园时机正好,登顶之后,眼前豁然开朗。葱郁的绿里点缀着红黄,极目远望,水光山色尽收眼底。BurnabyMountain Park 奇特的雕塑群(日本雕塑Kamui Mintara sculptures,由日本雕塑家 Nuburi Toko 和他的儿子所创作,象征着人类、动物、自然和上帝的和谐统一),秋日叶色微红的树林,和湖、云、山、峡、光融为一体,远处一两汪不知是湖还是海的碧水,让人仿佛置身在童话世界中,这才是美丽的秋景呀,色彩艳丽,如诗如梦。

与SFU大学连成一片的本那比山公园BurnabyMountainPark

10月20、21、22日天气不好,雨季的温哥华,对它没脾气,我们只得去商场和奥特莱斯转了转,顺道去参观一家湖北籍华人正在建造的木结构三层大别墅。

温哥华小区中林荫道

10月23日星期六,早晨天气转好,我们准备去惠斯勒(Whistler),没出城前有两处路边景点让我们兴奋不已,都在河边。

一处在弗雷泽河旁,是朦胧美,轻雾如纱,烟锁秋波,一层轻纱般薄雾在河面上飘来荡去,河对岸的树林和农舍时隐时现,一切都掩映在宁静、纯净、虚幻的晨雾怀里。那淡淡的地雾,那隐隐的树林,那静静的水面,活灵灵显出一个世外桃源。

另一处在皮特河旁,当车路过皮特河大桥时,我们眼前又一亮,被眼前美丽深深地震撼,赶紧下了主干道,停在河傍,端起相机不停地拍摄,机会难得。远处山峦披着多层薄雾,河两侧处处可见红的黄的树,湛蓝的河水倒映流光山色,黄草依依在河边随风而笑,宛如仙境。这情让人心灵涤净,这景让人飘飘欲仙。

弗雷泽河旁朦胧美,轻雾如纱

皮特河流光山色

走出温哥华前

走出温哥华,进入海天高速公路 Sea to Sky Highway (99号高速公路)。海天公路蜿蜒於太平洋和群山之间,由海洋、河谷、冰川及高山峻岭所铺陈。道路内侧是山峰林立、悬崖陡峭,另一面则是碧波万顷、海水湛蓝,沿途经过多处海湾、瀑布等风景区,景色如诗如画,还有绿色的小岛、高高的雪山,蜿蜒而上的公路,海天相连的景色尽收眼底,一如其名,完全展现由海至天的丰富样貌。

海天公路路过一个非常有特色的高尔夫球场。整个球场顺着山势而建,背山面海,高低错落,真不知老外的灵感怎么得来,设计出如此之美的球场。只是我们找不到俯视这球场的高台,照不出它的神采来。

到香农瀑布(Shannon Falls),虽落差335米,但我们看瀑布太多,就没停下来了。后来路过一个不知名的观景台,居高临下拍了一下海湾的大景观。在斯阔米什小镇(Squanmish),我们休整了一下,发现加拿大民众都在为万圣节筹备各色南瓜及物品,看来西方对这个节日挺重视的。村对面的史坦沃斯峰 Stawanus Chief 很雄奇,很高耸,坐缆车可以登顶,可一览海天高速公路四周美景,可惜我们没上去。

海天高速公路 Sea to Sky Highway

路边山坡高尔夫

斯阔米什小镇

 

【相关】

【立委兄:温哥华都市游 1/3】

【立委兄:温哥华都市游 2/3】

【立委兄:温哥华都市游 3/3】

《朝华午拾:温哥华,我的梦之乡》 

温哥华的美是骨子里的

【立委兄:二十八天加拿大自驾游 前言】

【立委兄:二十八天加拿大自驾游 结语】

【立委兄:加拿大西部山水游 (1/16)】

【立委兄:加拿大东部枫叶游(1/11)】

【立委兄:多伦多都市游1/2】

【立委兄:多伦多都市游2/2】

【立委兄:尼亚加拉大瀑布】

【立委兄:加拿大自驾游东部风光摄影选粹 1/2】

【立委兄:加拿大自驾游西部风光摄影选粹 2/2】 

【立委兄:加拿大东部枫叶游(8/11)】

二十八天加拿大自驾游东部篇

在小城停留几个小时后,我们驱车赶往距圣安妮大峡谷大约9公里处小村庄 Sainte-Anne-de-Beaupré,我们在小村庄著名的圣安妮大教堂旁预订了一个汽车旅馆–海岸公寓汽车旅馆 Condo & Motel des Berges.  多伦多到魁北克的40号高速路,两侧都是茂密的枫树,如果时间合适,枫景会很壮观,可惜我们今年来的不是时候,枫叶推迟盛红期了。到旅馆服务处,其大门紧闭,留下一个条子,让我们自已在门口小盒子里拿钥匙进房。国外旅馆手续简便,走时也不查房,交钥匙就可走人。安顿好住处,我们立即去圣安妮大峡谷,这是一处赏枫名地。圣安妮大峡谷的门票CAD13.50一人。刚进圣安妮大峡谷大门,还有些金色的枫叶,但到峡谷后,只能偶尔见到变黄变红的叶子,看不出是魁北克的“枫”景之最。说什么:山谷红黄的枫叶漫山遍野,峡谷由于在谷底、山腰和山顶枫叶变红时光不一,有丰盛层次感,我完全没感觉到。瀑布很一般,枫景很一般,没有了秋色,峡谷完全无法吸引到我们。该死的气候,今年天气热的太长,延缓了枫叶变色时间,今年我们武汉桂花也延缓15至20天才盛开哟,理解理解。

圣安妮大峡谷

从峡谷返回我们来到住处,欣赏住处的圣安妮大教堂及四周美景。圣安妮大教堂是一座宏伟的哥特式建筑,矗立在圣劳伦斯河边的这座纯白色教堂,17世纪1658年建立,历史悠久。350多年里,五次扩重建,可惜在20世纪初毁于一场大火。1926年重建。教堂前面的广场上有一个喷水池,青铜铸造的圣安妮怀抱着幼小的圣母玛丽亚安详地站在那里,雕像和喷泉融为一体。大教堂的内部金碧辉煌,气势震撼,我们进去时,主教正在宣教。

第二天一早。我们起床想照圣劳伦斯河日出和朝霞。这一天早晨天气十分寒冷,在河边我们穿了冬季衣物,仍感寒气逼人。可我们旁边一对老外老夫妇,穿着短裤也在河边拍照,本认为他们从车里出来不会很长时间,没想到他们比我们还久,真佩服他们,老外就是不怕冷,我们惭愧。天气虽冷,也时阴时情,但风景确实不错,我们照得尽兴。

圣安妮大教堂

小村庄Sainte-Anne-de-Beaupré

 

 

【相关】

【立委兄:二十八天加拿大自驾游 前言】

【立委兄:二十八天加拿大自驾游 结语】

【立委兄:加拿大西部山水游 (1/16)】

【立委兄:加拿大东部枫叶游(1/11)】

【立委兄:加拿大东部枫叶游(2/11)】

【立委兄:加拿大东部枫叶游(3/11)】

【立委兄:加拿大东部枫叶游(4/11)】

【立委兄:加拿大东部枫叶游(5/11)】

【立委兄:加拿大东部枫叶游(6/11)】

【立委兄:加拿大东部枫叶游(7/11)】

【立委兄:加拿大东部枫叶游(8/11)】

【立委兄:加拿大东部枫叶游(9/11)】

【立委兄:加拿大东部枫叶游(10/11)】

【立委兄:加拿大东部枫叶游(11/11)】

【立委兄:加拿大西部自驾山水游(13/16)】

二十八天加拿大自驾游西部篇

2日一早,当我起床打开窗户,不由得惊叫:太美呢,太美呢!蓝蓝天空下,白白的雪山,一条云雾缠绕在山间,朝霞印照在山顶,多彩多姿,金光闪烁,不是仙境胜似仙境。原来昨晚是山下下雨,山上下雪,清晨突然转晴,就展现出这神奇的景色来。我们呼着极清新的空气,冒着寒凤,拿着相机,在住宿门外不停地拍摄,太让人心动了。

落基山旅馆四周的好照片

随着我们赶紧打包上路,去朱砂湖拍日出下梦幻般的湖景,我们两天前就对那个地方充满期待。果然上天眷顾我们,给我们很多惊喜,枯黄的草,绚丽的霞,碧蓝的水,山顶的雪,多层的云,洁净的天,这些要素全都具备,实在是可遇不可求。诸要素构成了一幅难得的美丽画卷,让人爱不释手。不停的拍,不停的拍,说来你可能不信,当我们拍完照后,天气又大变,一股厚云从西向东飘来,不一会儿大雨倾盆。虽然我们无法再去我们发现的另一摄像点即1号公路与1A号公路交汇处去取景拍照,但我们心愿已足,便开车直奔优鹤 Yoho National Park 国家公园。

阴雨下的朱砂湖

日出下的朱砂湖

 

 

【相关】

【立委兄:二十八天加拿大自驾游 前言】

【立委兄:二十八天加拿大自驾游 结语】

【立委兄:加拿大西部山水游 (1/16)】

【立委兄:加拿大西部山水游 (2/16)】

【立委兄:加拿大西部山水游 (3/16)】

【立委兄:加拿大西部山水游 (4/16)】

【立委兄:加拿大西部山水游 (5/16)】

【立委兄:加拿大西部山水游 (6/16)】

【立委兄:加拿大西部山水游 (7/16)】

【立委兄:加拿大西部山水游 (8/16)】

【立委兄:加拿大西部自驾山水游 (9/16)】

【立委兄:加拿大西部自驾山水游 (10/16)】

【立委兄:加拿大西部自驾山水游 (11/16)】

【立委兄:加拿大西部自驾山水游 (12/16)】

【立委兄:加拿大西部自驾山水游(13/16)】

【立委兄:加拿大西部自驾山水游(14/16)】

【立委兄:加拿大西部自驾山水游(15/16)】

【立委兄:加拿大西部自驾山水游(16/16)】

【立委兄:加拿大西部自驾山水游 (12/16)】

二十八天加拿大自驾游西部篇

 

我们今晚住处不在班芙镇,而在离班芙20公里外的坎莫尔Canmore小镇的落基山旅馆,我们在这住两晚。这是我们这几天住得最好的旅馆,楼上楼下,日式联排别墅式的,二室二厅二厕一厨,十分干净和方便。

坎莫尔Canmore小镇的落基山旅馆

10月1日国庆节,天仍下着雨,我们在班芙镇转了一转,天空雾蒙蒙的,自然去硫磺山顶也无意义了,就再去路易斯湖了。环绕湖畔有许多条健行路径,加拿大国家公园大多建有许多许多步行道,人家对体育与锻练都很执着。另外湖边还有一条登山路径,可一直到达山顶。在山顶可俯览翡翠般的露易斯湖,由于天气不佳,我们只在湖边走了一下,没有上山去拍路易斯湖全景了。露易斯湖三面环山,层峦叠嶂的露易斯湖,仍然翠绿静谧,在宏伟山峰及壮观的冰川的衬映下秀丽迷人。我相信如果不是天气太差,这里一定是现实中的世外桃源,毕竟它久富盛名。下午就回到住处,自己做晚歺去了。傍晚雨仍在下,我们心情自然糟透了,明天就要离开班芙了,没拍几张班芙四周的好照片,该死的天气不给力呀。

班芙镇Banff


班芙镇Banff四周自然风光

 

 

 

【相关】

【立委兄:二十八天加拿大自驾游 前言】

【立委兄:二十八天加拿大自驾游 结语】

【立委兄:加拿大西部山水游 (1/16)】

【立委兄:加拿大西部山水游 (2/16)】

【立委兄:加拿大西部山水游 (3/16)】

【立委兄:加拿大西部山水游 (4/16)】

【立委兄:加拿大西部山水游 (5/16)】

【立委兄:加拿大西部山水游 (6/16)】

【立委兄:加拿大西部山水游 (7/16)】

【立委兄:加拿大西部山水游 (8/16)】

【立委兄:加拿大西部自驾山水游 (9/16)】

【立委兄:加拿大西部自驾山水游 (10/16)】

【立委兄:加拿大西部自驾山水游 (11/16)】

【立委兄:加拿大西部自驾山水游 (12/16)】

【立委兄:加拿大西部自驾山水游(13/16)】

【立委兄:加拿大西部自驾山水游(14/16)】

【立委兄:加拿大西部自驾山水游(15/16)】

【立委兄:加拿大西部自驾山水游(16/16)】

【立委兄:加拿大西部自驾山水游 (11/16)】

二十八天加拿大自驾游西部篇

沿1号公路继续往班芙赶,突然发现在1号公路与1A号公路交汇处,风景很是独特,山、水、林、铁路浑然一体,特适合摄影。天气已开始下雨,我们下车观察一下,决定明早一定要来这里,这里太有特色了。

1号公路与1A号公路交汇处

快到班芙,雨忽停忽下,没完没了。突然一处景观让我们眼前一亮,惊呼起来。美,实在是美。这就是朱砂湖Vermilion Lakes。朱砂湖位于班夫镇入口的高速公路旁,这里十月初湖畔风景线真是美的无语。虽然天气不好,仍让我们心旷神怡,十分振憾。我们赶紧下车猛拍了一番,不能辜负此处如朱砂一般色彩斑斓的动人景色。我们不用说,自然明早还会来,祈祷明天天气会好起来。

到班芙镇 Banff 了,天又下雨了。班芙镇比贾斯帕镇大很多,没有贾斯帕镇淳朴,是加拿大著名旅游城市,被誉为落基山脉的灵魂,加拿大国皇冠上的明珠。班芙镇群山环绕,冬天可以滑雪,夏季可远足。如果登小镇旁硫磺山,可以居高翘望落基山脉磅礴的气势,俯瞰班芙全镇景貌和弓河蜿蜒曲折的美景。硫磺山海拔2285米,有双向缆车到山顶,但因天气与时间原因,我们没有去硫磺山顶,错失俯瞰班芙全景的机会。到班芙镇我们首先到火车站,打听火车时刻表,目的就是希望明天去我们探寻的那个景点时,正好有火车经过,照出一批有特色的照片。天气说好就好,在火车站时天气转晴,太阳出来了,紧赶慢赶照了几张百年老火车站照片,这鬼天气说变就变,不一会又阴转雨了。

班芙镇百年老火车站

 

 

【相关】

【立委兄:二十八天加拿大自驾游 前言】

【立委兄:二十八天加拿大自驾游 结语】

【立委兄:加拿大西部山水游 (1/16)】

【立委兄:加拿大西部山水游 (2/16)】

【立委兄:加拿大西部山水游 (3/16)】

【立委兄:加拿大西部山水游 (4/16)】

【立委兄:加拿大西部山水游 (5/16)】

【立委兄:加拿大西部山水游 (6/16)】

【立委兄:加拿大西部山水游 (7/16)】

【立委兄:加拿大西部山水游 (8/16)】

【立委兄:加拿大西部自驾山水游 (9/16)】

【立委兄:加拿大西部自驾山水游 (10/16)】

【立委兄:加拿大西部自驾山水游 (11/16)】

【立委兄:加拿大西部自驾山水游 (12/16)】

【立委兄:加拿大西部自驾山水游(13/16)】

【立委兄:加拿大西部自驾山水游(14/16)】

【立委兄:加拿大西部自驾山水游(15/16)】

【立委兄:加拿大西部自驾山水游(16/16)】

【立委兄:加拿大西部自驾山水游 (10/16)】

二十八天加拿大自驾游西部篇

路易丝湖luise lake被誉为被誉为落基山脉最美丽得湖,以维多利亚女王的女儿路易丝公主的名字为其命名。93号公路到路易斯湖附近就转入最著名横贯加拿大东西的1号公路,路易丝湖就在1号公路附近,也是去班芙镇Banff主干道。到路易丝湖时天气不好,又阴沉沉的了,没有出太阳,拍不出她的风姿,更拍不出落日下的路易丝湖景观来,我们决定明早再过来碰碰运气。于是我们去童话城堡一般的露易斯湖费尔蒙城堡酒店Fairmont转了转,露易斯湖城堡酒店历史十分悠久,最早建于公元1886年,它紧临湖畔,气派十分宏伟,从宾馆窗户眺望窗外迷人的露易斯湖,那真是享受。

露易斯湖费尔蒙城堡酒店 Fairmont

路易丝湖 luise lake

中国大妈打遍天下

 

 

【相关】

【立委兄:二十八天加拿大自驾游 前言】

【立委兄:二十八天加拿大自驾游 结语】

【立委兄:加拿大西部山水游 (1/16)】

【立委兄:加拿大西部山水游 (2/16)】

【立委兄:加拿大西部山水游 (3/16)】

【立委兄:加拿大西部山水游 (4/16)】

【立委兄:加拿大西部山水游 (5/16)】

【立委兄:加拿大西部山水游 (6/16)】

【立委兄:加拿大西部山水游 (7/16)】

【立委兄:加拿大西部山水游 (8/16)】

【立委兄:加拿大西部自驾山水游 (9/16)】

【立委兄:加拿大西部自驾山水游 (10/16)】

【立委兄:加拿大西部自驾山水游 (11/16)】

【立委兄:加拿大西部自驾山水游 (12/16)】

【立委兄:加拿大西部自驾山水游(13/16)】

【立委兄:加拿大西部自驾山水游(14/16)】

【立委兄:加拿大西部自驾山水游(15/16)】

【立委兄:加拿大西部自驾山水游(16/16)】

 

【李白之28:“天就是这样被聊死的”】

白:
“天就是这样被聊死的。”

李:
说谁呢 ?

梁:
@wei ,I also admired your “层次纠缠“ comment.

李:
哦 那是刘少奇主义, 吃小亏占大便宜。真的,反单层parsing的传统潮流,悄悄地不知道占了多少便宜了,不吃点亏都觉得不好意思了

白:
“的”可以“买一送一”或者“卖一送一”。比如“卖火柴的小女孩”中,“小女孩”既可以作为整个定中结构的代表正常对外填坑,也可以“无偿”填定语从句内部“卖”的坑;“这本书的出版”既可以作为整个定中结构的代表正常对外填坑,也“无偿”对内接受定语部分“这本书”的填坑。“无偿”的意思是,一个括号配一个反方向括号后,对方消失了自己还在。多“饶”了一次匹配机会。现在从坑的角度看,最不情愿的一个处理就是把形容词的使动用法处理成一可选的坑。如果坑可以强制出来就好了。

李:
多一次匹配机会,就是一个儿子可以有两个老子,类似一个是生父,一个养父。对外填坑是句法的显性 dependency,对内无偿就是逻辑语义的隐性 dependency,中外皆然。“小女孩”就是如此,“小女孩”做“卖火柴”的【逻辑主语】。而“卖火柴”其实是“小女孩”的【定语】:让自己的显性的儿子去做自己的隐性的逻辑老子。到了谓词指称化就不同了,对外填坑不变,爱啥啥。对内的逻辑关系则反过来,自己的显性的句法定语儿子(adjunct),被用来填充为自己的隐性的逻辑儿子(argument)。“这本书” 做 “出版” 的逻辑宾语。与英语平行。

this book‘s publication;girl selling matches
? the sell-matches girl

白:
这两件事,被我统一成一件事了。

李:
统一的好处是?

白:
词典化,没有规则,只有词典和原则,一条原则管两头。到原则层面,只需要解决什么条件下谁提供bonus。

李:
一边是 NP , 一边是 V 或 VP。如果是 NP 的 V,V有坑,尽管 V 指称化了。那么 NP 就去填坑(宾语,或主语),如果是 VP 的 N,那么 VP 有主语的坑,N正好填。

白:
根本不看POS,只有买单和卖单。一对多的订单、一对一的订单、可以提供bonus的一对一订单。

李:
POS 也没啥,不过就是填坑中几百上千个可能的约束条件的一个而已。我要 Human 来填坑,与我要名词来填坑,对系统不过就是条件的宽窄不同而已。这是从填坑的角度看 POS。对于设坑的一方,当然无所谓 POS,V 也好 N 也好 A 也好,他们有没有坑,有几个坑,都可以认为是词典化的 subcat 规定的。都直接量(词)驱动了,自然就不谈 POS 了,因为 literal 永远比任何抽象信息量更足。

据说当年一个叫 Small 的人首创专家词典(Expert Lexicon),里面全部是词例化的规则,没有任何抽象规则,可以想象这样的系统在一个狭窄的 domain 里面的可行性。譬如 在天气预报的 domain 里。词条 “下雨”里面规定 要到左边去找 “老天”,到右边去找 “很猛、很急”,等等。

白:
肯定不是这样的。

李:
在一个小词汇表中 是可行的的 而且准确 有点组合爆炸而已。这是没有任何抽象的本义。一旦有抽象,哪怕是词例化,也必须引入 features,而不是直接量之间的交易了。lexical-POS 就是最简单的一个 feature。

白:
原则不看POS,matcher要看。但原则不放水,matcher就没有bonus给。

“张三这两条建议是关于修宿舍的。”

这里面,“建议”有两个坑,“张三”填了一个,然后这个词的“母体”填给“是”了,剩下一个坑成了没娘的孩子。后面当“关于修宿舍的”作为一个整体与“修宿舍”剩下的没娘孩子(human)不匹配的时候,匹配点会迁移到前一个没娘孩子(info,建议的内容)进行匹配。

李:

白:
这不是说,建议的内容就一定是“关于修宿舍”,万一“是”换成了“不是”呢?只是说,这一萝卜一坑,存在着语义类型上的关联。至于肯定否定,那是由围绕着谓词的修饰语决定的。

李:
肯定否定是另一层次的东西,无需牵扯进来。说到底就是 “建议” 有 Subcat,里面有 human agent,和 “关于 content” 的 PP 的规定。human S 连上了,【关于】的坑暂时没连上,但也不难。

白:
建议谁修宿舍,文本中找不到,作罢

“他死去多年的战友就埋葬在这里。”

“他”要憋着不参加后面的VP,直到被定语从句修饰的中心语露面,填入中心语所带的坑,才算了结。什么样的“过程性”控制策略能给出这个选择?

宋:
他死去多年,儿女都已经长大的战友埋葬在这里。

白:
嗯,其实谓词部分是收束的,只有谓词部分对外需要消解。所以,需要栈,但栈不必很深。栈和RNN是不矛盾的。栈顶元素可以作为输入的一部分,对栈的操作可以作为本轮输出的一部分。

李:
查一下后条件不就解决了?在主谓规则中 ,一个 np 与 一个 vp 不着急结为 s,往后查一下条件再决定。

没问后条件,错了。可以加上:

白:
往前看一个,只能做等待与否的决策,不能做结合与否的决策。等待就意味着要记忆某种东西。

李:
等待与否与决策与否,这里不是一个简单的答案。因为涉及两个问题:一个是“他死” 的主谓问题,一个是“死”做定语(兼“战友”的逻辑谓语)的问题。如果不考虑二者相交,第一个问题当成一个独立的问题,当然可以决策,不过是问合适的条件包括后条件而已。这样“他死”本来的主谓错误可以避免,但还是需要有人(“埋葬”)接盘。从相交的角度看,关键是定从句型的处置安放在何处合适的问题,定从解决好了,顺带也就解决了“他死”要不要就近连主谓的问题。涉及的句型也不那么复杂:

NP+VP+de+N

就是一个四元组。把上述句型在做主谓之前 fine-tune 到正好涵盖【定从】,问题就解决了。宋老师的句子是难一些,难在那个 VP 复杂化了,VP 实际是两个 VP 用逗号并列了(其实应该用顿号的,可国人把逗号当万金油,没办法)。这倒也罢,第二个谓语本身也是一个主谓结构:“儿女都已经长大”。“儿女长大” 与 “身体健康” 类似,都是那种句型紧凑的典型的【主谓谓语】。这类主谓只能有限扩展,跟通常主谓的自由度无法比,也因此可以考虑先行解决,给个标签,作为整体,它有一个逻辑主语的坑(通常是其前的 Topic 去填):实质上是对付一层的中心递归(center recursion)。总之是有些难缠,但并非无迹可寻,要做也可以做,考验的是细活。等低枝果实都摘差不多了,再去磨这个细活好了,现在不必。

白:

他那些杀红了眼,刺刀上沾满血的战友们可管不了那些了。

“儿女”有坑,把“战友”捎带上还可以解释。“刺刀”的坑是“枪”,“枪”的主人是human,这弯儿拐的。句法非标配的坑,靠语义中间件凌空凿开一个坑,才能把定语从句的钩子钩上。第一个“那些”如果去掉:

?他杀红了眼,刺刀上沾满血的战友们可管不了那些了。

好像不通了。或者说链条断开了。所以凌空开凿的坑无法填装远距离的“友元”。

李:
看样子这个 “那些” 是个关键的小词,应该善加利用:

human+那些+[human action] + 的+human

麻烦的是 human action 的谓语的扩充性。如果这个句型足够的频繁(感觉上是的),那么一个策略是,对于那个【定从谓语】的界定可以放的很宽,一路扫描下去,直到发现 【的+human】,就把这个 【定从】 的性质卡住了。定语定性以后,再慢慢对付里面的monsters,这个策略可能管用。

他的那些blahblah的朋友们

管它 blah 有多长、多复杂。一个 token* 就卡住了。还有一个策略就是 patching,对上面的那颗“循规蹈矩”而出错了的树做修补:

S1[X那些] + Pred1 + Conj + Mod(Pred2)+S2+Pred3

要问五个链条才能修补全,也不知道能概括多少现象,值得费这么大力气,会不会弄巧成拙。道理上是可行,问了这五个链条了,然后

(1)离婚 S1 和 Pred1
(2)结合 S1 与 S2,让 S1 zuo S2 的定语
(3) 切断 Conj
(4) 用新的 Conj 链接 Pred1 和 Pred2

可以做个实验玩玩,看这条路可行不。
MY GOD 值不值得做先放在一边,可的确做成了!

这个太tm牛了。我都不得不崇拜自己了。

还是那句话,没有翻不了的案子,毛太祖钦定的文化大革命都彻底否定了。这样的翻案 patching 应该没有副作用,因为都是 word driven 和非常 restricted 的现象。

同一条规则略加微调(没有“那些”但原主语是 human),就把宋老师的难题一并解决了。休眠唤醒术好使,以后要多使,这比条件不成熟的时候霸王硬上弓轻松多了。

白:
不对呀……
怎么是“他”死去?应该是“战友”死去才对。另外,“战友”并没有“长大”,“长大”的是战友的“儿女”。

李:
鸡蛋里挑骨头啊。明明“儿女”是“长大” 的 S。长大的战友,不过是一个边界不合适的 XP 懒得在 patching 的时候再动手术去重新修理边界而已。

白:
就是说,定语从句的两个分句,第一个“死去”的坑被提取出来,由中心语“战友”反填;第二个“长大”的坑由“儿女”填充,同时“儿女”挖了一个新的human类的坑,由中心语“战友”反填。

李:
真要修理也不是不可以,但已经没有啥意义,因为逻辑语义上已经阻止了 “战友” 做 “长大” 的主语。对,“他” 不该是 “死去” 的 S,这个我去查查 code

白:
要简化也是“儿女长大的 战友”,而不是“长大的 战友”

李:
那是因为偷懒 共享了 “那些”的规则。得,我分开来一步步来。
目前的机制可以改关系,暂时不可改边界。有空了写个 specs 让工程师增加边界调整的功能。不该有的 S 没有删去,是个 bug,规则是对的。对数据结构做手术,要做干净、不拖泥带水、不留后遗症,还需要磨一阵子。不过苗头是好的。

白:

“目前尚未毕业、导师已经超过六十三岁且不是院士的博士研究生要来教务处登记。”

谁不是院士?导师,还是博士研究生?如何确定?两个conj并列,是一股势力;“导师”辖域延展,是另一股势力。

李:
不用 common sense 出场?

白:
后一股势力因为“导师”自带的坑得到“博士研究生”在右侧遥相呼应而得到加强。当然“博士研究生”自身也因距离更近参与“不是院士”的逻辑主语坑的争夺。定性分析这是打个平手。common sense之外似乎看不到一锤定音的结构性要素。或者换个说法,大数据里,“导师”和“院士”共现的频度,与“博士研究生”和“院士”共现的频度比起来,谁高?

一提common sense就有把问题搞复杂的嫌疑,提大数据则明显把问题简化了。

李:
不错。现在的问题是,应该怎么挖掘和表达大数据的这些隐含了常识的知识,使得需要用的时候,够得着。人手工费那么大劲精心构建的 ontology 和常识,目前用起来还是不能得心应手,挖掘的东西应该呈现怎样的形态才好用呢。

白:
词向量可直接反映共现。

李:
在两个词抢夺同一个词的时候,最简单的办法就是看他们的两两共现来决定力量对比。这个听起来简单,但这种三角争夺是 on-the-fly 的,共现数据可以预先计算并 index,三角计算必须是 at run time,感觉上有一个不小的 overhead

白:
现场直接变成算距离了,index出来是向量,向量的距离直接反映共现。而且是“应该的”共现而不是“现实的”共现,中间加上了互通有无。互通有无是数学模型帮我们做的。

李:
大数据出来的统计不都是“应该的”么?都只是一种趋向。增加一个砝码,不是铁定。(一定有违反大数据统计的反例在。)

白:
不是这个意思,是说很多数据是稀疏的

宋:

(1)应该做大数据挖掘,与专家的规则结合起来。白硕建议比较两对共现频次,我觉得比常识知识库靠谱。

(2)这种大数据中的知识挖掘应当是实时的。应该有某种大数据存放的中间形式,支持快速的实时统计。这种中间形式会比原始的线性字符串对于统计更高效,同时应当有一定的通用性。

白:
在降维中稠密化了,原来语料中直接没搭上钩的,经过降维处理也可以搭上钩了。

宋:
(3)恐怕会有一些问题不是单纯的词语共现所能解决的。

白:
算距离,复杂性主要跟维度有关。维度降下来了,不仅数据稠密了,而且计算开销也下来了。@宋 老师的(3)完全赞同。共现的数学模型,build和run的确是分离的。李:

李:
synonym 或 clustering 就是 降维 和 数据稠密化吧,但同时也抹平了。不知道目前有没有哪个系统真地在歧义判别时候用到大数据统计的。

白:
word embedding并不严格抹平,但可以拉近,而且如果只为了比较大小,距离算到平方和足矣,没必要再开方。

李:
对,根本不需要精确计算,只需要相对的结论,谁强谁弱,或打平。

康:【首届语言与智能高峰论坛会议邀请函】 …

白:
这种会怎么不请伟哥啊……

阮:
第一届会议重在推动,伟老师估计会泼凉水。

白:
我们大家还在混圈子,伟哥已经高处不胜寒了。

李:
一觉醒来 左眼发跳 原来是白老师。冷不丁开个涮 由头却是啥高峰会议。
认真滴说 休眠唤醒是正道 开始尝甜头了 。感觉以前syntax下力可能太大太苦 ,不如把负担更多转嫁给语义唤醒。

 

【相关】

【立委科普:结构歧义的休眠唤醒演义】

【李白对话录系列】

中文处理

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

【李白之23:“一切都在变,只有变本身不变”种种】

白:
“一切都在变,只有变本身永远不变。”
前后两个“变”动词特征明显,中间那个“变”怎么说?

我:
有了“本身”来构成chunk,那就是动名化的意思。
x 本身 –》 NP

白:
“这个学校就爱拿本身的那点光荣历史说事儿。”
好像这规则有副作用
在贴这个例子的时候,伟哥说的规则已经在脑子里转了N转,但还是感觉不那么简单。似乎一堆爹在抢儿子,儿子归谁跟爹们的场上表现有关。

我:
那就 fine-tune 呗:

“本身” 可以独立成 NP,“x+本身” 也可以成 NP,怎么区分呢?
tricky,不过磨细活总是可以磨。只说 x 是动词的情形:一种是限定 x 是什么样的 v;一种是限定x不能是怎样的 v。后者的话,第一个条件可以是 这个 v 不能是可以做 prep 的 v,于是“拿”踢出去了。当然这感觉还是 underkill。

白:
糊弄老板可以,如果自己是老板,不情愿这么干。

我:
还有一种就是拓宽 context:不过那个法子也 tricky 因为每拓宽一个 token,又增加很多可能性要 include or exclude,但所有的歧义区分如果需要看上下文条件手工调教,都是这么个事儿:precontext and/or post-context and/or include-or-exclude conditions on self, 手工系统让人抓狂就在这里。

白:
我是这么考虑的:作名词的“本身”一定有先行词,因此先行词的匹配特征一定会继承到“本身”上来。以“这个学校就爱拿本身的那点光荣历史说事儿。”为例,“学校”与“历史”的匹配特征,会被“本身”继承下来。于是,从“历史”反推,可以给先行词候选“学校”加分,而另一个先行词候选“拿”却得不到这样的加分。“爹们”的角力,就这样决定了“本身”的命运。

我:
这个有理。把 chunking 的边界问题 转化为 self 的 binding 的问题。不过上面的那套思路实现起来 也不是那么直接。听上去是一个 procedure, 而不是一个 pattern

白:
不需要procedure,确定一个标配的先行词,匹配特征差再唤醒非标配的。
比如,最近出现的名词作为标配的先行词。如果名词太远或者名词匹配特征得分太低,再启用左侧邻近词作为先行词。当然与“本身”呼应的也可能不是先行词而是后继词,比如“以本身的实力而论,张三是考不进清华的。”

我:
这个机制不是通常的 pattern matching 可以实现的 吧。通常 FSA 的 runner,运行的时候可以加一些 config。暂不考虑“本身”的 binding,句法捋顺倒也不难:

白:
后继词也可以设定标配。这就显出坑论的好处了。“本身”挖个坑,左侧名词优先填坑,左侧邻近词次优(如果不是名词还要给它变性)。由于是共指关系填坑,不耗费萝卜指标(还可以填其他坑)。
不做pattern matching
btw,昨天讨论的闭环填坑也不耗费萝卜指标。用填坑来取代pattern matching,规则按词典化的方式组织更方便。

我:
有意思。

白:
“那些质疑凯文·凯利的朋友,我真心怀疑是否认真读过他的书”

我:
他?
一共就俩 candidates,都联上算了,爱谁谁。到语义落地,大不了生成两个 insights,至少 recall 是保全了。

白:
(1)怀疑谁?(2)他是谁?(3)前一小句本可不作定语从句解,如何排除的?
这些问题是连带的,一环扣一环。

“三十里有几个五?-六个。”
即使数量结构这么铁,也有不该在一起的时候。
“三十里有多远?-不远,十来分钟就到。”
等判据出来,不知道要几个词。
“因强烈不认可挪威国家石油终止合同 中海油服提起民事诉讼”
谁不认可?
“因不满老师虐待儿童口出狂言张三愤然辍学。”
谁口出狂言?
理由?

我:
先 parse 看看:

最后一句掉链子情有可原,貌似人理解也有困难。原句似有语病,总之不对劲儿。

 

【相关】

【李白对话录系列】

中文处理

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

 

【李白之22:兼语式的处置及其结构表达】

白:
hownet坑的供给比较充分,但是也很难说一定不会超出上限。(董老师可以试试“这场火多亏消防队来得及时”)

有很多其他类型,比如,“穿着拖鞋就走出来了”。不知道该贴啥标签。还有“撸起袖子加油干”。这两个例子中,在时间上,伴随行为本身在前,伴随行为的遗留状态持续时段包含核心行为持续时段。比如,撸袖子的动作在前,然后遗留状态持续,然后加油干持续,然后加油干结束(或有),然后放下袖子(或有)。

李:
with sentiment:

at least for semantic landing to sentiment, the parse gives decent support。

宋:
O和ObjV是什么关系?

李:
宾语和宾语补足语

白:
比如:知道你来了。你是O,来了是objV

李:
not really

宋:
@wei 你的论元关系分析,相当炉火纯青了。

李:
宋老师果酱。

白:
这俩的区别?“消防队来得及时”为啥不可以是objClause?

李:
“多亏” 直接 link “消防队”,“知道” 不直接 link “你”,only linking to the event

白:
我是问,是否直接link,准则是什么?有什么必要区分这两者?

李:
语言学家的多数有共识。词典subcats 印象也是区分的,落地时候也发现有益。

白:
但很多研究汉语语法的人并不认为汉语有“宾语补足语”的。准则是什么呢?一词一议?
sentiment传递?
“我讨厌他总出差”
这里的“总出差”成了objV,“他”是O。“讨厌”的负面属性可以传递给O,是吗?如果这样,O必须是objV的逻辑S,否则,“我讨厌花摆在阳台上(,掉下来砸到人怎么办?)”
里面的“花”就要受牵连了。

李:
对。

我讨厌ipod
我讨厌iPod老死机
我讨厌花儿
我讨厌花儿老养不活

这事儿真细究的话,属于 subcats 同形打架:带 clause 的句型与带宾补的句型。

白:
“我喜欢厕所门朝北开”
问题是要给宾补一个存在的理由。理由不存在,打架就不存在了。

李:
几乎所有的汉语文法(英语文法亦大同小异)都区别下面三种句型:(1)动宾式;(2)兼语式;(3)宾语从句式。动+宾+宾补的 representation 反映的是兼语式句型。兼语是一种简省的说法,不是一个单纯的(atomic)的关系成分标签。兼语说的是既当V1的宾语又当V2的主语。表面上看,只要我们连上了 O,也连上了 S,所谓的宾补V2 也就间接连上了 V1,因此 把 V2 直接联系上 V1 作为宾补似乎没有必要。问题是,那样的 representation 不能表达 V2 在结构configuration上是低于 V1 的。事实上,这是一个典型的 right branching recursion,V2 是一个 VP(V-bar according to X-bar theory),V1 只是一个 lexical V (V no bar),V2 代表的 VP 整体都是包含在 V1 所辖的边界内。帮助 V1 构成一个更大的 VP。V2 的 VP 也可以是一个兼语式 (递归)。

白:
对。兼语句前后句的坑共享没有疑问。有疑问的是后句不必是前句的objV。在能清晰表达坑共享的前提下,V2有必要成为V1的直接成分吗?如果是,那就做小句宾。如果不是,就是松散的两个谓词做兼语式。为什么要弄出个第三条道路?

李:
小句宾 与 兼语式 有相同的地方,有不同的地方,也有灰色地带。

白:
我知道你来。我喜欢你来。
这是一样的结构。

李:
不能拿灰色说事儿。“得悉”,只能带小句,不能带兼语,这是黑白分明的 case。

得悉你来
* 得悉你

灰色地带最突出的案例在英语的 minimal pair:

I demand him to leave immediately.
I demand that he leave immediately.

白:
这是生成时才用得到的区别,分析时可忽略。

李:
英语句法上泾渭分明的两个句型,在逻辑语义深层其实是大同小异的,很灰色。

白:
汉语里没必要弄出第三条道路

李:
不一定只有生成才需要区分。语义落地也可能得益。如果没有宾补的直接联系,我问 O,就不能区分单纯的 O 和带宾补的 O,如果所有的兼语都变成 宾语小句。

白:
我派你去,就是“我派你”“你去”不需要让“去”从属于“派”。
或者说,“派”不需要给“去”预留一个坑。

李:
那就切断了动词 V1 与兼语的直接联系。必须通过 V2 才能间接联系到兼语。

白:
错,萝卜共享就是联系

李:
有时候我们不在乎是 宾语 还是 兼语,我们就没办法用同一个子树句型做抽取。譬如,“多亏”:

多亏老李。
多亏老李送我回来。

这两句要找感谢的对象,就无需区分 宾语和兼语。

白:
这没什么不好。多亏我处理成副词。“多亏了”也是一个副词

李:
如果第一句是 VO 第二句是 V+Clause,就必须用两个句型才能捕捉。

白:
多亏老李,是副词升格为动词。

李:
关键是,有时候我们需要区分宾语和兼语,有时候我们不要区分,为了同时照顾这两种需要,把兼语处理成子句是不妥的。

白:
可以不从句法上区分,而从词典来区分。

李:
退一万步,多一个宾补的直接联系,只有好处,没有坏处。

白:
我的处理是要么包进来要么切断。多了歧义打架需要处理,而这本来可以是伪歧义。

李:
是真歧义,那就该处理。是伪歧义,也可以硬走一线,系统内是可以保持一致性的。你这里所谓伪歧义的情形实际是灰色地带,或两可地带,系统内部可以规定走哪一线,内部协调即可。伪歧义的真正困扰是系统无法区分,随机走路径。如果有办法确定性地走一条路径,理论上的伪歧义在实践中就化解了。传统 parser 的伪歧义困扰是,伪歧义与真歧义混淆在一起,使得真歧义无法 stand out (being  identified)。这里的情形不是。

白:
中间没有N的,也有助动词和补语两种可能性。助动词带的谓宾真包进来,补语我处理成共享萝卜的对等合并,与所谓核心动词并无隶属关系。只不过形式上把根让渡给前面核心动词而已。
看看前面的例子,“我喜欢厕所朝北开门”,什么特征决定了“厕所朝北开门”是小句宾?

李:
不好说。
假如有一家厕所公司专门建厕所,就好像苹果公司造 iPhone 一样,“喜欢厕所朝北开门” 与 “喜欢iPhone照相清晰” 有什么根本区别?再如,“喜欢厕所清洁卫生”。

与其花力气区分灰色地带的兼语 from 子句,不如一条路走到黑,对灰色的那些词规定一条路径。到落地的时候,如果需要,再进一步 fine-tune 来决定。如果是 sentiment 落地,就可以这样来 fine-tune:“喜欢”的兼语必须是产品或其他实体专名,才可以让其得到 positive sentiment,这是从严的 fine tuning。从宽的 fine-tuning 可以要求兼语不能是带有贬义色彩的名词,assuming “厕所”是这种类别。但是这种 fine-tuning 的拿捏,只对做“兼语”的名词需要,如果“喜欢”所带的不是兼语,而是纯粹的名词宾语,那么不管宾语是王八蛋还是杀人犯,喜欢的 positive sentiment 还是落在宾语身上。

当年的德国,很多人喜欢甚至崇拜希特勒。” 对于希特勒,这是 positive sentiment。但是,cf:

我喜欢希特勒被处以绞刑,而不是自杀身亡,逃避人民的审判。

这句中的“喜欢”,对于 希特勒 就不是 positive sentiment,因为 parser 把希特勒parse成有宾补的O(即兼语),而不是纯粹的 O

白:
喜欢厕所朝北开门 与 喜欢iPhone照相清晰 有什么根本区别?—这话也是我想说的,但我认为它们都是小句宾句式,与兼语无关。

李:
我要说的是,对于这样的 cases,要不一律处理成兼语。要不一律处理成小句宾语,只要 consistent 即可。

白:
希特勒那句,也是小句宾,没兼语什么事儿
什么情况下sentiment可以穿透到下面一层的成分,不是由句法决定,而是由另外因素决定。

李:
“我喜欢少年希特勒努力发奋。” 这句呢,也是小句?

白:

李:
我要说的是,对于有灰色地带的那些词和句型,可以人为规定一条路径。
区分:“我喜欢少年希特勒努力发奋” “我喜欢老年希特勒被处绞刑”,一个是对“希特勒”是褒义,一个不是。

白:
褒义针对整体,是否传导到部分,it depends, 针对整体都是褒义。

李:
说的是部分。我喜欢【human】和 我喜欢【human】VP,对于这个 human,默认是褒义的。

白:
要区分的只是能传导到部分的褒义和不能传导到部分的褒义。我喜欢【human】vp 是天经地义的,喜不喜欢其中的【human】,不由句法决定。
在我这里不默认。而且默认sentiment设定,和搞出一个句法类别,也是两回事。

李:
默认有很多便利。偷懒的时候,默认在统计上对数据质量有好处。默认这东西,不用白不用,尤其是从工程和实用上考量。我目前的 “喜欢” 的 sentiment 规则,不论中文英文,都是走默认的道路:管它后面的 NP 带不带 VP,只要 NP 是 entity,就默认是 positive 落地了。这个 practice 在实践中有莫大的好处。 “喜欢希特勒 VP” 这样的可能的例外极其罕见,以致于根本不必在默认之外再费力气去区分。而 “喜欢厕所VP”不算例外,无害:depending on 厕所是否看成是 entity,或者被排除在语义落地的雷达之外;或者落地了,也是正确的。

白:
充其量是个heuristic

我喜欢张老师讲古代史,不喜欢张老师讲现代史

李:
这个不是反例,前半句是 褒,后半句是 贬,都落地到 张老师身上。

白:
加分减分都没问题。问题是逻辑推论有没有。喜欢张老师和不喜欢张老师如果都是逻辑推论就有问题。讲逻辑的人是分得很清楚的,喜欢,对事不对人。sentiment也是很清楚的,一个给张老师加分,一个给张老师减分。但是确实没有“喜欢张老师”和“不喜欢张老师”这两个逻辑推论。

李:
回到原来的 arguments,如果 (1)我喜欢张老师;与 (2)我喜欢张老师讲古代史,是两个不同的 parses,sentiment 落地的时候,就必须有两个规则来对付这两个不同的结构

白:
这个很难吗?我不觉得。爱屋及乌,爱小句及小句逻辑主。一句话的事情.

李:
天下无难事。
但是,对于如此基本而普遍的 sentiment 的表达(love类 和 hate 类),本来可以直接搞定,非要绕一个弯儿再搞定。本来一组规则,非要增加一倍的工作量去写规则,去维护,这是很不合算的。

即便不谈 sentiment 这个具体的落地应用,抽象来说策略,这个争论的本质就是:两个句型有相同的部分,有不同的部分,如何处理?其实没有普世标准,就是系统内的一致性和方便性,系统内协调。

白:
不是这样的,就是一个sentiment penetration属性赋值yes还是no的问题,直接写在相关动词的词典里。其他都不受影响。标配赋值no,需要设yes的明示。

李:
我喜欢他。
我喜欢他老实。
我喜欢他做好事。

换一个角度看,第一句和第二、三句在“我喜欢他”这个部分是一样的。后面的 AP 和 VP 也可以看成是喜欢的某个部分或方面(aspect)(或理由),这个角度是先对人后对事儿。与你的“喜欢”应该对事不对人,或由事及人,具有等效的逻辑性。即便我说“我喜欢X”,常识告诉我们,世界上没有完美的X。“喜欢X” 后面隐藏着后面的 likable aspects,X 与 aspects 是整体与部分的关系。

白:
我喜欢他,我喜欢他老实,我喜欢他的老实。VS
我喜欢这本书,我喜欢这本书出版,我喜欢这本书的出版。

也就我们这种领域,讨论一个语言学概念的立和废,也要扯上系统甚至应用。

李:
是从部分的角度来推及整体,还是从整体的角度推及部分,不过是角度不同。

白:
如果一个arg,是这样。角度不同。两个坑就未必了。

李:
就好比半瓶水,一个角度看到了半瓶水,另一个角度看到了半瓶空气,都是有效的逻辑。

白:
我喜欢这本书出版,和我喜欢这本书,不是延展后者的关系。我们还可以说,我喜欢这本书春节后再出版。喜欢的重点既不在书也不在出版,而是在出版的时间。可以既不喜欢书,也不喜欢出版,只是对那个出版时间情有独钟。一个坏人干了件好事,我们总可以喜欢那件好事吧。不需要扯上喜欢坏人。

一个出版商可能对某本书恶心到吐,但是那本书的出版可以帮他赚大钱。他喜欢“那本书出版”但不喜欢“那本书”也是情理之中的。heuristic可以进入系统,但进入语言学是否合适要画个问号。进入系统也有“诛心”之嫌。喜欢坏人干的某件好事,被当成支持坏人的证据,这样的系统大家都要当心了。

李:
喜欢坏人干的某件好事,被当成支持坏人的证据,在情感分析上没有错。在大数据分析的时候,点点滴滴皆证据。坏人干了好事儿,只要被提到,就给这个坏人加了一分(褒)。这一点儿也不影响对于这个坏人的舆情分析全貌。恰恰相反,这才是真实的舆情。如果坏人干了好事儿被提及 m 次,坏人干了坏事儿提到了 n 次,纯粹厌恶坏人的情绪表达提到了 o 次,纯粹喜欢坏人的情绪表达提到了 p 次(p 通常接近零),那么这个坏人的褒贬指数,就是 (m+p) 与 (n+o)的比例。请放心,p 基本是 0,而 m 也会远远小于 n,这个大众舆情不仅靠谱,而且真实,bias free。

宋:
我喜欢希特勒自杀。

李:
宋老师的这个例子前面已经论及。回顾一下就是:情感分析中,“喜欢”的默认规则的确可能把它算成是对“希特勒”的正面评价。因为这个默认规则是把宾语和兼语同等看待,不去问后面的宾补 VP。理论上,这个结构根本就不是【宾语+宾补】的结构,而是【宾语子句】的结构,但是由于区分二者对于“喜欢”这样的词,有一定的难度,所以我们可以在 parsing 阶段一律当成兼语结构处理和表达。这样一来,默认的sentiment规则就会犯错。

犯错怎么办?如果这种案例极其罕见,不值得做,那就算了,因为默认的sentiment规则在绝大多数的场合是保证了数据质量的。如果觉得还是值得做,那就在默认sentiment规则之上再做一条规则去 override 默认。这条规则不需要改变 parser,而是利用 parsing 在这类结构上的 consistency(错也错得可以预测) ,将错就错,矫枉过正,把这个错纠正过来。换句话说,这个策略等于是休眠唤醒,不过这个休眠唤醒不是在 parsing 的后期进行,而是在 sentiment 语义落地的时候进行,其效果等价于把【兼语式】重新分析为【宾语小句】,切断“喜欢”与“希特勒”的语义直接联系。

不知道我说清了没有。可能有人会问:既然在语义落地时候要做类似休眠唤醒的工作,为什么不索性在parsing里面(parsing的后期,语义模块)里面做休眠唤醒呢?理论上,提前做休眠唤醒,使得parsing更精准,可以benefit不仅仅这个 sentiment 的语义落地,还可以 benefit 其他的语义落地和应用的场合。但是,实践中在哪个阶段做合算,不是那么简单。因为休眠唤醒这些事儿大多是长尾现象,鸡零狗碎,做不胜做。在还不能确认到底有多少好处前,往往顾不过来在 parsing 中统一处理好。而且很多时候,做了也白做,你需要用到那个落地或那批落地也许根本就用不到。

Anyway,point is,落地的时候是可以包容 parsing 的错误和不足做包容性产品开发(adaptive dev),只要 parsing 是可预测的数据结构,是有内部直通车的,而不是 offshelf  拿来的 parser,缺乏足够的内部支持和协调。Having said that,如果确实事先可以 identify 一些带有普遍性的休眠唤醒现象,以及可以惠及很多可能的语义落地应用,而且也有足够的时间和资源去做这些细线条的深度语义工作,那么不要等到落地就在提前在 deep parser 的语义模块里面做好,当然是更理想的情况。

白:
对坏人的褒贬判断,对事不对人自然左右不了大局,没什么可担心的。问题是拿对事不对人的表述做以坏人为参照的站队分析,这就很可怕了。

李:
可怕啥?或者就是大海里的一颗老鼠屎,丝毫没有影响。或者就是,用休眠唤醒杜绝它(它=“拿对事不对人的表述做以坏人为参照的站队分析”)。休眠唤醒之所以可以进行,是因为 parser,已经准备好了结构基础:要啥有啥,哪怕这个 parse 是不完善的。要 O 有 O,要 宾补 有 ObjV,要逻辑 S 有对于 V2 的逻辑 S,四通八达的路径都在。driving word “喜欢”也在,可以词驱动。所有的结构的节点词的信息,包括 ontology 都在,包括 “希特勒” 是个独裁者、通常是坏人这样的信息也都在。有了直通车,啥事儿做不成?什么案子不能翻?什么深度休眠不能唤醒?什么具有统计性的“可怕”的后果不可以杜绝?

白:
分析成小句宾,诛心或误伤的责任就不在分析器的开发者。而在后道工序。否则人家会说,都是你分析的我喜欢希特勒,我哪儿喜欢了?一颗老鼠屎,也那个。对于小句宾来说,没分析出我喜欢希特勒,这大不一样。但是小句宾结构与“喜欢”结合,大概率被穿透,这句话可以说,之后也是要啥有啥。谁用了谁负责,但分析器的开发者也没说错话。

李:
话说到这份上了,其实也没有多少进一步需要辩护各自做法的 arguments 了。选择这个兼语的表达,肯定不是因为明明知道处理成宾语小句更好,而不为之。一定是自有苦衷和盘算。

白:
我若干年前也用过这样的表达。最典型就是“我是县长派来的”,把其中的“来”也作为“派”的一个arg。后来发现,其实去掉它没什么损失。因为“派”和“来”共享了一个萝卜,所有联系都在里面了。

李:
隐约的感觉是,去掉它在逻辑语义深层没有损失,可能的损失是句法表层的痕迹(语法学界叫做 configuration info,就是 X-bar 理论里面的 bar 的信息)。

白:
这是有很多其他途径可以记录的

李:
留住它就是留下了这种 bar 的痕迹,就好比我们利用小词解构了实词之间的句法语义关系以后,我们并不把小词当敲门砖丢弃,而是用 X 挂上一样。虽然 理论上,这些小词对于深层语义已经没有意义。另一个例子是主动被动,到了逻辑语义的核心,这个 voice 的不同色彩,没有地位,可是语言分析中,留下表层的 voice 信息,还是可能对语义落地和应用有好处的。

 

 

【相关】

【李白对话录系列】

中文处理

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

【李白之20:得字结构的处置及其结构表达】

白:
他写字好
他人品好

多么平行呀,为什么两个“好”待遇这么悬殊

李:
好 是一个百搭词。因为百搭 所以赶上谁谁:他好;人品好;写字好。

白:
其实就是有一个pos为wildcard的坑,我这里标记是X

李:
人品和写字 都有一个 “人” 坑:一个是【所有】,一个是【施事】。

白:


李:
没看清 “得”怎么带的“好”。

白:
箭头方向是坑,箭尾方向是萝卜。得的输入是谓词,输出是体词。好的输入既可以是谓词,也可以是体词。语义上是把动作转化为为对象,然后以对象的身份填坑。

李:
“好” 没疑问。说 “得” 输出 N, N 被“好”吸收,这个说法有些不好理解,也不大看得出好处。传统说法是 “得字结构” 做谓语的【补语】。不一致的地方在于得字结构怎么构成、谁先谁后。得字结构的构成是“得”有坑,而不是“得”先与谓语结合,然后去填坑。你这里parsing 的先后正好与传统做法次序相反。

白:
结合以后做主语,让“好”做谓语。不是“好”或者“得好”做补语。

李:
这个句法分析与众不同。“得字结构”构成很容易:

【得 ➕ 评价类形容词】 是一种; 干得漂亮。
【得➕vp】:跑得快(这时候 vp 的逻辑主语沿用前一个谓语的主语)。
【得 ➕ s】 是另一种: 气得他哭了。

白:
没问题,“他”反填回“气”不占名额,可以处理成一个。无非就是约定什么情况下一个萝卜填俩坑不占名额的问题。跑得快,和跑得上气不接下气,结构上没有本质不同。病得很严重,和病得起不来床,也是一个结构。并不因是否使用了评价类形容词而不同。传统所谓程度补语,都可以用这个套路。

但是可能补语好像不一样。搬得动,睡得着,考得上一类。跑得快,跑快了,跑快些,跑赢了 填坑结果一样吗?

李:
可能补语是词典扩展 lexical rule,算是词典一级的延伸扩展,不难处理。

白:


因循守旧最简单了,我之前就是这么玩的。垂直方向是填坑关系,水平方向是修饰关系。但是不知道谁填“严重”的坑。root也和语感不符。这样的结构也是可以有的:
“他的病很严重”和“他病得很严重”不应该在“严重”的填坑方面有大的不同才对。要么这样:

“得”的填入体“很严重”留下了一个待填的坑X。作为root的“病”,向自己修饰语的体内回填S,不占萝卜指标。这就成环了。

同理,“他病得起不来床”可以处理成:这次轮到体外的“他”向“起不来床”回填N。在体外不占名额,所以“病”照填不误;体内“起”“来”为合并关系,共享除了“床”之外的一个坑,也只用一个名额。所以从“他”出发的三个萝卜,竟有两个是不占名额的。

这么处理,如果不耽误回填,也挺好。反正再怎么折腾都只折腾“得”一个词条,跟别人无关。补语还是补语,但需要回填。的[N+/X]、地[S+/X]、得[+S/S]都有了。它们都是单进单出,去掉确实不影响拓扑,但是放在那里可以揭示语义类型转换的逻辑。

李:

关键是 “病”与“很严重”具有直接联系。另外,【human】作为 N 的 Mod 与 【human】作为 V 的 S,具有相似性。这个甚至在 X-bar theory 中就有所揭示(specifier and subj are both external args)。进入深层逻辑,可以认为这是同一个关系的两种变式。

白:
可以再推广:
这本书的出版,这套房子的装修,这款软件的开发。

不是S,不是human,也有同样的暗通款曲。

李:

白:
看不到“这本书”是如何填“出版”的坑的。

李:
还没做呢,自然看不到。都是 Mod 作为句法桥梁,语义中间件目前还没全到位。加上这个逻辑 O 不难,只要与逻辑 S 区别就好:“出版”的宾语坑,需要一个【publication】的萝卜,放松一点也起码是一个非【human】或非【organization】的萝卜。

白:
这一部分完全是探讨,与formalism无关,只是在formalism框架内探讨小词“得”负载结构的不同表示方法的优劣。

 

 

【相关】

【李白对话录系列】

中文处理

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

【李白之18:白老师的秘密武器再探】

白:
伟哥对主谓vs.述补,️“得”字的作用有何评述?

李:
题目大了点儿
小半部汉语语法了 都。

白:
动作敏捷,行动迅速,打仗内行,排兵布阵有章法,运行平稳……
执行命令坚决,干家务不情愿

root感觉上都是在后面,是谓词在填谓词的坑。为什么被填坑的反而成了陪衬,这说不通。第一个“动作”是名词,我拿来跟后面对比的。说大不大,就是上面这些表达,head是谁,谁填谁的坑。如果跟主流是反着的,有什么后果。

李:
拿 practice 说话吧。

他努力工作
他工作努力

一般来说 “工作” 只有一个坑,“努力” 有两个坑: 一个人,一个是人的动作。这就是句法 subcat 与 逻辑 subcat 不尽相同之处。句法上,老子的 subcat 里面有儿子 args 的坑, 没有 mods 的坑。譬如 动词有 主宾等 但没有状语的坑。名词有补足语的坑 但没有修饰语的坑。但是逻辑上不同,逻辑上 任何 mods 定语啊 状语啊 对于与它直接联系的老子,都有一个语义相谐的要求。这种要求可以写进做修饰语的那个词的词典去。但对于语义搭配稍强的情况 也不仿写进做老子的词条去。关系是mutual 的 选择谁去 encode 这种细线条信息 其实没有一定之规。

问白老师一个问题,这样的借助小词接力,一步一步、一对 tokens 一对 tokens 的两两向前推进 parsing,会不会造成过多的假 parses (所谓伪歧义的困扰)?
DG中,语言单位虽然都可以 binary 关系表示,但是 parsing 的时候却不宜两两推进,因为那样的话,条件往往失于过宽。还是我理解有误?

白:
不会啊。(1)小词对距离很敏感,太远了肯定要靠实词自身的互联。(2)小词对结合方向很敏感。实词语序乱了不要紧,坑可以把它们找回来,但是小词语序乱了一定不知所云。什么语法不借助中间件都必然失之于宽。所以,WSD也好,二元关系也好,树也好,最后都要靠中间件摆平。

李:
对着例子谈吧:譬如 是AP的
这个汉语表达 affirmative 的 pattern,就是这么简单的一个 pattern,
用 “是。。。的” 把形成的 AP 包裹一层。因为这种小词没有啥意义,最多不过是给这个谓语加一个 affirmative 的 feature 的信息,其他的关系连接,还是要直接通向 AP:

她是漂亮的 == 她漂亮

白:
AP有两个表达方式,一个是紧靠名词的,一个是松散的(作谓语,作补语,通过“的”作定语)。是……的,是后一种。我们先解决句法成分“一个都不能少”的问题,再解决语义关系的抽取问题。

李:
可是看白老师的图,感觉在小词里面绕。

白:
没有无缘无故的绕。

李:
包裹一层的话,就不需要绕,就当成 AP 的前缀后缀了。说的是 pattern 包裹,结构根本就不用包裹,不过加一点信息而已,甚至丢掉那信息也无大碍。

白:
问题是,不光“是……的”,还有“有……的”还有其他“V……的”。我们可以统一处理。到了语义关系抽取阶段,衣服会脱下来的。二元关系进行到底,应该看不见pattern。pattern体现为判断二元关系成立与否的前后条件。

李:
的字负载结构 统一处理的好处何在?

白:
一视同仁

李:
这个好处貌似不实惠,也不必需。

白:
所有词、所有二元关系一视同仁,用同样的formalism处理,不排除神经。

李:
哦,有无缝挂靠RNN的便利?
一视同仁本身最多是显得 elegant,这种二元推进一视同仁的处理与用长短不同、随机决定的 patterns 处理,除了所用的机制不同,还有什么特别的说法?

白:
没有规则,没有pattern,bug都在词典里,要de,就改词典。语义构造最大限度地平行于句法提供的二元关系。

李:
patterns 甭管怎么写,说到底也是统一的机制,某种有限状态及其延伸罢了。

白:
pattern写了也不矛盾。不改变核心机制,只改变前后条件。

李:
只改词典,不改规则的例子见过,那就是 HPSG,但那是建立在词典结构无限复杂化的基础上。现如今用的是简单得多的词典结构。Categorial grammar 那种吃了吐、吐了吃在 cat 基础上的填坑挖坑,要实现只改词典就能 parsing 的开发,感觉哪里缺了什么。

白:
但是把pattern固化,就会引进来不robust的问题。很多情况下,系统自己找对象比用pattern拉郎配更聪明。

李:
同意,这是一个问题,不好拿捏。但换一个角度,用 patterns 直观易读,与人对语言现象的捕捉非常近似,而且 patterns 其实也还是立足于自己找对象的 subcats 的词典化信息基础。问题不在 patterns 上,而是在 patterns 的条件拿捏上。

白:
对,我就是在扬弃范畴语法复杂结构的方面跨出了一大步,坑和萝卜,都是“单体化”的。直到目前还没有发现什么语法现象必须引入复杂结构的。
都一样,吃了吐的路线,关键也在拿捏。

李:
又要简单统一,又要避免伪歧义,感觉是一个矛盾(当然 结构复杂本身也 adds to 伪歧义 ,那是另一个话题)。

白:
伪歧义在外面,中间件来搞。但是中间件面对二元关系搞非常清晰,中间件面向pattern搞就累了。二元关系是最简单的结构。就是一个词典词一个义项,其pos定义只有一层。只有一个“/”号。

李:
这就是我还没有理解的地方,感觉机制太简陋。机制简单统一,词典信息也简单,pos 只有一层,不过是 encode 了一些坑的信息,用的也是简单的 x/y,规定了输入(挖谁的坑)和输出(填什么坑)。如此简单统一,对付自然语言的窍门在哪里呢?

中间件通过二元关系搞定原则上没有问题。所谓语义中间件,在我这里,不过是把已经成串的珍珠链,经过某个子链,把一些语义相谐的珍珠挑出来,让间接关系变成直接的二元语义关系。假如初始的二元图是: 1–》2 –》3 –》4 –》5,语义中间件可以做到:1 –》3; 2 –》5,揭示诸如此类的hidden的逻辑语义关系。白老师的中间件有所不同,用的也是语义相谐(通过某种无监督训练而来),但目的是确保parsing不受伪歧义的羁绊。

白:
复杂的地方是什么时候有免费额度。

荀:
白老师是把这些简单的范畴放到RNN中,这个Rnn中间件性能决定了分析器性能

白:
句法“是什么”在这种机制下确实不复杂但管用。“怎么达到”是另一个问题。

李:
免费额度怎么讲?

白:
比如定语从句里面的坑,就是不占萝卜名额的。填了里面还可以再填外面。还有“NP1的NP2”,如果NP2有坑且与NP1语义相谐或统计意义上搭配,则NP1填入NP2也不占萝卜指标。比如“张三的弟弟”,“这本书的封面”。

李:
好,作为 syntax 表达,这些都不是问题。说说 ”怎么达到“ 吧。

白:
这一部分是parser最核心的地方了。

荀:
就是白老师的“毛毛”,一种利用大数据无监督的subcat嵌入算法。

李:
不是说只要 debug 词典 就可以达到吗?词典也没太多 debug 的余地,假设挖坑填坑都基本在词一级标注清楚的话。然后就灌输大数据?无监督的大数据在这些词典信息的基础上,学习出来的结果是什么形式呢?应该是词典 subcat 的语义相谐的条件。这些条件一旦学出来,就成为 parsing 的伪歧义的克星。

荀:
借助subcat嵌入的分析器要是突破了,短语和pattern也就嵌入在网中了

李:
换句话说,词典的每个词的粗线条的挖坑填坑先由人工敲定,而这些坑的语义条件 让大数据来填,从而粗线条变成细线条。从而伪歧义急剧减少。我是这么个理解。

白:
WSD和Matcher学出来的东西不一样的。

荀:
原来的做法是借助词的语言模型或词的Rnn消解伪歧义,白老师是把Subcat嵌入了网中。但一直担心前期中间件的可信赖程度,中间件错了,休眠唤醒,那么唤醒时是否用了patten?

李:
利用大数据无监督的subcat嵌入算法,这个方向没有问题,词(直接量)不够,一定要加subcat。唤醒可以看作二阶中间件。唤醒的都是局部的、个别的现象(子图 pattern),至少词驱动这一级唤醒,机制上与中间件利用语义相谐没区别,是靠谱的事儿。唤醒在我的实现中,一定是用 pattern (子树)为基。

WSD和Matcher 各自怎么学?

困了,回头好好学习,明儿天天向上。

荀:
这种唤醒的词典知识,主要是利用词搭配信息

李:
那当然。

荀:
李老师休息吧,明天搬着板凳听你们聊,到了关键地方了

白:
为了保证WSD和Matcher之间在大概率情形下都是串行的,被WSD压抑的候选都休眠。此时流程是线状的。为了对付有迹象可察的小概率情形,休眠的候选可以唤醒。此时流程是闭环。

李:
前半段懂了,后半段,闭环是什么?

白:
唤醒

李:
线状是说的 deterministic 吗?

荀:
如果被唤醒,wsd也会跟着改变

李:
唤醒在我看来就是局部重组。nondeterministic 是从一开始就留多条路径,唤醒是在一开始不留多条的情况下,activate 另一条子路径,摧毁现下的子路径。

荀:
如果是局部的,就是打补丁的机制。白老师的闭环是指唤醒和wsd联动?

李:
唤醒都是打补丁。
既然一开始不留多条路径,那么怎么唤醒另一条路径了?

白:
局部局到什么程度是不以主观意志为转移的,可以理解为一种lazy的、保相谐性的subcat传播。

李:
我这里的诀窍就在,开始不在实现的 parse 中留,但词典里面的潜在路径的种子还留着,因此词驱动可以唤醒它。

白:
唤醒的不是路径,而是词典的pos候选

李:
唤醒了不同的萝卜

荀:
唤醒了另外一个候选,也就是到引导不同的subcat路径,可以这样理解吗?

李:
不就等价于唤醒了不同的路径吗?因为萝卜换了,原来的萝卜所填的路径就废了,新的萝卜要填的坑也变了,新的路径也就唤醒(新生)了。

荀:
填好的萝卜重新拔出来

李:
对呀。
这是摧毁无效路径 为建立更好的路径做准备。

荀:
如果wsd需要频繁被唤醒,唤醒的中间件压力山大。

李:
wsd 常常被夸大 实际中没那么大。太细的也不用做。

白:
所以三省吾身

@荀 是这样子:休眠唤醒的动态工作空间专属wsd,它的范围是受控的,但之间逻辑较为复杂。中间件依据ontology和大数据统计结果,体现了subcat嵌入,是相对静态的,一组二元关系一个查询,彼此逻辑独立,既为wsd服务也为matcher服务,因此根本不在乎频繁程度。忙了就并发,不会惹到谁。wsd局限到一个句子,最多加上定宽滑动窗口的语境变量,工作负载是可控的。wsd和matcher一切照章行事,没有任何语言知识和常识,只有“过程性的、机械化的”程序。语言知识和常识都在词典和subcat嵌入里。

荀:
明白了,就是根据需要,随时被调用的“语言模型”,这个语言模型嵌入了ontology和subccat信息的RNN网络。

白:
今天说多了,打住。

荀:
开启了一个大话题

白:
过去统计意义上的“语言模型”是一勺烩的,所以难以精准地把有结构潜质的二元关系筛选出来。一旦仅针对这样的二元关系上统计手段,前途就一片光明了。萝卜是第一性的,路径是第二性的。

王:
@白硕 wsd和matcher一切照章行事,没有任何语言知识和常识,只有“过程性的、机械化的”程序。语言知识和常识都在词典和subcat嵌入里。

白:
怎么

王:
白老师,要走”小程序,大词典”路线?

白:
对,小程序、大词典、特大ontology、数据被ontology揉碎消化。前面再加个“零规则”。

王:
不知是否每个词做为Agent?然后多个Agent之间相互自适应?

白:
没那么自主

王:
请教白老师,何为“零规则”?是预留的待扩展的规则?还是压根就不用规则?

白:
是根本不要规则。

我们先解决“谁和谁发生关系”而不必具体明确“是何种关系”,只笼统地分成:“a是b的直接成分”、“a是b的修饰成分”以及“a是b的合并成分”三种情况。

现在还都没说定性,只说定位,谁跟谁有关系。结论是,就这么糙的事儿,也得动用ontology。

李:
句法不必要太细。语义可以细,但那个活儿可以悠着点,做多少算多少。

白:
关系不对,上标签何用。标签可以是句法的,也可以是逻辑语义的

李:
句法的本身就模糊一些。很多语言的主语与谓语是有一致关系的。这就给“主语”这个标签一个独立的句法层面的支持。虽然细究起来,这个所谓的主语,可能是 human agent,也可能是 instrument。

白:
粗粒度不等于错位。位置对上了不知什么标签这叫粗粒度。
位置不对叫啥

李:
位置不对就是父子认错了。这是最大的错,皮之不存,句法或逻辑语义标签也就谈不上。

看看这个: 这些纸我能写很多字

“这些纸”无从着落了。那就用 Topic 或 Next 耍个流氓:它们的句法意义与句首状语(全句状语)差不多,至于是什么状语(时间、地点、让步、工具、结果、原因、。。。),这是逻辑语义的标签。想做的话,让后面的语义模块去做:这些纸 Next 能写字。

其实拿目前的 parser parse 一下大数据,对于 Next 前后的词做一个统计,基本肯定可以挖掘出不少强搭配或弱搭配来。Next 虽然标签模糊,它把有关系的 tokens 的距离拉近了,虽然句法不知道是何种关系。

白:
暂不确定逻辑语义我赞同,但没有坑就不填坑,没有加号就不修饰也是铁律。于是需要一个节点做这个对接。大数据中这个节点有实例支持,引进就更理直气壮而已。
在这个阶段根本没有人去做逻辑语义标记。

李:
逻辑语义是 semantic parsing 的目标;syntactic parsing 可以不管。乔老爷说句法独立于语义肯定是有瑕疵的,但句法确实有相当地独立于语义的部分。这个独立性在形态语言中表现很充分,到了裸奔的汉语表现就差一些。但也不是一点独立性也没有。换句话说,总可以把一部分现象看成是纯粹句法的现象,不牵扯语义,也可以一路走下去。

白:
现在我是在定义syntax,自己定义的东东,自己要认账。
说好了不耍流氓的地方,就是坚决不耍,可以耍的地方也一定当仁不让。

李:
系统内部怎么协调,没法说对错优劣。我是要说,耍流氓也有其耍的道理。不耍,又不想牵扯太多语言外知识,那就只有断链。耍流氓比断链强。

白:
引入虚节点,有大数据背书,挺好。同样达到不断链的效果。

李:
还有一个更重要的特点是:句法模块与语义模块分开,有开发和维护的便利。比一锅炒感觉有优势。

白:
相谐问中间件可以,补虚节点问中间件当然也可以,毕竟大多数情况不需要补。
wsd和matcher现在连一点语言学知识都没有,是最不一锅炒的架构了。
内事不决问词典,外事不决问数据

李:
开发一个模块有两个模式,一个是轻装粗线条,knowledge-poor。另一种是细线条,knowledge intensive,前者的好处不仅在轻装,不仅在覆盖面好,而且在鲁棒性好。后者则是精准度好,而且可以聚焦去做,一步一步 peace-meal 地去做。很多人做了前者,但是带来了一个巨大的伪歧义泛滥的问题。我们做到了前者,而且基本对伪歧义免疫,这算是一个成就。至于后者,那是一张无边无际的网,不急,慢慢做。

白:
大部分不鲁棒都是伪语序造成的。让萝卜和坑自由恋爱,是鲁棒性的最好体现。

李:
白老师主张先不利用语序作为句法的制约,而是立足于词典的对萝卜的预期,以及查与周边 candidates 在中间件表达出来的语义相谐度。这样做自然是增加了鲁棒性(我以前提过,汉语实词之间的语序灵活到了超出想象),但同时也隐隐觉得,不问语序也可能是自废武功的不必要的损失。其实是可以把语序作为一个option加入坑的预期的。

 

【相关】

【李白之15:白老师的秘密武器探幽】

【李白对话录系列】

中文处理

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

 

 

 

 

【杞人忧天:可怕的信息极乐世界】

今天想信息过载的问题,有点感触。

我们生在大数据信息过载的时代。以前一直觉得作为NLPer,自己的天职就是帮助解决这个过载的问题。就好像马云的宏愿是天下没有难做的生意,我们玩大数据的愿景应该就是,天下没有不能 access 的信息。于是谷歌出现了,用粗糙的关键词和数不厌大的气概,解决了信息长尾问题。于是我们开始批判谷歌,信息长尾解决的代价是数据质量太差。于是人智(AI)派来了,借力深度作业(deep processing, whether deep learning or deep parsing),企图既要解决大数据的长尾,也要大幅提升数据质量,让全世界对于信息感兴趣的心灵,都有一个源源不断的信息流。这是从我们从业者的角度。
今天换了个角度想这个问题,从信息受众的角度。作为消费者,作为白领,我们从人类的信息过载的战役不断优化的过程中得到了什么?我们得到的是,越来越高质量的、投我所好的信息流。以前是在过载的海洋、信息垃圾里淹死,如今是在精致的虚假的满足里噎死。感受不同了,但反正都是死。哪怕做鬼亦风流,死鬼却从不放过我们。于是我们花费在朋友圈、新闻apps、娱乐apps的时间越来越多。无数天才(很多是我的同行高人)绞尽脑汁研究我们的喜好,研究如何黏住我们,研究什么诡计让我们拼死吃河豚。
一个人敌不过一个世界,这是铁律。七情六欲血肉之躯的消费者个体敌不过无数盯着消费者喜好的商家及其帮凶(包括在下)。于是我们沉沦了,成为了信息的奴隶。我们同时也不甘心,在努力寻求自救,不要在糖罐里甜腻死,虽然这甜越来越幽香、巧妙,充满诱惑。我们就这么一路挣扎着。但随着信息技术的提升,中招的越来越多,能自救的越来越少。
世界有n十亿人,m千万个组织,在每时每刻产生信息。假如我们把自我信息满足的门槛,用各种 filters 无限拔高,拔高到千万分之一,我们面对的仍然是 n百人和m个组织的产出。当技术提升到我们可以 access 这个高纯度但仍然能淹死人的信息的时候,我们一定相见恨晚,乐不思蜀,有朝闻道夕死可矣的感觉。这是一个可怕的极乐世界。
我们作为消费者在打一个注定失败的自虐之仗,试图抵制抵制不了的诱惑。说一点个人的应对体会,结束这个杞人早忧天的议论。这个体会也从朋友中得到印证过。
体会就是,有时候我们可以学林彪副统帅,不读书不看报,突然就掐了信息源和apps,专心做自己的事儿。一个月甚至半年过去,回头看,自己其实没有损失什么,而且完成了拖得很久的工作(其中包括如何去用语言技术提高信息质量诱惑别人的工作,不好意思,这颇滑稽,但无奈它是在下借以安身立命的天职)。
同行刘老师有同感,他是做事儿的人。我问他要不要加入群,咱们大伙儿聊聊NLP啥的。刘老师说,我这人经不起诱惑,曾经加入了n多群,一看话题有趣,就忍不住要看、要回应、要投入。结果是做不完手头的事儿。后来一横心,退了所有的群,就差把手机扔了。刘老师的做法也是一种自救。
其实我们最后还是要回到信息流中,再坚强的灵魂也不可能苦行僧一样长时期拒绝高品质信息以及消遣式信息享受。一味拒绝也自有其后果。意志力强的是在这两种状态中切换。更多的人意志力不够,就一步步淹没。退休了被淹没,也可算是福气。年轻人被淹没,这就是罪过,而恰恰是后者才是最 vulnerable 的群体。“忽视信息视而不见”乃是白领劳动者的生存技巧,但对于涉世未深的年轻人很难很难。据观察,在信息轰炸中淹没(info-addiction),其问题的严重性已经不亚于吸毒和酗酒,感觉与游戏的泛滥有一拼,虽然我没有统计数据。
因此,我想,人智可以缓行,我们没必要那么急把全世界的人生和时间都吞没,可以积点德或少点孽。同时,希望有越来越多的人研究如何帮助人抵制信息诱惑,抵抗沉沦。理想的世界是,我们既有召之即来的高质量信息,又有挥之即去的抵制工具在(类似戒毒program)。虽然后者的商业利益少,但却是拯救世界和人类的善举。
最可怕的是在下一代,可以看到他们的挣扎和无助。games、social media 和 internet 吞噬了无数青春。而世界基本是束手无策,任其沉沦。家长呢,只有干着急。我们自己都不能抵制诱惑,怎么能指望年青一代呢。充满 curiosity 和躁动的心灵,注定受到信息过载的奴役最深。其社会成本和代价似乎还没有得到应有的深入研究。
今天就扯到这儿,希望不是信息垃圾。
【相关】

Trap of Information Overdose

【置顶:立委NLP博文一览】

《朝华午拾》总目录

 

【李白之17:“我的人回来了, 可心还在路上”】

李:
圣诞醒来 心里突然冒出一句貌似开涮海龟海鸥的打油:

我的人回来了
可心还在路上

问一下: 谁回来了?“我”还是“我的人”?
谁的心在路上?“我”的心还是“我的人”的心?
或者 either?

我的身子虽然回家 可【我的】心还在路上
我的【心上】人虽然回家 可【他的】心还在路上

白:
人,是身的借喻,我的人=我的身。心=我的心
伟哥这个浪漫语境,更需要一个带优先机制的逻辑门了。

“我的人吃饱了,可细胞还饿着。”
“我的人吃饱了,但马还饿着。”

前者“人”指身体,后者“人”指将士。

开学了。孩子们人回到了学校,心还在假期。
某某,人虽然死了,思想还在影响着我们。

所以,“人”成为一个有坑名词,至少在述及灵肉分离的语境里,还是需要的。
是一个body-part,不知道这个义项知网收了没?

李:
除去“的”就不浪漫了 诗意了无:

我人回心没回

其实 身心分离 或 人心分裂 乃是诗歌、心理学和宗教哲学的永恒话题。形神合一反而难得,需要修为或运气。

白:
“人走了,香味还在房间里飘荡”

李:
那是仙女 不是人。
换成仙女 还可以说义项问题吗,更像是是临时延伸或活用。

白:
给“人“老老实实加个body-part义项是其一,语境驱动的逻辑门是其二,一个都不能少。在人与body-part对举的语境下,这个body-part义项被激活,可以颠覆标配。

李:
人 的默认(标配)是 body 而不是 heart,不说 body 就是 body,但 heart 永远不可省去:

仙女走了 香味还在
== 仙女 body 走了

某某同志与我们永别了
== 某某同志 body 永别了
他的精神长存
说明其灵魂没有走

滑稽的是:

某同志与我们永别了
但他的body永存
— 在水晶棺里
红场或什么场。
此 body 非彼 body

顺便一提,功过不论,留存 body 的做法感觉是违背伟人的意愿。为了留存 body,还要掏心掏肺 去除五脏 想想都让人悚然。

白:
楼歪了……

李:
歪了;也没歪。
“掏心掏肺” 语义计算要休眠唤醒,政治敏感性先放一边。
这种语义计算需要进入成语词典吗?一般认为不要进词典 但是不进入词典先休眠起来,想唤醒都没有依据。所谓词驱动休眠唤醒 就是把休眠的可能性预先隐藏在词典深处。细想起来这是一个巨大的虽然是可行的工程。起码 每个成语都要增加一个或数个字面意义的种子 。没有种子,context 再温床 另一种计算也难发芽。context 有 五脏 可以激发埋在种子里面的字面意义的心和肺,从而唤醒了 “掏” 的原义 而不是标配义。但这类激发不是驱动 驱动应该从成语本身发端 而不是从 context 发端。否则太发散 难以想象有实现唤醒的可行性。

 

 

【相关】

【李白对话录系列】

 

 

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

【研发心得:sentiment 的诡异】

我:
domain 很有意思,今天 study 客服的数据,发现我们现系统 tag “loyal customer” as positive,但实际上几乎总是 negative 抱怨。因为很多人(包括我自己,譬如跟电话公司打交道的时候)的抱怨总是这样开始:
I am your loyal customer for n years, blah blah [complaints: how can u treat me like that]

还有一些有意思的发现:new 这个词并不是啥褒义词,第一即便原义有一丝褒,但太弱,用得也太多。更重要的是,这个词最常出现在 promotion 的广告里面。客户情报里面很少用它表示褒义。那么 brand-new 呢?似乎稍微褒一些,但也很灰色,放过它可能更好。还有一个词 叫 available,以前以为是好话,其实用起来很 tricky:说 no Pepsi available, 不是说的 Pepsi 的坏话,很可能是说的好话 ,抱怨的是这么好的东西怎么没提供呢。诸如此类,不看 data 不知道。

白:
这么好的数据,不神经,糟蹋了

我:
大而言之,语义是泥坑;具体到 sentiment,几乎就是粪坑了。跳进去不仅可能被淹死,还会被呛死。

我等着神经在shentiment上来一个绝活。

目前为止,对手的 sentiment 全部用的学习,用没用 神经 就不知道了,但他们的 data quality 实在不敢恭维。

还有就是: I would kill for Pepsi,这样的说法是强烈的褒义:NND 为了 Pepsi 让我杀人都可以。这个倒不难逮住,无论是 pattern 去拿它,还是数据够了去训练出来它。

白:
可怜的米国人

我:
记得还有这么一句: Hell no man, pepsi or die. 这是对 pepsi 的极度褒扬:没 Pepsi 毋宁死(与自由同价: 不自由 毋宁死)宋柔:

宋:
@wei 可见,要做情感分析,对于所处理的语言必须有相当强的语感。国内的人做英语的情感分析恐怕不容易。

我:
昨天说了 sentiment data 的貌似诡异的事儿,明明说的是正面的词“loyal customer”,却几乎总是抱怨,至少是客服领域:原来人在表达情绪的时候,不仅会正话反说,而且还会先退一步。

今天再说一个案例:care about(关心) 一般认为是正面的动词,甚至关心钱财,从正面角度也可以说明这个企业懂得为 stake holders 创造价值,可能是一个兢兢业业的好企业吧。但是在客服领域,绝大多数场合,这却是抱怨的开始。

“All AT&T cares about is money. Worst service ever.”
AT&T is one of the most GREEDY companies I have ever saw. All they care about is the $$$$$.
“there aledged customer service is beyond ridiculous, they seem to care more about being paid than helping there customers”

等于说:你他妈只认钱,贪得无厌。

顺便一提,我也是 AT&T 很多年的“loyal”customer,完全认同上面的抱怨。看这家公司的账单,那真是跟天书一样,变着法儿跟你要钱。昨天来账单,我的电视涨了近30元,原来是 promotion 到期了,我就 call 他们,说,我们基本不看电视,时间和视屏都耗在网上了,不过是多年的习惯而已,你怎么一个月 charge 我 80 多刀,比互联网的 70 多刀的 charge 还大?你不给 discount,我就掐了电视算了。不过真要掐电视,领导怕不同意,如今的乐视盒子小米盒子啥的,节目虽多,还是不那么灵光,有时连不上。

结果客服说,现下没有新的 promotion 可以提供 discount,不过不久会有。说你要是电视少看,那就降一级吧,从 TV-family 降级为 TV-basic,那个才 19 块钱,来 20多个频道,你们也该够了。于是,我就降级了,然后一查看,说是TV 19 块,其实是 50 多块。什么 HD 费10快,receiver 费 15 块,录像费,等等等等。这种企业真该死。可是美国电信企业,好人不多,也就懒得挪动了。互联网+++ 再发达一些,这 cable TV 就该自生自灭了。

【相关】

舆情挖掘

【置顶:立委NLP博文一览】

《朝华午拾》总目录

【从IBM沃森平台的云服务谈AI热门中的热门 bots】

我:
哥仨老革命在去 IBM 的 traffic 中 去大名鼎鼎的沃森(Watson)系统探秘

洪:
讲者是这位印度籍女士:http://researcher.watson.ibm.com/researcher/view.php?person=us-vibha.sinha:

郭:
比较有意思的是她后面讲的三点:

1. LSTM based intent recognition and entity extraction

2. “tone” recognition
这里tone指的是从一句话(书面语)反应出的说话人的喜怒哀乐和处事方式等

3. personality recognition
主要基于心理学的分类,用200到2000条tweets训练


她重点强调的是,通过增加tone和personality的识别,人机对话可以有更高的可接受度。

我:
唐老师 诸位 汇报一下昨天的听闻。上面 郭老师也总结了几条,很好。我再说几点。
话说三位老革命慕名而去,这个 meet-up 一共才来了20几位听众吧 大概湾区此类活动甚多 marketing 不够的话 也难。据说北京的 AI 沙龙,弄个花哨一点的题目 往往门庭若市。

1. 没有什么 surprises 但参加沙龙的好处是可以问问题和可以听别人问问题,而主讲人常常在回答的时候给出一些书面没有的数据和细节。否则的话,各种资料都在网上(最后的 slide 给了链接),要写利人似的调研报告,只要不怕苦,有的是资料。

听讲的另一个好处是,主讲人事先已经组织好材料讲解,可以快速了解一个项目的概貌。

2. 特地替唐老师问了他钟情的 Prolog,问你们有用吗,在什么模块用。主讲人说,没有用。我说有报道说有用到。她说,她没听说,至少在她主讲的已经产品化的这个沃森 chatbot 的组建 toolkit 里面没有 Prolog。当然她不排除某个小组或个人 在沃森的某个项目或模块用到。IBM 对 AI 的投入增大,在沃森的名号下的各种研究项目和小组很多。

马:
我问过了IBM中国的,在沃森参加电视节目版本中没有用prolog,但是后续的版本中,确实用到了prolog

陈:
它是很多services构成,用不会奇怪,尤其是某些既有系统

我:

3. 现在不少巨头都在 offer 这样的 toolkit,问微软 offer 的 cortana 的 toolkit 与你们沃森的这套有啥不同。回答是,非常类似,不过她自认为沃森质量更好。亚马逊也有类似的 offer。

所以回来路上,我们就谈到这个 bots 遍地开花的场景。郭老师说,现如今谁要想做一个领域内的 bot,或自己的 app 做一个 bot 接口,根本就不需要编程。只要准备好领域的 experts,把数据准备好,用这些巨头的工具箱就可以构建一个出来。也一样可以 deploy 到 messenger 或嵌入其他场景,这几乎是一条龙的云服务。

当然 用这些服务是要交钱的,但主讲人说很便宜很便宜的,郭兄说,真用上了,其实也不便宜。便宜与否放一边,至少现如今,bots 的门槛很低,需要的不是软件人才,而是领域数据的人。于是,我看到一种前景,以前毕业即失业的语言学家、图书馆业人士,将来可能成为 AI 的主力,只有对数据和细节敏感的人,最终才是 AI 接口的血肉构筑者,反正架构是现成通用的。这个细想想是有道理的。这是沃森 API calls 的价格。

我:
这就回到我们以前议论过的话题。AI 创业,如果做平台或工具箱,初创公司怎么敌得过巨头呢?我觉得几乎是死路。

大而言之 做平台和工具箱创业的,历史上就没见过什么成功案例(不排除做了被收购那种,那也是“成功”,如果你的技术有幸被巨头看中:其实昨晚介绍的沃森系统的一个重要组件 AlchomyLanguage 就是收购的,洪爷知道收购的来路和细节)。

白:
麦当劳玩法,方便,质量可控,但绝非美食,虽然是“美”食。

我:
不错,这些巨头的 offerring 都是麦当劳式的流程。创业的空间,从工具角度,可以是中华料理的配方辅助工具之类。不过,还是那句话,最好绕过平台本身创业的思维,而是用巨头的工具或者自家建造匕首去做领域的 AI,这样的创业应该具有更大的空间和更多的可能性。

对于 NLP(AI之一种) 我写过 n 篇博文强调,所有的 offshelf 的平台和toolkit(譬如 历史悠久的GATE),甚至一个小插件(譬如 Brill Tagger or some Chinese word segmenter)都不好用。可以 prototyping 但如果稍微有点长期观点 要建一个大规模的NLP的应用,还是一切自家建造为好。当然,自家建造的门槛很高,多数人造不起,也没这个 architect 来指挥。但最终是,自家建造的胜出,从质量上说(质量包括速度、鲁棒性、精度广度、领域的可适应性等关键综合指标)。

巨头的工具箱的产品 offers 一开始也不赚钱,但他们的研发积累已经做了,且还在不断投入,不产品化成工具箱不是傻瓜吗,赚多少算多少。如果真到了AI bots 遍地开花的时候,他们凭借巨大的平台优势,赚钱也是可能的。小公司这条路没门吧。如果你的 offer 的确 unique,譬如是中华料理,譬如是伟哥的 parsing,你可能会吸引一批使用者。但想赚钱必须有规模,而 component tech 或平台工具之类,在小公司的环境中,是成不了规模的。所以不要想赚钱的事儿。

赚钱靠的是产品,而不是工具,这是AI创业铁律。

当然,通过平台或工具打出影响,做 marketing,曲线救国创业,另当别论。
回到 meet-up:

4. bots 构建的核心当然是 conversations 的训练工具。IBM沃森的工具用的是深度神经。

对于 bots,input 是确定的,就是用 bots 的人的输入。自然语言的语音也好 文字也好,语音反正也要转化为文字 所以我们面对的就是人机接口中的“人话”,理论上无止境 千变万化。

bots 的 output 呢?

在目前的框架里,在绝大多数实际场景,这个 output 都是以极为有限的集合
最典型的案例是为 apps(天气、股票、时间之类) 做 bots 作为 apps 的人机接口,
其 output 就是 app 里面的 commands 集合。于是 bot 产品定义为从无限到有限的映射,这是一个典型的分类场景。于是沃森提供这个深度学习为基础的工具帮助你训练你所需要的 classifiers,这是标准做法 无甚新意。

数据越多,分类质量越好。千变万化的死敌是稀疏数据。好在对于 bots,数据的收集会是一个边使用边加强的过程。如果你的 bots 开始有用户,你就形成了正循环,数据源源而来,你不断打磨、训练,这些都是可以 streamline 的流水作业,就越来越好。Siri 如此,Echo 也如此。

白:
分类本身是不带参数的,而bots的应对必须是带参数的,这是硬伤。
拿分类来做对话是看得到天花板的。

我:
I cannot agree more :=)

这里其实是有历史渊源的。IBM 做问答,一直是把问题简化为分类。18 年前我们在第一次 QA 竞赛(TREC-8)中交流 就是如此,这么多年这个核心做法一直不变。当时我们的QA成绩最好,得分66%,沃森的系统印象是40%左右,他们的组长就追在后面问,我们思路差不多呀,都是 question intents(我们叫 asking points,比多数 intents 其实更聚焦),外加 Named Entity 的support。我说我们还用到了语言结构啊。

直到今天他们仍然是没有句法分析,更甭提深度分析。他们当年的 QA 就是基于两点:
1. 问句分类:试图了解 intents;2. NE。有了这两条,通过 keywords 检索作为 context,在大数据中寻找答案,对于 factoid questions 是不难的(见【立委科普:问答系统的前生今世】)。这就是沃森打败人类的基本原理,一点也不奥秘,从来没有根本改变。现在这一套继续体现在其 bots 工具箱 offering 里面。

洪:

昨晚Watson讲座听,
今早广告已跟进。
IBM可真下本,
今天我试Bluemix云。

我:
2.  因此 conversations 训练,其核心就是两条:一个是 intents classification (这个 intents 是根据 output 的需求来定义的),一个 NE,不过 NE 是他们已经训练好的模块(NE有一定的domain独立性),用户只是做一些微调和增强而已。

顺便插一句,这几天一直在想,AI 现在的主打就是深度神经,所有的希望都寄托在神经上。但无论怎么神经,都不改 supervised learning 的本性:所以,我的问题是:你怎么克服缺乏带标大数据的知识瓶颈?

ok 你把机器翻译玩转了。因为 MT 有几乎无限的 “自然” 带标数据(其实也不是自然了,也是人工,幸运的是这些人力是历史的积累,是人类翻译活动的副产品,是不需要开发者花钱的 free ride)。可其他的 ai 和 nlp 应用呢,你还可以像 MT 这样幸运 这样享用免费午餐吗?

现在想,紧接着 MT 的具有大数据的热门应用是什么?非 bots 莫属。
对于 bots,数据已经有一定的积累了,其最大的特点在于,bots 的使用过程,数据就会源源而来。问题是 这些数据是对路的,real life data from the field,但还是不带标啊。所以,bots 的前景就是玩的跟数据打仗:可以雇佣人去没完没了地给数据做标注。这是一个很像卓别林的【摩登时代】的AI工厂的场景,或者是列宁同志攻打冬宫的人海战术。看上去很笨,但可以确定的是,bots 会越来越“智能”,应对的场景也越来越多。应了那句老话,有多少人工,就有多少智能。然而,这不是、也不应该是 唯一的克服知识瓶颈的做法。

毛:
嗯,有多少人工,就有多少智能。这话说得好。

我:
但这个景象成为常规 也不错 至少是帮助解决了一些白领就业。是用高级的专家知识去编写规则来提高系统质量,还是利用普罗标注去提高质量,从帮助就业和维稳角度看,几乎蛮力似的深度神经对于标注大数据的无休止的渴望和胃口,对于社会似乎更为有利。为了社会稳定和世界和平,我们该看好这种蛮力。我们做深度分析和理解的专家,试图尽可能逼真地去模拟人的智能过程,但对蛮力也应该起一份敬意。

将来的AI,什么人都可做:1. 你发现一个领域的 AI 需求; 2. 你雇佣一个对这个需求可以形式化定义的设计家; 3. 你调用巨头的一个通用的 AI 工具箱(譬如 TensorFlow) 或面向专项产品的工具箱(譬如 bot 的沃森工具箱) 4 你雇佣一批失业但受过教育的普罗,像富士康一样训练他们在流水线上去根据设计家的定义去标注数据、测试系统,你于是通过 AI 创造了价值,不排除你的产品会火。因为产品火不火已经不是技术了,而是你满足需求的产品角度。

3. 但是 正如白老师说的 这种用分类来简化问题的 AI 产品化,走不远。它可能满足一些特定领域的特定的需求 但是后劲不足是显然的。其中一个痛点或挑战就是,这种东西走不出三步,三步以上就抓瞎。如果你的应用可以在三步之内就基本满足需求,没问题。

bots 最显然的有利可图的应用场景是客服。一般而言,bots 取代和补充客服是大势所趋,因为客服的知识资源和记忆,根本没法与我们可以灌输给 bots 的知识来相比。利用知识去回答客户疑问,人不如机,是可以想见的。但是 观察一个好的客服与客户的交互 可以发现,三步的交流模型是远远无法满足稍微复杂一点的场景的。三步的说法是一个比喻,总之是目前的工具箱,对于较长时期的对话,还是束手无策。

bots 对用户话语的理解简化为 classification,以此为基础对用户的回答就不是那么简单了。目前提供的做法是:因为 intents 是有限的集合,是 classification 的结果,那么对于每一个 intent 可以预知答案(存在数据库的 hand-crafted text snippet)或回应(譬如展示一个图,譬如天气app的今日天气图表)。 这些预制的答案,听上去非常自然、生动甚至诙谐,它们都是领域专家的作品。且不说这些预制的 snippets,如何根据classification hierarchy 本身需要做不同组装,在存于数据库里面的核心应答的预制以外,还可以加上情感的维度,还可以加上 personalized 的维度,这些都可以使得对话更加人性化、自然化,但每加一个维度就意味着我们开始接近组装式策略的组合爆炸后果。三步、三维以上就无法收拾。

我问主讲人,你的这些预先制定好的应答片段,按照你的工具的组装方式,不就是一个 decision tree 吗?回答是,的确,就是一个 decision tree 的做法。然后她说,有不少研究想突破这种应答模式,但都是在探索,没有到可以产品化工具化的阶段。

郭老师说,谁要是有本事把人机的 “自然对话”能够延长到 20 分钟,换句话说 就是突破图灵测试,谁就是 AI bots 的真正破局者。如果你证明你能做到,巨头会抢着来高价收购你的。这是所有做 bots 的所面临的共同挑战。

据说小冰最高记录是与单一的人谈了九个小时的心。但那不是真正的突破,那是遇到了一个异常人类。正常的人,我的体会是两分钟定律,你与小冰谈话 超不过两分钟。我试过多次,到了两分钟,它所露出来的破绽就让你无法忍受,除非自己铁心要自我折磨。其实 工业界要求的连续对话,不是小冰这种闲扯。而是针对一个稍微复杂一点的任务场景(譬如订票)如何用自然对话的假象去把相关的信息收集全,来最大限度地满足客户需求。

累了,先笔记和评论如上。其余还有一些有趣的点儿可以讨论,以后再说。这是交给我们唐老师的作业。

郭:
Amazon’s $2.5M ‘Alexa Prize’ seeks chatbot that can converse intelligently for 20 minutes

洪:
亚马逊正设大奖,
chatbot赛悬赏。
对话若超廿分长,
两半米粒到手上。// 2.5M

【相关】

立委科普:问答系统的前生今世

Amazon’s $2.5M ‘Alexa Prize’ seeks chatbot that can converse intelligently for 20 minutes

微软小冰,两分钟定律

中文处理

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

【语义计算:精灵解语多奇智,不是冤家不上船】

白:
“他分分钟就可以教那些不讲道理的人做人的道理。”

我:

1016a

一路通,直到最后的滑铁卢。
定语从句谓语是“做人”而不是“可以教”,可是定语从句【【可以教。。。的】道理】与 vp定语【【做人的】道理】,这账人是怎么算的?

白:
还记得“那个小集合”吗?sb 教 sb sth,坑已经齐活儿了
“道理”是一般性的,定语是谓词的话一定要隐含全称陈述,不能是所有坑都有萝卜的。当然这也是软性的。只是在比较中不占优而已。单独使用不参与比较就没事:“张三打李四的道理你不懂”就可以,这时意味着“张三打李四背后的逻辑你不懂”。
“他分分钟就可以把一个活人打趴下的道理我实在是琢磨不透。”这似乎可以。

我:
教 至少两个 subcats:
教 sb sth
教 sb todo sth

白:
这个可以有
刚刚看到一个标题起:没有一滴雨会认为自己制造了洪灾。
这个句法关系分析的再清楚,也解释不了标题的语义。

宋:
有意思。

我:
教他
教他做人
教他道理
教他做人的道理
教他的道理
教他做人的往事儿

这个 “道理” 和 “往事”,是属于同一个集合的,我们以前讨论过的那个集合,不参与定语从句成分的 head n。

白:

我:
这个集合里面有子集 是关于 info 的,包括 道理 新闻 公告 往事。。。

白:
但是于“道理”而言,坑不满更显得有抽象度。是没“提取”,但坑不满更顺更优先,因为隐含了全称量词。

我:
就是说 这个集合里面还有 nuances 需要照顾。滑铁卢就在 “教他做人的往事儿” 上,照顾了它 就照顾不了 “做人的道理”。
就事论事 我可以词典化 “做人的道理”,后者有大数据的支持。

白:
这可是能产的语言现象。
试试这个:“你们懂不懂做人要低调的道理?”

我:
我试试 人在外 但电脑带了 只好拍照了

371656522530864097

你们懂不懂道理,这是主干
什么道理?
要低调的道理。
谁要低调?
你们。
懂什么类型的道理?
做人的道理。
谁做人?
你们。
小小的语义计算图谱 ,能回答这么多问题 ,这机器是不是有点牛叉?

白:
图上看,“要低调”是“懂道理”的状语而不是“道理”的定语?

我:
这个是对的,by design。但我们设计vn合成词的时候,我们要求把分离词合成起来。如果 n 带有定语,合成以后就指向 合成词整体。这时候 为了留下一些痕迹,有意在系统内部 保留定语的标签,以区别于其他的动词的状语修饰语。否则,“懂【要低调的】道理” 与 “【要低调的】懂道理”,就无法区分了。这样处理 语义落地有好处 完全是系统内部的对这种现象的约定和协调 system internal。定语 状语 都是修饰语 大类无异。

白:
“做人要低调”是一个整体,被拆解了。逻辑似乎不对。
拆解的问题还没解决:不管x是谁,如果x做人,x就要低调。
两个x是受全称量词管辖的同一个约束变元。
@宋 早上您似乎对“没有一滴雨会认为自己制造了洪灾”这个例子有话要说?

宋:
@白硕 主要是觉得这句话的意思有意思。从语义分析看应该不难,因为这是一种模式:没有NP V。即任何x,若x属于NP,则否定V(x)。

白:
首先这是一个隐喻,雨滴是不会“认为”如何如何的,既然这样用,就要提炼套路,看把雨滴代换成什么:雨滴和洪水的关系,是天上的部分和地上的整体的关系,是无害无责任的个体和有害有责任的整体的关系。

“美国网约车判决给北上广深的启示”

洪:
中土NLP全家福,
烟台开会倾巢出。
语言架桥机辅助,
兵强马壮数据足。

chinanlp
中国nlp全家福啊@wei

白: 哈
李白无暇混贵圈,一擎核弹一拨弦。精灵解语多奇智,不是冤家不上船。

洪:
冤家全都上贼船,李白有事别处赶。天宫迄今无甚关,Alien语言亟需练。

我:
白老师也没去啊 敢情。
黑压压一片 吾道不孤勒。

 

【相关】

【李白对话录:RNN 与语言学算法】

【李白对话录:如何学习和处置“打了一拳”】

【李白对话录:你波你的波,我粒我的粒】

【李白对话录- 从“把手”谈起】

【李白对话录:如何学习和处置“打了一拳”】 

【李白对话录之六:NLP 的Components 及其关系】

乔姆斯基批判

[转载]【白硕 – 穿越乔家大院寻找“毛毛虫”】

泥沙龙笔记:parsing 是引擎的核武器,再论NLP与搜索

泥沙龙笔记:从 sparse data 再论parsing乃是NLP应用的核武器

【立委科普:NLP核武器的奥秘】

【立委科普:语法结构树之美】

【立委科普:语法结构树之美(之二)】

中文处理

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

 

【Question answering of the past and present】

  1. A pre-existence

The traditional question answering (QA) system is an application of Artificial Intelligence (AI).  It is usually confined to a very narrow and specialized domain, which is basically made up of a hand-crafted knowledge base with a natural language interface. As the field is narrow, the vocabulary is very limited, and its pragmatic ambiguity can be effectively under control. Questions are highly predictable, or close to a closed set, the rules for the corresponding answers are fairly straightforward. Well-known projects in the 1960s include LUNAR, a QA system specializing in answering questions about the geological analysis on the lunar samples collected from the Apollo’s landing on the Moon.  SHRDLE is another famous QA expert system in AI history, it simulates the operation of a robot in the toy building world. The robot can answer the question of the geometric state of a toy and listen to the language instruction for its operation.

These early AI explorations seemed promising, revealing a fairy-tale world of scientific fantasy, greatly stimulating our curiosity and imagination. Nevertheless, in essence, these are just toy systems that are confined to the laboratory and are not of much practical value. As the field of artificial intelligence was getting narrower and narrower (although some expert systems have reached a practical level, majority AI work based on common sense and knowledge reasoning could not get out beyond lab), the corresponding QA systems failed to render meaningful results. There were some conversational systems (chatterbots) that had been developed thus far and became children’s popular online toys (I remember at one time when my daughter was young, she was very fond of surfing the Internet to find various chatbots, sometimes deliberately asking tricky questions for fun.  Recent years have seen a revival of this tradition by industrial giants, with some flavor seen in Siri, and greatly emphasized in Microsoft’s Little Ice).

2. Rebirth

Industrial open-domain QA systems are another story, it came into existence with the development of the Internet boom and the popularity of search engines. Specifically, the open QA system was born in 1999, when the TREC-8 (Eighth Text Retrieval Conference) decided to add a natural language QA track of competition, funded by the US Department of Defense’s DARPA program, administrated by the United States National Institute of Standards and Technology (NIST), thus giving birth to this emerging QA community.  Its opening remarks when calling for the participation of the competition are very impressive, to this effect:

Users have questions, they need answers. Search engines claim that they are doing information retrieval, yet the information is not an answer to their questions but links to thousands of possibly related files. Answers may or may not be in the returned documents. In any case, people are compelled to read the documents in order to find answers. A QA system in our vision is to solve this key problem of information need. For QA, the input is a natural language question, the output is the answer, it is that simple.

It seems of benefit to introduce some background for academia as well as the industry when the open QA was born.

From the academic point of view, the traditional sense of artificial intelligence is no longer popular, replaced by the large-scale corpus-based machine learning and statistical research. Linguistic rules still play a role in the field of natural language, but only as a complement to the mainstream machine learning. The so-called intelligent knowledge systems based purely on knowledge or common sense reasoning are largely put on hold by academic scholars (except for a few, such as Dr. Douglas Lenat with his Cyc). In the academic community before the birth of open-domain question and answering, there was a very important development, i.e. the birth and popularity of a new area called Information Extraction (IE), again a child of DARPA. The traditional natural language understanding (NLU) faces the entire language ocean, trying to analyze each sentence seeking a complete semantic representation of all its parts. IE is different, it is task-driven, aiming at only the defined target of information, leaving the rest aside.  For example, the IE template of a conference may be defined to fill in the information of the conference [name], [time], [location], [sponsors], [registration] and such. It is very similar to filling in the blank in a student’s reading comprehension test. The idea of task-driven semantics for IE shortens the distance between the language technology and practicality, allowing researchers to focus on optimizing tasks according to the tasks, rather than trying to swallow the language monster at one bite. By 1999, the IE community competitions had been held for seven annual sessions (MUC-7: Seventh Message Understanding Conference), the tasks of this area, approaches and the then limitations were all relatively clear. The most mature part of information extraction technology is the so-called Named Entity (NE tagging), including identification of names for human, location, and organization as well as tagging time, percentage, etc. The state-of-the-art systems, whether using machine learning or hand-crafted rules, reached a precision-recall combined score (F-measures) of 90+%, close to the quality of human performance. This first-of-its-kind technological advancement in a young field turned out to play a key role in the new generation of open-domain QA.

In industry, by 1999, search engines had grown rapidly with the popularity of the Internet, and search algorithms based on keyword matching and page ranking were quite mature. Unless there was a methodological revolution, the keyword search field seemed to almost have reached its limit. There was an increasing call for going beyond basic keyword search. Users were dissatisfied with search results in the form of links, and they needed more granular results, at least in paragraphs (snippets) instead of URLs, preferably in the form of direct short answers to the questions in mind.  Although the direct answer was a dream yet to come true waiting for the timing of open-domain QA era, the full-text search more and more frequently adopted paragraph retrieval instead of simple document URLs as a common practice in the industry, the search results changed from the simple links to web pages to the highlighting of the keywords in snippets.

In such a favorable environment in industry and academia, the open-domain question answering came onto the stage of history. NIST organized its first competition, requiring participating QA systems to provide the exact answer to each question, with a short answer of no more than 50 bytes in length and a long answer no more than 250 bytes. Here are the sample questions for the first QA track:

Who was the first American in space?
Where is the Taj Mahal?
In what year did Joe DiMaggio compile his 56-game hitting streak?

3. Short-lived prosperity

What are the results and significance of this first open domain QA competition? It should be said that the results are impressive, a milestone of significance in the QA history. The best systems (including ours) achieve more than 60% correct rate, that is, for every three questions, the system can search the given corpus and is able to return two correct answers. This is a very encouraging result as a first attempt at an open domain system. At the time of dot.com’s heyday, the IT industry was eager to move this latest research into information products and revolutionize the search. There were a lot of interesting stories after that (see my related blog post in Chinese: “the road to entrepreneurship”), eventually leading to the historical AI event of IBM Watson QA beating humans in Jeopardy.

The timing and everything prepared by then from the organizers, the search industry, and academia, have all contributed to the QA systems’ seemingly miraculous results. The NIST emphasizes well-formed natural language questions as appropriate input (i.e. English questions, see above), rather than traditional simple and short keyword queries.  These questions tend to be long, well suited for paragraph searches as a leverage. For competition’s sake, they have ensured that each question asked indeed has an answer in the given corpus. As a result, the text archive contains similar statements corresponding to the designed questions, having increased the odds of sentence matching in paragraph retrieval (Watson’s later practice shows that from the big data perspective, similar statements containing answers are bound to appear in text as long as a question is naturally long). Imagine if there are only one or two keywords, it will be extremely difficult to identify relevant paragraphs and statements that contain answers. Of course, finding the relevant paragraphs or statements is not sufficient for this task, but it effectively narrows the scope of the search, creating a good condition for pinpointing the short answers required.  At this time, the relatively mature technology of named entity tagging from the information extraction community kicked in.  In order to achieve the objectivity and consistency in administrating the QA competition, the organizers deliberately select only those questions which are relatively simple and straightforward, questions about names, time or location (so-called factoid questions).  This practice naturally agrees with the named entity task closely, making the first step into open domain QA a smooth process, returning very encouraging results as well as a shining prospect to the world. For example, for the question “In what year did Joe DiMaggio compile his 56-game hitting streak?”, the paragraph or sentence search could easily find text statements similar to the following: “Joe DiMaggio’s 56 game hitting streak was between May 15, 1941 and July 16”.  An NE system tags 1941 as time with no problem and the asking point for time in parsing the wh-phrase “in what year” is also not difficult to decode. Therefore, an exact answer to the exact question seems magically retrieved from the sea of documents to satisfy the user, like a needle found in the haystack. Following roughly the same approach, equipped with gigantic computing power for parallel processing of big data, 11 years later, IBM Watson QA beat humans in the Jeopardy live show in front of the nationwide TV audience, stimulating the entire nation’s imagination with awe for this technology advance.  From QA research perspective, the IBM’s victory in the show is, in fact, an expected natural outcome, more of an engineering scale-up showcase rather than research breakthrough as the basic approach of snippet + NE + asking-point has long been proven.

A retrospect shows that adequate QA systems for factoid questions are invariably combined with a solid Named Entity module and a question parser for identifying asking points.  As long as there is an IE-indexed big data behind, with information redundancy as its nature, factoid QA is a very tractable task .

4. State of the art

The year 1999 witnessed the academic community’s initial success of the first open-domain QA track as a new frontier of the retrieval world.  We also benefited from that event as a winner, having soon secured a venture capital injection of $10 million from the Wall Street. It was an exciting time shortly after AskJeeves’ initial success in presenting a natural language interface online (but they did not have the QA technology for handling the huge archive for retrieving exact answers automatically, instead they used human editors behind the scene to update the answers database).  A number of QA start-ups were funded.  We were all expecting to create a new era in the information revolution. Unfortunately, the good times are not long, the Internet bubble soon burst, and the IT industry fell into the abyss of depression.  Investors tightened their monetary operations, the QA heat soon declined to freezing point and almost disappeared from the industry (except for giants’ labs such as IBM Watson; in our case, we shifted from QA to mining online brand intelligence for enterprise clients). No one in the mainstream believes in this technology anymore. Compared with traditional keyword indexing and searching, the open domain QA  is not as robust and is yet to scale up to really big data for showing its power. The focus of the search industry is shifting from depth back to breadth, focusing on the indexing coverage, including the so-called deep web. As the development of QA systems is almost extinct from the industry, this emerging field stays deeply rooted in the academic community, developed into an important branch, with increasing natural language research from universities and research labs. IBM later solves the scale-up challenge, as a precursor of the current big data architectural breakthrough.

At the same time, scholars begin to summarize the various types of questions that challenge QA. A common classification is based on identifying the type of questions for their asking points.  Many of us still remember our high school language classes, where the teacher stressed the 6 WHs for reading comprehension: who / what / when / where / how / why. (Who did what when, where, how and why?)  Once answers to these questions are clear , the central stories of an article are in hands. As a simulation of human reading comprehension, the QA system is designed to answer these key WH questions as well. It is worth noting that these WH questions are of different difficulty levels, depending on the types of asking points (one major goal for question parsing is to identify the key need from a question, what we call asking point identification, usually based on question parsing of wh-phrases and other question clues). Those asking points corresponding to an entity as an appropriate answer, such as who / when / where, are relatively easy questions to answer (i.e. factoid questions). Another type of question is not simply answerable by an entity, such as what-is / how / why, there is consensus that answering such questions is a much more challenging task than factors questions.  A brief introduction to these three types of “tough” questions and their solutions are presented below as a showcase of the on-going state to conclude this overview of the QA journey.

What/who is X? This type of questions is the so-called definition question, such as What is iPad II? Who is Bill Clinton? This type of question is typically very short, after the wh-word and the stop word “is” are stripped in question parsing, what is left is just a name or a term as input to the QA system.  Such an input is detrimental to the traditional keyword retrieval system as it ends up with too many hits from which the system can only pick the documents with the most keyword density or page rank as returns.  But from QA perspective, the minimal requirement to answer this question is a definition statement in the forms of “X is a …”.  Since any entity or object is in multiple relationships with other entities and involved in various events as described in the corpus, a better answer to the definition question involves a summary of the entity with all the links to its key associated relations and events, giving a profile of the entity.  Such technology is in existence, and, in fact, has been partly deployed today. It is called knowledge graph, supported by underlying information extraction and fusion. The state-of-the-art solution for this type of questions is best illustrated in the Google deployment of its knowledge graph in handling queries of a short search for movie stars or other VIP.

The next challenge is how-questions, asking about a solution for solving a problem or doing something, e.g. How can we increase bone density? How to treat a heart attack?  This type of question calls for a summary of all types of solutions such as medicine, experts, procedures, or recipe.  A simple phrase is usually not a good answer and is bound to miss varieties of possible solutions to satisfy the information need of the users (often product designers, scientists or patent lawyers) who typically are in the stage of prior art research and literature review for a conceived solution in mind.  We have developed such a powerful system based on deep parsing and information extraction to answer open-domain how-questions comprehensively in the product called Illumin8, as deployed by Elsevier for quite some years.  (Powerful as it is, unfortunately, it did not end up as a commercial success in the market from revenue perspective.)

The third difficult question is why.  People ask why-questions to find the cause or motive of a phenomenon, whether an event or an opinion.  For example, why people like or dislike our product Xyz?  There might be thousands of different reasons behind a sentiment or opinion.   Some reasons are explicitly expressed (I love the new iPhone 7 because of its greatly enhanced camera) and more reasons are actually in some implicit expressions (just replaced my iPhone , it sucks in battery life).  An adequate QA system should be equipped with the ability to mine the corpus and summarize and rank the key reasons for the user.  In the last 5 years, we have developed a customer insight product that can answer why questions behind the public opinions and sentiments for any topics by mining the entire social media space.

Since I came to the Silicon Valley 9 years ago, I have been lucky, with pride, in having had a chance to design and develop QA systems for answering the widely acknowledged challenging questions.  Two products for answering the open-domain how questions and why-questions in addition to deep sentiment analysis have been developed and deployed to global customers.  Our deep parsing and IE platform is also equipped with the capability to construct deep knowledge graph to help answer definition questions, but unlike Google with its huge platform for the search needs, we have not identified a commercial opportunity to deploy that capability for a market yet.

This  piece of writing first appeared in 2011 in my personal blog, with only limited revisions since. Thanks to Google Translate at https://translate.google.com/ for providing a quick basis, which was post-edited by myself.  

 

[Related]

Http://en.wikipedia.org/wiki/Question_answering

The Anti-Eliza Effect, New Concept in AI

Knowledge map and open-domain QA (1)” (in Chinese)

knowledge map and how-question QA (2)”  (in Chinese)

Ask Jeeves and its million-dollar idea for human interface in 】(in Chinese)

Dr Li’s NLP Blog in English

 

【立委科普:谷歌NMT,见证奇迹的时刻】

微信最近疯传人工智能新进展:谷歌翻译实现重大突破!值得关注和庆贺。mt 几乎无限量的自然带标数据在新技术下,似乎开始发力。报道说:

十年前,我们发布了 Google Translate(谷歌翻译),这项服务背后的核心算法是基于短语的机器翻译(PBMT:Phrase-Based Machine Translation)。

自那时起,机器智能的快速发展已经给我们的语音识别和图像识别能力带来了巨大的提升,但改进机器翻译仍然是一个高难度的目标。

今天,我们宣布发布谷歌神经机器翻译(GNMT:Google Neural Machine Translation)系统,该系统使用了当前最先进的训练技术,能够实现到目前为止机器翻译质量的最大提升。我们的全部研究结果详情请参阅我们的论文《Google`s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation》。

几年前,我们开始使用循环神经网络(RNN:Recurrent Neural Networks)来直接学习一个输入序列(如一种语言的一个句子)到一个输出序列(另一种语言的同一个句子)的映射。其中基于短语的机器学习(PBMT)将输入句子分解成词和短语,然后在很大程度上对它们进行独立的翻译,而神经机器翻译(NMT)则将输入的整个句子视作翻译的基本单元。

这种方法的优点是:相比于之前的基于短语的翻译系统,这种方法所需的工程设计更少。当其首次被提出时,NMT 在中等规模的公共基准数据集上的准确度,就达到了可与基于短语的翻译系统媲美的程度。

自那以后,研究者已经提出了很多改进 NMT 的技术,其中包括模拟外部对准模型(external alignment model)来处理罕见词,使用注意(attention)来对准输入词和输出词 ,以及将词分解成更小的单元应对罕见词。尽管有这些进步,但 NMT 的速度和准确度还没能达到成为 Google Translate 这样的生产系统的要求。

我们的新论文描述了怎样克服让 NMT 在非常大型的数据集上工作的许多挑战、如何打造一个在速度和准确度上都足够能为谷歌 用户和服务带来更好的翻译体验的系统。

来自对比评估的数据,其中人类评估者对给定源句子的翻译质量进行比较评分。得分范围是 0 到 6,其中 0 表示「完全没有意义的翻译」,6 表示「完美的翻译」。

…………

使用人类评估的并排比较作为一项标准,GNMT 系统得出的翻译相比于之前基于短语的生产系统有了极大提升。

在双语人类评估者的帮助下,我们在来自维基百科和新闻网站的样本句子上测定发现:GNMT 在多个主要语言对的翻译中将翻译误差降低了 55%-85% 甚至更多。

今天除了发布这份研究论文之外,我们还宣布将 GNMT 投入到了一个非常困难的语言(汉语-英语)的翻译的生产中。

现在,移动版和网页版的 Google Translate 的汉英翻译已经在 100% 使用 GNMT 机器翻译了——每天大约 1800 万条翻译。GNMT 的生产部署是使用我们公开开放的机器学习工具套件 TensorFlow 和我们的张量处理单元(TPU:Tensor Processing Units),它们为部署这些强大的 GNMT 模型提供了足够的计算算力,同时也满足了 Google Translate 产品的严格的延迟要求。

汉语到英语的翻译是 Google Translate 所支持的超过 10000 种语言对中的一种,在未来几个月,我们还将继续将我们的 GNMT 扩展到远远更多的语言对上。

from 谷歌翻译实现重大突破

作为老机译,不能不被吸引。准备小试一下这最新版的谷歌神经翻译。
此前试过谷歌在线翻译,总体不如百度,可现如今说汉语mt已经很神经了:深度神经,接近人类。我有几百篇待译 正好一试,先试为快。期待谷歌的神译。

董:
@wei 但愿不致让你失望。我曾半开玩笑地说:规则机译是傻子,统计机译是疯子,现在我继续调侃:神经机译是“骗子”(我绝不是指研发者)。语言可不是猫脸或马克杯之类的,仅仅表面像不行,内容也要像!

我:现在是见证奇迹的时刻:

The automatic speech generation of this science blog of mine is attached here, it is amazingly clear and understandable. If you are an NLP student, you can listen to it as a lecture note from a seasoned NLPer (definitely clearer than if I were giving this lecture myself with my strong accent).   More amazingly, the original blog was in Chinese and I used the newest Google Translate claimed to be based on deep learning using sentence-based translation as well as character-based techniques.  My original blog in Chinese is here, you can compare:【立委科普:自然语言系统架构简说】。

董老师,您是知道我的背景和怀疑的。但是,面对这样的进步,这种远远超出我们当初入行的时候可以想象的极限的自动翻译质量和鲁棒性,我们不得不,不得不,不得不叹服。

董:
用他们的术语叫“less adequate,but more fluent”。机译已经历了三次paradigm shift,当人们发现无论如何它只能是一种很好的信息处理工具,而无法取代人类翻译时,那就选取代价较少的。

我:
无论如何,这个小小的测试让我这个老机译有点瞠目结舌。还没有从这种冲击回过味来。当然,赶巧我测试的是正规文体,谈的又是电脑和NLP话题,肯定在语料库的涵盖范围内,撞上枪眼了。可比起此前我使用的前神经时代的谷歌SMT和百度SMT,这个飞跃式突破还是让人惊异的。向我们的神经同行致敬。这是一批绝顶聪明的疯子。

毛老,这是我对谷歌最近的 claim 的一个反馈。上次对他们的 parser 嘲笑了一通,这次对他们用同样技术带来的MT的突破,表达一下深深的敬佩。这种 contrast 不是我神经了,或分裂了,而是在 parsing,他们苦于没有自然带标数据,巧妇难为无米之炊,因此无法跟符号逻辑派比试。可是 MT 不同,几乎有无穷无尽的自然带标数据(人的翻译从来没有终止过,留下的对照译文浩如烟海)。

毛: @wei 这就是说,你服了基于神经元的MT,改变了自己的见解和主张?

我: 我服了,但没真地改变。

毛: 怎么说?

我:
无论多少门户之见,基本事实总应该看清吧。听一下上面列出的他们的 SMT 翻译,其流利程度和对我原文的忠实程度,已经超过了一个普通的人做翻译。因为一个口译如果不懂我这一行,我如果拿着这个讲稿讲课,让这样的 average interpreter 做现场翻译,他是比不过机器的,无论信还是达。(翻译高手不论。)这一点不得不服。另一方面,我以前说的,神经再深度,我看不出它在今后几年内可以赶上我的深度 parser,这一点还没改变。尤其是应对不同 domains 和 genres 的能力,他们不可能做到。因为在自然世界里面,没有带标的句法树,有的只是线性句子。而迄今看到的突破都是有监督的深度学习(supervised learning),离开海量带标数据(labeled data)就抓瞎了。

毛: 我被你弄糊涂了。你究竟是说哪一派强哪?@wei 究竟谁是世界第0?

我: parsing 我是第0,谷歌赶不上。MT 谷歌有重大突破,估计符号逻辑派的 MT 的日子不好过。

毛: 我问的是,MT谁是第0,不管用什么方法。

我: 这不是说,MT 规则系统就没有活路了,但是总体而言,SMT(statistical MT)占据上风的 trends 还在增强。

云: THKS. 我来试试能不能翻译我写的公司白皮书?

我:
你要是加一点译后人工编辑的话,我估计会很好的。再不要傻傻地从头请人工做翻译了。翻译公司如果不使用 MT 做底,将会被淘汰,成本上看很难存活。

董:
学习上,初二是一个分水岭,学科数量明显增多,学习方法也有所改变,一些学生能及时调整适应变化,进步很快,由成绩中等上升为优秀。但也有一部分学生存在畏难情绪,将心思用在学习之外,成绩迅速下降,对学习失去兴趣,自暴自弃,从此一蹶不振,这样的同学到了初三往往很难有所突破,中考的失利难以避免。
Learning, the second is a watershed, the number of subjects increased significantly, learning methods have also changed, some students can adjust to adapt to changes in progress, progress quickly, from the middle to rise to outstanding. But there are some students there is fear of hard feelings, the mind used in the study, the rapid decline in performance, loss of interest in learning, self-abandonment, since the devastated, so the students often difficult to break through the third day,

毛: 这翻译没什么好呀?

我:
要的就是这句话 🙂 @毛,需要一个对比,才能回答你的问题。

毛: 那就拿你的出来比比?

我: 我早就不做 MT 了,我是逃兵。近20年前,我就转移到信息抽取 IE(以及sentiment)挖掘了,这方面我有底气,不怕比。

刘:转:谷歌新版翻译有多神?英文教授亲证后告诉你…

我: thanks 似乎评论得比较中肯。对于口语现在肯定还不行,它的训练集一直对口语 cover 的很不够。以前测过,有些常用的简单的口语表达也弄错。不知这次这方面加强多少。

董老师上面给的那段谷歌翻译的段落,毛老说翻译不咋样。不过我做过MT多年,知道达到这一步其实是有很大进步的。以前的汉译英的不可读,到现在读起来大体能听得懂。这里面有很多的进步值得留意。

刘: @wei 转一个: 话说大数据干的一些事属于人工智能操练(不能用“研究”这词了)的范畴吗,那本来不就是传统计算机系的人干的?动不动嘲笑每开掉一个语言学家就往前多走几步这眼界太浅了

马: 在数据充足的领域,这几年DL方法都突飞猛进,我认识的好几个以前对DL有偏见的都多少改变了看法。在IR领域DL还基本不见功效,但也在慢慢渗透中。

毛: 不认同“传统计算机系”这个话。计算机系应该跟着实践走,而不是反过来。

董:
NMT的关键是个“像”。于是出了一个有时不懂原文的人以为翻的很顺溜。没了信的翻译,不就成骗子了吗?如何知道自己的翻译是满拧了?这也是NMT的死穴。

马: 董老师,我觉得统计方法都有这个死穴

我:
寸有所长尺有所短,这也不奇怪。我今天自己听这个对我的blog的翻译已经三篇了,一步一叹。NND 咋这么溜呢。找碴子 找翻译错 总是有的。可是人也有译错啊。从可懂度和流畅程度看,我反正是服了。而这发生在没有亲属关系的两个语言之间。

董:
想当年有的领导干部对我说,“其实机器翻译只有百分之50的正确性,也不要紧,问题是你能不能把那个一半告诉我,我可以找人专翻那部分。”我回答说我做不到。从那时起我一直在关注这个问题。直到如今很多人在叫嚷要取代人工翻译了。这真有点像有了麦当劳就说不要法式大餐了一样。何况机译还做不到麦当劳。计算机、以致机译是上帝给人类玩的,上帝没有给人类那种可以复制自己的本领。

洪:

我的观点很简单:
影子不能三维变。
人若二维非压扁,
自叹弗如影子前。

人工智能影子般,
随人活动数据攒。
深度学习模型建,
类似皮影戏好玩。

董:
是的。我曾对照过10多本英国名著,曾经发现有一本译著明显的是译者故意大段大段地漏译的,那里面有太多的花草等,估计大师懒得查。就不译了。

为什么GNMT首先选择的语言对是汉英,而不是英汉?这是非常精明的。人工翻译即使错了或漏了,译文通常会是顺溜的,至少绝不会像传统的机译那样有傻又疯的,诘屈聱牙的,而这正是NMT的特点,它挑选的是译文中最大相似的。那样一来广大的英语读者,多数不懂中文,就容易被它“唬住”了。

我:
对。仔细看来,这次“突破”是达有余而信不足,矫枉过正了。
但一切才刚开始。我可以理解做NMT的人面对突破的欣喜心情。

洪:
伟爷久玩nlp,
一直孤傲头不低。
今朝服膺叹奇迹,
深度神经已皈依!

我:
皈依还不至于,也不够格。赞佩是由衷的,希望今后有合作的机会,取长补短,达成共赢。人家要是看不上咱呢,咱就单干。deep parsing 是 NLP 的皇冠。神经 parsing 何时全方位超过在下,咱就退休。现在仍然觉得,照这个标准,估计这辈子也退休不了。但愿自己错了,可以提早周游世界。

 

【相关】

Wei’s Introduction to NLP Architecture

谷歌翻译实现重大突破

谷歌新版翻译有多神?英文教授亲证后告诉你…

立委科普:NLP 联络图】(姐妹篇)

机器翻译

Wei’s Introduction to NLP Architecture Translated by Google

Introduction to NLP Architecture
by Dr. Wei Li
(fully automatically translated by Google Translate)

The automatic speech generation of this science blog of mine is attached here, it is amazingly clear and understandable, if you are an NLP student, you can listen to it as a lecture note from a seasoned NLPer (definitely clearer than if I were giving this lecture myself with my strong accent):

To preserve the original translation, nothing is edited below.  I will write another blog to post-edit it to make this an “official” NLP architecture introduction to the audiences perused and honored by myself, the original writer.  But for time being, it is completely unedited, thanks to the newly launched Google Translate service from Chinese into English at https://translate.google.com/ 

[Legislature science: natural language system architecture brief]

For the natural language processing (NLP) and its application, the system architecture is the core issue, I blog [the legislature of science: NLP contact diagram] which gave four NLP system architecture diagram, now one by one to be a brief .
I put the NLP system from the core engine to the application, is divided into four stages, corresponding to the four frame diagram. At the bottom of the core is deep parsing, is the natural language of the bottom-up layer of automatic analyzer, this work is the most difficult, but it is the vast majority of NLP system based technology.

160213sg5p2r8ro18v17z8

The purpose of parsing is to structure unstructured languages. The face of the ever-changing language, only structured, and patterns can be easily seized, the information we go to extract semantics to solve. This principle began to be the consensus of (linguistics) when Chomsky proposed the transition from superficial structure to deep structure after the linguistic revolution of 1957. A tree is not only the arcs that express syntactic relationships, but also the nodes of words or phrases that carry various information. Although the importance of the tree, but generally can not directly support the product, it is only the internal expression of the system, as a language analysis and understanding of the carrier and semantic landing for the application of the core support.

160216n8x8jj08qj2y1a8y

The next layer is the extraction layer (extraction), as shown above. Its input is the tree, the output is filled in the content of the templates, similar to fill in the form: is the information needed for the application, pre-defined a table out, so that the extraction system to fill in the blank, the statement related words or phrases caught out Sent to the table in the pre-defined columns (fields) to go. This layer has gone from the original domain-independent parser into the face-to-face, application-oriented and product-demanding tasks.
It is worth emphasizing that the extraction layer is domain-oriented semantic focus, while the previous analysis layer is domain-independent. Therefore, a good framework is to do a very thorough analysis of logic, in order to reduce the burden of extraction. In the depth analysis of the logical semantic structure to do the extraction, a rule is equivalent to the extraction of thousands of surface rules of language. This creates the conditions for the transfer of the domain.
There are two types of extraction, one is the traditional information extraction (IE), the extraction of fact or objective information: the relationship between entities, entities involved in different entities, such as events, can answer who dis what when and where When and where to do what) and the like. This extraction of objective information is the core technology and foundation of the knowledge graph which can not be renewed nowadays. After completion of IE, the next layer of information fusion (IF) can be used to construct the knowledge map. Another type of extraction is about subjective information, public opinion mining is based on this kind of extraction. What I have done over the past five years is this piece of fine line of public opinion to extract (not just praise classification, but also to explore the reasons behind the public opinion to provide the basis for decision-making). This is one of the hardest tasks in NLP, much more difficult than IE in objective information. Extracted information is usually stored in a database. This provides fragmentation information for the underlying excavation layer.
Many people confuse information extraction and text mining, but in fact this is two levels of the task. Extraction is the face of a language tree, from a sentence inside to find the information you want. The mining face is a corpus, or data source as a whole, from the language of the forest inside the excavation of statistical value information. In the information age, the biggest challenge we face is information overload, we have no way to exhaust the information ocean, therefore, must use the computer to dig out the information from the ocean of critical intelligence to meet different applications. Therefore, mining rely on natural statistics, there is no statistics, the information is still out of the chaos of the debris, there is a lot of redundancy, mining can integrate them.

160215hzp5hq5pfd1alldj

Many systems do not dig deep, but simply to express the information needs of the query as an entrance, real-time (real time) to extract the relevant information from the fragmentation of the database, the top n results simply combined, and then provide products and user. This is actually a mining, but is a way to achieve a simple search mining directly support the application.
In fact, in order to do a good job of mining, there are a lot of work to do, not only can improve the quality of existing information. Moreover, in-depth, you can also tap the hidden information, that is not explicitly expressed in the metadata information, such as the causal relationship between information found, or other statistical trends. This type of mining was first done in traditional data mining because the traditional mining was aimed at structural data such as transaction records, making it easy to mine implicit associations (eg, people who buy diapers often buy beer , The original is the father of the new people’s usual behavior, such information can be excavated to optimize the display and sale of goods). Nowadays, natural language is also structured to extract fragments of intelligence in the database, of course, can also do implicit association intelligence mining to enhance the value of intelligence.
The fourth architectural diagram is the NLP application layer. In this layer, analysis, extraction, mining out of the various information can support different NLP products and services. From the Q & A system to the dynamic mapping of the knowledge map (Google search search star has been able to see this application), from automatic polling to customer intelligence, from intelligent assistants to automatic digest and so on.

16221285l5wkx8t5ffi8a9

This is my overall understanding of the basic architecture of NLP. Based on nearly 20 years in the industry to do NLP product experience. 18 years ago, I was using a NLP structure diagram to the first venture to flicker, investors themselves told us that this is million dollar slide. Today’s explanation is to extend from that map to expand from.
Days unchanged Road is also unchanged.

Where previously mentioned the million-dollar slide story. Clinton said that during the reign of 2000, the United States to a great leap forward in Internet technology, known as. Com bubble, a time of hot money rolling, all kinds of Internet startups are sprang up. In such a situation, the boss decided to hot to find venture capital, told me to achieve our prototype of the language system to do an introduction. I then draw the following three-tier structure of a NLP system diagram, the bottom is the parser, from shallow to deep, the middle is built on parsing based on information extraction, the top of the main categories are several types of applications, including Q & A system. Connection applications and the following two language processing is the database, used to store the results of information extraction, these results can be applied at any time to provide information. This architecture has not changed much since I made it 15 years ago, although the details and icons have been rewritten no less than 100 times. The architecture diagram in this article is about one of the first 20 editions. Off the core engine (background), does not include the application (front). Saying that early in the morning by my boss sent to Wall Street angel investors, by noon to get his reply, said he was very interested. Less than two weeks, we got the first $ 1 million angel investment check. Investors say that this is a million dollar slide, which not only shows the threshold of technology, but also shows the great potential of the technology.

165325a3pamcdcdr3daapw

Pre – Knowledge Mapping: The Structure of Information Extraction Engine

【Related】
[Legislature science: NLP contact map (one)]
Pre – Knowledge Mapping: The Architecture of Information Extraction Engine
[Legislature science: natural language parsers is to reveal the mystery of the language LIGO-type detector]
【Essay contest: a dream come true
OVERVIEW OF NATURAL LANGUAGE PROCESSING

NLP White Paper: Overview of Our NLP Core Engine

White Paper of NLP Engine

“Zhaohua afternoon pick up” directory

[Top: Legislative Science Network blog NLP blog at a glance (regularly updated version)]

nmt1

nmt2

nmt3

nmt4

nmt5

nmt6

nmt7

retrieved 10/1/2016 from https://translate.google.com/

translated from http://blog.sciencenet.cn/blog-362400-981742.html

Not an ad. But a historical record.

Although not updated for long, this wiki remains like this until today 9/28/2016
from https://en.wikipedia.org/wiki/NetBase_Solutions,_Inc.

wikinetbase

NetBase Solutions, Inc.

From Wikipedia, the free encyclopedia
  (Redirected from NetBase)
NetBase Solutions, Inc.
Private
Industry Market Research
Founded 2004
Founder Jonathan Spier and Michael Osofsky
Headquarters Mountain View, CA, USA
Area served
Worldwide
Key people
Peter Caswell, CEO
Mark Bowles, CTO
Lisa Joy Rosner, CMO
Dr. Wei Li, Chief Scientist
Products NetBase Insight Workbench
Website www.netbase.com

NetBase Solutions, Inc. is a Mountain View, CA based developer of natural language processing technology used to analyze social media and other web content. It was founded by two engineers from Ariba in 2004 as Accelovation, before changing names to NetBase in 2008. It has raised a total of $21 million in funding. It’s sold primarily on a subscription basis to large companies to conduct market research and social media marketing analytics. NetBase has been used to evaluate the top reasons men wear stubble, the products Kraft should develop and the favorite tech company based on digital conversations.

History

NetBase was founded by Jonathan Spier and Michael Osofsky, both of whom were engineers at Ariba, in 2004 as Accelovation, based on the combination of the words “acceleration” and “innovation.”[1][2] It raised $3 million in funding in 2005, followed by another $4 million in 2007.[1][3] The company changed its name to NetBase in February 2008.[4][5]

It developed its analytics tools in March 2010 and began publishing monthly brand passion indexes (BPI) comparing brands in a market segment using the tool shortly afterwards.[6] In 2010 it raised $9 million in additional funding and another $2.5 million in debt financing.[1][3] NetBase Insight Workbench was released in March 2011 and a partnership was formed with SAP AG that December for SAP to resell NetBase’s software.[7] In April 2011, a new CEO Peter Caswell was appointed.[8] Former TIBCO co-inventor, patent author and CTO Mark Bowles is now the CTO at NetBase and held responsible for many technical achievements in scalability.[9]

Software and services

Screenshot of NetBase Insight Workbench dashboard

NetBase sells a tool called NetBase Insight Workbench that gives market researchers and social marketers a set of analytics, charts and research tools on a subscription basis. ConsumerBase is what the company calls the back-end that collects and analyzes the data. NetBase targets market research firms and social media marketing departments, primarily at large enterprises with a price-point of around $100,000.[10][11] NetBase is also white-labeled by Reed Elsevier in a product called illumin8.[12]

Uses

For the average NetBase user, 12 months of activity is twenty billion sound bytes from just over seven billion digital documents. The company claims to index 50,000 sentences a minute from sources like public-facing Facebook, blogs, forums, Twitter and consumer review sites.[13][14]

According to a story in InformationWeek, Kraft uses NetBase to measure customer needs and conduct market research for new product ideas.[15] In 2011 the company released a report based on 18 billion postings over twelve months on the most loved tech companies. Salesforce.com, Cisco Systems and Netflix were among the top three.[16] Also in 2011, NetBase found that the news of Osama Bin Laden eclipsed the royal wedding and the Japan earthquake in online activity.[17]

External links

References

  1. ^ Jump up to:a b c By Matt Marshall, VentureBeat. “Accelovation Raises $4M for online software for IT market research.” December 3, 2007.
  2. Jump up^ BusinessWeek profile
  3. ^ Jump up to:a b By Jon Xavier, BizJournals. “NetBase filters social media for what clients need to know.” June 3, 2011.
  4. Jump up^ By Barbara Quint, Information Today. “Elsevier and NetBase Launch illumin8.” February 28, 2008.
  5. Jump up^ The Economist. “Improving Innovation.” February 29, 2008.
  6. Jump up^ By Rachael King, BusinessWeek. “Most Loved — And Hated — Tech Companies.”
  7. Jump up^ Darrow, Barb (December 12, 2011). “SAP taps NetBase for deep social media analytics”. GigaOm. Retrieved May 8, 2012.
  8. Jump up^ San Jose Mercury News. “People on the Move.” May 15, 2011.
  9. Jump up^ By David F. Carr, InformationWeek. “How Much is your Brand Loved (or Hated)?” June 16, 2011.
  10. Jump up^ By Eric Schoenfeld, TechCrunch. “NetBase Offers Powerful Semantic Indexing Platform That Reads The Web.” April 22, 2009.
  11. Jump up^ By Jon Xavier, BizJournals. “NetBase filters social media for what clients need to know.” June 3, 2011.
  12. Jump up^ By Barbara Quint, Newsbreak. “Elsevier and NetBase Launch illumin8.” February 28, 2008.
  13. Jump up^ By Neil Glassman, Social Times. “What Every Social Media Marketer Should Know About NetBase.” August 24, 2010.
  14. Jump up^ By Ryan Flinn, BusinessWeek. “Wanted: Social Media Sifters.” October 21, 2010.
  15. Jump up^ By David F. Carr, InformationWeek. “How Kraft Foods Listens to Social Media.” June 30, 2011.
  16. Jump up^ By Ryan Flinn, Bloomberg. “Tech companies measure online sentiment.” May 19, 2011.
  17. Jump up^ By Geoffrey Fowler and Alexandra Berzon, Wall Street Journal. “Social Media Buzzes, Comes Into Its Own.” May 2, 2011.

【一日一parsing:走火入魔,parser 貌似发疯了】

我:
系统调试也上瘾。今夜无眠,调着调着,parser 貌似发疯了,大概是嫌我啥都喂给它,闹情绪了??

0927a
仔细瞅瞅,好像也没啥大错,没疯。与鲁爷【狂人日记】不同,我怀疑得没理。

自然语言的任何并列(Conj)结构,到了逻辑层,都必须分列。赶上遇到好几个并列就热闹了,关系有组合爆炸的趋向。都是汉语的顿号惹的祸。用恁多顿号做啥,多写几个小句能死吗?纯句法parsing不管这些,图面倒是显得干净。可是 deep parsing 的语义计算是逻辑的,就不能不管。

白:
“或”的结合能力弱于“与”,顿号在被“或”绑架不成情况下标配解释为“与”。

我:
这几天净出怪,不知是机器走火入魔了,还是玩机器的走火入魔,总之,出来一些奇奇怪怪的 graphs,远远不是教科书上展示的句法树形图给人留下的印象。教科书都是这样的,太过优雅

parse_tree_1

前两天出了一个葫芦形的图,昨天又出了双伞形的,今天是发飙,明天还不知会咋样。

这是昨天的两把伞。瞅了一瞅,好像也没错:

0926a

白:
吗的位置不对。两把伞那个,能……吗,才是一对。

我:
对,“吗“”应该更上一层楼。如果没有上一层,“吗”疑似就对了。为个小词爬楼不值当了,不是不可以爬 (patching). 当然这里面其实牵涉到决定 yes-no question 的所属问题,最终可能还是要上。

如果说 “电子签证是什么吗。”那就是活用。表面上用疑问,实际是应该是感叹?不是“吗”的标准用法。因为“吗”的本性是一般疑问句,而“什么”是特殊疑问句的疑问词(wh-word),不相谐。

白:
那个是“嘛”,不是“吗”

我:
肯定这里不可以用 “吗” 吗?

白:
他知道电子签证是什么

我:
感觉上可以,好像也不等同于“嘛”。

是那个什么吗。
真地忘了是那个什么了。

白:
你说的感叹义,应该用“嘛”。遗忘义,可以用“吗”
不过现在白字用的,早乱套了。

我:
这是前天的葫芦,白老师的名句。就是“与之”没挂上arg,差强人意,但总体逻辑语义的计算还都对。“你”(S)与“女人”(S)结了婚,而且这事儿修饰的(Mod-S:定语从句)是“女人”。

0925a

你说机器神不神,parser 好玩不好玩,这算不算对人类语言的机器理解的敲门砖:芝麻开门!芝麻芝麻快开门。

 

【相关】

【立委科普:语法结构树之美】

【立委科普:语法结构树之美(之二)】

中文处理

Parsing

【置顶:立委NLP博文一览】

《朝华午拾》总目录

 

Chart Parsing Chinese Character Strings

W. Li. 1997. Chart Parsing Chinese Character Strings. In
Proceedings of the Ninth North American Conference on Chinese
Linguistics (NACCL-9). Victoria, Canada.

Chart Parsing Chinese Character Strings [1]

 

Wei  LI

Simon Fraser University
Burnaby B.C. V5A 1S6 CANADA (lio@sfu.ca) 

 

ABSTRACT

This paper examines problems in word identification for a Chinese natural language processing system and presents our solution to these problems. In conventional systems, written Chinese parsing takes two steps: (1) a segmentation preprocessor for word identification (segmenter); (2) a grammar parsing the string of identified words. Morphological analysis, when required, as in the case of productive word formation, has to be incorporated in the segmenter. This matches the conventional morphology-before-syntax architecture. We will demonstrate the theoretical defect of this architecture when applied to Chinese. This leads to the conclusion that segmentational approach, despite its being the mainstream in Chinese computational morphology, is in general not adequate for the task of Chinese word identification. To solve this problem, a full grammar should be made available. Therefore, we take an alternative one-step approach. We have implemented an integrated grammar of morphology and syntax for directly parsing a string of Chinese characters, building both morphological and syntactic structures. Compared with the conventional two-step approach, our strategy has advantages in resolving ambiguity in word identification and in handling productive word formation.

  1. Introduction

A written Chinese sentence is a string of characters with no blanks to mark word boundaries. In conventional systems, Chinese parsing takes two steps as shown in the following Figure 1: (1) a segmentation preprocessor (called segmenter) for word identification; (2) a word based parsing grammar, building syntactic structures (Feng 1996, Chen & Liu (1992).

hpsg4

 

In contrast, we take an alternative one-step approach, as shown in Figure 2 below. We have implemented a grammar named W‑CPSG (for Wei’s Chinese Phrase Structure Grammar). W‑CPSG integrates morphology and syntax for character based parsing, building both morphological and syntactic structures.

hpsg5

In the two-step architecture, the purpose for the segmenter is to properly identify a string of words to feed syntax. This is not an easy task due to the possible involvement of the segmentation ambiguity. For example, given a string of 4 Chinese characters 研究生命, the segmentation ambiguity is shown in (1.a) and (1.b) below.

(1.)  研究生命

(a)        研究生               |
graduate student         | life or destiny

(b)        研究    | 生命
study   | life

The resolution of the above ambiguity in the segmenter is a hopeless job because such ambiguity is syntactically conditioned. For sentences like 研究生命金贵 (life for graduate students is precious), (1.a) is the right identification. For the phrase 研究生命起源 (to study the origin of life), (1.b) is right. So far there are no segmenters which can handle this properly and guarantee right word segmentation (Feng 1996). In fact, there can never be such segmenters as long as a grammar is not brought in. This is a theoretical defect of all Chinese analysis systems in the conventional architecture. We have solved this problem in our morphology-syntax integrated W‑CPSG. Word identification in our design becomes a by-product of parsing instead of a pre-condition for parsing.

In the text below, Section 2 investigates problems with the conventional two-step approach. In Section 3, we will present W‑CPSG one-step approach and demonstrate how W‑CPSG parsing solves these problems. The following is a list for abbreviations used in this paper.

A (Adjective); AF (Affix); BM (Bound Morpheme);
CLA (Classifier); CLAP (Classifier Phrase);
DE (Chinese particle introducing a modifier of noun); DEP (DE Phrase);
DE3 (Chinese particle introducing a modifier of result or capability);
DET (Determiner); LE (Chinese perfective aspect marker);
N (Noun); NP (Noun Phrase); P (Preposition); PP (Prepositional Phrase);
S (Sentence); V (Verb); VP (Verb Phrase); Vt (Transitive Verb)

  1. Problems Challenging Segmenters

In general, there are two basic problems for segmenters, namely, segmentation ambiguity and productive word formation.

2.1. segmentation ambiguity

This sub-section studies the segmentation ambiguity for Chinese word identification. We indicate that this ambiguity is structural in nature. Therefore it should be captured by structural trees via parsing. We conclude that a parsing grammar is indispensable in the resolution of the segmentation ambiguity.

Behind all segmenters are procedure based segmentation algorithms. Most proposals are some modified versions of large-lexicon based matching algorithms. As an underlying hypothesis, a longer match overrides a shorter match, hence the name maximum match. Decided by the  direction of the  procedure, i.e. whether  the segmentation proceeds from left (the beginning of a string) to right (the end of the string) or from right to left, we have two general types of maximum match: (1) FMM (Forward Maximum Match) algorithm; (2) BMM (Backward Maximum Match) algorithm (Feng 1996).

According to Liang 1987, segmenters have trouble with cases involving the segmentation ambiguity. There are two types of segmentation ambiguity: the cross ambiguity (AB|C vs. A|BC) and the embedded ambiguity (AB vs. A|B).

To detect possible ambiguity, many researchers use the technique of combining the FMM algorithm and the BMM algorithm. When the output of FMM and BMM are different, there must be some ambiguity involved. The following table lists the cases associated with the FMM and BMM combined approach.[2]

hpsg6

The following 3 examples all contain a cross ambiguity sub-string 研究生命 with 2 segmentation possibilities: 研究生|命 and 研究|生命. Example (4.) is a genuinely ambiguous case. Genuinely ambiguous sentences cannot be disambiguated within the sentence boundary, rendering multiple readings.

(2.) case 1:      研究生命金贵。

(a)        研究生                |      | 金贵                  (FMM: correct)
graduate student         | life   | precious
Life for graduate students is precious.

(b) * 研究 | 生命    |起源                                   (BMM: incorrect)
study        | life     | precious

(3.) case 2:       研究生命起源。

(a) *     研究生              | 命     | 起源                       (FMM: incorrect)
graduate-student       | life   | origin

(b)        研究     | 生命    | 起源                                (BMM: correct)
study   | life     | origin
to study the origin of life

(4.) case 3:       研究生命不好。

(a)        研究生                   | 命                 |        |      (FMM: correct)
graduate student         | destiny        | not     | good
The destiny of graduate students is not good.

(b) 研究 | 生命   | 不      | 好                                      (BMM: correct)
study    | life     |  not    | good
It is not good to study life.

The following example is a complicated case of cross ambiguity, involving more than 2 ways of segmentation. Both the FMM segmentation 出现|在世|界 and the BMM segmentation 出|现在|世界 are wrong. A third segmentation 出现||世界 is right.

(5.)  case 4:      出现在世界东方。

(a) * 出现 | 在世          |      | 东方                       (FMM: incorrect)
appear     | be-alive   | BM   | east

(b) * 出  | 现在  | 世界    | 东方                               (BMM: incorrect)
out        | now   | world | east

(c)  出现  |     | 世界     | 东方                               (correct)
appear    | at    | world  | east
to appear in the east of the world

In the following examples (6.) through (8.), ¿¾°×Êí involves embedded ambiguity. As separate words, the verb ¿¾ (bake) and the NP °×Êí (sweet potato) form a VP. As a whole, it is a compound noun ¿¾°×Êí (baked sweet potato). In cases of the embedded ambiguity, FMM and BMM always make the same segmentation, namely AB instead of A|B. It may be the only right choice, as seen in (6.). It may be wrong as shown in (7.). It may only be half right, as in the case of genuine ambiguity shown in (8.).

(6.) case 5:       他吃烤白薯。

(a)        他       |       | 烤白薯                                 (FMM&BMM: correct)
he       | eat     | baked sweet potato
He eats baked sweet potatoes.

(b) *     他       |       |       | 白薯                        (incorrect)
he       | eat     | bake | sweet potato

(7.) case 6:       他会烤白薯。

(a) *     他       |       | 烤白薯                                 (FMM&BMM: incorrect)
he       | can    | baked sweet potato

(b)        他      |       |       | 白薯                         (correct)
he      | can   | bake | sweet potato
He can bake sweet potatoes.

(8.) case 7:       他喜欢烤白薯。

(a)       他       | 喜欢 | 烤白薯                                  (FMM&BMM: correct)
he      | like  | baked sweet potato
He likes baked sweet potatoes.

(b)        他       | 喜欢   |       | 白薯                       (correct)
he      | like     | bake | sweet potato
He likes baking sweet potatoes.

Compare the above examples, we see that there are severe limitations for the FMM-BMM combined approach. First, it only serves the purpose of ambiguity detection (when the results of FMM and BMM do not match), and contributes nothing to its resolution. It has no way to tell which segmentation is right (compare case 1 and case 2), and, worse still, whether both are right (case 3) or wrong (case 4). Second, even when the results of FMM and BMM do match, it by no means guarantees right segmentation (case 6). Third, as far as detection is concerned, it is only limited to the problems for the cross ambiguity. The existence of the embedded ambiguity defines a blind area for this way of detection (case 6 and case 7). This is because the underlying maximum match hypothesis assumed in the FMM and BMM segmentation algorithms is directly contradictory to the phenomena of the embedded ambiguity.

In face of ambiguity, how do people judge which segmentation is right in the first place? It really depends on whether we can understand the sentence or phrase based on the segmentation. In computational linguistics, this is equivalent to whether the segmented string can be parsed by a grammar. The segmentation ambiguity is one type of structural ambiguity, not in essence different from typical structural ambiguity like, say, PP attachment ambiguity. In fact, PP attachment problem is a counterpart of the cross ambiguity in English syntax, as shown below.

(9.)       Cross ambiguity in PP attachment: V NP PP

(a) [V NP] [PP]
(b) [V] [NP PP]

Therefore, like English PP attachment, Chinese word segmentation ambiguity should also be captured by a parsing grammar. A parser resolves the ambiguity if it can, or detects the ambiguity in the form of multiple parses when it cannot. As shall be demonstrated in Section 3, wrong segmentation will not lead to a parse. Right segmentation results in at least one successful parse. In any case, at least a parser (hence a grammar on which the parser is based) is required for proper word identification.

The important thing is that the ambiguity in word identification is a grammatical problem. The attempt to solve this problem without a grammar is bound to be crippled. Since traditional segmentation algorithms are non-grammatical in nature, they are theoretically not equipped for handling such ambiguity. A successive model of segmentater-before-grammar attempts to do what it is not yet able to do. This is the theoretical defect for almost all existing segmentation approaches.

(10.)     Conclusion for 2.1.

The segmentation ambiguity in word identification is one type of structural ambiguity. In order to solve this problem, a parsing grammar is indispensable.

2.2. productive word formation

Unless morphological analysis is incorporated, lexicon match based segmenters will have trouble with new words produced by Chinese productive word formation, including reduplication, derivation and the formation of proper names. When the morphology component is incorporated in the segmenter, the two-step design becomes a variant of the conventional morphology-before-syntax architecture. But this architecture is not effective when the segmentation ambiguity is at issue.

In the following, we investigate reduplication, derivation and proper names one by one. In each case, we find that there is always a possible involvement of the segmentation ambiguity. This problem cannot be solved by a morphology component independent of syntax. We therefore propose a  grammar incorporating both morphology and syntax.

2.2.1. reduplication

Reduplication in Chinese serves various grammatical and/or lexical functions. Not all reduplications pose challenges to segmentation algorithms. Assume that a word consists of 2 characters AB, reduplication of the type AB –> ABAB is no problem. What becomes a problem for word segmentation is the reduplication of the type AB –> AABB or its variants like AB –> AAB. For example, a two-morpheme verb with verb-object relation at the level of morphology has the following way of reduplication.

(11.) Verb Reduplication: AB –> AAB  (for diminutive use)

分心 (get distracted) –> 分分心 (get distracted a bit)

让他分分心。

让       | 他     | 分分心
let       | he    | get distracted a bit
Let him relax a while.

It seems that reduplication is a simple process which can be handled by incorporating some procedure-based function calls in the segmentation algorithm. If a 3-character string, say 分分心, cannot be found in the lexicon, the reduplication procedure will check whether the first 2 characters are the same, and if yes, delete one of them and consult the lexicon again. But, such expansion of the segmentation algorithm is powerless when the segmentation ambiguity is involved. For example, it is wrong to regard 分分心 as of reduplication in the following sentence.

(12.)   这件事十分分心。

(a) *     这       |      |         |       | 分分心
this      | CLA  | thing  | ten    | get distracted a bit

(b)        这       |       |         | 十分    | 分心
this      | CLA  | thing  | very   | distracting
This thing is very distracting.

2.2.2. derivation

In Contemporary Mandarin, there have come to be a few morphemes functioning similarly to English affixes, e.g. 可 (-able) turns a transitive verb into an adjective.

(13.)     可 (-able) + Vt –> A

可 (-able) + 读 (Vt: read) –>   可读 (A:readable)

这本书非常可读。

这       | 本     | 书       | 非常   | 可读
this    | CLA  | book  | very  | readable
This book is very readable.

The suffix 性 works just like ‘-ness’,  changing an adjective into an abstract noun.  The derived noun 可读性 (readability) in the following example, similar to its English counterpart, involves a process of double affixation.

(14.)     A + 性 (-ness)  –> N
可 (-able) + 读 (Vt: read) –>   可读 (A:readable)
可读 (A:readable) + 性 (-ness) –> 可读性 (N:readability)

这本书的可读性

这       | 本      | 书       |      | 可读性
this    | CLA  | book  | DE    | readability
this book’s readability

The suffix Í· can change a transitive verb into an abstract noun adding to it the meaning “worth-of”.

(15.) Vt + 头 (AF:worth of) –> N

吃 (Vt:eat) + 头 (AF:worth of) –> 吃头 (N:worth of eating)

这道菜没有吃头

这       | 道     | 菜      | 没有             | 吃头
this    | CLA  | dish  | not-have    | worth-of-eating
This dish is not worth eating.

It is not difficult to incorporate in the segmenter these derivation rules for the morphological analysis. But, as in the case of reduplication, there is always a danger of wrongly applying the rules due to possible ambiguity involved. For example, 吃头 is a sub-string of embedded ambiguity. It can be both a derived noun ‘worth of eating’ or two separate words as seen in the following example.

(16.)  他饿得能吃头牛。

(a) *     他      | 饿             |       |      | 吃头·                       |
             he     | hungry    | DE3  | can  | worth-of-eating   | ox

(b)        他      | 饿              |      |      |       |       |
              he     | hungry    | DE3  | can  | eat    | CLA  | ox
He is so hungry that he can eat an ox.

2.2.3. proper name

Proper names are of 2 major types: (1) Chinese names; (2) transliterated foreign names. In this paper, we only target the identification of Chinese names and leave the problem of transliterated foreign names for further research (Li, 1997b).

A Chinese human name usually consists of a family name followed by a given name. Chinese family names form a clear-cut closed set. A given name is usually either one character or two characters. For example, the late Chinese chairman 毛泽东 (Mao Zedong) used to have another name 李得胜 (Li Desheng). In the lexicon, 李 is a registered family name. Both 得胜 and 胜 mean ‘win’. This may lead to 3 ways of word segmentation: (1) 李得胜; (2) 李|得胜; (3) 李得|胜, as seen in the following examples.

(17.)    李得胜了

(a)  李    | 得胜 | .
       Li    | win  | LE
Li won.

(b)   李得   |      |
        Li De | win  | LE
Li De won.

(c) *  李得胜          | .
          Li Desheng | LE

(18.)   李得胜胜了 。

(a) *  李 | 得胜 |     | .
         Li  | win | win | LE

(b) *  李得   |      |      |
          Li De | win  | win  | LE

(c)   李得胜            |      |
Li Desheng   | win  | LE
Li Desheng won.

Since the given name like µÃʤ is an arbitrary string of 1 or 2 characters, the morphological analysis of the full name should start with family name which can optionally combine with any 1 or 2 characters to form candidate proper names Àî, ÀîµÃ and ÀîµÃʤ. In other words, family name serves as the left boundary of a full name and the length is used to determine candidates. The right segmentation can only be made via sentence analysis as shown in the above examples.

Most Chinese place proper names are made of 1 to 3 characters, for example, 武汉市(Wuhu City), 南陵县 (Nanling County). The arbitrariness of these names makes any sub-strings of n characters (0<n<4) in the sentence a suspect. Fortunately, in most cases we may find boundary indicators of these names, like 省 (province), 市 (city), 县 (county), etc. Once the boundary indicator is located, the similar technique in using Chinese family name to identify the given name can be applied to select candidates of place proper names for verification through grammatical analysis.

In general, there is always a possibility of ambiguity involvement in the formation of all types of proper names.

(19.)     Conclusion for 2.2.

Due to the possible involvement of ambiguity, a parsing grammar for morphological analysis as well as for sentence analysis is required for the proper identification of the words produced by Chinese productive word formation.

  1. W‑CPSG Grammatical Approach

This section presents W‑CPSG approach to Chinese word identification and morphological analysis. We will demonstrate how a parser based on W‑CPSG solves the problems of the word identification ambiguity and productive word formation.

3.1. rationale of W‑CPSG approach

There have been a number of word identification algorithms based on both morphological and syntactic information (see survey in Feng 1996 and Sun & Huang 1996). Most such approaches do not use a self-contained grammar to parse the complete sentence. They are confined to the conventional two-step process of the segmentation-before-grammar design. As long as the word identification procedure is independent of a parsing grammar, it is extremely difficult to make full use of grammatical information to resolve ambiguity in word identification. Careful tuning up and sophisticated design improves the precision but will not change the theoretical defect of all such approaches. Chen & Liu acknowledges the limitation of their approach due to the lack of a grammar.  “However”, they say,  “it is almost impossible to apply real world knowledge nor to check the grammatical validity at this stage”. (Chen & Liu 1992, p.105) Why impossible at this stage? Because these segmentation systems are based on the concept of  two-step architecture and the grammar is not yet available! As we have demonstrated, the final judgment for proper word identification can hardly be made until the whole sentence is parsed, hence the requirement of a full grammar. Therefore, we are forced to make a compromise in involving how much of grammatical information depending on how much word identification precision we can afford to sacrifice. Needless to say, there is significant double-labor between such a word segmentation procedure and the following stage of parsing. As more and more grammatical information is used to achieve better precision, the overhead of this double labor becomes more serious. We consider the double labor as one strong argument against the two-step approach. If enough grammatical information is incorporated, it is essentially equivalent to a grammar. And  the segmenter will be equivalent to a parser.  Then why two grammars, one for word identification, and one for sentence parsing? Why not combine them? That is exactly what we are proposing in W‑CPSG – one-step approach based on an integrated grammar,  eliminating the necessity of a segmentation preprocessor.

3.2. W‑CPSG character-based parsing

W‑CPSG (Li. 1997a, 1997b) is a lexicalized Chinese unification grammar. The work on W‑CPSG is taken in the spirit of the modern linguistic theory Head-driven Phrase Structure Grammar (Pollard & Sag 1994). W‑CPSG consists of two parts: a minimized general grammar and an enriched lexicon. The general grammar only contains a handful of PS (phrase structure) rules, covering complement structure, modifier structure, conjunctive structure and morphological structure. This is the nature of lexicalized grammars. PS rules in such grammars are very abstract. Essentially, all they say is one thing, that is, 2 signs can combine so long as the lexicon so indicates. The lexicon houses lexical entries with their linguistic description in feature structures. Potential morphological structures as well as potential syntactic structures are lexically encoded. In syntax, a word expects another sign to form a phrase. In morphology, a morpheme expects another sign to form a word. For example, the prefix 可 (-able) expects a transitive verb to form an adjective. The morphological PS rule will build the morphological structure when a transitive verb does appear after the prefix 可 (-able) in the input string.

We now illustrate how W‑CPSG parses a string of Chinese characters by a sample parsing chart. The prototype of W‑CPSG was written in ALE, a grammar compiler developed on top of Prolog by Carpenter & Penn (1994). ALE compiles W‑CPSG into a Chinese parser, a Prolog program ready to accept a string of characters for analysis. W‑CPSG parse tree embodies both morphological analysis and syntactic analysis, as shown below.

hpsg12

 

This is so-called bottom-up parsing. It starts with lexicon look-up. Edges 1 through 7 are lexical edges. Other edges are phrasal edges. Each edge represents a sign, i.e. a character (morpheme), a word, a phrase or a sentence. Lexical edges result from a successful match between the signs in the input string and the entries in the lexicon during lexicon look-up. For example, 可 (-able), 读 (read) and 性 (-ness) are all registered entries in the lexicon, so they get matched and shown by edge 5, edge 6 and edge 7. Words produced by productive word formation present themselves as phrasal edges, e.g. edge ((5+6)+7) for 可读性 (readability). For the sake of concise illustration, we only show two pieces of information for the signs in the chart, namely category and interpretation with a delimiting colon (lexical edges are only labeled for either category or interpretation). The parser attempts to combine the signs according to PS rules in the grammar until parses are found. A parse is an edge which ranges over the whole string. The parse ((((1+2)+3)+4)+((5+6)+7)) for (20.) represents a binary structural tree based on the W‑CPSG analysis, as shown below.

hpsg13

3.3. ambiguity resolution in word identification

Given the resources of a phrase structure grammar like W‑CPSG, a parser based on standard chart parsing algorithms can handle both the cross ambiguity and the embedded ambiguity provided that a match algorithm based on exhaustive lookup instead of maximum match is adopted for lexicon lookup. All candidate words in the input string are presented to the parser for judgment. Ambiguous segmentation becomes a natural part of parsing: different ways of segmentation add different edges, a successful parse always embodies right identification. In other words, word identification in our design becomes a by-product of parsing instead of a pre-condition for parsing. The following example of the complicated cross ambiguity illustrates how the W‑CPSG parser resolves ambiguity. As seen, both the FMM segmentation (represented by the edge sequence 8-9-5-10) and the BMM segmentation (represented by 1-11-12-10) are in the chart as a result of exhaustive lexicon lookup. They are proved to be wrong because they do not lead to a successful parse according to the grammar. As a by-product, the final parse (8+(3+(12+10))) automatically embodies rightly identified word sequence 8-3-12-10, i.e. 出现  (appear) |在  (at) |世界 (world) |东方 (east).

hpsg10

 

Exhaustive lookup also makes an embedded ambiguity sub-string like 烤红薯 no longer a blind area for word identification, as shown in (22.) below. All the candidate words in the sub-string including 烤 (bake), 红薯 (sweet potato), 烤红薯 (baked sweet potato) are added to the chart as lexical edges (edge 4, edge 8 and edge 10). This is a case of genuine ambiguity, resulting in 2 parses corresponding to 2 readings. The first parse (1+(7+10)) identifies the word sequence 他|喜欢|烤红薯, and the second parse (1+(9+(4+8))) a different sequence 他|喜欢|烤|红薯. Edge 7 and edge 9 represent two lexical entries for the verb 喜欢 (like), with different syntactic expectation (categorization). One expects an NP object, notated in the chart by like<NP>, and the other expects a VP complement, notated by like<VP>.

hpsg11

 

We now illustrate how Chinese proper names are identified in W‑CPSG parsing. In the W‑CPSG lexicon, Chinese family name is encoded to optionally expect the given name. Due to the arbitrariness of given names, no other constraint except for the length (either 1 character or 2 characters) is specified in the expectation. Therefore, we have three candidates for proper names in the following example, namely 李 (Li), 李得 (Li De), 李得胜 (Li Desheng), represented respectively by edge 1, edge (1+2) and the NP edge (1+5).[3] The first two candidates contribute to two valid parses while the third does not, hence the identification of the word sequences 李|得胜|了 and 李得|胜|了.

hpsg8

 

Now we add one more character 胜 (win) to form a new sentence, as shown in (24.) below.

hpsg9

 

The first two candidate proper names 李 (Li) and 李得 (Li De) no longer lead to parses. But the third candidate 李得胜 (Li Desheng) becomes part of the parse as a subject NP. The parse (((1+6)+4)+5) corresponds to the identification of the only valid word sequence 李得胜|胜|了.

Finally, we give an example to demonstrate how W‑CPSG handles reduplication in parsing and word identification. The sample sentence to be processed by the parser is 让他分分心 (Let him relax a while), involving the AB–>AAB type verb reduplication for diminutive use.

In most lexicons, 分心 (distract-heart: get distracted) is a registered 2-morpheme verb with internal morphological verb-object relation. Therefore, the reduplication is considered morphological. But in Chinese syntax, we also have a  general verb reduplication rule of the type A–>AA for diminutive use, for example, 看(look) –> 看看(have a look). This morphological verb reduplication rule AB–>AAB and the syntactic verb reduplication rule A–>AA are essentially the same rule in Chinese grammar. 分心 sits in the gray area between morphology and syntax. It looks both like a word (verb) and a phrase (VP). Lexically, it corresponds to one generalized sense (concept) and the internal combination is idiomatic, i.e. 分 (distract) must combine with 心 (heart) to mean ‘get distracted’. But, structurally, the combination of 分 and 心 is not fundamentally different from a VP consisting of Vt and NP, as in the phrase 看电影 (see a film). In fact, there is no clear-cut boundary between Chinese morphology and syntax. This morphology-syntax isomorphic fact serves as a further argument to support the W‑CPSG design of integrating morphology and syntax in one grammar module. Although the boundary between Chinese morphology and syntax is fuzzy, hence no universal definition of basic notions like word and phrase, the division can be easily defined system internally in an integrated grammar. In W‑CPSG,  分心 is treated as a phrase (VP) instead of a word (verb). The lexical entry 分 (distract) is coded to obligatorily expect the literal 心 (heart) as its syntactic object, shown in the following chart by the notation V<>. This approach has the advantage of eliminating the doubling of the reduplication rule for diminutive use in both syntax and morphology, making the grammar more elegant. The verb reduplication rule is implemented as a lexical rule in W‑CPSG.[4] This lexical rule creates a reduplicated verb with added diminutive sense, shown by edge 8 (a lexical edge).  The whole parsing process is illustrated below.

hpsg7

 

 

REFERENCES

Carpenter, B. & Penn, G. (1994): ALE, The Attribute Logic Engine, User’s Guide, Carnegie Mellon University

Chen, K-J., & S-H. Liu (1992): “Word identification for mandarin Chinese sentences”. Proceedings of the 15th International Conference on Computational Linguistics, Nantes, 101-107.

Feng, Z-W. (1996): “COLIPS lecture series – Chinese natural language processing”,  Communications of COLIPS, Vol.6, No.1 1996, Singapore

Li, W. (1997a): “Outline of an HPSG-style Chinese reversible grammar”, Proceedings of The Northwest Linguistics Conference-97 (NWLC-97, forthcoming), UBC, Vancouver, Canada

Li, W. (1997b): W‑CPSG: A Lexicalized Chinese Unification Grammar And Its Application, Doctoral dissertation (on-going), Simon Fraser University, Canada

Liang, N. (1987): “Shumian Hanyu Zidong Fenci Xitong – CDWS” (Automatic word segmentation system for written Chinese – CDWS), Journal of Chinese Information Processing, No.2 1987, pp 44-52, Beijing

Pollard, C.  & I. Sag (1994): Head-Driven Phrase Structure Grammar,  Centre for the Study of Language and Information, Stanford University, CA

Sun, M-S. & C-N. Huang  (1996): “Word segmentation and part of speech tagging for unrestricted Chinese texts” (Tutorial Notes for International Conference on Chinese Computing ICCC’96), Singapore

~~~~~~~~~~~~~~~~~~~

[1] The author benefited from the insightful discussion with Dr. Dekang Lin on the feasibility of parsing Chinese character strings instead of word strings. Thanks also go to Paul McFetridge and Fred Popowich for their supervision and encouragement.

[2] This table is adapted from the following table in Sun & Huang (1996).

case 1 The output of FMM and BMM are different, but both are incorrect 0.054%
case 2 The output of FMM and BMM are different, but only one is correct 9.24%
case 3 The output of FMM and BMM are identical, but incorrect 0.41%
case 4 The output of FMM and BMM are identical, and correct 90.30%

The 4 cases which they listed are not logically exhaustive in terms of sentence based processing (i.e. when discourse is not involved in a system). In particular, there is another case when the output of FMM and BMM are different, and both are correct. We call this a case of genuine cross ambiguity.

[3] Note that there is another S edge (1+5) in the chart. These two edges are structurally different, created via different PS rules. The NP edge (1+5) is formed through the morphological PS rule, combining the family name (edge 1) and its expected given name (edge 5). In the S edge (1+5). however, it is the subject rule (one of the complement PS rules) that decides the combination of the predicate (edge 5) and its expected subject NP (edge 1).

[4] Lexical rules are favored by many linguists to capture redundancy in the lexicon instead of the conventional approach of syntactic transformation. Lexical rules are applied at compile time to form an expanded lexicon before parsing starts.

 

[Related]

Interaction of syntax and semantics in parsing Chinese transitive verb patterns 

Handling Chinese NP predicate in HPSG 

Notes for An HPSG-style Chinese Reversible Grammar

Outline of an HPSG-style Chinese reversible grammar

PhD Thesis: Morpho-syntactic Interface in CPSG (cover page)

PhD Thesis: Chapter I Introduction

PhD Thesis: Chapter VII Concluding Remarks

Overview of Natural Language Processing

Dr. Wei Li’s English Blog on NLP

PhD Thesis: Chapter II Role of Grammar

 

2.0. Introduction

This chapter examines the role of grammar in handling the three major types of morpho-syntactic interface problems.  This investigation  justifies the mono-stratal design of CPSG95 which contains feature structures of both morphology and syntax.

The major observation from this study is:  (i) grammatical analysis, including both morphology and syntax, plays the fundamental role in contributing to the solutions of the morpho-syntactic problems;  (ii)  when grammar alone is not sufficient to reach the final solution, knowledge beyond morphology and syntax may come into play and serve as “filters” based on the grammatical analysis results.[1]  Based on this observation, a study in the direction of interleaving morphology and syntax will be pursued in the grammatical analysis.  Knowledge beyond morphology and syntax is left to future research.

Section 2.1 investigates the relationship between grammatical analysis and  the resolution of segmentation ambiguity.  Section 2.2 studies the role of syntax in handling Chinese productive word formation.  The borderline cases and their relationship with grammar are explored in 2.3.  Section 2.4 examines the relevance of knowledge beyond syntax to segmentation disambiguation.  Finally, a summary of the presented arguments and discoveries is given in 2.5.

2.1. Segmentation Ambiguity and Syntax

Segmentation ambiguity is one major problem which challenges the traditional word segmenter or an independent morphology.  The following study shows that this ambiguity is structural in nature, not fundamentally different from other structural ambiguity in grammar.  It will be demonstrated that sentential structural analysis is the key to this problem.

A huge amount of research effort in the last decade has been made on resolving segmentation ambiguity (e.g. Chen and Liu 1992; Gan 1995; He, Xu and Sun 1991; Liang 1987; Lua 1994; Sproat, Shih, Gale and Chang 1996; Sun and T’sou 1995; Sun and Huang 1996; X. Wang 1989; Wu and Su 1993; Yao, Zhang and Wu 1990; Yeh and Lee 1991; Zhang, Chen and Chen 1991; Guo 1997b).  Many (e.g. Sun and Huang 1996; Guo 1997b) agree that this is still an unsolved problem.  The major difficulty with most approaches reported in the literature lies in the lack of support from sufficient grammar knowledge.  To ultimately solve this problem, grammatical analysis is vital, a point to be elaborated in the subsequent sections.

2.1.1. Resolution of Hidden Ambiguity

The topic of this section is the treatment of hidden ambiguity.   The conclusion out of the investigation below is that the structural analysis of the entire input string provides a sound basis for handling this problem.

The following sample sentences illustrate a typical case involving the hidden ambiguity string 烤白薯 kao bai shu.

(2-1.) (a)      他吃烤白薯
ta         | chi  | kao-bai-shu
he      | eat  | baked-sweet-potato
[S [NP ta] [VP [V chi] [NP kao-bai-shu]]]
He eats the baked sweet potato.

(b) * ta       | chi  | kao          | bai-shu
he      | eat  | bake         | sweet-potato

(2-2.) (a) *    他会烤白薯
ta         | hui  | kao-bai-shu.
he      | can | baked-sweet-potato

(b)     ta       | hui  | kao          | bai-shu.
he      | can | bake         | sweet-potato
[S [NP ta] [VP [V hui] [VP [V kao] [NP bai-shu]]]]
He can bake sweet potatoes.

Sentences (2-1) and (2-2) are a minimal pair;  the only difference is the choice of the predicate verb, namely chi (eat) versus hui (can, be capable of).  But they have very different structures and assume different word identification.  This is because verbs like chi expect an NP object but verbs like hui require a VP complement.  The two segmentations of the string kao bai shu provide two possibilities, one as an NP kao-bai-shu and the other as a VP kao | bai-shu.  When the provided unit matches the expectation, it leads to a successful syntactic analysis, as illustrated by the parse trees in (2‑1a) and (2-2b).  When the expectation constraint is not satisfied, as in (2-1b) and (2-2a), the analysis fails.  These examples show that all candidate words in the input string should be considered for grammatical analysis.  The disambiguation choice can be made via the analysis, as seen in the examples above with the sample parse trees.  Correct segmentation results in at least one successful parse.

He, Xu and Sun (1991) indicate that a hidden ambiguity string requires a larger context for disambiguation.  But they did not define what the ‘larger context’ should be.  The following discussion attempts to answer this question.

The input string to the parser constitutes a basic context as well as the object for sentential analysis.[2]  It will be argued that this input string is the proper context for handling the hidden ambiguity problem.  The point to be made is, context smaller than the input string is not reliable for the hidden ambiguity resolution.  This point is illustrated by the following examples of the hidden ambiguity string ge ren in (2-3).[3]  In each successive case, the context is expanded to form a new input string.   As a result, the analysis and the associated interpretation of ‘person’ versus ‘individual’ change accordingly.

(2-3.)  input string                            reading(s)

(a)      人  ren                                       person (or man, human)
[N ren]

(b)      个人  ge ren                               individual
[N ge-ren]

(c)      三个人  san ge ren                               three persons
[NP [CLAP [NUM san] [CLA ge]] [N ren]]

(d)      人的力量  ren de li liang                      the human power
[NP [DEP [NP ren] [DE de]] [N li-liang]]

(e)      个人的力量  ge ren de li liang                        the power of an individual
[NP [DEP [NP ge-ren] [DE de]] [N li-liang]]

(f)       三个人的力量  san ge ren de li liang              the power of three persons
[NP [DEP [NP [CLAP [NUM san] [CLA ge]] [N ren]] [DE de]] [N li-liang]]

(g)      他不是个人  ta bu shi ge ren.
           (1)    He is not a man. (He is a pig.)
[S [NP ta] [VP [ADV bu] [VP [V shi] [NP [CLAP ge] [N ren]]]]]
(2)  He is not an individual. (He represents a social organization.)
[S [NP ta] [VP [ADV bu] [VP [V shi] [NP  ge-ren]]]]

Comparing (a), (b) with (c), and (d), (e) with (f), one can see the associated change of readings when each successively expanded input string leads to a different grammatical analysis.  Accordingly, one segmentation is chosen over the other on the condition that the grammatical analysis of the full string can be established based on the segmentation.  In (b), the ambiguous string is all that is input to the parser, therefore the local context becomes full context.  It then acquires the lexical reading individual as the other possible segmentation ge | ren does not form a legitimate combination.  This reading may be retained, as in (e), or changed to the other reading person, as in (c) and (f), or reduced to one of the possible interpretations, as in (g), when the input string is further lengthened.  All these changes depend on the sentential analysis of the entire input string, as shown by the associated structural trees above.  It demonstrates that the full context is required for the adequate treatment of the hidden ambiguity phenomena.  Full context here refers to the entire input string to the parser.

It is necessary to explain some of the analyses as shown in the sample parses  above.  In Contemporary Mandarin, a numeral cannot  combine with a noun without a classifier in between.[4]  Therefore, the segmentation san (three) | ge-ren (individual) is excluded in (c) and (f), and the correct segmentation san (three) | ge (CLA) | ren (person) leads to the NP analysis.  In general, a classifier alone cannot combine with the following noun either, hence the interpretation of ge ren as one word ge-ren (individual) in (b) and (e).  A classifier usually combines with a preceding numeral or determiner before it can combine with the noun.  But things are more complicated.  In fact, the Chinese numeral yi (one) can be omitted when the NP is in object position.  In other words, the classifier alone can combine with a noun in a very restricted syntactic environment.  That explains the two readings in (g).[5]

The following is a summary of the arguments presented above.   These arguments have been shown to account for the hidden ambiguity phenomena.  The next section will further demonstrate the validity of these arguments for overlapping ambiguity as well.

(2-4.) Conclusion
The grammatical analysis of the entire input string is required for the adequate treatment of the hidden ambiguity problem in word identification.

2.1.2. Resolution of Overlapping Ambiguity

This section investigates overlapping ambiguity and its resolution.  A previous influential theory is examined, which claims that the overlapping ambiguity string can be locally disambiguated.   However, this theory is found to be unable to account for a significant amount of data.  The conclusion is that both overlapping ambiguity and hidden ambiguity require a context of the entire input string and a grammar for disambiguation.

For overlapping ambiguity, comparing different critical tokenizations will be able to detect it, but such a technique cannot guarantee a correct choice without introducing other knowledge.  Guo (1997) pointed out:

As all critical tokenizations hold the property of minimal elements on the word string cover relationship, the existence of critical ambiguity in tokenization implies that the “most powerful and commonly used” (Chen and Liu 1992, page 104) principle of maximum tokenization would not be effective in resolving critical ambiguity in tokenization and implies that other means such as statistical inferencing or grammatical reasoning have to be introduced.

However, He, Xu and Sun (1991) claim that overlapping ambiguity can be resolved within the local context of the ambiguous string.  They classify the overlapping ambiguity string into nine types.  The classification is based on the categories of the assumably correctly segmented words in the ambiguous strings, described below.

Suppose there is an overlapping ambiguous string consisting of ABC;  both AB and BC are entries listed in the lexicon.  There are two possible cases.  In case one, the category of A and the category of BC define the classification of the ambiguous string.  This is the case when the segmentation A|BC is considered correct.  For example, in the  ambiguous string 白天鹅 bai tian e, the word AB is  bai-tian (day-time) and the word BC is tian-e (swan).  The correct segmentation for this string is assumed to be A|BC, i.e. bai (A: white) | tian-e (N: swan) (in fact, this cannot be taken for granted as shall be shown shortly), therefore, it belongs to the A-N type.  In case two, i.e. when the segmentation AB|C is considered correct, the category of AB and the category C define the classification of the ambiguous string.   For example, in the ambiguous string 需求和 xu qiu he, the word AB is  xu-qiu (requirement) and the word BC qiu-he (sue for peace).  The correct segmentation for this string is AB|C, i.e. xu-qiu (N: requirement) | he (CONJ: and) (again, this should not be taken for granted), therefore, it belongs to the N-CONJ type.

After classifying the overlapping ambiguous strings into one of nine types, using the two different cases described above, they claim to have discovered a rule.[6]  That is, the category of the correctly segmented word BC in case one (or AB in case two) is predictable from AB (or BC in case two) within the local ambiguous string.  For example, the category of tian-e (swan) in bai | tian-e (white swan) is a noun.  This information is predictable from bai tian within the respective local string bai tian e.  The idea is, if ever an overlapping ambiguity string is formed of bai tian and C, the judgment of bai | tian-C as the correct segmentation entails that the word tian-C  must be a noun.  Otherwise, the segmentation A|BC is wrong and the other segmentation AB|C is right.  For illustration, it is noted that tian-shi (angel) in the ambiguous string 白天使 bai | tian-shi (white angel) is, as expected, a noun.  This predictability of the category information from within the local overlapping ambiguous string is seen as an important discovery (Feng 1996).  Based on this assumed feature of the overlapping ambiguous strings, He,  Xu and Sun (1991) developed their theory that an overlapping ambiguity string can be disambiguated within the local string itself.

The proposed disambiguation process within the overlapping ambiguous string proceeds as follows.  In order to correctly segment an overlapping ambiguous string, say, bai tian e or bai tian shi, the following information needs to be given under the entry bai-tian (day-time) in the tokenization lexicon:  (i) an ambiguity label, to indicate the necessity to call a disambiguation rule;  (ii) the ambiguity type A-N, to indicate that it should call the rule corresponding to this type.  Then the following disambiguation rule can be formulated.

(2-5.) A-N type rule       (He,  Xu and Sun 1991)
In the overlapping ambiguous string A(1)…A(i) B(1)…B(j) C(1)…C(k),
if        B(1)…B(j) and C(1)…C(k) form a noun,
then  the correct segmentation is A(1)…A(i) | B(1)…B(j)-C(1)…C(k),
else    the correct segmentation is A(1)…A(i)-B(1)…B(j) | C(1)…C(k).

This way, bai tian e and bai tian shi will always be segmented as bai (white) | tian-e (swan) and bai (white) | tian-shi (angel) instead of bai-tian (daytime) | e (goose) and bai-tian (daytime) | shi (make).  This can be easily accommodated in a segmentation algorithm provided the above  information is added to the lexicon and the disambiguation rules are implemented.  The whole procedure is running within the local context of the overlapping ambiguous string and uses only lexical information.  So they also name the overlapping ambiguity disambiguation morphology-based disambiguation, with no need to consult syntax, semantics or discourse.

Feng (1996) emphasizes that He, Xu and Sun’s view on the overlapping ambiguous string constitutes a valuable contribution to the theory of Chinese word identification.  Indeed, this overlapping ambiguous string theory, if it were right, would be a breakthrough in this field.  It in effect suggests that the majority of the segmentation ambiguity is resolvable without and before a grammar module.  A handful of simple rules, like the A-N type rule formulated above, plus a lexicon would solve most ambiguity problems in word identification.[7]

Feng (1996) provides examples for all the nine types of overlapping ambiguous strings as evidence to support He, Xu and Sun (1991)’s theory.   In the case of the A-N type ambiguous string bai tian e, the correct segmentation is supposed to be bai | tian-e in this theory.  However, even with his own cited example, Feng ignores a perfect second reading (parse) when the time NP bai-tian (daytime) directly acts as a modifier for the sentence with no need for a preposition, as shown in (2‑6b) below.

(2-6.)           白天鹅游过来了
bai tian e you guo lai le       (Feng 1996)

(a)      bai     | tian-e       | you          | guo-lai      | le.
          white | swan        | swim        | over-here  | LE
[S [NP bai tian-e] [VP you guo-lai le]]
The white swan swam over here.

(b)      bai-tian       | e              | you          | guo-lai      | le.
          day-time      | goose        | swim        | over-here  | LE
[S [NP+mod bai-tian] [S [NP e] [VP you guo-lai le]]]
In the day time the geese swam over here.

In addition, one only needs to add a preposition zai (in) to the beginning of the sentence to make the abandoned segmentation bai-tian | e the only right one in the changed context.  The presumably correct segmentation, namely bai | tian-e, now turns out to be wrong, as shown in (2-7a) below.

(2-7.)           在白天鹅游过来了
zai bai tian e you guo lai le

(a) *   zai     | bai           | tian-e       | you          | guo-lai      | le.
          in      | white        | swan        | swim        | over-here  | LE

(b)      zai     | bai-tian    | e              | you          | guo-lai      | le.
          in      | day-time   | goose        | swim        | over-here  | LE
[S [PP+mod zai bai-tian] [S [NP e] [VP you guo-lai le]]]
In the day time the geese swam over here.

The above counter-example is by no means accidental.  In fact, for each cited ambiguous string in the examples given by Feng, there exist counter-examples.  It is not difficult to construct a different context where the preferred segmentation within the local string, i.e. the segmentation chosen according to one of the rules, is proven to be wrong.[8]  In the pairs of sample sentences (2‑8) through (2-10), (a) is an example which Feng (1996) cited to support the view that the local ambiguous string itself is enough for disambiguation.  Sentences in (b) are counter-examples to this theory.  It is a notable fact that the listed local string is often properly contained in a more complicated ambiguous string in an expanded context, seen in (2-9b) and (2-10b).  Therefore, even when the abandoned segmentation can never be linguistically correct in any context, as shown for tu-xing (graph) | shi (BM) in (2-9) where a bound morpheme still exists after the segmentation, it does not entail the correctness of the other segmentation in all contexts.  These data show that all possible segmentations should be retained for the grammatical analysis to judge.

(2-8.)  V-N type of overlapping ambiguous string

研究生命
          yan jiu sheng ming:
          yan-jiu (V:study) | sheng-ming (N:life)
yang-jiu-sheng (N:graduate student) | ming (life/destiny)

(a)      研究生命的本质
          yan-jiu    sheng-ming de      ben-zhi
          study          life               DE     essence
Study the essence of life.

(b)      研究生命金贵
           yan-jiu-sheng      ming  jin-gui
          graduate-student  life     precious
Life for graduate students is precious.

(2-9.)  CONJ-N type of overlapping ambiguous string
和平等 he ping deng:
          he (CONJ:and) | ping-deng (N:equality)
he-ping (N:peace) | deng (V:wait)?

(a)      独立自主和平等互利的原则
           du-li-zi-zhu           he      ping-deng-hu-li               de      yuan-ze
          independence       and    equal-reciprocal-benefit  DE     principle
the principle of independence and equal reciprocal benefit

(b)      和平等于胜利 he-ping       deng-yu       sheng-li
          peace           equal           victory
Peace is equal to victory.

(2-10.)  V-P type of overlapping ambiguous string
看中和 kan zhong he:
          kan-zhong (V:target) | he (P:with)
kan (V:see) | zhong-he (V:neutralize)

(a)      他们看中和日本人生意的机会
ta-men    kan-zhong   he      ri-ben          ren              zuo     sheng-yi      de      ji-hui    
they         target           with    Japan          person          do      business     DE   opportunity
They have targeted the opportunity to do business with the Japanese.

(b)      这要看中和作用的效果
zhe          yao    kan    zhong-he-zuo-yong                   de          xiao-guo
this    need  see     neutralization                DE     effect
This will depend on the effects of the neutralization.

The data in (b) above directly contradict the claim that an overlapping ambiguous string can be disambiguated within the local string itself.  While this approach is shown to be inappropriate in practice, the following comment attempts to reveal its theoretical motivation.

As reviewed in the previous text, He, Xu and Sun (1991)’s overlapping ambiguity theory is established on the classification of the overlapping ambiguous strings.  A careful examination of their proposed nine types of the overlapping ambiguous strings reveals an underlying assumption on which the classification is based.  That is, the correctly segmented words within the overlapping ambiguous string will automatically remain correct in a sentence containing the local string.   This is in general untrue, as shown by the counter-examples above.[9]   The following analysis reveals why.

Within the local context of the overlapping ambiguous string, the chosen segmentation often leads to a syntactically legitimate structure while the abandoned segmentation does not.  For example,  bai (white) | tian-e (swan) combines into a valid syntactic unit while there is no structure which can span bai-tian (daytime) | e (goose).  For another example,  yan-jiu (study) | sheng-ming (life) can be combined into a legitimate verb phrase [VP [V yan-jiu] [NP sheng-ming]], but  yan-jiu-sheng (graduate student) | ming (life/destiny) cannot.  But that legitimacy only stands locally within the boundary of the ambiguous string.  It does not necessarily hold true in a larger context containing the string.  As shown previously in (2-7a),  the locally legitimate structure bai | tian-e (white swan) does not lead to a successful parse for the sentence.  In contrast, the locally abandoned segmentation bai-tian (daytime) | e (goose) has turned out to be right with the parse in (2-7b).   Therefore, the full context instead of the local context of the ambiguous string is required for the final judgment on which segmentation can be safely abandoned.  Context smaller than the entire input string is not reliable for the overlapping ambiguity resolution.  Note that exactly the same conclusion has been reached for the hidden ambiguous strings in the previous section.

The following data in (2-11) further illustrate the point of the full context requirement for the overlapping ambiguity resolution, similar to what has been presented for the hidden ambiguity phenomena in (2-3).  In each successive case, the context is expanded to form a new input string.  As a result, the interpretation of ‘goose’ versus ‘swan’ changes accordingly.

(2-11.)  input string                reading(s)

(a)      鹅 e                                goose
[N e]

(b)      天鹅 tian e                                swan
[N tian-e]

(c)      白天鹅 bai tian e                       white swan
[N [A bai] [N tian-e]]

(d)      鹅游过来了 e you guo lai le.
The geese swam over here.
[S [NP e] [VP you guo-lai le]]

(e)      天鹅游过来了 tian e you guo lai le.
The swans swam over here.
[S [NP tian-e] [VP you guo-lai le]]

(f)      白天鹅游过来了 bai tian e you guo lai le.
          (i)       The white swan swam over here.
[S [NP bai tian-e] [VP you guo-lai le]]
          (ii)      In the daytime, the geese swam over here.
S [NP+mod bai-tian] [S [NP e] [VP you guo-lai le]]]

(g)       在白天鹅游过来了 zai bai tian e you guo lai le.
            In the daytime, the geese swam over here.
[S [PP zai bai-tian] [S [NP e] [VP you guo-lai le]]]

(h)      三只白天鹅游过来了 san zhi bai tian e you guo lai le.
           Three white swans swam over here.
[S [NP san zhi bai tian-e] [VP you guo-lai le]]

It is interesting to compare (c) with (f), (g) and (h) to see their associated change of readings based on different ways of  segmentation.  In (c), the overlapping ambiguous string is all that is input to the parser, therefore the local context becomes full context.  It then acquires the reading white swan corresponding to the segmentation bai | tian-e.  This reading may be retained, or changed, or reduced to one of the possible interpretations when the input string is lengthened.  That is respectively the case in (h), (g) and (f).  All these changes depend on the grammatical analysis of the entire input string.  It shows that the full context and a grammar are required for the resolution of most ambiguities;  and when sentential analysis cannot disambiguate – in cases of ‘genuine’ segmentation ambiguity like (f), the structural analysis can make explicit the ambiguity in the form of multiple parses (readings).

In the light of the inquiry in this section, the theoretical significance of the distinction between overlapping ambiguity and hidden ambiguity seems to have diminished.[10]  They are both structural in nature.  They both require full context and a grammar for proper treatment.

(2-12.) Conclusion

(i)  It is not necessarily true that an overlapping ambiguous string can be disambiguated within the local string.

(ii) The grammatical analysis of the entire input string is required for the adequate treatment of the overlapping ambiguity problem as well as the hidden ambiguity problem.

2.2. Productive Word Formation and Syntax

This section examines the connection of productive word formation and segmentation ambiguity.  The observation is that there is always a possible involvement of ambiguity with each type of word formation.  The point to be made is that no independent morphology systems can resolve this ambiguity when syntax is unavailable.  This is because words formed via morphology, just like words looked up from lexicon, only provide syntactic ‘candidate’ constituents for the sentential analysis.  The choice is decided by the structural analysis of the entire sentence.

Derivation is a major type of productive word formation in Chinese.   Section 1.2.2 has given an example of the involvement of hidden ambiguity in derivation, repeated below.

(2-13.)         这道菜没有吃头  zhe dao cai mei you chi tou.

(a)      zhe    | dao          | cai            | mei-you    | chi-tou
          this    | CLA                   | dish         | not-have   | worth-of-eating
[S [NP zhe dao cai] [VP [V mei-you] [NP chi-tou]]]
This dish is not worth eating.

(b) ?   zhe    | dao          | cai            | mei-you    | chi  | tou
          this    | CLA                   | dish         | not have   | eat  | head
[S [NP zhe dao cai] [VP [ADV mei-you] [VP [V chi] [NP tou]]]]
This dish did not eat the head.

(2-14.)         他饿得能吃头牛 ta e de neng chi tou niu.

(a) *   ta       | e              | de            | neng         | chi-tou               | niu
he      | hungry     | DE3         | can           | worth-of-eating  | ox

(b)      ta       | e              | de            | neng         | chi  | tou            | niu
he      | hungry     | DE3         | can           | eat  | CLA          | ox
[…[VP [V e] [DE3P [DE3 de] [VP [V neng] [VP [V chi] [NP tou niu]]]]]]
He is so hungry that he can eat an ox.

Some derivation rule like the one in (2-15) is responsible for combining the transitive verb stem and the suffix –tou (worth-of) into a derived noun for (2-13a) and (2-14a).

(2-15.)         X (transitive verb) + tou –> X-tou (noun, semantics: worth-of-X)

However, when syntax is not available, there is always a danger of wrongly applying this morphological rule due to possible ambiguity involved, as shown in (2-14a).  In other words, morphological rules only provide candidate words;  they cannot make the decision whether these words are legitimate in the context.

Reduplication is another method for productive word formation in Chinese.  An outstanding problem is the AB –> AABB reduplication or AB –> AAB reduplication if AB is a listed word.   In these cases, some reduplication rules or procedures need to be involved to recognize AABB or AAB.  If reduplication is a simple process confined to a local small context, it may be possible to handle it by incorporating some procedure-based function calls during the lexical lookup.  For example, when a three-character string, say 分分心 fen fen xin, cannot be found in the lexicon, the reduplication function will check whether the first two characters are the same, and if yes, delete one of them and consult the lexicon again.  This method is expected to handle the AAB type reduplication, e.g. fen-xin (divide-heart: distract) –> fen-fen-xin (distract a bit).

But, segmentation ambiguity can be involved in reduplication as well.  Compare the following examples in (2-16) and (2-17) containing the sub-string fen fen xin, the first is ambiguity free but the second is ambiguous.  In fact, (2‑17) involves an overlapping ambiguous string  shi fen fen xinshi (ten) | fen-fen-xin (distract a bit) and shi-fen (very) | fen-xin (distract).  Based on the conclusion presented in 2.1, it requires grammatical analysis to resolve the segmentation ambiguity.  This is illustrated in (2‑17).

(2-16.)         让他分分心

rang     | ta    | fen-fen-xin
let      | he   | distracted-a-bit
Let him relax a while.

(2-17.)         这件事十分分心

zhe jian shi shi fen fen xin.

(a) *   zhe    | jian          | shi           | shi  | fen-fen-xin
          this    | CLA         | thing       | ten  | distracted a bit

(b)      zhe    | jian          | shi            | shi-fen     | fen-xin
           this    | CLA         | thing       | very         | distract
[S [NP zhe jian shi] [VP [ADV shi-fen] [V fen-xin]]]
This thing is very distracting.

Finally, there is also possible ambiguity involvement in the proper name formation.  Proper names for persons, locations, etc. that are not listed in the lexicon are recognized as another major problem in word identification (Sun and Huang 1996).[11]  This problem is complicated when ambiguity is involved.

For example, a Chinese person name usually consists of a family name followed by a given name of one or two characters.  For example, the late Chinese chairman mao-ze-dong (Mao Zedong) used to have another name li-de-sheng (Li Desheng).  In the lexicon, li is a listed family name.  Both de-sheng and sheng mean ‘win’.  This may lead to three ways of word segmentation, a complicated case involving both overlapping ambiguity and hidden ambiguity:  (i) li | de-sheng;  (ii) li-de | sheng;  (iii) li-de-sheng, as shown in (2-18) below.

(2-18.)         李得胜了 li de sheng le.

(a)      li        | de-sheng  | le
           Li       | win          | LE
[S [NP li] [VP de-sheng le]]
Li won.

(b)      li-de   | sheng       | le
           Li De | win          | LE
[S [NP li de] [VP sheng le]]
Li De won.

(c) *    li-de-sheng  | le
           Li Desheng  | LE

For this particular type of compounding, the family name serves as the left boundary of a potential compound name of person and the length can be used to determine candidates.[12]  Again, the choice is decided by the grammatical analysis of the entire sentence, as illustrated in (2-18).

(2-19.) Conclusion

Due to the possible ambiguity involvement in productive word formation, a grammar containing both morphology and syntax is required for an adequate treatment.  An independent morphology system or separate word segmenter cannot solve ambiguity problems.

2.3. Borderline Cases and Grammar

This section reviews some outstanding morpho-syntactic borderline phenomena.  The points to be made are:  (i) each proposed morphological or syntactic analysis should be justified in terms of capturing the linguistic generality;  (ii) the design of a grammar should facilitate the access to the knowledge from both morphology and syntax in analysis.

The nature of the borderline phenomena calls for the coordination of morphology and syntax in a grammar.  The phenomena of Chinese separable verbs are one typical example.  The co-existence of their contiguous use and separate use leads to the confusion whether they belong to the lexicon and morphology, or whether they are syntactic phenomena.  In fact, as will be discussed in Chapter V, there are different degrees of ‘separability’ for different types of Chinese separable verbs;  there is no uniform analysis which can handle all separable verbs properly.  Different types of separable verbs may justify different approaches to the problems.  In terms of capturing linguistic generality, a good analysis should account for the demonstrated variety of separated uses and link the separated use and the contiguous use.

‘Quasi-affixation’ is another outstanding interface problem.  This problem requires careful morpho-syntactic coordination.  As presented in Chapter I, structurally, ‘quasi-affixes’ and ‘true’ affixes demonstrate very similar word formation potential, but ‘quasi-affixes’ often retain some ‘solid’ meaning while the meaning of ‘true’ affixes are functionalized.  Therefore, how to coordinate the semantic contribution of the derived words via ‘quasi-affixation’ in the context of the building of the semantics for the entire sentence is the key.  This coordination requires flexible information flow between data structures for morphology, syntax and semantics during the morpho-syntactic analysis.

In short, the proper treatment of the morpho-syntactic borderline phenomena requires inquiry into each individual problem in order to reach a morphological or syntactic analysis which maximally captures linguistic generality.  It also calls for the design of a grammar where information between morphology and syntax can be effectively coordinated.

2.4. Knowledge beyond Syntax

This section examines the roles of knowledge beyond syntax in the resolution of segmentation ambiguity.  Despite the fact that further information beyond syntax may be necessary for a thorough solution to segmentation ambiguity,[13] it will be argued that syntax is the appropriate place for initiating this process due to the structural nature of segmentation ambiguity.

Depending on which type of information is essential for the disambiguation, disambiguation can be classified as structure-oriented, semantics-oriented and pragmatics-oriented.  This classification hierarchy is modified from that in He, Xu and Sun (1991).  They have classified the hidden ambiguity disambiguation into three categories:  syntax-based, semantics-based and pragmatics-based.  Together with the morphology-based disambiguation which is equivalent to the overlapping ambiguity resolution in their theory, they have built a hierarchy from morphology up to pragmatics.

A note on the technical details is called for here.  The term X‑oriented (where X is either syntax, semantics or pragmatics) is selected here instead of X-based in order to avoid the potential misunderstanding that X is the basis for the relevant disambiguation.  It will be shown that while information from X is required for the ambiguity resolution, the basis is always syntax.

Based on the study in 2.1, it is believed that there is no morphology-based (or morphology-oriented) disambiguation independent of syntax.  This is because the context of morphology is a local context, too small for resolving structural ambiguity.  There is little doubt that the morphological analysis is a necessary part of word identification in terms of handling productive word formation.  But this analysis cannot by itself resolve ambiguity, as argued in 2.2.  The notion ‘structure’ in structure-oriented disambiguation includes both syntax and morphology.

He, Xu and Sun (1991) exclude the overlapping ambiguity resolution in the classification beyond morphology.  This exclusion is found to be not appropriate.  In fact, both the resolution of hidden ambiguity and overlapping ambiguity can be classified into this hierarchy.   In order to illustrate this point, for each such class, I will give examples from both hidden ambiguity and overlapping ambiguity.

Sentences in (2-20) and (2-21) which contain the hidden ambiguity string 阵风zhen feng  are examples for the structure-oriented disambiguation.  This type of disambiguation relying on a grammar constitutes the bulk of the disambiguation task required for word identification.

(2-20.)         一阵风吹过来了
yi zhen feng chui guo lai le.          (Feng 1996)

(a)      yi       | zhen         | feng         | chui          | guo-lai      | le
          one    | CLA          | wind        | blow         | over-here  | LE
[S [NP [CLAP yi zhen] [N feng]] [VP chui guo-lai le]]
A gust of wind blew here

(b) *   yi       | zhen-feng                    | chui                   | guo-lai      | le
          one    | gusts-of-wind    | blow         | over-here  | LE

(2-21.)         阵风会很快来临 zhen feng hui hen kuai lai lin.

(a)      zhen-feng              | hui  | hen          | kuai         | lai-lin
          gusts-of-wind       | will | very                   | soon         | come
[S [NP zhen-feng] [VP hui hen kuai lai-lin]]]
Gusts of wind will come very soon.

(b) *   zhen  | feng                   | hui  | hen          | kuai         | lai-lin
          CLA   | wind        | will | very                   | soon         | come

Compare (2-20a) where the ambiguity string is identified as two words zhen (CLA) feng (wind) and (2-21a) where the string is taken as one word zhen-feng (gusts-of-wind).  Chinese syntax defines that a numeral cannot directly combine with a noun, neither can a classifier alone when it is in non-object position.  The numeral and the classifier must combine together before they can combine with a noun.  So (2-20b) and (2‑21b) are both ruled out while (2-20a) and (2-21a) are structurally well-formed.

For the structure-oriented overlapping ambiguity resolution,  numerous examples have been cited before, and one typical example is repeated below.

(2-22.)         研究生命金贵 yan jiu sheng ming jin gui

(a)      yan-jiu-sheng       | ming         | jin-gui
graduate student | life            | precious
[S [NP yan-jiu-sheng] [S [NP ming] [AP jin-gui]]]
Life for graduate students is precious.

(b) *   yan-jiu        | sheng-ming        | jin-gui
study          | life                     | precious

As a predicate, the adjective jin-gui (precious) syntactically expects an NP as its subject, which is saturated by the second NP ming (life) in (2-22a).   The first NP serves as a topic of the sentence and is semantically linked to the subject ming (life) as its possessive entity.[14]  But there is no parse for (2-22b) despite the fact that the sub-string yan-jiu sheng-ming (to study life) forms a verb phrase [VP [V yan-jiu] [NP sheng-ming]] and the sub-string sheng-ming jin-gui (life is precious) forms a sentence [S [NP sheng-ming] [AP jin-gui]].  On one hand, the VP in the subject position does not satisfy the syntactic constraint (the category NP) expected by the adjective jin-gui (precious) – although other adjectives, say zhong-yao ‘important’, may expect a VP subject.  On the other hand, the transitive verb yan-jiu (study) expects an NP object.  It cannot take an S object (embedded object clause) as do other verbs, say ren-wei (think).

The resolution of the following hidden ambiguity belongs to the semantics-oriented disambiguation.

(2-23.)         请把手抬高一点儿 qing ba shou tai gao yi dian er            (Feng 1996)

(a1)    qing             | ba   | shou         | tai            | gao | yi-dian-er
          please          | BA  | hand        | hold         | high| a-little
[VP [ADV qing] [VP ba shou tai gao yi-dian-er]]
Please raise your hand a little higher.

(a2) * qing   | ba   | shou         | tai            | gao           | yi-dian-er
          invite | BA  | hand        | hold         | high         | a-little

(b1) * qing             | ba-shou    | tai            | gao           | yi-dian-er
          please          | N:handle  | hold         | high         | a-little

(b2) ? qing   | ba-shou    | tai            | gao           | yi-dian-er
          invite | N:handle  | hold         | high         | a-little
[VP [VG [V qing] [NP ba-shou]] [VP tai gao yi-dian-er]]
Invite the handle to hold a little higher.

This is an interesting example.  The same character qing is both an adverb ‘please’ and a verb ‘invite’.  (2-23b2) is syntactically valid, but violates the semantic constraint or semantic selection restriction.  The logical object of qing (invite) should be human but ba-shou (handle)  is not human.  The two syntactically valid parses (2-23a1) and (2-23b2), which correspond to two ways of segmentation, are expected to be somehow disambiguated on the above semantic grounds.

The following case is an example of semantics-oriented resolution of the overlapping ambiguity.

(2-24.)         茶点心吃了 cha dian xin chi le.

(a1)    cha    | dian-xin   | chi  | le
tea     | dim sum  | eat  | LE
[S [NP+object cha dian-xin] [VP chi le]]
The tea dim sum was eaten.

(a2) ? cha    | dian-xin   | chi  | le
tea     | dim sum  | eat  | LE
[S [NP+agent cha dian-xin] [VP chi le]]
The tea dim sum ate (something).

(a3) ? cha    | dian-xin   | chi  | le
tea     | dim sum  | eat  | LE
[S [NP+object cha ] [S [NP+agent dian-xin] [VP chi le]]]
Tea, the dim sum ate.

(a4) ? cha    | dian-xin   | chi  | le
tea     | dim sum  | eat  | LE
[S [NP+agent cha ] [VP [NP+object dian-xin] [VP chi le]]]
The tea ate the dim sum.

(b1) ? cha-dian               | xin           | chi  | le
tea dim sum         | heart       | eat  | LE
[S [NP+object cha-dian] [S [NP+agent xin] [VP chi le]]]
The tea dim sum, the heart ate.

(b2) ? cha-dian               | xin           | chi  | le
tea dim sum         | heart        | eat  | LE
[S [NP+agent cha-dian] [VP [NP+object xin] [VP chi le]]]
The tea dim sum ate the heart.

Most Chinese dictionaries contain the listed compound noun cha-dian (tea-dim-sum), but not cha dian-xin which stands for the same thing, namely the snacks served with the tea.  As shown above, there are four analyses for one segmentation and two analyses for the other segmentation.  These are all syntactically legitimate, corresponding to six different readings.  But there is only one analysis which makes sense, namely the implicit passive construction with the compound noun cha dian-xin as the preceding (logical) object in (a1).  All the other five analyses are nonsense and can be disambiguated if the semantic selection restriction that animate being eats (i.e. chi) food is enforced.   Syntactically, (a2) is an active construction with the optional object omitted.  The constructions for (a3) and (b1) are of long distance dependency where the object is topicalized and placed at the beginning.   The SOV (Subject Object Verb) pattern for (a4) and (b2) is a very  restrictive construction in Chinese.[15]

The pragmatics-oriented disambiguation is required for the case where ambiguity remains after the application of both structural and semantic constraints.[16]  The sentences containing this type of ambiguity are genuinely ambiguous within the sentence boundary, as shown with the multiple parses in (2-25) for the hidden ambiguity and (2-26) for the overlapping ambiguity below.

(2-25.)         他喜欢烤白薯 ta xi huan kao bai shu.

(a)      ta       | xi-huan    | kao          | bai-shu.
          he      | like           | bake         | sweet-potato
[S [NP ta] [VP [V xi-huan] [VP [V kao] [NP bai-shu]]]]
He likes baking sweet potatoes.

(b)      ta       | xi-huan    | kao-bai-shu.
          he      | like           | baked-sweet-potato
[S [NP ta] [VP [V xi-huan] [NP kao-bai-shu]]]
He likes the baked sweet potatoes.

(2-26.)         研究生命不好 yan jiu sheng ming bu hao

(a)      yan-jiu-sheng       | ming         | bu   | hao.
          graduate student | destiny     | not | good
[S [NP yan-jiu-sheng] [S [NP ming] [AP bu hao]]]
The destiny of graduate students is not good.

(b)      yan-jiu        | sheng-ming        | bu   | hao.
          study          | life                     | not | good
[S [VP yan-jiu sheng-ming] [AP bu hao]]
It is not good to study life.

An important distinction should be made among these classes of disambiguation.  Some ambiguity must be solved in order to get a reading during analysis.  Other ambiguity can be retained in the form of multiple parses, corresponding to multiple readings.  In either case, it demonstrates that at least a grammar (syntax and morphology) is required.  The structure-oriented ambiguity belongs to the former, and can be handled by the appropriate structural analysis.  The semantics-oriented ambiguity and the pragmatics-oriented ambiguity belong to the latter, so multiple parses are a way out.  The examples for different classes of ambiguity show that the structural analysis is the foundation for handling ambiguity problems in word identification.  It provides possible structures for the semantic constraints or pragmatic constraints to work on.

In fact, the resolution of segmentation ambiguity in Chinese word identification is but a special case of the resolution of structural ambiguity for NLP in general.  As a matter of fact, the grammatical analysis has been routinely used to resolve, and/or prepare the basis for resolving, the structural ambiguity like the PP attachment.[17]

2.5. Summary

The most important discovery in the field of Chinese word identification presented in this chapter is that the resolution of both types of segmentation ambiguity involves the analysis of the entire input string.  This means that the availability of a grammar is the key to the solution of this problem.

This chapter has also examined the ambiguity involvement in productive word formation and reached the following conclusion.  A grammar for morphological analysis as well as for sentential analysis is required for an adequate treatment of this problem.  This establishes the foundation for the general design of CPSG95 as consisting of morphology and syntax in one grammar formalism. [18]

The study of the morpho-syntactic borderline problems shows that  the sophisticated design of a grammar is called for so that information between morphology and syntax can be effectively coordinated.  This is the work to be presented in Chapter III and Chapter IV.  It also demonstrates that each individual borderline problem should be studied carefully in order to reach a morphological or syntactic analysis which maximally captures linguistic generality.  This study will be pursued in Chapter V and Chapter VI.

 

 

———————————————————-

[1]  Constraints beyond morphology and syntax can be implemented as subsequent modules, or “filters”, in order to select the correct analysis when morpho-syntactic analysis leads to multiple results (parses).  Alternatively, such constraints can also be integrated into CPSG95 as components parallel to, and interacting with, morphology and syntax.  W. Li (1996) illustrates how semantic selection restriction can be integrated into syntactic constraints in CPSG95 to support Chinese parsing.

[2] In theory, if discourse is integrated in the underlying grammar, the input can be a unit larger than sentence, say, a paragraph or even a full text.  But this will depend on the further development in discourse theory and its formalization.  Most grammars in current use assume sentential analysis.

[3]  Similar examples for the overlapping ambiguity string will be shown in 2.1.2.

[4]  But in Ancient Chinese, a numeral can freely combine with countable nouns.

[5] These two readings in written Chinese correspond to an obvious difference in Spoken Chinese:  ge (CLA) in (g1) is weakened in pronunciation, marked by the dropping of the tone, while in (g2) it reads with the original 4th tone emphatically.

[6] It is likely that what they have found corresponds to Guo’s discovery of “one tokenization per source” (Guo 1998).  Guo’s finding is based on his experimental study involving domain (“source”) evidence and seems to account for the phenomena better.  In addition, Guo’s strategy in his proposal is also more effective, reported to be one of the best strategies for disambiguation in word segmenters.

[7] According to He, Xu and Sun (1991)’s statistics on a corpus of 50833 Chinese characters, the overlapping ambiguous strings make up 84.10%, and the hidden ambiguous strings 15.90%, of all ambiguous strings.

[8] Guo (1997b) goes to the other extreme to hypothesize that “every tokenization is possible”.   Although this seems to be a statement too strong, the investigation in this chapter shows that at least domain independently, local context is very unreliable for making tokenization decision one way or the other.

[9] However, this assumption may become statistically valid within a specific domain or source, as examined in Guo (1998).  But Guo did not give an operational definition of source/domain.  Without such a definition, it is difficult to decide where to collect the domain-specific information required for disambiguation based on the principle one tokenization per source, as proposed by Guo (1998).

[10] This distinction is crucial in the theories of Liang (1987) and He,  Xu and Sun (1991).

[11] This work is now defined as one fundamental task, called Named Entity tagging, in the world of information extraction (MUC-7 1998).  There has been great advance in developing Named Entity taggers both for Chinese (e.g. Yu et al 1997; Chen et al 1997) and for other languages.

[12] That is what was actually done with the CPSG95 implementation.  More precisely, the family name expects a special sign with hanzi-length of 1 or 2 to form a full name candidate.

[13] A typical, sophisticated word segmenter making reference to knowledge beyond syntax is presented in Gan (1995).

[14] This is in fact one very common construction in Chinese in the form of NP1 NP2 Predicate.  Other examples include ta (he) tou (head) tong (ache): ‘he has a head-ache’ and ta (he) shen-ti (body) hao (good): ‘he is good in health’.

[15] For the detailed analysis of these constructions, see W. Li (1996).

[16] It seems that it may be more appropriate to use terms like global disambiguation or discourse-oriented disambiguation instead of the term pragmatics-oriented disambiguation for the relevant phenomena.

[17] It seems that some PP attachment problems can be resolved via grammatical analysis alone.  For example, put something on the table; found the key to that door.  Others require information beyond syntax (semantics, discourse, etc.) for a proper solution.  For example, see somebody with telescope. In either case, the structural analysis provides a basis.  The same thing happens to the disambiguation in Chinese word identification.

[18] In fact, once morphology is incorporated in the grammar, the identification of both vocabulary words and non-listable words becomes a by-product during the integrated morpho-syntactic analysis.  Most ambiguity is resolved automatically and the remaining ambiguity will be embodied in the multiple syntactic trees as the results of the analysis.  This has been shown to be true and viable by W. Li (1997, 2000) and Wu and Jiang (1998).

 

[Related]

PhD Thesis: Morpho-syntactic Interface in CPSG (cover page)

PhD Thesis: Chapter I Introduction

PhD Thesis: Chapter II Role of Grammar

PhD Thesis: Chapter III Design of CPSG95

PhD Thesis: Chapter IV Defining the Chinese Word

PhD Thesis: Chapter V Chinese Separable Verbs

PhD Thesis: Chapter VI Morpho-syntactic Interface Involving Derivation

PhD Thesis: Chapter VII Concluding Remarks

Overview of Natural Language Processing

Dr. Wei Li’s English Blog on NLP

PhD Thesis: Chapter I Introduction

1.0. Foreword

This thesis addresses the issue of the Chinese morpho-syntactic interface.  This study is motivated by the need for a solution to a series of long-standing problems at the interface.  These problems pose challenges to an independent morphology system or a separate word segmenter as there is a need to bring in syntactic information in handling these problems.

The key is to develop a Chinese grammar which is capable of representing sufficient information from both morphology and syntax.  On the basis of the theory of Head-Driven Phrase Structure Grammar (Pollard and Sag 1987, 1994), the thesis will present the design of a Chinese grammar, named CPSG95 (for Chinese Phrase Structure Grammar).  The interface between morphology and syntax is defined system internally in CPSG95.  For each problem, arguments will be presented for the linguistic analysis involved.  A solution to the problem will then be formulated based on the analysis.  The proposed solutions are formalized and implementable;  most of the proposals have been tested in the implementation of CPSG95.

In what follows, Section 1.1 reviews some important developments in the field of Chinese NLP (Natural Language Processing).  This serves as the background for this study.  Section 1.2 presents a series of long-standing problems related to the Chinese morpho-syntactic interface.  These problems are the focus of this thesis.  Section 1.3 introduces CPSG95 and sketches its morpho-syntactic interface by illustrating an example of the proposed morpho-syntactic analysis.

1.1. Background

This section presents the background for the work on the interface between morphology and syntax in CPSG95.  Major development on Chinese tokenization and parsing, the two areas which are related to this study, will be reviewed.

1.1.1. Principle of Maximum Tokenization and Critical Tokenization

This section reviews the influential Theory of Critical Tokenization (Guo 1997a) and its implications.  The point to be made is that the results of Guo’s study can help us to select the tokenization scheme used in the lexical lookup phase in order to create the basis for morpho-syntactic parsing.

Guo (1997a,b,c) has conducted a comprehensive formal study on tokenization schemes in the framework of formal languages, including deterministic tokenization such as FT (Forward Maximum Tokenization) and BT (Backward Maximum Tokenization), and non-deterministic tokenization such as CT (Critical Tokenization), ST (Shortest Tokenization) and ET (Exhaustive Tokenization).  In particular, Guo has focused on the study of the rich family of tokenization strategies following the general Principle of Maximum Tokenization, or “PMT”.  Except for ET, all the tokenization schemes mentioned above are PMT-based.

In terms of lexical lookup, PMT can be understood as a heuristic by which a longer match overrides all shorter matches.  PMT has been widely adopted (e.g. Webster and Kit 1992; Guo 1997b) and is believed to be “the most powerful and commonly used disambiguation rule” (Chen and Liu 1992:104).

Shortest Tokenization, or “ST”, first proposed by X. Wang (1989), is a non-deterministic tokenization scheme following the Principle of Maximum Tokenization.  A segmented token string is shortest if it contains the minimum number of vocabulary words possible – “short” in the sense of the shortest word string length.

Exhaustive Tokenization, or “ET”, does not follow PMT.  As its name suggests, the ET set is the universe of all possible segmentations consisting of all candidate vocabulary words.  The mathematical definition of ET is contained in Definition 4 for “the character string tokenization operation”  in Guo (1997a).

The most important concept in Guo’s theory is Critical Tokenization, or “CT”.  Guo’s definition is based on the partially ordered set, or ‘poset’, theory in discrete mathematics (Kolman and Busby 1987).  Guo has found that different segmentations can be linked by the cover relationship to form a poset.   For example, abc|d and ab|cd both cover ab|c|d, but they do not cover each other.

Critical tokenization is defined as the set of minimal elements, i.e. tokenizations which are not covered by other tokenizations, in the tokenization poset.  Guo has given proof for a number of mathematical properties involving critical tokenization.  The major ones are listed below.

  • Every tokenization is a subtokenization of (i.e. covered by) a critical tokenization, but no critical tokenization has a true supertokenization;
  • The tokenization variations following the Principle of Maximum Tokenization proposed in the literature, such as FT, BT, FT+BT and ST, are all true sub-classes of CT.

Based on these properties, Guo concludes that CT is the precise mathematical description of the widely adopted Principle of Maximum Tokenization.

Guo (1997c) further reports his experimental studies on relative merits of these tokenization schemes in terms of three quality indicators, namely, perplexity, precision and recall.  The perplexity of a tokenization scheme gives the expected number of tokenized strings generated for average ambiguous fragments.  The precision score is the percentage of correctly tokenized strings among all possible tokenized strings while the recall rate is the percentage of correctly tokenized strings generated by the system among all correctly tokenized strings.  The main results are:

  • Both FT and BT can achieve perfect unity perplexity but have the worst precision and recall;
  • ET achieves perfect recall but has the lowest precision and highest perplexity;
  • ST and CT are simple with good computational properties.  Between the two, ST has lower perplexity but CT has better recall.

Guo (1997c) concludes, “for applications with moderate performance requirement, ST is the choice;  otherwise, CT is the solution.”

In addition to the above theoretical and experimental study, Guo (1997b) also develops a series of optimized algorithms for the implementation of these generation schemes.

The relevance and significance of Guo’s achievement to the research in this thesis lie in the following aspect.  The research on Chinese morpho-syntactic interface is conducted with the goal of  supporting Chinese morpho-syntactic parsing.  The input to a Chinese morpho-syntactic parser comes directly from the lexical lookup of the input string based on some non-deterministic tokenization scheme (W. Li 1997, 2000; Wu and Jiang 1998).  Guo’s research and algorithm development can help us to decide which tokenization schemes to use depending on the tradeoff between precision, recall and perplexity or the balance between reducing the search space and minimizing premature commitment.

1.1.2. Monotonicity Principle and Task-driven Segmentation

This section reviews the recent development on Chinese analysis systems involving the interface between morphology and syntax.  The research on the Chinese morpho-syntactic interface in this thesis echoes this new development in the field of Chinese NLP.

In the last few years, projects have been proposed for implementing a Chinese analysis system which integrates word identification and parsing.  Both rule-based systems and statistical models have been attempted with good results.

Wu (1998) has addressed the drawbacks of the conventional practice on the development of Chinese word segmenters, in particular, the problem of premature commitment in handling segmentation ambiguity.  In his A Position Statement on Chinese Segmentation, Wu proposed a general principle:

Monotonicity Principle for segmentation:

A valid basic segmentation unit (segment or token) is a substring that no processing stage after the segmenter needs to decompose.

The rationale behind this principle is to prevent premature commitment and to avoid repetition of work between modules.   In fact, traditional word segmenters are modules independent of subsequent applications (e.g. parsing).  Due to the lack of means for accessing sufficient grammar knowledge, they suffer from premature commitment and repetition of work, hence violating this principle.

Wu’s proposal of the monotonicity principle is a challenge to the Principle of Maximum Tokenization.  These two principles are not always compatible.  Due to the existence of hidden ambiguity (see 1.2.1), the PMT-based segmenters by definition are susceptible to premature commitment leading to “too-long segments”.  If the target application is designed to solve the hidden ambiguity problem in the segments, “decomposition” of some segments is unavoidable.

In line with the Monotonicity Principle, Wu (1998) proposes an alternative approach which he claims “eliminates the danger of premature commitment”, namely task-driven segmentation.  Wu (1998) points out, “Task-driven segmentation is performed in tandem with the application (parsing, translating, named-entity labeling, etc.) rather than as a preprocessing stage.  To optimize accuracy, modern systems make use of integrated statistically-based scores to make simultaneous decisions about segmentation and parsing/translation.”  The HKUST parser, developed by Wu’s group, is such a statistical system employing the task-driven segmentation.

As for rule-based systems, similar practice of integrating word identification and parsing has also been explored.  W. Li (1997, 2000) proposed that the results of an ET-based lexical lookup directly feed the parser for the hanzi-based parsing.  More concretely, morphological rules are designed to build word internal structure for productive morphology and non-productive morphology is lexicalized via entry enumeration.[1]  This approach is the background for conducting the research on Chinese morpho-syntactic interface for CPSG95 in this dissertation.

The Chinese parser on the platform of multilingual NLPWin developed by Microsoft Research also integrates word identification and parsing (Wu and Jiang 1998).  They also use a hand-coded grammar for word identification as well as for sentential parsing.  The unique part of this system is the use of a certain lexical constraint on ET in the lexical lookup phase.  This effectively reduces the parsing search space as well as the number of syntactic trees produced by the parser, with minimal sacrifice in the recall of tokenization.  This tokenization strategy provides a viable alternative to the PMT-based tokenization schemes like CT or ST in terms of the overall balance between precision, recall and perplexity.

The practice of simultaneous word identification and parsing in implementing a Chinese analysis system calls for the support of a grammar (or statistical model) which contains sufficient information from both morphology and syntax.  The research on Chinese morpho-syntactic interface in this dissertation aims at providing this support.

1.2. Morpho-syntactic Interface Problems

This section presents a series of outstanding problems in Chinese NLP which are related to the morpho-syntactic interface.  One major goal of this dissertation is to argue for the proposed analyses of the problems and to provide solutions to them based on the analyses.

Sun and Huang (1996) have reviewed numerous cases which challenge the existing word segmenters.  As many of these cases call for an exchange of information between morphology and syntax, an appropriate solution can hardly be reached within the module of a separate word segmenter.  Three major problems at issue are presented below.

1.2.1. Segmentation ambiguity

This section presents the long-standing problem in Chinese tokenization, i.e. the resolution of the segmentation ambiguity.  Within a separate word segmenter, resolving the segmentation ambiguity is a difficult, sometimes hopeless job.  However, the majority of ambiguity can be resolved when a grammar is available.

Segmentation ambiguity has been the focus of extensive study in Chinese NLP for the last decade (e.g. Chen and Liu 1992; Liang 1987;  Sproat, Shih, Gale and Chang 1996; Sun and Huang 1996; Guo 1997b).  There are two types of segmentation ambiguities (Liang 1987; Guo 1997b):  (i) overlapping ambiguity:  e.g. da-xue | sheng-huo vs. da-xue-sheng | huo as shown in (1-1) and (1-2);  and (ii) hidden ambiguity:  ge-ren vs. ge | ren, as shown in (1-3) and (1-4).

(1-1.) 大学生活很有趣
da-xue         | sheng-huo          | hen          | you-qu
university    | life                     | very          | interesting
The university life is very interesting.

(1-2.)  大学生活不下去了
da-xue-sheng                 | huo          | bu | xia-qu      | le
university student          | live           | not | down        | LEs
University students can no longer make a living.

(1-3.)  个人的力量
ge-ren         | de   | li-liang
individual   | DE  | power
the power of an individual

(1-4.) 三个人的力量
san    |  ge            | ren           | de   | li-liang
three  | CLA          | person      |DE   | power
the power of three persons

These examples show that the resolution of segmentation ambiguity requires larger syntactic context and grammatical analysis.   There will be further arguments and evidence in Chapter II (2.1) for the following conclusion:  both types of segmentation ambiguity are structural by nature and require sentential analysis for the resolution.  Without access to a grammar, no matter how sophisticated a tokenization algorithm is designed, a word segmenter is bound to face an upper bound for the precision of word identification.  However, in an integrated system, word identification becomes a natural by-product of parsing (W. Li 1997, 2000;  Wu and Jiang 1998).  More precisely, the majority of ambiguity can be resolved automatically during morpho-syntactic parsing;  the remaining ambiguity can be made explicit in the form of  multiple syntactic trees.[2]  But in order to make this happen, the parser requires reliable support from a grammar which contains both morphology and syntax.

1.2.2. Productive Word Formation

Non-listable words created via productive morphology pose another challenge (Sun and Huang 1996).  There are two major problems involved in this issue:  (i) problem in identifying lexicon-unlisted words;  (ii) problem of possible segmentation ambiguity.

One important method of productive word formation is derivation.  For example, the derived word 可读性 ke-du-xing (-able-read-ness: readability) is created via morphology rules, informally formulated below

(1-5.) derivation rules

ke + X (transitive verb) –> ke-X (adjective, semantics: X-able)

Y (adjective or verb) + xing –> Y-xing (abstract noun, semantics: Y-ness)

Rules like the above have to be incorporated properly in order to correctly identify such non-listable words.  However, there has been little research in the literature on what formalism should be adopted for Chinese morphology  and how it should be interfaced to syntax.

To make the case more complicated, ambiguity may also be involved in productive word formation.  When the segmentation ambiguity is involved in word formation, there is always a danger of wrongly applying morphological rules.  For example, 吃头 chi-tou (worth of eating) is a derived word (transitive verb + suffix tou);   however, it can also be segmented as two separate tokens chi (eat) | tou (CLA), as shown in (1-6) and (1-7) below.

(1-6.)  这道菜没有吃头
zhe    | dao           | cai            | mei-you    | chi-tou
this    | CLA          | dish         | not have   | worth-of-eating
This dish is not worth eating.

(1-7.) 他饿得能吃头牛
ta       | e               | de             | neng        | chi  | tou           | niu
he      | hungry     | DE3         | can           | eat  | CLA                   | ox
He is so hungry that he can eat an ox.

To resolve this segmentation ambiguity, as indicated before in 1.2.1, the structural analysis of the complete sentences is required.  An independent morphology system or a separate word segmenter cannot handle this problem without accessing syntactic knowledge.

1.2.3. Borderline Cases between Morphology and Syntax

It is widely acknowledged that there is a remarkable gray area between Chinese morphology and Chinese syntax (L. Li 1990; Sun and Huang 1996).  Two typical cases are described below.  The first is the phenomena of Chinese separable verbs.  The second case involves interfacing derivation and syntax.

Chinese separable verbs are usually in the form of V+N and V+V or V+A.  These idiomatic combinations are long-standing problems at the interface between compounding and syntax in Chinese grammar (L. Wang 1955; Z. Lu 1957; Lü 1989; Lin 1983; Q. Li 1983; L. Li 1990; Shi 1992; Zhao and Zhang 1996).

The separable verb 洗澡 xi zao (wash‑bath: take a bath) is a typical example.  Many native speakers regard xi zao as one word (verb), but the two morphemes are separable.  In fact, xi+zao shares the syntactic behavior and the pattern variations with the syntactic transitive combination V+NP:  not only can aspect markers appear between xi and zao,  but this structure can be passivized and topicalized as well.  The following is an example of topicalization (of long distance dependency) for xi zao.

(1-8.)(a)       我认为他应该洗澡
wo     ren-wei        ta       ying-gai       xi zao.
I         think           he      should        wash-bath
I think that he should take a bath.

(b)      澡我认为他应该洗
zao    wo     ren-wei        ta       ying-gai       xi.
bath  I         think           he      should        wash
The bath I think that he should take.

Although xi zao behaves like a syntactic phrase, it is a vocabulary word in the lexicon due to its idiomatic nature.  As a result, almost all word segmenters output xi-zao in (1-8a) as one word while treating the two signs[3] in (1-8b) as two words.  Thus the relationship between the separated use of the idiom and the non-separated use is lost.

The second case represents a considerable number of borderline cases often referred to as  ‘quasi-affixes’.  These are morphemes like 前 qian (former, ex-) in words like 前夫 qian-fu (ex-husband), 前领导 qian-[ling-dao] (former boss) and -盲 mang (person who has little knowledge of) in words like 计算机盲 [ji-suan-ji]-mang (computer layman), 法盲 fa-mang (person who has no knowledge of laws).

It is observed that ‘quasi-affixes’ are structurally not different from other affixes.  The major difference between ‘quasi-affixes’ and the few generally honored (‘genuine’) affixes like the nominalizer 性 -xing (-ness) lies mainly in the following aspect.  The former retain some ‘solid’ meaning while the latter are more functionalized.  Therefore, the key to this problem seems to lie in the appropriate way of coordinating the semantic contribution of the derived words using ‘quasi-affixes’ to the building of the semantics for the entire sentence.  This is an area which has not received enough investigation in the field of Chinese NLP.  While many word segmenters have included some type of derivational processing for a few typical affixes, few systems demonstrate where and how to handle these ‘quasi-affixes’.

1.3. CPSG95:  HPSG-style Chinese Grammar in ALE

To investigate the interaction between morphological and syntactic information, it is important to develop a Chinese grammar which incorporates morphology and syntax in the same formalism.  This section gives a brief presentation on the design and background of CPSG95 (including lexicon).

1.3.1. Background and Overview of CPSG95

Shieber (1986) distinguishes two types of grammar formalism:  (i) theory-oriented formalism;  (ii) tool-oriented formalism.  In general, a language-specific grammar turns to a theory-oriented formalism for its foundation and a tool-oriented formalism for its implementation.  The work on CPSG95 is developed in the spirit of the theory-oriented formalism Head-driven Phrase Structure Grammar (HPSG, proposed by Pollard and Sag 1987).  The tool-oriented formalism used to implement CPSG95 is the Attribute Logic Engine (ALE, developed by Carpenter and Penn 1994).

The unique feature of CPSG95 is its incorporation of Chinese morphology in the HPSG framework.[4]  Like other HPSG grammars, CPSG95 is a heavily lexicalized unification grammar.  It consists of two parts:  a minimized general grammar and an information-enriched lexicon.  The general grammar contains a small number of Phrase Structure (PS) rules, roughly corresponding to the HPSG schemata tuned to the Chinese language.[5]  The syntactic PS rules capture the subject-predicate structure, complement structure, modifier structure, conjunctive structure and long-distance dependency.  The morphological PS rules cover morphological structures for productive word formation.  In one version of CPSG95 (its source code is  shown in APPENDIX I), there are nine PS rules:  seven syntactic rules and two morphological rules.

In CPSG95, potential morphological structures and potential syntactic structures are both lexically encoded.  In syntax, a word can expect (subcat-for or mod in HPSG terms) another sign to form a phrase.   Likewise, in Chinese morphology, a morpheme can expect another sign to form a word.[6]

One important modification of HPSG in designing CPSG95 is to use an atomic approach with separate features for each complement to replace the list design of obliqueness hierarchy among complements.  The rationale and arguments for this modification are presented in Section 3.2.3 in Chapter III.

1.3.2. Illustration

The example shown in (1-9) demonstrates the morpho-syntactic analysis  in CPSG95.

(1-9.) 这本书的可读性
zhe    ben    shu    de      ke               du      xing
this    CLA   book  DE     AF:-able      read   AF:-ness
this book’s readability
(Note: CLA for classifier; DE for particle de; AF for affix.)

Figure 1 illustrates the tree structure built by the morphological PS rules and the syntactic PS rules in CPSG95

cpsgtree

Figure 1. Sample Tree Structure for CPSG95 Analysis

As shown, the tree embodies both morphological analysis (the sub-tree for ke-du-xing) and syntactic analysis (the NP structure).  The results of the morphological analysis (the category change from V to A and to N and the building of semantics, etc.) are readily accessible in building syntactic structures.

1.4. Organization of the Dissertation

The remainder of this dissertation is divided into six chapters.

Chapter II presents arguments for the need to involve syntactic analysis for a proper solution to the targeted morpho-syntactic problems.   This establishes the foundation on which CPSG95 is based.

Chapter III presents the design of CPSG95.  In particular, the expectation feature structures will be defined.  They are used to encode the lexical expectation of both morphological and syntactic structures.  This design provides the necessary means for formally defining Chinese word and the interface of morphology, syntax and semantics.

Chapter IV is on defining the Chinese word.  This is generally recognized as a basic issue in discussing Chinese morpho-syntactic interface.  The investigation leads to a way of the wordhood formalization and a coherent, system-internal definition of the work division between morphology and syntax.

Chapter V studies Chinese separable verbs.  It discusses  wordhood judgment for each type of separable verbs based on their distribution.   The corresponding morphological or syntactic solutions will then be presented.

Chapter VI investigates some outstanding problems of Chinese derivation and its interface with syntax.  It will be demonstrated that the general approach to Chinese derivation in CPSG95 works both for typical cases of derivation and the two special problems, namely ‘quasi-affix’ phenomena and zhe-affixation.

The last chapter, Chapter VII, concludes this dissertation.  In addition to a concise retrospect for what has been achieved, it also gives an account of the limitations of the present research and future research directions.

Finally, the three appendices give the source code of one version of the implemented CPSG95 and some tested results.[7]

 

————————————————–

[1] In line with the requirements by Chinese NLP, this thesis places emphasis on the analysis of productive morphology:  phenomena which are listable in the lexicon are not the major concern.  This is different from many previous works on Chinese morphology (e.g. Z. Lu 1957; Dai 1993) where the bulk of discussions is on unproductive morphemes (affixes or ‘bound stems’).

[2] Ambiguity which remains after sentential parsing may be resolved by using further semantic, discourse or pragmatic knowledge, or ‘filters’.

[3] In CPSG95 and other HPSG-style grammars, a ‘sign’ usually stands for the generalized notion of grammatical units such as morpheme, word, phrase, etc.

[4] Researchers have looked at the incorporation of morphology of other natural languages in the HPSG framework (e.g. Type-based Derivation Morphology by Riehemann 1998).  Arguments for  the inclusion of morphological features in the definition of sign will be presented in detail in Chapter III

[5] Note that ‘phrase structure’ in terms like Phrase Structure Grammar (PSG) or Phrase Structure rules (PS rules) does not necessarily refer to structures of (syntactic) phrases. It stands for surface-based constituency structure, in contrast to, say, dependency structure in Dependency Grammar.  In CPSG95, some productive morphological structures are also captured by PS rules.

[6] Note that in this dissertation, the term expect is used as a more generalized notion than the terms subcat-for (subcategorize for) and mod (modify).  ‘Expect’ is intended to be applied to morphology as well as to syntax.

[7]  There are differences in technical details between the proposed grammar in this dissertation and the implemented version.  This is because any implemented version was tested at a given time while this thesis evolved over a long period of time.  It is the author’s belief that it best benefits readers (including those who want to follow the CPSG practice) when a version was actually tested and given as was.

 

[Related]

PhD Thesis: Morpho-syntactic Interface in CPSG (cover page)

PhD Thesis: Chapter I Introduction

PhD Thesis: Chapter II Role of Grammar

PhD Thesis: Chapter III Design of CPSG95

PhD Thesis: Chapter IV Defining the Chinese Word

PhD Thesis: Chapter V Chinese Separable Verbs

PhD Thesis: Chapter VI Morpho-syntactic Interface Involving Derivation

PhD Thesis: Chapter VII Concluding Remarks

Overview of Natural Language Processing

Dr. Wei Li’s English Blog on NLP

【语义计算群:借定语的壳装状语的瓤】

看一组例子:
“洗了一个痛快的澡”
“痛快地洗了一个澡”
“洗澡洗得痛快”

t0708p

好在我们把动宾离合词“洗澡”的搭配问题解决了,定语(Mod)、状语(Adv)同是附加语(adjunct),都挂到了同样的动词“洗澡”身上了,加上部分补语(Buyu)也是附加语,可谓世界大同了。原先较真的话,要问“痛快”的是“澡”,还是“洗”,还是“洗澡”, who cares?其实都是一个意思。类似的,英语也有:
live a happy life
live (a life) happily

白:
do了一个痛快的“洗澡”
程序还是要care的。

我:
如果程序在此类情形下 只选一个路径,或不做规约也是可以的。到语义落地的时候 只要系统适应性鲁棒即可:Adv:happily OR Mod:happy。

白:
借定语的壳装状语的瓤,总要有一个地方碰上的。
“开了一个无聊的会”
工程师可以不 care,架构师必须给说法。
我要说的是,伪定语伪状语在formalism层面就是可以解决的,并不带来额外负担。

我:
do + Adjunct + core pred
已经做了相当努力去规约这些本质上相同的说法了,如前面图中的“洗澡”:Mod 也好 Adv 也好 Buyu 也好,大体属于同样性质的附加语:
adjunct 痛快 —-》 pred 洗澡

白:
“张三做出了一个追悔莫及的决定。”
“张三遇上了这个倒霉的天气。”
“倒霉的”修饰“天气”,但倒霉的不是天气。
同理,“追悔莫及的”修饰“天气”,但追悔莫及的不是天气。
修饰关系和修饰语内置的填坑关系是脱钩的。

我:
“追悔莫及” 本义 有一个 human 的坑
“做出决定” 也有一个 human 的坑
现在 human (张三) 与 “做出决定” 发生了直接联系(S) 与 “追悔莫及” 发生了间接关系(通过“做出决定”)。离开让 human (张三) 与 需要 human 坑的 “追悔莫及”直接联系 只有一步之遥了。

白:
由此可见,有了的字结构,就由“的”统一应对被修饰语。至于修饰语内部的坑由谁填,被修饰语不过只是一个普通的候选而已。选不上不勉强,有更好的候选完全可以进来。所以我对把“的”这种重要的词仅仅处理成x,是有保留看法的。

我:
“的” 是敲门砖。句法树出来了, x它意思意思,比扔掉它也许好一些。

白:
我有更好的处理办法,绝非仅是敲门砖。

我:
关键是,第一个句子是一步之遥,第二个句子是两步之遥,几乎不可能超过两步。也就是说 从ngram角度看 也不过是dag中的 bigram 或 trigram 的语义规则,如果真想做的话。只要证明从间接联系到直接联系 在语义中间件做 对应用有益处 这个工作是非常 tractable 的。
一个有语义的坑 一个正好符合语义可以填坑 近在咫尺 有何难处?给我五分钟 我两条线都可以勾搭上,而且保证不是权宜之计 不引起副作用。其所以这些语义中间件的细活 虽然不难 但并没去全做 是因为不很确定做了 到底能带来多大好处,虽然理论上是有好处的。

白:
这些后缀,几乎每个case都是一样的。

我:
要的是这个结果吗?
t0708r

白:
一点不错,就是它

我:
我做一下 regression testing 看看有无副作用,没有的话,这个 trigram 的语义填坑规则就留下来。

我:
trigram
具体到这个 case 是从线性 5-gram 缩小成 graph 的 trigram
5 与 3 在组合爆炸的考量中是天壤之别
何况完全可以造出比 5 更加远距离的同样合适的例子来 这就是句法的威力。
更主要的是,即便一个线性系统用得起 5-gram
没有结构支撑,也不敢乱用

白:
5-gram配得上的不稀疏的数据哪里来?

我:
说的是一回事儿 5gram 必然是稀疏数据 不足以支撑远距离选取。不能因为一个token需要human 另一个token恰好是human 中间隔了四个词,就可以填坑了。总之是,没有结构,这事儿就做不成。

 

【相关】

【置顶:立委NLP博文一览】

《朝华午拾》总目录

【置顶:立委NLP博文一览】

NLP University

【立委NLP相关博文汇总一览】

NLP University 开张大吉

 《朝华午拾》总目录

余致力自然语言处理(NLP,Natural Language Processing)凡30年,其目的在求交流之通畅,信息之自由,语言之归一,世界之大同。积30年之经验,深知欲达此目的,必须启蒙后进,普及科学,同心协力,共建通天之塔,因作文鼓而吹之。处理尚未成功,同志仍需努力。

 

1. 关于NLP体系及方法论

【立委科普:自然语言parsers是揭示语言奥秘的LIGO式探测仪】

泥沙龙笔记:漫谈语言形式

《泥沙龙笔记:沾深度神经的光,谈parsing的深度与多层》

【立委科普:语言学算法是 deep NLP 绕不过去的坎儿】

《OVERVIEW OF NATURAL LANGUAGE PROCESSING》

《NLP White Paper: Overview of Our NLP Core Engine》

White Paper of NLP Engine

【新智元笔记:工程语法和深度神经】

【新智元笔记:李白对话录 – RNN 与语言学算法】

《新智元笔记:再谈语言学手工编程与机器学习的自动编程》

《新智元笔记:对于 tractable tasks, 机器学习很难胜过专家》

《新智元笔记:【Google 年度顶级论文】有感》

《新智元笔记:NLP 系统的分层挑战》

《泥沙龙笔记:连续、离散,模块化和接口》

《泥沙龙笔记:parsing 的休眠反悔机制》

【立委科普:歧义parsing的休眠唤醒机制初探】

【泥沙龙笔记:NLP hard 的歧义突破】

【立委科普:结构歧义的休眠唤醒演义】

【新智元笔记:李白对话录 – 从“把手”谈起】

《新智元笔记:跨层次结构歧义的识别表达痛点》

立委科普:NLP 中的一袋子词是什么

一切声称用机器学习做社会媒体舆情挖掘的系统,都值得怀疑

立委科普:关键词革命

立委科普:关键词外传

《立委随笔:机器学习和自然语言处理》

【泥沙龙笔记:语法工程派与统计学习派的总结】

【科普小品:NLP 的锤子和斧头】

【新智元笔记:两条路线上的NLP数据制导】

《立委随笔:语言自动分析的两个路子》

Comparison of Pros and Cons of Two NLP Approaches

why hybrid? on machine learning vs. hand-coded rules in NLP

Why Hybrid?

钩沉:Early arguments for a hybrid model for NLP and IE

【李白对话录:你波你的波,我粒我的粒】

【泥沙龙笔记:学习乐观主义的极致,奇文共欣赏】

《泥沙龙笔记:铿锵众人行,parsing 可以颠覆关键词吗?》

泥沙龙笔记:铿锵三人行

《泥沙龙铿锵三人行:句法语义纠缠论》

【科普随笔:NLP主流的傲慢与偏见】

【科普随笔:NLP主流最大的偏见,规则系统的手工性】

再谈机器学习和手工系统:人和机器谁更聪明能干?

乔姆斯基批判

Chomsky’s Negative Impact

[转载]【白硕 – 穿越乔家大院寻找“毛毛虫”】

【新智元笔记:语法糖霜论不值得认真对待】

【科研笔记:NLP “毛毛虫” 笔记,从一维到二维】

【泥沙龙笔记:NLP 专门语言是规则系统的斧头】

【新智元:理论家的围墙和工程师的私货】

泥沙龙笔记:从乔姆斯基大战谷歌Norvig说起

【Church – 钟摆摆得太远(2):乔姆斯基论】

【NLP主流的反思:Church – 钟摆摆得太远(1):历史回顾】

【Church – 钟摆摆得太远(3):皮尔斯论】

【Church – 钟摆摆得太远(4):明斯基论】

【Church – 钟摆摆得太远(5):现状与结论】

《泥沙龙笔记:【钟摆摆得太远】高大上,但有偏颇》

自给自足是NLP王道

自然语言后学都应该看看白硕老师的“自然语言处理与人工智能”

语言创造简史

Notes on Building and Using Lexical Semantic Knowledge Bases

【NLP主流成见之二,所谓规则系统的移植性太差】

Domain portability myth in natural language processing (NLP)

【科普随笔:NLP的宗教战争?】

Church – 计算语言学课程的缺陷 (翻译节选)

【科普随笔:NLP主流之偏见重复一万遍成为反真理】

坚持四项基本原则,开发鲁棒性NLP系统

NLP 围脖:成语从来不是问题

NLP 是一个力气活:再论成语不是问题

立委围脖:对于用户来说,抓住老鼠就是好猫

《科普随笔:keep ambiguity untouched》

【科研笔记:NLP的词海战术】

在构筑一个模型时,枚举法是常用的必要的强盗分类

没有语言学的 CL 走不远

[转载]为什么谷歌搜索并不像广泛相信的那样主要采用机器学习?

手工规则系统的软肋在文章分类

老教授回函:理性主义回摆可能要再延迟10几年

每隔二十年振荡一次的钟摆要多长?

【系统不能太精巧,正如人不能太聪明】

《泥沙龙李白对话录:关于纯语义系统》

【泥沙龙笔记:语义可以绕过句法吗】

一袋子词的主流方法面对社交媒体捉襟见肘,结构分析是必由之路

《新智元:通用的机器人都是闹着玩的,有用的都是 domain 的》

SBIR Grants

 

2. 关于NLP分析(parsing)

语义计算沙龙:Parsing 的数据结构和形式文法

【语义计算群:句法语义的萝卜与坑】

【语义计算群:李白侃中文parsing】

【语义计算群:借定语的壳装状语的瓤】

【语义计算群:带歧义或模糊前行,有如带病生存】

【一日一parsing:”钱是没有问题”】

【一日一parsing:休眠唤醒的好例子】

【一日一parse:长尾问题种种】

【语言学小品:送老婆后面的语言学】 

【一日一parsing:NLP应用可以对parsing有所包容】

泥沙龙笔记:骨灰级砖家一席谈,真伪结构歧义的对策(1/2)

泥沙龙笔记:骨灰级砖家一席谈,真伪结构歧义的对策(2/2)

【语义计算沙龙:巨头谷歌昨天称句法分析极难,但他们最强】

语义计算沙龙:parsing 的鲁棒比精准更重要】

语义计算沙龙:基本短语是浅层和深层parsing的重要接口》

【做 parsing 还是要靠语言学家,机器学习不给力】

《泥沙龙笔记:狗血的语言学》

语义计算沙龙:关于汉语介词的兼语句型,兼论POS】

泥沙龙笔记:在知识处理中,很多时候,人不如机

《立委科普:机器可以揭开双关语神秘的面纱》

《泥沙龙笔记:漫谈自动句法分析和树形图表达》

泥沙龙笔记:语言处理没有文法就不好玩了

泥沙龙笔记:parsing 是引擎的核武器,再论NLP与搜索

泥沙龙笔记:从 sparse data 再论parsing乃是NLP应用的核武器

【立委科普:NLP核武器的奥秘】

【立委科普:语法结构树之美】

【立委科普:语法结构树之美(之二)】

【立委科普:自然语言理解当然是文法为主,常识为辅】

语义计算沙龙:从《知网》抽取逻辑动宾的关系】

【立委科普:教机器识英文】

【立委科普:及物、不及物 与 动词 subcat 及句型】

泥沙龙笔记:再聊乔老爷的递归陷阱

【泥沙龙笔记:人脑就是豆腐,别扯什么递归了】

泥沙龙笔记:儿童语言没有文法的问题

《自然语言是递归的么?》

Parsing nonsense with a sense of humor

【科普小品:文法里的父子原则】

Parent-child Principle in Dependency Grammar

乔氏 X 杠杠理论 以及各式树形图表达法

【泥沙龙笔记:依存语言学的怪圈】

【没有语言结构可以解析语义么?浅论 LSA】

【没有语言结构可以解析语义么?(之二)】

自然语言中,约定俗成大于文法教条和逻辑

泥沙龙笔记:三论世界语

泥沙龙笔记:再聊世界语及其文化

泥沙龙笔记:聊一聊世界语及老柴老乔以及老马老恩

《泥沙龙笔记:NLP component technology 的市场问题》

【泥沙龙笔记:没有结构树,万古如长夜】

Deep parsing:每日一析

Deep parsing 每日一析:内情曝光 vs 假货曝光

Deep parsing 每日一析 半垃圾进 半垃圾出

【一日一parsing: 屈居世界第零】

【研发随笔:植树为林自成景(10/n)】

【deep parsing:植树为林自成景(20/n)】

【deep parsing:植树为林自成景(30/n)】

语义计算沙龙:植树为林自成景(40/n)】

【deep parsing 吃文化:植树为林自成景(60/n)】

【deep parsing (70/n):离合词与定语从句的纠缠】

【deep parsing (80/n):植树成林自成景】

【deep parsing (90/n):“雨是好雨,但风不正经”】

【deep parsing (100/n):其实 NLP 也没那么容易气死】

 

3. 关于NLP抽取

【立委科普:NLU 的螺旋式上升及其 open知识图谱的趋向】

【语义计算沙龙:知识图谱无需动用太多知识 负重而行】

【立委科普:信息抽取】

《朝华午拾:信息抽取笔记》

泥沙龙笔记:搜索和知识图谱的话题

《知识图谱的先行:从Julian Hill 说起》

《有了deep parsing,信息抽取就是个玩儿》

【立委科普:实体关系到知识图谱,从“同学”谈起】

泥沙龙笔记: parsing vs. classification and IE

前知识图谱钩沉: 信息抽取引擎的架构

前知识图谱钩沉: 信息体理论

前知识图谱钩沉,信息抽取任务由浅至深的定义

前知识图谱钩沉,关于事件的抽取

钩沉:SVO as General Events

Pre-Knowledge-Graph Profile Extraction Research via SBIR (1)

Pre-Knowledge-Graph Profile Extraction Research via SBIR (2)

Coarse-grained vs. fine-grained sentiment extraction

【立委科普:基于关键词的舆情分类系统面临挑战】

【“剩女”的去向和出路】

SBIR Grants

 

4.关于NLP大数据挖掘

 

“大数据与认识论”研讨会的书面发言(草稿)

【立委科普:自动民调】

Automated survey based on social media

《立委科普:机器八卦》

言多必露,文本挖掘可以揭示背景信息

社媒是个大染缸,大数据挖掘有啥价值?

大数据挖掘问答2:会哭的孩子有奶吃

大数据挖掘问答1:所谓数据完整性

为什么做大数据的吹鼓手?

大数据NLP论

On Big Data NLP

作为公开课的大数据演讲

【立委科普:舆情挖掘的背后】

【立委科普:所谓大数据(BIG DATA)】

【科研笔记:big data NLP, how big is big?】

文本挖掘需要让用户既能见林又能见木

【社媒挖掘:《品牌舆情图》的设计问题】

研究发现,国人爱说反话:夸奖的背后藏着嘲讽

立委统计发现,人是几乎无可救药的情绪性动物

2011 信息产业的两大关键词:社交媒体和云计算

《扫了 sentiment,NLP 一览众山小:从“良性肿瘤”说起》

 

5. 关于NLP应用

 

【河东河西,谁敢说SMT最终一定打得过规则MT?】

【立委科普:NLP应用的平台之叹】

【Bots 的愿景】

《新智元笔记:知识图谱和问答系统:how-question QA(2)》

《新智元笔记:知识图谱和问答系统:开题(1)》

【泥沙龙笔记:NLP 市场落地,主餐还是副食?】

《泥沙龙笔记:怎样满足用户的信息需求》

立委科普:问答系统的前生今世

《新智元笔记:微软小冰,人工智能聊天伙伴(1)》

《新智元笔记:微软小冰,可能的商业模式(2)》

《新智元笔记:微软小冰,两分钟定律(3)》

新智元笔记:微软小冰,QA 和AI,历史与展望(4)

泥沙龙笔记:把酒话桑麻,聊聊 NLP 工业研发的掌故

泥沙龙笔记:创新,失败,再创新,再失败,直至看上去没失败

泥沙龙笔记:parsing 是引擎的核武器,再论NLP与搜索

立委科普:从产业角度说说NLP这个行当

【立委科普:机器翻译】

立委硕士论文【附录一:EChA 试验结果】

社会媒体(围脖啦)火了,信息泛滥成灾,技术跟上了么?

2011 信息产业的两大关键词:社交媒体和云计算

再说苹果爱疯的贴身小蜜 死日(Siri)

从新版iPhone发布,看苹果和微软技术转化能力的天壤之别

非常折服苹果的技术转化能力,但就自然语言技术本身来说 …

科研笔记:big data NLP, how big is big?

与机器人对话

《机器翻译词义辨识对策》

【立委随笔:机器翻译万岁】

 

6. 关于中文NLP

【语义计算群:李白侃中文秀parsing】

【parsing 在希望的田野上】

语义计算沙龙:其实 NLP 也没那么容易气死

【deep parsing (70/n):离合词与定语从句的纠缠】

【立委科普:deep parsing 小讲座】

【新智元笔记:词的幽灵在NLP徘徊】

《新智元笔记:机器的馅饼专砸用心者的头》

【新智元笔记:机器的馅饼(续篇)】

【新智元笔记:parsing 汉语涉及重叠的鸡零狗碎及其他】

【新智元笔记:中文自动分析杂谈】

【deep parsing:“对医闹和对大夫使用暴力者,应该依法严惩”

【让机器人解读洪爷的《人工智能忧思录》(4/n)】

【让机器人解读洪爷的《人工智能忧思录》(3/n)】

【让机器人解读洪爷的《人工智能忧思录》(2/n)】

【让机器人解读洪爷的《人工智能忧思录》(1/n)】

《新智元笔记:找茬拷问立氏parser》

【新智元笔记:汉语分离词的自动分析】

《新智元笔记:与汉语离合词有关的结构关系》

《新智元笔记:汉语使动结构与定中结构的纠缠》

《新智元笔记:汉语parsing的合成词痛点》

《新智元:填空“的子结构”、“所字结构”和“者字结构“》

【沙龙笔记:汉语构词和句法都要用到reduplication机制】

钩沉:博士阶段的汉语HPSG研究 2015-11-02

泥沙龙小品:小词搭配是上帝给汉语文法的恩赐

泥沙龙笔记:汉语牛逼,国人任性!句法语义,粗细不同

泥沙龙笔记:汉语就是一种“裸奔” 的语言

【NLP笔记:人工智能神话的背后是汗水】

【立委随笔:中文之心,如在吾庐】

汉语依从文法 (维文钩沉)

《立委科普:现代汉语语法随笔》

“自由”的语言学至少有三种理论

应该立法禁止切词研究 :=)

再谈应该立法禁止切词研究

中文处理的迷思之一:切词特有论

中文处理的迷思之二:词类标注是句法分析的前提

中文NLP迷思之三:中文处理的长足进步有待于汉语语法的理论突破

专业老友痛批立委《迷思》系列搅乱NLP秩序,立委固执己见

后生可畏,专业新人对《迷思》争论表面和稀泥,其实门儿清

突然有一种紧迫感:再不上中文NLP,可能就错过时代机遇了

社会媒体舆情自动分析:马英九 vs 陈水扁

舆情自动分析表明,谷歌的社会评价度高出百度一倍

方寒大战高频情绪性词的词频分析

方韩大战的舆情自动分析:小方的评价比韩少差太多了

研究发现,国人爱说反话:夸奖的背后藏着嘲讽

立委统计发现,人是几乎无可救药的情绪性动物

研发笔记:粤语文句的情报挖掘

《立委随笔: 语言学家是怎样炼成的》

《立委科普:汉语只有完成体,没有过去时》

《科研笔记:中文图灵试题?》

立委统计发现,汉语既适合吹嘘拍马亦长于恶意构陷

比起英语,汉语感情更外露还是更炽烈?

科研笔记:究竟好还是不好

《科普随笔:汉字和语素》

《科普随笔:汉语自动断词 “一次性交500元”》

《科普随笔:“他走得风一样地快” 的详细语法结构分析》

【立委科普:自动分析 《偉大的中文》】

《立委随笔:汉语并不简单》

语言学小品:结婚的远近距离搭配

中文处理的模块化纠结

【立委科普:《非诚勿扰》中是谁心动谁动心?】

曙光在眼前,轻松过个年

挺反自便,但不要欺负语言学!

当面对很烦很难很挑战的时候

创造着是美丽的

汉语依从文法 (维文钩沉)

《新智元:挖掘你的诗人气质,祝你新年快乐》

 

7. 关于NLP社会媒体舆情挖掘的实践

 

【语义计算沙龙:sentiment 中的讽刺和正话反说】

【喋喋不休论大数据(立委博文汇总)】

【新智元笔记:再谈舆情】

舆情挖掘系统独立验证的意义

【社煤挖掘:雷同学之死】

《利用大数据高科技,实时监测美国总统大选舆情变化》

世人皆错nlp不错,民调错大数据也不会错

社媒大数据的困境:微信的风行导致舆情的碎片化

从微信的用户体验谈大数据挖掘的客户情报

社媒挖掘:社会媒体疯传柴静调查,毁誉参半,争议趋于情绪化

奥巴马赢了昨晚辩论吗?舆情自动检测告诉你

全球社交媒体热议阿里巴巴上市

到底社媒曲线与股市曲线有没有、有多少相关度?

再谈舆情与股市的相关性

【『科学』预测:A-股 看好】

舆情挖掘用于股市房市预测靠谱么?

大数据帮助决策实例:《走进“大数据”——洗衣机寻购记》

【社媒挖掘:外来快餐店风光不再】

【社媒挖掘:中国手机市场仍处于战国争雄的阶段】

世界杯是全世界的热点,纵不懂也有义务挖掘一哈

【大数据挖掘:方崔大战一年回顾】(更正版)

【大数据挖掘:转基因一年回顾】

【大数据挖掘:“苦逼”小崔2013年5-7月为什么跌入谷底?】

【大数据挖掘:转基因中文网络的自动民调,东风压倒西风?】

【大数据挖掘:转基因英文网络的自动民调和分析】

只认数据不认人:IRT 的鼓噪左右美国民情了么?

继续转基因的大数据挖掘:谁在说话?发自何处?能代表美国人民么

关于转基因及其社会媒体大数据挖掘的种种问题

【美国网民怎么看转基因:英文社交媒体大数据调查告诉你】

【社媒挖掘:必胜客是七夕节情侣聚餐的首选之地?】

【社媒挖掘:大数据时代的危机管理】

测试粤语舆情挖掘:拿娱乐界名人阿娇和陈冠希开刀

【社媒挖掘:不朽邓丽君】

【社媒挖掘:社会媒体眼中的李开复老师】

【社媒挖掘:糟糕透顶的方韩社会形象】

社媒挖掘:关于狗肉的争议

社媒挖掘:央视的老毕

社媒挖掘:老毕私下辱毛事件再挖掘

大数据淹没下的冰美人(之一)

大数据淹没下的冰美人(之二)

大数据淹没下的冰美人(之三): 喜欢的理由

大数据淹没下的冰美人(之四): 流言蜚语篇(慎入)

大数据淹没下的冰美人(之五): 星光灿烂谁为最?

【社媒挖掘:成都暴打事件中的男司机和女司机】

【社媒挖掘:社会媒体眼中的陳水扁】

【社媒挖掘:社会媒体眼中的李登輝】

【社媒挖掘:馬英九施政一年來輿情晴雨表】

【社媒挖掘:臺灣政壇輿情圖】

【社媒挖掘:社会媒体眼中的臺灣綠營大佬】

舆情挖掘:九合一國民黨慘敗 馬英九時代行將結束?

社会媒体舆情自动分析:马英九 vs 陈水扁

社媒挖掘:争议人物方博士被逐,提升了其网路形象

方韩大战高频情绪性词的词频分析

方韩大战的舆情自动分析:小方的评价比韩少差太多了

社媒挖掘:苹果CEO库克公开承认同志身份,媒体反应相当正面

苹果智能手表会是可穿戴设备的革命么?

全球社交媒体热议苹果推出 iPhone 6

互联网盛世英雄马云的媒体形象

革命革到自身头上,给咱“科学网”也挖掘一下形象

两年来中国红十字会的社会媒体形象调查

自动民调Walmart,挖掘发现跨国公司在中国的日子不好过

【社媒挖掘:“剩女”问题】

【舆情挖掘:2013央视春晚播后】

【舆情挖掘:年三十挖一挖央视春晚】

新浪微博下周要大跌?舆情指数不看好,负面评价太多(疑似虚惊)

【大数据挖掘:微信(WeChat)】

【大数据解读:方崔大战对转基因形象的影响】

【微博自动民调:薄熙来、薛蛮子和李天一】

【社媒挖掘:第一夫人光彩夺目赞誉有加】

Chinese First Lady in Social Media

Social media mining on credit industry in China

Sina Weibo IPO and its automatic real time monitoring

Social media mining: Teens and Issues

立委元宵节大数据科技访谈土豆视频上网

【大数据挖掘:中国红十字会的社会媒体形象】

【社媒挖掘:社会媒体眼中的财政悬崖】

【社媒挖掘:美国的枪支管制任重道远】

【舆情挖掘:房市总体看好】

【社媒挖掘:社会媒体眼中的米拉先生】

【社会媒体:现代婚姻推背图】

【社会媒体:现代爱情推背图】

【科学技术之云】

新鲜出炉:2012 热点话题五大盘点之五【小方vs韩2】

【凡事不决问 social:切糕是神马?】

Social media mining: 2013 vs. 2012

社会媒体测试知名品牌百度,有惊人发现

尝试揭秘百度的“哪里有小姐”: 小姐年年讲、月月讲、天天讲?

舆情自动分析表明,谷歌的社会评价度高出百度一倍

圣诞社媒印象: 简体世界狂欢,繁體世界分享

WordClouds: Season’s sentiments, pros & cons of Xmas

新鲜出炉:2012 热点话题五大盘点之一【吊丝】

新鲜出炉:2012 热点的社会媒体五大盘点之二【林书豪】

新鲜出炉:2012 热点话题五大盘点之三【舌尖上的中国】

新鲜出炉:2012 热点话题五大盘点之四【三星vs苹果】

社会媒体比烂,但国骂隐含舆情

肮脏语言研究:英语篇

肮脏语言研究:汉语篇(18岁以下勿入)

新年新打算:【社媒挖掘】专栏开张大吉

 

8. 关于NLP的掌故趣闻

《朝华午拾:创业之路》

《朝华午拾 – 水牛风云》

《朝华午拾:用人之道》

《朝华午拾:欧洲之行》

《朝华午拾:“数小鸡”的日子》

《朝华午拾:一夜成为万元户》

《朝华午拾:世界语之恋》

《朝华午拾:我的考研经历》

80年代在国内,社科院的硕士训练使我受益最多

科研笔记:开天辟地的感觉真好

《朝华午拾:今天是个好日子》

【朝华午拾:那天是个好日子】

10 周年入职纪念日有感

《立委随笔: 语言学家是怎样炼成的》

说说科研立项中的大跃进

围脖:一个人对抗一个世界,理性主义大师 Lenat 教授

《泥沙龙笔记:再谈 cyc》

围脖:格语法创始人菲尔墨(Charles J. Fillmore)教授千古!

百度大脑从谷歌大脑挖来深度学习掌门人 Andrew Ng

冯志伟老师以及机器翻译历史的一些事儿

《立委随笔:微软收购PowerSet》

NLP 历史上最大的媒体误导:成语难倒了电脑

立委推荐:乔姆斯基

巧遇语言学上帝乔姆斯基

[转载]欧阳锋:巧遇语言学新锐 - 乔姆斯基

【科普小品:伟哥的关键词故事】

不是那根萝卜,不做那个葱

【随记:湾区的年度 NLP BBQ 】

女怕嫁错郎,男怕入错行,专业怕选错方向

据说,神奇的NLP可以增强性吸引力,增加你的信心和幽会成功率

【立委科普:美梦成真的通俗版解说】

【征文参赛:美梦成真】

【创业故事:技术的力量和技术公司的命运】

把酒话桑麻,再泡一壶茶,白头老机译,闲坐说研发

MT 杀手皮尔斯 (翻译节选)

ALPAC 黑皮书 1/9:前言

《眼睛一眨,来了王子,走了白马》

职业随想曲:语言学万岁

立委随笔:Chomsky meets Gates

钩沉:《中国报道》上与导师用世界语发表的第一篇论文

钩沉:《中国报道》上用世界语发表的第二篇论文

贴身小蜜的面纱和人工智能的奥秘

有感于人工智能的火热

泥沙龙笔记微博议摘要

【泥沙龙笔记:没有结构树,万古如长夜】

【泥沙龙笔记:机器 parsing 洪爷,无论打油或打趣】

老革命遇到新问题,洪爷求饶打油翁

我要是退休了,就机器 parse 《离骚》玩儿


【语言学小品:送老婆后面的语言学】

456822675539882531

谁会误读?为什么误读?研究一下背后的语言学 and beyond。

双宾两个坑 human 默认的坑是对象 “老婆”是“送”的对象,这是正解。
对于心术不正的人 human 也可以填受事的坑,“老婆”跟礼物一样,成了“送”的受事。
这是 “送” 的歧义,到了 caption 里面的合成词 “送给”,subcat 有细微变化,就没歧义了。为什么 “送-个” 也没歧义呢?因为“个”是不定的,而对象这个角色通常是有定的。
这里面细说起来还有一摞的语言学。

(1)双宾句型的对象一般是有定的,不定的对象不是绝对不可以,譬如:
“我把一大批书送(给)一所学校了。”
“一所” 是不定数量词,作为对象。
汉语中的 “一+量词”与光杆“量词”通常认为是等价的,范畴都是不定(indefinite),后者是前者省略了“一”而得。但是二者并非完全等价。
对象这个角色默认有定(definite,虽然汉语没有定冠词),如果是有定,不可以省略“一”,或者说,不可以由带光杆量词的NP充当。
汉语句法里面可以总结出这么一条细则:带有光杆量词的NP只能充当直接宾语,不能充当间接宾语(对象)或其他。

(2)再看合成词 “送给” 里面的语言学。
汉语反映双宾概念的语词,常常可以进一步与“给”组成合成动词,意义不变,但注意合成前后的subcat的微妙变化:“送” vs “送给” (寄给,赠给,赠送给,等)
“送”的 subcat patterns:
(1) 送 + 对象NP + 受事NP: 送她一本书
(2) “把”受事NP+送+对象: 把一本书送她
(3)受事NP+送+对象: 这本书送她了
(4)送+受事NP: 送个老婆
(5)送+对象NP(human,definite):送(我)老婆。

请留心(4)和(5):两个patterns有相交竞争的时候,于是歧义产生。当“送+给”构成合成动词后,subcat 的 patterns(1)(2)(3)(5) 保持不变,而(4)基本失效(退出)了。说基本失效,是因为:虽然 “送给老婆”只能循 pattern 5,但“送给个老婆”(稍微有限别扭,但仍在语言可接受之列)似乎仍然需要理解为 pattern 4,这是怎么回事呢?
这就是语言的微妙之处:pattern 4 本来应该退出,因为“给”已经决定了后面是对象而不是受事;但是因为汉语有另一条很细但是很强的规则说,光杆量词的NP只能做受事,不能做对象或其他。在这两条规则(pattern 5的对象规则与光杆受事规则)发生冲突的时候,后一条胜,因此“送给个老婆”就不得不做 pattern 4 的受事解了。这叫规则与规则打架,谁胜谁输也是语言学的一部分,电脑实现的时候可以运用一个priority的机制来model。

上图还涉及一个常见的促销句式: 买NP1送NP2
买iPhone 6 送耳机
买 Prius 送三年保修
这个语用句式的存在,加强了NP2作为受事的可能性,使得 human 本来默认为对象的力量受到制衡。这似乎涉及语用与句法的交界了。

这些算是语言学。Beyond 语言学,也可以从文化上看这个误解或歧义的现象:

对于来自落后农村的人,老婆作为受事的理解几乎是理所当然,因为农村的封建落后使得娶不起媳妇的光棍汉太多,白捞一个媳妇的渴望诱使他们更多向受事而不是对象方面联想,何况手机对于他们是天价,卖肾才可得之,因此对于促销句式也就更加敏感。反之,对于一个知识分子或富裕阶层人士,“送老婆”可能更偏向于理解为对象。

就跟王若水老老年谈桌子的哲学类似,这则小品主要是想谈谈日常的语言学。哲学家满眼都是哲学,语言学家以语言学看世界。语言人人会说,背后的语言学却不是老妪能解。语言如水如空气,一般人熟视无睹了,语言学家来揭示。这是 real life linguistics,琐碎而不乏规律,似海却仍可见底。

【相关】

《立委随笔: 语言学家是怎样炼成的》

《朝华午拾》总目录

【关于立委NLP的《关于系列》】

【置顶:立委NLP博文一览(定期更新版)】

立委NLP频道