Lab · Word map词图实验 · 词图
When was “these words are close” drawn by hand?“这些词很近”是什么时候亲手画上的?
The press officer scores the same two headlines a second way before writing “the coverage agrees.” Counts only notice shared spelling, so “raised” and “lowered” barely change the score. On this page each content word has a place that was typed in: “raised” sits opposite “lowered,” and “pay” sits near “earnings.” The map score therefore drops on the opposite-claim pair and rises on the different-words pair. A trained embedding would learn those places from a large collection of headlines. These places were typed.新闻官在写下“报道口径一致”之前,用第二种方法给同样两则标题打分。计数只看见相同的拼写,所以 raised 和 lowered 几乎不改变分数。这一页每个实词有一个敲进去的位置:raised 在 lowered 对面,pay 靠近 earnings。于是地图分数在相反主张那一对下降,在用词不同那一对上升。训练出来的嵌入会从大量标题里学习这些位置。这些位置是敲进去的。
Why this matters.为何重要。 A briefing can quote whichever score looks convenient. The count score says the wage lines agree when one verb has flipped the story. The map score says two different wordings are close, because the map was drawn that way. Neither score was learned from a newspaper archive.简报可以引用看起来顺手的那个分数。计数分数会说两则工资新闻一致,而一个动词已经把故事说反了。地图分数会说两种措辞很近,因为地图就是这样画的。两个分数都不是从报纸档案里学来的。
Try this试一试 Stay on “Opposite claims” and compare the two numbers. Then move to “Different words.” The count and the map should trade places.停在“相反的主张”,比较两个数。再移到“用词不同”。计数和地图应该对调。
A.A. The new training program raised wages for the workers in the plant.
B.B. The new training program lowered wages for the workers in the plant.
Read the chart读这张图
If you misread this读错了会怎样
If the press officer tells the editor a high map score means a model understood the headlines, the editor may print “coverage agrees.” The marks sit close because pay was typed near earnings. Those places were not learned from a pile of headlines.如果新闻官告诉编辑,地图分数高表示模型读懂了标题,编辑可能会印出“报道口径一致”。标记靠在一起,是因为 pay 被敲在 earnings 附近。这些位置不是从一堆标题里学来的。
Enough, and not enough够用,以及不够
Enough to see the word-count score and the map score disagree on the same two headlines.够用来看见:词数分数和地图分数在同样两则标题上不一致。
Not enough for a note that says the coverage agrees. It still needs places learned from headlines you can name, and a check that those places separate “pay rose” from “pay fell.”不够写一份说报道口径一致的说明。说明仍需要从你能点名的标题里学来的位置,并检查这些位置把“工资涨了”和“工资降了”分开。
Equations, assumptions, and limits方程、假设与限度
A sentence vector is the average of the coordinates of the words that have one. Cosine similarity is the angle between the two averages. The count cosine is the bag-of-words cosine from the other lab, with function words kept. The two scores answer different questions because they use different representations.句子向量是有坐标的那些词的坐标平均。余弦是两个平均之间的夹角。计数余弦是另一个实验里的词袋余弦,功能词保留。两个分数回答不同的问题,因为表示不同。
A trained embedding would place words by a learning rule on a large collection of headlines. This page types the places directly. Words with no coordinate are ignored.训练出来的嵌入会按大量标题上的学习规则放置词语。这一页直接写下位置。没有坐标的词不计入。
Common pitfall.常见陷阱。 A high vector cosine is easy to read as “the model understands the sentences.” Here the coordinates were chosen so that some synonyms sit together and one contrasting pair sits apart. Change the coordinates and the ranking of the pairs can change, with the sentences held fixed. Another pitfall is to average a sentence and then talk about a single word. The mark is a mixture.向量余弦高,很容易被读成“模型理解了这些句子”。这里的坐标是选定的,让一些同义词靠在一起,并让一对反义词分开。改坐标,句子不动,配对的排序也可以变。另一个陷阱是把句子平均之后还当成在谈论某一个词。那个标记是一个混合。