Tag

Conditional Random Field

3 posts

Conditional random fields (CRFs) are a class of statistical modelling method often applied in pattern recognition and machine learning, where they are used for structured prediction. Whereas an ordinary classifier predicts a label for a single sample without regard to "neighboring" samples, a CRF can take context into account; e.g., the linear chain CRF popular in natural language processing predicts sequences of labels for sequences of input samples. CRFs are a type of discriminative undirected probabilistic graphical model. It is used to encode known relationships between observations and construct consistent interpretations. It is often used for labeling or parsing of sequential data, such as natural language text or biological sequences and in computer vision. Specifically, CRFs find applications in shallow parsing, named entity recognition, gene finding and peptide critical functional region finding, among other tasks, being an alternative to the related hidden Markov models (HMMs). In computer vision, CRFs are often used for object recognition and image segmentation.

Code

ParsCit 一个开源的CRF参考文献解析包

译自:ParsCit: An open-source CRF reference string parsing package (部分)

0 摘要

我们这样描述ParsCit:一个自由,开源的参考文献解析包。ParsCit的核心是一个被训练完成的CRF模型,它被用来标记参考文献字符串的序列。这个启发式的模型对一个纯文本文件进行分割和识别。这个包可以直接运行,或者作为一个web服务器使用。

我们用3个不同的数据集来比较ParsCit,并用它和之前已出版的作品比较。

Code

CRF++ 的测试结果统计

crf++ 是一个开源工具,是自然语言处理的一把利器。它可以编译为 crf_learn 和 crf_test 两个 bin 文件。测试的结果是将预测的结果和真实的结果进行比对。可惜这东西比对完也不给个统计,真教人不爽。 幸亏,有人做了个 perl 工具 conlleval 可以用来分析测试结果。

Code

ParsCit: An open-source CRF reference string parsing package

当今世界,文献资料的数量几乎是以指数级别的增长,而文献的传播过程中,往往并非仅仅通过电子版的数据传输. 由于各种条件限制,文献数据有时候甚至是Word->PDF->纸质打印资料->扫描进文件->再次整理资料这样螺旋式的传播. 而在再次整理资料的时候,往往不免要遇到这样或者那样的奇怪的数据,比如1变成了i,有些数据彻底错位等等. ParsCit ,一个开源的package,正是试图通过CRF的分析方法,将文献后的reference再次整理为有序的数据.