Paper Reading - A Joint Many-Task Model, Growing a Neural Network for Multiple NLP Tasks

Problem Perform multi-tasks in a hierarchical manner. Train a multi-layer model for multitasks. Different layers handle different tasks, from morphology, syntax to semantics. Key Ideas Different layers handle different tasks. Low-level layer handle easy task,... Continue

NLP Tasks

Introduction Here we summarize some natural language processing tasks. Compare their semantic levels, difficulty, and similarity. We rank and group them into a table. NLP Tasks Task Input Type Word tagging     Word segmentation... Continue

NLP Pretraining - from BERT to XLNet

Introduction The emergence of BERT brought NLP into a new era. Recent research works usually apply a similar “pre-training + finetuning” manner. In this post, we briefly summarize recent works after BERT. Some of them... Continue

Paper Reading - A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks

问题 训练层级多任务。 关键想法 根据任务关系组合不同的任务,简单任务在下,复杂在上,下层任务对上层进行辅助。这里选择NER,EMD,CR,RE四个任务,分三层。NER-〉EMD-〉CR和RE 模型 结构 Embedding: character emb + Glove + ELMo Encoder: Multi-layer BiLSTM for each task Decoder: CRF for NER and EMD, Scorer for CR and RE 数据 利用标注好的不同训练数据 训练 随机选择任务,选择batch 技巧... Continue