A POS Tagger for Social Media Texts trained on Web Comments

Authors

Abstract

Using social media tools such as blogs and forums have become more and more popular in recent years. Hence, a huge collection of social media texts from different communities is available for accessing user opinions, e.g., for marketing studies or acceptance research. Typically, methods from Natural Language Processing are applied to social media texts to automatically recognize user opinions. A fundamental component of the linguistic pipeline in Natural Language Processing is Part-of-Speech tagging. Most state-of-the-art Part-of-Speech taggers are trained on newspaper corpora, which differ in many ways from non-standardized social media text. Hence, applying common taggers to such texts results in performance degradation. In this paper, we present extensions to a basic Markov model tagger for the annotation of social media texts. Considering the German standard Stuttgart/Tübinger TagSet (STTS), we distinguish 54 tag classes. Applying our approach improves the tagging accuracy for social media texts considerably, when we train our model on a combination of annotated texts from newspapers and Web comments.

Keywords

Natural language processing, part-of-speech tagging, opinion mining, German

Ressources

Polibits 48 -- Research journal on Computer science and computer engineering with applications.

BibT_EX Reference Entry

@article{NeReMa13,
	author = {Melanie Neunerdt and Michael Reyer and Rudolf Mathar},
	title = "A POS Tagger for Social Media Texts trained on Web Comments",
	pages = "61-68",
	journal = "Polibits",
	volume = "48",
	month = Dec,
	year = 2013,
	hsb = hsb999910325580 ,
	}

Downloads

_{Download paper} _{Download bibtex-file}

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights there in are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Institute for Theoretical Information Technology