corpustools: Managing, Querying and Analyzing Tokenized Text

Provides text analysis in R, focusing on the use of a tokenized text format. In this format, the positions of tokens are maintained, and each token can be annotated (e.g., part-of-speech tags, dependency relations). Prominent features include advanced Lucene-like querying for specific tokens or contexts (e.g., documents, sentences), similarity statistics for words and documents, exporting to DTM for compatibility with many text analysis packages, and the possibility to reconstruct original text from tokens to facilitate interpretation.

Version:	0.5.2
Depends:	R (≥ 3.5.0)
Imports:	methods, wordcloud (≥ 2.5), stringi, Rcpp (≥ 0.12.12), R6, udpipe (≥ 0.8.3), digest, data.table (≥ 1.10.4), quanteda (≥ 1.5.1), igraph, tokenbrowser (≥ 0.1.5), RNewsflow (≥ 1.2.1), Matrix (≥ 1.2), parallel, pbapply (≥ 1.4), rsyntax (≥ 0.1.1)
LinkingTo:	Rcpp, RcppProgress
Suggests:	testthat, tm (≥ 0.6), topicmodels, knitr, rmarkdown
Published:	2025-07-07
DOI:	10.32614/CRAN.package.corpustools
Author:	Kasper Welbers [aut, cre], Wouter van Atteveldt [aut]
Maintainer:	Kasper Welbers <kasperwelbers at gmail.com>
License:	GPL-3
URL:	https://github.com/kasperwelbers/corpustools
NeedsCompilation:	yes
Materials:	README, NEWS
CRAN checks:	corpustools results

Documentation:

Reference manual:	corpustools.html , corpustools.pdf
Vignettes:	Introduction to corpustools (source, R code)

Downloads:

Package source:	corpustools_0.5.2.tar.gz
Windows binaries:	r-devel: corpustools_0.5.2.zip, r-release: corpustools_0.5.2.zip, r-oldrel: corpustools_0.5.2.zip
macOS binaries:	r-release (arm64): corpustools_0.5.2.tgz, r-oldrel (arm64): corpustools_0.5.2.tgz, r-release (x86_64): corpustools_0.5.2.tgz, r-oldrel (x86_64): corpustools_0.5.2.tgz
Old sources:	corpustools archive

Reverse dependencies:

Reverse imports:	text2sdg
Reverse suggests:	LexisNexisTools

Linking:

Please use the canonical form https://CRAN.R-project.org/package=corpustools to link to this page.