TermExtractor

Entry point for the unsupervised methods. Choose a method with method=...; see the unsupervised extractors for what each one does.

TermExtractor(method='tfidf', alpha=0.5, beta=0.5, threshold=None, n=1, num_topics=20, top_words=200, window_size=5, seed_terms=200, context_words=50, stoplist=None, phrase_delimiters=None, reference_corpus=None, tie_break='ratio', n_topics=10, max_features=None, stopwords='english', words_layer=None, lemma_layer=None, min_frequency=1, word_pattern='alphanumeric', pos_pattern=None, pos_layer=None)

A wrapper class for selecting the term extraction method.

TermExtractor.METHODS maps each method name to its extractor class and the parameters passed to it, so list(TermExtractor.METHODS) lists the available methods.

Initialize the TermExtractor with the desired method and parameters.

Each method only uses some of the parameters; the others are ignored.

Parameters:

Name	Type	Description	Default
method	str	The term extraction method, one of the keys of TermExtractor.METHODS (e.g. 'tfidf', 'cvalue', 'rake', 'weirdness').	'tfidf'
alpha	float	Weight for ( e_t ) ('basic' and 'combobasic').	0.5
beta	float	Weight for ( e'_t ) ('combobasic').	0.5
threshold	Optional[float]	Minimum score threshold for term inclusion.	None
n	int	Maximum length of n-grams to consider (all methods except 'rake').	1
num_topics	int	Number of LDA topics ('topicmodeling').	20
top_words	int	Number of most probable words of each topic that count towards the scores ('topicmodeling').	200
window_size	int	The number of words on either side of a candidate that count as its context ('domaincoherence').	5
seed_terms	int	The number of best candidates by Basic used to choose the context words ('domaincoherence').	200
context_words	int	The number of context words that candidates are scored against ('domaincoherence').	50
stoplist	Optional[List[str]]	List of stop words (required by 'rake').	None
phrase_delimiters	Optional[List[str]]	List of phrase delimiters (required by 'rake').	None
reference_corpus	Optional[Union[Corpus, List[str], List[Union[Corpus, List[str]]]]]	General-language reference corpus, required by 'domainpertinence', 'weirdness' and 'relevance'. It must have the same type as the corpus passed to extract. A list of reference corpora is compared one by one by 'domainpertinence', and read as one corpus by the others.	None
tie_break	Optional[str]	How to order the candidates that tie at 1 ('domainpertinence'): "ratio", "frequency" or None.	'ratio'
n_topics	int	Number of NMF topics ('nmf').	10
max_features	Optional[int]	Maximum vocabulary size ('nmf').	None
stopwords	Union[str, Iterable[str], None]	Drop candidates that start or end with a stopword (all methods except 'rake', which uses stoplist): the name of a built-in list in termxtract.stopwords.STOPWORD_LISTS, such as "english" (the default) or an ISO 639-1 code such as "fr", a list of words, or None to keep every candidate. Stopwords inside a multi-word term, as in "degree of freedom", are kept.	'english'
words_layer	Optional[str]	For Teanga input, the span layer holding the words: None (the default) for words or, if the corpus has no words layer, tokens.	None
lemma_layer	Optional[str]	For Teanga input, a sequence layer, based on the words layer, holding the lemma of each word (all methods except 'rake'). If given, words are read as their lemmas, so inflected forms such as "cell" and "cells" are counted, and returned, as one term. None (the default) reads the words as written.	None
min_frequency	int	Drop candidates that occur fewer times than this in the corpus (all methods except 'rake'). ATR4S uses 2. The default of 1 keeps every candidate.	1
word_pattern	Optional[str]	Drop candidates with a word that does not fully match this regular expression (all methods except 'rake'), or the name of a pattern in termxtract.candidates.WORD_PATTERNS: "alphanumeric" (the default) drops words without a letter or digit, such as "%", and "atr4s" keeps words of three or more letters, digits and hyphens, as ATR4S's noise filter does. None keeps every word.	'alphanumeric'
pos_pattern	Optional[str]	For Teanga input, count only the occurrences of candidates whose part-of-speech tags match this pattern, so a term must match it in at least one occurrence. Tags are written in angle brackets, as in "<ADJ|NOUN>*<NOUN>", or use the name of a pattern in termxtract.pos.POS_PATTERNS, such as "noun_phrase". None (the default) allows every tag sequence. RAKE's terms are filtered after scoring instead, so its scores are unchanged.	None
pos_layer	Optional[str]	The part-of-speech layer used by pos_pattern, based on the words layer: None (the default) for upos or, if the corpus has no upos layer, pos.	None

Raises:

Type	Description
ValueError	If the method is unknown, or a parameter it requires is missing.

extract(corpus)

Extract terms from a given corpus.

Parameters:

Name	Type	Description	Default
corpus	Union[Corpus, List[str]]	Either a Teanga Corpus object or a list of strings (target corpus).	required

Returns:

Name	Type	Description
ATEResults	ATEResults	An object containing the corpus, terms, and offsets (if applicable).

Raises:

Type	Description
ValueError	If the corpus type is not supported or if a required reference corpus is missing.