After a case in which the reference group (150 respondents) was smaller than the batch under examination (210): the functions refused to run, and with subsamples of 100 the level-matched control samples came out almost identical (the same gap in every sample), so the p values overstated the evidence.
compare_batches() and neighbor_test() gain
min_ratio (default 3). When the reference has fewer than
min_ratio times the focal respondents, including a
reference smaller than the batch, each sample takes a random subsample
of the batch of floor(n_reference / min_ratio) respondents
and pairs the reference with it. The focal value is the median over the
focal subsamples (focal_gap_sample in
neighbor_test()). min_ratio = 1 gives the
former behaviour whenever the reference is at least as large as the
batch.settings$matched_overlap) and warn when it is above 80%:
the control then barely varies. In the case above the share was 100%
with the whole reference, 87% with min_ratio = 2 and 63%
with min_ratio = 3; with 5 the focal samples had 30
respondents and the near-duplicate test lost its power, hence the
default of 3.compare_batches()$tests gains n_sample,
the size of the samples compared.Tools to judge a whole batch of data, after a case in which 40 late answers of a survey looked suspicious (many 4s, weak item correlations) and turned out to be explained by their level and by the flat responders of the first batch.
compare_batches(): compares the item correlation
structure of a batch with a reference group by permutation (mean
inter-item correlation and the within-minus-between-scales structure),
after leaving out flat responders (drop_flat) and against
reference samples paired by response level (match_level);
adds the Jennrich (1970) test. plot() types
"null", "heatmap" and
"structure".order_test(): are consecutive rows (items and,
optionally, demographic profile columns) more alike than in
a random order? Detects rows copied from the previous one without a
submission time. plot() types "null" and
"sequence".neighbor_test(): distance to the most similar
respondent, against data shuffled by item and against level-matched
control samples; returns the closest pairs. plot() types
"null" and "distance".scan_responses() gains the even-odd consistency index
and flag (evenodd, evaluated with
eo_min_scales = 6 scales or more), and the
reference and ref_quantile arguments: the
cut-offs of low_irv, mahal,
psychsyn and evenodd are then taken from a
reference group without flat responders.flat is judged only in scales of
flat_min_items (6) items or more, or in all items together
when no scale is that long. In three- or four-item scales many genuine
respondents give the same answer everywhere.batch_sim: two phases, with a
hand-filled block in the second one.Adjustments after screening two short one-factor scales (7 and 10 items) in a large real sample (N = 4911), where three criteria flagged far too many people.
psychsyn uses disjoint pairs: each item enters one pair
only, strongest first. In a short one-factor scale almost every pair
passes syn_r and the pairs reused the same items, so the
within-person correlation was noise (45 % flagged). New argument
syn_min_pairs (default 5): with fewer disjoint pairs the
criterion is reported as not evaluable.duplicate is evaluated only with at least
dup_min_items items (default 15). With few items, genuine
respondents share answers by chance (28-45 % flagged in the real case).
Vectors that are flat in every scale are no longer counted as
duplicates; they are already flagged by flat.low_irv is computed among respondents with some
variability (IRV > 0). Before, when more than 5 % of the sample
responded flat the cut fell to 0 and low_irv repeated
flat exactly.exclude_responses() warns, and
compare_fit() adds a note, when a rule removes more than
half of the sample.compare_fit() adds a note when an exclusion removes
respondents for their Zh: Zh is estimated from the same items, so part
of the gain in fit is circular.plot() returns the ggplot object invisibly, so it can be
modified or saved with ggsave(); new argument
base_size.plot.response_scan() and of
plot.fit_comparison() explains how to read each view.exclude_responses(): returns the data without the
respondents that meet the chosen flags, patterns, number of flags or
rows, ready for any model.compare_fit(): fits the same CFA (lavaan syntax) or
EFA with each exclusion and compares fit indices, mean loading and
factor correlations; improper solutions are reported.
print() and plot() methods.plot() gains five views: "days",
"overlap", "profiles", "styles"
and "scatter".as.data.frame() prefixes the flags with
flag_ so that they do not clash with the index of the same
name.scan_responses() with
print(), summary(),
as.data.frame() and plot() methods (overview,
indices, patterns, heatmap and time), and the simulated data set
survey_sim.