Package {growthcleanr}

growthcleanr: a data cleaner for anthropometric measurements

Description

growthcleanr flags implausible anthropometric (height, weight, and head circumference) measurements in longitudinal datasets, such as those from electronic health records. It also identifies other measurements that should often be excluded from analyses.

Getting started

The main function is cleangrowth. Pass it a data frame with one row per measurement and columns subjid, param, agedays, sex, and measurement (plus an optional per-row id). Semi-wide and wide layouts, alternate column names, and date-of-birth / visit-date ages are also supported. growthcleanr will not delete or edit data: it returns your data with an added exclude column labeling each row "Include" or an exclusion code naming why it recommends exclusion.

The function growthcleanr_app opens a Shiny app that runs growthcleanr without writing code. A command-line driver is documented in vignette("command-line").

New to growthcleanr? Start with vignette("new-start-here"). See vignette("output") for more information including the returned columns and vignette("configuration") for the full index of arguments and thresholds.

Author(s)

Maintainer: Carrie Daymont cdaymont@pennstatehealth.psu.edu [copyright holder]

Authors:

• Robert W Grundmeier

• Jeffrey Miller

• Diego Campos

• Dan Chudnov

• Hannah De los Santos

• David Freedman

• Wen Hu

• Chris Penney

Other contributors:

• Jesse Anderson [contributor]

• Matt Boyas [contributor]

• Lusha Cao [contributor]

• Sara Couture [contributor]

• Prashila Dullabh [contributor]

• Jess Fernandez [contributor]

• Andrew Gregorowicz [contributor]

• Lyudmila Kompaniyets [contributor]

• A Russell Localio [contributor]

• Ruth Nwefo [contributor]

• Max Olivier [contributor]

• Samantha Pierce [contributor]

• Michelle E Ross [contributor]

• Steffani Silva [contributor]

• Mithuna Srinivasan [contributor]

• Marissa Sucosky [contributor]

• Jennifer Wiltz [contributor]

• Hanzhe Zhang [contributor]

• Alexander G Fiks [scientific advisor]

• Richard C (Mort) Wasserman [scientific advisor]

• Andreas Achilleos [scientific advisor]

• Nabeel Akhtar [scientific advisor]

• Sarah Akhtar [scientific advisor]

• Wilbert Beachy [scientific advisor]

• Colin Bridgeman [scientific advisor]

• Jessica Butts [scientific advisor]

• Emily Despeaux [scientific advisor]

• Christine Irvin [scientific advisor]

• Madiha Jamil [scientific advisor]

• Robert Marquis [scientific advisor]

• Abigail Nelson [scientific advisor]

• Sheila Nguyen [scientific advisor]

• Taraneh Soleymani [scientific advisor]

• Ronald Williams [scientific advisor]

See Also

Useful links:

• https://growthcleanr.org/

• https://github.com/growth-data-tools/growthcleanr

• Report bugs at https://github.com/growth-data-tools/growthcleanr/issues

CDC BMI reference data

Description

Used for extended BMIz computation

CDCref_d.csv.gz

Used for extended BMI computation

Removed function: adjustcarryforward()

Description

adjustcarryforward() was removed in growthcleanr 3.0.0. Carried-forward handling is now part of cleangrowth(); use its cf_rescue argument. This function remains only to stop with that message, so that packages that refer to it do not fail their checks.

Usage

adjustcarryforward(...)

Arguments

Value

None; always stops with an error.

See Also

vignette("cf-rescue")

Initialize EWMA cache for a set of observations. Builds the full weight matrix once (O(n^2)), computes EWMA values.

Description

Initialize EWMA cache for a set of observations. Builds the full weight matrix once (O(n^2)), computes EWMA values.

Usage

adult_ewma_cache_init(agedays, meas, ewma_exp = -5, ewma_window = 15)

Arguments

Value

Cache list with delta matrix, weighted sums, row sums, and EWMA values

Update EWMA cache by removing one observation. O(n) instead of O(n^2).

Description

Update EWMA cache by removing one observation. O(n) instead of O(n^2).

Usage

adult_ewma_cache_update(cache, pos_j)

Arguments

Value

Updated cache list with n-1 observations, or NULL if n would be 0

Child unit-error-range reference values

Description

Part of default reference tables

Details

Contains, for each age in days, sex, and param, the measurement values at recentered CSD z-scores of -3 (column zn3) and +3 (column zp3), computed with the same WHO/CDC blending and default recentering used by the child algorithm. Covers weight and height from birth to 20 years and head circumference from birth to 5 years (1826 days). Used to compute the informational unit_error_range output column and the unit-error check in Evil Twins (Child Step 9). See vignette("algorithm-functions").

child_z3_reference.csv.gz

Used in function cleangrowth() (when unit_error_range = TRUE, the default)

Clean growth measurements (main growthcleanr function)

Description

growthcleanr flags implausible values in longitudinal anthropometric measurements – weight, height/length, and head circumference – for children and adults. growthcleanr is specifically designed to evaluate electronic health record data for outpatients. It does not delete or edit your data. It returns your data with an added exclude column labeling each row either "Include" or an exclusion code naming why it was set aside. Each subject is cleaned independently, using that subject's own measurement trajectory over time, so splitting, subsetting, or batching the data does not change any subject's result.

The default input is one row per measurement in long format with columns subjid, param, agedays, sex, and measurement (plus an optional per-row id); semi-wide and wide layouts are also accepted (see input_format). You can use date of birth and visit date instead of agedays. Most runs need only the input-column arguments; see "Arguments at a glance" under Details for how the rest are grouped.

New to growthcleanr? Start with vignette("new-start-here"). To check that growthcleanr is reading your columns, sex coding, and parameters correctly before a full run, use gc_runtrial on a sample of your data. See vignette("output") for the returned columns and vignette("configuration") for the full argument index. For ready-to-run scripts covering common setups, see vignette("example-scripts").

Usage

cleangrowth(
  data = NULL,
  subjid = NULL,
  param = NULL,
  agedays = NULL,
  sex = NULL,
  measurement = NULL,
  piv.z.wt.high = 22,
  piv.z.ht.low.young = -25,
  piv.z.ht.low.old = -15,
  piv.z.ht.high = 8,
  piv.z.hc.low = -15,
  piv.z.hc.high = 15,
  error.load.mincount = 2,
  error.load.threshold = 0.5,
  sd.recenter = NA,
  cf_rescue = "standard",
  cf_detail = FALSE,
  corr_detail = FALSE,
  recenter_detail = FALSE,
  qm_detail = FALSE,
  ref.data.path = "",
  log.path = NA,
  parallel = FALSE,
  cores = NULL,
  quietly = FALSE,
  verbose = FALSE,
  adult_cutpoint = 20,
  adult_permissiveness = "looser",
  adult_scale_max_lbs = Inf,
  ewma_window = 15,
  adult_overrides = NULL,
  id = NULL,
  dob = NULL,
  visit_date = NULL,
  age_source = NULL,
  date_order = NULL,
  sex_coding = .gc_default_sex_coding,
  param_coding = NULL,
  carry_through = TRUE,
  input_format = "long",
  output_format = NULL,
  semiwide_cols = NULL,
  widened_emit = "exclude",
  wide_cols = NULL,
  max_blocks = 25,
  internal_detail = FALSE,
  ref_tables = NULL,
  tri_exclude = FALSE,
  debug = FALSE,
  full_detail = FALSE,
  unit_error_range = TRUE,
  display_gc_id = FALSE,
  exclude_detail = FALSE,
  mean_ht = TRUE,
  batch_size = 5000,
  length.adjust = FALSE,
  checkpoint = TRUE,
  checkpoint_dir = NULL,
  checkpoint_size = 5000,
  checkpoint_resume = FALSE,
  keep_checkpoints = TRUE,
  log_file = NULL,
  .checkpoint_driven = FALSE,
  ...
)

Arguments

Details

Calling styles:

1. Data-frame style (recommended). Pass a single data.frame or data.table as data. The subjid, param, agedays, sex, measurement, and id arguments are then column-name strings identifying which columns in data to use. Each defaults to the canonical name (e.g. "subjid"), so if your columns already use those names you can just call cleangrowth(data = my_df). Use the column-name arguments only to remap (e.g. subjid = "patid").

2. Vector style (legacy). Omit data and pass each of subjid, param, agedays, sex, and measurement as a vector. Retained for backward compatibility with existing code; new code should use the data-frame style.

All vectors (or the columns referenced in data) must have the same length and be aligned row-for-row. Pulling values from different datasets is not supported and not checked for.

Arguments at a glance. The many arguments fall into groups; most runs touch only the first one or two:

• Core input column names – data, subjid, sex, agedays, param, measurement, id.

• Age from dates (instead of agedays) – dob, visit_date, age_source, date_order.

• Sex and parameter coding – sex_coding, param_coding.

• Input/output shape – input_format, output_format, semiwide_cols, wide_cols, max_blocks, widened_emit, carry_through.

• Run control and performance – parallel, cores, batch_size, quietly, verbose, ref_tables, ref.data.path, log.path.

• Checkpointed (resumable) runs – checkpoint, checkpoint_dir, checkpoint_size, checkpoint_resume, keep_checkpoints, log_file.

• Adult algorithm – adult_cutpoint, adult_permissiveness, adult_scale_max_lbs, adult_overrides.

• EWMA window (child and adult) – ewma_window.

• Length vs. height handling – length.adjust.

• Recentering – sd.recenter.

• Carried-forward rescue – cf_rescue.

• Optional output columns – exclude_detail, unit_error_range, tri_exclude, cf_detail, corr_detail, recenter_detail, internal_detail, display_gc_id, mean_ht, full_detail, qm_detail, debug.

• Advanced threshold overrides (rarely changed) – piv.z.*, error.load.*. See vignette("algorithm-functions") for the CSD z-scores (sd.orig) the piv.z.* cutoffs apply to.

Value

By default, a data.table in long format (one row per input measurement, in the same order as the input); the exclusion codes are in its exclude column. (Prior versions returned a character vector of exclusion codes.) Eight columns are always present, plus unit_error_range unless you turn it off:

id

User-provided row identifier (or the auto-generated 1:N if none was usable).

subjid

Subject identifier, as submitted.

param

Measurement parameter, as submitted.

agedays

Age in days, as submitted.

sex

Sex, as submitted.

measurement

Measured value, as submitted.

exclude

The result (a factor): "Include" or a Summary-level exclusion code naming why the value was set aside.

bin_exclude

A two-way collapse of exclude (character): "Include" or "Exclude".

unit_error_range

On by default (set unit_error_range = FALSE to omit). A logical informational flag, TRUE for a value whose magnitude looks like a unit-conversion error; never affects exclude.

mean_ht

Returned by default when the data has adult rows (set mean_ht = FALSE to omit). The adult algorithm's calculated mean height (cm), computed from included heights only and populated on all of a subject's adult height rows; NA for child rows, adult weight rows, and subjects with no included adult height.

Exclusion codes changed in version 3; for the mapping from 2.x codes, see "Mapping from prior exclusion codes" in vignette("experienced-start-here").

Output fidelity. The five echoed input columns (subjid, param, agedays, sex, measurement) carry your verbatim submitted values – growthcleanr never silently alters or drops an input value. For example, imperial values are returned as submitted (not converted to metric), even though internally, growthcleanr converts them to metric. A value growthcleanr cannot use – an uninterpretable sex; a 0, negative, or missing measurement; a missing param; or a negative/NA agedays – is flagged Exclude-Missing-Info; a param that is not one growthcleanr recognizes is flagged Exclude-Not-GC-Param; and a head-circumference value at 5 years or older (beyond the reference) is flagged Exclude-HC-Out-of-Range. In every case the value is still returned as submitted. The internal working values are available via internal_detail/debug (see below).

Column naming. The six data columns (id, subjid, param, agedays, sex, measurement) are renamed to match your original argument expressions when those resolve to a simple name – e.g. cleangrowth(subjid = dt$patid, ...) returns a column named patid. Complex expressions fall back to the canonical name. In data-frame style, output columns keep your column names (e.g. subjid = "patid" returns patid).

Optional columns. Each of these switches adds columns; all default to off unless noted:

exclude_detail

exclude_detailed – the step-level (Detailed) exclusion code behind each Summary exclude value.

tri_exclude

tri_exclude – "Include", "Same-Day", or "Exclude".

cf_detail

cf_status, cf_deltaz, cf_nextz, and three carried-forward design aids (imperial, cf_dage_orig, cf_dage_next).

corr_detail

corr_val (per value) and corr_pt (per subject) gestational-age correction status.

recenter_detail

The recentered CSD z-score columns tbc.sd, ctbc.sd, and rcmedian.sd (child rows; NA for adults) – specialized output for a related tool in development.

internal_detail

growthcleanr's working values for your inputs: gc_internal_meas, gc_internal_param, gc_internal_sex, gc_internal_id, gc_internal_agedays (the whole-day age used for cleaning) (and parsed gc_internal_dob/gc_internal_visit_date if dates were supplied).

display_gc_id

gc_internal_id – the id growthcleanr actually used internally.

debug

Intermediate z-score diagnostics for child rows (sd.orig, sd.corr, tbc.sd, ctbc.sd, and the plus/minus perturbation z-scores) plus gc_internal_sex; for development, not routine use.

qm_detail

Ten per-step child eligibility flags (qm_elig_*); for eligibility-normalized quality measures used for a related package in development. Must be requested explicitly – not turned on by full_detail.

full_detail

Convenience switch turning on all of the above except qm_detail (and independent of unit_error_range).

Non-long output. With output_format = "semi_wide" or "wide" the returned object is your submitted table with the columns named in widened_emit inserted beside the value they describe (see output_format).

The full catalog of exclusion codes (child + adult, Summary and Detailed) is in vignette("exclusion-codes").

Examples

# Check your setup on a sample of your data first: gc_runtrial(syngrowth)

# Data-frame style (recommended): canonical column names, just pass the
# data.frame.
# syngrowth is a small example dataset bundled with growthcleanr.
df_stats <- as.data.frame(syngrowth)
# Use 5 subjects so the example runs quickly
df_stats <- df_stats[df_stats$subjid %in% unique(df_stats[, "subjid"])[1:5], ]
# We recommend including a unique per-row `id` column (syngrowth already
# does; this shows how to add one)
df_stats$id <- seq_len(nrow(df_stats))
clean_stats <- cleangrowth(data = df_stats)

# How many values were included, and why the others were excluded:
table(clean_stats$exclude)

# The result is your data with an added `exclude` column; keep only the
# measurements growthcleanr says to include:
clean_only <- clean_stats[clean_stats$exclude == "Include", ]

# Data-frame style with remapped column names
my_df <- df_stats
names(my_df)[names(my_df) == "subjid"] <- "patid"
clean_stats <- cleangrowth(data = my_df, subjid = "patid")

# Vector style (legacy)
clean_stats <- cleangrowth(subjid = df_stats$subjid,
                           param = df_stats$param,
                           agedays = df_stats$agedays,
                           sex = df_stats$sex,
                           measurement = df_stats$measurement,
                           id = df_stats$id)

# Parallel processing: run using 2 cores
clean_stats <- cleangrowth(data = df_stats,
                           parallel = TRUE,
                           cores = 2)

Run cleangrowth() with automatic checkpointing (deprecated wrapper)

Description

Deprecated. Checkpointing is built into cleangrowth via its checkpoint_dir argument – call cleangrowth(data, checkpoint_dir = ...) directly. This wrapper forwards to cleangrowth() and will be removed in a future version. The use_existing_checkpoint_results argument maps to cleangrowth(checkpoint_resume = ...). Calling cleangrowth() directly also lets a checkpointed run use wide / semi-wide input and output (via its input_format / output_format).

Usage

cleangrowth_checkpoint(
  data,
  checkpoint_dir,
  checkpoint_size = 5000,
  log_file = NULL,
  keep_checkpoints = TRUE,
  use_existing_checkpoint_results = FALSE,
  quietly = FALSE,
  verbose = FALSE,
  ...
)

Arguments

Value

A data.table – see cleangrowth for the output columns.

See Also

cleangrowth, cleangrowth_checkpoint_combine

Examples

## Not run: 
library(data.table)
dt <- fread("large_dataset.csv")

# Preferred: call cleangrowth() directly with checkpoint_dir.
result <- cleangrowth(dt, checkpoint_dir = "results/gc_checkpoints",
                      checkpoint_size = 5000, log_file = "results/gc_run.log")

## End(Not run)

Combine saved checkpoint files from a checkpointed cleangrowth() run

Description

Reads all checkpoint .rds files from a directory and combines them into a single data.table. Use this when a checkpointed cleangrowth run (checkpoint_dir set) completed all checkpoints but the combine step failed - you can re-combine without re-running the algorithm.

Usage

cleangrowth_checkpoint_combine(
  checkpoint_dir,
  id_order = NULL,
  log_file = NULL,
  quietly = FALSE,
  verbose = FALSE
)

Arguments

Details

All .rds files matching checkpoint_NNNN.rds in checkpoint_dir are read and combined in numeric order. Every matching file is included, so the directory should hold checkpoints from a single run only. (A checkpointed cleangrowth() run with checkpoint_resume = FALSE, the default, removes earlier checkpoint files from the directory at the start, so after a run only that run's files remain.)

Value

A long-format data.table with growthcleanr's standard column names (id, subjid, param, agedays, sex, measurement, plus result columns). Unlike cleangrowth(), it does not restore the user's column names, carried-through columns, or wide/semi-wide layout; if growthcleanr generated id values (because id was missing or not unique), id holds those generated values.

See Also

cleangrowth_checkpoint

Examples

## Not run: 
# If the combine step failed at the end of a run, re-combine like this:
result <- cleangrowth_checkpoint_combine(
  checkpoint_dir = "results/gc_checkpoints",
  id_order       = dt$id   # original row order; needs a unique id column named id
)
saveRDS(result, "results/gc_output.rds")

## End(Not run)

Compute the Evil Twins / Extreme EWMA cap for given intervals and weights. Vectorized: interval_months and uw can be vectors.

Description

For built-in formulas (piecewise, piecewise-lower, allofus15), ET caps are derived from the formula's wtallow caps + 20 (for PW-H/PW-L) or equal to the allofus15 12m cap. For custom CSV formulas, fixed caps of 70/100 are used.

Usage

compute_et_limit(interval_months, formula = "piecewise", uw = NULL)

Arguments

Value

Numeric vector of ET limits

References

Guide A, Sulieman L, Garbett S, Cronin RM, Spotnitz M, Natarajan K, Carroll RJ, Harris P, Chen Q. Identifying erroneous height and weight values from adult electronic health records in the All of Us research program. J Biomed Inform. 2024 Jul;155:104660. doi:10.1016/j.jbi.2024.104660. (Source of the adapted "allofus15" formula.)

Compute observation-level percentage criterion limit. Vectorized over meas.

Description

Compute observation-level percentage criterion limit. Vectorized over meas.

Usage

compute_perc_limit(meas, perclimit_low, perclimit_mid, perclimit_high)

Arguments

Value

Numeric vector of perc_limits

Pre-computes trajectory rescue for all observations. Returns TRUE = fails all rescue (not rescued).

Description

Pre-computes trajectory rescue for all observations. Returns TRUE = fails all rescue (not rescued).

Usage

compute_trajectory_fails(meas, age_days, err = 5)

Arguments

Compute weight allowance from interval in months (vectorized)

Description

Built-in formulas: "piecewise" (PW-H, default), "piecewise-lower" (PW-L), "allofus15". Custom: pass a file path to a CSV with columns 'months' and 'wtallow'. Values are linearly interpolated; months beyond the table are clamped to the nearest row.

Usage

compute_wtallow(months, formula = "piecewise", uw = NULL)

Arguments

Details

UW (upper weight) scaling:

• PW-H: scaled up for UW > 120 (slope 0.25/kg, UW capped at 180) and down for UW < 120, with 2/3 ceiling.

• PW-L: NOT scaled up for UW > 120. Scaled down for UW < 120, with 2/3 ceiling.

• allofus15: not adjusted by UW, except its 12m cap is limited to never exceed the effective PW-L at 12m for that UW.

• Custom CSV: no UW adjustment.

Value

Numeric vector of weight allowances

References

Guide A, Sulieman L, Garbett S, Cronin RM, Spotnitz M, Natarajan K, Carroll RJ, Harris P, Chen Q. Identifying erroneous height and weight values from adult electronic health records in the All of Us research program. J Biomed Inform. 2024 Jul;155:104660. doi:10.1016/j.jbi.2024.104660. (Source of the adapted "allofus15" formula.)

Detect consecutive runs of TRUE in a logical vector Returns list with run_id, run_len, run_pos for each element

Description

Detect consecutive runs of TRUE in a logical vector Returns list with run_id, run_len, run_pos for each element

Usage

detect_runs(flagged)

Arguments

Evaluate single distinct value observations

Description

Evaluate single distinct value observations

Usage

eval_1d(subj_results, params)

Arguments

Value

Character vector of internal_ids to mark Exclude-A-Single

Evaluate 2D non-ordered weight pairs for one subject

Description

Evaluate 2D non-ordered weight pairs for one subject

Usage

eval_2d_nonord(w_subj_df, w_subj_keep, wtallow_formula = "piecewise")

Arguments

Value

Character vector of internal_ids to exclude with code Exclude-A-2D-Non-Ordered

Evaluate error load for one subject

Description

Evaluate error load for one subject

Usage

eval_error_load(subj_results, error_threshold = 0.41)

Arguments

Value

Character vector of internal_ids to mark Exclude-A-Unevaluable-Trajectory

Evil twins detection for one subject (anchor rule)

Description

Adult Step 9Wa, weight only, in kg measurement-space. Mirrors the child Step 9 anchor rule and one-exclusion-per-iteration selection (calc_otl_evil_twins + the Step 9 driver in child_clean.R); the scale differs (raw kg + interval/UW etcaps in place of z-scores).

Usage

evil_twins(w_subj_df, wtallow_formula = "piecewise")

Arguments

Details

A weight is OTL ("over-the-limit") only if it is an inner member (B or C) of an evaluable over-the-limit pair: four age-ordered eligible values A, B, C, D where the inner B-C jump exceeds etcap(B,C) and BOTH outer pairs (A-B, C-D) anchor within 60% of their own etcap. A missing A or D disqualifies the pair, so single spikes (record ends and singles embedded between normals) cede to remove_ewma_wt(), which has the better-informed directional 90% rule and per-observation thresholds. All comparisons to etcap carry the 0.12 kg rounding tolerance used throughout the adult algorithm.

RVs (repeated values) participate as ordinary values: no special handling.

Selection per iteration (one exclusion): if a single OTL candidate, exclude it. Otherwise compute absd_med = |meas - windowed median| (median over +/-4 age-ordered positions, kg space) and route by the UER3 unit-error check and a median band, identically in structure to the child:

• UER3 (computed ONCE on the initial Include set): if the SP splits cleanly into a unit-error group (unit_error_range flag, all one CSD-z sign) and a plausible group (|adult CSD z| < 3) whose mean kg differ by ~2.2046 (the lb<->kg factor, band 1.7-2.7), the unit-error block is the implausible one and the median can be pulled toward it, so exclude the highest |z|.

• Otherwise: band = candidates whose absd_med is within band_width of the max absd_med, band_width = 0.20 * etcap(B,C) of the max candidate's inner pair (the adult analogue of the child's fixed z-unit band). If >= 2 in band (the windowed median cannot discriminate), exclude the highest |adult CSD z|; if exactly 1 (one absd_med stands clearly apart), absd_med is decisive.

• Tiebreak: lowest internal_id.

Adult CSD z (sex-specific, NA-safe via the averaged sex=NA reference row) is used for the UER3 plausible group and the |z| tiebreak; weights are looked up as WEIGHTKG (meas_m is already kg).

Value

Character vector of internal_ids to exclude (Exclude-A-Evil-Twins)

Exponentially Weighted Moving Average (EWMA)

Description

ewma calculates the exponentially weighted moving average (EWMA) for a set of numeric observations over time.

Usage

ewma(agedays, z, ewma.exp, ewma.adjacent = TRUE, window = 15, cache_env = NULL)

Arguments

Value

A named list of numeric vectors, each the same length as agedays.

• ewma.all - EWMA at each observation's age, excluding only the observation itself.

• ewma.before - also excludes the immediate prior observation (sorted by agedays); equals ewma.all for the first observation and when n <= 2.

• ewma.after - also excludes the immediate subsequent observation; equals ewma.all for the last observation and when n <= 2.

When ewma.adjacent = FALSE, only ewma.all is returned (still wrapped in a one-element named list).

Examples

# Run on 1 subject, 1 type of parameter
df_stats <- as.data.frame(syngrowth)
df_stats <- df_stats[df_stats$subjid == df_stats$subjid[1] &
                       df_stats$param == "HEIGHTCM", ]

# Get the uncentered z-scores
measurement_to_z <- read_anthro(cdc.only = TRUE)
sd <- measurement_to_z(df_stats$param,
                       df_stats$agedays,
                       df_stats$sex,
                       df_stats$measurement,
                       TRUE)

# Calculate exponentially weighted moving average
e_df <- ewma(df_stats$agedays, sd, ewma.exp = -1.5)

Calculate extended BMI measures

Description

ext_bmiz Calculates the sigma (scale parameter for the half-normal distribution), extended BMI percentile, extended BMIz, and the CDC LMS Z-scores for weight, height, and BMI for children between 2 and 19.9 years of age. Note that for BMIs <= 95th percentile of the CDC growth charts, the extended values for BMI are equal to the LMS values. The extended values differ only for children who have a BMI > 95th percentile.

Usage

ext_bmiz(
  data,
  age = "agem",
  wt = "wt",
  ht = "ht",
  bmi = "bmi",
  adjust.integer.age = TRUE,
  ref.data.path = ""
)

Arguments

Details

This function produces output equivalent to CDC software and the cdcanthro R package.

The extended BMIz is the inverse cumulative distribution function (CDF) of the extended BMI percentile. If the extended percentile is very close to 100, the qnorm function in R produces an infinite value. This occurs only if the extended BMI percentile is > 99.99999999999999. This occurs infrequently, such as a 48-month-old with a BMI > 39, and it is likely that these BMIs represent data entry errors. For these cases, extended BMIz is set to 8.21, a value that is slightly greater than the largest value that can be calculated.

The output columns are listed under Value below.

data must have columns for at least age, sex, weight, height, and bmi.

age should be coded in months, using the most precise values available. To convert to months from age in years, multiply by 12. To convert to months from age in days, divide by 30.4375 (365.25 / 12).

sex: we recommend M and F, which is what longwide() returns. Male can be coded as boy, Boy, b, B, male, Male, m, or M, and female can be coded as girl, Girl, g, G, female, Female, f, or F. Numeric sex is accepted only if sex has both 1s and 2s and no other values, read as 1 = male, 2 = female; otherwise ext_bmiz() stops. This is because 0/1 is the numeric coding in cleangrowth(), and a column of only 1s could be either coding. The 1/2 coding matches the CDC reference data, CDC software, and the cdcanthro package. Other text values, and logical values, are not recognized: their z-scores are NA, with a warning.

wt should be in kilograms.

ht should be in centimeters.

Specify the input data column names for age, wt, ht, bmi using quotation marks. See example below.

ext_bmiz() does not change data. It returns a copy of data with the new columns added; your column names and values are kept as they are (the 0.5 added by adjust.integer.age is used only in the calculation). If data already has a column with the name of one of the columns ext_bmiz() adds (bmiz, bmip, waz, wp, haz, hp, p95, p97, bmip95, mod_bmiz, mod_waz, mod_haz, sigma, original_bmip, original_bmiz, sev_obese, obese), for example from an earlier run, it warns and the returned data has the newly calculated values in that column; rename the column first to keep both.

Values are calculated for ages 24 through 240 months, as in the cdcanthro package. Age 730 days is 23.98 months (730 / 30.4375), which is below 24 months, so a row at 730 days has NA values even though a child at that age may be exactly 2 years old; values begin at 731 days. A warning gives the number of rows at about 729-730 days.

If the argument adjust.integer.age is TRUE (the default), 0.5 will be added to all age values for the calculation if all input values are integers. Set to FALSE to disable.

By default, the reference data file CDCref_d.csv, made available at https://www.cdc.gov/growth-chart-training/hcp/computer-programs/index.html, is included in this package for convenience. If you are developing this package, use ref.data.path to adjust the path to this file from your working directory if necessary.

Value

A data.table: a copy of data (same rows, row order, column names, and values) with these columns added: bmiz, bmip, waz, wp, haz, hp, p95, p97, bmip95, mod_bmiz, mod_waz, mod_haz, sigma, original_bmip, original_bmiz, sev_obese, obese.

References

Wei R, Ogden CL, Parsons VL, Freedman DS, Hales CM. A method for calculating BMI z-scores and percentiles above the 95th percentile of the CDC growth charts. Ann Hum Biol. 2020 Sep;47(6):514-521. doi:10.1080/03014460.2020.1808065.

Examples

# Run on a small subset of given data
df <- as.data.frame(syngrowth)
df <- df[df$subjid %in% unique(df[, "subjid"])[1:5], ]
# cleangrowth() returns the data with an 'exclude' column
clean_df <- cleangrowth(df)
df_wide <- longwide(clean_df) # convert to wide format for ext_bmiz
df_wide_bmi <- simple_bmi(df_wide) # compute simple BMI

# Calling the function with default column names
df_bmiz <- ext_bmiz(df_wide_bmi)

# Specifying different column names; note that quotes are used
dfc <- simple_bmi(df_wide)
colnames(dfc)[colnames(dfc) %in% c("agem", "wt", "ht")] <-
  c("agemos", "weightkg", "heightcm")
df_bmiz <- ext_bmiz(dfc, age="agemos", wt="weightkg", ht="heightcm")

# Disabling conversion of all-integer age in months to (age + 0.5)
dfc <- simple_bmi(df_wide)
df_bmiz <- ext_bmiz(dfc, adjust.integer.age=FALSE)

Approximated Fenton Median Weights by Gestational Age

Description

A table of approximated Fenton median weights by gestational age, derived from printed Fenton (2025) charts. Maps sex and integer weight (grams, rounded to nearest 10) to the gestational age in days at which that weight is the median. These are approximations derived from the charts, not values published by the Fenton authors. See vignette("wrapper"), "Gestational-age reference values".

fent_foraga.csv.gz

Used in function cleangrowth() for GA correction (Step 2b).

References

Fenton TR, Elmrayed S, Alshaikh BN. Fenton Third-Generation Growth Charts of Preterm Infants Without Abnormal Fetal Growth: A Systematic Review and Meta-Analysis. Paediatr Perinat Epidemiol. 2025 Aug;39(6):543-555. doi:10.1111/ppe.70035. Epub 2025 Jun 19. PMID: 40534585; PMCID: PMC12391854.

Approximated Fenton CSD Values (M, S_upper, S_lower)

Description

Approximated Fenton CSD values: approximated reference values derived from printed Fenton (2025) charts, for weight, length, and head circumference by sex and gestational age in days. Contains the median (M), S_upper, and S_lower, where S_upper x M is half the distance between the median and +2 SD and S_lower x M is half the distance between the median and -2 SD. Used with the CSD method for z-score calculation. These are approximations derived from the charts, not values published by the Fenton authors. See vignette("wrapper"), "Gestational-age reference values".

fenton2025_ms_lookup_smoothed.csv

Used in function cleangrowth() for GA correction (Step 2b).

References

Fenton TR, Elmrayed S, Alshaikh BN. Fenton Third-Generation Growth Charts of Preterm Infants Without Abnormal Fetal Growth: A Systematic Review and Meta-Analysis. Paediatr Perinat Epidemiol. 2025 Aug;39(6):543-555. doi:10.1111/ppe.70035. Epub 2025 Jun 19. PMID: 40534585; PMCID: PMC12391854.

Path to the growthcleanr command-line driver script

Description

Returns the location of the installed gcdriver.R command-line driver, so it can be invoked with Rscript without hunting for the file:

DRIVER=$(Rscript -e 'cat(growthcleanr::gc_cli_path())')
Rscript "$DRIVER" input.csv output.csv --quiet

Usage

gc_cli_path()

Value

The absolute path to gcdriver.R in the installed package.

See Also

run_gc_cli; vignette("command-line").

Incremental (partial) growthcleanr run

Description

Re-clean only the subjects whose data changed since a previous cleangrowth run, and reuse the earlier cleaned results for everyone else. This is intended for workflows that repeatedly clean a mostly unchanging dataset – for example a simulation loop that perturbs a subset of subjects and re-cleans – where a full re-run would waste most of its work.

Usage

gc_partial_run(new_data, cached_output, changed_subjids, ...)

Arguments

Details

gc_partial_run() is a thin wrapper: it slices new_data down to the changed (and any newly added) subjects, hands that subset to a single ordinary cleangrowth() call, and row-binds the freshly cleaned rows to the retained rows of cached_output. All arguments in ... – including checkpoint_dir, checkpoint_size, batch_size, and every output-column switch – are forwarded verbatim to that inner cleangrowth() call, so a large changed subset can still be checkpointed. The cache itself is never passed into cleangrowth(); the merge happens entirely at the output level.

Why the result is correct. After reference-based recentering, growthcleanr cleans each subject independently of every other subject, so cleaning only the changed subjects yields the exact same rows those subjects would receive in a full run. The combined result is therefore identical to cleangrowth(new_data, ...) for a given set of options (see Examples).

Caller responsibilities.

• ... must specify the same options used to produce cached_output (same cf_rescue, permissiveness, output switches, etc.). If the requested output columns differ from those in cached_output, the function stops rather than silently produce NA-filled rows.

• id must be present and unique per row in both new_data and cached_output, and must be stable across runs (the same measurement keeps the same id). id is how retained cached rows and freshly cleaned rows are ordered in the output.

• Input must be long format (one measurement per row). Wide and semi-wide input are not supported by this wrapper.

Value

A data.table with the same columns as cleangrowth() would return under the same ..., containing every subject in new_data, ordered by id.

See Also

cleangrowth

Examples

# Baseline clean of the full dataset
baseline <- cleangrowth(syngrowth)

# Suppose one subject's data was corrected; re-clean only that subject
changed <- syngrowth$subjid[1]
updated <- gc_partial_run(syngrowth, baseline, changed_subjids = changed)

# Identical to a full re-run (same options), for every subject
full <- cleangrowth(syngrowth)
identical(updated[order(id)]$exclude, full[order(id)]$exclude)

Pre-load growthcleanr reference closures

Description

gc_preload_refs loads all reference table closures used by cleangrowth once, avoiding repeated disk reads across repeated calls (e.g., simulation loops). Pass the result as the ref_tables argument to cleangrowth.

Usage

gc_preload_refs(path = "")

Arguments

Value

A named list with two read_anthro closures:

mtz_cdc_prelim

CDC-only (used by child algorithm)

mtz_who_prelim

WHO+CDC (used by child algorithm)

Examples

refs <- gc_preload_refs()
result <- cleangrowth(
  subjid = syngrowth$subjid,
  param  = syngrowth$param,
  agedays = syngrowth$agedays,
  sex = syngrowth$sex,
  measurement = syngrowth$measurement,
  ref_tables = refs
)

Trial run of growthcleanr on a sample of your data

Description

gc_runtrial() helps you confirm that your data are formatted correctly and that growthcleanr is reading your columns, sex coding, and parameters the way you intend – before you commit to a full cleangrowth run. It draws a small sample of patients, runs cleangrowth() on them with the exact configuration you pass, and prints a short diagnostic report: how the input was interpreted, how sex and parameters were mapped (the two highest-risk settings, shown prominently), anything growthcleanr flagged while reading, and a summary of the cleaning result. The cleaned sample is returned invisibly for inspection. For a ready-to-run script, see "Check your setup first" in vignette("example-scripts").

Usage

gc_runtrial(
  data = NULL,
  n = 100,
  sampling = c("random", "first"),
  seed = NULL,
  ...
)

Arguments

Details

Sampling is by patient: all of a sampled subject's rows are kept, so trajectories stay intact. Random sampling (the default) is effectively free – the only real cost is running growthcleanr on the sampled patients, which is the same whether they were chosen at random or as the first n. Pass a seed for a reproducible random sample, or sampling = "first" for a deterministic peek that needs no seed.

Only the data-frame calling style is supported (pass your table as data); vector-style input is not. The trial always analyses and returns results in long format so the diagnostic is well defined, regardless of any output_format you set.

If your configuration makes cleangrowth() stop (for example a forgotten input_format, or a sex coding it will not guess), the trial reports that stop as the headline finding instead of re-throwing it – turning a hard error into setup guidance. The sex and parameter mapping sections still print, since they are derived from your configuration and data rather than the run.

Value

Invisibly, a data.table: the cleaned sample in long format on success (empty on a configuration error), with the structured diagnostic attached as attr(x, "gc_trial"). Prints the report as a side effect.

Examples

# quick check that a dataset is set up right
gc_runtrial(syngrowth)


## Not run: 
# semi-wide data, custom sex coding, reproducible 50-patient sample
# (my_visits stands in for your own semi-wide dataset)
gc_runtrial(my_visits, n = 50, seed = 1,
            input_format = "semi_wide",
            sex_coding = list(male = "M", female = "F"))

## End(Not run)

CDC Growth Reference Table

Description

Part of default CDC-derived tables

Details

Contains CDC LMS parameters and CSD values (median plus half the distance to +/-2 SD) for weight, height, head circumference, and BMI, by age in days and sex.

growthfile_cdc_ext_infants.csv.gz

Used in function cleangrowth()

WHO Growth Reference Table

Description

Part of default WHO-derived tables

Details

Contains WHO LMS parameters and CSD values for weight, height, head circumference, and BMI, by age in days (0 to 5 years) and sex.

growthfile_who.csv.gz

Used in function cleangrowth()

Run the growthcleanr Shiny application

Description

Launches the interactive growthcleanr Shiny interface in a local web browser. The app runs entirely on the user's own machine: data is read into the local R session, cleaned with cleangrowth(), and results are saved to the user's computer. Nothing is uploaded anywhere.

Usage

growthcleanr_app(...)

Arguments

Details

The app needs the shiny, DT, shinyjs, and shinyFiles packages, which growthcleanr imports, so a standard growthcleanr install is enough to run it.

Value

This function is called for its side effect of launching the app and does not return a value; interrupt R to stop the app.

Examples

## Not run: 
growthcleanr_app()

## End(Not run)

Transform data in growthcleanr format into wide structure for BMI calculation

Description

longwide transforms data from long to wide format. Ideal for transforming output from growthcleanr::cleangrowth() into a format suitable for growthcleanr::ext_bmiz().

Usage

longwide(
  long_df,
  id = "id",
  subjid = "subjid",
  sex = "sex",
  agedays = "agedays",
  param = "param",
  measurement = "measurement",
  gcr_result = "exclude",
  include_all = FALSE,
  inclusion_types = c("Include"),
  extra_cols = NULL,
  keep_unmatched_data = FALSE,
  sex_coding = .gc_default_sex_coding,
  param_coding = NULL
)

Arguments

Value

Returns a data frame transformed from long to wide. Includes only values flagged with indicated inclusion types. Potentially includes additional columns if arguments are passed to extra_cols. For each subject, heights without corresponding weights for a given age (and vice versa) will be dropped unless keep_unmatched_data is set to TRUE. The subject and age columns keep the names you passed as subjid and agedays (for example patid); the other columns are agey, agem, sex (recoded to M or F), ht_id, wt_id, wt (kg), and ht (cm). If no rows are kept, a data frame with these columns and no rows is returned.

Examples

# Run on a small subset of given data
df <- syngrowth[syngrowth$subjid %in% unique(syngrowth$subjid)[1:5], ]
gc_out <- cleangrowth(df)
df$exclude <- gc_out$exclude[match(df$id, gc_out$id)]

# Convert to wide format (default gcr_result = "exclude")
wide_df <- longwide(df)

# Include all inclusion types
wide_df <- longwide(df, include_all = TRUE)

# Specify which exclusion codes to retain (default `exclude` column uses
# the Summary codes; pass `exclude_detail = TRUE` to cleangrowth() and use
# `gcr_result = "exclude_detailed"` for the step-level Detailed codes)
wide_df <- longwide(df, inclusion_types = c("Include", "Exclude-CF"))

Return preset setting values for each permissiveness level

Description

Returns a named list of four permissiveness levels (loosest, looser, tighter, tightest), each containing the full set of adult algorithm settings for that level. Default level is "looser".

Usage

permissiveness_presets()

Value

Named list of four lists, each containing setting values

See Also

Section 1D of vignette("configuration") for what each level is intended for and the preset values; vignette("adult-algorithm") for what each setting does.

Examples

presets <- permissiveness_presets()
names(presets)
presets$looser$overall_wt_max

Propagate firstRV exclusions forward to RV copies and extraneous values (linked mode). For each excluded value, finds all other values with the same measurement that are still "Include" – including extraneous values whose is_rv flag may have been cleared by redo_identify_rv(). Marks them with "<code>-RV-Propagated". This is forward-only propagation (firstRV -> later RVs + extraneous), unlike Step 4W's bidirectional scale-max propagation.

Description

Propagate firstRV exclusions forward to RV copies and extraneous values (linked mode). For each excluded value, finds all other values with the same measurement that are still "Include" – including extraneous values whose is_rv flag may have been cleared by redo_identify_rv(). Marks them with "<code>-RV-Propagated". This is forward-only propagation (firstRV -> later RVs + extraneous), unlike Step 4W's bidirectional scale-max propagation.

Usage

propagate_to_rv(exc_codes, w_subj_df, w_subj_keep)

Arguments

Value

List with updated w_subj_keep and propagated_ids

Recentering reference medians

Description

Contains reference median z-scores (column rcmedian.sd) by age in days, sex, and param, used for the default recentering in the child algorithm (sd.recenter = NA). See vignette("recentering").

rcfile-resmoothed.csv.gz

Used in function cleangrowth()

Function to calculate z-scores and csd-scores based on anthro tables.

Description

Function to calculate z-scores and csd-scores based on anthro tables.

Usage

read_anthro(path = "", cdc.only = FALSE)

Arguments

Value

A function f(param, agedays, sex, measurement, csd = FALSE) that returns, for each measurement, its LMS z-score (default) or, when csd = TRUE, its CSD ("standard deviation") score against the selected reference (WHO below age 5 years and CDC from age 5 years, or CDC at all ages when cdc.only = TRUE; the 2-5 year WHO/CDC blending used by cleangrowth() is not applied here).

Examples

# Return calculating function with all defaults
afunc <- read_anthro()

# Return calculating function while specifying a path and using only CDC data
afunc <- read_anthro(path = system.file("extdata", package = "growthcleanr"),
                     cdc.only = TRUE)

Recode binary sex variable for compatibility

Description

recode_sex recodes a binary sex variable for a given source column in a data frame or data table. Only exact matches of sourcem/sourcef are recoded; any other value (e.g. "M"/"F") becomes NA in targetcol without a warning. ext_bmiz() reads only the column named sex, so to prepare data for it use targetcol = "sex" (with targetm = "M", targetf = "F" if the data might have only one sex, since ext_bmiz() accepts numeric sex only when both 1 and 2 are present). Not needed between cleangrowth() and ext_bmiz(): longwide() already outputs sex as M or F.

Usage

recode_sex(
  input_data,
  sourcecol = "sex",
  sourcem = "0",
  sourcef = "1",
  targetcol = "sex_recoded",
  targetm = 1L,
  targetf = 2L
)

Arguments

Value

Returns a data table with recoded sex variables.

Examples

# Run on given data
df <- as.data.frame(syngrowth)

# Run with all defaults
df_r <- recode_sex(df)

# Specify different targets
df_rt <- recode_sex(df, targetcol = "sexr", targetm = "Male", targetf = "Female")

# Specify different inputs
df_ri <- recode_sex(df_rt, sourcecol = "sexr", sourcem = "Male", sourcef = "Female")

Remove extreme EWMA weight outliers (Adult Step 9Wb)

Description

Iteratively excludes values whose EWMA deviation exceeds an interval-specific threshold (ET cap). Each round removes the single worst outlier (largest |dewma|, with age then internal_id as tiebreakers). Iterates until no candidates remain or <3 values.

Usage

remove_ewma_wt(
  subj_df,
  wtallow_formula = "piecewise",
  exc_label = "Exclude-A-Traj-Extreme",
  ewma_window = 15
)

Arguments

Details

Threshold: ET cap derived from wtallow formula and upper weight (UW). Per-observation threshold based on min neighbor gap and max(wt, ewma).

90% rule: directional dewma (before/after) must exceed 90% of threshold. For edge values, the missing directional EWMA falls back to the overall EWMA, so the 90% rule does not block exclusion; a missing neighbor gap is treated as Inf when choosing the interval.

Value

Named character vector: internal_id -> exc_label for excluded values

Remove implausible weight values using 7-step moderate EWMA flow.

Description

Remove implausible weight values using 7-step moderate EWMA flow.

Usage

remove_mod_ewma_wt(
  full_inc_df,
  exc_label = "Exclude-A-Traj-Moderate",
  max_rounds = 100,
  wtallow_formula = "piecewise",
  ewma_window = 15,
  mod_ewma_f = 0.75,
  perclimit_low = 0.5,
  perclimit_mid = 0.4,
  perclimit_high = 0,
  endpair_gap_days = 90
)

Arguments

Value

Named character vector: internal_id -> exclusion code for excluded values.

Resolve permissiveness: fill NULLs from preset, keep explicit values

Description

Resolve permissiveness: fill NULLs from preset, keep explicit values

Usage

resolve_permissiveness(permissiveness, ...)

Arguments

Value

Named list of all resolved parameter values

Run growthcleanr from the command line (CSV in, CSV out)

Description

Testable core behind the inst/scripts/gcdriver.R command-line driver. Reads a CSV, runs cleangrowth on it (data-frame style), and writes the result to a CSV. Only the options in opts are forwarded, so cleangrowth()'s own defaults apply to everything else.

Usage

run_gc_cli(infile, outfile, opts = list())

Arguments

Value

The output path, invisibly.

See Also

cleangrowth; the command-line driver inst/scripts/gcdriver.R.

Examples

## Not run: 
run_gc_cli("visits.csv", "cleaned.csv",
           opts = list(adult_cutpoint = 18, quietly = TRUE))

## End(Not run)

Compute BMI using standard formula

Description

simple_bmi Computes BMI using standard formula. Assumes input compatible with output from longwide().

Usage

simple_bmi(wide_df, wtcol = "wt", htcol = "ht")

Arguments

Value

Returns a data table with the added column "bmi"

Examples

# Simple usage
# Run on a small subset of given data
df <- as.data.frame(syngrowth)
df <- df[df$subjid %in% unique(df[, "subjid"])[1:2], ]
# cleangrowth() returns the data with an 'exclude' column
clean_df <- cleangrowth(df)
# Convert to wide format
wide_df <- longwide(clean_df)
wide_df_with_bmi <- simple_bmi(wide_df)

# Specifying different column names; note that quotes are used
colnames(wide_df)[colnames(wide_df) %in% c("wt", "ht")] <-
  c("weight", "height")
wide_df_with_bmi <- simple_bmi(wide_df, wtcol = "weight", htcol = "height")

syngrowth

Description

A synthetic set of 77,721 measurements from 2,697 subjects generated using Synthea, with measurement errors for testing with growthcleanr. Contains both pediatric and adult data.

Usage

syngrowth

Format

A data frame with six variables: id, subjid, sex, agedays, param, and measurement

Details

Example electronic health record (heightcm, weightkg) data.

Tanner Growth Velocity Table

Description

Part of default reference tables

Details

Contains height velocities from Tanner and Davies (1985)

tanner_ht_vel.csv.gz

Used in function cleangrowth()

References

Tanner JM, Davies PS. Clinical longitudinal standards for height and height velocity for North American children. J Pediatr. 1985 Sep;107(3):317-329. doi:10.1016/s0022-3476(85)80501-1.

WHO Maximum Head Circumference Velocity for (3\sigma)

Description

Part of default WHO-derived tables

Details

Contains three standard deviations for the World Health Organization values of maximum head circumference velocities.

who_hc_maxvel_3sd_infants.csv.gz

Used in function cleangrowth()

References

World Health Organization. WHO Child Growth Standards: Growth velocity based on weight, length and head circumference: Methods and development. Geneva: World Health Organization; 2009. ISBN 9789241547635.

WHO Head Circumference Velocity for (3\sigma)

Description

Part of default WHO-derived tables

Details

Contains three standard deviations for the World Health Organization values of head circumference velocities.

who_hc_vel_3sd_infants.csv.gz

Used in function cleangrowth()

References

World Health Organization. WHO Child Growth Standards: Growth velocity based on weight, length and head circumference: Methods and development. Geneva: World Health Organization; 2009. ISBN 9789241547635.

WHO Maximum Height Velocity for (3\sigma)

Description

Part of default WHO-derived tables

Details

Contains three standard deviations for the World Health Organization values of maximum height velocities.

who_ht_maxvel_3sd.csv.gz

Used in function cleangrowth()

References

World Health Organization. WHO Child Growth Standards: Growth velocity based on weight, length and head circumference: Methods and development. Geneva: World Health Organization; 2009. ISBN 9789241547635.

WHO Height Velocity for (3\sigma)

Description

Part of default WHO-derived tables

Details

Contains three standard deviations for the World Health Organization values of height velocities.

who_ht_vel_3sd.csv.gz

Used in function cleangrowth()

References

World Health Organization. WHO Child Growth Standards: Growth velocity based on weight, length and head circumference: Methods and development. Geneva: World Health Organization; 2009. ISBN 9789241547635.