Overview

The filtering_audit output provides an item-level provenance record for the AIGENIE / GENIE reduction process.

Each row corresponds to an item removed during filtering and documents:

1. what item was removed,
2. when it was removed,
3. which formal criterion caused removal,
4. which statistic triggered the decision, and
5. what the item’s structural profile looked like before removal.

Conceptually:

What item was removed?
        |
        v
At what stage was it removed?
        |
        v
What formal criterion caused removal?
        |
        v
What statistic triggered that criterion?
        |
        v
What structural evidence characterizes the item?

The combined audit is available from:

results$filtering_audit

Type-specific audits are available from:

results$item_type_level$openness$filtering_audit
results$item_type_level$conscientiousness$filtering_audit

Loading the bundled worked example

AIGENIE includes a frozen GPT-5.4 item pool and matching OpenAI text-embedding-3-small embeddings so that users can inspect a realistic GENIE example without regenerating items or embeddings.

library(AIGENIE)
#> Loading required package: EGAnet
#> 
#> EGAnet (version 2.5.0) 
#> 
#> For help getting started, see <https://r-ega.net> 
#> 
#> For bugs and errors, submit an issue to <https://github.com/hfgolino/EGAnet/issues>
#> AI-GENIE loaded. Python dependencies will be configured on first use.
#> For GPU support, run: AIGENIE::install_gpu_support()
#> For local LLM support, run: AIGENIE::install_local_llm_support()

data("items.gpt5.4.example")
data("embeddings.gpt5.4.example")

dim(items.gpt5.4.example)
#> [1] 180   4
dim(embeddings.gpt5.4.example)
#> [1] 1536  180

head(items.gpt5.4.example)
#>                type            attribute
#> 1 conscientiousness        self-efficacy
#> 2 conscientiousness        self-efficacy
#> 3 conscientiousness achievement-striving
#> 4 conscientiousness achievement-striving
#> 5 conscientiousness         perseverance
#> 6 conscientiousness         perseverance
#>                                                        statement ID
#> 1       I am someone who can handle demanding tasks effectively.  1
#> 2     I am someone who feels capable of meeting difficult goals.  2
#> 3   I am someone who sets high standards for my accomplishments.  3
#> 4 I am someone who pushes myself to achieve outstanding results.  4
#> 5        I am someone who keep working until a task is finished.  5
#> 6      I am someone who stay focused even when progress is slow.  6

The full GENIE analysis is intentionally not evaluated while building the vignette because bootstrapped EGA is computationally expensive.

Users can reproduce the analysis interactively with:

gpt54 <- GENIE(
  items = items.gpt5.4.example,
  embedding.matrix = embeddings.gpt5.4.example,
  EGA.model = "glasso",
  EGA.algorithm = "walktrap",
  EGA.uni.method = "louvain",
  uva.cut.off = 0.20,
  run.overall = TRUE,
  all.together = FALSE,
  plot = FALSE,
  silently = FALSE
)

gpt54$filtering_audit

Complete definition of every output element

Formal filtering criteria versus structural diagnostics

A critical distinction is that filtering_audit contains both:

• formal filtering criteria, which determine whether an item is removed, and
• descriptive structural diagnostics, which characterize the item’s psychometric profile.

diagnostic_name  = wTO
diagnostic_value = weighted topological overlap
cutoff           = UVA cutoff

diagnostic_name  = item_stability
diagnostic_value = empirical item stability
cutoff           = stability cutoff

Network loadings do not determine removal

The pre_reduction_*network_loading* fields are diagnostic evidence, not filtering thresholds.

This distinction is essential.

In the bundled GPT-5.4 example, four UVA-removed items had strong primary network loadings and very small cross-loadings:

These items were not removed because they were weak indicators. They were removed because they provided locally redundant information relative to other items.

Example: a UVA removal

Consider Item 4:

ID                    4
removal_stage         UVA
diagnostic_name       wTO
diagnostic_value      0.2797578
cutoff                0.20
uva_sweep             1
redundant_with_ID     46
redundant_wTO         0.2797578

Removed statement:

I am someone who pushes myself to achieve outstanding results.

Redundancy counterpart:

I am someone who push myself to deliver results that stand out.

The appropriate interpretation is:

During the first UVA sweep, Item 4 and Item 46 showed weighted topological overlap of approximately .280. Because this exceeded the prespecified .20 redundancy threshold, Item 4 was removed.

Both items may be strong indicators of achievement striving. The issue is that they provide highly overlapping information.

Why all_redundant_* exists

Redundancy does not always occur as a single isolated pair.

An item may exceed the wTO threshold with several other items:

all_redundant_with_IDs
"15; 22; 41"

all_redundant_wTO
"15=0.231; 22=0.287; 41=0.219"

In that situation:

• redundant_with_ID identifies the primary redundancy counterpart,
• all_redundant_with_IDs preserves the complete redundancy set, and
• all_redundant_wTO preserves the corresponding numerical evidence.

Understanding bootEGA item stability

Conceptually, bootEGA asks:

Across bootstrap replications, does this item continue to belong to the same dimension?

An item with:

item_stability = 0.95

has highly reproducible structural placement.

An item with:

item_stability = 0.414

shows considerably weaker reproducibility.

With a cutoff of .75, the latter item fails the stability requirement.

Example: a bootEGA removal

Item 95 in the bundled worked example had:

removal_stage       bootEGA
item_stability      0.414
cutoff              0.750
stability_deficit   0.336

Its pre-reduction network-loading profile was approximately:

primary loading          = 0.195
strongest cross-loading  = 0.170
loading gap              = 0.025

The primary and strongest cross-loading were almost identical:

\[ 0.195 - 0.170 = 0.025. \]

This is consistent with poorly differentiated dimensional placement.

The correct interpretation is:

Item 95 was removed because its empirical bootEGA item stability was below the prespecified criterion. Its weakly differentiated network-loading profile provides convergent structural evidence for that instability.

The incorrect interpretation is:

Item 95 was removed because its loading was too low.

Network loadings are not themselves the filtering rule.

Understanding the network-loading diagnostics

pre_reduction_strongest_cross_community
pre_reduction_strongest_cross_loading
pre_reduction_strongest_cross_loading_abs

Why the pre_reduction_ prefix matters

The variables deliberately use names such as:

pre_reduction_primary_network_loading
pre_reduction_strongest_cross_loading
pre_reduction_loading_gap

because they describe the item before it was removed.

This prevents a methodological misunderstanding that network loadings constitute the filtering rule. Instead, they provide a snapshot of the item’s structural profile before filtering.

Type-level and pooled audit scopes

For a type-specific audit:

results$item_type_level$openness$filtering_audit

the network diagnostics describe the type-level pre-reduction structure.

The combined audit is available from:

results$filtering_audit

and provides a unified, publication-ready record across item types.

When run.overall = TRUE, GENIE also returns a pooled post-reduction fit:

results$overall

This pooled analysis evaluates the union of the items that survived the type-level reductions. It does not perform another UVA or bootEGA reduction.

Four questions answered by the audit

ID
type
attribute
statement

removal_stage
reason
diagnostic_name
diagnostic_value
cutoff

uva_sweep
redundant_with_ID
redundant_with_statement
redundant_wTO
all_redundant_with_IDs
all_redundant_wTO

boot_run
item_stability
stability_deficit

pre_reduction_EGA_community
pre_reduction_primary_network_loading
pre_reduction_primary_network_loading_abs
pre_reduction_strongest_cross_community
pre_reduction_strongest_cross_loading
pre_reduction_strongest_cross_loading_abs
pre_reduction_loading_gap

Worked GPT-5.4 result

The bundled regression example produces:

Conscientiousness: 90 -> 87 items
Initial NMI:       1.000
Final NMI:         1.000

Openness:          90 -> 88 items
Initial NMI:       0.9555
Final NMI:         1.000

Overall:           180 -> 175 items
Initial NMI:       0.9864
Final NMI:         1.000

The audit identifies:

• four UVA redundancy removals, and
• one bootEGA instability removal.

The two stages therefore serve complementary purposes:

• UVA removes local redundancy.
• bootEGA removes insufficiently stable dimensional assignments.

Reduction summaries

In addition to the item-level audit, each type-level result contains a stage-by-stage reduction_summary:

results$item_type_level$openness$reduction_summary
results$item_type_level$conscientiousness$reduction_summary

These summaries document the trajectory from the initial item pool through UVA and bootEGA to the final retained pool, including changes in item count and NMI.

Summary

The filtering_audit is more than a deletion log.

It provides an item-level psychometric provenance record for the AIGENIE / GENIE reduction process.

For every removed item, it records:

• the item itself,
• the filtering stage,
• the formal removal criterion,
• the exact diagnostic statistic,
• the decision threshold,
• the relevant redundancy or stability evidence, and
• the item’s pre-reduction network-loading profile.

AIGENIE / GENIE therefore return not only a reduced item pool, but also a transparent and publication-ready justification for every filtering decision.