pairwiseLLM 1.3.1

New Features

Adaptive linking

• Added cross-set adaptive linking through adaptive_rank(), with one-spoke and multi-spoke hub-and-spoke run modes for placing separately ranked sets on a shared scale.
• Phase A can run canonical within-set ranking, import saved Phase A artifacts, or combine imported and newly run sets. Phase B uses anchored-joint estimation with hard-locked hub parameters, globally shared judge parameters, and concurrent spoke processing.
• Added D-optimal cross-set candidate selection, spoke-aware comparison budgets, and fixed per-refit held-out probes with explicit coverage, calibration, and stopping gates.
• Added persistent Phase A manifests and linking sessions, resume support, link_stage_log, and explicit blocker and invalid-judgment reporting for auditable long-running workflows.

Model support & live API improvements

• Added a separate Vertex AI Gemini live backend via backend = "vertex" with VERTEX_API_KEY.
  • Vertex remains live-only in this series; generic batch wrappers reject backend = "vertex" explicitly rather than routing it through Gemini batch helpers.
• Clarified provider-specific service_tier behavior for Gemini Developer API and Vertex AI Gemini API in the package docs, README, and vignettes.
• Added request-shape handling for the OpenAI GPT-5.x and GPT-5.6 model families.

Bayesian BTL usability

• Added exported build_btl_results_data() to convert conventional ID1/ID2/better_id outcomes into the canonical input required by fit_bayes_btl_mcmc().
• Standalone Bayesian BTL fits now report reliability and sampler diagnostics through the same result and log surfaces used by adaptive workflows.

Improvements

• Reduced adaptive-linking refit, candidate-generation, logging, and checkpoint overhead for large runs, including bounded candidate construction and analytic anchored-joint derivatives.
• Hardened adaptive-linking resume behavior, Phase A stopping scope, spoke routing, probe scheduling, and Phase B pair accounting.

Documentation

• Updated the backend support matrix to distinguish Gemini Developer API from Vertex AI Gemini API and to document the live-only Vertex scope.
• Replaced the undated supported-model list with a dated, machine-readable compatibility registry that separates backend implementation, live/batch testing, and current provider availability.
• Corrected stale provider identifiers, API-key and CmdStan setup guidance, OpenAI Flex wording, cost-estimator summaries, and broken public examples.

Bug fixes

• Fixed Anthropic extended-thinking live requests when users explicitly set the required temperature = 1.
• OpenAI Responses live requests now forward and enforce an explicitly supplied max_output_tokens limit.
• Changed omitted temperature and top_p controls to use model/provider defaults instead of injecting deterministic sampling values. Provider-required reasoning constraints are still enforced.
• Provider smoke runs now exit unsuccessfully when a selected API key is absent, unless missing-key skips were explicitly allowed.

pairwiseLLM 1.3.0

New Features

Adaptive pairing & ranking framework

• Introduced an adaptive pairing / adaptive ranking framework designed to reduce unnecessary comparisons through uncertainty-aware pair selection and Bayesian inference.

• Added adaptive_rank(), the primary user-facing wrapper that runs the complete adaptive workflow end-to-end, including warm start, adaptive pairing rounds, Bayesian BTL refits, diagnostics, and stopping.

• Advanced control is available via:

  • adaptive_rank_start() — initialize an adaptive run and state
  • adaptive_rank_run_live() — execute live adaptive comparisons
  • adaptive_rank_resume() — resume interrupted or long-running runs
    These functions are intended for custom orchestration and fault-tolerant execution.

• Adaptive pairing is organized into rounds that balance global scale identification and local refinement using a mixture of anchor, long-range, mid-range, and local comparisons.

• The adaptive controller tracks a global identifiability state based on Bayesian diagnostics and agreement between online (TrueSkill) and global (BTL) rankings. Once the global scale is identified:

  • long-range comparisons are automatically tapered,
  • comparison budget is reallocated toward local and boundary-refining pairs,
  • exploration rates are reduced to focus on decision-relevant uncertainty.

• Long-range comparisons are additionally posterior-gated in later stages, preventing wasted comparisons on pairs that are already decisively ordered.

• Late-stage local pairing prioritizes near-tie pairs, with limited, auditable overrides to degree caps when especially informative comparisons are blocked.

• Adaptive runs produce fully auditable step-, round-, and refit-level logs, recording candidate generation, fallbacks, gating decisions, quota reallocations, and stopping criteria.

• All adaptive workflows use standardized configuration, state, and logging contracts to ensure reproducibility and future extensibility.

Bayesian Bradley–Terry–Luce (BTL) modeling

• Added a fully Bayesian Bradley–Terry–Luce (BTL) model implemented via CmdStan, providing posterior uncertainty estimates for item skill parameters.
• New entrypoint fit_bayes_btl_mcmc() enables direct posterior inference from pairwise comparison data, independent of or integrated with adaptive workflows.
• Supports multiple model variants (including error and positional bias extensions) and optional refitting on increasing subsets of comparisons.
• Bayesian BTL outputs integrate seamlessly with adaptive ranking utilities (summarize_items(), summarize_refits()), serving as the statistical backbone for adaptive pairing decisions.

Model support & live API improvements

• Added support for the Gemini Flash model gemini-3-flash-preview for live pairwise comparisons.
• Added support for OpenAI service tiers / priority routing via service_tier for applicable live models.
  • Enables tiers such as "flex" and "priority" when supported by the selected model.
  • Integrated into the live submission path without requiring changes to calling code.

Documentation

• Expanded and clarified documentation for adaptive ranking, Bayesian BTL, and live model configuration.
• Updated examples to reflect new adaptive and Bayesian APIs.

pairwiseLLM 1.2.0

New Features

• Parallel Processing:
  • submit_llm_pairs() and backend-specific live functions (OpenAI, Anthropic, Gemini, Together, Ollama) now support parallel execution via parallel = TRUE and workers = n (requires the {future} package).
• Incremental Saving & Resume:
  • Added save_path argument to live submission functions. Results are saved to CSV incrementally, allowing interrupted jobs to resume automatically by skipping previously processed pairs.
• Robust Error Handling:
  • Failed API calls no longer stop the entire process. Failures are captured and returned separately, allowing for easier inspection and re-submission.
• Added estimate_llm_pairs_cost() to estimate costs in live and batch mode.
• Introduced llm_submit_pairs_multi_batch() and llm_resume_multi_batches() to split large comparison sets across multiple batches and resume polling later. These helpers support writing per‑batch and combined results, along with an optional jobs registry.

Bug fixes

• The prompt format for anthropic batch comparisons now match the anthropic live format.
• Reverse consistency functions can now handle duplicate pairs.

Breaking Changes

• submit_llm_pairs() and its backend-specific counterparts now return a list containing two elements: $results (a tibble of successful comparisons) and $failed_pairs (a tibble of inputs that failed). Previous versions returned a single tibble.

pairwiseLLM 1.1.0

Models

• Added GPT-5.2
• Ensured models can be called with date format, e.g. gpt-5.2-2025-12-11
• Default temperature setting is set to 0 for non-reasoning models, provider default for reasoning models (typically 1)

Tests

• Tests added to improve coverage

Documentation

• Changed pkgdown site layout
• Added codemeta.json
• Added repo logo
• Updated function examples
• Add references to Description

Miscellaneous

• No longer set global variables, now done in individual functions
• Added verbose option in fit_bt_model() and summarize_bt_fit()
• Moved null coalescing helper to separate R file
• Changed validation of API keys in multiple functions

pairwiseLLM 1.0.0

• Initial release.
• Unified live and batch LLM comparison framework (OpenAI / Anthropic / Gemini).
• Live support for Together.ai and local Ollama backends.
• Tools for Bradley–Terry and Elo models, positional bias checks