edgemodelr 0.4.3

CRAN Feedback Fixes

• Missing <cstdlib> in four more translation units: llama-mmap.cpp (posix_memalign, free), llama-model-loader.cpp (getenv, atoi), llama-vocab.cpp (strtol) and llama-batch.cpp (getenv, atoi, malloc, free) used C library functions without including the header that declares them. Recent libc++ releases no longer supply these through transitive includes, so clang 23 and fedora-clang rejected them. llama-batch.cpp was not in the compiler report, because the build stopped before reaching it; it was found by auditing every file in src/ for this defect rather than by patching the reported lines.

• 38 further missing standard includes across 26 files: the first audit covered only the C headers (<cstdlib>, <cstring> and so on), but the libc++ change behind the clang 23 failures removes transitive includes of C++ headers too. A second audit covering the C++ standard library found std::min, std::copy, std::advance, std::distance, std::unique_ptr, std::is_same, std::out_of_range, std::array, std::pair, std::move and isspace/toupper used in files that never included <algorithm>, <iterator>, <memory>, <type_traits>, <stdexcept>, <array>, <utility> or <cctype>. These build under libstdc++, which supplies the declarations transitively, and would have failed under libc++.

• False -Warray-bounds in llama-grammar.cpp: with optimisation and -Wall now applied to the vendored sources, GCC 12 and later reported a false positive against a zero-length array when llama_grammar_advance_stack() copied a vector it had already proven empty. R CMD check treats that warning as significant. The branch now default-constructs the empty stack instead of copying it, which is equivalent and does not trip the warning.

• CFLAGS and CXXFLAGS now reach the whole package: the pattern rules for ggml/ and llama/ passed only package-local flags to the compiler, so flags set by the user or the site Makevars were applied to 6 of 161 source files and silently dropped for the other 155. Every rule in src/Makevars and src/Makevars.win now appends its engine-specific additions to $(ALL_CFLAGS) or $(ALL_CXXFLAGS) instead of replacing them. The variables were renamed to GGML_EXTRA_* and GENERIC_EXTRA_* to make that contract explicit.

edgemodelr 0.4.2

Compiler Portability Fixes

• Installation failure on macOS x86_64 (CRAN r-release-macos-x86_64): path_str() in ggml/ggml-backend-reg.cpp chose between two conversion branches using #if defined(__cpp_lib_char8_t). The libc++ shipped in MacOSX11.sdk defines that macro under -std=gnu++20 but its std::filesystem::path::u8string() still returns std::string, so the branch guarded for std::u8string failed to compile. The function now deduces the return type with auto and converts byte-wise, which is correct whether the element type is char or char8_t.

• C++ standard pinned to C++17 (CXX_STD = CXX17 in src/Makevars and src/Makevars.win): the bundled llama.cpp and GGML sources target C++17. Without a pin, each platform applied its own default, so the same source compiled as C++17 under R 4.5 and as C++20 under R 4.6 on macOS. Every CRAN flavour now builds the dialect the vendored engine is written for.

• Installation failure on clang 23: three translation units used symbols they never included a header for, relying on transitive includes that newer libc++ releases no longer provide. ggml/gguf.cpp calls errno without <cerrno>, and llama/llama-graph.h and llama/llama-context.cpp call getenv() and atoi() without <cstdlib>. Each file now includes what it uses.

edgemodelr 0.4.1

CRAN Resubmission Fixes

• Stderr references in compiled objects (CRAN auto-check NOTE on Debian): the previous CRAN cleanup (commit d8870bd) added stdio suppression to 7 upstream files but missed ggml/ggml.c and ggml/ggml-opt.cpp. Both now include the same #ifdef USING_R macro block that neutralizes printf, fprintf, fputs, fflush, stderr, and stdout. These calls were diagnostic-only and were already silent at runtime via the installed log callback; now the symbols never reach the compiled object files either.

edgemodelr 0.4.0

Structured Output, Embeddings, RAG, and API Server

New Features

• Grammar-constrained generation (edge_grammar_completion()): Force model output to conform to a GBNF grammar specification. Ensures valid, parseable structured output (JSON, enums, numbers, etc.) using llama.cpp’s native grammar sampler.

• JSON schema helper (edge_json_grammar()): Convert a simple R list schema into a GBNF grammar string. Supports string, number, integer, boolean fields and enum (character vector) constraints.

• Structured data extraction (edge_extract()): High-level function that combines prompt construction with grammar-constrained generation to extract structured data from text. Returns a parsed R list (requires jsonlite).

• Text classification (edge_classify()): Classify text into predefined categories using grammar constraints. Supports single text and batch (vectorized) classification. Output is guaranteed to be one of the specified categories.

• Text embeddings (edge_embeddings()): Extract dense vector embeddings from any loaded model. Returns a numeric matrix (n_texts x n_embd) suitable for clustering, semantic search, similarity computation, and RAG pipelines. Supports optional L2 normalization.

• Cosine similarity (edge_similarity(), edge_similarity_matrix()): Compute pairwise cosine similarity between embedding vectors. Matrix version efficiently computes all-pairs similarity using normalized matrix multiply.

• Embedding dimension query (edge_model_n_embd()): Query the embedding dimension of a loaded model.

• Batch processing (edge_map()): Apply a prompt template over a vector of texts with progress reporting. Supports both string templates with {text} placeholder and custom prompt functions. Optional grammar constraint for structured batch output.

• Batch extraction (edge_extract_batch()): Extract structured data from multiple texts, returning a data frame with one row per input.

• RAG document indexing (edge_index_documents()): Build a semantic embedding index from a directory of text files or a character vector. Automatic chunking with configurable size and overlap.

• RAG semantic search (edge_search()): Find the most relevant text chunks for a query using cosine similarity over the embedding index.

• RAG question answering (edge_ask()): Retrieval-augmented generation that retrieves relevant context from an index and generates a grounded answer. Supports custom system prompts and optional context return for debugging/transparency.

• Plumber API server (edge_serve()): Serve a model as a local OpenAI-compatible REST API. Endpoints: /v1/completions, /v1/chat/completions, /v1/embeddings, /v1/models, /health. Supports optional API key authentication and CORS. Requires plumber.

• Qwen3 model family in edge_list_models(): Added Qwen3-0.6B, 1.7B, 4B, and 8B pre-configured entries from the unsloth GGUF repository.

• Friendly names in edge_download_model(): Now accepts model names from edge_list_models() (e.g., edge_download_model("Qwen3-0.6B")) in addition to HuggingFace repo IDs. Filename is auto-resolved from the model registry.

• httr download fallback: .robust_download() now tries httr::GET before R’s download.file, improving reliability on corporate networks with custom SSL certificates or proxy configurations.

• SIMD optimization warning: On package load, warns if running without SIMD (generic mode) and suggests reinstalling from source with EDGEMODELR_SIMD=NATIVE for faster inference.

Bug Fixes

• Fixed grammar-constrained generation failures (issue #41): edge_grammar_completion(), edge_extract(), and edge_extract_batch() were unusable due to two bugs. First, edge_json_grammar() emitted rule names like field_1 containing underscores, which llama.cpp’s grammar parser rejects (only [a-zA-Z0-9-] is allowed in rule identifiers). Renamed to field-1. Second, llama_sampler_accept() throws “Unexpected empty grammar stack” when a token fully satisfies the grammar; the binding now catches this and terminates cleanly, same as end-of-generation handling.

• Fixed crash from silent context size override (issue #40 item 11): Removed the auto-reduction of n_ctx for small models that silently changed the user’s requested context size. This caused segfaults when prompts exceeded the reduced context. Context is now used as-is. Minimum n_ctx lowered from 512 to 128 for short-task use cases.

• Fixed prompt echo in completion output (issue #40 item 1): edge_completion() previously returned prompt + generated_text. Now returns only the generated text, matching user expectations.

• Added prompt length validation: All completion functions now validate that the tokenized prompt fits within the model’s context window before calling llama_decode(). Exceeding the context now raises a clear R error instead of crashing the process.

• Model-native chat templates (issue #40 item 7): New edge_chat_completion() function reads the model’s chat template from GGUF metadata (via llama_chat_apply_template) and formats messages correctly for each model architecture (ChatML, Llama, Gemma, etc.). build_chat_prompt() updated to accept an optional ctx parameter for native template formatting, with ChatML as the generic fallback (replacing the old Human:/Assistant: format).

Use Cases Unlocked

• Sentiment analysis: edge_classify(ctx, text, c("positive", "negative", "neutral"))
• Entity extraction: edge_extract(ctx, text, list(name = "string", role = "string"))
• Data labeling: Batch classify thousands of rows with guaranteed valid labels
• Semantic search: Embed documents and queries, find nearest neighbors
• Document clustering: Compute similarity matrices, feed to hclust/kmeans
• RAG foundations: Embed corpus, retrieve relevant context for generation

edgemodelr 0.3.0

CUDA GPU Support and Qwen3 Tokenizer Fix

New Features

• CUDA GPU acceleration (Windows): New edge_install_cuda() and edge_install_cuda_toolkit() functions set up GPU inference automatically.
  • edge_install_cuda() downloads the matching ggml-cuda dynamic backend from llama.cpp releases and extracts the companion ggml-base.dll / ggml.dll runtime libraries.
  • edge_install_cuda_toolkit() copies nvcudart_hybrid64.dll from the Windows DriverStore (already on any NVIDIA-driver machine, no download required) and fetches cublas64 / cublasLt64 from NVIDIA’s redistrib server.
  • edge_reload_cuda() activates the CUDA backend in the current R session without restarting R.
  • edge_cuda_info() reports whether CUDA is installed and active.
  • Pass n_gpu_layers = -1L to edge_load_model() for full GPU offload.
  • Tested on NVIDIA RTX 5070 Ti (Blackwell sm_120, CUDA 13.1, 12 GB VRAM): Qwen3-14B loads in 3.4 s with full VRAM offload.
• Updated llama.cpp to build b8179 (GGML 0.9.7): Brings all upstream model architecture updates, sampler improvements, and quantization fixes.

Bug Fixes

• Qwen3 / QWEN2 tokenizer 40-minute load time (8000× speedup): The QWEN2 byte-level regex pattern caused GCC’s std::regex to spend 40+ minutes in exponential backtracking. Added a hand-written fast path unicode_regex_split_custom_qwen2() in unicode.cpp, matching the logic of the existing llama-3 fast path. Qwen3-14B now loads in 0.3 s on CPU (3.4 s on GPU including VRAM transfer). Covers QWEN2 and QWEN3.5 variants.

CRAN Compliance

• Replaced abort() in ggml_abort() with raise(SIGABRT) under #ifdef USING_R; replaces abort() token in ggml.cpp with std::terminate().
• Guarded ggml_print_backtrace() body and fflush(stdout) / fprintf(stderr, …) in ggml_abort() with #ifndef USING_R to remove _Exit, stdout, and stderr symbol references from ggml.o on macOS.
• Added #define _GNU_SOURCE to ggml-cpu.c (required for SCHED_BATCH, CPU_ZERO, pthread_setaffinity_np on Linux).
• CXX_STD = CXX17 replaces -std=c++17 in PKG_CXXFLAGS in both Makevars and Makevars.win.
• -fno-builtin-printf added to GGML_CFLAGS to suppress printf → puts optimizations.
• Man pages added for edge_install_cuda, edge_install_cuda_toolkit, edge_reload_cuda, edge_cuda_info.

edgemodelr 0.2.0

SIMD Optimizations for Faster CPU Inference

New Features

• Flash attention support: Enabled by default in edge_load_model() via flash_attn = TRUE. Reduces memory usage and improves attention computation speed on CPU.

• Full hardware thread utilization: Removed the 4-thread cap for small contexts. edge_load_model() now uses all available CPU threads by default, with n_threads_batch set to max for prompt processing.

• User-configurable threading: New n_threads parameter in edge_load_model() allows explicit control over CPU thread count. Pass NULL (default) for auto-detect or an integer to limit cores.

• Apple Accelerate framework (macOS): Automatically links the Accelerate framework on macOS builds, enabling hardware-accelerated vDSP vector operations for faster matrix math.

• Compiler auto-vectorization: Added -ftree-vectorize to GGML compilation flags on all platforms, allowing GCC/Clang to generate SIMD instructions for eligible loops beyond the hand-tuned GGML kernels.

Existing Features

• SIMD-optimized build system: Replaced generic scalar fallback with architecture-aware SIMD detection in both Makevars (Unix) and Makevars.win (Windows)

  • x86_64: Enables SSE4.2 baseline by default (universal since Intel Nehalem 2008)
  • aarch64/arm64: NEON support built into the ABI (no extra flags needed)
  • Other architectures: Automatic generic fallback

• User-configurable SIMD levels: Set EDGEMODELR_SIMD environment variable before install to select optimization level:

  • GENERIC: Scalar fallback (maximum compatibility)
  • SSE42: SSE4.2 baseline (default on x86_64)
  • AVX: AVX + F16C (Intel Sandy Bridge 2011+)
  • AVX2: AVX2 + FMA + F16C (Intel Haswell 2013+, recommended)
  • AVX512: AVX-512 (Intel Skylake-X 2017+)
  • NATIVE: Uses -march=native for maximum performance on the build machine

• edge_simd_info(): New function to query compile-time SIMD status including architecture, compiler features, and GGML optimization flags

• x86 architecture-specific quantization: Enabled optimized x86 quantization kernels (arch/x86/quants.c, arch/x86/repack.cpp) with SIMD-accelerated dot products and matrix operations

Performance

• 15-40% faster inference on x86_64 with SSE4.2 baseline vs generic scalar
• Up to 2-3x faster with AVX2 for quantized model operations
• SSSE3-accelerated integer multiply-accumulate for quantized dot products

edgemodelr 0.1.5

CRAN Policy Fixes

Bug Fixes

• Fixed donttest examples: Changed resource-intensive examples from \donttest{} to \dontrun{} to prevent downloading multi-GB models during CRAN checks

• Fixed M1 Mac compiler warnings: Added explicit static_cast<> for:

  • double to float conversions for temperature/top_p parameters
  • size_type to int32_t conversions for buffer size parameters

• Fixed connection handling: Replaced on.exit() with tryCatch/finally for proper connection cleanup in loops (thanks @eddelbuettel)

edgemodelr 0.1.4

Performance Optimizations for Small Language Models

New Features

• Small Model Configuration Helper: New edge_small_model_config() function provides optimized settings for small models (1B-3B parameters)
  • Device-specific presets: mobile, laptop, desktop, and server
  • Adaptive configuration based on model size and available RAM
  • Built-in performance tips and recommendations
  • Automatic parameter tuning for optimal inference speed
• Adaptive Batch Processing: Intelligent batch size optimization based on context length
  • Small contexts (≤512): Uses up to full context for batching
  • Medium contexts (512-2048): Uses 1/2 context for optimal throughput
  • Large contexts (2048-4096): Uses 1/4 context to balance speed and memory
  • Very large contexts (>4096): Caps at 2048 tokens for stability
• Smart Thread Allocation: Context-aware CPU thread management
  • Small models automatically limit threads to avoid overhead
  • Reduces CPU contention on resource-constrained devices
  • Improves inference speed for models with contexts ≤2048 tokens
• Automatic Context Optimization: Model size-based context tuning
  • Small models (<1GB): Optimized to 1024 tokens for faster inference
  • Medium models (1-2GB): Set to 1536 tokens for balanced performance
  • Large models (>2GB): Maintains 2048+ tokens for quality
  • User override available via n_ctx parameter

Performance Improvements

• Faster Small Model Inference: 15-30% speed improvement for small models through optimized batch and thread settings
• Reduced Memory Footprint: Better memory efficiency for resource-constrained environments
• Lower Latency: Optimized thread allocation reduces context switching overhead
• Better Scalability: Adaptive configurations scale from mobile devices to servers

Examples and Documentation

• Small Model Optimization Example: Comprehensive example demonstrating all optimization features
  • Configuration comparison across device types
  • Performance benchmarking workflow
  • Best practices for different model sizes
  • Manual tuning guidelines
• Enhanced Testing: New test suite for small model configuration
  • Tests for all device target configurations
  • Validation of adaptive parameter adjustments
  • Safety checks for edge cases

Technical Details

• Improved C++ bindings with adaptive batch size calculations
• Enhanced R API with intelligent parameter defaults
• Better integration between model size detection and configuration
• Comprehensive documentation for optimization features

edgemodelr 0.1.2

Major New Features

Ollama Integration

• Native Ollama Support: Complete integration with Ollama models through automatic model discovery and SHA-256 hash-based loading
• edge_find_ollama_models() - Discover all locally available Ollama models across platforms (Windows, macOS, Linux)
• edge_load_ollama_model() - Load Ollama models using convenient SHA-256 hash prefixes instead of full file paths
• test_ollama_model_compatibility() - Built-in compatibility testing for Ollama models
• Cross-platform Model Detection: Robust model discovery supporting standard installations, snap packages (Linux), and various Windows configurations
• Windows OneDrive Compatibility: Enhanced path detection that properly handles Windows OneDrive document folder redirections

Comprehensive Examples Suite

• Structured Learning Path: Complete examples directory with progressive difficulty levels (Beginner → Intermediate → Advanced)
• 01_basic_usage.R: Fundamental operations including model loading, text generation, parameter tuning, and error handling
• 02_ollama_integration.R: Complete Ollama workflow with model discovery, hash-based loading, and compatibility testing
• 03_streaming_generation.R: Real-time streaming text generation with interactive chat interfaces and callback processing
• 04_performance_optimization.R: Advanced performance tuning including GPU acceleration, benchmarking, memory management, and batch processing
• examples/README.md: Comprehensive documentation with learning paths, troubleshooting guide, and customization instructions

Package Structure Improvements

• Organized File Structure: Consolidated all examples into structured examples/ directory with consistent formatting
• Enhanced Documentation: Improved inline documentation and example comments throughout

edgemodelr 0.1.1

Bug Fixes and Improvements

Compilation Fixes

• macOS Boolean Conflicts: Completely resolved Boolean enum conflicts by avoiding problematic system headers and using direct function declarations
• Filesystem Compatibility: Added comprehensive fallback implementation for disabled std::filesystem on macOS builds
• Header Protection: Implemented robust cross-platform header inclusion strategy that works with R, Rcpp, and system headers
• System Header Workarounds: Replaced <mach-o/dyld.h> inclusion with direct function declarations to avoid enum conflicts
• Format Attribute Warnings: Suppressed unsupported printf format attribute warnings on macOS Apple Clang compiler
• CRAN Compliance: Removed non-portable optimization flags (-march=native, -mtune=native, etc.) from Makevars for CRAN compatibility
• Cross-platform Build: Enhanced Makevars configuration for better macOS compatibility with R package requirements

Demo and Documentation Updates

• Modern UI: Updated streaming chat demo with modern bslib interface for enhanced user experience
• Documentation: Improved documentation for edge_clean_cache() function
• Examples: Enhanced streaming chat example with better UI components

Technical Improvements

• Build System: Updated Makevars files for improved compilation on Windows and Unix systems
• Core Bindings: Enhanced C++ bindings for better performance and stability

edgemodelr 0.1.0

Initial CRAN Release

New Features

• Local LLM Inference: Complete R interface for running large language models locally using llama.cpp and GGUF model files
• Model Management: Built-in functions for downloading and managing popular models from Hugging Face
• Text Generation: Support for both blocking and streaming text completion
• Interactive Chat: Real-time streaming chat interface with conversation history
• Privacy-First: All processing happens locally without external API calls

Core Functions

• edge_load_model() - Load GGUF model files for inference
• edge_completion() - Generate text completions
  
• edge_stream_completion() - Stream text generation with real-time callbacks
• edge_chat_stream() - Interactive chat session with streaming responses
• edge_free_model() - Memory management and cleanup
• is_valid_model() - Model context validation

Model Management

• edge_list_models() - List pre-configured popular models
• edge_download_model() - Download models from Hugging Face Hub
  
• edge_quick_setup() - One-line model download and setup

System Support

• Self-contained: Includes complete llama.cpp implementation
• Cross-platform: Works on Windows, macOS, and Linux
• CPU optimized: Runs efficiently on standard hardware
• Memory efficient: Support for quantized models

Documentation

• Comprehensive getting started vignette
• Complete API documentation with examples
• README with extensive usage examples
• Test coverage for all major functionality

Technical Implementation

• C++17 integration via Rcpp
• Real-time token streaming with callback support
• Automatic memory management with RAII
• Robust error handling and validation
• Thread-safe model operations

This release provides a complete, production-ready solution for Local Large Language Model Inference Engine in R, enabling private, offline text generation workflows.