starburst 0.3.9 (2026-07-22)

Cost & cleanup fixes (important)

• The max_hourly_cost guard now works on the EC2 backend (the default). Previously estimate_cost() returned per_hour for Fargate but total_per_hour for EC2, and the guard read per_hour — which is NULL on EC2 — so the cost ceiling was silently never enforced on the default backend. estimate_cost() now returns a normalized hourly_rate for every backend, and the guard, cost display, and starburst_map()/starburst_cluster() all use it.
• cost_alert_threshold is now enforced — it warns when a plan’s estimated hourly rate exceeds it (previously it was stored but never read).
• starburst_config(auto_cleanup_s3 = …) now actually controls S3 cleanup. The runtime read a different key (getOption("starburst.cleanup_s3")) and ignored the stored setting; cleanup now resolves option → auto_cleanup_s3 config → default TRUE.
• starburst_map()/starburst_cluster() cost estimates match the backend used — they forward launch_type/instance_type/use_spot, so an EC2 job no longer reports a Fargate price.
• Cost estimates now use live AWS pricing (Pricing API for On-Demand, EC2 spot-price history for Spot), cached per session with a built-in static-rate fallback when offline. Reported costs are estimates (measured runtime × current rate), not AWS billing figures; docs reworded accordingly.

New features

• starburst_setup() now provisions the default EC2 capacity provider (for c7g.xlarge) so the default EC2 backend works out of the box — previously the first starburst_map()/plan(starburst) run failed because no capacity provider existed. It is created at DesiredCapacity = 0 (no billable instances launch during setup). Pass setup_ec2 = FALSE to skip (Fargate-only users). (#37)
• Custom EC2 instance types are now provisioned automatically on first use. Selecting a non-default instance_type (e.g. c8a.xlarge) previously failed with an opaque “AutoScalingGroup not found” because only the default was provisioned. Now the capacity provider/ASG is created on demand at first launch (a one-time ~1–2 min step; no instances launched by provisioning), or on failure you get the exact starburst_setup_ec2(instance_types = "…") command to run.

Behavior changes

• Detached session$collect() now returns an entry for every terminal task, including failures. A failed task comes back as a structured failure (list(error = TRUE, message = …, task_id = …)) alongside successful results, instead of being silently omitted — matching the documented contract.
• Clarified session$cleanup(): by default it stops workers and marks the session terminated but preserves S3 objects; pass force = TRUE to also delete them. (Behavior unchanged; the docs were wrong and are now correct.)

Breaking changes

• Removed the platform argument from starburst_map() and starburst_cluster(). It was ignored (architecture is inferred from instance_type: Graviton *g.* → ARM64, else x86_64), and its "X86_64" default even contradicted the ARM64 c7g.xlarge default. Drop it from calls; the backend picks the architecture from instance_type. (platform remains on starburst_estimate(), where it is functional.)
• Renamed starburst_config(max_cost_per_job=) to max_hourly_cost. The limit was always enforced as an hourly rate (USD/hour), not a total-job cap; the new name matches the behavior. Update any starburst_config(max_cost_per_job = ...) calls to max_hourly_cost = ....

Bug fixes

• starburst_map() and starburst_cluster() now accept launch_type, instance_type, and use_spot and forward them to the backend. Previously these functions had no backend arguments, so a call like starburst_map(x, f, launch_type = "FARGATE") silently passed launch_type to the mapped function .f (typically an “unused argument” error) instead of switching the backend — contradicting the 0.3.7 migration note. Backend selection now works uniformly across plan(starburst), starburst_map(), starburst_cluster(), and starburst_session().
• Fixed broken/misleading examples: the README detached-session example used a nonexistent detached = TRUE argument; S3-read examples used base read.csv()/ readRDS(url()) which cannot read s3://; and the API rate-limit example aggregated to 10× its stated global limit across workers.
• Public base images are now built multi-arch (amd64 + arm64). They were published amd64-only, so use_public_base = TRUE failed the multi-arch environment build with “no match for platform in manifest” for arm64. The build-base-images workflow now builds both architectures (matching the env build), so the public base path works. (#39)

Documentation

• Consistency sweep addressing an external documentation review: EC2 is now consistently presented as the default/recommended backend with Fargate as the optional alternative; the README and example console output match the current engine (ASCII status markers, one task per input element — no chunking step); the Getting Started tutorial leads with starburst_map(); a grouped reference index and an architecture diagram were added; and starburst_config() keys are now catalogued.
• Trimmed the Getting Started tutorial: the inline Monte Carlo / bootstrap / genomics snippets (which showed unbatched future_map(1:10000, …) and hand-written speedup claims) now link to the canonical, measured example articles and the Workload Shapes / Performance guides. The doc-consistency guard (tools/check-docs.R) now also flags naive future_map()/future_lapply() big-count calls, not just starburst_map().
• Removed the historical ROADMAP.md (a pre-1.0, Fargate-first implementation plan that also carried a stale “MIT” license note); the README Roadmap section is the single source of truth.

starburst 0.3.8 (2026-03-06)

Reliability & CRAN readiness

Bug fixes

• Worker script: Removed an invalid timeout parameter from the paws client configuration that could cause workers to fail on startup.
• Environment sync: ensure_environment() now walks up the directory tree to find the package-root renv.lock when invoked from a subdirectory (e.g. during tests), and the environment image hash is computed consistently between build and lookup, so cached worker images are reused correctly.
• Hardened safe_system() usage and resolved qs2 serialization edge cases surfaced by integration tests.

Documentation & packaging

• All \donttest{} examples are guarded with starburst_is_configured() so they are safe to run without AWS credentials.
• Pre-built vignettes shipped in inst/doc/ and assorted R CMD check / CRAN incoming-feasibility fixes. Package accepted on CRAN.

starburst 0.3.7 (2026-02-17)

Default backend is now EC2

• BREAKING CHANGE: the default launch_type changed from FARGATE to EC2, with spot instances enabled by default (use_spot = TRUE, instance_type = "c7g.xlarge"). EC2 avoids Fargate cold starts, is 50–90% cheaper with spot, and supports warm pools and a wider range of instance types. Fargate remains fully supported — pass launch_type = "FARGATE" to plan(starburst, ...), starburst_map(), starburst_cluster(), or starburst_session() to keep the previous behavior.
• Completed the EC2 backend implementation with working end-to-end execution (warm pools, capacity providers, spot support).

Migration

• If you relied on the Fargate default, add launch_type = "FARGATE" to your plan()/starburst_*() call. No other changes are required; the task API is unchanged across backends.

starburst 0.3.6 (2026-02-16)

AWS Integration Testing & Documentation

Major milestone: Complete AWS integration testing infrastructure and documentation site launch.

New Features

• Comprehensive AWS Integration Testing (#4b36310)
  • GitHub Actions workflow for automated AWS testing
  • Local test runner script (run-aws-tests.sh)
  • Multiple test suites: quick, detached-sessions, integration-examples, ec2, cleanup
  • OIDC authentication for secure CI/CD
  • Comprehensive TESTING.md documentation
  • Weekly scheduled testing runs
• Documentation Site (https://starburst.ing)
  • Custom domain with HTTPS enabled
  • Full pkgdown site with all 29+ exported functions
  • 12 vignettes including security and troubleshooting guides
  • Example scripts and runnable code

Bug Fixes

• Test Suite
  • Fixed missing readLines() stubs in environment tests (#6514229)
  • Fixed missing get_starburst_config() stub in Docker tests (#dad7124)
  • Fixed environment variable handling in integration tests (#78dbc2c)
  • Improved AWS credential handling in test script (#0e916f2)
• CI/CD
  • Removed docs/ directory conflict in pkgdown workflow (#bc97dca)
  • Complete pkgdown configuration for all functions and vignettes (#7a562f3)
  • Removed non-existent function from pkgdown config (#4fe5d97)

Test Results

• Unit Tests: 202 passing, 0 failures
• Integration Tests: 34 ready to run (local + CI)
• CI Status: All checks passing on 5 platforms (Ubuntu, Windows, macOS)

Documentation

Full documentation now available at starburst.ing

starburst 0.3.5 (2026-02-15)

Bug Fixes & Improvements

Changes

• Docker Image Versioning (#707ee78)
  • Include package version in Docker image hash
  • Ensures environment rebuilds when package version changes
  • Prevents stale Docker images with old code
• Serialization Update (#cbfad21)
  • Changed worker scripts from qs to qs2 package
  • Improved compatibility and performance
  • Consistent with main package dependencies

Assessment

Minor version bump with critical bug fixes for Docker caching and serialization consistency.

starburst 0.3.4 (2026-02-14)

Code Quality Fix

Changes

• Fixed vapply calls (#b96ace1)
  • Added missing FUN.VALUE parameters to all vapply() calls
  • Ensures type safety in vectorized operations
  • Follows R best practices for safe functional programming

Assessment

Quick patch release addressing R CMD check warnings about unsafe vapply usage.

starburst 0.3.3 (2026-02-13)

Zero Lints - Idiomatic R Code Quality

Goal: Achieve zero linting warnings while maintaining idiomatic R code style.

Changes

• Fixed trivial lint issues (#11)
  • Fixed infix spacing: collapse= → collapse =
  • Split long lines in R/ec2-pool.R and R/plan-starburst.R
  • 3 quick wins for cleaner code
• Configured lintr for R best practices
  • Explicit return() statements (clarity over implicit)
  • Descriptive variable names (clarity over brevity)
  • Suppressed false positives for internal functions
  • Accepted minor indentation variations (cosmetic only)

Quality Metrics

Lint Progression: - v0.3.0: 325 total lints - v0.3.1: 198 lints (-39%) - v0.3.2: 113 lints in R/ code (-65% from v0.3.1) - v0.3.3: 0 lints in R/ code ✅ (-100%)

Philosophy: This release establishes lintr configuration that prioritizes: 1. Code clarity over terseness 2. Explicit over implicit 3. Meaningful names over short names 4. R idioms over arbitrary style rules

Assessment

The package now has zero linting warnings while maintaining: - Explicit return statements (R best practice) - Descriptive variable names (self-documenting code) - Standard R indentation patterns - Internal function patterns recognized by R

Result: Clean, idiomatic R code with zero false-positive lint warnings.

starburst 0.3.2 (2026-02-13)

Idiomatic R Code - Go-Level Quality

Goal: Achieve Go-level code quality standards for R - clean, consistent, idiomatic.

Changes

• Removed unused variables (#10)
  • Cleaned up 5 truly unused assignments
  • Fixed cat_warning → cat_warn typo
  • Simplified code by removing unnecessary intermediate variables
• Code style improvements
  • Additional trailing whitespace cleanup
  • Improved code readability
  • More idiomatic R patterns

Quality Metrics

Lint Reduction Progress: - v0.3.0: 325 lints - v0.3.1: 198 lints (-39%) - v0.3.2: 195 lints (-40% total, -2% this release)

R/ Package Code Only (excluding examples/vignettes): - 113 lints (down from ~200+) - Breakdown: - 46 indentation (cosmetic, consistent style) - 34 object_usage (mostly false positives - internal functions) - 27 return (style preference - explicit vs implicit returns) - 3 object_length (descriptive variable names) - 2 line_length (complex expressions) - 1 infix_spaces (formatting)

Assessment

The remaining lints are: 1. Style preferences (indentation, returns) - subjective, not bugs 2. False positives (object_usage) - lintr doesn’t recognize internal functions 3. Descriptive names (object_length) - clarity over brevity

Code quality achieved: The package now meets high standards for production R code. Remaining lints are acceptable trade-offs for code clarity and maintainability.

Next Steps (Optional)

For absolute zero-lint perfection (0.3.3 if desired): - Manual indentation review (46 instances) - Add lintr suppressions for false positives - Shorten some variable names

starburst 0.3.1 (2026-02-12)

Code Quality Improvements

Complete: All 3 issues from v0.3.1 milestone (#18, #19, #20)

Changes

• Replaced all emojis with ASCII equivalents (#19)
  • ✓ → [OK] (success messages)
  • ⚠ → [WARNING] (warning messages)
  • 💡 → [TIP] (recommendations)
  • 📖 → [INFO] (documentation links)
  • 🚀 → [Starting] (initialization messages)
  • 🧹 → [Cleaning] (cleanup messages)
  • ✗ → [ERROR] (error messages)
  • 14 files updated, better compatibility with older systems
• Applied goodpractice suggestions (#20)
  • Replaced all sapply() with vapply() for type safety (10 instances in R/)
  • More predictable return types
  • Prevents unexpected list returns
  • Better error handling for edge cases
• Fixed lintr warnings (#18)
  • Removed 127 trailing whitespace instances
  • Down from 325 to 198 remaining lints (39% improvement)
  • Remaining lints are cosmetic (indentation, style preferences)

Impact

• No functional changes
• Better code readability
• Improved compatibility
• More robust type safety

Remaining Lints (198)

Acceptable cosmetic issues for future polish: - 85 indentation inconsistencies - 49 unused variable warnings - 29 return() style preferences - 15 seq_len() suggestions (in examples/vignettes) - 20 other minor style issues

starburst 0.3.0 (2026-02-12)

🎉 Production-Ready Release

staRburst is now enterprise-grade and ready for production deployment! This release focuses on security hardening, operational excellence, and comprehensive documentation.

Major Features

• Complete resource cleanup - session$cleanup() now fully implemented
  • Stops all running ECS tasks when session ends
  • Deletes S3 session files with force = TRUE option
  • Tracks ECS task ARNs in session manifest for reliable cleanup
  • Verification step ensures all resources are properly released
  • Prevents orphaned workers and runaway costs
• Detached session mode - Long-running jobs that persist after R session ends
  • Create sessions with starburst_session()
  • Submit tasks and disconnect: session$submit(expr)
  • Reattach later with starburst_session_attach(session_id)
  • Check progress anytime: session$status()
  • Workers stay running until absolute timeout (default 24h)
• Comprehensive troubleshooting guide - 15+ common issues documented
  • Accessing CloudWatch Logs (console and CLI)
  • Tasks stuck in pending (quota, IAM, network)
  • Permission errors (ECS, S3, ECR)
  • High costs and runaway workers
  • Package installation failures
  • Each issue includes symptoms, diagnosis, solutions, and prevention
• Security best practices guide - Enterprise security documentation
  • Credential management (IAM roles, profiles, STS)
  • S3 bucket security (encryption, versioning, policies)
  • Network isolation (VPCs, security groups, endpoints)
  • Cost controls and budget alerts
  • Audit logging (CloudTrail, CloudWatch)
  • Compliance considerations (HIPAA, GDPR)

Security Improvements

• Command injection prevention - Replaced unsafe system() calls
  • New safe_system() wrapper using processx::run()
  • Command whitelist validation
  • Automatic argument escaping (no shell expansion)
  • Prevents code execution via Docker/AWS CLI parameters
  • 25 new security regression tests
• Worker cost controls - Enforced maximum worker limits
  • Hard limit of 500 workers per cluster (prevents accidental runaway costs)
  • Validation at plan() time with helpful error messages
  • Clear guidance on requesting quota increases if needed
  • Estimated cost validation before worker launch
• Secure ECR authentication - Fixed credential exposure vulnerability
  • ECR password no longer exposed in process listings
  • Uses stdin for Docker login (not command line arguments)
  • Credentials never visible in ps aux output
  • Temporary files cleaned up immediately after use

Reliability Improvements

• Atomic S3 manifest updates - Prevents race conditions
  • ETag-based optimistic locking for concurrent updates
  • Automatic retry with exponential backoff on conflicts
  • Ensures no manifest updates are lost when multiple workers update simultaneously
  • Critical for detached sessions with many workers
• Comprehensive retry logic - Handles transient AWS failures gracefully
  • Exponential backoff with jitter for all AWS operations
  • Retries throttling, timeouts, 5xx errors automatically
  • Configurable retry limits (default 3 attempts)
  • Specialized retry wrappers: with_s3_retry(), with_ecs_retry(), with_ecr_retry()
  • Reduces job failures from temporary AWS service issues
• Improved error messages - Context, solutions, and documentation links
  • New starburst_error() helper for rich error messages
  • Every error includes relevant context (quota limits, resources, regions)
  • Actionable solutions provided (not just “something failed”)
  • Links to troubleshooting guide for detailed help
  • Specialized errors: quota_error(), permission_error(), task_failure_error()

New Functions & API

• starburst_session() - Create detached session for long-running jobs
• starburst_session_attach() - Reattach to existing session
• starburst_session_list() - List all active sessions
• session$submit() - Submit tasks to detached session
• session$status() - Check session progress and task states
• session$collect() - Retrieve completed results
• session$cleanup() - Stop workers and clean up resources

Infrastructure

• New R modules:
  • R/aws-retry.R - Centralized retry logic (167 lines)
  • R/errors.R - Rich error message helpers (286 lines)
  • R/session-api.R - Detached session API (600+ lines)
  • R/session-backend.R - Session backend initialization (332 lines)
  • R/session-state.R - S3 state management with atomic updates (487 lines)
• New vignettes:
  • vignettes/troubleshooting.Rmd - 15+ common issues (~15KB)
  • vignettes/security.Rmd - 10+ security topics (~17KB)
• Development infrastructure:
  • CLAUDE.md - Comprehensive AI assistant development guide
  • GitHub issues/milestones for project tracking
  • 30 standardized labels for issue classification

Testing

• 39 new tests - Comprehensive test coverage for production features
  • 25 security tests (command injection prevention, validation)
  • 14 cleanup tests (ECS task stopping, S3 deletion)
  • All tests passing (179 total tests now)
  • AWS integration tested against real infrastructure
• Package quality improvements
  • Documentation regenerated with no warnings
  • .Rbuildignore updated to exclude development files
  • Internal .Rd files properly namespaced
  • Top-level directory cleaned up (internal docs moved to docs/)

Bug Fixes

• Fixed: Worker error handling now catches and reports task failures (#2)
• Fixed: Manifest race condition causing concurrent update conflicts
• Fixed: S3 timeout errors now retried automatically
• Fixed: ECR password exposure in Docker login command
• Fixed: Missing cleanup implementation (was just printing message)
• Fixed: Documentation warnings for internal modules

Breaking Changes

• session$cleanup() signature changed: now accepts stop_workers and force parameters
• Default behavior: cleanup stops workers but preserves S3 files (use force=TRUE to delete)

Performance

• No performance regressions
• Retry logic adds minimal overhead (only on failures)
• Atomic updates have negligible latency impact (<50ms)

Documentation

• 2 new comprehensive vignettes (~32KB of documentation)
• All errors now link to troubleshooting guide
• Security guide covers 10+ enterprise security topics
• Examples added for all new API functions

Production Readiness

✅ Command injection prevention ✅ Worker cost controls (max 500) ✅ Complete resource cleanup ✅ Race condition prevention ✅ Transient failure handling ✅ Comprehensive documentation ✅ Professional error messages ✅ 179 passing tests

This release makes staRburst suitable for enterprise production deployments.

Known Issues (to be addressed in 0.3.1)

• Code style: 325 lintr warnings (mostly indentation, trailing whitespace)
  • Does not affect functionality
  • Will be cleaned up in 0.3.1
• Non-ASCII characters: Emojis in user-facing messages (✓, ⚠, 💡, etc.)
  • Modern R handles UTF-8 correctly
  • May cause warnings on older systems
  • Can be replaced with ASCII equivalents if needed
• Best practices: goodpractice suggests improvements
  • Replace sapply() with vapply() (30+ instances)
  • Replace 1:length() with seq_len() (14+ instances)
  • These are minor optimizations, not bugs

Static Analysis & Security

• Security scanning: Snyk enabled for dependency vulnerabilities
• Static analysis: lintr and goodpractice configured
• .lintr configuration: Ignores examples/, focuses on package code
• All critical security issues from audit resolved (command injection, credential exposure, etc.)

starburst 0.2.0 (2026-02-04)

Major Features

• Multi-stage base image system for dramatically faster builds
  • Base images contain system dependencies + core R packages
  • Project images only install project-specific packages
  • Reduces typical build times from 20 min to 3-5 min
  • One-time base build per R version, reused across all projects
• Complete Docker dependency support
  • Added 15 system packages for comprehensive R package compilation
  • Supports graphics packages (ragg, systemfonts, textshaping)
  • Supports data packages (httpuv, readr, haven)
  • All common CRAN packages now compile successfully
• Fixed globals serialization (#1)
  • Proper function closure capture for remote execution
  • Converts plain lists to globals::Globals objects
  • Ensures variables are correctly serialized to workers

Performance Improvements

• ECR image caching validated with 40x speedup
  • First run: ~42 min (one-time Docker build)
  • Subsequent runs: ~1 min (cached image from ECR)
  • No rebuild needed when renv.lock unchanged
• Build time optimizations
  • Dev environment (112 packages): 20 min → 6-8 min
  • Production (30 packages): 8-10 min → 3-5 min
  • Minimal projects: 3-5 min → 1-2 min

New Functions

• build_base_image() - Build base Docker image with common dependencies
• ensure_base_image() - Check for/create base image as needed
• get_base_image_uri() - Get ECR URI for base image

Bug Fixes

• Fixed globals serialization causing empty results from workers
• Added missing system dependencies for package compilation
• Resolved Docker build failures for graphics packages

Infrastructure

• New inst/templates/Dockerfile.base for base image builds
• Simplified inst/templates/Dockerfile.template (42 → 19 lines)
• Base images tagged by R version: base-{R.VERSION}

Known Limitations

• No GPU support (planned for v1.0)
• No Spot instance support (planned for v1.0)
• Limited to Fargate resources (16 vCPU, 120GB RAM max)
• Public base images not yet available (coming in 0.3.0)

starburst 0.1.0 (2026-02-03)

Initial Release

• Initial development version
• Core features:
  • future backend for AWS Fargate
  • Automatic environment synchronization with renv
  • Wave-based quota management
  • Cost estimation and tracking
  • One-time setup wizard
  • Transparent quota handling with automatic increase requests