max_hourly_cost guard now works on the EC2 backend (the default).
Previously estimate_cost() returned per_hour for Fargate but total_per_hour
for EC2, and the guard read per_hour — which is NULL on EC2 — so the cost
ceiling was silently never enforced on the default backend. estimate_cost() now
returns a normalized hourly_rate for every backend, and the guard, cost display,
and starburst_map()/starburst_cluster() all use it.cost_alert_threshold is now enforced — it warns when a plan's estimated
hourly rate exceeds it (previously it was stored but never read).starburst_config(auto_cleanup_s3 = …) now actually controls S3 cleanup. The
runtime read a different key (getOption("starburst.cleanup_s3")) and ignored the
stored setting; cleanup now resolves option → auto_cleanup_s3 config → default TRUE.starburst_map()/starburst_cluster() cost estimates match the backend used —
they forward launch_type/instance_type/use_spot, so an EC2 job no longer
reports a Fargate price.starburst_setup() now provisions the default EC2 capacity provider (for
c7g.xlarge) so the default EC2 backend works out of the box — previously the
first starburst_map()/plan(starburst) run failed because no capacity provider
existed. It is created at DesiredCapacity = 0 (no billable instances launch
during setup). Pass setup_ec2 = FALSE to skip (Fargate-only users). (#37)instance_type (e.g. c8a.xlarge) previously failed with
an opaque "AutoScalingGroup not found" because only the default was provisioned.
Now the capacity provider/ASG is created on demand at first launch (a one-time
~1–2 min step; no instances launched by provisioning), or on failure you get the
exact starburst_setup_ec2(instance_types = "…") command to run.session$collect() now returns an entry for every terminal task,
including failures. A failed task comes back as a structured failure
(list(error = TRUE, message = …, task_id = …)) alongside successful results,
instead of being silently omitted — matching the documented contract.session$cleanup(): by default it stops workers and marks the session
terminated but preserves S3 objects; pass force = TRUE to also delete them.
(Behavior unchanged; the docs were wrong and are now correct.)platform argument from starburst_map() and
starburst_cluster(). It was ignored (architecture is inferred from
instance_type: Graviton *g.* → ARM64, else x86_64), and its "X86_64"
default even contradicted the ARM64 c7g.xlarge default. Drop it from calls; the
backend picks the architecture from instance_type. (platform remains on
starburst_estimate(), where it is functional.)starburst_config(max_cost_per_job=) to max_hourly_cost. The limit
was always enforced as an hourly rate (USD/hour), not a total-job cap; the new
name matches the behavior. Update any starburst_config(max_cost_per_job = ...)
calls to max_hourly_cost = ....starburst_map() and starburst_cluster() now accept launch_type,
instance_type, and use_spot and forward them to the backend. Previously
these functions had no backend arguments, so a call like
starburst_map(x, f, launch_type = "FARGATE") silently passed launch_type
to the mapped function .f (typically an "unused argument" error) instead of
switching the backend — contradicting the 0.3.7 migration note. Backend
selection now works uniformly across plan(starburst), starburst_map(),
starburst_cluster(), and starburst_session().detached = TRUE argument; S3-read examples used base read.csv()/
readRDS(url()) which cannot read s3://; and the API rate-limit example
aggregated to 10× its stated global limit across workers.use_public_base = TRUE failed the multi-arch
environment build with "no match for platform in manifest" for arm64. The
build-base-images workflow now builds both architectures (matching the env
build), so the public base path works. (#39)starburst_map(); a grouped reference
index and an architecture diagram were added; and starburst_config() keys are
now catalogued.future_map(1:10000, …) and
hand-written speedup claims) now link to the canonical, measured example articles
and the Workload Shapes / Performance guides. The doc-consistency guard
(tools/check-docs.R) now also flags naive future_map()/future_lapply()
big-count calls, not just starburst_map().ROADMAP.md (a pre-1.0, Fargate-first implementation plan
that also carried a stale "MIT" license note); the README Roadmap section is the
single source of truth.timeout parameter from the paws client
configuration that could cause workers to fail on startup.ensure_environment() now walks up the directory tree to
find the package-root renv.lock when invoked from a subdirectory (e.g. during
tests), and the environment image hash is computed consistently between build
and lookup, so cached worker images are reused correctly.safe_system() usage and resolved qs2 serialization edge cases
surfaced by integration tests.\donttest{} examples are guarded with starburst_is_configured() so they
are safe to run without AWS credentials.inst/doc/ and assorted R CMD check / CRAN
incoming-feasibility fixes. Package accepted on CRAN.launch_type changed from FARGATE to EC2,
with spot instances enabled by default (use_spot = TRUE, instance_type = "c7g.xlarge"). EC2 avoids Fargate cold starts, is 50–90% cheaper with spot, and
supports warm pools and a wider range of instance types. Fargate remains fully
supported — pass launch_type = "FARGATE" to plan(starburst, ...),
starburst_map(), starburst_cluster(), or starburst_session() to keep the
previous behavior.launch_type = "FARGATE" to your
plan()/starburst_*() call. No other changes are required; the task API is
unchanged across backends.Major milestone: Complete AWS integration testing infrastructure and documentation site launch.
Comprehensive AWS Integration Testing (#4b36310)
run-aws-tests.sh)Documentation Site (https://starburst.ing)
Test Suite
CI/CD
Full documentation now available at starburst.ing
Docker Image Versioning (#707ee78)
Serialization Update (#cbfad21)
qs to qs2 packageMinor version bump with critical bug fixes for Docker caching and serialization consistency.
FUN.VALUE parameters to all vapply() callsQuick patch release addressing R CMD check warnings about unsafe vapply usage.
Goal: Achieve zero linting warnings while maintaining idiomatic R code style.
Fixed trivial lint issues (#11)
collapse= → collapse =Configured lintr for R best practices
return() statements (clarity over implicit)Lint Progression:
Philosophy: This release establishes lintr configuration that prioritizes:
The package now has zero linting warnings while maintaining:
Result: Clean, idiomatic R code with zero false-positive lint warnings.
Goal: Achieve Go-level code quality standards for R - clean, consistent, idiomatic.
Removed unused variables (#10)
cat_warning → cat_warn typoCode style improvements
Lint Reduction Progress:
R/ Package Code Only (excluding examples/vignettes):
The remaining lints are:
Code quality achieved: The package now meets high standards for production R code. Remaining lints are acceptable trade-offs for code clarity and maintainability.
For absolute zero-lint perfection (0.3.3 if desired):
Complete: All 3 issues from v0.3.1 milestone (#18, #19, #20)
Replaced all emojis with ASCII equivalents (#19)
Applied goodpractice suggestions (#20)
sapply() with vapply() for type safety (10 instances in R/)Fixed lintr warnings (#18)
Acceptable cosmetic issues for future polish:
staRburst is now enterprise-grade and ready for production deployment! This release focuses on security hardening, operational excellence, and comprehensive documentation.
Complete resource cleanup - session$cleanup() now fully implemented
force = TRUE optionDetached session mode - Long-running jobs that persist after R session ends
starburst_session()session$submit(expr)starburst_session_attach(session_id)session$status()Comprehensive troubleshooting guide - 15+ common issues documented
Security best practices guide - Enterprise security documentation
Command injection prevention - Replaced unsafe system() calls
safe_system() wrapper using processx::run()Worker cost controls - Enforced maximum worker limits
plan() time with helpful error messagesSecure ECR authentication - Fixed credential exposure vulnerability
ps aux outputAtomic S3 manifest updates - Prevents race conditions
Comprehensive retry logic - Handles transient AWS failures gracefully
with_s3_retry(), with_ecs_retry(), with_ecr_retry()Improved error messages - Context, solutions, and documentation links
starburst_error() helper for rich error messagesquota_error(), permission_error(), task_failure_error()starburst_session() - Create detached session for long-running jobsstarburst_session_attach() - Reattach to existing sessionstarburst_session_list() - List all active sessionssession$submit() - Submit tasks to detached sessionsession$status() - Check session progress and task statessession$collect() - Retrieve completed resultssession$cleanup() - Stop workers and clean up resourcesNew R modules:
R/aws-retry.R - Centralized retry logic (167 lines)R/errors.R - Rich error message helpers (286 lines)R/session-api.R - Detached session API (600+ lines)R/session-backend.R - Session backend initialization (332 lines)R/session-state.R - S3 state management with atomic updates (487 lines)New vignettes:
vignettes/troubleshooting.Rmd - 15+ common issues (~15KB)vignettes/security.Rmd - 10+ security topics (~17KB)Development infrastructure:
CLAUDE.md - Comprehensive AI assistant development guide39 new tests - Comprehensive test coverage for production features
Package quality improvements
.Rbuildignore updated to exclude development files.Rd files properly namespaceddocs/)session$cleanup() signature changed: now accepts stop_workers and force parametersforce=TRUE to delete)✅ Command injection prevention ✅ Worker cost controls (max 500) ✅ Complete resource cleanup ✅ Race condition prevention ✅ Transient failure handling ✅ Comprehensive documentation ✅ Professional error messages ✅ 179 passing tests
This release makes staRburst suitable for enterprise production deployments.
Code style: 325 lintr warnings (mostly indentation, trailing whitespace)
Non-ASCII characters: Emojis in user-facing messages (✓, ⚠, 💡, etc.)
Best practices: goodpractice suggests improvements
sapply() with vapply() (30+ instances)1:length() with seq_len() (14+ instances)Multi-stage base image system for dramatically faster builds
Complete Docker dependency support
Fixed globals serialization (#1)
globals::Globals objectsECR image caching validated with 40x speedup
Build time optimizations
build_base_image() - Build base Docker image with common dependenciesensure_base_image() - Check for/create base image as neededget_base_image_uri() - Get ECR URI for base imageinst/templates/Dockerfile.base for base image buildsinst/templates/Dockerfile.template (42 → 19 lines)base-{R.VERSION}