These were invisible to the test suite, which mocks AWS, and to earlier live
runs, which used starburst_session(). Sessions were not affected.
plan(starburst) futures work again. Since 0.3.0 the worker uploaded the
result of every plan(starburst) future under an empty S3 key, so each
future failed with SignatureDoesNotMatch (HTTP 403). The worker now uses
the task ID it was launched with (#63). Environment images are rebuilt
automatically: their tag now also covers the worker script.starburst_map() and starburst_cluster() work again. They failed before
launching any workers with "environment did not shut itself down properly",
because they registered a pre-built backend with future::plan(). They now
submit to their own backend without changing your active plan, so
cluster$shutdown() no longer resets the plan to sequential (#64). Past
that error they had further EC2 failures, now fixed:
assignPublicIp, which EC2 rejects ("Assign public IP
is not supported for this launch type"), and ignored the configured
subnets and security groups;cpu = 4, memory = "8GB" was not resized to the instance (as
plan(starburst) and sessions already did), so tasks on e.g. c7g.xlarge
could never be placed and stayed PROVISIONING forever;future.apply/furrr no longer hang after a worker error. When one
element failed they cancel the remaining futures, and a canceled starburst
future never counted as resolved, so collecting results waited forever.future "INTERNAL ERROR: Unexpected structure for condition
set element"; value() and starburst_map() now raise the worker's own
error message (#65).plan(starburst) backend now always submits to its own cluster, even after
a starburst_map() or starburst_cluster() call in the same session.plan(starburst), future.apply, starburst_map() and
starburst_cluster().tools/live-smoke.R (also the live-smoke suite of the AWS integration
workflow) runs every entry point on real EC2; it is now a release
requirement.qs2. When no public
base image matches your exact R version (only R 4.5.3 and 4.6.1 are published),
staRburst builds one from inst/templates/Dockerfile.base. Since RcppParallel
6.2.0 (2026-07-30) requires cmake at configure time, that build failed to
install qs2 — but install.packages() only warns, so the image built
"successfully" and workers later failed to deserialize tasks. The template now
installs cmake and stops the build if any worker package fails to install
(#55, #56).qs2 is installed but cannot be
loaded (e.g. a stale binary built against an older RcppParallel), instead of
failing with misleading name-mismatch errors.max_hourly_cost guard now works on the EC2 backend (the default).
Previously estimate_cost() returned per_hour for Fargate but total_per_hour
for EC2, and the guard read per_hour — which is NULL on EC2 — so the cost
ceiling was silently never enforced on the default backend. estimate_cost() now
returns a normalized hourly_rate for every backend, and the guard, cost display,
and starburst_map()/starburst_cluster() all use it.cost_alert_threshold is now enforced — it warns when a plan's estimated
hourly rate exceeds it (previously it was stored but never read).starburst_config(auto_cleanup_s3 = …) now actually controls S3 cleanup. The
runtime read a different key (getOption("starburst.cleanup_s3")) and ignored the
stored setting; cleanup now resolves option → auto_cleanup_s3 config → default TRUE.starburst_map()/starburst_cluster() cost estimates match the backend used —
they forward launch_type/instance_type/use_spot, so an EC2 job no longer
reports a Fargate price.starburst_setup() now provisions the default EC2 capacity provider (for
c7g.xlarge) so the default EC2 backend works out of the box — previously the
first starburst_map()/plan(starburst) run failed because no capacity provider
existed. It is created at DesiredCapacity = 0 (no billable instances launch
during setup). Pass setup_ec2 = FALSE to skip (Fargate-only users). (#37)instance_type (e.g. c8a.xlarge) previously failed with
an opaque "AutoScalingGroup not found" because only the default was provisioned.
Now the capacity provider/ASG is created on demand at first launch (a one-time
~1–2 min step; no instances launched by provisioning), or on failure you get the
exact starburst_setup_ec2(instance_types = "…") command to run.session$collect() now returns an entry for every terminal task,
including failures. A failed task comes back as a structured failure
(list(error = TRUE, message = …, task_id = …)) alongside successful results,
instead of being silently omitted — matching the documented contract.session$cleanup(): by default it stops workers and marks the session
terminated but preserves S3 objects; pass force = TRUE to also delete them.
(Behavior unchanged; the docs were wrong and are now correct.)platform argument from starburst_map() and
starburst_cluster(). It was ignored (architecture is inferred from
instance_type: Graviton *g.* → ARM64, else x86_64), and its "X86_64"
default even contradicted the ARM64 c7g.xlarge default. Drop it from calls; the
backend picks the architecture from instance_type. (platform remains on
starburst_estimate(), where it is functional.)starburst_config(max_cost_per_job=) to max_hourly_cost. The limit
was always enforced as an hourly rate (USD/hour), not a total-job cap; the new
name matches the behavior. Update any starburst_config(max_cost_per_job = ...)
calls to max_hourly_cost = ....starburst_map() and starburst_cluster() now accept launch_type,
instance_type, and use_spot and forward them to the backend. Previously
these functions had no backend arguments, so a call like
starburst_map(x, f, launch_type = "FARGATE") silently passed launch_type
to the mapped function .f (typically an "unused argument" error) instead of
switching the backend — contradicting the 0.3.7 migration note. Backend
selection now works uniformly across plan(starburst), starburst_map(),
starburst_cluster(), and starburst_session().detached = TRUE argument; S3-read examples used base read.csv()/
readRDS(url()) which cannot read s3://; and the API rate-limit example
aggregated to 10× its stated global limit across workers.use_public_base = TRUE failed the multi-arch
environment build with "no match for platform in manifest" for arm64. The
build-base-images workflow now builds both architectures (matching the env
build), so the public base path works. (#39)starburst_map(); a grouped reference
index and an architecture diagram were added; and starburst_config() keys are
now catalogued.future_map(1:10000, …) and
hand-written speedup claims) now link to the canonical, measured example articles
and the Workload Shapes / Performance guides. The doc-consistency guard
(tools/check-docs.R) now also flags naive future_map()/future_lapply()
big-count calls, not just starburst_map().ROADMAP.md (a pre-1.0, Fargate-first implementation plan
that also carried a stale "MIT" license note); the README Roadmap section is the
single source of truth.timeout parameter from the paws client
configuration that could cause workers to fail on startup.ensure_environment() now walks up the directory tree to
find the package-root renv.lock when invoked from a subdirectory (e.g. during
tests), and the environment image hash is computed consistently between build
and lookup, so cached worker images are reused correctly.safe_system() usage and resolved qs2 serialization edge cases
surfaced by integration tests.\donttest{} examples are guarded with starburst_is_configured() so they
are safe to run without AWS credentials.inst/doc/ and assorted R CMD check / CRAN
incoming-feasibility fixes. Package accepted on CRAN.launch_type changed from FARGATE to EC2,
with spot instances enabled by default (use_spot = TRUE, instance_type = "c7g.xlarge"). EC2 avoids Fargate cold starts, is 50–90% cheaper with spot, and
supports warm pools and a wider range of instance types. Fargate remains fully
supported — pass launch_type = "FARGATE" to plan(starburst, ...),
starburst_map(), starburst_cluster(), or starburst_session() to keep the
previous behavior.launch_type = "FARGATE" to your
plan()/starburst_*() call. No other changes are required; the task API is
unchanged across backends.Major milestone: Complete AWS integration testing infrastructure and documentation site launch.
Comprehensive AWS Integration Testing (#4b36310)
run-aws-tests.sh)Documentation Site (https://starburst.ing)
Test Suite
CI/CD
Full documentation now available at starburst.ing
Docker Image Versioning (#707ee78)
Serialization Update (#cbfad21)
qs to qs2 packageMinor version bump with critical bug fixes for Docker caching and serialization consistency.
FUN.VALUE parameters to all vapply() callsQuick patch release addressing R CMD check warnings about unsafe vapply usage.
Goal: Achieve zero linting warnings while maintaining idiomatic R code style.
Fixed trivial lint issues (#11)
collapse= → collapse =Configured lintr for R best practices
return() statements (clarity over implicit)Lint Progression:
Philosophy: This release establishes lintr configuration that prioritizes:
The package now has zero linting warnings while maintaining:
Result: Clean, idiomatic R code with zero false-positive lint warnings.
Goal: Achieve Go-level code quality standards for R - clean, consistent, idiomatic.
Removed unused variables (#10)
cat_warning → cat_warn typoCode style improvements
Lint Reduction Progress:
R/ Package Code Only (excluding examples/vignettes):
The remaining lints are:
Code quality achieved: The package now meets high standards for production R code. Remaining lints are acceptable trade-offs for code clarity and maintainability.
For absolute zero-lint perfection (0.3.3 if desired):
Complete: All 3 issues from v0.3.1 milestone (#18, #19, #20)
Replaced all emojis with ASCII equivalents (#19)
Applied goodpractice suggestions (#20)
sapply() with vapply() for type safety (10 instances in R/)Fixed lintr warnings (#18)
Acceptable cosmetic issues for future polish:
staRburst is now enterprise-grade and ready for production deployment! This release focuses on security hardening, operational excellence, and comprehensive documentation.
Complete resource cleanup - session$cleanup() now fully implemented
force = TRUE optionDetached session mode - Long-running jobs that persist after R session ends
starburst_session()session$submit(expr)starburst_session_attach(session_id)session$status()Comprehensive troubleshooting guide - 15+ common issues documented
Security best practices guide - Enterprise security documentation
Command injection prevention - Replaced unsafe system() calls
safe_system() wrapper using processx::run()Worker cost controls - Enforced maximum worker limits
plan() time with helpful error messagesSecure ECR authentication - Fixed credential exposure vulnerability
ps aux outputAtomic S3 manifest updates - Prevents race conditions
Comprehensive retry logic - Handles transient AWS failures gracefully
with_s3_retry(), with_ecs_retry(), with_ecr_retry()Improved error messages - Context, solutions, and documentation links
starburst_error() helper for rich error messagesquota_error(), permission_error(), task_failure_error()starburst_session() - Create detached session for long-running jobsstarburst_session_attach() - Reattach to existing sessionstarburst_session_list() - List all active sessionssession$submit() - Submit tasks to detached sessionsession$status() - Check session progress and task statessession$collect() - Retrieve completed resultssession$cleanup() - Stop workers and clean up resourcesNew R modules:
R/aws-retry.R - Centralized retry logic (167 lines)R/errors.R - Rich error message helpers (286 lines)R/session-api.R - Detached session API (600+ lines)R/session-backend.R - Session backend initialization (332 lines)R/session-state.R - S3 state management with atomic updates (487 lines)New vignettes:
vignettes/troubleshooting.Rmd - 15+ common issues (~15KB)vignettes/security.Rmd - 10+ security topics (~17KB)Development infrastructure:
CLAUDE.md - Comprehensive AI assistant development guide39 new tests - Comprehensive test coverage for production features
Package quality improvements
.Rbuildignore updated to exclude development files.Rd files properly namespaceddocs/)session$cleanup() signature changed: now accepts stop_workers and force parametersforce=TRUE to delete)✅ Command injection prevention ✅ Worker cost controls (max 500) ✅ Complete resource cleanup ✅ Race condition prevention ✅ Transient failure handling ✅ Comprehensive documentation ✅ Professional error messages ✅ 179 passing tests
This release makes staRburst suitable for enterprise production deployments.
Code style: 325 lintr warnings (mostly indentation, trailing whitespace)
Non-ASCII characters: Emojis in user-facing messages (✓, ⚠, 💡, etc.)
Best practices: goodpractice suggests improvements
sapply() with vapply() (30+ instances)1:length() with seq_len() (14+ instances)Multi-stage base image system for dramatically faster builds
Complete Docker dependency support
Fixed globals serialization (#1)
globals::Globals objectsECR image caching validated with 40x speedup
Build time optimizations
build_base_image() - Build base Docker image with common dependenciesensure_base_image() - Check for/create base image as neededget_base_image_uri() - Get ECR URI for base imageinst/templates/Dockerfile.base for base image buildsinst/templates/Dockerfile.template (42 → 19 lines)base-{R.VERSION}