v1.5.6

Sling v1.5.6 (2026-02-02T10:19:20Z)

• Enhanced delete_missing with configurable source and target WHERE clauses
• Added secret encoding support for HMAC authentication in API specs
• Added EscapeBackslash option for MySQL LOAD DATA operations
• Added row-start-only range format support for Excel

Bug Fixes

• Fixed Fabric ROWTERMINATOR to use hexadecimal representation
• Disabled AWS EC2 metadata retrieval for improved security
• Restored escape character handling in MySQL/MariaDB LOAD DATA INFILE templates

v1.5.5

Sling v1.5.5 (2026-01-26T16:09:41Z)

New Features

• DB2 Database Support - Initial support for IBM DB2 with connection templates and full test suite
• definition-only Mode - Create table/file definitions without transferring data (parquet/arrow only for files)
• _sling_synced_at Column - Enable with SLING_SYNCED_AT_COLUMN=true for timestamp tracking
• _sling_synced_op Column - Tracks operation type: I (insert), U (update), D (soft delete)
• slugify Function - Convert strings to URL-friendly slugs

Bug Fixes

• DDL primary key with WITH clauses - Fixed incorrect placement when table options present (#694)
• Prometheus streaming deadlock - Fixed stall on large time ranges (#700)
• MySQL boolean CSV streaming - Fixed boolean handling in LOAD DATA LOCAL INFILE
• delete_missing with transforms - Fixed errors in PK-only queries when transforms present
• Connection caching - Fixed hash method and ODBC initialization

Improvements

• Go 1.25 - Upgraded build toolchain
• Docker ODBC packages - Added unixodbc and odbcinst
• Oracle XMLTYPE - Support for BigQuery transfers
• MySQL LOAD DATA LOCAL - Enhanced NULL value handling
• S3 multi-bucket access - Improved support in replications
• Schema discovery - Thread-safe schemata merging, column-level support

v1.5.4

Sling v1.5.4 (2026-01-05T20:59:07Z)

ODBC Connection Support

• New ODBCConn type for generic ODBC database connections

Environment Variable Expansion in Env Files

• Environment variables can now be used within ~/.sling/env.yaml files
• Variables are expanded before YAML unmarshalling
• Loaded env vars are set into the OS environment if not already present

Enhanced Boolean Type Handling

• Boolean columns now cast to SmallIntType instead of IntegerType for better compatibility
• Improved boolean handling for MySQL, MariaDB, and StarRocks databases

Pipeline Step Order Preservation

• Updated YAML handling to use v3
• Pipeline steps now maintain their defined order during execution

Bug Fixes

• ADBC Safety Check: Ensure ADBC connection is initialized before use (conn.adbc != nil check)
• Google API Auth: Fixed "multiple credential options provided" error with google.golang.org/api >= v0.258.0
• Windows Build Dynamic Linking issue

v1.5.3

Sling v1.5.3 (2025-12-23T14:49:39Z)

Fix for boolean casting logic for v1.5.2

• Added CastBoolColumnForSelect method to cast boolean columns when target expects integer or string
  • Boolean → Integer: true→1, false→0
  • Boolean → String: true→"true", false→"false"
• Enhanced CastColumnsForSelect to detect boolean/integer or boolean/string mismatches and apply casting

API Incremental Sync Improvements

• New GetSyncUpdateKey function - Automatically detects sync key and update key from API endpoint configuration when:
  • Endpoint has a sync array defined
  • A processor uses aggregation: maximum that outputs to state.<sync_key>
  • update_key is defined in response records
• DB-based incremental fallback - When SLING_STATE is not set, API replications can now fetch the incremental value directly from the target database

v1.5.2

Sling v1.5.2 (2025-12-20T13:40:11Z)

GeoJSON Export Support

• New file format for exporting spatial data to GeoJSON
• Supports geometry column detection with configurable column name (defaults to geometry)
• Added go-geom library dependency

ADBC (Arrow Database Connectivity) Support

• New connection type ArrowDBC for high-performance database connectivity via Apache Arrow
• Supports PostgreSQL and SQL Server via ADBC drivers
• Includes bulk import/export methods leveraging Arrow format
• New tests: r.77.adbc_postgres.yaml, r.78.adbc_sqlserver.yaml, r.79.adbc_write.yaml

OAuth2 Enhancements

• Improved OAuth2 initialization and callback handling
• Added Gmail API connector with OAuth support
• Enhanced encryption for OAuth tokens in env files
• Added prompt=consent parameter to force refresh token retrieval

Runtime State Management

• Enhanced combined state file handling for replications
• File persistence for runtime state with exec ID tracking
• Improved thread state synchronization

Hook Improvements

• Added OnFailRetry option for automatic retry on hook failure
• Better handling of replication hooks prefixing

New Functions

• object_zip: Creates an object from arrays of keys and values

Infrastructure & Fixes

• Docker: Updated base image from Ubuntu Focal to Jammy
• Database Templates: Unified quoting/unquoting methods across connections; added YAML-based type mappings (deprecating TSV)
• User Template Overrides: Support for ~/.sling/templates/ custom templates
• API Spec: Added PrimaryKey field to stream configuration
• Connection Handling: Thread-safe access to local connections; improved MSSQL boolean handling
• Pipeline: Fixed EndTime for skipped steps; improved zombie job cleanup logging

v1.5.1

Sling v1.5.1 (2025-12-07T10:54:11Z)

New Features

Directory-Based Runs

• Run all replications/pipelines in a directory: You can now pass a directory path to sling run, and it will automatically discover
  and execute all valid replication and pipeline YAML files within it
• Files are executed in order with clear visual separators showing progress
• Summary shows successes/failures after completion

API Specification Enhancements

• Modifier syntax for response rules/processors: New +rules, rules+, +processors, processors+ syntax allows prepending or appending
  rules and processors to API spec endpoints
• Setup/Teardown modifiers: Similar +setup, setup+, +teardown, teardown+ syntax for endpoint setup and teardown configurations
• JSON object/array detection in template variables: Template variables in API specs now properly detect and handle JSON objects and
  arrays

New Transform Functions

• regex_replace(string, pattern, replacement): Replace regex pattern matches in strings, supports backreferences (e.g., $1, $2)

Database Improvements

• ClickHouse keep-alive mechanism: Prevents idle connection timeouts during long bulk imports by periodically sending heartbeat
  queries
• DuckDB updated to v1.4.2: Latest DuckDB version with improvements and fixes
• DuckDB binary pre-download for threaded replications: Avoids race conditions when multiple threads need DuckDB simultaneously

Snowflake Enhancements

• MFA token caching: Improved support for Snowflake connections with MFA

OpenTelemetry Integration

• OTEL logging support: New OpenTelemetry dependencies for enhanced logging and observability
• Configure via SLING_OTEL_ENDPOINT environment variable

Pipeline Improvements

• Skip steps on previous error: Pipeline steps are now automatically skipped if a previous step errored
• Parameter rendering in hooks: Dynamic value handling in hook parameters

Bug Fixes

• Fixed table creation check: CREATE TABLE statements now properly check for existence before creation
• Removed deprecated --config CLI flag: Cleanup of legacy options
• Improved progress message formatting: Cleaner task execution output

v1.5.0

Sling v1.5.0 (2025-11-30T16:59:21Z)

Key Features

API Specification Enhancements

• Endpoint-specific authentication: Refactored API authentication to allow individual endpoints to define and manage their own authentication, including endpoint-level auth expiry and re-authentication
• Dynamic endpoint rendering: Added Dynamic field to Endpoint for generated state indication
• New rule types: Added RuleTypeSkip to skip records and RuleTypeBreak to stop processing requests for current iteration
• Enhanced state management: Implemented contextMap for improved state management across endpoints
• Renamed AggregationTypeFlatten to AggregationTypeCollect for clearer semantics

Database & Connection Improvements

• DuckDB 1.4.1: Updated to latest DuckDB version
• Native pgx connection for Cloud SQL: Implemented native pgx connection for Cloud SQL COPY operations
• ClickHouse boolean handling: Fixed boolean-to-string conversion in CastColumnForSelect function
• DuckLake SQLite catalog: Automatically create parent directory for SQLite catalog if needed
• GCP credentials: Added base64 decoding for GCP credentials in BigQuery, MySQL, PostgreSQL, and Redshift connections

Replication & Pipeline Enhancements

• Wildcard hooks: Added support for wildcard hooks in replication configuration (e.g., pre.*, post.*)
• Dynamic SQL rendering: Enhanced evaluator integration for dynamic SQL rendering with JMESPath compatibility
• Incremental mode improvements: Added hasRange method and improved incremental mode warnings
• Periodic state updates: Implemented periodic state updates during replication execution
• Run state tracking: Added optional fields for incremental value and range in RunState

Environment & Configuration

• EnvFile refactoring: Refactored Variables to Env in EnvFile structure for improved clarity
• Legacy support: Improved handling for empty Env and Variables maps with backward compatibility
• LocalConnections: Initialize and populate LocalConnections map with connection data

Error Handling & Debugging

• Enhanced error messages: Improved error messages for CSV input errors, URL parsing failures, and transaction commit issues
• S3 region guidance: Added guidance for S3 region specification in DuckLake error messages
• Goroutine stack dumps: Dump goroutine stack on critical events (Ctrl+C, timeout) for debugging

SSH Enhancements

• NewSession method: Added NewSession method to create SSH sessions with proper error handling
• Comprehensive SSH tests: Added comprehensive SSH command tests with various scenarios

Concurrency Improvements

• Concurrent maps: Replaced standard maps with concurrent-map for LocalConnections and transforms to prevent race conditions
• Synchronized stopIters: Added synchronization during endpoint setup

Bug Fixes

• Fixed debug logging for HTTP stream initialization in DuckDB
• Fixed overwriting existing state in renderEndpointTemplate
• Fixed nil Pipeline handling in Context method of PipelineStepExecution
• Fixed replication step configuration and environment variable handling
• Fixed timeout duration to use seconds instead of milliseconds
• Removed redundant check for disabled streams during replication preparation
• Fixed missing filepath import in database_ducklake.go
• Removed unused duckURI variable in filesystem operations

Test Infrastructure

• Added comprehensive routine tests for logging, validation, file operations, and HTTP fetch
• Added test orchestration script for CLI and database testing
• Added dependency management for test cases with after field
• Added group classification for test organization
• Added MSSQL to Postgres decimal precision and cast_as tests
• Added nested JSON to SQLite test scripts
• Added test for wildcard streams with disabled table exclusion
• Added ClickHouse boolean-to-string conversion test
• Added test for {fields} placeholder with select parameter
• Added HTTP POST hooks test suite for JSON payloads

v1.4.26

Sling v1.4.26 (2025-11-06T13:15:03Z)

See https://github.com/slingdata-io/sling-cli/ for more details.

v1.4.25

Sling v1.4.25 (2025-11-06T11:06:26Z)

See https://github.com/slingdata-io/sling-cli/ for more details.

v1.4.24

Sling v1.4.24 (2025-10-17T16:09:36Z)

See https://github.com/slingdata-io/sling-cli/ for more details.