v0.21.5

Re-release of v0.21.3 after fixing publishing config

v0.21.4

Republish of v0.21.3 (ZIO 1 compatible version)

v0.21.3

• Compile java source files to Java 8 version @mmalik23 (#798)

v0.30.1

Dependency upgrades. Please note that there are some compatibility issues with ZIO 2.0.3, we're still investigating.

Changes

• Deprecate Producer.sinkChunked and introduce sink @svroonland (#791)
• Update scala-library to 2.13.10 @scala-steward (#769)
• Update zio-aws-cloudwatch, zio-aws-core, ... to 5.17.295.6 @scala-steward (#787)
• Update sbt-protobuf to 0.7.2 @scala-steward (#777)
• Update logback-classic to 1.4.4 @scala-steward (#765)
• Update sbt-ci-release-early to 2.0.37 @scala-steward (#781)
• Update scala3-library to 3.2.1 @scala-steward (#782)
• Update scalafmt-core to 3.6.1 @scala-steward (#784)
• Update scala-library to 2.12.17 @scala-steward (#768)
• Update sbt-pgp to 2.2.0 @scala-steward (#776)
• Update sbt-scalafmt to 2.5.0 @scala-steward (#790)
• Update sbt-assembly to 2.0.0 @scala-steward (#771)
• Update sbt to 1.8.0 @scala-steward (#789)
• Update zio-logging, zio-logging-slf4j to 2.1.4 @scala-steward (#788)
• Update sbt-sonatype to 3.9.14 @scala-steward (#785)
• Update sbt-tpolecat to 0.4.1 @scala-steward (#764)

v0.30.0

This release is an upgrade to ZIO 2.0. Everything should be fully usable, but be aware of #761.

Changes

• Misc Upgrades @svroonland (#760)
• Upgrade to ZIO 2 @svroonland (#623)
• Update sbt-tpolecat to 0.3.1 @scala-steward (#733)
• Update scala3-library to 3.1.2 @scala-steward (#726)
• Update sbt-tpolecat to 0.2.3 @scala-steward (#728)
• Update scalafmt-core to 3.5.2 @scala-steward (#729)
• Update zio-interop-reactivestreams to 1.3.10 @scala-steward (#730)
• Update sbt-ci-release-early to 2.0.29 @scala-steward (#731)
• Producer speedup: Parallel shard prediction (MD5 hashing) @svroonland (#732)
• Update zio, zio-streams, zio-test, ... to 1.0.14 @scala-steward (#727)
• Update sbt-tpolecat to 0.2.2 @scala-steward (#722)
• Update sbt-scalajs to 1.10.0 @scala-steward (#724)
• Update scalafmt-core to 3.5.0 @scala-steward (#723)
• Update sbt-tpolecat to 0.2.1 @scala-steward (#721)
• Update amazon-kinesis-client to 2.4.1 @scala-steward (#719)
• Update sbt-tpolecat to 0.2.0 @scala-steward (#720)
• Update zio-aws-cloudwatch, zio-aws-core, ... to 3.17.104.7 @scala-steward (#718)
• Update scala-collection-compat to 2.7.0 @scala-steward (#717)
• Update sbt-scalajs-crossproject to 1.2.0 @scala-steward (#716)

v0.21.2

Same as 0.21.1, but the previous release did not publish all artifacts (#715).

v0.21.1

Changes

• PollingFetcher: Get a new iterator if the old one is expired @mschuwalow (#711)
• Scala 3 @svroonland (#539)
• Upgrade to Scala 2.13.8 @svroonland (#713)
• Update sbt-assembly to 1.2.0 @scala-steward (#710)
• Update logback-classic to 1.2.11 @scala-steward (#709)
• Update amazon-kinesis-client to 2.4.0 @scala-steward (#708)
• Update sbt-ci-release-early to 2.0.28 @scala-steward (#705)
• Update sbt-tpolecat to 0.1.22 @scala-steward (#704)
• Update sbt-sonatype to 3.9.12 @scala-steward (#707)
• Update sbt-scalajs to 1.9.0 @scala-steward (#706)

v0.21.0

Changes:

• Native consumer: add Settings object with additional DynamoDB CreateTable parameters for DynamoDbLeaseRepository (#698)
• Forward forked fibers' failures to the DefaultLeaseCoordinator.acquiredLeases stream, hence failing Consumer.shardedStream (#699)
• DynamicConsumer: warn when getting offered records after shard queue shutdown (#678)
• Some DefaultLeaseCoordinator refactorings (#690)

Bug fixes:

• ProducerLive#produceChunk will suspend indefinitely if used with an empty Chunk (#701)

Dependency updates:

• Update zio to 1.0.13
• Update amazon-kinesis-client to 2.3.10 (#681)
• Update zio-aws-cloudwatch, zio-aws-core, ... to 3.17.104.5 (#692)
• Update scala-library to 2.12.15 (#594)
• Update scala-library to 2.13.7 (#671)
• Update zio-logging to 0.5.13 (#624)
• Update izumi-reflect to 2.0.3 (#634)
• Update zio-interop-reactivestreams to 1.3.9 (#679)
• Update sbt-protobuf to 0.7.1 (#605)

v0.20.0

KCL consumer improvements:

• Make all configuration for the KCL Scheduler configurable via the new configureKcl: SchedulerConfig => SchedulerConfig argument to shardedStream and consumeWith (#558)
  • Add optional metricsNamespace argument
  • Remove isEnhancedFanOut argument. To use polling, use shardedStream(..., configureKcl = _.withPolling) or consumeWith(..., configureKcl = _.withPolling).
• Bug fix: any unprocessed records were removed from the shard stream immediately after a shard end was encountered. The DynamicConsumer now throws an exception when the last record has not been checkpointed by your application code when a shard end is encountered and the shard stream closes. (#555, #556)
• Fix possible failure when terminating Producer (#544)

Dependency upgrades:

• ZIO 1.0.10
• zio-aws 3.17.9.1
• zio-logging 0.5.11
• amazon-kinesis-client 2.3.6
• logback-classic 1.2.5

v0.19.0

This release features many performance optimizations and bug fixes for the Producer.

Optimizations:

• Increased buffer sizes for higher throughput (#498)
• Use Chunk instead of List in Producer (#501)
• Remove an unnecessary copy in Protobuf aggregation (#507)
• More efficient incremental metrics collection using Chunks (#507)
• Only allocate 1 ref + promise per chunk instead of per element (#508)
• Increase PutRecords call utilization by pipelining the response processing (#508)
• Use Chunk[Byte] instead of ByteBuffer in Serdes as serialization target (#508)
• More efficient metrics combination using a map-reduce kind of method (less Ref updates necessary) (#508)
• Do not lookup the correct shard throttler for every chunk but only once per shard stream (#508)
• Optimize some hot code paths (#514)

Fixes:

• Fix issue with partition key length calculation for unicode keys (#499)
• Fix shard prediction errors for UTF8 partition keys (#500)
• Build in some margin against too large records under aggregation (#503)

Also:

• Upgrade to zio 1.0.8 (#521)
• Upgrade to zio-logging 0.5.10 (#521)
• Upgrade to zio-aws 3.16.70.3 (#521)

Keep in mind when upgrading:

• Producer#produceChunk now returns a Task[Chunk[ProduceResponse]] instead of Task[Seq[ProduceResponse]].
• Serdes now work with Chunk[Byte] instead of ByteBuffer. You can use Chunk.fromByteBuffer and ByteBuffer.wrap(chunk.toArray) to convert between the two if necessary