From what I understand these are the main differences:
1. Deployment Model
- Flink is centralized and Kafka Streams is distributed
2. Fault Tolerance
- Flink maintains a snapshot of the workload so in case of failure it starts up right away while Kafka Streams has to rebuild its state from backup (if RocksDB state is not maintained, like with stateful k8s pods)
3. Flink is not limited to Kafka while Kafka Streams only works with Kafka
4. Transaction support
- Flink has some limited transaction support across systems like Kafka, HDFS and Cassandra. Kafka Streams EOS only works within Kafka. (There is a KIP open for this in Kafka)
5. Bounded and Unbounded data support
- Flink supports both natively while Kafka only supports unbounded. It is possible to also support bounded (batched) in Kafka streams but you need to grow your own. https://medium.com/swlh/etl-batch-processing-with-kafka-7f66f843e20d