Enhancement
If there are problems with Kafka or TL-Sync, sending and receiving is repeated indefinitely at high frequency. This floods the log.
Improvement
When a problem occurs, the interval between attempts should increase exponentially, to a maximum of perhaps once per hour, or maybe even just once per day.
Application
No migration or customization of the application is necessary. Sending and receiving slows down automatically when errors occur. A "truncated exponential backoff" is used.
If the default settings for this in an application are not suitable, they can be adjusted. See the configurations of tl:KBDataProducerTask and tl:ConsumerDispatcher.
Test
- Break sending via Kafka. For example by starting the application without Kafka.
- Create data that will be transmitted.
- Sending must fail with stacktraces in Kafka log. The stacktraces must become less frequent over time.
- Fix the sending.
- Break the receiving.
- Create data that will be transmitted.
- Receiving must fail with stacktraces in the Kafka Log. The stacktraces must become rarer with time.