enhancement
If problems occur with Kafka or TL-Sync, sending and receiving are repeated an unlimited number of times at a high frequency. This floods the log.
Improvement
If a problem occurs, the interval between attempts should increase exponentially, up to a maximum of perhaps once per hour or perhaps even just once per day.
Application
No migration or modification of the application is necessary. Sending and receiving automatically slow down in the event of errors. A "truncated exponential backoff" is used.
If the default settings for this are not suitable in an application, they can be adjusted. See the configurations for tl:KBDataProducerTask and tl:ConsumerDispatcher.
Test
- Disrupt sending via Kafka. For example, by starting the application without Kafka.
- Create data to be transmitted.
- Sending must fail with stack traces in the Kafka log. The stack traces must become less frequent over time.
- Fix the sending.
- Break the receiving process.
- Create data to be transmitted.
- Receiving must fail, with stack traces appearing in the Kafka log. The stack traces must become less frequent over time.