major
#29383
A dead SOCKET_APPENDER (Chainsaw, localhost:4445) in the default logging configuration can cause logging to stall and operations to abort under heavy load
The default logging configuration (tl-core, WEB-INF/conf/logging/default.xml) defines a log4j2 Socket appender named SOCKET_APPENDER that targets localhost:4445 (intended for a Chainsaw dev receiver). It is controlled by the tl_developerMode=true SystemPropertyArbiter and referenced from <Root>, so in developer mode, every logged event is routed to it.
When no receiver is listening on port 4445 (which is usually the case in developer mode), log4j’s TcpSocketManager maintains a background Reconnector and buffers writes to the unreachable endpoint. Under a burst of log messages, the socket’s send buffer fills up. Because the framework runs under log4j’s AwaitCompletionReliabilityStrategy, the logging call then blocks and/or throws an AppenderLoggingException from within Logger.info(...). Two consequences:
- The entire logging pipeline stalls—the FILE_APPENDER stops receiving events, so top-logic.main.log freezes while the application continues to run.
- The failure propagates into the calling application code that issued the log call.
Observed impact: a long-running background operation that reported progress via the TL-Script log(...) function (~3,200 messages within a few seconds) experienced a break in its logging pipeline; the main log froze mid-run, and the operation aborted silently at a clean batch boundary. Abbreviated stack trace:
org.apache.logging.log4j.core.appender.AppenderLoggingException: Error writing to TCP:localhost:4445: socket not available
at org.apache.logging.log4j.core.net.TcpSocketManager.write(TcpSocketManager.java:264)
at org.apache.logging.log4j.core.appender.SocketAppender.directEncodeEvent(SocketAppender.java:598)
at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(...)
at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92)
at org.apache.logging.log4j.core.Logger.log(Logger.java:187)
at com.top_logic.basic.Logger.info(Logger.java:174)
A dead/optional dev appender must never be able to stall file/console logging or cause the application to fail. Requested fix (any of):
- Remove SOCKET_APPENDER from the default configuration; or
- Make it robust: set ignoreExceptions="true", wrap it in an Async appender, and/or only enable it when a receiver on port 4445 is actually reachable.
Workaround in an application: override WEB-INF/conf/logging/default.xml and remove the SOCKET_APPENDER appender definition and its <Root> AppenderRef.
Migration
Developers who forward log events to an external log viewer (e.g., Chainsaw listening on localhost:4445) must now explicitly enable the SOCKET_APPENDER: add
-Dtl_logReceiver=true
to the VM arguments of the launch configuration. Developer mode (tl_developerMode=true) alone no longer activates the appender. The host, port, and serialized event format remain unchanged. Developers who do not run a log receiver do not need to make any changes—the appender is no longer part of their logging configuration.
Note for users who have opted in: the appender runs with `ignoreExceptions="true"`; while no receiver is reachable, events destined for the socket are dropped silently—file and console logging are unaffected.