#26626 Systematic logging for Kafka

enhancement

critical

#25645

OpenAPI server

#26207

REST calls via TL script to external APIs according to OpenAPI specification

#26368

Migrate dependency management and build to Maven

#26910

Optimize TL-Sync for larger loads

major

#25000

Set unversioned TLAnnotation's in AttributeSetting's

#25536

TL-Doc: Revise TL-Script pages

#25776

Column configuration for containment attributes when displayed as a form table

#25881

No more automatic creation of "table types" for database tables

#26365

Enable explicit option ordering in selector.template.xml

#26392

Selection filter for tree selection

#26411

Tiles: User-defined commands in the context menu of object tiles

#26415

Embedding properties of a list of polymorphic objects into a form

#26421

XIO: Configurable import handler

#26444

Specification of the context object in TL-Script Object creation

#26501

PDF export of an InApp form

#26532

Edit report/template type attributes via HTML WYSIWYG editor

#26542

yFiles: onDoubleClick and context menu on node

#26563

Checking dependencies for security vulnerabilities

#26588

TL-Script format for text with embedded expressions

#26620

Drag and drop with multiple selection

#26626

Systematic logging for Kafka

#26644

Kafka and TL-Sync should use an exponential backoff in case of problems

#26696

Increase minimum Java version to 11

#26715

Uniform multi-selection display for grid and tables

#26719

WrapperGenerator: Maven plugin for wrapper generation

#26726

Code" data type

#26732

Layout editor: alternative views for different model types

#26739

WrapperGenerator: Typed factory methods and addXxx(), removeXxx()

#26746

Model: Password attributes

#26761

TL-Script: contextPath() function

#26772

List-valued attributes with primitive type

#26775

Automatic translation from I18NConstants

#26811

Kafka & TL-Sync swap log messages to own file

#26816

Add Kafka and TL-Sync status to monitor page

#26864

Force password change when administrator assigns password

#26893

Optional dependencies between services (ManagedClass)

#26905

Extinguishing callback at the subject object

#26920

Suggested values for number and string attributes

#26936

WrapperGenerator: lookup methods for enums/classifiers, better typing for references

#26959

Drag&Drop: Generalization of the DropByExpression API

#26986

Do not use theme variables directly but only via CSS variables

#26999

Typed configuration: short-cut format for list-value properties

#27001

ProgressDialog: Better reaction possibility to errors in the background process

#27053

TLScript for resolving a model part

#27057

BulkIdLoad should also load flex attributes

#27060

New type: tl.util:JSON

#27067

Better tooltips for model elements, icons for TLClassifier

#27068

ResKey with fixed number of values

#27109

Show the process cockpit by default in new app

#27118

Install TopLogic via Debian packages

#27135

Delimit layout editor functions from specialist functions

#27159

Model editor: translation of all module names plus display of technical name

#27186

Context-local environment variables: Assigning application properties via JNDI

#27196

DeepL: Use glossaries for translations

#27214

TL-Script unversioned object comparison

#27215

Historical to-n references

#27234

Include ChartJS and OpenAPI in BPE Standard Configuration

minor

#15276

Customize AssociationCache for mixed MOReference

#24564

Creating ScriptRecorder templates not in the deploy folder

#25031

Adapt directory structure to Maven conventions

#26215

TL-Doc: Plugin for templates

#26288

Display of the application uptime

#26317

Enable resources normalization of a module

#26363

Structure modification of a database table via SQL language

#26377

FileManager: Remove dependency on java.io.File

#26385

Template Arguments Minify Configuration

#26387

Configure set of editable columns in grids

#26397

Easy configuration of model updates via MigrationProcessor

#26398

Abolish automated creation of TableInterfaces

#26404

Update CKEditor to version 4.17.1

#26408

Configuration format for TLClassifier

#26409

XIO: Optimized insertion into a multi-reference

#26410

XIO: Assigning an object ID from text content

#26417

Safety: Label output even in error situations and with invalid system status

#26418

Initializer for object creation dialog

#26420

Refactoring: Remove ResPrefix from AbstractFormDialog

#26423

Access to the detailed information of the type index

#26425

Property editor for editing `ConfigurationItem`s as XML

#26427

Restrict annotations to compositions

#26458

Sorting the options in the ItemEditor

#26468

Sequence of dialog buttons when configuring new list items

#26495

Scripting: Referencing objects compactly in drag-and-drop operations

#26506

Abort application start in time in case of errors in Model XML files

#26508

TableDataExport should also provide only export

#26510

Update library log4j to version >= 2.16.0

#26512

ProgressDialog: Accesses the current progress counter

#26524

App deployment from pre-built repository artifacts

#26533

FormMember names for TypeParts should be independent of overrides

#26549

Form tables: Customization standard columns

#26551

TLScript: Variable definition directly in HTML-embedded expressions.

#26552

TLScript: Script-embedding in HTML via script tag

#26568

Maven: Normalize resources easily

#26593

Display form groups from configurations initially collapsed

#26595

Combine .info and .tooltip at resource keys

#26597

Literal ResKeys with suffix keys

#26604

Table columns for resources

#26614

Translations for data types

#26700

Non-existing groups in Security Storage cause commits to fail

#26703

Update MSSQL JDBC driver to version 10.2.0.jre11

#26707

API consistency checking tool

#26708

Check for missing translations of the I18NConstants in the nightly build.

#26712

Maven configuration for calling normalize layouts

#26722

TL-Maven plugin: Goal `translate` with encrypted server passphrase for DeepL

#26723

Redefine table column for primitive model attribute (simple)

#26728

CodeEditorControl should not write ACE editor for invisible fields

#26734

Schema editor: column selection when defining key attributes

#26747

JavaDoc: Warning for CameCase content inside <p>-tags in

#26752

Removing the tl-help module

#26755

Update CKEditor to version 4.19.1

#26764

Model editor: improve the design of the context menu

#26765

Model Editor: Extend context menu with GoTo Definition

#26766

Model Upgrade: Make-Abstract and Make-Concrete

#26784

Migrate TLDoclet to jdk.javadoc.doclet

#26799

Better loading cursor support for multiple long-running GUI actions

#26814

Faster resolving of files in FileManager (without Files.exists(...))

#26815

KBDataProducerTask should react on stop request of the scheduler

#26822

Migration: DB schema update after migration

#26835

Perform instanceof checks unversioned

#26838

Reduce selection for SingleSelectionModel if necessary

#26841

Migration: moving instances to another table

#26858

Reduce calls to ModelBuilder.supportsModel()

#26871

Simplified mail configuration for IMAPS and SMTPS

#26872

Selectable ID column on type

#26877

Better context information for deleted objects

#26887

Specify database passwords encrypted

#26889

Encrypt Pepper in Argon Hashing

#26913

Improve performance when deleting multiple objects

#26917

Default selection for value suggestions: Options via printout

#26918

Prioritize implementations for in-app annotations

#26935

Make Close dialog configurable by Create

#26938

Allow configuring control tag and CSS classes for all configurable control renderers

#26940

Preset size of button icons in theme

#26946

Show resource path of the application in the environment variables

#26952

Customizability of breadcrumb display

#26953

Icon library: Bootstrap Icons

#26966

Configuring StorageMapping on PrimitiveStorage

#26982

Control width of selection and detail column via theme variable

#26983

Adjust width of filter dialogs via theme variable(s)

#26985

Introduce styleable DIV container around demo license display

#27000

Rebuild form on invalidation

#27006

Add tooltip description for LoginMessage interval attribute.

#27008

Enable tooltips via tabs

#27011

Update Maven Sources plugin to version 3.3.0

#27015

Logging in KBDataProducerTask should cover error cases better

#27024

Support for multi-file upload in declarative forms

#27055

Service editor commands should not always be active

#27063

Java class cache for TLObject implementation

#27075

Navigation of calculated config properties via collections

#27095

Bulk processing of object deletions during layout update

#27106

Delete command should (can) close open dialog

#27108

Risk analysis: Move coordinate origin down/left

#27111

Get log output even without log viewer

#27116

Support for MariaDB in combination with JNDI configuration

#27133

Adjust width of minimized sidebar via theme variable

#27151

ApplicationAction, which calls other action scripts

#27154

TL-Sync is to stop reception in case of missing messages

#27158

Shortening of button labels in the change check dialog

#27169

Declarative forms: Set property editors via configuration instead of annotation

#27173

Automatic language determination during wrapper generation

#27175

Auto-update of Debian deployments

#27180

Default label for components in the Inspector

#27225

Bulk Dependency Update

#27236

Model definition for revision

#27251

TLModelUtil: Replace declared ConfigurationException with TopLogicException

nice-to-have

#26706

Better logging when building the App-WAR

#26737

Eclipse Maven settings: current TL version as minimum version for plugin exclude

#26738

Archetype: Avoid fully qualified module names

#26914

Alias and System Properties for ZooKeeper Port and Kafka Port

defect

critical

#26850

Enums of a module disappear when creating and deleting enums in this module

#27205

TL-Sync transmitter stops on association update

major

#25761

Race Condition in TL-Sync in case of model changes, e.g. new TLClassifier

#26318

TL-Doc: Pages sometimes cannot be saved

#26367

Error when updating the layout storage

#26391

Tree selection cannot be adjusted to the current object

#26456

DatabaseLayoutCache may offer data for wrong themes

#26534

Memory hole in GraphComponent

#26565

Resource Leak: Files.list(...).foreach(...)

#26612

Invalid session revision for multiple threads

#26622

Missing test configurations for tests in Maven workspaces

#26631

XMain based tools do not start anymore

#26716

WrapperGenerator: derivation of StructuredElement not for all classes of a module

#26790

Error loading objects with long texts with H2

#26791

Knowlege Base: Error when updating validity periods when reloading object data

#26802

Migration scripts are not executed

#26808

Replay migration with H2 fails with long CLOB and BLOB values

#26809

Replay migration fails due to access to read-only columns

#26930

Long values lead to unusable column filters

#26968

Missing items in refetch update event

#27123

Meaningless selection at "Configure Tabs

#27137

In-app configured tables not exportable

#27138

Error when dragging the selected row of a non-initial slice of a table

#27139

Button menu in button bar does not open

#27193

Configuration export via context menu in model editor defective

#27207

Search suggestions of SelectFields and popups of the breadcrumb are placed incorrectly

#27228

Missing display of Flex data for historical objects

#27241

OpenAPI: Error although query successful

minor

#19344

Attributes of type TLEnumeration return collections despite multiple=false

#23801

Label calculation in Excelexport does not consider table configuration

#24788

Missing deployment of top-logic.js and CSS for production mode

#25322

Form field for mandatory DateTime attribute incorrectly marked as changed

#25343

Missing filename check with multi-upload

#25501

EncodeConfigurationValues.jsp writes unquoted parameters to the GUI

#25606

Model Editor: Missing change check for form and annotation editing

#25612

Model Editor displays link ends in the attribute table when references are changed

#25613

BPE: Creation of new process instances with mandatory fields not possible

#25652

In-app modeling: backreferences of compositions are created as multiple=true

#25732

Model-based search: Input validation: Filter query leads to NullpointerException

#25834

Error with system without "maintenance pages

#25852

Missing check for null when receiving references in TL-Sync

#25858

Model editor reloads page on script execution

#25859

JavaScript error when expanding nodes in TreeTables

#26013

Incorrect removal of item layout resources

#26373

Icon of a toolrow button cannot be changed InApp

#26379

Replay migration fails because DependencyInjection does not take place

#26382

Error messages "Duplicate tag name..." when starting an application in the IDE from a TL-Studio

#26383

ConcurrentModificationException in progress dialog

#26384

partnerGroup is not transferred to the component in selector.template.xml

#26386

Duplicate icons for ID column _self in TreeTables and TreeGrids

#26388

Hide root node at null model

#26393

Minimize configuration resets properties with ValueInitializer

#26396

Drag and Drop: Icon from table header to drop zone: NPE

#26402

Model Editor: Diagram: Error when dragging a class from another module to your own worksheet

#26405

TTypeRewriter logs warning "Unable to resolve items by external reference" also unnecessarily

#26406

WYSIWYG YouTube plugin has problems with embedding

#26407

No thread context in ProgressDialog.run(I18NLog)

#26422

CodeEditor: Error with context completion in XML mode

#26424

Model Diagram: Missing Upper Type Connection for New Classes

#26426

LogControl: Error message due to concurrent updates

#26428

ChangeCheck defective during tree selection

#26430

ProgressDialog does not visually end at 100% if Step-Cnt != 100 is selected

#26431

Changed superclass relationship leads to changed attribute list in the form editor only after a restart

#26438

Checkbox to select all entries cannot be clicked

#26441

No session invalidation during login and password change process

#26443

Layout Editor: Remove button of the "Custom transaction" type

#26446

Generic forms: Visibility adjustments of overridden attributes are not taken into account.

#26447

Reference checks: Are only checked in the form, not when committing

#26457

Wrapper generation for modules without singletons leads to broken classes

#26460

Model import no longer possible

#26461

Overridden attribute inherits mandatory "Default value" customization

#26465

Module names with parts from digits allowed but not working

#26484

In-app template for grid and tables: Function "Verifier for use as list item" does not get component model

#26496

Temp folder name clash for multi-user development on Linux

#26500

Unnecessary indentation for non-existent images

#26513

TL-Doc: External links not clickable

#26515

Error in "util.js", line 765: "services" is undefined

#26517

Test modules must provide test web application

#26520

Form creation with "foreign objects" and individual template fails

#26523

Missing image replacement in WYSIWYG editor

#26525

Import of BPML files with StructuredText with some images erroneous

#26529

Model transfer to development environment: ResKeys remain in dynamic properties

#26531

FormEditor: "Macro" configuration error

#26536

When rendering HTML from TLScript expressions, configured renderers are not taken into account

#26540

LayoutStorage indexes layouts by layout paths with different path separator

#26543

Log error: Web application resources must start with a '/' character: ...

#26550

TLScript: Quotes in HTML text content are not removed

#26554

Calculated web folder references delete folder when deleting the base object

#26555

Bookmarks from TL-Doc always use the internal address

#26557

TL-Doc: Highlighting of code blocks leads to RegEx errors

#26559

Not all options of `SelectField`s can be scrolled through with the keyboard

#26564

Cryptic symbols in tile representation

#26569

Missing constraint violations when deleting objects referenced by mandatory fields

#26573

Missing icons for Document's in WebFolder'n

#26575

Model event during rendering phase in ThreadDetailComponent

#26577

Import of documentation does not work on Maven workspaces

#26579

ClassCastException when changing the type for the FormDefinitionEditor

#26583

Encoding problems in system overload hint page

#26584

Wiki formatting in multiline text attributes / text fields broken

#26587

Option providers do not update calculated suggested values

#26591

Unwanted display "Fully qualified name" when creating attributes

#26599

TL-Doc: At startup the pages from the workspace are not loaded

#26602

No admin button in Modern theme

#26608

Requirejs logs errors when using highlightjs

#26609

DeepL translation buttons are no longer displayed

#26611

Missing GUI update in StructuredTextControl after field change

#26619

Inlining a file in LayoutModelProcessor broken

#26623

TestComments fails by default in a new non-TopLogic internal module

#26624

Incremental updates in TreeGrid do not always work

#26625

Failure to apply application configuration to calculated columns

#26629

Test JSP compile: Jetty not found

#26630

Invalid resource names when searching for migration scripts

#26635

Remove dependency on apache-mime4j-0.3.jar

#26637

Raise dependency pac4j to version 5.3.1

#26638

Raise dependency H2 to version 2.1.210

#26639

Remove Ext Module for Jetty

#26640

Remove dependency openxml4j

#26641

Update Jetty to 9.4.45.v20220203

#26645

TestComponentConfiguration should not test templates

#26646

Udate POI to version 5.2.2

#26647

Update jsoup to version 1.14.3 and guice to version 5.1.0

#26649

Update commons-io to version 2.11.0

#26650

Attribute "picture" is displayed twice on profile

#26651

StatusReportGrid: Import / Export broken

#26653

Invalid MySQL Dependency in ProjectDemo

#26656

Changes to InApp components leaves locks behind

#26660

Remove TL Remote

#26661

Memory hole in tile layout

#26663

TreeGridComponent: Selection is not reset after discarding a transient row

#26664

Work breakdown structure: Missing code migration

#26665

ProjectStructureEditComponent: Application of column styles broken

#26666

StatusReportOverview: StatusReportDialog does not initialize context component.

#26670

Earned Value: Incorrect calculation of accumulated actual values

#26671

Status report: Improvements

#26679

Fix Jps pages.

#26680

Multiselection support in Tree, Table and TreeTable

#26684

Milestone dialog: Show name also in viewmode, otherwise form fields jump

#26686

Disable cache of the last messages sent via TL-Sync

#26691

Webfolder: Erroneous behavior of multiple upload file selection

#26694

replace itext 2.1.7 with openpdf 1.3.27

#26698

Raise MySQL Connector to version 8.0.27

#26699

Collapsing the selection changes the selection in TreeGrids

#26704

Dependency Analysis Tool: No Duplicate Classes in ClassPath

#26710

ModelBuilder for Services view inserts user sessions into the table

#26711

ACE Editor and requirejs define the global variable require

#26717

JavaDoc writes outside the module by default

#26720

CodeEditorControl is to report client-side errors of the ACE editor

#26724

Meta-model: access to the index of a classifier

#26725

Schema editor: error when saving

#26727

Missing adjustment of (default) selection after tree update

#26730

Column definitions are not applied to composition tables

#26731

Model Editor: Delete in detail view removes wrong diagram element

#26735

Form editor: crash if attribute no longer exists

#26736

Double configuration of the description of a TLType

#26744

Post-create action in trees does not work with in-app commands

#26748

Insufficient quoting when writing JavaScript

#26750

Prohibit drop from non-TL object to tables and trees

#26757

Inconsistent API of StructuredElement

#26758

Model editor loses selection during relayout

#26760

Project Demo: Use H2 database config as local default.

#26763

TL-Doc: Copying old help documentation does not work

#26767

Model export writes resource files to wrong module

#26768

Parents are not always expanded after selection change

#26770

Model Editor: Error during further editing after deleting element

#26771

NPE when filtering all threads in the thread monitor

#26773

TL script: Self-expression in concat() is not taken into account

#26774

TL script: Error message when sublist() is called with too large index

#26783

WYSIWYG CSS uses variable defined in com.top_logic.icons

#26786

Launch-Configs still reference Java-8 VM

#26795

Data migration for ticket #25881 and #26398

#26797

Transaction with user input: invisible properties of the form model cannot be assigned values (initialized)

#26800

Scripting: Record StructuredElement with arbitrary singleton root

#26803

KnowledgeBaseRuntimeException when accessing deleted reference

#26804

No consistent deletion of model elements during model upgrade

#26806

Script recorder: record branch and revision only if necessary

#26807

Possible NPE in MonitorEventAccessor

#26812

Subtree update of an invisible root node faulty

#26827

No error message when accessing foreign key references if foreign key cannot be resolved

#26836

StackOverflow due to configuration error: Calculated string attribute returns integer

#26842

Scripting recorder: Playback of multi-selection faulty

#26845

NPE in the translation service if no network connection is available at startup.

#26848

Document management does not work in non-versioning system

#26849

Maps of ConfiguredInstances should keep order

#26852

ApplicationTesting: No two sessions possible for the same user in JUnit test

#26853

EditComponent loses edit mode after deletion

#26854

Tab delete of a legacy tabbar is not recorded

#26855

Missing dependency DynamicComponentService -> SafeHTML

#26856

Lack of documentation adaptation to new + context

#26857

Inconsistent "source" properties in documentation pages

#26859

Correct various Eclipse settings

#26865

Missing selection marker in composition tables

#26866

Instance Browser: New transient object does not show type

#26867

Configuration uses unencrypted SMTP and IMAP password

#26868

End animation when table is cleared from GUI

#26870

Inactive scheduler crashes the application

#26873

Application monitor "Environment variables" may display safety-critical values

#26874

Service Editor displays "secret" of the OpenID service

#26880

Selection of icons in the icon chooser is not scriptable

#26883

Drag preview image in script recorder does not use label

#26884

Declarative forms: Missing GUI update for programmatic changes of a list-valued property

#26885

Constraints on declarative forms with arguments from a container reference lead to errors for new elements

#26890

Overridden properties are not initialized correctly when booting from model definition

#26891

Jerky table contents in FrozenTables

#26894

Missing dependency ModelBasedSearch -> SearchBuilder

#26895

Missing dependency MailServer -> MailReceiverService

#26896

DnD: Drop operation with drop type "child" on table gets the wrong row object

#26903

Under unfavorable circumstances data from future revisions visible in AssociationCache's

#26904

CSS editor does not work with Top-Logic theme variables

#26906

Default annotations cannot be inherited from the attribute type

#26911

OutOfMemoryError on App-WAR generation

#26921

ClassCastException when evaluating security rules that refer to (non-structuredElement) singletons of a module.

#26922

With generated subject classes, a default provider of an attribute in a non-structure class does not get a create context

#26923

ModelNamingSchemes do not get "value context" in some cases

#26924

Horizontally displayed radio button selection extends beyond form boundaries

#26925

BreadCrumbs do not update when inner tabbars are not visible

#26926

Rare ArrayIndexOutOfBoundsException on tab change

#26928

Hidden component does not appear anymore, although it gets a supported model

#26929

Scripting recorder: No templates are loaded in project demo

#26934

Update yFiles to version 2.4.0.6

#26939

Changed order of standard columns in new table

#26942

Frozen table: drag selection contains duplicates

#26945

Inconsistent lock timeout at application startup

#26957

No update of node properties after model change

#26960

Schedule: Automatic collision avoidance does not work

#26962

TL-Sync: Tabs are lost with string attributes

#26965

Theme icon with value "none" is not resolved correctly

#26970

Wrong DB configuration at replay after changing FastList table

#26972

Scripting recorder: stops execution after reloading the page

#26984

Not all tab bars take into account the theme variable setting 'LEVEL_ONE_BAR_SIZE'.

#26988

In-app documentation generator does not extract documentation for overwritten config properties

#26989

Broken GWT Launch Config

#26992

Asynchronous loading of layouts for themes crashes short-running test cases

#26993

Application monitor uses wrong locale for displaying working memory values

#26996

ChartJS Update to 3.9.1

#26997

JSoup Update to 1.15.3

#26998

Update Batik Graphics to 1.15

#27012

Potential NullPointerException with Drag&Drop

#27018

SchedulerGui: Detail view "Selected run" always empty

#27019

Inconsistent logging: scheduler uses its own log file, but its tasks do not

#27021

Missing search buttons in Modern theme

#27022

Cyclic FileSystemProvider dependency in Log4J2

#27025

Inconsistent font type and size for number fields

#27026

Creating company contact: Half of the values are not saved

#27027

Declarative forms: SelectField disappears after upload if option list depends on mandatory property

#27028

Data migration: Delay migration processors

#27029

Failure to use PropertyInitializer for list-valued properties.

#27030

SafeHTML not started during automatic data migration

#27031

Missing adaptation of the stored model in model changing EventRewriter

#27032

Unexpected order of backup log files with Log4j2

#27035

Missing constraint evaluation for list-valued properties

#27036

The ThemeImage "NoIcon" does not disappear, but is displayed as an empty white image

#27039

Administration: Contact cleanup produces error message

#27040

Jackson FasterXML Update to 2.13.4

#27042

MaintenanceJspBase should write to the log first, then to the client, instead of the other way around.

#27043

Batik update to 1.16

#27045

Maintenance page ScanData.jsp does not compile in openJDK 1.8

#27047

Avoiding data migration in #25732: Saved search queries may no longer be readable

#27049

Transaction monitor renders collapse buttons twice

#27065

Missing CDATA quoting when serializing ConfigurationItem

#27069

Minimize an EditComponent discards changes

#27070

Duplicate error icons for TL script fields

#27073

LDAPAccessService: connection establishment using the host name instead of the IP address

#27078

Prevent unnecessary, incorrect conversion of simple attribute values

#27085

Create in tree grid on misconfiguration leads to NullpointerException

#27087

Map-valued properties in configurations of form editor plugins do not work

#27088

Multiline text in layout template parameters is reformatted

#27090

TreeTableComponent shall initially select the first visible row, if initially one row shall be selected

#27094

Reconfiguration of a component in the tile environment removes toolbar commands

#27096

Tree does not scroll the selected node into the viewport on (external) selection

#27098

Default-for definition in in-app tile layouts does not work

#27107

Tab "Process Cockpit" is available twice for selection at "Configure Tabs

#27110

Launch new app without test code

#27113

In the Process Cockpit, the number of tasks is not displayed directly after login

#27115

Error in deployed app without autoconf directory

#27119

Mail dispatch: Startup notification leads to NullpointerException

#27121

In-app reconfiguration and re-export of a modular layout crashes the application

#27122

Reconfiguring and exporting an inherited (modular) layout is not applied

#27129

Declarative forms: Values of @MapBinding properties are not taken over

#27130

TLScript fields not visible in BPE

#27132

Chart.js JavaScript is included in the application CSS

#27140

Personalization of forms for local types crashes the application

#27142

Missing error with wrong command clique

#27146

Memory leak due to multiple registration of selection listeners in the GridComponent

#27147

Update Dependency Check to version 7.4.4 or above

#27148

Scriptrecorder: Input of date values via date popup control is not recorded

#27153

Error when running tests in a build reactor via Maven

#27164

Missing sender address in mails

#27165

Platform-Specific Folder Separator for Mail Folders

#27167

Normalizing JSPs leads to wrong indentation for if-else blocks

#27170

ClassCastException on representation of a tl.core:DateTime attribute in a composition table

#27172

LayoutEditor: StackOverflowError with wrong model declaration

#27176

Unfolding in the model element tree causes selection of another unfolded element

#27178

Infinite tree in table selection dialogs leads to errors

#27179

Service configurations cannot be changed in a deployed system

#27182

Memory overflow during execution of scripted tests

#27191

Time selection logs errors

#27197

Missing component model for grid object initialization with multiple input channels

#27203

Application tests with chart.js report on home page fail

#27204

PersistentEnumeration.tValue(...) leads to ClassCastException

#27218

Grid tables: Calculated column: Single-Value SelectField requires list value.

#27221

Crash with inconsistent layout model (user trap)

#27227

User time zone and thus SubSession time zone can be zero

#27231

Incremental modification of a set-valued attribute not possible.

#27239

Typo in database configurations at "password template

#27240

OpenAPI: Missing content type for JSON request content

#27249

TLScript suggestions are always displayed in the same language

#27250

Missing Maven profile "ide" in Eclipse

#27259

PostCreateAction on Drag and Drop is executed at the wrong time.

nice-to-have

#25295

Refresh empty search returns NPE

#26792

Scripting: Assertions try to access invisible columns

#27128

Double pop-up in view "External selection in the tree

enhancement

major

#26626

Systematic logging for Kafka

Monitoring RequiresMigration TLSync

Challenge

The DiFa systems within a network architecture make extensive use of Kafka interfaces, whether via TL-Sync or custom-implemented protocols (e.g., JSON).

With HTTP servers, it is common practice to log all communication on the server side, such as request logs that record the metadata of all requests. These logs make it possible to trace communication across systems and various network components. In the event of an error, it is also possible to see at which point communication breaks down or which components are refusing to process the request further.

Communication via Kafka, on the other hand, cannot be traced in the current situation for several reasons:

Producers do not adequately log the sending of messages
Consumers do not log the receipt of messages
Recipients do not adequately log the processing of messages
Log files are flooded with message content
Log information is retained for only about 20 minutes; after that, log entries are overwritten due to the sheer volume; file sizes and rolling file numbers are insufficient
Message contents are encrypted and therefore cannot be located or inspected using Daimler tools such as Lenses.

Solution Approach

Our Kafka implementation generates a communication log. The sending and receiving of every single message is logged before further business-logic processing and regardless of the processing result (exception save/finally). The logs contain the necessary metadata to trace a message’s path from the producer to the consumer.

The log format is standardized and largely symmetrical between the send log and the receive log to facilitate analysis.

The log has two log levels:

INFO: Metadata for a message per line
DEBUG: Metadata + message content, possibly multiple lines

There is a send log for all producers and a receive log for all consumers (it is unclear whether this is sufficient, or whether each producer and consumer generates its own log (its own file)).

The metadata must include (obda):

Timestamp of the send/receive
Topic
Message ID
Chunk information (if available)
Offset (if possible)
Application server (IP, hostname, cluster node ID, etc.)
Kafka server (IP, hostname, etc.)
Name/ID/Class of the producer or consumer
TBD...

The log settings are configured sufficiently to track communication at the INFO level for at least 1 week.

This solution is not limited to EPP but applies equally to all ASCon systems in the DiFa landscape.

Implementation

I would just ask that we review the implementation together before final completion, particularly regarding the log information.

Code Migration

Kafka senders and receivers have a new mandatory configuration option: CommonClientConfig.getLogWriter()
- The tl:KafkaLogWriter is used to log the metadata and content from Kafka messages in a usable format.
- The tl:TLSyncRecordLogWriter and the tl:KafkaStringLogWriter can be used as templates for your own implementations.
- To find all instances where a tl:KafkaLogWriter must be specified, search for KafkaProducerService and KafkaConsumerService in all .config.xml files.
- As an example configuration, you can use the file com.top_logic.kafka/src/main/webapp/WEB-INF/conf/kafkaConf.config.xml, which contains both the tl:KafkaProducerService and the tl:KafkaConsumerService.
The constructor of the tl:ProducerProxy class now requires a tl:KafkaLogWriter as an argument.
The classes `tl:CommonClientConfig` and `tl:KafkaCommonClient` now have an additional type parameter specifying the type of the value being sent or received.

Test

Start the Kafka demo once with log level DEBUG and once with INFO. Create at least one object in each case. Then check whether the expected amount of data for the Kafka messages is present in the log.

INFO: Only the metadata.
DEBUG: The entire content as well.

The data must be logged by both the sender and the receiver.