#25035 StructuredTextControl has problems with special characters in image filenames

enhancement

critical

#23472

In-app configured drag and drop with pre-target validation

#25424

Meta-Model: Generic navigation to container: TLObject.tContainer()

#25542

Supplement tables with calculated columns

#25646

In-app transactions with user input

#25647

Multiple relationships between components

#25704

Layout editor: enable in-app configuration of tile cockpits

#25863

Layout Editor: Selection Component

#26004

Tables with dynamic columns

#26229

Introspection Layout Editor

major

#25634

Creation of elements of polymorphic compositions in a form table

#25787

TL Script Task

#25841

Option provider: access to the context of objects being created

#25872

Create test cases in-app

#25921

State-dependent executability of commands

#25923

Better heuristics for English model element names / automatic translation

#25925

"Real" overlay as deployment configuration

#26000

Display type options for polymorphic configurations in EditorFactory as drop-downs

#26011

TL script: Command to produce log entry

#26033

Fill references about tree selection

#26063

Embed Youtube videos in tl-doc

#26066

Tree tables with dynamic columns

#26067

Status-dependent object icons via annotation

#26068

Context-dependent number ranges via annotation

#26080

TL script: $obj.set($ref, $value) with dynamic reference

#26096

Drag and drop scripting in views that have been configured in-app

#26097

Automatic application restart from the development environment

#26117

SecurityProviders in tables and grids

#26127

IDE startup: Automatic redirect to the right context path

#26141

Define buttons on selection components

#26158

Transaction with input: reset form, easy access to context in option providers

#26182

Support for bidirectional transforming channels

#26268

Tool for importing TopLogic dumps

#26276

New API for Progress dialogs

minor

#23563

Selection of a CalendarMarker implementation in the model

#25878

Remove EWE from TLCommittee / TLBoard

#25899

TL-Doc: Simplify TLObjectLinks

#25922

Replace DelegateStructureHtmlTree in POS and Prime with TreeComponent

#25935

Move definition of navigation role for org structure from contact to projectdemo

#25937

Include logback in TL Studio

#25942

DownloadButton for BinaryData attributes in Grid

#25954

Change ViewStateManager in tl-project to CommandApprovalService

#25960

Deprecation: remove com.top_logic.mig.html.HTMLTree

#25961

Deprecation: Remove EWE from tl-demo

#25962

Deprecation: Remove EWE and Tickt/TTS

#25966

Schedule: Collision detection for schedule shifts

#26005

Cache for TLModel type hierarchy

#26017

Utility to create singleton or empty list

#26022

Prevent unnecessary dialog during InApp component creation

#26045

Assertion on the model of a component

#26062

Youtube Plugin for CKEditor

#26082

CreateTypeOptionsByExpression: Include subtypes

#26106

Jump to default jump target in grid detail opener

#26119

Speaking error message for RegExpConstraint

#26179

Enable standard security for InApp components

#26184

Enforce encryption of configured passwords for kafka SSL

#26189

Default-For for generic dialogs

#26195

Introduce "commons" classifier for in-app implementations.

#26221

Environment variable tl_storage_dir also for start from development environment

#26222

TL script: list.reverse(), extension list.subList(...)

#26225

Also provide properties of subtypes for columns of tables to be displayed.

#26241

Remove CommandInvocation

#26263

No configuration defaults for `content-layouting` in layout configurations

#26274

Short-cut WindowScope.deliverContent(BinaryDataSource)

#26280

Functional interface: com.top_logic.layout.form.Constraint

#26295

Configure basic set of table columns in-app

#26297

Type index with annotation and configuration information

#26300

Prevent unnecessary copying when reading stacked configurations

#26316

Configuration to disable branches

#26329

Change user menu to standard popup

#26340

Disable table reset as single button

#26341

XML import: object refs with multiple ID attribute

#26344

set partnerGroup in selector.template.xml

defect

critical

#25842

TL-Sync deletes random objects if it does not find the actual ones to be deleted

#26311

GridComponent: Programmatic invalidation after multi-selection leads to undefined behavior

major

#22838

TLSync imports HAS_WRAPPER_ATT_VALUE.REV_CREATE from the source system

#24187

ValueDumper / ValueParser ignores BinaryData.getName()

#24189

BinaryDataFactory discards names for files > MAX_MEMORY_SIZE

#25598

NPE in TL-Sync when sending overwritten attributes whose definition is not exported.

#25934

Layout tooling fails in linked workspaces

#25958

Ant targets in Eclipse workspace fails with "The linked resource '/com.top_logic.basic.core/src' in project '.org.eclipse.jdt.core.external.folders' does not exist

#25967

TL-Doc: After a link the line does not break anymore

#25968

TL-Doc: view always jumps to the top when editing if heading format is selected

#25969

Application hangs constantly with wait cursor

#25974

NPE in deployed demo

#25975

Copy images between pages does not work

#25982

JS error when editing long texts in CKEditor

#25985

TL-Studio: Application startup fails due to unresolved dependency

#25998

TL-Script context help broken

#26037

MemoryLeak due to OptionsByExpression -> AttributeFormContext

#26059

MemoryLeak because the SecurityStorage is a ClusterManagerListener that does not deregister

#26084

On english pages you can find german TLObject links

#26088

Memory leak in DefaultLayoutGraphCleaner

#26093

Implementations of calculated attributes cannot be overwritten

#26112

Calculated attributes on historical stands may calculate incorrectly

#26152

Model editor: UML class diagram bounces after each selection

#26217

TL-Doc: No error message in case of failed import

#26353

Crash after deleting tile

#26358

Option lists are no longer loaded lazy

minor

#24425

ClassCastException on live change of a persistent list attribute

#25035

StructuredTextControl has problems with special characters in image filenames

#25603

Null argument on generation of JavaDoc for tl.element

#25801

NamedConfiguration and NamedConfigMandatory must be @Abstract

#25818

Documentation: link suggestions always in German even when editing the English translation

#25819

Too many link suggestions during link creation

#25883

Unnecessarily wide index in `hasStructureChild`.

#25884

Duplicate indices in `hasWrapperAttValueBaseAssociation` and all derivatives

#25907

Layout editor: inconsistent behavior of unfold state, selection and reaction to object creation

#25928

Invalid configurations for Producer and Consumer

#25949

TL-Doc: Copy&Paste transfers unwanted styles

#25951

Application tests: CheckLabeledExecutabilityOp fails to assert non-executability of literal ResKey reasons.

#25952

Model Editor: Enum default provider does not work for enum references for which singleton link storage was explicitly selected

#25953

Two enums with the same name cannot be created in different modules

#25957

Error when updating assization caches from the commit

#25970

DataItemControl not usable with multiple DataField's

#25972

There are no help IDs for the in-app editors stored in the "Development" area

#25976

Test Error with TL Script in Executability Rules in Application Configuration

#25977

No fallback for display names of languages

#25999

Disappearing toolbar in tile layout

#26002

ConfigurationItem-valued properties with format annotations may cause an attempt to create abstract config items when deserializing null values

#26006

Scrollbars in burger and context menus when resolution is changed

#26007

Unnecessary scrollbar in HTMLTree

#26008

No visual feedback for possible dropzones in FrozenTables

#26009

Monitoring cleanup tasks crashes if they are not registered

#26012

Help editor crashes when a user has chosen a language that is not in the supported languages of the application

#26019

Buttons in filter dialogs disappeared

#26021

DeepL translation service crashes the application if the service is not available

#26024

I18N for "Create page" is wrong in dialog

#26030

Two classifiers with the same name can be created in the same enum

#26031

Layout editor: Delete view and Delete dialog do not ask for confirmation

#26039

Gridcomponent: Display of context menu for new (transient) row object leads to UnsupportedOperationException

#26040

Partially no label for custom model type

#26041

In the context menu for a new grid line the command "Delete" is offered, crash on selection

#26042

Calendar control in filter dialogs does not work in Modern theme

#26043

Milestones / Grid: When a new milestone is created, a warning message is logged.

#26048

Enable GUI action after custom transaction

#26054

Misplacement of error icon for text fields in grid

#26055

TestComileJSPs fails in linked workspaces

#26064

Error with Boolean and Number support in JSON configuration types

#26069

TL script: Evaluate ToString only at compile time if arguments are primitives.

#26075

Incorrect options for configurations of components

#26077

Declarative forms: @Encrypted properties are displayed at the GUI in plain text

#26085

An attribute of type tl.model.search:Expr is not displayed in the form above a TL script editor

#26090

Memory Leak because the TLObject "IMAPMailFolder" is listener at IMAPFolder

#26092

Hidden attributes are offered to the user for selection in the table columns by default

#26095

Missing constraints for enumeration and classifier names

#26101

Attributes of type tl.model.search:Expr cannot be edited in the grid

#26103

JSR-305 annotations for Guava

#26110

POS: Risk elements: Warnings in log files when new instances are created

#26111

Context menu commands for table rows are erroneously executed on the table model when opening the context menu in the background of the table

#26114

Incorrect quoting in TagWriter for XML with CDATA sections in attributes

#26120

Misleading name for type options field in declarative forms

#26124

Missing replacement of %LAYOUT_RESPREFIX% in typed templates

#26125

Error "Invalid attribute name aria-label" in TL Doc after cut & paste

#26135

Cockpit tile: Schedule: Milestones cannot be edited, authorization check incorrect

#26137

No error message for invalid tooltips

#26138

Unfavorable DeepL configuration for translation of HTML fragments

#26139

JavaDoc: Embedded values are not displayed

#26140

TLModelCacheService slows down object creation by about 70%.

#26145

Infinite loop when updating KB caches

#26150

BPE still has old online help

#26151

tl-doc still uses legacy overlay to show the help editor

#26153

Migration: Replay fails if string values with line breaks are contained in the dataset where a line ends with a semicolon

#26154

Missing entries in the authorization view for InApp components

#26156

CommandApprovalService: Declared exceptions do not take effect.

#26159

Crash in design mode when a create form gets a list model via a channel combination

#26163

No feedback if layout overlay cannot be applied

#26166

Too many prompts when deleting in-app dialog

#26168

Attribute defaults cannot make transactional changes

#26171

A TL script expression entered in a wizard template is reformatted when transferred to the final template

#26172

Buttonbar in dialogs differs in Modern and Sidebar

#26175

Error at startup: I18NStructuredText cannot be indexed by Lucene

#26180

Still too many buttons in the sidebar layout

#26185

Script recorder: Templates can no longer be created

#26192

UuidDefaultProvider is displayed for all attribute types

#26198

Application does not store service config files in configured autoconf folder

#26199

Drag&Drop of text in trees and tables causes NPE

#26206

Configuration option "Enable help" always checked

#26210

Views cannot be deleted

#26214

#26218

TypedConfiguration: Crash when saving nullable enums

#26220

Transaction with input: button order inconsistent

#26223

TL script: Under unfavorable conditions wrong parameter values when evaluating local functions

#26224

Descriptions with line breaks cannot be saved at model elements

#26226

Dynamic columns: NPE if the column model is zero

#26233

Chrome Browser: Users Are Not Logged Out After Closing the Browser Window

#26239

KPI reports are no longer displayed

#26249

Invalid option for selection component

#26250

TL script: CCE when sorting in reverse order

#26251

Display of DateTime objects difficult to read

#26254

Layout export fails when changing layouts with overlays

#26259

Scriptrecorder template creation does not respond to new actions / Saves outdated work state

#26260

Error message in ObjectTreeNaming if the searched label is correct

#26262

Replay of tests replacing components not possible

#26264

TL-Doc: RequestLock when copying images

#26269

Single Selection in TableControl causes duplicate events

#26270

LastLoginAccessor loads all user session data

#26271

TL-Doc: Drag and drop of images does not work in some situations

#26272

Component buttons have no icon in disabled state

#26273

No tooltip for MessageBox buttons

#26289

CommandHandlerProxy are not displayed in the context menu

#26290

Unnecessary display "Edit/cite comment" in context menu

#26293

Stakeholder: Creation dialog allows any number of characters as "Description", the form only 255

#26294

Stakeholder: Description at creation is not a mandatory field / may be empty

#26296

Initializer of configuration dialogs do not set form model

#26301

NPE when importing dumps via the interface

#26304

Faulty analysis in DeleteChecker

#26305

ConcurrentModificationException in TLModelCacheEntry

#26308

With every click in the service editor "funny" messages appear in the log

#26310

Create executable WAR by default

#26312

Missing tooltip for internationalized names

#26313

Properties of type "Set" should not be mandatory

#26314

No selection of xlsx files possible in ScriptRecorder

#26320

Error when operating the application via keyboard

#26325

"Reset form" does not ask for confirmation

#26326

Form does not hide if type is not appropriate

#26335

Display of online help fails on Windows computers

#26338

Remove BoundCommandGroup id logic from defaultFor

#26342

I18NConstantsChecker does not make entries in missing.properties

#26343

Zero components by ReplaceComponentVisitor

#26346

TLProject: Structure planning: "Designation" column is not displayed

#26349

ClassCastException in ConfigurationPropertyListModelBuilder

#26352

Component name selection no longer a drop-down

#26359

Safety: Do not crash when looking up instances for a type whose table does no longer exist

#26360

Active color in decent theme disappeared

#26361

Too much choice in Security Object Provider

nice-to-have

#26089

Template for new app: Default module is not loaded via Autoconf

#26211

URLPathParser crashes with the empty string

#26212

NPE on form display, if the object does not have the displayed attribute, but a visibility is set

defect (User-visible)

minor

#25035

StructuredTextControl has problems with special characters in image filenames

DataTypes Documentation

The StructuredTextControl has at least the following issues with images that have special characters in their names:

Characters that are URL-encoded cause the image to be deleted immediately after upload. This is caused by the code that deletes images that are no longer in use. In this process, the name of each image is compared to the names of the images currently in use. However, the names of the images in use are URL-encoded. As a result, " Koala - 128px.jpeg " is compared to " Koala%20-%20128px.jpeg." This means the entry is not found, and the image is deleted as "no longer in use."
Even if this is fixed, such images will still not be displayed. The URL for the image would then be Koala%2520-%2520128px.jpeg. The percent signs in the encoded name have thus been re-encoded. The name is therefore double-encoded. This is likely why the image cannot be found.
Characters not included in ISO-8859-1 cause a problem when the response is sent to the client after the image is uploaded:

java.io.CharConversionException: Not an ISO 8859-1 character: €
    at javax.servlet.ServletOutputStream.print(ServletOutputStream.java:130)
    at com.top_logic.layout.wysiwyg.ui.StructuredTextControl.sendResponse(StructuredTextControl.java:735)
    at com.top_logic.layout.wysiwyg.ui.StructuredTextControl.uploadFile(StructuredTextControl.java:635)

The problem here is that while the StructuredTextControl correctly specifies that the response should be encoded in UTF-8, the implementation of `javax.servlet.ServletOutputStream ` used by Jetty apparently ignores the character encoding and uses ISO-8859-1 instead. (To understand the source code: Java characters are internally encoded in UTF-16. This corresponds to ISO-8859-1 as long as the high byte is not set.)

{{{#!java

public void print(String s) throws IOException {
if (s==null) s="null";
int len = s.length();
for (int i = 0; i < len; i++) {
char c = s.charAt (i);

//
// XXX NOTE: This is clearly incorrect for many strings,
// but is the only consistent approach within the current
// servlet framework. It must suffice until servlet output
// streams properly encode their output.
//
if ((c & 0xff00) != 0) { // high-order byte must be zero
String errMsg = lStrings.getString("err.not_iso8859_1");
Object[] errArgs = new Object[1];
errArgs[0] = Character.valueOf(c);
errMsg = MessageFormat.format(errMsg, errArgs);
throw new CharConversionException(errMsg);
}
write (c);
}
}

}}}

Improvement

Before checking whether images can be deleted, the image names are now decoded.
Decoding must not be performed only after com.top_logic.layout.wysiwyg.ui.StructuredTextControl.linkImageSource(String, Element), as suggested in the patch, because the encoded string is already set as the src value in the image at that point, whereas the decoded version must also be used there. Therefore, decoding is now performed in com.top_logic.layout.wysiwyg.ui.StructuredTextControl.getImageID(String, String).
In `com.top_logic.layout.wysiwyg.ui.StructuredTextControl.sendResponse(DisplayContext, FileItemBinaryData, String)`, the `PrintWriter` is now used because it can handle strings, whereas the `OutputStream` only processes binary data and then ignores the encoding.

Test

In the documentation: Inserting images with special characters, e.g., ÄÖÜ, €, spaces, etc. The image must remain intact even after saving.

Related Tickets

Noticed in the context of #19000