#30031 The database does not enforce uniqueness of reference values without duplicates: a unique index on the link tables, separate link tables for bags

enhancement

major

#29993

Configuration: Each configured function can be implemented in either Java or TL-Script, using the same configuration syntax (including in the React configuration editor)

#30000

Design System UI pass, part 1: one selection form; surface-interactive-selected becomes surface-interactive-strong

#30006

Engine Extensions for the Tax Application: writable secret storage in the database, application context path in development mode (tl.contextPath), and instance search by attribute value

#30025

React UI: Hide commands and units for which the logged-in user does not have the required role on any object

#30031

The database does not enforce uniqueness of reference values without duplicates: a unique index on the link tables, separate link tables for bags

#30041

Self-service accounts: invitations expire and record their acceptance, invitations can be accepted and managed in the React UI, localized mail templates

#30045

View layer slots: report a <slot-content> that reaches no <slot>, empty fallback of a <slot>, slot positions in split panels and the sidebar

#30053

Agent runs: bound the pool of parallel tool calls (AgentLoop.TOOL_WORKERS) and tell a user whose chat message waits for a free worker

#30054

Agent screenshots belong to their chat: deleted with it, shown as thumbnails in the chat

#30057

Administration of the React UI: scheduler (task list, run now, stop, activate/deactivate, cluster lock, run history) with one unified active/inactive switch per task

#30058

Administration of the React UI: memory trend chart (memory use over time incl. garbage collection) and the remaining monitoring views of the classic UI

#30059

Administration of the React UI: maintenance functions of the classic maintenance pages (database export/import/history compaction, SQL console, reload, search index, security storage, encryption)

#30061

React UI: recent changes of the user with revert of a commit (user menu)

#30062

Model editor of the React UI: instance browser and instance import of the selected type

#30063

Administration of the React UI: people (contacts with optional login) in one form

#30064

Administration of the React UI: login messages (messages shown at login, with their validity)

#30065

React Interface: Workflow Administration with BPMN Editor, Step Views from the View Designer, and Task Editing (BPE)

#30066

React Administration: View and Explain an Account's Effective Permissions on an Object (Role Inspector)

#30074

React Interface: Signing In with Passkeys (WebAuthn) — passwordless or as a second factor alongside TOTP; managing your own passkeys in the settings

#30075

Log View (React UI): Display the details of an entry (stack trace) in a dialog in a readable format

#30076

React Interface: Password change with confirmation message and validation before the identity prompt; protection against brute-force attacks on passwords and one-time codes

#30081

TL Views: Change Authenticator App (New TOTP Secret), Reset Two-Factor Authentication by Administrator

#30086

XML import: <update> of <unless-known> must restate the structure of the descendants it updates

#30088

View layer: document how a ChannelSource keeps a monitor running while its view is hidden and ends it with the window

#30095

OpenAPI Client: Parameter Value from a Secret Source

#30096

Rights System: Remove the configurable group mapper from the ElementAccessManager

#30097

Generate JavaScript libraries (umljs, ace, require.js, bpmn-js) in the Maven build instead of checking them in; update bpmn-js to version 18

#30098

Design system as an engine module: built by Maven instead of a checked-in copy with seals

#30104

Nightly preview builds of the engine with a fixed Maven version and a changelog instead of a rolling SNAPSHOT

#30108

pac4j Registration: The account mapping receives the identity provider's full identity (subject, claims) rather than just a login name

#30116

CommonMark: Displaying Tables (GFM) as HTML Tables

#30117

Refetch timeout does not specify the thread holding the commit/refetch lock

#30120

View layer: options of a string attribute cannot have display labels

#30122

Remove TLDoclet exceptions in ext/javadoc/javadoc-ignores

#30126

tl-chat: Chat actions run as React commands within the window rather than via a separate ChatServlet; the chat's proxy workaround is no longer used in favor of engine polling

#30131

View layer: show the fallback of AttributeWithFallbackStorage as placeholder of a form field

#30132

View layer: value listeners of the model (<value-listeners>) react to field changes in the React form, through a UI-independent listener contract

#30148

RecollateStringColumnsProcessor: Run ANALYZE after changing the collation (PostgreSQL)

#30149

Let a chat agent run page commands that the page of the chat allows, e.g. open a dialog

#30150

Ship a provider with recorded model answers for the agent tests of applications

#30151

React View Layer: Commands Without HandlerResult - View commands report errors as exceptions; React command handlers return their response as a msgbuf message

#30157

React Interface: Insert images into the HTML editor using Ctrl+V and drag-and-drop; resize using handles; never wider than the content

#30158

TL Chat: Let a chat agent pass the input of a page command it runs

#30160

Adding Albert to an application: shipped React chat page, how-to article and archetype defaults

#30163

SQLFactory: Generate column changes (modifyColumnName/Type/Mandatory) only with a complete column definition

#30167

Remove database defaults for columns: Add required columns using "create, populate, set as required"

#30172

Create accounts at an external login and assign their groups from a claim

#30179

An application generated with useSSO=true offers the generic OpenID login, switched on by the tl_pac_* environment variables, also on the development server

#30183

Self-Service: Open self-registration without an invitation

#30190

Remove the control application tl-factory from the engine, it is developed in its own repository

#30200

Agent Access and Recording: Commands operate on stable object IDs; technical keys are used only for recording and playback

#30215

View layer: a value channel whose value is remembered per user

#30223

Self-service mails: texts editable in the service configuration, language of an invitation selectable

#30225

Model: handlers declared on an attribute (change) or a type (creation, deletion), run within the commit

#30227

Design System UI pass, part 1b: tables with depth and visible column boundaries

#30250

TL Views: <start-job> / <job-status> - close button for an ended job, point of no return, partial result of a cancelled job, closing the dialog from the follow-up, follow-up sees the job's commits

#30253

tl-ai-core pulls the Playwright browsers (driver-bundle, 200 MB) into every application: browser tools move into the optional module tl-ai-browser

#30266

React Configuration Editor: Multi-line text (HTML, Markdown, code, multi-line) uses the entire line by default

#30267

Setting and removing administrative secrets; writable directory storage

#30268

System Environment: Display the source of configuration variables and set variables from the administration interface

#30276

AI Agents: Meaningful commit messages for changes made by an agent instead of "Agent edit"

minor

#30139

TL Views: Unit and Custom Number Format for Number Fields

#30188

TL Script: Read a binary value as text; Read a web app resource as a binary value

#30209

React Table: Show and Hide Columns Based on Channel Values (exists-if)

#30218

ModificationListener: no supported way to report to the user what a Modification changed, once the commit succeeded

defect

critical

#30146

Data migration is not protected against parallel execution when multiple cluster nodes are started simultaneously

#30196

Canceling a background job (TL Views <start-job>, Scheduler Task, JMS Consumer) interrupts the thread in the middle of a database operation and closes an embedded H2 database (MVStore panic, application becomes unusable)

#30224

TestFlowSvgPortable does not compile: FlowFactory.chart() has the parameter fixedView

#30301

React-Views: TestTreeTableRowEdit no longer compiles on master (ExpandArguments.ROW_INDEX replaced with ROW_ID)

major

#29847

Scripted test TestTransactionMonitor-01_RevertRevision fails when a scheduled task commits concurrently

#30043

TL Views: a value the server sets while the React rich-text editor has the focus is never applied; the composer keeps its text after sending

#30067

React Interface: Logging in in one browser tab immediately logs you out if a second tab for the application is open

#30077

TL Views: Wizard—re-entering an "Auto-Next" step without a separate navigation path leaves the user with no way to return to the workflow

#30078

React Interface: When setting up two-factor authentication, the key for manual entry is not displayed; only the QR code is shown.

#30079

Configured SecurityParents do not work with StructuredElementWrapper

#30082

React UI: Tabs in a dialog change the page's address and remain in the URL after the dialog is closed

#30090

Scheduler: A task that is due at startup is detached during shutdown while it is still running (NullPointerException in TaskImpl.getLog())

#30099

Scheduler tests: testStartTaskImmediately and testPeriodically have occasionally timed out since the scheduler was revised

#30103

tl-maven-plugin: If merging the view overlays fails during application build, the build remains green

#30106

tl-chat: Commands at the top of a data-list do not display their status (hidden, disabled with reason); clicking on a locked command produces no response

#30107

tl-maven-plugin: If generating a layout fails during application build, the build remains green

#30109

tl-chat: After logging in, the chat in the same browser window remains at "Log in to use the chat"

#30110

An aborted external login (OIDC) leaves a logged-in session without tokens

#30115

delete-module / delete-class do not clean up hasWrapperAttValue data

#30118

Passkeys per application can be used either for password-less login or as a second factor, but not both at the same time

#30119

AlterColumnProcessor does not update stored schema for abstract types

#30123

The scheduler incorrectly reports "There is no thread to remove" when a task fails to start

#30124

OpenAPI Server: The error message for a failed script is unreadable and is generated outside the interaction context

#30125

Agent access: /agent-api answers a request without a valid credential with a redirect to the login page instead of 401

#30127

An application at the server root keeps its session: the session cookie is sent to all its paths, and the React page announces its icon

#30128

A run without provider credentials ends as an authentication failure, and HTTP 402 is a billing failure for every provider

#30133

View layer: a field mandatory by dynamic visibility loses its required marker on a validation change

#30141

TypeIndexer: Java 21 as a supported source version; Index records

#30143

TL Chat: While an agent is performing multiple steps, their last message remains visible, and the status bar displays the current step instead of just showing "…" and "Working on …"

#30144

Chat sessions are invisible to their owner in an application with model-based security

#30147

React Interface: The "More Actions" menu does not close when the Escape key is pressed if all items are disabled

#30159

React UI Script Recorder: Recordings of tree, multiple-selection, and dropdown assertions can be replayed in a different session

#30164

TL Chat: A run that fails with an exception leaves no error in the conversation; it is gone after a reload

#30165

React: A command with ignoreStaleUi to a control that left the display still fails with 404 "Control not found"

#30166

TL Chat: runPageCommand gives the model the refusal of a page command in English instead of the user's language

#30173

OIDC login button is offered although the client has neither discovery URI nor provider metadata

#30176

The development server stops on /admin/stop from any address

#30177

The development server ignores X-Forwarded-* headers, so a relative callback URL behind a proxy names the wrong origin

#30184

DeleteTableProcessor does not remove " <alternative>" elements from the stored schema

#30201

Commit veto of the model constraints: a failed job (and its log) shows only "Cannot store, constraints violated." without the problems

#30216

Background processing fails with the error "Session revision invalid" if the window saves data while it is running (jobs, AI runs)

#30231

HtmlResKey: Also read the value as an attribute (add format)

#30244

tl-archetype-deploy: Declare the archetype integration test's dependency on tl-demo

#30247

Migrating to PostgreSQL with `autosave=NEVER`: Expected SQL errors abort the transaction

#30251

tl-ai-core, Anthropic provider: Claude 5.5 models missing from the bundled model catalog, and claude-haiku-5-5 sent a thinking budget (HTTP 400)

#30252

tl-ai-mcp-server: tools/list is not filtered by the caller's tool access, and /mcp/ with a trailing slash answers 404

#30254

tl-chat: A missing API key is reported as a rejected key ("The model rejected the access key")

#30255

tl-ai-core: agent turn rolled back with "Create on objects of type ... without permission" when a tool creates a part in an object created in the same turn - remove the duplicate pre-commit guard, check initial values in TL-Script

minor

#29961

React applications include modules from the classic interface (self-service, WYSIWYG editor, code editor, search, monitoring, AI administration)

#30044

TL Views: a sidebar separator stays visible when every item after it is hidden; the anonymous session sees the system separator with nothing below it

#30085

Default values <country value="..."/> and <time-zone value="..."/> (CountryDefaultProvider, TimeZoneDefaultProvider) cannot be used

#30089

The TL script functions `binarySize()` and `modelHealthReset()` return integers instead of doubles and log an error on every call

#30121

Compile-Errors in Eclipse in TL 8 alpha12

#30134

View layer: <composition-table> ignores the visibility (dynamic-visibility, hidden, read-only) of its composite attribute

#30140

TL Views: TestViewsLoad does not detect an undefined variable in a computed-column's value expression

#30142

CommaSeparatedStringSet: Non-deterministic serialization creates duplicate key attributes in the schema baseline

#30180

Flowchart (Gantt): Clicking on an element does not select anything if frozenRows > 0

#30181

Flowchart (Gantt): The chart is always displayed in full—no vertical scrolling, but it becomes unreadable when there are many rows

#30185

Flowchart (Gantt): LineDeco labels (e.g., “Today”) are cut off beneath fixed headers; edges without arrowheads

#30187

Exporting diagrams as SVG (reactFlowToSvg, flowToSvg) is not suitable for Batik or printing

#30191

Module shutdown stops no module at all if the class of its InContext job cannot be loaded (database stays open in the container's JVM)

#30197

React Table: Date cell in in-place edit accepts any partial keyboard input (e.g., year 0002)

#30198

React Views: Tree Table ({{{<tree-table>}}}) without row editing ({{{row-edit}}}) – not a replacement for the classic tree grid

#30199

React-Views: When a constraint is violated, the editable table displays only a red border, not an error message

#30204

React Table: When there are multiple row types, attribute columns are resolved only for the first type (label, sorting, filter)

#30205

React Table with Row Edit: A column based on a derived attribute is editable, but saving fails

#30206

React: A disabled `upload-command` does not display a reason; `null-input-disabled` always disables it

#30207

React: Button with an upload action does not open the file dialog

#30208

React Table: No cell display for external links (URL)

#30211

React: The built-in rules for `delete-object` and similar take precedence over the configured ones (making `visible-if` ineffective)

#30212

React: `null-input-disabled` only checks the first input

#30213

Binary attribute: Querying the content type loads the entire content; BLOB columns in object tables are loaded only for small values

#30214

View layer: tooltips drop the line breaks of their text (.tl-tooltip__body has white-space: normal)

#30226

React Table: Commands are unaware of the displayed rows or their order

#30237

React-UI: No display while a command is running on the server

#30256

React-UI: Set the color role of a row, cell, or view as a separate script (row-role, cell-role, role) instead of the ineffective status classes tl-status-*

#30257

React-UI: Commands from a window in a detached session generate two WARN lines per command

#30258

React-UI: The only way to display tabs for a single type is by using a type deviation in `access-control`, and that is logged as a WARN.

#30259

KB: Committable.rollback() is called after a successful database commit if the changes or caches fail to be published

#30265

TL Script: When `throw()` is used or a `TopLogicException` occurs, `try()` passes only the message text to the `catch` block instead of the `ResKey`.

#30269

ModelAccessRights.isAllowed(Person, ...): Checks whether another person is granted bypass or admin status in the current context

task

major

#30046

Remove the unused service ReactComponentRegistry

update

critical

#30111

Update dependency requirejs to ^2.3.7 [SECURITY]

enhancement

major

#30031

The database does not enforce uniqueness of reference values without duplicates: a unique index on the link tables, separate link tables for bags

KnowledgeBase Migration

Problem

By default, the links for all references with link storage (SetStorage, ListStorage with and without isBag()) are stored in the same tables, e.g., hasWrapperAttValue (LinkStorage.LinkStorageConfig#getTable(), selection in LinkStorage.defaultConfig()). Their indexes are not unique (com.top_logic.element/src/main/webapp/WEB-INF/kbase/ElementMeta.xml, hasWrapperAttValueBaseAssociation): {{{#!xml <mo_index name="source" unique="false"> source, metaAttribute </mo_index> <mo_index name="dest" unique="false"> dest, metaAttribute </mo_index> }}} The annotation <key-attributes attributes="source,dest,metaAttribute"/> is purely metadata and is not enforced by the database.

Consequently, the database cannot prevent a reference without duplicates (TLStructuredTypePart#isBag() = false) from containing the same value multiple times. Uniqueness depends solely on the storage code. Prior to #29975, ListStorage did not enforce this: calling tAdd() on a value already present created a second link ([a, b, a]). Such data may exist in existing databases. Accesses based on the assumption of uniqueness—such as the target index behind TLObject#tContains() (a BidiMap target → link)—return incorrect results for such data.

Solution

In applications, bags are the rare exception; almost all references are unique. Therefore, the existing link tables become unique, and bags get their own tables.

Unique Index, Dedicated Tables for Bags

The source index of hasWrapperAttValueBaseAssociation, hasHistoricValue, and hasMixedValue is unique on (source, metaAttribute, dest). Thus, it applies to all link tables derived from them (including the unversioned ones, the composition tables, and the applications’ link tables). For historical/mixed references, it covers the entire target, including the target revision (DEST_REV), since “X currently” and “X in revision 5” are different values. Branch and REV_MAX automatically supplement the index strategy (DefaultIndexColumnsStrategy) so that the index applies only to the current rows of a branch.
Bag tables with the previous, non-unique indexes: hasWrapperAttValueBag, hasWrapperDeleteRefererBag, hasWrapperVetoBag, hasHistoricValueBag, hasMixedValueBag, each also ...Unversioned; shared abstract base ` hasWrapperAttValueBagBaseAssociation`. Composite references do not require one: ` uniqueParent ` already prevents a part from being included twice.
LinkStorage.defaultConfig() additionally selects the table based on isBag(). If a bag reference is explicitly stored (<storage table="..."/>) in a table with a unique index, LinkStorage.init() reports an error in the log.
The knowledge base writes a commit in the order of deletions, modifications, and insertions across all tables (DefaultDBContext.commitTransaction()); replication, import, and ChangeSet all run under the same commit. Therefore, deleting and reinserting the same value never temporarily violates the index. A link created twice without being stored causes the commit to fail with a KnowledgeBaseException.

Changing the Storage of a Reference

A reference’s table depends on ` bag`, `deletionPolicy `, and `historyType`, while the storage type depends on ` multiple ` and `ordered`.

In the running system (automatic model upgrade at startup from *.model.xml, module import in the interface, AI model editor): If any of these properties of a persistent reference changes, its storage is recalculated within the same transaction (PersistentEnd, ReferenceStorageChange), and the existing values are read from the old storage and written to the new one. Parts of a composition are preserved in the process. When switching to a reference without duplicates, the first occurrence of a value is retained; when switching from historical to current, a value is mapped to its current object; when switching to a single-value reference, the first value is retained (with a warning in the log). A change to `composite` is not adjusted. Previously, the cached storage of a reference was only recalculated when its annotations changed, so such changes took effect only after a restart, and the values were then no longer visible.
In a migration (ApplyModelPatch with migration processors): A change to a reference’s ` bag ` or ` deletionPolicy ` with default storage generates `<move-links>` in the new table; when switching to a reference without duplicates, it generates `<remove-duplicates>` beforehand. A warning is issued if an explicit table is used, if there are overrides, or if the ` historyType ` is changed simultaneously.

Migration Processors

<align-indexes> (new): aligns the indexes of the selected tables with the application schema (creates missing indexes, recreates modified ones, and deletes obsolete ones). Automatic schema migration does not create indexes for existing tables.
Table selection: tables, subtypes-of, exclude for <align-indexes> and <remove-duplicates>.
<remove-duplicates>: Duplicates are detected per attribute (reference column) and, if applicable, per target revision (previously, identical values of different attributes of the same object were grouped together); Fixed an error in the overlap check following a deleted row; `skip-bags` option; candidates are pre-determined using `GROUP BY ... HAVING COUNT(DISTINCT IDENTIFIER) > 1 `, so that tables without duplicates are not processed.
<move-links-by-bag> (new): moves the links of all bag references (or all non-bag references) between configured table pairs.
<move-links> invalidates the revision indexes of the involved tables.

Migration

The Ticket_30031_unique_link_tables (tl-element) migration runs automatically:

The bag tables are created.
The links of all Bag references are moved from the standard tables to the Bag tables.
In the standard tables (including the application tables derived from them), duplicates (same source, metaAttribute, and dest in overlapping revisions) are cleaned up across the entire history: The first link remains; the others are deleted or their validity start date is shortened. The migration logs every cleaned-up link.
The “source” index of these tables is replaced by the unique index.

Applications with their own link tables derived from ` hasWrapperAttValueBaseAssociation ` inherit the unique index. If Bag references are stored in them (explicitly via `<storage table="..."/>`), their links are not cleaned up; the migration reports them as errors, and creating the index fails if duplicates are found. Such references must be reconfigured to a bag table (derived from ` hasWrapperAttValueBagBaseAssociation`) and their links moved using `<move-links> `.

Java API: ListStorage.listConfig(...), SetStorage.setConfig(...), and ImageGalleryStorage.imageGalleryConfig(...) have an additional parameter, a boolean `bag ` (previous behavior: ` false`; `true` for a bag reference).

Test

TestLinkTableUniqueness: A duplicate link created outside the storage process fails on commit, but succeeds in the bag table; table selection; deletion and reinsertion in a single commit; historical reference to the same object in two revisions.
TestUniqueLinkTablesMigration: the migration script on a database with the old layout, containing bag links, duplicates (fully and partially overlapping), and historical values.
TestAlignIndexesProcessor, TestRemoveDuplicatesInLinkTables, TestMoveLinksProcessor: the migration processors.
TestModelPatch: Model patches that modify `bag ` or ` deletionPolicy ` and move the links.
TestReferenceStorageChange: Modification of `bag`, `deletionPolicy`, `historyType`, and `ordered` on a reference with values in the running system, composition, and rollback.
TestListStorage and TestSetStorage run unchanged.