# Migrating a TopLogic application from 7.10.0 to 8.0.0-alpha1

Audience: an AI coding agent (or developer) upgrading a TopLogic application project.
Sources: release notes https://dev.top-logic.com/tl/content/releases/TL_8.0.0-alpha1/ and the
TopLogic Trac tickets flagged `RequiresMigration` for milestone `TL_8.0.0-alpha1` (ticket numbers
are given as `#NNNNN`; the per-ticket page is `https://dev.top-logic.com/tl/content/releases/TL_8.0.0-alpha1/NNNNN/`).

## How to use this document

1. Work through the sections in order. Section 1 must be done first (the project does not build
   otherwise); the rest can be done in any order.
2. Every step has a **Detect** line: run the search in the application's source tree. If it finds
   nothing, the step does not apply — skip it. Do not invent changes for steps that do not apply.
3. Many changes are reported as **configuration errors at application start or when a layout is
   loaded** (not at compile time), e.g. unknown configuration properties. After migrating, start the
   application, log in, and open every view; read the log for `ConfigurationException` /
   configuration errors.
4. Several automatic data migrations run at the first start on the new version (named in the steps).
   **Back up the database before the first start.** Some steps below must be done *before* that
   start, because the automatic migration drops data.
5. Ticket descriptions on Trac are partly German; this document translates the instructions.

Search commands below assume the project root as working directory and GNU grep.

---

## 1. Build and platform

### 1.1 Version bump
Set the TopLogic version to `8.0.0-alpha1` (parent `tl-parent-app` / `tl-parent-core` version, or
the `tl.version` property).

### 1.2 Custom resource keys in `I18NConstants` (#29111) — Java
`ResKey` fields in `I18NConstants` classes that get a custom resource key through `legacyKey()`,
`legacyKey1()`, `legacyKey2()`, … must use the new annotation `@CustomKey`
(`com.top_logic.basic.i18n.CustomKey`) instead. The field must **not** be initialized —
`initConstants()` assigns it from the annotation. The same applies to fields initialized with
`ResKey.internalCreate("...")`.

Before:
```java
public static ResKey MY_KEY = legacyKey("my.custom.key");
public static ResKey1 MY_KEY_1 = legacyKey1("my.custom.key");
```
After:
```java
@CustomKey("my.custom.key")
public static ResKey MY_KEY;

@CustomKey("my.custom.key")
public static ResKey1 MY_KEY_1;
```
Reason: only then do the runtime (`I18NConstantsBase`) and the build-time doclet (`TLDoclet`) use
the same key, so `@en` JavaDoc translations are written to `messages_en.properties` under the custom
key. Note: in 8.0.0-alpha1 the `legacyKey*()` methods still exist (deprecated), so unconverted code
still compiles — but `@en` JavaDoc translations of such fields end up under the wrong key.

Optional tool (only if asked): `com.top_logic.layout.tools.cleanup.MigrateResourcesToJavaDoc` moves
English translations from `WEB-INF/conf/resources/*_en.properties` into `@en` JavaDoc tags of the
matching `ResKey` fields (respecting `@CustomKey`) and removes the moved entries from the properties
files.

Detect:
```
grep -rn -e 'legacyKey[0-9]*(' -e 'ResKey.internalCreate(' --include=*.java .
```

---

## 2. Server configuration and login (`*.config.xml`, `web.xml`, layouts)

### 2.1 Login page replaced by an in-app dialog (#29092) — mandatory if referenced
The new module `tl-security-selfservice` (password reset, second-factor reset, invitations) is an
optional dependency and needs no action. The core login, however, was rewritten for a two-step
flow (multi-factor authentication):

`LoginPageServlet` (mapping `/servlet/login`), `login.jsp`, `login.banner.inc`, `doOnload.inc`,
`jsp/main/loginError.jsp`, `jsp/util/administration/changePwd.jsp` and `script/tl/loginError.js` are
deleted. A request without a session now gets an anonymous session, and the layout offers the
`LoginViewDialog` through the new `<login-hooks>` of `MainLayout$GlobalConfig` (framework default:
`OpenLoginDialogHook`).

- Delete application overlays of the removed JSPs and includes, and `web.xml` entries for
  `LoginPageServlet`. Bookmarks and monitoring probes pointing to `/login.jsp` or `/servlet/login`
  must use `/servlet/LayoutServlet`.
- Remove `login`, `loginRetryPage`, `loginErrorPage` and `changePassword` from an
  `ApplicationPages$Config` override; these properties no longer exist (unknown property = startup
  error). `loginPage`, `logoutPage` and `loginRetrySSO` default to `/servlet/LayoutServlet`.
- Login-messages addon: `LoginMessagesMainLayout` and its `GlobalConfig` are replaced by the login
  hook `LoginMessagesHook`. Remove a `MAIN_LAYOUT_CLASS` theme setting naming the old class and the
  `LoginMessagesMainLayout$GlobalConfig` block; the addon's `loginMessagesConf.config.xml` registers
  the hook itself:
  ```xml
  <config config:interface="com.top_logic.mig.html.layout.MainLayout$GlobalConfig">
      <login-hooks>
          <login-hook class="com.top_logic.addons.loginmessages.layout.LoginMessagesHook" showLoginMessages="true"/>
      </login-hooks>
  </config>
  ```
- Resource keys: `tl.logout` is now
  `class.com.top_logic.layout.component.configuration.I18NConstants.LOGOUT`;
  `layouts.admin.persons.changePassword.*` moved to
  `class.com.top_logic.knowledge.gui.layout.person.I18NConstants.CHANGE_PASSWORD_FORM.*`;
  `MAX_USERS_EXCEEDED` is gone. Update own references to and own translations of these keys.
- A session may now belong to the `anonymous` account (`PersonManager.getAnonymous()`,
  `TLContext.isAnonymous()`), and `Person.all()` contains that account. Check own code that iterates
  all persons or assumes a logged-in user.
- Application copies of engine layout files must be adapted to the anonymous session:
  - In a copy of `toolrow/homeButton.xml`, wrap the user menu (`PopupViewConfiguration`) in a
    `ConditionalViewConfiguration` with `condition="com.top_logic.util.IsAnonymousUser"` and
    `invert="true"`; otherwise anonymous users see the user menu.
  - In a copy of `themes/core/templates/navigationComponent.xml`, add
    `login-image="theme:IMAGE_BUTTONS_LOGIN"` to the `logoutView` (icon of the login action while no
    user is logged in). The `coreSidebar` variant no longer contains a `logoutView`.

Detect:
```
grep -rn -e 'LoginPageServlet' -e 'servlet/login' -e 'login.jsp' -e 'login.banner.inc' -e 'doOnload.inc' \
  -e 'loginError' -e 'changePwd.jsp' -e 'ApplicationPages' -e 'loginRetryPage' -e 'loginErrorPage' \
  -e '<changePassword>' -e 'LoginMessagesMainLayout' -e 'MAIN_LAYOUT_CLASS' -e 'tl\.logout' \
  -e 'layouts.admin.persons.changePassword' -e 'MAX_USERS_EXCEEDED' .
find . \( -path '*WEB-INF/layouts/toolrow/homeButton.xml' -o -path '*templates/navigationComponent.xml' \) -not -path '*/target/*'
```

### 2.2 Authentication devices, module list, account form (#29092) — mandatory if configured
- `mfa-requirement` is mandatory on `DBAuthenticationAccessDevice.Config` and
  `LDAPAuthenticationAccessDevice.Config`. The core sets `optional` for `dbSecurity` and
  `%LDAP_MFA_REQUIREMENT%` (default `optional`) in `ldapConf.config.xml`. An application declaring its
  own `<security-device>` of one of these classes (or a copied `ldapConf`) must add
  `mfa-requirement="optional|required|disabled"`.
- Applications that maintain their own `<modules>` list (ModuleSystem configuration) must enable
  `com.top_logic.base.accesscontrol.LoginFailuresModule$Module`.
- Automatic data migration `Ticket_29092_multi_factor_authentication` (`tl-element`) adds
  `Person#mfaSecret` and `Person#mfaRequirement`. The `tl-layout-formeditor` migration of the same
  name **replaces** the form definition annotation of `tl.accounts:Person`: an in-app customisation
  of the account form is lost and must be re-applied after the update. If the account form was
  customised in-app, record it before the first start and tell the user.

Detect:
```
grep -rn -e 'DBAuthenticationAccessDevice' -e 'LDAPAuthenticationAccessDevice' -e '<security-device' \
  -e 'ldapConf' -e '<modules>' .
```

### 2.3 `SessionService` / `UserMonitor` options (#29106) — mandatory if configured
- The options `excludeUIDs` and `onlyOneSession` of `com.top_logic.base.accesscontrol.SessionService`
  are removed → delete them (left over → configuration error at startup).
- If `excludeUIDs` of `com.top_logic.knowledge.monitor.UserMonitor` is configured, configure it on the
  `SessionService` instead:
  ```xml
  <config service-class="com.top_logic.base.accesscontrol.SessionService">
    <instance>
      <listeners>
        <listener name="storeEvent">
          <impl class="com.top_logic.knowledge.monitor.StoreUserEventListener" excludeUIDs="..."/>
        </listener>
      </listeners>
    </instance>
  </config>
  ```
- Java: `UserMonitor.login(...)`, `logout(...)`, `notifyUserEvent(...)`, `findUserSession(...)` and
  `findSessionOnServer(...)` are removed.

Detect:
```
grep -rn -e 'onlyOneSession' -e 'excludeUIDs' -e 'UserMonitor' .
```

---

## 3. Security / access checks

### 3.1 Role rules navigate only over the model; `hasRole` / `hasGlobalRole` (#28710) — mandatory

#### 3.1.1 Role rule configuration
Role rules of the `AccessManager` (e.g. `InitialRoleRule.xml`) must be converted. Security works only
over the model, i.e. a rule necessarily needs an `attribute` and possibly a `meta-element`.
- `<step association="…">`: instead of navigating over the association table in general, find the
  model attribute whose values are stored in that table and navigate over it:
  ```xml
  <step association="hasStructureChild" inverse="false" />
  ```
  becomes
  ```xml
  <step attribute="children" inverse="false" />
  ```
- `<rule meta-object="…">`: instead of the table type, give the TL model type:
  ```xml
  <rule meta-object="StoredQuery" ...
  ```
  becomes
  ```xml
  <rule meta-element="tl.search:StoredQuery" ...
  ```
- Likewise, the attribute `source-meta-object` is removed: instead of the table type, give the TL
  model type with `source-meta-element`.
- The extension points `ExternalRoleProvider` (`<role-provider>`) and `SecurityStorageCommitObserver`
  (`<commit-observer>`) of the `ElementAccessManager` and the class `FallbackAccessManager` are removed
  without replacement; express programmatic role assignment as a configured role rule.

Detect:
```
grep -rn -e '<step association=' -e 'meta-object=' -e '<role-provider' -e '<commit-observer' \
  -e 'ExternalRoleProvider' -e 'SecurityStorageCommitObserver' -e 'FallbackAccessManager' .
```

#### 3.1.2 Data migration — check before the first start
The migrations `Ticket_28710_Removed_legacy_types`, `Ticket_28710_Update_application_types` and
`Ticket_28710_RoleAssignment_TLSearch` (module `tl-element`) run automatically at the first start.
- **Directly assigned global roles are lost.** The table `hasGlobalRole` is dropped; its data is not
  transferred. Only applications that assigned roles programmatically via `Person#addGlobalRole(...)`
  are affected (the engine had no UI for it). Map such assignments to a group membership plus role
  rule **before** the update.
- **The application must delete its own legacy table types.** The modules `tl.legacy.tabletypes` and
  `tl.tables` no longer exist; the engine migration deletes only the engine's types
  (`tl.legacy.tabletypes:PersonTable`, `tl.tables:PersonTableInterface`, …). If the application's
  database still contains own types of these modules (from the update TL 6 → TL 7 or from
  `TableInterface`-based models), the application deletes them in its own migration and retypes
  attributes pointing to such types (`change-part-type ... target="tl.accounts:Person"`). Template in
  the engine: `com.top_logic.demo/src/main/webapp/WEB-INF/kbase/migration/tl-demo/Ticket_28710_Removed_legacy_types.migration.xml`
  (it depends on `Ticket_28710_Removed_legacy_types` of module `tl-element` and deletes the types with
  `<delete-class name="tl.legacy.tabletypes:…" skip-model-baseline-change="true"/>`).
  Remove configuration references such as `IndexedObjectNaming` `<type name="tl.legacy.tabletypes:...">`
  or `defaultFor="tl.legacy.tabletypes:..."`.
- `hasRole` is no longer a `KnowledgeAssociation` but an `MOKnowledgeObject` of model type
  `tl.accounts:RoleAssignment` (references `source`, `dest`, `owner`); `hasGlobalRole` is gone. Adapt
  own `*Meta.xml` files referencing these tables.

Detect:
```
grep -rn -e 'addGlobalRole' -e 'hasGlobalRole' -e 'hasRole' -e 'tl.legacy.tabletypes' -e 'tl\.tables:' \
  -e 'TableInterface' -e 'IndexedObjectNaming' .
```
Own legacy types that exist only in the database are not found by grep: check the types of the
existing database (model editor or table `META_ELEMENT`) for the modules `tl.legacy.tabletypes` and
`tl.tables` and report them to the user.

#### 3.1.3 Java API
- `BoundedRole.HAS_ROLE_ASSOCIATION` is removed. Role assignments are written with
  `BoundedRole.assignRole(context, person|group, role)` and read with
  `getLocalAndGlobalRoles(context, person)` and the queries `roleAssignmentsForContext(...)`,
  `roleAssignmentsForRole(...)` (they return a `CloseableIterator` that must be closed). Code
  navigating `getOutgoingAssociations(HAS_ROLE_ASSOCIATION)` must be converted.
  `Person#getGlobalRoles()`, `addGlobalRole(...)` and `removeGlobalRole(...)` are removed.
- `LegacyFlexWrapper` and `StoredFlexWrapper` are removed: wrapper classes extending them (like
  `StoredQuery` and `StoredReport` in the engine) extend
  `com.top_logic.element.meta.kbbased.AttributedWrapper`;
  `MapBasedPersistancySupport.getObjects(KnowledgeItem)` and `setObjects(Collection, KnowledgeItem)`
  replace the `FlexData` variants.
- `AccessManager#handleSecurityUpdate(KnowledgeBase, Map, Map, Map, CommitHandler)` is now
  `handleSecurityUpdate(TLObjectChangeSet, CommitHandler)`; likewise
  `LogHandler#logSecurityUpdate(TLObjectChangeSet, Map, Set)`. **An override with the old signature is
  silently no longer called if it lacks `@Override`** — check every subclass.
- The constructors of `RoleRule` taking a `MetaObject` and `PathElement(String associationType, boolean)`
  are removed; rules navigate only over attributes. `ElementAccessManager#getRules()` returns
  `Map<TLClass, Collection<RoleProvider>>`.

Detect:
```
grep -rn -e 'HAS_ROLE_ASSOCIATION' -e 'getGlobalRoles' -e 'removeGlobalRole' -e 'LegacyFlexWrapper' \
  -e 'StoredFlexWrapper' -e 'FlexData' -e 'MapBasedPersistancySupport' -e 'handleSecurityUpdate' \
  -e 'logSecurityUpdate' -e 'new RoleRule(' -e 'new PathElement(' -e 'getRules(' --include=*.java .
```

---

## 5. Classic UI, model and other Java APIs

### 5.1 Authentication and account Java API (#29092) — Java
- `Login.login(userName, request, response)` → `Login.checkUserPassword(userName, char[] password, request, response)`;
  `Login.login(request, response, credentials)` → `Login.checkLoginCredentials(credentials, request, response)`.
  Both only check the credentials; the caller creates the session with
  `SessionService.getInstance().loginUser(request, response, person)`.
  `MaxUsersExceededException` is replaced by `LoginHookFailedException`.
- `LoginCredentials` is no longer `AutoCloseable`: call `clearPassword()` in a `finally` block
  (instead of try-with-resources).
- `ExternalAuthenticationServlet` (base class of SSO servlets) extends `NoContextServlet` instead of
  `LoginPageServlet`: `forwardPage(...)` → `forwardToPage(...)`, `forwardToStartPage(...)` →
  `redirectToStartPage(...)`; `checkRequest(...)` and `forwardToTarget(...)` can no longer be
  overridden; `forwardToSSOLoginFailed(...)` declares `IOException, ServletException`. Subclasses that
  only implement `retrieveLoginCredentials(...)` need no change.
  `LoginPageServlet.appendCustomParameters(...)` and `PARAM_START_PAGE` moved to `AbstractTopLogicServlet`.
- `forwardPage(...)` is also removed from `TopLogicServlet` and `AJAXServlet`; `TopLogicServlet` now
  extends `AbstractTopLogicServlet`. The classes `com.top_logic.base.accesscontrol.SessionAttributeKeys`
  and `com.top_logic.util.DispatchException` are deleted.
- `TLPersonManager` is deleted: configure and extend `com.top_logic.knowledge.wrap.person.PersonManager`
  (as `ContactPersonManager` does; its `Config` is no longer generic). `PersonManager` extends
  `KBBasedManagedClass`, so subclasses call `super(context, config)`. Replacement for
  `getAllAliveFullPersons()`: `Person.all().stream().filter(Person.FULL_USER_FILTER)`.
- `Person.create(kb, name, String deviceId)` → `Person.create(kb, name, AuthenticationDevice device)`.
- `AuthenticationDevice.getMFARequirement()` is a new abstract method: own devices implement it (e.g.
  returning `MfaRequirement.DISABLED`).
- `FormMember.setLabel(ResKey)`, `setTooltip(ResKey)` and `setTooltipCaption(ResKey)` were added next
  to the `String` variants; a call with a literal `null` is ambiguous and must be cast
  (`setLabel((String) null)`).
- `ListStorage.listConfig(...)`, `SetStorage.setConfig(...)` and
  `SingletonLinkStorage.singletonLinkConfig(...)` take a trailing `boolean unversioned` (pass `false`
  for the previous behaviour).
- The deprecated `MetaElementUtil.getAllInstancesOf(TLClass)` and `getAllDirectInstancesOf(TLClass)`
  are removed; use the overloads with the expected class (`getAllInstancesOf(type, Wrapper.class)`).
- `Base32` is renamed to `EncodeTypable` (same static method names).

Detect:
```
grep -rn -e 'Login\.' -e 'MaxUsersExceededException' -e 'LoginCredentials' -e 'ExternalAuthenticationServlet' \
  -e 'forwardPage(' -e 'forwardToStartPage' -e 'LoginPageServlet' -e 'SessionAttributeKeys' \
  -e 'DispatchException' -e 'TLPersonManager' -e 'getAllAliveFullPersons' -e 'getAllAliveRestrictedPersons' \
  -e 'Person.create(' -e 'AuthenticationDevice' -e 'setLabel(null)' -e 'setTooltip(null)' \
  -e 'setTooltipCaption(null)' -e 'listConfig(' -e 'setConfig(' -e 'singletonLinkConfig(' \
  -e 'getAllInstancesOf(' -e 'getAllDirectInstancesOf(' -e 'Base32' .
```

### 5.2 Table rows after changes in other sessions: `supportsListElement` / `supportsElement` (#29080)
Tables, tree tables and grids no longer add a changed or created object as a row just because the
list builder "supports" it. If object types are configured on the table, an `instanceOf` check is
done automatically, and only matching objects are passed to `ListModelBuilder#supportsListElement`.
The new constant `com.top_logic.mig.html.ElementUpdate.UNKNOWN` makes the table rebuild its row list
via the model builder. `ListModelByExpression` returns `UNKNOWN` if no `supportsElement` script is set
(new in-app tables/grids are created without `supportsElement`).

Steps:
- Java list builders: if it is unclear whether an object must be added to or removed from the list,
  `ListModelBuilder#supportsListElement` must return `ElementUpdate.UNKNOWN`.
- In-app tables and grids must be checked: if the table/grid shows **all** objects of a type, the
  `supportsElement` script must return `true`; if the table/grid should rebuild its list when an
  element of the supported type is created, leave `supportsElement` empty. The script may return
  `"unknown"` if it cannot decide whether an element must be added or removed (corresponds to
  `ElementUpdate.UNKNOWN`).
- Subclasses of `SelectableBuilderComponent` that override `receiveModelCreatedEvent(Object, Object)`
  override `handleTLObjectCreations(Stream<? extends TLObject>)` instead.
- The classes `com.top_logic.element.layout.security.GlobalDomain` and `AttSecMetaElementLabelProvider`
  are removed.

Detect:
```
grep -rn -e 'supportsListElement' -e 'supportsElement' -e 'receiveModelCreatedEvent' -e 'GlobalDomain' \
  -e 'AttSecMetaElementLabelProvider' .
```
In-app layouts stored in the database are not found by grep: after the start, list the in-app tables
and grids with a `supportsElement` script (layout editor) and report them to the user for review.

### 5.3 Abstract attributes are not accessible any more (#29073) — model
Reading an abstract attribute used to return `null` (e.g. on transient objects). Now every access to
an abstract attribute throws an error. This affects domain models in which an inherited abstract
attribute (such as `StructuredElement#parent`) was not redeclared properly (usually in the root
type). A structure based on `StructuredElement` must be built like this:
```
// Top-level type of the structure - each type must extend it.
MyNode extends StructuredElement {
    parent: MyContainer
}

// Abstract child type in the structure - each type with substructure must extend it.
abstract MyContainer extends MyNode, StructuredElementContainer {
}

// Concrete type with children.
NodeA extends MyContainer {
    children: ...
}

NodeB extends MyContainer {
    children: ...
}

// Concrete type without children.
NodeC extends MyNode {
}
```

Detect:
```
grep -rln -e 'StructuredElement' -e 'abstract="true"' --include=*.model.xml .
```
For each hit, check that every abstract attribute inherited by a concrete class is redeclared.
Violations otherwise appear at runtime when the attribute is accessed (forms, tables, scripts).

### 5.4 Workflows deployed in `WEB-INF/workflow` (#29037) — tl-bpe only
Workflows in `src/main/webapp/WEB-INF/workflow` are now re-imported automatically when the content
of their file changes (a hash per file is stored). At the first start on the new version, only the
hashes of the already imported workflows are recorded — **the workflows are not updated**. If an
update is wanted on this occasion, add a migration that replaces the stored value `loaded` by a hash
that does not match the current file, using the migration processor
```xml
<update-tl-properties property="initial-workflow.<resource>" value="<wf-name>:XXX" />
```
- `<resource>`: name of the resource below `/WEB-INF/workflow/` that defines the initial workflow,
  e.g. `Ticketsystem.bpml.xml` for `/WEB-INF/workflow/Ticketsystem.bpml.xml`.
- `<wf-name>`: name of the workflow as given in the file; if none is given, the file name up to the
  first `.` (e.g. `Ticketsystem`).
- `XXX` stands for the "old hash". Since the file's hash differs from `XXX`, the workflow is updated.

Java: `com.top_logic.bpe.modeler.upload.Updater` is now `com.top_logic.bpe.util.Updater` (module `tl-bpe`).

Detect:
```
find . -path '*WEB-INF/workflow/*' -not -path '*/target/*'
grep -rn 'bpe.modeler.upload.Updater' --include=*.java .
```

---

## 7. Further notes (no action required)

- #29092: a request without a session gets an anonymous session; the login is an in-app dialog.
  Accounts can set up a TOTP second factor; whether it is enforced is controlled per account by
  `mfaRequirement` (default from the `mfa-requirement` of the authentication device). An administrator
  can remove the secret of an account.
- #29092: in 8.0.0-alpha1 the service configuration of the optional module `tl-security-selfservice`
  (`InvitationModule`) is located in its `WEB-INF/autoconf` folder; an application that wants to
  change it has to override that configuration completely (from a Trac comment on #29092).
- #29037: existing workflows are not re-imported at the first start (an earlier version of the ticket
  said in-app modified workflows had to be saved because of a re-import; that no longer applies).

---

## 8. Verification checklist

1. `mvn clean install` of the application (with tests if possible).
2. Back up the database; start the application against a copy of the production data.
3. Read the startup log: no configuration errors (removed `ApplicationPages` properties,
   `onlyOneSession`/`excludeUIDs`, missing `mfa-requirement`, `TLPersonManager`, role-rule attributes
   `association`/`meta-object`, `role-provider`/`commit-observer`), and the automatic migrations
   `Ticket_28710_*` and `Ticket_29092_*` ran successfully.
4. Open the application without a session: the login dialog appears; log in as a normal user and as
   administrator; anonymous users must not see the user menu.
5. Open every view; check tables/trees/grids showing objects created or changed in another session
   (#29080) and forms of structure types (#29073).
6. Check access: users get the roles defined by the role rules (no lost global roles).
7. Report to the user every step that was skipped, needed a decision, or touches data (global roles,
   own legacy table types, account form customisation, workflow re-import).
