enhancement
major
minor
major
minor
major
#29630
TL Views: a chunked, transactional job body for <start-job> - scripted init / elements / steps / finish, one transaction per chunk, per-item retry and skip, phases and progress derived from the steps
Problem
<start-job> (#29600) runs its body on a worker thread outside any transaction; "persisting what the job produced is the business of the actions after it". That holds for a computation, not for an import: the Trac import of the issue tracker tl-dev creates or updates ~15 000 tickets with comments and attachments over XML-RPC, and its classic-UI predecessor (ChunkedProgressCommand in the app) committed every chunk of 200 items in a transaction of its own, with per-item retry and skip, so that memory stays bounded, a failure late in the run keeps what was committed, and a re-run resumes idempotently.
Ported to the React UI, a <start-job function="…"> body can neither create persistent objects (new() outside a transaction fails) nor commit chunks: TL-Script has no function that runs a block in a transaction (modelSearchConf.config.xml offers try, log, revisionForCommit, nothing that begins or commits one), and ScriptJobBody opens none. A single <with-transaction> after the job would hold 15 000 tickets in one commit and lose everything on a fault.
The app therefore carries a generic Java JobBody of its own (<body class="…"> inside <start-job>) configured with init / elements / chunk-size / steps / finish scripts, which opens a transaction per chunk, retries and skips per item, reports phase, progress and message through the JobMonitor, and mirrors the log(...) lines of the scripts onto the monitor's message - the logic of the classic ChunkedProgressCommand re-hosted as a job body.
Request
One of the two in the engine, so an application declares a transactional long job without Java:
- a chunked, transactional job body beside ScriptJobBody: <body class="…ChunkedScriptJobBody" chunk-size="200"><init>…</init><elements>…</elements><step>…</step><finish>…</finish></body>, one transaction per chunk, per-item retry and skip reported as messages, phases derived from the steps; or
- a TL-Script function running a block in a transaction, transaction(fn) (commit on return, rollback on failure), which a script job body then uses per chunk, together with try for the per-item skip.
The first fits <start-job>'s phase display directly; the second is the smaller primitive.
Solution
The first option: the chunked, transactional job body is part of the engine, in the package of <start-job> (com.top_logic.layout.view.job), as two classes:
- ChunkedJobBody - the transactional frame with Java hooks (init, elements, step, finish, and hasInit / hasFinish / stepCount telling the frame which of them exist): it walks the work items in chunks of a configured size, commits every chunk in a transaction of its own, retries a failed chunk item by item and skips the items that cannot be processed (each one named in a message and counted), announces the phases of the job (preparation, one per pass, completion), reports the progress of a pass in chunks, and answers a cancel between two chunks. A failure of the preparation or the completion ends the job with its own error. A Java-written import builds on this class directly.
- ChunkedScriptJobBody - the frame with its hooks bound to TL-Script:
<start-job job="importJob" cancelable="true">
<body class="com.top_logic.layout.view.job.ChunkedScriptJobBody" chunk-size="200">
<init-label><en>Project structure</en></init-label>
<init>job -> request -> { … ; $project }</init>
<elements>job -> project -> $project.get(`…#tickets`)</elements>
<steps>
<step>
<label><en>Tickets</en></label>
<expr>job -> chunk -> project -> $chunk.foreach(t -> …)</expr>
</step>
</steps>
<finish-label><en>Index</en></finish-label>
<finish>job -> project -> { … ; $project }</finish>
</body>
</start-job>
Shared state. init runs once, in a transaction of its own, and what it returns is the state every later script receives: elements yields the work items from it, every step is called with the chunk and the state, finish with the state. The state is whatever the passes need to reach - the container the items are created in, a map of several objects, or a transient object where the passes have to change it. Without an init, the first argument the job was started with is the state.
Reporting. Every script is called with the monitor of the job as its first argument, exactly as the function of a <start-job> is, and narrates with $job.jobMessage(…); the passes are the phases the display shows, with init-label, the labels of the steps and finish-label as their texts (defaults: Preparation, Pass n, Completion), so the <start-job> declares no <phases> for such a body. A report from within a chunk answers a cancel at once: the chunk in progress is rolled back, the chunks committed before it stay. The log(...) mirror of the app body is not carried over.
Result. The job ends with what finish returns; without a finish, with the text naming how many items were processed and how many skipped. That text is set as the last message of the job in either case, so the display shows the counts.
elements is evaluated once, read-only and outside any transaction. The TL-Script primitive transaction(fn) is not part of this ticket.
Demo and documentation. The long-job demo of tl-demo-react (demo/long-job-demo.view.xml) has a chunked import creating 500 tickets in one pass (every 97th number fails and is skipped) and closing every second one in a next pass, and the chunked removal of what it created; docs/faq/react-view-layer.md describes the body in the section on long-running jobs.