Problem

The current server-side OpenAPI implementation does not allow defining an endpoint that returns binary data. Any operation result is either JSON-serialized or coerced to a String before being written to the HTTP response — there is no path for streaming raw bytes (e.g. application/octet-stream, image/*, application/pdf) to the caller.

Root Cause

In com.top_logic.service.openapi.server.impl.ServiceMethodByExpression#handleRequest, the response body is always written via the character Writer:

{{{#!java resp.getWriter().write(content); // content is always a String }}}

The value of content is built in exactly two ways:

• Default (no Response wrapper): content = JSON.toString(result) — any return value is JSON-serialized.
• With Response wrapper (HTTPResponse TL-Script function): if the MIME type is application/json, then content = JSON.toString(response.getResult()); otherwise content = String.valueOf(response.getResult()).

HttpServletResponse#getOutputStream() is never used for user endpoint responses. There is no special-casing for BinaryData, byte[], or InputStream return values — they fall into the else branch and are either JSON-serialized or coerced via String.valueOf(...), which destroys the binary content.

The Javadoc of com.top_logic.service.openapi.server.impl.Response already documents this limitation:

> "written to HttpServletResponse#getWriter()" — i.e., character output only.

Schema vs. Runtime Mismatch

com.top_logic.service.openapi.server.parameter.ParameterFormat defines a BINARY entry, which is reused by OperationResponse#getFormat(). BINARY is emitted into the generated OpenAPI spec document as a schema annotation, but there is no corresponding runtime handling in the response writer. The published contract therefore lies about what the endpoint can actually deliver.

Asymmetry with the Client Side

The client side (consuming a remote API) does support binary responses: com.top_logic.service.openapi.client.registry.impl.response.ResponseHandlerByExpression calls EntityUtils.toByteArray(entity) for BINARY format and returns a byte[] to TL-Script. Only the server side (producing a response) is broken.

Proposed Fix

In ServiceMethodByExpression#handleRequest, branch on the result type:

• If the result is a BinaryData, byte[], or InputStream (or if the declared/response ParameterFormat is BINARY), write raw bytes via resp.getOutputStream() instead of resp.getWriter().
• Preserve the current JSON/string path for all other cases.

Additionally, Response (the HTTPResponse TL-Script wrapper) should accept binary payloads so that a script can explicitly return application/octet-stream, application/pdf, image/*, etc., with the correct Content-Type and Content-Length set.

Affected Files

• com.top_logic.service.openapi.server/src/main/java/com/top_logic/service/openapi/server/impl/ServiceMethodByExpression.java (lines 79–112)
• com.top_logic.service.openapi.server/src/main/java/com/top_logic/service/openapi/server/impl/Response.java (Javadoc + possibly new binary payload field)
• com.top_logic.service.openapi.server/src/main/java/com/top_logic/service/openapi/server/parameter/ParameterFormat.java (runtime hook for BINARY on the response side)