LSP does not work over tramp when using direct-async-process

[CeleritasCelery]

Thank you for the bug report

• I am using the latest version of lsp-mode related packages.
• I checked FAQ and Troubleshooting sections
• You may also try reproduce the issue using clean environment using the following command: M-x lsp-start-plain

Bug description

Lsp does not work over tramp when direct-async-process is used in tramp 2.7.1. I tested this with the jedi-tramp server. calling lsp will have a delay and then the minibuffer will prompt

Server jedi-tramp:50133/starting exited (check corresponding stderr buffer for details). Do you want to restart it? (y or n) n

Restarting it just leads to a loop where it exits again. The *jedi-tramp:stderr* has this message:

Process jedi-tramp stderr<1> finished

Process jedi-tramp stderr finished

I was able to work around the issue by disabling the direct aysnc processes using cl-letf when launching the server here.

(list :connect (lambda (filter sentinel name environment-fn workspace)
                   (if (and (functionp 'json-rpc-connection)
                            (not (file-remote-p default-directory)))
                       (lsp-json-rpc-connection workspace (lsp-resolve-final-command command))
                     (let ((final-command (lsp-resolve-final-command command))
                           (process-name (generate-new-buffer-name name))
                           (process-environment
                            (lsp--compute-process-environment environment-fn)))
                       (cl-letf (((symbol-function 'tramp-direct-async-process-p) ;; disable direct async predicate
                                  (lambda (&rest args) nil)))
                         (let* ((stderr-buf (get-buffer-create (format "*%s::stderr*" process-name)))
                                (default-directory (lsp--default-directory-for-connection))
                                (tramp-use-ssh-controlmaster-options 'suppress)
                                (tramp-ssh-controlmaster-options "-o ControlMaster=no -o ControlPath=none")
                                (proc (make-process
                                       :name process-name
                                       :connection-type 'pipe
                                       :buffer (format "*%s*" process-name)
                                       :coding 'no-conversion
                                       :command final-command
                                       :filter filter
                                       :sentinel sentinel
                                       :stderr stderr-buf
                                       :noquery t
                                       :file-handler t)))
                           (set-process-query-on-exit-flag proc nil)
                          (set-process-query-on-exit-flag (get-buffer-process stderr-buf) nil)
                          (with-current-buffer (get-buffer stderr-buf)
                            ;; Make the *NAME::stderr* buffer buffer-read-only, q to bury, etc.
                            (special-mode))
                          (cons proc proc))))))
        :test? (or
                test-command
                (lambda ()
                  (lsp-server-present? (lsp-resolve-final-command command t)))))

Steps to reproduce

Enable direct async process:

(connection-local-set-profile-variables
 'remote-direct-async-process
 '((tramp-direct-async-process . t)))


(connection-local-set-profiles
 '(:application tramp :machine "remotehost")
 'remote-direct-async-process)

Try to connect to a LSP client via TRAMP and it will terminate with an error.

Expected behavior

LSP should work over tramp.

Which Language Server did you use?

lsp-jedi

OS

MacOS

Error callstack

No response

Anything else?

No response