Guile over nREPL, guile-ares-rs integration

[abcdw]

Hi @Olical!

On fediverse, you mentioned that stdio Guile REPL is far from perfect and that it would be cool to make a Guile nREPL integration for Conjure. A few weeks ago we released guile-ares-rs (RPC server based on nREPL protocol), it implements all the basic functionality (evaluation, interruption, completion) and seems to be a very reliable.

There is an emacs-arei client, which can be used as a source for inspiration.

I don't use vim for almost a decade already and thus it would be not very rational to write a client/integration on my own, however, I would be glad to share the knowledge and implement missing functionality on the server side to help someone to implement the client.

Cc people using guile and vim I know: @ekaitz-zarraga, @Millak

[Millak]

I know of vim-slime, which I've heard is like SLIME for emacs, but I've never used emacs or actually used vim-slime. https://github.com/jpalardy/vim-slime

[Olical]

This is really cool, thanks for sharing! This should work great when I can refactor the nREPL support into a reusable layer.

[abcdw]

On 2023-12-30 04:35, Oliver Caldwell wrote: This is really cool, thanks for sharing! This should work great when I can refactor the nREPL support into a reusable layer.

Nice! Don't hesitate to contact me if you have any questions, need support or changes on ares-rs side. Wish you great holidays! :)

…

-- Best regards, Andrew Tropin

[Futurile]

Tried the basic parts of this today and the nREPL side works - would be great to have it in Conjure ;-)

This is what worked for me:

1. Install Guix as a package manager on top of any Linux distribution
   Install it and do a guix pull to get the latest package definitions: already had this.

2. Start a guix shell environment / install guile-ares-rs

$ guix shell 
$ guix package --install guile-next guile-ares-rs

$ guile -c '((@ (nrepl server) run-nrepl-server) #:port 7888)'

This starts a nrepl and I can connect to it with netcat

3. Start a Clj nrepl

  clj -Sdeps '{:deps {nrepl/nrepl {:mvn/version "0.5.3"}}}' -m nrepl.cmdline -c --host 127.0.0.1 --port 7888

We can send really basic stuff:

user=> (+ 1 1)
2

user=> (list 1 2 3 4)
(1 2 3 4)

user=> (reverse '(1 5 9))
(9 5 1)

<ctrl>c doesn't seem to actually work to interrupt it though.

Anything more complicated hits the fact that we're inside a Clojure environment.

I couldn't get Conjure to connect to it with ConjureConnect.

[Olical]

Started work on this, I can't get guix installed on arch so far though, the AUR builds the package ~3 of them have failing tests and won't install. I guess I'll need to find another way to install it or maybe use a docker container that already has guix installed like https://github.com/metacall/guix?

[abcdw]

Hi Oliver!

You actually don't need guix, if you can install guile 3.0.10 (this version or newer is required). After that just add fibers and guile-ares-rs to load path like that: guile -L ../fibers -L ../guile-ares-rs ....

If you want to use guix in container, you can use debian:testing image and do sudo apt install guix and guix pull (from user, without sudo). BTW, in guix guile is not yet updated to 3.0.10, so you need to use guile-next at the moment, I think it will be updated soon.

We are preparing the last bits for 1.0 release, so I think we will do it next week or just after guile is updated in Guix repo:
https://todo.sr.ht/~abcdw/tickets?search=label%3A%22milestone%3Aarei-1.0%22

Let me know if you have any questions or need a hand.

[Olical]

So guix has been removed from debian testing (not sure why) so I tried to use stable but that requires a daemon which requires extra dependencies and I still can't get to work.

I've tried https://github.com/metacall/guix but can't get that to work, I think it requires some very specific buildx based commands in a chain? I couldn't get the insecure flags to work for me for some reason.

I've tried cloning both of the dependencies and I have guile 3.0.10 installed, I used this command:

guile -L ./deps/fibers -L ./deps/guile-ares-rs -c '((@ (ares server) run-nrepl-server))'

I get:

Backtrace:
In ice-9/boot-9.scm:
  1755:12  8 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
           7 (apply-smob/0 #<thunk 7827a1d0e300>)
In ice-9/boot-9.scm:
    724:2  6 (call-with-prompt ("prompt") #<procedure 7827a1d1ebc0 …> …)
In ice-9/eval.scm:
    619:8  5 (_ #(#(#<directory (guile-user) 7827a1d11c80>)))
In ice-9/command-line.scm:
   185:19  4 (_ #<input: custom-port 7827a1d0c850>)
In unknown file:
           3 (eval ((@ (ares server) run-nrepl-server)) #<directory …>)
In ice-9/eval.scm:
   182:19  2 (proc #f)
   142:16  1 (compile-top-call #f (5 (ares server) # . #t) ())
In unknown file:
           0 (%resolve-variable (5 (ares server) # . #t) #f)

ERROR: In procedure %resolve-variable:
In procedure module-variable: Wrong type argument in position 1 (expecting module): #f

So I think maybe I'm either missing an argument or I need to run some build commands in the fibers and guile-ares-rs directories? Just sharing this here to show exactly what I've tried and what I've failed at 😅 I like the idea of using docker most because that'll be easily reproducible but it seems like running guix under docker is a bit tricky because it needs to fork some processes (I think?).

Edit: Trying out installing guix from the AUR again, just in case it was a temporary issue.

[Olical]

The AUR package failed sadly:

curl: (28) Failed to connect to download.savannah.gnu.org port 443 after 135248 ms: Couldn't connect to server
Warning: Problem : timeout. Will retry in 3 seconds. 3 retries left.

I tried the manual install script from guix that sidesteps the arch package management but that also failed sadly

[1721389986.054]: Starting installation (Fri 19 Jul 12:53:06 BST 2024)
[1721389986.058]: [ PASS ] verification of required commands completed
[ INFO ] The following OpenPGP public key is required to verify the Guix binary signature: 27D586A4F8900854329FF09F1260E46482E63562.
Would you like me to fetch it for you? [Y/n]
2024-07-19 12:53:08 URL:https://savannah.gnu.org/people/viewgpg.php?user_id=127547 [55/55] -> "-" [1]
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
[1721389988.932]: [ FAIL ] Missing OpenPGP public key (27D586A4F8900854329FF09F1260E46482E63562).
Fetch it with this command:

  wget "https://sv.gnu.org/people/viewgpg.php?user_id=127547" -O - | sudo -i gpg --import -
[ INFO ] The following OpenPGP public key is required to verify the Guix binary signature: 3CE464558A84FDC69DB40CFB090B11993D9AEBB5.
Would you like me to fetch it for you? [Y/n]
2024-07-19 12:53:12 URL:https://savannah.gnu.org/people/viewgpg.php?user_id=15145 [55/55] -> "-" [1]
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
[1721389992.036]: [ FAIL ] Missing OpenPGP public key (3CE464558A84FDC69DB40CFB090B11993D9AEBB5).
Fetch it with this command:

  wget "https://sv.gnu.org/people/viewgpg.php?user_id=15145" -O - | sudo -i gpg --import -

I tried to import those keys by running those commands but got

gpg: no valid OpenPGP data found.
gpg: Total number processed: 0

So I think I'm just going to have to park this concept for now until I can find some better way to run the software, sorry 😢 maybe it's just me and others can easily get this working in a very portable way.

Edit: I think various GNU servers are down right now maybe? So some unlucky timing!

[abcdw]

Your first attempt looks almost correct.

guile -L ./deps/fibers -L ./deps/guile-ares-rs -c '((@ (ares server) run-nrepl-server))'

should be
guile -L ./deps/fibers -L ./deps/guile-ares-rs/src/guile -c '((@ (ares server) run-nrepl-server))'

[Olical]

Oh thanks! I didn't spot that, I should've had a look at where the actual guile source was in the repo. This got us further but not all the way, here's my latest output:

nREPL server started on port 50668 on host localhost - nrepl://localhost:50668
Backtrace:
In ice-9/boot-9.scm:
  1755:12  8 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
           7 (apply-smob/0 #<thunk 7b967cd0e300>)
In ice-9/boot-9.scm:
    724:2  6 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
In ice-9/eval.scm:
    619:8  5 (_ #(#(#<directory (guile-user) 7b967cd11c80>)))
In ice-9/command-line.scm:
   185:19  4 (_ #<input: custom-port 7b967cd0c850>)
In unknown file:
           3 (eval ((@ (ares server) run-nrepl-server)) #<directory …>)
In ares/server.scm:
     91:3  2 (run-nrepl-server #:host _ #:family _ #:addr _ #:port _ …)
In fibers.scm:
   135:23  1 (run-fibers _ #:hz _ #:scheduler _ #:parallelism _ # _ # …)
In fibers/scheduler.scm:
   125:22  0 (make-scheduler #:parallelism _ #:prompt-tag _)

fibers/scheduler.scm:125:22: In procedure make-scheduler:
Unbound variable: events-impl-create

[Olical]

I have tried with the latest commit in both dependencies and with the latest tag (0.9.5 for guile-ares-rs, v1.3.1 for fibers), the all result in the same error. I'm using guile (GNU Guile) 3.0.10 which I think should be good enough?

Maybe I'm missing a system dependency or library?

[Olical]

Finally got it to work, I had to go into the fibers repo, compile it, installed it globally (which maybe isn't required, that didn't actually fix the issue even though it really should?) and then used the env script the fibers repo provides to wrap my guile instance.

./deps/fibers/env guile -L ./deps/guile-ares-rs/src/guile -c '((@ (ares server) run-nrepl-server))'

So now I have a Guile nREPL but it's not scripted yet, I had to build manually. I'll try to automate that in Conjure's dev tooling directory to make it a little easier to reproduce.

[Olical]

Annoyingly the guix installer is working today 😆 so I'll see if I can use that finally, that'll make it easier to reproduce I think. I guess some GNU project servers have been down every time I tried to use them.

[abcdw]

Finally got it to work, I had to go into the fibers repo, compile it, installed it globally (which maybe isn't required, that didn't actually fix the issue even though it really should?) and then used the env script the fibers repo provides to wrap my guile instance.

Haha, I just reproduced an issues you mentioned, did exactly that and wanted to share with you :)

[abcdw]

Yep, gnu infra is a bit wonky the last few weeks.

I glad that you succeed to run server, you probably would like to pull the latest ares-rs version, it has a lot of updates and it's better to develop against it. I think we almost finished breaking APIs before 1.0, but I can potentially change a couple more things next week.

You probably also want to add (@ (ares-extension ares logging) ares.logging) to the list of bootstrap extensions to get the following debug output on server side:

[Olical]

So if guix works (still doing my first guix pull) would you say that it's the best way to run things with the latest versions or should I rely on the git repo to stay in the loop?

And thanks for the tip on the logging, that'll be helpful!

[abcdw]

You should rely on latest git checkout, it has a lot of changes since 0.9.5. Plus it's easier to hack things in-place for server side part, when you have git repo at your disposal.

If you have a working guix, you can spawn an nREPL with make ares-rs or just make. It will pull a guix version defined in rde/channels.scm and all needed dependencies (guile and fibers) and will run ares with them. It will make us to use the same dev environments. However, it's not necessary, the option with ./env in fibers will also work.