Nickel as a scripting language

[ad-si]

I was thinking of embedding Nickel in a project of mine as a scripting language (think spreadsheet formulas or workflow automation). Usually Lua would be a good fit for something like this I guess, but I'd prefer the Nickel syntax a lot. Also I can probably do without IO functions like network- and file-access.

Are there any drawbacks / concerns I should be aware of?

[yannham]

Hello!

I believe Nickel would be quite fitting for such a task - in fact I've talked with other people experimenting with the same ideas (use Nickel as a "typed workflow" description language). We also wrote the interpreter in Rust in part to make it easy to interface with other languages (although, as of now, we don't expose C bindings yet, but at least here's no runtime in the way 🙂 ), so it shouldn't be too hard to embed. I believe the merging approach of Nickel can lead to new and interesting programming techniques for some domains, that haven't been really explored yet.

First, the elephant in the room is that Nickel is just less mature than a language like Lua. It's been in development for years, and past 1.0, so it shouldn't be totally unstable, but it's still nowhere as tested.

A second possible drawback could be performance. Lua has very, very fast engines available, and is a much simpler language. Nickel, on the other hand, have had performance problems already for mid sized codebase - although it's not really something Nickel-specific, but rather than we invested in other aspects of language engineering before performance. The good news is that we're working on it right now (see e.g. #1622), and already started implementing optimizations. Still, I think we have some serious work to compete with modern, efficient scripting VM or JIT compilers.

I would say it depends on the nature what you're trying to do: is it a more experimental project? Do you anticipate manipulating data structures (say, array) of typical size 10,100,1k,10k,100k,1M elements? Do you expect scripts to stay small or to grow larger? Is the Nickel evaluation part going to be potentially a non-negligible part of the whole thing?

[matklad]

Related question: if I do want to have IO and side-effects, how would that look in nickel?

Context: suppose we ignore all existing prior art whatsoever, and decide to implement something of the same overall shape as nix, but with nickel as the main language. nickel obviously works as a replacement of nix for describing a set of mutually-dependent derivations.

However, one also needs some executable language to describe build function of the derivation itself. In nix currently, that is bash. So, you derivation build function is roughly the following bash script

./configure
make
make install

It is very tempting to get rid of this second language (bash) and just use nickel for building stuff as well. How bad ./configure && make && make install would look in Nickel?

[yannham]

This is a broad question, depending on whether you expect Nickel to handle everything - including "sending the missiles" - but I think something along the lines of Guix G-epressions (which is one inspiration of Nickel symbolic strings) could do.

In general, I imagine that instead of actually performing actions, you would build an AST or a data representation of the whole effectful computation within Nickel, and handle that to a part that can actually perform the effects, à la Haskell free monads. Could be as simple as converting that to bash, and that could be done in pure Nickel. You would just need some glue to run the generated action.sh at the end or something. In some sense you can do that in Nix as well, as you don't have to use Make but you can provide your own builder which can just be a string representing self-contained bash - but untyped strings aren't a great interface, and there is no real advantage over just writing them in a bash file directly.

So, put differently, you probably want to define a DSL for scripting, the way you like it. Then you can encode it in Nickel, with the two usual approaches, the initial encoding and the final encoding.

An initial encoding (very broadly, represent the AST directly as data) could leverage enums to do that.

A final encoding is more akin to a normal library (my personal of view is that the interface of any library out there in any language define its own DSL - it might just be a very simple or a very common one, but it's still a DSL)

run [
  file "/foo/bar/baz" |> create,
  file_exists (file "/borg") |> exit_if,
  _unsafe_run_bash "dark_magic_that_can't_be_expressed_as_nice_building_blocks",
]

which could generate bash on-the-fly as a growing string, or actually use enums under the hood to represent this computation as e.g.:

'Sequence [
  'Command { command = 'create, arg = 'File "/foo/bar/baz" },
  'Command { command = 'exit_if, arg = 'Command { command = ''file_exists, arg = 'File "/borg" }  },
  'UnsafeRunBash "...some bash...",
]

Which could be then compiled to something different than bash by another part of the Nickel code or even an external tool.

Nickel's Symbolic strings could also be helpful to interpolate as-of-yet-unknown values inside such scripts (after tall, that is their very raison d'être), although they might be not powerful enough right now. Maybe they should be updated to use the new enum variants and be a powerful and proper quote/unquote mechanism.

Is it more or less what you're asking about, or is it a bit off-topic?