[WIP] feat(std): add monad transformer interface, StateT, and LazyT

[Etherian]

Description

Adds a monad-transformer interface Transformer, a State monad transformer StateT, and a Lazy monad transformer LazyT to the standard library.

Related PRs

Lays groundwork for Streamlike and the new parser #687.

Status

Merged

Outstanding design questions

• Can and should StateT be replaced by the effects system? (not for the time being, at least)
• Should Transformer require monad field? (Basing monads on their transformers is the better option)
• Should StateT and LazyT (and other monad transformers) be merged with their non-transformer counterparts?

To Do

• Fix StateT's implicit parameter bugs (fix(check): Subsume implicit functions with forall correctly #688 & fix(check): The alias reduction stack must be cleared between unifyin… #689)
• add tests
• add inline docs

Prior Art

• StateT
  • Haskell
  • Idris
  • Implementing Applicative (<*>) for StateT
• MonadTrans
  • Haskell
  • Idris

[Marwes]

Left some nits for some more idiomatic code but otherwise looks good 👍 . Some tests, (doc or in tests/pass) would be nice but maybe that can wait till later as State could be implemented in terms of StateT and it would then get tested by the current tests.

Something I have been thinking about is that std's top level (https://gluon-lang.org/doc/nightly/std/std.html) has become rather crowded so for the next version I will likely try and move some stuff into sub-modules. Maybe it makes sense to put these under std.transformer or std.monad.transformer? Without being sure about where to put them it might make more sense to wait with that however.

[Etherian]

Left some nits for some more idiomatic code but otherwise looks good 👍 . Some tests, (doc or in tests/pass) would be nice but maybe that can wait till later as State could be implemented in terms of StateT and it would then get tested by the current tests.

Thanks. I'll make those changes as soon as I get the chance. Any tests I add might take a bit longer, since that's another area I don't have much experience with.

Something I have been thinking about is that std's top level (https://gluon-lang.org/doc/nightly/std/std.html) has become rather crowded so for the next version I will likely try and move some stuff into sub-modules. Maybe it makes sense to put these under std.transformer or std.monad.transformer? Without being sure about where to put them it might make more sense to wait with that however.

I think it makes sense to merge std.statet with std.state and std.lazyt with std.lazy like in Haskell and Idris. Then they could be moved to std.monads with all the other monads and monad transformers.

[Marwes]

Can and should StateT be replaced by the effects system?

It think there is still value in having monad transformers as they have their uses. If they end up being replaced by effects in 99% of cases monad transformers could still be factored out of the standard library and into an external library for those who need it.

[Etherian]

I was writing tests for StateT when I ran into some issues. (Ignore the error mentioned on line 26. Looking back at my logs, that appears to have been due to a typo.)

error: Implicit parameter with type `forall m . std.functor.Functor m` could not be resolved.
- <tests.pass.statet>:32:29
32 |     assert_eq (eval_state_t ((mx >>= f) >>= g) s) (eval_state_t (mx >>= (\x -> f x >>= g)) s)
   |                             ^^^^^^^^^^^^^^^^^^^^

et alter...

The type checker sometimes seems to have trouble finding buried Functor instances.

error: Implicit parameter with type `std.transformer.Transformer (std.statet.StateT b)` could not be resolved.
- <tests.pass.statet>:29:40
- 29 |     let mx : StateT _ _ _ = wrap_monad mx
-    |                                        ^^

I'm not sure what I'm doing wrong here.

error: Expected the following types to be equal
Expected: std.list.List Int
Found: Int
1 errors were found during unification:
Types do not match:
    Expected: std.list.List Int
    Found: Int
- <tests.pass.statet>:44:103
44 |         test "modify eval_state_t" <| \_ -> (assert_eq (eval_state_t (modify (\x -> x + 2) *> get) 0) 2),
   |                                                                                                       ^

et cetera...

There's not actually an issue here. I left the unfinished code in lines 41-44 to point out an anomaly in the error messages: they refer to the List type when that type is not relevant to those lines. The type checker seems to be assuming the missing types in those lines are Lists instead of referring to them generically.

[Marwes]

error: Implicit parameter with type forall m . std.functor.Functor m could not be resolved.

• <tests.pass.statet>:32:29
  32 | assert_eq (eval_state_t ((mx >>= f) >>= g) s) (eval_state_t (mx >>= (\x -> f x >>= g)) s)
  | ^^^^^^^^^^^^^^^^^^^^

It appears that records are only traversed once to pull out the implicit instances so it finds Applicative but not Functor. I have a fix which recurses fully but unfortunately it cause ambiguity errors else where since the resolver then finds Num -> Ord and Ord and there is no way to figure out that those are actually the same instance.

Going to have to sleep on this one.

#[implicit]
type Functor f = {
    map : forall a b . (a -> b) -> f a -> f b
}
#[implicit]
type Applicative f = {
    functor : Functor f,
}
#[implicit]
type Monad m = { functor : Applicative m }

let any x = any x

let eval_state_t f : [Functor m] -> m a -> () = ()

let associativity mx : [Monad m] -> m a -> () =
    eval_state_t mx

()

error: Implicit parameter with type std.transformer.Transformer (std.statet.StateT b) could not be resolved.

• <tests.pass.statet>:29:40
• 29 | let mx : StateT _ _ _ = wrap_monad mx
• |

Here you have forgotten to export the transformer binding from statet.glu, add transformer alongside monad etc and it works.

error: Expected the following types to be equal
Expected: std.list.List Int
Found: Int
1 errors were found during unification:
Types do not match:
Expected: std.list.List Int
Found: Int

• <tests.pass.statet>:44:103
  44 | test "modify eval_state_t" <| _ -> (assert_eq (eval_state_t (modify (\x -> x + 2) *> get) 0) 2),
  | ^

Implicit resolution seems to do something weird here, haven't looked into it further.

[Marwes]

👍 Looks good to me. Either remove or fix the TODO and I would be happy to merge!

[Marwes]

That might be useful but we don't have to do that here and now.

[Etherian]

I decided to not include the unfinished tests, so the PR should be ready to merge. I do have a question, though, about one of the issues I had while trying to write them. Why does the implicit parameter of (+) not automatically resolve in the code below?

let { wrap, (<*>) } = import! std.applicative
let { Option, ? } = import! std.option
let int @ { ? } = import! std.int
let { wrap_monad } = import! std.transformer

let mplus : Option _ = wrap (+) // requires adding ?int.num
let ma : Option Int = wrap 10
let mb : Option Int = None
let c : Option Int = mplus <*> ma <*> mb
c

[Marwes]

Why does the implicit parameter of (+) not automatically resolve in the code below?

Implicits are resolved just before generalisation just as in https://arxiv.org/pdf/1512.01895.pdf . The short of it is that since type inference and implicit resolution depend on each other when we chose to do implicit resolution affects type inference and therefore whether the program is well typed.

Since we also want typing to be predictable, choosing to do implicit resolution just before generalization on the closest let binding is the easiest to follow solution. Most of the time it is not an issue since the type will be concrete (say (+) : Int -> Int -> Int) and in the other cases it can be fixed with a type signature let mplus : [Num a] -> Option a = wrap (+).

As an aside, we could chose to infer that signature instead of demanding a concrete type. However I would argue that it is usually not what you want. Introducing an implicit argument makes the binding a function so using it elsewhere involves a function call, making it more costly than one might expect.

While this is basically https://wiki.haskell.org/Monomorphism_restriction which has a fair share of critque, gluon has scoped type variables by default so I think it avoids most of actual problems.

Looks like tests are failing after the rebase since you now need to import Lazy (and some other types) explicitly https://travis-ci.org/gluon-lang/gluon/jobs/527568322#L1771 . Went through and made the standard library types act more like library types.

[Etherian]

Thanks for the explanation! Do you think it would be viable to suggest the inferred signature in the error message? That way we could keep the difference in signature explicit while also smoothing the learning curve for newbies like myself.

Looks like tests are failing after the rebase since you now need to import Lazy (and some other types) explicitly https://travis-ci.org/gluon-lang/gluon/jobs/527568322#L1771 . Went through and made the standard library types act more like library types.

Ah, that explains it.

I won't have access to my computer over the weekend, but if you made the tweaks I could accept them over the phone. Or I could just do it Monday.

[Marwes]

Do you think it would be viable to suggest the inferred signature in the error message?

Good idea! Created an issue #725

[Marwes]

bors r+

[bors]

Build succeeded

• continuous-integration/travis-ci/push