Code generation

[janwirth]

Hey there!

i love create-elm-app. It's awesome.

I have an idea.
Writing elm JSON encoders / decoders means writing a lot of boilerplate.

I suggest code generation as an approach to improve developer happiness.
I think this is a good feature for create-elm-app because

1. virtually every elm project uses coders
2. there is some complexity involved that is better hidden within a tool like create-elm-app

Existing solutions

Fortunately there are efforts for making this happen:

elm-cog

https://github.com/boxed/elm-cog#full-example-of-code-generation

Pros

• In-line declaration and code-generation

Cons

• Written in python (not self-containable with elm or JS code bases)
• Source of truth for the generated code is in the declarations. Not in the elm code. I think it would be much cooler to make a declaration that results in code based on existing elm code. A type alias for example.

decgen

https://github.com/dkodaj/decgen

Pros

• Written in elm
• uses elm types from

Cons

• Not well-integrated with the code base. You will have to roll your own CLI or resort to copy/pasting stuff. Not cool.

Next Steps

I am currently porting this thing to elm 0.19. Once I am done I will create a headless version that can read files for example and spit out coders for each type in the file.

Integration with create-elm-app

The codegen runs before elm-make.
It searches all elm files for codegen declarations and runs the generator

Usage

I imagine usage similar to the following:

SomeModule/Types.elm

SomeModule.Types.elm
import Dict
{- generate Coders -}

type alias myType = Dict Int String

type myOtherType = Yes | No

Where generate Coders is just an elm function call. In the comments though.

Result

SomeModule.Types.elm
import Dict
{- generate Coders -}
import Codegen.SomeModule.Types_298h2neist exposing (..)
{- generated end -}

type alias MyType = Dict Int String

type MyOtherType = Yes | No

Codegen/SomeModule/Types12345.elm is automatically generated and placed in a source-directory.
12345 is a hash of the source file.

{- generated by elm-code-generator 0.0.1 -}
module Codegen.Types12345 exposing (..)

decodeMyOtherType =
   let
      recover x =
         case x of
            "Yes"->
               Dec.succeed Yes
            "No"->
               Dec.succeed No
            other->
               Dec.fail <| "Unknown constructor for type MyOtherType: " ++ other
   in
      Dec.string |> andThen recover

decodeMyType =
   let
      decodeMyTypeTuple =
         Dec.map2
            (\a1 a2 -> (a1, a2))
               ( field "A1" Dec.int )
               ( field "A2" Dec.string )
   in
      Dec.map Dict.fromList (Dec.list decodeMyTypeTuple)

encodeMyOtherType a =
   Enc.string <| toString a

encodeMyType a =
   let
      encodeMyTypeTuple (a1,a2) =
         object
            [ ("A1", Enc.int a1)
            , ("A2", Enc.string a2) ]
   in
      (Enc.list << List.map encodeMyTypeTuple) (Dict.toList a)

[janwirth]

I think putting the generated code in a separate file right now is problematic because decgen will only create coders for types definitions it sees. It will then reference non-primitive type coders using the built-in convention. That will lead to those not being found in the scope of that separate file.

[kornicameister]

I personally do not believe in those sort of tooling. It somehow contradicts to me with idea standing behind Elm and coding by type. This tooling will not account for things that fall into business domain standing behind the data. Not sure if that's what you meant by your comment, but if not just imagine that you have a business data where two properties are connected. Having a certain value in of those means that there can be no value in the second or other way around. Just by decoding this in non-naive way you can ensure that nowhere in the codebase a contract that has been signed on data level is violated. Not to mention that it moves a way a need to introduce complex logic to ensure that. Or what about opaque types or hiding the types behind an alias? Just imagine having an Id that you wish to be generated only by the module that decodes that making it an only place where Id is generated. Effectively that means that this is Id that you got from the server and an only place to be fair. Having it done otherwise means that your code opens up a hole where this ID is generated accidently by another developer, even inside a view function for a weird display reason.

Just my POV why I do not see adding this tooling into widely used project thus making it easy to use. I know that elm decoders are tricky to learn. But once you got it right, you have a powerful tool in your hands to increase application reliability. Doing this in naive way effectively removes that possibility.
It is sort of the same story as removing the native modules support. It was easy to write native module instead of figuring it type-safe solution in Elm, so it was abused and later removed :-)

[kornicameister]

Well, TBH I guess I somehow disagreed with seeing writing decoders/encoders as boilerplate. Perhaps encoders are boilerplate. But with powerful typing system Elm offers, considering decoders just as a boilerplate is not very fair especially in context of coding by type philosophy.

[janwirth]

I understand create-elm-app as a tool for accelerate elm development. I am aware of the benefits of having a slowly changing codebase. Manually writing elm coders forces you to think about schemas and schema evolution. However, I can imagine it is possible to save a large amount of time by generating trivial coders on the fly while the application is in early development. That means if I want to encode a record into a JSON object and decode it back and it maps over well then no amount of human creativity is required.

I believe we should automate all the tasks that do not require human creativity or decision making. After writing a few thousand lines of decoders I believe that some of the work can be automated in an efficient way without sacrificing the benefits of coding by type.

Speaking of coding by type - if you look at ports or programWithFlags arguments you can see that elm is well able to derive decoders from primitive types.

[janwirth]

Just imagine having an Id that you wish to be generated only by the module that decodes that making it an only place where Id is generated. [...]

I am aware that some amount of complex business logic - a product of creativity hand human decision making can not and should not be automated at the current state of the technology. I am not advocating for automating the generation of all decoders. I just think that the task of generating decoders can be solved pareto-efficiently - meaning that we do 80% of the coders in 20% of the time by using code generation and focusing 80% of the time on the 20% of the coders that require human thought to be valid.

[kornicameister]

Yes, but I wasn't talking about primitive types. I was talking about unions, decoding into unions etc. But perhaps you're right in advocating for POC development with Elm, where reliability means less. With Elm it wouldn't be really dangerous to refactor the code later.

Perhaps, it's just me who preferred to hand craft the solution to control how it evolves even if it means some extra load in initial phase.

[janwirth]

I contradict. Refactoring coders is dangerous. It is the one thing that can not be checked. Which is even more so a case for generation because a generator makes less mistakes in complicated but primitive structure.

[janwirth]

control how it evolves even if it means some extra load in initial phase.

Your coders are derived from your types. If your types evolve, your coders co-evolve. As long as there is not data loss implied, you can just discard all data and automate the evolution.

[kornicameister]

I contradict. Refactoring coders is dangerous. It is the one thing that can not be checked.

Ah, yeah it is...from POV of accepting the backend response but not from POV of Elm. You change the type you use and you have it account for it everywhere. So if you make a mistake, in Elm the mistake is cheap. Luckily for this discussion, it does not mean whether you're using the generated code or not.

TBH. I feel like we're getting into ideological discussion. I'd say that both of us have a point somewhere and let's just use whatever tooling makes our life nicer for us. Still, it will be for @halfzebra to figure out if there's a place for such a thing inside a create-elm-app. Perhaps so, perhaps not...even if it is here, I do not see a problem with it as long as I can do what I am doing now :)

BTW. What's the problem with building custom command in package.json and watch the types to generate the model inside Elm src directory that would trigger rebuilding Elm project? You're basically getting the same thing. It is just that it is accessible for everyone. Do not get me wrong. But as our conversation goes on, I am sure that there might be more than just me who would prefer to write types and decoders/encoders on its own. It's not as common abstraction as elm-app start or elm-app build, right?

[janwirth]

I guess you are probably right. I think it is hard to talk baout programming languages without having an ideological discussion :D

[kornicameister]

I know and it sucks. I am trying to avoid that as much as possible. I present my own POV explaining why it matters from technical POV and perhaps that view that matters a most when it comes to learning how to think in certain methodology. I hope at least that part went ok :)

Anyway, it was nice to hear your POV. It certainly has some advantages and use cases. It might matter a lot for POC and for people learning the language. Like you might want to concentrate on getting the architecture right instead of coding by type which can be added later.