Refactor compiler to separate impure and pure code

[madstap]

Currently the compiler has quite a bit of io spread out in many functions. It would be nice to refactor it into three distinct phases.

• reading posts, pages and the config into a data-structure
• transforming that data structure into a new one
• writing the transformed data structure to disk

All the difficult things are moved into the pure middle part, which becomes a lot more convenient to test and nicer to work with in general. (It's probably not necessary to be militant about purity, the printlns that are scattered around are kind of nice.)

The output data structure could be a map from url to finished html. There's a quite nice library that takes a data structure like that and either writes it to disk or serves it locally with ring. Using that we could save quite a bit of code in the compiler and, more importantly, code in the template.

It also has some other niceties.

• Reporting on conflicting urls.
• Reporting on differences in the built files.

Any thoughts on this?

[lacarmen]

Sounds like a good idea! The compiler namespace has become quite overgrown as new features were added and I think the three phases that you proposed is a nice, logical way to split all of that up. Perhaps this would a be a good time to take a look at #98 as well. Stasis sounds like a good library to use in Cryogen as well.

Is this something that you're interested in working on?

[madstap]

Yeah, I'm interested in working on this.

I could start by refactoring the way cryogen reads files as a relatively self-contained step. I'm thinking that there should be a function called read-markup-dirs that takes a collection of directories and a collection of markups and reads the posts/pages from all combinations thereof, encapsulating reading posts/pages. This would then make both #98 and #48 trivial to implement.

[lacarmen]

Yeah, I'm interested in working on this.

Awesome!

I could start by refactoring the way cryogen reads files as a relatively self-contained step.

That sounds like a good place to start. There's going to be a lot of code moving around for the full refactor so I definitely want this to be approached in an iterative fashion.

[madstap]

I'm thinking that dealing with markups could be made better by:

• Removing the concept of a markup having a folder. Allow using both Asciidoc and Markdown for individual pages and posts #48
• Having the parser take a string instead of a reader.
• Moving the prefixing of local links to the compiler, as a pure function, one of the transformations done on a post.