Custom Blocks Proposal

[joshuataylor]

Custom Blocks Proposal

In Jinja2, there is a feature to add custom blocks to the templates. This is a proposal to add a similar feature to MiniJinja.

Motivation

Having custom blocks is a very handy feature in Jinja2, as you can do things like this:

{% markdown %}
# Hello World
hello
{% endmarkdown %}

Which could pass to a custom function called "markdown", which tags in the content. This is very useful, and with Jinja2 this is fairly complicated to implement, and will be here as well.

Jinja2

How do projects use the functionality in Jinja2?

I searched on GitHub code search (which only gives popular results), and it seems pretty popular.

dbt:

Used to define custom blocks for materialisation, documentation etc. This allows the user to do things like:

{% docs user_id %}
     The id of the user.
{% enddocs %}

Then you can use this elsewhere (using a function, this is just an example), like {{ docs('user_id') }}.

How do projects integrate this in Jinja2?

From what I can tell, they define the tag, then they implement the parse function from Extension. I found a blog post explaining how this is done.

Minijinja

Implementation in MiniJinja will be tricky, as they will need to do a bit of work to make it work. But given that adding this is a pretty rare occurrence, I think it's fine (and might make people rethink if they really should be doing it :-)).

Implementation in MiniJinja itself

Let's say we want to add a custom block called "backwards", which will reverse all text. This will be a super naive implementation for this example, where we just reverse the text and don't account for Unicode and other edge cases: https://stackoverflow.com/a/38083610/1353681

let reversed_string = "hello".chars().rev().collect::<String>();

To do this today in MiniJinja requires doing the following:

1. Parser (parser.rs):
   1a. Adding a struct for it:

/// Reverse block.
#[cfg_attr(feature = "internal_debug", derive(Debug))]
#[cfg(feature = "multi-template")]
pub struct Reverse<'a> {
    pub name: &'a str,
    pub body: Vec<Stmt<'a>>,
}

1b. Adding it to Stmt:

pub enum Stmt<'a> {
    Template(Spanned<Template<'a>>),
    ...
    Reverse(Spanned<Reverse<'a>>),
}

1c. Adding it to parse_stmt_unprotected

Token::Ident("reverse") => ast::Stmt::Reverse(respan!(ok!(self.parse_reverse()))),

1d. Adding the parsing logic to it..

2. Adding it to codegen in compile_stmt (codegen.rs):

ast::Stmt::Reverse(reverse_block) => {
    self.compile_reverse(reverse_block);
}

And lots of other logic...

3. Adding it to meta find_macro_closure:

#[cfg(feature = "multi-template")]
ast::Stmt::Reverse(stmt) => {
    state.push();
    state.assign("super");
    stmt.body.iter().for_each(|x| walk(x, state));
    state.pop();
}

Ideas for implementation

I'm not sure what the best way is. add_function, add_filter etc are how it's done right now if you want to add custom functionality, and that's great. Maybe add_block?

It would also be worth thinking about the fact that this is now a global block.

[mitsuhiko]

I'm afraid that going there would expose the entire internal machinery of MiniJinja and stabilizing this would be risky. It would also greatly increase the total surface area of the library beyond what it was designed for. Jinja2 custom extensions are generally quite rare compared to Django templates for instance, partially because of the complexity of implementing it, on the other hand because you can usually avoid them with the expression language or {% call %} blocks.

One potential way to address this would be expose a preprocessing step in minijinja so that other crates could perform a transformation themselves before feeding it in. I am however not sure if I want to expose the AST, instructions or code generator ever, so that would probably be limited to source level transformations.

[mitsuhiko]

There is also an issue filed on this #200