#rakulang grammar hints about creating a templating system

Raku’s grammar system is very powerful, but often mystifying and frustrating. Some worked examples go a long way. This post isn’t a “how to create a templating system”, it just contains a few hints on getting started.

It turns out that a simple system is easy. I have in mind a way of extending my assembler written in Raku with a very naive substitution system. This will enable me to write assembler at a higher-level. Really, it will land me a third of the way towards BASIC.

Where might I be going with this? Well, instead of specifying a grammar for BASIC as is normally the case, I had an interesting idea of creating a kind of extension mechanism inspired by Forth. So you have a kind of BASIC, but with a plug-in extensible grammar system.

I’ll describe how such a curious feat might be achieved in a later post, assuming I decide to implement it. But I have digressed too far already. so, back to the topic at hand.

What I’m trying to achieve in the first instance is that you may have triggering syntax embedded inside general text, and you want to perform actions on this special text.

The example I chose to work with is wherever I see the word IF, I want to replace it by --GOT IF--, and where I see the word IFX, I replace it with ..GOT IFX.. . All other text should be left alone. So, pretty simple, but not necessarily easy to figure out how to achieve if, like me, you’re new to Raku’s grammars.

It turns out that the implementation (available as a gist) is not difficult at all:

grammar G {
	token TOP { <atom>* }
	token atom { 	IFX { print "\n.. GOT IFX..\n"; } 
			| IF {print "\n--GOT IF--\n"; }  
			| .  { print $/; }
		}
}

my $str = "heIFllIFXo\n";

G.parse($str);


# outputs
# he
# --GOT IF--
# ll
# .. GOT IFX..
# o

There we go, simple as that. Some points to note: I have defined inputs as being made of “atoms”. An atom is either one of the keywords, or anything else. You can see token atom selects from a list of alternatives via the | operator.

Note also that the action that we perform is dependent on the actual match that succeeded. This facility may be obvious to some, but I didn’t know about it until a couple of days ago. I had always assumed that the action was based on the whole token. Like I said before, Raku grammars have a lot of stuff available, but it’s not necessarily obvious for the beginner. Trying to wade through a morass of documentation often doesn’t help. It is often tempting to ditch the whole approach and just hand-roll a grammar.

Note that IFX is define before IF, because if they were defined the other way around, then you could only match IF, and not IFX. For the same reason, the . , which matches anything, is defined last.

A final point to note is that $/ represents the “current match”. In the context of the code above, if we don’t match IF or IFX, we match something else, which we just echo out.

Great! Now go and implement the Unix tools m4 and groff in Raku. Whilst you’re at it, be sure to add programmability, a la awk.

That’s all for now. Stay safe. May you all be happy.