Extending the syntax analyzer

And as in uffish though he stood,
The jabberwock, with eyes of flame,
Came whiffling through the tulgey wood,
And burbled as it came!

Introduction.

Both the debugger and the syntax analyzer of the editor must have a notion of the syntax LISP. The debugger need to know the syntax to figure out where he has to add instrumentation code and the syntax analyzer must known how to divide the source in the different elements needed for syntax coloring. Because the syntax of LISP can be extended via macros, a fixed syntax hardcoded in the debugger or the analyzer is out of the question. Jabberwocky uses a special language to define the syntax. This makes it possible to extend the debugger and syntax coloring (for example when you have added your own control structures). The basic principle is as follows : When Jabberwocky is installed a default syntax file is used to generate the necessary code, so that the standard LISP syntax is handled correctly. If you extend the syntax of LISP and you want to see these extensions reflected in the debugging and highlighting you must modify this syntax file and then generate the necessary code.

If you want to modify the debugger keep in mind the following points :

Modifying the syntax

The steps to modify the debugger are as follows:

Language to define the syntax

The extension language is strongly based on the syntax descriptions used in ANSI LISP, so that writing extensions is as simple as writing a syntax diagram.

The default file used by the debugger

To give you an idea of how the extension language looks like, look at 'lispsyntax' the source used to generate the parser/transformer of the debugger/editor delivered with this package. Although the language used is not yet defined it should look familiar to you.

Syntax of the language.

The language is composed of the following elements : A source in our language is a text file containing definitions and expressions.

Semantics of the language

The syntax tells us what the wellformed expressions and definitions are in our language but it says nothing about their meaning, for this we need a little bit of semantics. The best way to understand the semantics of the language is the consumer/producent metaphor. When a expression in our language is applied on a lisp expression two things can happens :

Lets now put these ideas in practice on the different type of expressions of our language. Let P be the list produced, E a expression in our language and L the lisp expression on which we applies expressions in our language. The parser/generator in the debugger will apply each expression on a given lisp expression until it gets not a throw and the lisp expression is fully consumed, the produced list is then the lisp expression with debug code added. If this sounds inefficient you are right this is just a semantic explanation, our language is actually compiled to become the parser/generator of the debugger which has the same effect as our semantic explanation, but he does it in a more efficient way.
The syntax highlighter works in the same fashion, but instead of generating code, it notes which symbols are variables,functions or macros in a lispform.
Lets look now at the semantics of the different elements in our language.

Cavecats