An update on the Wul programming language

programming, language

Wul (pronounced wool) is a toy programming language I’ve been working on as a learning exercise. In my school days, I was asked to implement a very simple tree-walking interpreter using ANTLR to generate the parser. I found this to be one of the more interesting programming assignments. I spent many hours simplifying the EBNF grammar from which the parser was generated. Wul is the language that I began to design after that course. My motivation was to construct a language that was flexible yet allowed me to understand every detail. This forces the language to remain rather small despite drawing on both Lua and Lisp. It has, however, reached a point where I think it may prove useful to document its current features.

Functionality

Every value can act like a function. This is achieved by per value metatypes (similar to Lua’s metatables). A metatype is a collection of named methods that define the functionality of a value. Most importantly. it defines whether or not it operates like a function (via the invoke metamethod). This also means you can replace the invoke method of an individual function. You may do this for the function metatype to change the behavior of all functions, including special forms. But this is not advised. Consider: how would Wul be able to interpret your new method if it must call it each time any function is invoked in its definition?

By default the numbers 5 and 6 share the same metatype: the Number metatype. This metatype allows the values to be added together, converted to strings and so on. However, it is possible to define a new metatype for 5. Perhaps adding 5 to 6 (but not vice versa) should raise an error. In that case, a per value metamethod is desirable. If, however, you want to prevent the addition of 5 and 6 in any way, then replacing the add metamethod on the Number metatype is desirable. Either is possible, it’s to the user to decide if either is necessary.

Multiple returns values

While functions are capable of returning multiple values in many lisps, they usually require helpers like multiple-value-bind or match-define. I found this makes multiple return values more trouble than they’re worth. They are rarely used and seemingly discouraged in the lisp community. Wul, like Lua, takes the opposite approach. Use them whenever it makes sense. It’s valid to consume functions with multiple values with very little syntax. As an example:

(defn n-square (n) (return n (* n n)))
(+ (n-square 5) 10) ; this returns 40

This may allows subtle bugs but Wul prioritizes flexibility. If you want only the first value of a multiple return function, use the destructuring form of let. Multiple return values are used in many standard library functions, notably the pcall (protected call) function as seen here here. The function unpack turns a list into multiple values.

Ranges

Ranges are created with a special list syntax [start end increment] where the increment is optional. The default increment is 1 or -1 depending on the values of start and end. A third special form exists [1] is the range of only one. This seems useless except that ranges can be used as function to index lists. ([1] somelist) returns the second element of somelist. ([0 4 2] somelist) will return a list consisting of the 1st, 3rd, and 5th elements of somelist. Ranges are currently the preferred idiom for reversing a list. Ranges can be enumerated using the first and rem (remainder) operators like lists and maps. They can be eagerly converted to lists with the list function. The default map function uses next and remainder to enumerate values only as they are used.

Interoperability

The wul is written in C# because I’m rather familiar with the language and because I thought I could reuse most of its standard library. I’m not interested in writing a standard library, but useful examples often require it.

Wul can access the .Net libraries via the poorly named :: operator. Framework types can be constructed with the new-object operator. This makes Wul useful as a .Net lisp-like glue language. In principle, if a function is not contained in the Wul standard library look for a .Net alternative. If you find yourself using many files, create a Wul file to define wrapper functions. As an example:

(using System.Math)

(defn Cos (n) (:: Cos n))
(defn Sin (n) (:: Sin n))
(defn Tan (n) (:: Tan n))
(defn Abs (n) (:: Abs n))
(defn Sqrt (n) (:: Sqrt n))
(defn Max (a b) (:: Max a b))
(defn Min (a b) (:: Min a b))

Oddities

I have never implemented a lisp before or, for that matter, seen the source of another lisp-like language. What I call “magic functions” in the source code seem to correspond to special forms. In earlier versions of Wul, special forms could be written in Wul itself but I removed them as I found it easier to reason about hygienic macros. I should also admit, at some point, that I prefer Lua’s tables and semantics but Lisp’s syntax and macros.

Wul’s metatypes allow simple recursive anonymous functions. Within the lambda and defn special forms the variable self refers to the function it defines. The following will produce an infinite loop:

((lambda (self)))

Future plans

  1. Replace all lists with an optimized associated array (like Lua’s tables). This unification removes the need for arrays, lists and maps/dictionaries. (add b: 5 a: 6) would construct a table with element 0 as the function add, element a as the number 6, and element b as the number 5. When add is invoked, all key-value pairs in the list being interpreted will be introduced into the scope of execution. Function definitions will use identifier-value pairs to introduce defaults in the case no value is supplied. Supplying nil will be considered a value. For example: (defn add (a: 0 b: 0) (+ a b)). Metatypes will then become associated arrays rather their current hybrid state.

  2. Coroutines or generators and other asynchronous programming support. Wul currently supports tasks but the interface is not complete. It works for calling threads in a background thread or waiting for a set of tasks to complete. As part of this change, I’d like to move most of the standard functions (e.g. list, unpack) to lazy enumeration.

  3. Reader macros. This will simplify the parser and make ranges, quote shorthand, list/table keys, and quoting escapes easier to add.

  4. Prototypical inheritance, if I ever need it. I rarely use inheritance, especially in highly dynamic languages, but if I find a use case it’ll be through as a metamethod called parent or index. And metamethod not defined in the value’s metatype will be searched for in its parent line. This would also need the ability to call the type’s corresponding metamethod in its replacement (something like base or super).

  5. Explicit type signatures for interoperability. Currently :: deduces the type signature of the method to from its runtime arguments. Adding the option to specify the return type and parameters will make it easier to deal with overloaded .Net functions and those with optional arguments.

  6. A metamethod for enumeration. The at metamethod already exists for indexed access making this a low priority. A next method should be sufficient but I’m curious if something like Lua’s pairs would be more useful. As a parameter it should take the value itself and the current enumeration state.

  7. Improved debugging facilities, improved error messages and trace backs. The implementation of these vital features is not as straight forward as I had initially hoped though they are all currently present in varying degrees of correctness and completeness.

  8. Performance is not and will not be a priority. Wul is about developing the semantics I find interesting. When the language is reduced to an unchanging state, I plan to write a compiler that targets Lua so I can mix and match Lua and Wul.