Learning Lua

Lua came to my attention because two of my favorite programs use it: pandoc is written supports custom Lua filters; NeoVim is written in Lua and extensively uses Lua (default Vim can also be compiled with lua support).

Just recording some of the idiosyncrasies of Lua here.

Data Structures

Tables

Source: https://www.lua.org/pil/3.6.html

The only data format in Lua is called a table, which can be understood as an associative array that manages pointers to an underlying object.

The type of indices can be mixed, the values of tables as well. So with this very general and flexible data format, you can build other common data formats.

Indices in Lua start at 1.

Initialization: "all tables are created equal; constructors only affect their initialization. "

The most general constructor expression is of the format:

{["x"]=0, ["y"]=1}.

The part within brackets can be an arbitrary expression, so also the result of some function, which is useful for automatically generating tables.

There are two shorthand styles for constructors: list-style and record-style.

The above expression in record-style is {x=y, y=1}.

List-style looks like this: { "red", "green", "blue" }.

All elements provided in this manner are automatically mapped onto an index expression that starts at 1. So the above is equivalent to:

{[1]="red", [2]="green", [3]="blue"}

These two styles can be mixed an matched. The following expression creates the first two elements record-style, and the last three list-style:

comb = {x=10, y=45; "one", {x=1, y=2}, "three"}

The use of the semicolon is identical to the use of the comma, but is conventionally used to indicate a new style of assignment. This is where the flexibility of the table may be confusing when you are used to accessing elements by index!

print(comb[1]) --> "one" and NOT 10

The table does not have a default index. Selecting the element with index 1 does not select the first element in the table, but the third! What happens is that for the first two elements you explicitly specify the indices "x" and "y":

print(comb['x']) --> 10
print(comb.x) --> 10 (shorthand notation)
print(comb.y) --> 45

For the unnamed values that are assigned with the list-style shorthand, numerical indices are automatically assigned. The element with index 2 is again a table.

print(comb[1]) --> "one" print(comb[2]x) --> 1

Create an array with a table

Arrays are nothing more than tables with numeric indexing. The following creates a zero-indexed array like we often use in other languages, but note that Lua starts indexing at 1 by default.

arr = {}
for i=0, 9 do
    arr[i] = 0
end

It's better to stick to the Lua convention of 1-indexing so you can use default functions. E.g. using a constructor will result in 1-indexing:

arr = {1, 2, 3}

Matrices / multi-dimensional arrays

To create a "true" 2D matrix (as a nested data structure) you have to manually declare each row. This is a bit more cumbersome, but it does give you the flexibility to define a non-square triangle matrix to save memory:

mt = {}          -- create the matrix
for i=1,N do
  mt[i] = {}     -- create a new row
  for j=1,i do   -- replace i with M for a square matrix
    mt[i][j] = 0
  end
end

Alternatively, just use a flat table but manage the indices to support 2D indexing:

mt = {}          -- create the matrix
for i=1,N do
  for j=1,M do
    mt[i*M + j] = 0
  end
end

Sparse matrix

Arrays implemented using tables are sparse by default! Providing a key that has not been set yet will just return nil. See bottom remark here.

Linked List

Tables are dynamic, so just make a table that refers to another table.

list = nil

list = {next = list, value = 1}
list = {next = list, value = 'yolo'}

local l = list
while l do
  print(l.value)
  l = l.next
end

Queues

Do not use table.insert and table.remove, but define as such: https://www.lua.org/pil/11.4.html .

Set

The idiomatic way to define sets in Lua is to add your set elements as indices! To test membership you do not have to search a list of members, but just index the table.

function Set (list)
  local set = {}
  for _, l in ipairs(list) do set[l] = true end
  return set
end

String buffers

Lua uses true garbage collection, which causes problems when reading in a buffer line by line and storing the results in a string. The idiomatic efficient way to read in a string buffer is using io.read("*all"), which uses an underlying C implementation with a better algorithm for concatenating parts into a large string. See here.

Basic syntax and concepts

Assignments

Source: https://www.lua.org/pil/4.1.html

In multiple assignments all values are evaluated before being assigned, so you can do this : x, y = y, x -- swapx' for y'.

If the numbers of variables do not match with the number of values, either a nil is assigned or one of the values is discarded. Python will throw an error in both cases!

This also means that you cannot do the following: a, b, c = 0. This only assigns a.

Scoping

Source: https://www.lua.org/pil/4.2.html

Variables are global unless specified as local.

The scope of local variables is their block.

A block is either a function body or a chunk. A chunk is a string of code or a file where the variable is declared.

Something to take into account: if you run local i = 0 interactively (so not in a file), the scope will not be a file but only this line itself (i.e. this line is a chunk in itself). To get the desired scope, run within a do block.

Important idiom: local foo = foo.

This code creates a local variable, foo, and initializes it with the value of the global variable foo. That idiom is useful when the chunk needs to preserve the original value of foo even if later some other function changes the value of the global foo; it also speeds up access to foo.

You can define a local variable anywhere in a block. The scope only starts at the moment of initialization. The book also says something very interesting about this:

Because many languages force you to declare all local variables at the beginning of a block (or a procedure), some people think it is a bad practice to use declarations in the middle of a block. Quite the opposite: By declaring a variable only when you need it, you seldom need to declare it without an initial value (and therefore you seldom forget to initialize it). Moreover, you shorten the scope of the variable, which increases readability.

You can more explicitly manipulate the scope with a do-end block:

Iterators

Numeric and generic for loop

Numeric for executes from 10, to 1, with step value -1 (optional with default 1):

for i=10,1,-1 do print(i) end

Generic for loop with an iterator over the numerical indices can be done with ipairs(x) (cf. Python's enumerate(x)). Used a do and end statement: for i,v in ipairs(a) do print(v) end.

Because I come from Python, I'm inclined to write a colon, but the do statement replaces this.

Important!: pairs(x) iterators over the keys of a table, whereas ipairs(x) iterates over the numerical indices only.

Due to syntax choices break and return have to be the last statement of a block, so if you want to break in the middle of a function you need to wrap them in their own block as such: do return end.

Iterate over characters in strings

Input string: str = 'hoi'.

There is a shorthand length operator #:

#str --> 3
string.len(str) --> 3

You can use this to define an iterator with a numeric for loop:

for i=1, #str do
    local c = str:sub(i,i)
    print(c)
end

Alternatively get an iterator directly on the string, which is the idiomatic thing to do if you do not need the index:

for c in str:gmatch"." do
    print(c)
end

IO

From HackerRank.

Input:

8
UDDDUDUU

IO example:

local ftpr = io.open(os.getenv("OUTPUT_PATH"), "w")
local steps = io.stdin:read("*n", "*l")
local path = io.stdin:read("*l")
local result = f(steps, path)
ftrp:write(result, "\n")
ftrp:close()

TODO Data files and persistence

Data as code

https://www.lua.org/pil/12.html

Lua is often used as a configuration language, probably because the difference between data en program code is very small so that data can be loaded in easily. You can use table constructors as an alternative to regular file formats, that is you can write data as Lua code.

Example:

Entry{"Donald E. Knuth",
      "Literate Programming",
      "CSLI",
      1992}

Note:

  • This really looks like a plain text format because calls to functions with single arguments can omit brackets Entry ({"a", "b"}) -> Entry {"a", "b"} / Entry{"a", "b"}. But in fact, this is proper Lua code.
  • You can now read in the data and do something with it by defining Entry (b), which then behaves as a callback function that is called in an event-driven manner, i.e. every time a data entry is read in.

Example:

-- Read in all authors
local authors = {}      -- a *set* to collect authors
function Entry (entry) authors[entry[1]] = true end
dofile("data.txt")
for name in pairs(authors) do print(name) end

Self describing data

  • Each field has a short description of its meaning
  • Human understandable
  • Easy to edit by hand
  • Easy to modify the data format because
    • No longer depends on order of data entries
    • Backwards compatible: When adding fields, do an existence check and supply a default value for missing fields

Example:

Entry{
  author = "Donald E. Knuth",
  title = "Literate Programming",
  publisher = "CSLI",
  year = 1992
}

Reading code:

local authors = {}      -- a set to collect authors
function Entry (entry)
  if entry.author then authors[entry.author] = true end
end
dofile("data")
for name in pairs(authors) do print(name) end

TODO Serialization

TODO

Object Oriented Programming in Lua

TODO

TODO Functional programming in Lua

Functions are first-class citizens in Lua and can be passed as arguments to other functions.

To continue.