This week in reinventing the wheel I present the
Lua preprocessor. This is a more or less finished project which allows you to use Lua as a text preprocessor. The basic idea is that we create a file of some kind (my preferred use is C code) and it contains <#lua#>/<#endlua#> blocks.
#include
int main( int argc, char ** argv ){
int n = 1000;
<#lua#>
malloced_vals = { "var1", "var2, "var3" };
map(function(str) write("double * "..str..";) end, malloced_vals);
map(function(str) write(str..." = malloc(sizeof(double*n));") end, malloced_vals);
<#endlua#>
/* Do lots of stuff */
<#lua#>
map(function(str) write("free("..str..")) end, malloced_vals);
<#endlua#>
}
(Note, here the function map is defined elsewhere)
This may seem like a stupid idea - but its not. The issue I was having was that I was writing model neurons in C. Biophysically realistic neurons are constructed by measuring and characterizing the voltage and ion concentration dependence of the channels in the cell wall. A sophisticated model might have tens of different channel types, each with their own set of variables which need to be kept track of and integrated separately. In addition to this, there are variables for synaptic currents and strengths which also must be taken care of.
Vincent four years ago might have tried to represent each neuron as a struct such as:
typedef struct neuron {
double * voltage;
double * channel1;
...
double * channelN;
} neuron;
But using arrays of structs like this can be computationally expensive - you have to pay for those pointer dereferences. In smaller models, I just use malloced straight arrays for each value and hope to name them in a way which makes writing a custom integrator easy. But as I faced more complicated tasks the complexity threatened to overwhelm the meager abstractions I was using. I wrote the lua pre-processor so that I could marshal the added abstraction of Lua and still control how my models compiled to C. Happily, the Lua preprocessor also makes it easy to write Matlab interfaces for C code
1.
The preprocessor is pretty well behaved. When you use the "write" function, the output is automatically indented to match the indentation of the <#lua#> block that produced it (I use only spaces for this, since I never use tabs). Additionally, it supports the ability to log the output into a variable available to the preprocessor code. This allows you, for example, to detect function definitions and automatically generate header files. As with any macro language, it makes debugging somewhat difficult, and I plan to eventually add a feature which keeps track of the line number which generated a line of code and appends it to the beginning of each output line. To make things a little easier, the LPP outputs the current code block in the event of any error with line number and error information.
It is a unix-command line friendly application which accepts standard in and prints to standard out in the absence of arguments, so it can be used within VIM (for example) to dynamically use Lua to produce text files. Without further ado, here is the code (under the GPL). It is proof of how sweet Lua is that it is this short.
#!/usr/local/bin/lua
-- LUA Preprocessor Copyright J.V. Toups 2007
--
--This program is free software; you can redistribute it and/or modify it under
--the terms of the GNU General Public License as published by the Free Software
--Foundation; either version 2 of the License, or (at your option) any later
--version.
--
--This program is distributed in the hope that it will be useful, but WITHOUT
--ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
--FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
--details.
--
--You should have received a copy of the GNU General Public License along with
--this program; if not, write to the Free Software Foundation, Inc., 51
--Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
--
stdout = io.output();
stdin = io.input();
i = 1;
while arg[i] and string.sub(arg[i],1,1) == '-' do
-- Parse command line arguments if we ever care about them
if arg[i] == '--noautoindent' then
_autoindent = false;
i = i + 1
elseif arg[i] == '--noautoendline' then
_autoendline = false;
i = i + 1;
elseif arg[i] == '--baseindent' then
_baseindent = string.rep(' ',arg[i+1]);
i = i + 2;
elseif arg[i] == '--output' or arg[i] == '-o' then
ofname = arg[i+1];
i = i + 2;
else
i = i + 1;
end
end
if not _autoindent then
_autoindent = true;
end
if not _autoendline then
_autoendline = true;
end
if not _baseindent then
_baseindent = '';
end
_replacements = {};
_buffer = {};
_keep_buffer = false;
_indent = '';
--print('*************************************************************')
--print('** **');
--print('** luacpp, a Lua c preprocessor copyright J.V. Toups 2006 **');
--print('** send questions or comments to toups@physics.unc.edu **');
--print('** **');
--print('*************************************************************');
--print('\tThere are '..table.getn(arg)..' argument(s).');
if arg[i] then
print('\tInput file is '..arg[i]);
else
--print('\tInput is standard input.');
end
ifname = arg[i];
--if not ofname then
-- ofname = string.gsub(arg[i],'.luacpp','');
-- if ifname == ofname then
-- ofname = ofname..'luacpp_output'
-- end
--end
if ofname then
print('\tOutput file is '..ofname);
else
--print('\tOutput file is standard output.');
end
--print('');
--print('** Beginning Parse **');
--print('');
if ofname then
outfile= assert(io.open(ofname,'w'));
else
outfile= stdout;
end
if ifname then
infile = assert(io.open(ifname,'r'));
else
infile = stdin;
end
function clear_buffer()
_buffer = {};
end
function write(...)
if _autoindent then
io.write(_baseindent.._indent);
end
for _,v in ipairs(arg) do
io.write(v);
end
if _autoendline then
io.write('\n');
end
end
io.output(outfile);
if ifname then
io.write('/* This file produced from '..ifname..' by luacpp */\n');
io.write('/* which is copy right 2006 J.V. Toups */\n');
end
lines = infile:lines();
for line in lines do
if string.find(line,'<#lua#>') then
--print(' *** FOUND LUA BEGIN *** ');
--print(line)
_indent = string.sub(line,string.find(line,'%s*'));
--print('Length of indent is '..string.len(_indent));
execlines = '';
numbered_lines = '';
line = lines();
i = 1;
while not string.find(line,'<#endlua#>') do
execlines = execlines..line..'\n'
numbered_lines = numbered_lines..i..' :'..line..'\n';
line = lines();
i = i + 1;
end
exec_chunk, errmsg = loadstring(execlines);
if exec_chunk then
exec_chunk();
else
print('I found an error in this chunk:');
print(numbered_lines);
assert(nil,errmsg);
end
--execlines = assert(loadstring(execlines));
--execlines();
else
rline = line;
for k, v in pairs(_replacements) do
repstr = '([%s%p]-)'..k..'([%s%p]-)';
print(k..': '..v);
rline = string.gsub(rline,repstr,'%1'..v..'%2');
end
io.write(rline..'\n');
if(_keep_buffer) then
table.insert(_buffer,rline);
end
end
end
1: I wrote this before getting into Lisp. Lisp's Macro system allows you to do exactly what I wrote this to do but in just one language.
