I have sent patches to GNU Autoconf to add new macros for testing Erlang modules, include files, and functions: AC_ERLANG_CHECK_MOD, AC_ERLANG_CHECK_HEADER,AC_ERLANG_CHECK_LIB_HEADER, and AC_ERLANG_CHECK_FUNC. I also sent a patch to fix the AC_RUN_IFELSE macro which executes Erlang test code, so that this macro cleanly fails if the code doesn’t compile, and another patch to fix the AC_COMPILE_IFELSE macro, which tests that Erlang test code compiles (this is a long known Autoconf bug).
Only the AC_RUN_IFELSE patch has been committed so far. The other patches are functional, but the Autoconf maintainers would prefer a better integration with the other Autoconf macros, for instance by making Autoconf’s AC_CHECK_HEADER macro working for Erlang code when the test language is set to Erlang, which could be used like that:
AC_LANG([Erlang]) AT_CHECK_HEADER([eunit/include/eunit.hrl])
I agree that this is the right way to go, and it will benefit the other supported languages, but it will take me a lot more time to implement.
Ralf Wildenhus, one of GNU Autoconf’s maintainers, has also found a bug in the AT_CHECK_EUNIT macro, which executes EUnit tests from a GNU Autotest testsuite: the macro was always failing when used with Erlang/OTP versions prior to R13. We finally fixed that bug, so that AT_CHECK_EUNIT will work with prior versions of Erlang/OTP, when it will be first available in the next release of Autoconf. Users don’t have to worry about the details of that bug, but I just want to share below my frustration of using Erlang to implement command-line tools.
The problem with AT_CHECK_EUNIT was that the Erlang module generated internally by AT_CHECK_EUNIT for calling EUnit was calling function init:stop/1 (taking an exit code as an argument). But that function was introduced only in R13, and only init:stop/0 (without argument) was available in previous versions. There are only three high-level ways to stop the Erlang VM:
- by calling
halt/1, which takes the exit code as an argument, and abruptly exits the VM with that exit code without flushing the standard output; - by calling
init:stop/0, which cleanly flushes the standard output, and always exits the VM with exit code 0; - by calling
init:stop/1, which both cleanly flushes the standard output, and exits the VM with the exit code given as an argument, but is available only from version R13.
In AT_CHECK_EUNIT, we need both to return custom exit codes, to integrate with Autotest, and to flush the standard output, to let Autotest capture the whole output produced by EUnit. So only init:stop/0 can be used. Internally, the Erlang test module was initially run within Autotest’s standard AT_CHECK macro, basically like:
AT_CHECK([erl -s foobar start], [0])
AT_CHECK checks the exit code of the command in argument, and compares it to the expected exit code (here, 0). If it is the expected exit code, then the test succeeds. If the exit code is 77, the test is skipped, e.g. to indicate that a requirement to the test is not met. Otherwise, the test fails.
In the executed Erlang module, we determine that the exit code is 77 if EUnit is not available (i.e., if module eunit cannot be loaded); otherwise if EUnit was run successfully and the EUnit test passed it is 0; otherwise it is 1. That exit code was passed to init:stop/1 to exit the VM. In the fixed version of AT_CHECK_EUNIT, the module now writes out the exit code into a temporary file, and then calls init:stop/0 (and therefore the VM always exits with code 0 and normally never fails). The real exit code is then read from the file and checked in a subsequent test, like:
AT_CHECK([erl -s foobar start], [0]) AT_CHECK([test -f tempfile && (exit `cat tempfile`)])