Swig, LLVM, LLILC

As I work to enhance Campy, a C# library for GPU programming I wrote, I’m trying to capitalize on some new code from the NET Foundation Projects. These include LLILC, a MS IL compiler based on LLVM. I want to be able to use some of the APIs in LLVM to perform SSA analysis rather than roll my own. But, that turns out to be easier said than done. This note describes some of the issues in building LLILC, LLVM, and SWIG.


SWIG

Simplified Wrapper and Interface Generator (SWIG) is a tool to generate a wrapper in one programming language from the API written in another language. SWIG has the capability to parse C/C++ header files and output C# classes and native C interfaces that perform PInvoke calls for C#. It is a reasonable choice when one wants to create a C# interface to LLVM, or any C++ API for that matter.

However, SWIG is baroque, if not outright grotesque. The rules for type mapping C++ to C# are difficult to understand, even with full debugging enabled (-debug-memory -debug-classes -debug-symtabs -debug-tmsearch -debug-tmused -debug-symbols -debug-csymbols -debug-lsymbols -debug-typemap -debug-template). Conversion of basics like void* -> IntPtr are not handled out of the box.

While SWIG can run on Windows, I find that it is easier to build in a Linux shell, and call the command-line tool via WSL, e.g., bash -c swig …. That said, Swig is one of those tools in which you sometimes need to debug the code in order to understand the type mapping rules–not that that will be easy as almost every object in SWIG is encoded as what the authors call a DOH object— which is just a void*.

Installation:

# Make sure required software installed.
sudo apt-get install gcc libpcre3 libpcre3-dev make 'g++' git automake flex bison
git clone https://github.com/swig/swig.git
cd swig
./autogen.sh
mkdir temp
./configure --prefix=`pwd`/temp
make
make install

Example:

git -clone  https://github.com/kaby76/SwigExample.git
cd SwigExample
bash -c ./generate.sh
# Open in Visual Studio 2017, rebuild, and run.

(Note! If you make your own API, make sure to update the .SLN file with a <ProjectReference> entry. See https://blogs.msdn.microsoft.com/kirillosenkov/2015/04/04/how-to-have-a-project-reference-without-referencing-the-actual-binary/)

Resources:

http://swig.org
https://sourceforge.net/projects/swig/

Documentation:

http://swig.org/Doc3.0/SWIGDocumentation.html


Swigged LLVM

LLVM (Low Level Virtual Machine) is a compiler infrastructure project, and is the defacto tool for compiler writers nowadays (which is too bad, because it lacks some flexibility, and it is better for people to learn and know compiler theory).

Building LLVM on Windows is well documented. However, if you want to call LLVM from C#, you will need to use SWIG to generate a thunking layer, or write your own. In fact, others have done so: LLVMSharp and SharpLang. However, SharpLang is no longer active (as of November 2015), and LLVMSharp does not have LLILC and does not expose the SSA analysis API.

Using SWIG to generate a C# API does require some finesse. A basic SWIG input file might be:

// Input file xxx.i
// To generate files:
// mkdir lib nlib
// swig -csharp -c++ -I./nlib -I../llvm/include -namespace CSLLVM -dllimport CSLLVM.Native.dll -outdir ./lib -o ./nlib/LLVM_wrap.cpp xxx.i

%include "typemaps.i"

%module LLVM

%{
#include <llvm-c/Core.h>
#include <llvm-c/BitReader.h>
#include <llvm-c/BitWriter.h>
#include <llvm-c/Transforms/IPO.h>
#include <llvm-c/Transforms/PassManagerBuilder.h>
#include <llvm-c/Transforms/Scalar.h>
#include <llvm-c/Transforms/Vectorize.h>
#include <llvm-c/Target.h>
#include <llvm-c/TargetMachine.h>
#include <llvm-c/Analysis.h>
#include "Additional.h"
#include "DebugInfo.h"
%}

%include "llvm-c/Core.h"
%include "llvm-c/BitReader.h"
%include "llvm-c/BitWriter.h"
%include "llvm-c/Transforms/IPO.h"
%include "llvm-c/Transforms/PassManagerBuilder.h"
%include "llvm-c/Transforms/Scalar.h"
%include "llvm-c/Transforms/Vectorize.h"
%include "llvm-c/Target.h"
%include "llvm-c/TargetMachine.h"
%include "llvm-c/Analysis.h"
%include "Additional.h"
%include "DebugInfo.h"

However, this will result in many poorly named C# files, e.g., SWIGTYPE_p_LLVMTypeRef.cs. Therefore, one should apply SWIG %typemap’s to rename the types appropriately. SharpLang uses this approach, and I’ve updated the rewrite rules slightly to handle more of the LLVM API. LLVMSharp uses a different approach, using a home-grown tool to parse C/C++ header files and convert them into wrappers.

NB: SWIG/LLVM is so difficult to understand, that I will be exploring it in a later post in this blog.

Installation:

# In CMD, as vcvarsall.bat is required to initialize environmental variables.
git clone http://llvm.org/git/llvm.git
mkdir build
cd build
"%VS150BASE%\VC\Auxiliary\Build\vcvarsall.bat" x64
cmake -G "Visual Studio 15 2017 Win64" ..\llvm
msbuild LLVM.sln /p:Configuration=Debug /p:Platform=x64
# (or you can open LLVM.sln in Visual Studio and rebuild in that environment.)

Example:

https://github.com/kaby76/swigged-llvm

Resources:

http://llvm.org/
https://github.com/llvm-mirror
http://www.telegrid.enea.it/SGI/Altix/Typemaps.pdf
http://swig.org/Doc3.0/SWIGDocumentation.html

Documentation:

http://llvm.org/docs/


LLILC

LLILC is a compiler backend for MS IL based on LLVM. It was started by Microsoft to address ahead-of-time (AOT) compilation of MS IL, different from the JIT compiler, RyuJIT, the compiler backend since Net 4.6. The JIT compiler is not LLVM based.

The main use I would have for LLILC is to convert MS IL into LLVM for code analysis. However, I am not interested in code generation; I want to use the SSA graph to answer some basic questions of what code is referenced when calling a Campy parallel-for.

Unfortunately, LLILC is poorly written code. LLILC merges the parsing of MSIL, construction of a control-flow graph, and translation of MSIL into LLVM IR with each instruction that is parsed. Further, LLILC is CoreCLR dependent, which means this cannot be used with a NET Framework application (the reverse is possible).

Installation:

See https://github.com/dotnet/llilc/blob/master/Documentation/Getting-Started-For-Windows.md

Resources:

https://dotnetfoundation.org/llilc
https://github.com/dotnet/llilc
https://github.com/dotnet/coreclr/blob/master/Documentation/botr/ryujit-overview.md
http://www.c-sharpcorner.com/UploadFile/b26131/ryujit-the-64-bit-compiler-for-net/
https://github.com/dotnet/coreclr/tree/master/src/jit
http://blog.llvm.org/2015/04/llilc-llvm-based-compiler-for-dotnet.html