Embedded Universal DSL: a good DSL for us, by us

This repository contains the source code for EUDSL, a toolkit for the construction of embedded DSLs, in arbitrary languages, for targeting MLIR.

FYI: this project is currently "alpha" quality.

Currently, there are three components:

1. eudsl-tblgen: Python bindings to LLVM's TableGen library;
2. eudsl-nbgen: A source-to-source translator that translates MLIR headers¹ into direct nanobind bindings;
3. eudsl-py: Direct Python bindings to MLIR, generated using eudsl-nbgen;
   • Currently only TableGen outputs (various *.h.inc files generated by mlir-tblgen) are automatically translated but the eudsl-nbgen tool can be manually run to translate any MLIR header (to varying degrees of success);
   • See projects/eudsl-py/tests for a few examples of what these bindings look like.

Getting started

Python wheels of all the tools are available at the latest release page. They are also pip install-able with .e.g

$ cd <EUDSL_CHECKOUT_DIR>
$ pip install eudsl-py -f https://github.com/llvm/eudsl/releases/expanded_assets/latest

eudsl-py has a slowly growing set of tests @ projects/eudsl-py/tests.

If you don't want to install locally, here is a colab notebook minimal working example.

Developing/building

Strong recommendation: check the CI scripts @ .github/workflows - they do a fresh checkout and build on every commit and are written to be read by a non-CI expert.

Firstly, you need a distribution of LLVM. You can build LLVM from source using our submodule by doing (on Mac/Linux or mingw):

$ cd <EUDSL_CHECKOUT_DIR>
$ ./build_tools/build_llvm.sh

Alternatively you can download a distribution for your platform from the llvm release page.

Then each of the above tools can both be built as a conventional, standalone, CMake project and as a Python wheel. The wheel build looks something like:

$ cd <EUDSL_CHECKOUT_DIR>
$ export CMAKE_PREFIX_PATH=$PWD/llvm-install 
$ pip wheel projects/eudsl-nbgen -w wheelhouse -v
$ pip wheel projects/eudsl-py -w wheelhouse -v --find-links $PWD/wheelhouse

Note, the trailing --find-links $PWD/wheelhouse on pip wheel projects/eudsl-py is because eudsl-nbgen is a dependency of eudsl-py (that can be satisfied using the eudsl-nbgen wheel).

If you want to build an individual tool via CMake you can do something like:

$ cd <EUDSL_CHECKOUT_DIR>
$ pip install -r requirements.txt
$ export CMAKE_PREFIX_PATH=$PWD/llvm-install 
$ cmake -B $PWD/eudsl-nbgen-build -S $PWD/projects/eudsl-nbgen -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$PWD/eudsl-nbgen-install
$ cmake --build "$PWD/eudsl-build" --target install

If you want to build all the tools at once using CMake you can use the root CMakeLists.txt. Note, in this case, eudsl-nbgen will automatically be built prior to eudsl-py.

Footnotes

Footnotes

1. Yes C++ headers... ↩