Thanos Stratikopoulos
- Dec 20, 2023
- 4 min read

Hardware Acceleration for Polyglot Runtimes

TornadoVM can be used with the GraalVM Truffle Polyglot API to invoke TaskGraphs from guest programming languages such as Python, Ruby, etc. This blog aims to showcase how to execute TornadoVM code that is offloaded on a GPU through code written in Python, JavaScript, and Ruby. In a nutshell, it has the following objectives:

Describing polyglot programming.
Discussing briefly the changes introduced in GraalVM 23.1.0 and the impact in TornadoVM.
Explaining how programmers can combine polyglot programming with hardware acceleration via TornadoVM.
Providing examples of programs that perform a computation with TornadoVM on GPUs, from Python, JavaScript and Ruby.

1. Polyglot Programming

Polyglot programming has been re-ignited by the Truffle Language Implementation Framework to enable the interoperability of Java with other programming languages, such as Python, JavaScript, Ruby, etc. A runtime system that can interoperate with multiple programming languages can increase maintainability and the performance of the underlying runtime.

2. Recent Changes from GraalVM 23.1.0

Since GraalVM 23.1.0, there have been three important changes as described in a dedicated GraalVM blog post:

The GraalVM Updater is removed from the distribution of GraalVM without replacement.
All polyglot language runtimes shipped by GraalVM can now be used as Java libraries from Maven Central.
More standalone distributions are shipped for languages, such as Node and LLVM, that did not have such a distribution. Additionally, dedicated builds are provided with a pre-installed GraalVM JDK called JVM-standalone for every language.

Note: Following the last change, the GraalVM JDK that is pre-installed with the standalone distributions does not include the compiler modules of Graal. Instead, it is built with libGraal (i.e., the Graal compiler compiled as a native library) to reduce the disk footprint. This has an impact when running with TornadoVM since the standalone distribution must be rebuilt to include the compiler modules of Graal.

The aforementioned changes enable users to use polyglot runtime implementations in two ways:

As Java libraries via Maven Central.
As standalone toolkits.

The former way enables Java programmers to use the polyglot runtime implementations as Java libraries. For instance, Java programmers can import the GraalPy dependency from Maven Central, and then embed and execute a code segment expressed in Python from Java. To do so, Java programmers must create a context object (org.graalvm.polyglot.Context) and declare the embedded programming language, as shown below:

Context context = Context.create();
context.eval("python", "print('Hello polyglot world from Python!')");
context.close();

On the other hand, the latter way enables programs written in Python, JavaScript and other languages to execute code written in Java. Python programmers can use the java module, and access any classes that are available in the classpath of the JVM standalone (i.e., the JVM distributed with the toolkit). For example:

import java
myclass = java.type("uk.ac.manchester.tornado.examples.polyglot.MyCompute")

However, this is possible only when the standalone toolkit is downloaded as JVM standalone (i.e., contains the --jvm suffix in the name graalpy-jvm-<version>-<os>-<arch>.tar.gz).

3. Combining Polyglot Programming with Hardware Acceleration

In this section, we will show how programmers can use both ways to access a TornadoVM task and execute it on a GPU, from Python, JavaScript and Ruby.

3.1 Using Polyglot Runtimes from Maven Central

Using the distributions of polyglot runtime implementations as Java libraries is an elegant and very familiar way for Java programmers who want to express a part of their codebase in a different programming language, and they do not want to change the programming environment (i.e., the runtime system).

In TornadoVM, programmers can leverage the polyglot runtime dependencies for Python, JS, and Ruby, via a new maven profile that is invoked during the built time if programmers use the --polyglot flag:

$ git clone https://github.com/beehive-lab/TornadoVM.git
$ ./bin/tornadovm-installer --jdk graalvm-jdk-21 --backend opencl --polyglot
$ source setvars.sh

Note: The example builds only the OpenCL backend. However, programmers can build TornadoVM with any backend (OpenCL, SPIR-V, PTX) and the polyglot runtime dependencies will be added accordingly.

Once TornadoVM is built, programmers can try the TornadoVM examples that access a Java class (uk.ac.manchester.tornado.examples.polyglot.MyCompute). The examples create contexts and embed programs written in Python, JavaScript or Ruby to access the MyCompute class. This class implements the compute method that uses the TornadoVM API to create two vectors, initialise them and perform the multiplication on a GPU.

For instance, the HelloPython.java example contains the following code to access and run the compute method of the MyCompute class from a Python context.

public static void runTornadoFromPython() {
   try (Context context = Context.newBuilder().allowAllAccess(true).build()){
       // @formatter:off
       float[] v = context.eval("python",
               "import java\n" +
               "myclass = java.type('uk.ac.manchester.tornado.examples.polyglot.MyCompute')\n" +
                       "output = myclass.compute()\n" +
                       "print(output.toString())\n" + "output")
               .asHostObject();
       // @formatter:on
       System.out.println(Arrays.toString(v));
   }
}

The Java implementation of the compute method is as follows:

public static float[] compute() {
   final int N = 512;
   float[] a = new float[N * N];
   float[] b = new float[N * N];
   float[] c = new float[N * N];

   IntStream.range(0, N * N).sequential().forEach(i -> {
       a[i] = 2.0f;
       b[i] = 1.4f;
   });
   if (executor == null) {
       TaskGraph taskGraph = new TaskGraph("s0") //
               .transferToDevice(DataTransferMode.EVERY_EXECUTION, a, b)//
               .task("t0", MyCompute::mxm, a, b, c, N)//
               .transferToHost(DataTransferMode.EVERY_EXECUTION, c);//


       ImmutableTaskGraph immutableTaskGraph = taskGraph.snapshot();
       executor = new TornadoExecutionPlan(immutableTaskGraph);
   }
   executor.execute();
   return c;
}

The code snippet of the compute method shows the TornadoVM API for the declaration of a TaskGraph that contains a task (MyCompute::mxm) that is compiled at runtime, and the declaration of a TornadoExecutionPlan to run on a GPU.

Note: The declaration of a TaskGraph can occur once, as long as the shape of the task remains unmodified. The execution plan can be executed every time that the compute method is invoked, and the input/output data will be transferred in every execution.

To run the example for HelloPython.java:

$ tornado --debug -m tornado.examples/uk.ac.manchester.tornado.examples.polyglot.HelloPython

To run the example for HelloJS.java:

$ tornado --debug -m tornado.examples/uk.ac.manchester.tornado.examples.polyglot.HelloJS

To run the example for HelloRuby.java:

$ tornado --debug -m tornado.examples/uk.ac.manchester.tornado.examples.polyglot.HelloRuby

3.2 Using Polyglot Runtimes as Standalone Toolkits

The standalone distributions of the GraalVM runtime implementations for GraalPy, GraalVM JavaScript, and TruffleRuby are available on GitHub. To access Java classes from those toolkits, you must download them as JVM standalone (i.e., contain the --jvm suffix in the name of toolkit <toolkit>-jvm-<version>-<os>-<arch>.tar.gz). However, the available JVM standalone toolkits are not compatible to run with TornadoVM. The reason is that they do not include the compiler modules of Graal, as mentioned earlier.

Thus, users must rebuild the toolkits.

Note: The process of rebuilding GraalPy, GraalVM JavaScript and TruffleRuby from the source code is described here. To simplify the building process, we have built and upstreamed in docker-hub, three docker images that offer TornadoVM shipped with the GraalPy, GraalVM JavaScript and TruffleRuby distributions.

The JAVA_HOME and the <TOOLKIT>_HOME variables are already set in the containers, as described here.

3.2.1 Pull the docker images

GraalPy-TornadoVM Docker Image

$ docker pull beehivelab/tornadovm-polyglot-graalpy-23.1.0-nvidia-opencl-container:latest

GraalJS-TornadoVM Docker Image

$ docker pull beehivelab/tornadovm-polyglot-graaljs-23.1.0-nvidia-opencl-container:latest

TruffleRuby-TornadoVM Docker Image

$ docker pull beehivelab/tornadovm-polyglot-truffleruby-23.1.0-nvidia-opencl-container:latest

3.2.2 Run the examples via docker

We have prepared some examples that show how to execute TornadoVM code that is offloaded on a GPU through code written in Python, JavaScript, and Ruby. The examples are available in the docker-tornadovm GitHub repository.

$ git clone https://github.com/beehive-lab/docker-tornadovm
$ cd docker-tornadovm

For each polyglot runtime implementation, we provide a launcher script (tornadovm-polyglot.sh) that facilitates the launching of the container or the execution of any (Python, JavaScript, Ruby) program. Below, we show how to run programs that interoperate with TornadoVM to execute a matrix multiplication operation on a GPU.

Run a Python program with GraalPy-TornadoVM:

## Launch the container
$ ./polyglotImages/polyglot-graalpy/tornadovm-polyglot.sh

## Run Matrix Multiplication from a Python program.
$ ./polyglotImages/polyglot-graalpy/tornadovm-polyglot.sh tornado --printKernel --truffle python example/polyglot-examples/mxmWithTornadoVM.py

Run a JavaScript program with GraalJS-TornadoVM:

## Launch the container
$ ./polyglotImages/polyglot-graaljs/tornadovm-polyglot.sh

## Run Matrix Multiplication from a JavaScript program.
$ ./polyglotImages/polyglot-graaljs/tornadovm-polyglot.sh tornado --printKernel --truffle js example/polyglot-examples/mxmWithTornadoVM.js

Run a Ruby program with TruffleRuby-TornadoVM:

## Launch the container
$ ./polyglotImages/polyglot-truffleruby/tornadovm-polyglot.sh

## Run Matrix Multiplication from a Ruby program.
$ ./polyglotImages/polyglot-truffleruby/tornadovm-polyglot.sh tornado --printKernel --truffle ruby example/polyglot-examples/mxmWithTornadoVM.rb

Note: If you use the launcher script with tornado, you can append the running command with any tornado flags, to print the kernel, enable the TornadoVM profiler, as usual. More info about the flags is provided here.