The

Ruby Logo is Copyright (c) 2006, Yukihiro Matsumoto. It is licensed under the terms of the Creative Commons Attribution-ShareAlike 2.5 agreement.

JRuby+Truffle
Why its important to optimise the tricky parts

Chris Seaton
Research Manager
Oracle Labs
2 June 2016

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement
The following is intended to provide some insight into a line of research in Oracle Labs. It
is intended for information purposes only, and may not be incorporated into any contract.
It is not a commitment to deliver any material, code, or functionality, and should not be
relied upon in making purchasing decisions. Oracle reserves the right to alter its
development plans and practices at any time, and the development, release, and timing
of any features or functionality described in connection with any Oracle product or
service remains at the sole discretion of Oracle. Any views expressed in this presentation
are my own and do not necessarily reflect the views of Oracle.

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Ruby
Imperative
Scripting (Perl)
Object-oriented (Smalltalk)
Batteries included

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
MRI
Simple bytecode interpreter
Implemented in C
Core library implemented in C

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 The JRuby logo is copyright (c) Tony Price 2011, licensed under the terms of Creative Commons Attribution-NoDerivs 3.0 Unported (CC BY-ND 3.0)

JRuby
JITs by emitting JVM bytecode
VM in Java
Core library mostly in Java

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 The Rubinius logo is copyright 2011 Shane Becker, licensed under the terms of Creative Commons Attribution-NoDerivatives 4.0 International CC BY-ND 4.0

Rubinius
JITs by emitting LLVM IR
VM in C++
Core library mostly in Ruby

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

+ Truffle and Graal
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
 Lexer Lexer
Lexer
Lexer
Parser Parser
Parser
Bytecode
Intermediate rep.
AST JIT
Bytecode generator
Core library Primitives
Core library
Core library
Core library

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

100% Compatibility with the
language (spec/ruby)

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

90% Compatibility with the
core library (spec/ruby)

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
 ?
Why arent you using more of JRuby?
Such as the existing Java implementation
of the core library?

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

What makes Ruby difficult to
optimise?

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

How do people want to
write Ruby?

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 20
Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 21
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 23
Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 24
Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 25
Why cant a conventional VM
optimise this?

Why cant JRuby make this as fast

as we want?

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

First problem: JRubys core library is
megamorphic

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Second problem: JRubys core library is
stateless

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Third problem: JRubys core library is
very deep

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Fourth problem: JRubys core library
isnt amenable to optimisations

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
The same problems apply to Rubinius,
even though the core library is mostly
written in Ruby

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Interlude: Truffle and Graal

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Hotspot

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Hotspot

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 Hotspot

JIT

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 Hotspot

JIT

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

x + y * z load_local x
load_local y
load_local z
call :*
call :+
+
pushq %rbp
movq %rsp, %rbp
x * movq
movq
%rdi, -8(%rbp)
%rsi, -16(%rbp)
movq %rdx, -24(%rbp)
movq -16(%rbp), %rax
movl %eax, %edx
y z movq
imull
-24(%rbp), %rax
%edx, %eax
movq -8(%rbp), %rdx
addl %edx, %eax
popq %rbp
ret

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

x + y * z load_local x
load_local y
load_local z
call :*
call :+
+
pushq %rbp
movq %rsp, %rbp
x * movq
movq
%rdi, -8(%rbp)
%rsi, -16(%rbp)
movq %rdx, -24(%rbp)
movq -16(%rbp), %rax
movl %eax, %edx
y z movq
imull
-24(%rbp), %rax
%edx, %eax
movq -8(%rbp), %rdx
addl %edx, %eax
popq %rbp
ret

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 Hotspot
JIT
JIT

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Truffle

Hotspot
JIT
JIT

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 Node Rewriting
U for Profiling Feedback

U U Node Transitions

Uninitialized Integer
U
U U I

S D
AST Interpreter
Uninitialized Nodes String Double
G
Generic

T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,

and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 Node Rewriting
U for Profiling Feedback G

U U Node Transitions I G

Uninitialized Integer
U
U U I I I

S D
AST Interpreter AST Interpreter
Uninitialized Nodes String Double Rewritten Nodes
G
Generic

T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,

and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Rewriting Compilation using
ng Feedback G Partial Evaluation
G

I G I G

Uninitialized Integer I I

I I I

D
AST Interpreter
Double Compiled Code
Rewritten Nodes

Generic

T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,

and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

codon.com/compilers-for-free
Presentation, by Tom Stuart, licensed under a Creative Commons Attribution ShareAlike 3.0

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 55

 Node Rewriting Compilation using
U for Profiling Feedback G Partial Evaluation
G

U U Node Transitions I G I G

Uninitialized Integer I I
U
U U I I I

S D
AST Interpreter AST Interpreter
String Double Compiled Code
Uninitialized Nodes Rewritten Nodes
G
Generic

T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,

and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 Deoptimization Node Rewriting to Update
to AST Interpreter G Profiling Feedback
G

I G I G D

I I

I I I

T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,

and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

03/06/2016 Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
 Node Rewriting to Update Recompilation using
G Profiling Feedback G Partial Evaluation
G

I G D G D G

I D

I I D

T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,

and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

03/06/2016 Copyright 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential Internal/Restricted/Highly Restricted
 Deoptimization Node Rewriting to Update Recompilation using
to AST Interpreter G Profiling Feedback G Partial Evaluation
G G

I G I G D G D G

I I I D

I I I D

T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,

and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

03/06/2016 Copyright 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential Internal/Restricted/Highly Restricted
Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 60
 Frequently executed call

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 61

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 62
 double

BigInteger
int

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 63

 double

BigInteger
int

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 64

 double

BigInteger

int

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 65

x + y * z load_local x
load_local y
load_local z
call :*
call :+
+
pushq %rbp
movq %rsp, %rbp
x * movq
movq
%rdi, -8(%rbp)
%rsi, -16(%rbp)
movq %rdx, -24(%rbp)
movq -16(%rbp), %rax
movl %eax, %edx
y z movq
imull
-24(%rbp), %rax
%edx, %eax
movq -8(%rbp), %rdx
addl %edx, %eax
popq %rbp
ret

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

x + y * z load_local x
load_local y
load_local z
call :*
call :+
+
pushq %rbp
movq %rsp, %rbp
x * movq
movq
%rdi, -8(%rbp)
%rsi, -16(%rbp)
movq %rdx, -24(%rbp)
movq -16(%rbp), %rax
movl %eax, %edx
y z movq
imull
-24(%rbp), %rax
%edx, %eax
movq -8(%rbp), %rdx
addl %edx, %eax
popq %rbp
ret

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Will I be able to use Truffle
and Graal for real?

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 JS R Ruby

Truffle Java

Graal
JVMCI
(JVM Compiler Interface)
Hotspot C++

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

JS R Ruby

Truffle via Maven etc

Graal

Hotspot Java 9

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 otn graal
03/06/2016 Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
How Truffle solves the problem
of optimising Ruby

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

First problem: JRubys core library is
megamorphic

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 Node Rewriting Compilation using
U for Profiling Feedback G Partial Evaluation

U U Node Transitions I G I

Uninitialized Integer I
U
U U I I I

S D
AST Interpreter AST Interpreter
String Double Compi
Uninitialized Nodes Rewritten Nodes
G
Generic

T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,

and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,
and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Second problem: JRubys core library is
stateless

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 Node Rewriting Compilation using
for Profiling Feedback G Partial Evaluation
G

Node Transitions I G I G

Uninitialized Integer I I
U
I I I

S D
AST Interpreter
String Double Compiled Code
Rewritten Nodes
G
Generic
T. Wrthinger, C. Wimmer, A. W, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon,
and M. Wolczko. One VM to rule them all. In Proceedings of Onward!, 2013.

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
send :bar
send :foo

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 80

send :bar
send :foo

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 81

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Third problem: JRubys core library is
very deep

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

send :bar
send :foo

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 84

Fourth problem: JRubys core library isnt
amenable to optimisations

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
A simple example

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

def min(a, b)
[a, b].sort[0]
end

puts min(2, 8)

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 89

def min(a, b)
[a, b].sort[0]
end

puts [2, 8].sort[0]

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 90

t0 = 2 <=> 8
t1 = t0 < 0 ? 2 : 8
t2 = t0 > 0 ? 8 : 2
t3 = [t1, t2]

puts t3[0]

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 91

t0 = 2 <=> 8
t1 = t0 < 0 ? 2 : 8
t2 = t0 > 0 ? 8 : 2
t3 = [t1, t2]

puts t1

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 92

t0 = -1
t1 = t0 < 0 ? 2 : 8

puts t1

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 93

t0 = -1
t1 = -1 < 0 ? 2 : 8

puts t1

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 94

t1 = true ? 2 : 8

puts t1

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 95

t1 = 2

puts t1

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 96

t1 = 2

puts 2

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 97

puts 2

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 98

t0 = a <=> b
t1 = t0 < 0 ? a : b

puts t1

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 99

t0 = a <=> b
t1 = (a <=> b) < 0 ? a : b

puts t1

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 100
t1 = (a <=> b) < 0 ? a : b

puts (a <=> b) < 0 ? a : b

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 101
puts (a <=> b) < 0 ? a : b

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 102
A deliberately extreme example

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
 = 22 !
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Node

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Control Data
flow flow

Node

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
C extensions

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

C extensions are a hack to workaround
performance, but now they stop us
really fixing performance

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

A lot of this has been about removing
barriers to the excellent optimisations
we already have

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
cmyk_to_rgb

psd_native_util_clamp

FIX2INT

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 126
cmyk_to_rgb

psd_native_util_clamp

FIX2INT

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 127
 C Extension Performance for psd_native and oily_png
35

Average Speedup Relative to MRI

30

Without C Extension (s/s)

25

20

15

10

Matthias Grimmer, Chris Seaton, Thomas Wuerthinger, Hanspeter Moessenboeck:

Dynamically Composing Languages in a Modular Way: Supporting C Extensions for Dynamic Languages
Modularity '14 Proceedings of the 14th International Conference on Modularity

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Conclusions

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

The blocker for performance of idiomatic
Ruby code is the core library, not basic
language features

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

This extends to everything that forms a
barrier including C extensions

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Specialisation, splitting, inlining, partial
evaluation, inline caching are all
solutions to this problem

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Truffle makes it easy to add these to a
language implementation

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

 Can result in an order of magnitude
performance increase with reasonable
effort

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |
Acknowledgements
Benoit Daloze Oracle Labs (continued) JKU Linz University of California, Irvine
Kevin Menard Michael Van de Vanter Prof. Hanspeter Mssenbck Prof. Michael Franz
Petr Chalupa Adam Welc Josef Eisl Codrut Stancu
Till Westmann Thomas Feichtinger Gulfem Savrun Yeniceri
Oracle Labs Christian Wimmer Matthias Grimmer Wei Zhang
Danilo Ansaloni Christian Wirth Christian Hub
Stefan Anzinger Paul Wgerer Josef Haider Purdue University
Daniele Bonetta Mario Wolczko Christian Hube Prof. Jan Vitek
Matthias Brantner Andreas W David Leopoltsederr Tomas Kalibera
Laurent Dayns Thomas Wrthinger Manuel Rigger Romand Tsegelskyi
Gilles Duboscq Stefan Rumzucker Prahlad Joshi
Michael Haupt Oracle Labs Interns Bernhard Urban Petr Maj Lei Zhao
Christian Humer Shams Imam
Mick Jordan Stephen Kell University of Edinburgh T. U. Dortmund
Peter Kessler Gero Leinemann Christophe Dubach Prof. Peter Marwedel
Hyunjin Lee Julian Lettner Juan Jos Fumero Alfonso Ranjeet Helena Kotthaus
David Leibs Gregor Richards Singh Ingo Korb
Tom Rodriguez Robert Seilbeck Toomas Remmelg
Roland Schatz Rifat Shariyar University of California, Davis
Chris Seaton LaBRI Prof. Duncan Temple Lang
Doug Simon Oracle Labs Alumni Floral Morandat Nicholas Ulle
Lukas Stadler Erik Eckstein
Christos Kotselidi

Copyright 2016, Oracle and/or its affiliates. All rights reserved. | 136
Safe Harbor Statement
The preceding is intended to provide some insight into a line of research in Oracle Labs. It
is intended for information purposes only, and may not be incorporated into any contract.
It is not a commitment to deliver any material, code, or functionality, and should not be
relied upon in making purchasing decisions. Oracle reserves the right to alter its
development plans and practices at any time, and the development, release, and timing
of any features or functionality described in connection with any Oracle product or
service remains at the sole discretion of Oracle. Any views expressed in this presentation
are my own and do not necessarily reflect the views of Oracle.

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |

Copyright 2016, Oracle and/or its affiliates. All rights reserved. |