Abstract
Optimizations are an omnipresent topic when working on Virtual Machines (VMs). There is a
plethora of different optimizations available ranging from simple tweaks and tricks to full
evaluation concepts requiring a complex infrastructure. Depending on the complexity of an
optimization and the performance increase it is important to choose the right kinds of optimizations.
Based on a high-level language VM as a case study we argue in favor of transparent optimizations
which do not require changes in the interpreted language's semantics. Furthermore it is necessary to
write and properly evaluate benchmarks to be able to track the performance impact of a certain
optimization. When building a high-level language VM the underlying system -- traditionally a C or
C++ core -- does not share many concepts with the implemented language. Hence some optimizations
emerging from the low-level VM core are orthogonal to the high-level concepts of the interpreted
language. Focusing on such optimizations can strongly limit the dynamic capabilities of a high-level
language. These non-transparent optimizations require the semantics of the interpreted language to be
changed. Changes in the language's semantics can require extensive changes in the sources which is an
undesired property. However transparent optimizations preserve semantics of the language. Using
transparent optimizations helps to separate the low-level requirements of the VM form the high-level
design decisions of the language. We argue that non-transparent optimizations should only be applied
to a high-level language VM in an early development stage. Furthermore each non-transparent
optimization should be paired with a compatible way to reintroduce the lost or altered semantics. To
make valid statements about optimizations it is necessary to write adequate benchmarks. The
benchmarks have to be reproducible and the evaluation has to be statistically sound, furthermore the
benchmarks should focus on covering specific use cases to help locating performance issues in the VM
code. Benchmarks are to be used in a similar way as unit tests. Optimizations can only be evaluated
in a sound way when the corresponding benchmarks produce deterministic values. Furthermore we state
that focusing on micro benchmarks helps to locate and track performance critical code segments of the
VM.
Users
Please
log in to take part in the discussion (add own reviews or comments).