Product overview. The OPT++ Suite tests whether C++ compiler optimziers
properly perform various optimizations.
The optimizations checked in version 2.0 of the suite include "traditional"
optimizations such as common subexpression elimination, dead code removal, forward substitution, expression simplification,
etc., as well as C++ specific optimizations such as return-value optimization, elimination of virtual function
calls and copy optimization.
Initial results with various C++ compilers have shown that several C++ compilers fail
to optimize several simple code-sequences, especially in the presence of objects that arenot built-in types. The
suite also checks each basic "test" for several built-in as well as user-defined data types, allowing
users to determine several optimizations which pass for some data types and fail for several other data types.
The OPT++ Suite (like all other test suites from Modena), is set up so that each test
includes only one header file and links in with one object file. Each test is completely self-contained. A bourne
shell script is supplied to run all of the tests. Individual tests can be run in "standalone" mode by
writing a trivial script.
Benefits. The OPT++ Suite can be very effectively used to tune the performance of C++ optimizers.
It allows users to pin-point cases in which the optimizer fails to perform common C++ optimizations. The suite
also checks C++ optimizers for correctness, by ensuring that and optimized function returns the same value as the
unoptimized version of the function.
Testing Methodology. The organization of each test in the OPT++ Suite is explained below.
Each test includes two kernels : a basic test kernel and a hand-optimized
source code kernel. The test kernel is run for time_limit seconds, and the number of iterations
it takes to get to complete time_limit seconds is recorded. The hand-optimized kernel is then run for
the same number of iterations as the test kernel. The time taken is recorded.
Test failure is determined as follows:
if (testKernelTime > (optimization_factor * optTime) then
<<< Assume that the test has failed >>>
else
<<< Assume that the test has passed >>>
Optimization_factor and time_limit are constants that are set in the
file "opthdr.h", and can be modified by the user. Optimization_factor should always be a real
number greater than 1. For example, if optimization_factor is 1.2, and testKernelTime = 1.0 and
optTime = 0.9, then, since 1.0 < (0.9 * 1.2), we assume that the test passes. i.e. the compiler executed
the non-optimal test kernel within a time of 20% of the hand-optimized test kernel.
By varying the time_limit and optimization_factor, users can experiment
with different runs of the suite. Users can tell the percentage of inefficiency of performance by determining the
largest optimization_factor for which a test just passes.
All tests in the OPT++ suite are completely "templatized". i.e., they make extensive
use of the C++ class and function template features. Basing the tests on templates makes it much easier to instantiate
the tests for various built-in and user-defined types, and exposes several more optimizer bugs.
Maintenance,
Upgrades and Evaluation. We back up our software with outstanding support. We try to solve any
problems within hours or days. When you call us or send us email, you get one of the developers of the software,
who will work with you to solve problems, whether they are yours or ours. We work with customers and welcome their
suggessions to continuously improve the quality and usefulness of our product.
Updates to test sources typically include seveal new optimization tests, together with bug fixes. Source
updates are made one or two times per year.
Evaluation. Prospective customers with a serious interest in the OPT++ Suite are allowed
to examine the complete suite for a period of 30 days. A simple software confidentiality agreement needs to be
signed for this purpose.
Please contact us via phone or email to obtain the evaluation NDA.
Customers.
The Test++ Suite has already been licensed by several software and computer systems companies. These
include
IBM
Silicon Graphics
Siemens-Nixdorf, AG
and several other systems and software companies. Our client list continues to grow, as more companies realize
the importance of C++ performance testing.
Latest
release. Release 2.0 of the OPT++ suite was made in May 1997. The next major release is scheduled
for spring 1998.
|