Skip to content

Commit

Permalink
[SPARK-14072][CORE] Show JVM/OS version information when we run a ben…
Browse files Browse the repository at this point in the history
…chmark program

## What changes were proposed in this pull request?

This PR allows us to identify what JVM is used when someone ran a benchmark program. In some cases, a JVM version may affect performance result. Thus, it would be good to show processor information and JVM version information.

```
model name	: Intel(R) Xeon(R) CPU E5-2697 v2  2.70GHz
JVM information : OpenJDK 64-Bit Server VM, 1.7.0_65-mockbuild_2014_07_14_06_19-b00
Int and String Scan:                Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
-------------------------------------------------------------------------------------------
SQL Parquet Vectorized                    981 /  994         10.7          93.5       1.0X
SQL Parquet MR                           2518 / 2542          4.2         240.1       0.4X
```

```
model name	: Intel(R) Xeon(R) CPU E5-2697 v2  2.70GHz
JVM information : IBM J9 VM, pxa6480sr2-20151023_01 (SR2)
String Dictionary:                  Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
-------------------------------------------------------------------------------------------
SQL Parquet Vectorized                    693 /  740         15.1          66.1       1.0X
SQL Parquet MR                           2501 / 2562          4.2         238.5       0.3X
```

## How was this patch tested?

Tested by using existing benchmark programs

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

Author: Kazuaki Ishizaki <[email protected]>

Closes apache#11893 from kiszk/SPARK-14072.
  • Loading branch information
kiszk authored and rxin committed Mar 23, 2016
1 parent 4700adb commit 0d51b60
Showing 1 changed file with 17 additions and 1 deletion.
18 changes: 17 additions & 1 deletion core/src/main/scala/org/apache/spark/util/Benchmark.scala
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ private[spark] class Benchmark(

val firstBest = results.head.bestMs
// The results are going to be processor specific so it is useful to include that.
println(Benchmark.getJVMOSInfo())
println(Benchmark.getProcessorName())
printf("%-35s %16s %12s %13s %10s\n", name + ":", "Best/Avg Time(ms)", "Rate(M/s)",
"Per Row(ns)", "Relative")
Expand Down Expand Up @@ -91,16 +92,31 @@ private[spark] object Benchmark {
* This should return something like "Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz"
*/
def getProcessorName(): String = {
if (SystemUtils.IS_OS_MAC_OSX) {
val cpu = if (SystemUtils.IS_OS_MAC_OSX) {
Utils.executeAndGetOutput(Seq("/usr/sbin/sysctl", "-n", "machdep.cpu.brand_string"))
} else if (SystemUtils.IS_OS_LINUX) {
Try {
val grepPath = Utils.executeAndGetOutput(Seq("which", "grep"))
Utils.executeAndGetOutput(Seq(grepPath, "-m", "1", "model name", "/proc/cpuinfo"))
.replaceFirst("model name[\\s*]:[\\s*]", "")
}.getOrElse("Unknown processor")
} else {
System.getenv("PROCESSOR_IDENTIFIER")
}
cpu
}

/**
* This should return a user helpful JVM & OS information.
* This should return something like
* "OpenJDK 64-Bit Server VM 1.8.0_65-b17 on Linux 4.1.13-100.fc21.x86_64"
*/
def getJVMOSInfo(): String = {
val vmName = System.getProperty("java.vm.name")
val runtimeVersion = System.getProperty("java.runtime.version")
val osName = System.getProperty("os.name")
val osVersion = System.getProperty("os.version")
s"${vmName} ${runtimeVersion} on ${osName} ${osVersion}"
}

/**
Expand Down

0 comments on commit 0d51b60

Please sign in to comment.