Crowd sourced benchmarks

Overview

A straightforward way of speeding up your analysis, is to buy a better computer. Modern desktops are relatively cheap, especially compared to user time. However, it isn’t clear if upgrading your computing is worth the cost. The benchmarkme package provides a set of benchmarks to help quantify your system. More importantly, it allows you to compare your timings with other systems.

Getting Started

To benchmark your system just run

library("benchmarkme")
res = benchmark_std(runs = 3)

The runs argument controls the number of replicates. If you have a fast system, then consider increasing this number. Currently it takes around a minute to run a single benchmark on a standard system (so around \(3\) minutes for \(3\) runs).

You can upload the results using

id = upload_results(res)

The object id contains a unique identifier that you can use to extract your dataset in future versions of the package.

Exploring the Results

Past results are included in a companion package and can be accessed via

data(past_results, package="benchmarkmeData")

The data set is a simple data frame

head(past_results, 3)

##                    id       date     time test_group
## 1 2015-12-14-98680060 2015-12-14  5.91600       prog
## 2 2015-12-14-98680060 2015-12-14 13.11433 matrix_fun
## 3 2015-12-14-98680060 2015-12-14 17.15367 matrix_cal
##                                        cpu        ram byte_optimize
## 1 Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz 8053540000             2
## 2 Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz 8053540000             2
## 3 Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz 8053540000             2
##   r_major r_minor sysname           release blas_optimize
## 1       3     2.2   Linux 3.13.0-43-generic         FALSE
## 2       3     2.2   Linux 3.13.0-43-generic         FALSE
## 3       3     2.2   Linux 3.13.0-43-generic         FALSE

and can plotted via

## shine() # if you have shiny
plot_past()

You can also see how you computer measures up with others

plot(res)

The three sets of graphs generated each correspond to a particular set of benchmarks. See the other vignette for details.

There is also a shiny interface that can be accessed via shine(res).

Interactive Tables

If you crave raw numbers, install the DT package`

install.packages("DT")

and run

## Your result is highlighted in orange.
get_datatable(res)
get_datatable_past()

Input/Output benchmarks

The function benchmark_std attempts to assess computational speed. To assess the speed of file access, use

res_io = benchmark_io()
upload_results(res_io)
## Sample size is small, so don't split by byte/blas
plot(res_io, byte_optimize=NULL, blas_optimize=NULL)

This benchmark creates a \(5\), \(50\) and \(200\) MB data frame and times reading/wrting of the file. By default the files are created in a temporary directory specified as

Sys.getenv("TMPDIR")

Depending on your set-up, this can be on a network drive. You can change where the files are created via