I recently worked on configuring and enabling Ehcache with our Hibernate objects for common domain objects we query on almost every page request. I was interested in whether this would improve performance, and how best to illustrate the effect of the pages across threads and actions.

I had written some JMeter tests I am able to automatically run with different thread loads (5-60 threads stepping by 5) so I generated some response time data for before and after enabling the cache. Although there is actually many more rows and columns, in general the data looks like:
label elapsed threads

Action Elapsed Threads
NtSP 25 10
DLgn 11 10
FtDV 6 10
NtMM 16 10
NtSS 20 15
SrfS 157 10


I then read the log files into two data frames, r1 for the pre-cache times and r2 for the post-cache times. Using tapply I then generated two matrices of mean response times by thread and action (r1 becomes m1, r2 becomes m2). From that point it is just a matter of subtracting the m1 matrix from m2 to obtain the differences measured.

z <- r1
m1 <- tapply(z$elapsed, list(z$label, z$threads), mean)
z <- r2
m2 <- tapply(z$elapsed, list(z$label, z$threads), mean)
m.diff <- m2 - m1

Because the final output of the previous calculations is a two-dimensional matrix, a heatmap or similar visualization style seemed like it might be appropriate. In the real version the action names are not obfuscated.

# Calculate breaks using midpoints of matrix histogram with initial extension
x <- c(-20000, hist(m.diff)$mids)
levelplot(t(m.diff), scales=list(cex=0.7), aspect="iso", col.regions=heat.colors, pretty=F,at=x,xlab='Threads',ylab='Action',main="Time Difference With Cache")

metrics <- read.csv('some_metrics.csv',header=T)
aggregate(dce$elapsed, by=list(CLAZZ=dce$clazz,METHOD=dce$method), median) -> medians
aggregate(dce$elapsed, by=list(CLAZZ=dce$clazz,METHOD=dce$method), mean) -> means
aggregate(dce$elapsed, by=list(CLAZZ=dce$clazz,METHOD=dce$method), min) -> mins
aggregate(dce$elapsed, by=list(CLAZZ=dce$clazz,METHOD=dce$method), max) -> maxes
aggregate(dce$elapsed, by=list(CLAZZ=dce$clazz,METHOD=dce$method), length) -> lengths
aggregate(dce$elapsed, by=list(CLAZZ=dce$clazz,METHOD=dce$method), sum) -> sums
s <- mins
s$MIN <- s$x
s$x <- NULL
s$MAX = maxes$x
s$MEAN = means$x
s$MEDIAN = medians$x
s$NUM = lengths$x
s$SUM = sums$x
rm(mins,means,maxes,medians,sums)

This was obviously less than ideal, although I could wrap this in a function it is a bit ugly and cumbersome. I searched the R-help mailing list and found some references to the doBy package, which “grew out of a need to calculate groupwise summary statistics in a simple way”. The summaryBy function in this package turned out to be exactly what I needed and simplified by code to:

summarize <- function(csvfile) {
	require(doBy)
	metrics.csv <- read.csv(csvfile,header=T)
	metrics <- summaryBy(elapsed ~ clazz + method, data=metrics.csv, FUN=c(mean,median,min,max,sum,length))
	write.csv(metrics, file='export.csv', quote=F, row.names=F)
	metrics
}
metrics <- summarize('some_metrics.csv')

I realized it should be pretty easy to use the approach as the previous R map I generated to make a smaller scale map at a country level. In this example I’m setting up an initial non-plotted map with limits on the X and Y ranges I want to display. Next I assign colors to the observations of this subset map, plot these filled areas, then plot all boundaries.

I’m not really happy with this map because (1) I don’t really like just throwing out maps with just geographic coordinates, and (2) using area coloring can be a bit misleading about the magnitude and dispersal of the data presented: in this case there are really small clusters of reported cases, not country wide. I think this map might be a good index map to get an overview of the spread and number of cases, but would need to be backed by more detailed maps that do not generalize the location of outbreaks as much.

Here’s the source:

# load required libraries
require(maps)
require(RColorBrewer)

# Create a dataframe with the reported observations
loc <- c('USA', 'Mexico', 'Canada', 'Spain')
cases <- c(40, 26, 6, 1)
flu <- data.frame(loc,cases)

# Setup the coordinate system
m <- map("world",plot=F, xlim=c(-180,5),ylim=c(10,90), fill=T)

# Match up our observations
stm <- match.map(m, flu$loc)

# Rank the cases and assign colors using the RColorBrewer YlOrRd palette
flu$rank <- rank(flu$cases, ties="min")
pal <- brewer.pal(max(flu$rank),"YlOrRd")
color <- pal[flu$rank]
flu.color <- color[stm]

# Do the drawing
map(m,col=flu.color,fill=T, lty=0,boundary=F,interior=F) # fill regions
map('world',interior=T,add=T,col="grey30") # plot boundaries
map.axes()
grid(col="grey50")
title("WHO Confirmed Cases of Swine Flu by Country (28 April 2009)")
legend('bottomleft', legend=paste(flu$loc,flu$cases),
	fill=color, bg="white", horiz=T, cex=0.75,
	title=paste("Jason B. Smith | 28 April 2009 | Source: CNN.com"))

Here is the R code I used to generate the map.

draw.flumap <- function() {
	# load required libraries
	require(maps)
	require(RColorBrewer)

	# Create a dataframe with the reported observations
	states <- c('California','Kansas','New York','Ohio','Texas')
	cases <- c(7,2,28,1,2)
	flu <- data.frame(states,cases)

	# Match up our observations with the "state" database
	stm <- match.map("state", states)

	# Rank the cases and assign colors using the RColorBrewer YlOrRd palette
	flu$rank <- rank(flu$cases, ties="min")
	pal <- brewer.pal(max(flu$rank),"YlOrRd")
	color <- pal[flu$rank]
	flu.color <- color[stm]

	# Actually do the drawing
	map("state",proj="albers",col=flu.color, parameters=c(30,40),fill=T,lwd=0.7)
	title("US Confirmed Cases of Swine Flu by State (27 April 2009)")
	legend('bottomleft', legend=paste(flu$states,flu$cases),fill=color, cex=0.75)
	text(0.26, -1.62, labels=paste("Albers Equal Area\n","Jason B. Smith\n", "27 April 2009"), cex=0.75)
}
# Just saves it out as a file
png(filename="~/Desktop/flu.png", width=600, height=500, bg="white")
print(draw.flumap())
dev.off()

The draw.flumap function has everything you would need to do this in an interactive R console, I really just wrapped it at the end to make it easier to save the image to a file. I had to make use of the locator() function for placement of the text.