devjason

I recently worked on configuring and enabling Ehcache with our Hibernate objects for common domain objects we query on almost every page request. I was interested in whether this would improve performance, and how best to illustrate the effect of the pages across threads and actions.

I had written some JMeter tests I am able to automatically run with different thread loads (5-60 threads stepping by 5) so I generated some response time data for before and after enabling the cache. Although there is actually many more rows and columns, in general the data looks like:
label elapsed threads

Action Elapsed Threads
NtSP 25 10
DLgn 11 10
FtDV 6 10
NtMM 16 10
NtSS 20 15
SrfS 157 10


I then read the log files into two data frames, r1 for the pre-cache times and r2 for the post-cache times. Using tapply I then generated two matrices of mean response times by thread and action (r1 becomes m1, r2 becomes m2). From that point it is just a matter of subtracting the m1 matrix from m2 to obtain the differences measured.

z <- r1
m1 <- tapply(z$elapsed, list(z$label, z$threads), mean)
z <- r2
m2 <- tapply(z$elapsed, list(z$label, z$threads), mean)
m.diff <- m2 - m1

Because the final output of the previous calculations is a two-dimensional matrix, a heatmap or similar visualization style seemed like it might be appropriate. In the real version the action names are not obfuscated.

# Calculate breaks using midpoints of matrix histogram with initial extension
x <- c(-20000, hist(m.diff)$mids)
levelplot(t(m.diff), scales=list(cex=0.7), aspect="iso", col.regions=heat.colors, pretty=F,at=x,xlab='Threads',ylab='Action',main="Time Difference With Cache")