I recently worked on configuring and enabling Ehcache with our Hibernate objects for common domain objects we query on almost every page request. I was interested in whether this would improve performance, and how best to illustrate the effect of the pages across threads and actions.
I had written some JMeter tests I am able to automatically run with different thread loads (5-60 threads stepping by 5) so I generated some response time data for before and after enabling the cache. Although there is actually many more rows and columns, in general the data looks like:
label elapsed threads
Action Elapsed Threads
NtSP 25 10
DLgn 11 10
FtDV 6 10
NtMM 16 10
NtSS 20 15
SrfS 157 10
I then read the log files into two data frames, r1 for the pre-cache times and r2 for the post-cache times. Using tapply I then generated two matrices of mean response times by thread and action (r1 becomes m1, r2 becomes m2). From that point it is just a matter of subtracting the m1 matrix from m2 to obtain the differences measured.
z <- r1 m1 <- tapply(z$elapsed, list(z$label, z$threads), mean) z <- r2 m2 <- tapply(z$elapsed, list(z$label, z$threads), mean) m.diff <- m2 - m1
Because the final output of the previous calculations is a two-dimensional matrix, a heatmap or similar visualization style seemed like it might be appropriate. In the real version the action names are not obfuscated.
# Calculate breaks using midpoints of matrix histogram with initial extension x <- c(-20000, hist(m.diff)$mids) levelplot(t(m.diff), scales=list(cex=0.7), aspect="iso", col.regions=heat.colors, pretty=F,at=x,xlab='Threads',ylab='Action',main="Time Difference With Cache")