r - How to test graphical output of functions? -


i wondering how test functions produce graphics. have simple plotting function img:

img <- function() {   plot(1:10) }

in package create unit test function using testthat. because plot , friends in base graphics return null simple expect_identical not working:

library("testthat")  ## example successful test expect_identical(plot(1:10), img()) ## equal (as expected)  ## example test failure expect_identical(plot(1:10, col="red"), img()) ## not fail! # (because both return null)

first thought plotting file , compare md5 checksums ensure output of functions equal:

md5plot <- function(expr) {   file <- tempfile(fileext=".pdf")   on.exit(unlink(file))   pdf(file)   expr   dev.off()   unname(tools::md5sum(file)) }  ## example successful test expect_identical(md5plot(img()),                  md5plot(plot(1:10))) ## equal (as expected)  ## example test failure expect_identical(md5plot(img()),                  md5plot(plot(1:10, col="red"))) ## not equal (as expected)

that works on linux not on windows. surprisingly md5plot(plot(1:10)) results in new md5sum @ each call. aside problem need create lot of temporary files.

next used recordplot (first creating null-device, call plotting function , record output). works expected:

recplot <- function(expr) {   pdf(null)   on.exit(dev.off())   dev.control(displaylist="enable")   expr   recordplot() }  ## example successful test expect_identical(recplot(plot(1:10)),                  recplot(img())) ## equal (as expected)  ## example test failure expect_identical(recplot(plot(1:10, col="red")),                  recplot(img())) ## not equal (as expected)

does know better way test graphical output of functions?

edit: regarding points @josilber asks in comments.

while recordplot approach works have rewrite whole plotting function in unit test. becomes complicated complex plotting functions. nice have approach allows store file (*.rdata or *.pdf, ...) contains image against compare in future tests. md5sum approach isn't working because md5sums differ on different platforms. via recordplot create *.rdata file not rely on format (from recordplot manual page):

the format of recorded plots may change between r versions. recorded plots can not used permanent storage format r plots.

maybe possible store image file (*.png, *.bmp, etc), import , compare pixel pixel ...

edit2: following code illustrate desired reference file approach using svg output. first needed helper functions:

## plot svg , return file contant character plot_image <- function(expr) {   file <- tempfile(fileext=".svg")   on.exit(unlink(file))   svg(file)   expr   dev.off()   readlines(file) }  ## ids differ @ each `svg` call, that's why simple remove them ignore_svg_id <- function(lines) {   gsub(pattern = "(xlink:href|id)=\"#?([a-z0-9]+)-?(?<![0-9])[0-9]+\"",        replacement = "\\1=\"\\2\"", x = lines, perl = true) }  ## compare svg character vs reference expect_image_equal <- function(object, expected, ...) {   stopifnot(is.character(expected) && file.exists(expected))   expect_equal(ignore_svg_id(plot_image(object)),                ignore_svg_id(readlines(expected)), ...) }  ## create reference image create_reference_image <- function(expr, file) {   svg(file)   expr   dev.off() }

a test be:

create_reference_image(img(), "reference.svg")  ## create tests library("testthat")  expect_image_equal(img(), "reference.svg") ## equal (as expected) expect_image_equal(plot(1:10, col="red"), "reference.svg") ## not equal (as expected)

sadly not working across different platforms. order (and names) of svg elements differs on linux , windows.

similar problems exist png, jpeg , recordplot. resulting files differ on platforms.

currently working solution recplot approach above. therefore need rewrite whole plotting functions in unit tests.

p.s.: completley confused different md5sums on windows. seems depending on creation time of temporary files:

# on windows table(sapply(1:100, function(x)md5plot(plot(1:10)))) #4693c8bcf6b6cb78ce1fc7ca41831353 51e8845fead596c86a3f0ca36495eacb #                              40                               60

mango solutions have published open source package, visualtest, fuzzy matching of plots, address use case.

the package on github, install using:

devtools::install_github("mangothecat/visualtest") library(visualtest)

then use function getfingerprint() extract finger print each plot, , compare using function issimilar(), specifying suitable threshold.

first, create plots on file:

png(filename = "test1.png") img() dev.off()  png(filename = "test2.png") plot(1:11, col="red") dev.off()

the finger print numeric vector:

> getfingerprint(file = "test1.png")  [1]  4  7  4  4 10  4  7  7  4  7  7  4  7  4  5  9  4  7  7  5  6  7  4  7  4  4 10 [28]  4  7  7  4  7  7  4  7  4  3  7  4  4  3  4  4  5  5  4  7  4  7  4  7  7  7  4 [55]  7  7  4  7  4  7  5  6  7  7  4  8  6  4  7  4  7  4  7  7  7  4  4 10  4  7  4  > getfingerprint(file = "test2.png")  [1]  7  7  4  4 17  4  7  4  7  4  7  7  4  5  9  4  7  7  5  6  7  4  7  7 11  4  7 [28]  7  5  6  7  4  7  4 14  4  3  4  7 11  7  4  7  5  6  7  7  4  7 11  7  4  7  5 [55]  6  7  7  4  8  6  4  7  7  4  4  7  7  4 10 11  4  7  7

compare using issimilar():

> issimilar(file = "test2.png", +           fingerprint = getfingerprint(file = "test1.png"), +           threshold = 0.1 + ) [1] false

you can read more package @ http://www.mango-solutions.com/wp/products-services/r-services/r-packages/visualtest/


Comments

Post a Comment

Popular posts from this blog

c++ - Difference between pre and post decrement in recursive function argument -

i have following sample code used pre-decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(--c); cout << c << endl; return; } } this function give output input 4 : dp3 dp2 dp1 dp0 0 1 2 3 but when use post decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(c--); cout << c << endl; return; } } output goes in infinite loop dp4 can explain me in detail why happens ? this happens because function(c--) called same value of c , when finishes c decremented. recursively called, hence called same value no chance of return , hit stack overflow error. assume this, int x = 1, y = 0; y = x++ result y == 1 , x == 2. if y = ++x , bot x , y 2.
Read more

php - Nothing but 'run(); ' when browsing to my local project, how do I fix this? -

i added new project. made mistake in projectname.conf file in sites-available apache2, led me reinstalling apache2. after reinstalling , re-enabling projects on every page of every project .. <body> run(); </body> .. i've run composer update, restarted service, restarted pc, checked config files, tried disabling new project. there nothing in error log, , access.log: 127.0.0.1 - - [15/may/2015:01:40:47 +0200] "get / http/1.1" 304 181 "-" "mozilla/5.0 (x11; ubuntu; linux x86_64; rv:38.0) gecko/20100101 firefox/38.0" maybe i'm forgetting apache2 reinstall? don't think did different last time installed it. looks haven't setup php correctly. take @ this great guide , maybe you're forgetting link php config apache2.
Read more

php - How can I echo out this array? -

i don't know how go grabbing data in variable , put foreach loop. class tree{ private $info = array( array('name'=>'jim','state'=>'nu','company'=>'nu','phone'=>array('cell'=>'5615111111','office'=>'5611111111'),'email'=>array('primary'=>'exs@example.com','ex@there.com')), array('name'=>'joe smith','city'=>'phoenix','phone'=>'4805551111','email'=>'jsmith@some_email.com'), array('name'=>'john doe','city'=>'chandler','company'=>'doe co.','email'=>array('jdoe@gmail.com','personal'=>'email@email.com'),'phone'=>'6025550002') ); } if you're inside of class, can access private variable using $this->variablename . example, i...
Read more