[Bbob-discuss] Error while post-processing noisy functions (using bbob v15.03)

Vladimir Perić vlada.peric at gmail.com
Thu Apr 28 13:39:05 CEST 2016

Hi all,

I hope this list is still active. :)

I'm using the latest "old code" of BBOB. Everything works great when
running noise-free functions. To run noisy functions, I had to make a
small change in bbobbenchmakrs.py but nothing major (in the functions
generating noise, if fval is a scalar, a different error occurs than
the one caught).

However, when trying to post-process noisy data, I'm getting the
following error when comparing two algorithms:

> python bbob.v15.03/python/bbob_pproc/rungeneric.py -o compare_noisy_DE --noisy --expensive --svg test_noisy_DE-1000/ test_noisy_DE_QM3_t2-1000/
  using: ./test_noisy_DE-1000
  using: ./test_noisy_DE_QM3_t2-1000
Using bestGECCO2009 based target values: now for each function the
target values differ, but the "level of difficulty" is "the same".
log ERT1/ERT0 vs target function values done.
ECDF runlength ratio graphs done.
ECDF runlength graphs done.
File compare_noisy_DE/templateBBOBcmp.html does not exist.
Scatter plots done.
Traceback (most recent call last):
  File "bbob.v15.03/python/bbob_pproc/rungeneric.py", line 277, in <module>
    res = main()
  File "bbob.v15.03/python/bbob_pproc/rungeneric.py", line 260, in main
    rungeneric2.main(genopts + ["-o", outputdir] + args)
  File "bbob.v15.03/python/bbob_pproc/../bbob_pproc/rungeneric2.py",
line 527, in main
    '%s%d' % (nGroup, i), genericsettings.verbose)
  File "bbob.v15.03/python/bbob_pproc/../bbob_pproc/comp2/pptable2.py",
line 420, in main
    with open(filename) as infile:
IOError: [Errno 2] No such file or directory:

Through some triaging, I was able to pinpoint the error to somewhere
between lines 469 and 489 of rungeneric2.py (after ECDF runlength
graps are done and before scatter plots) -- the *.html file isn't
being created for some reason. However, I was unable to find where the
file is supposed to be created (in ppfig/save_single_function_html?).
I'm also not sure what's the difference between noise-free and noisy
functions, as the former work fine. All the individual figures and
*.svg files also seem to be created normally.

I'm aware that this code is no longer actively maintained, but I'd be
very much obliged if someone could help me pinpoint the issue. Looking
at the project homepage, I see that no noisy function results have
been presented since 2012.

Thanks a lot,

Vladimir Perić

More information about the bbob-discuss mailing list