MacResearch Request: FORTRAN Numerical Test Suite

By dgohara at Fri, Jun 29 2007 8:31am |

News

As many of you know MacResearch is interested in moving the use of FORTRAN on Mac OS X forward. One way we do this is via by creating installers of gfortran builds that Gaurav Khanna produces. In an effort to streamline the testing/validation process we are looking to put together (or extend) a numerical testing suite that compiled FORTRAN source can be vetted against. The tests we are looking for should compile and validate a variety of scientific code that is robust and produces known results. We are looking for information, feedback, suggestions on how best to proceed. You can post in this thread (must be a registered user, it's the only way we can currently fight SPAM) or via email to: fortran@macresearch.org

Comments

Fri, 2007-06-29 09:48 — fxcoudert

Great idea, more testing can only help!

As a gfortran maintainer, everyday Mac user and macresearch.org reader (and also, producer of Mac installers located on http://gcc.gnu.org/wiki/GFortranBinaries), I welcome this idea! We do know that our testing on Mac is limited, especially because of missing hardware and testing (only a few of us have access to a Mac, which makes debugging Mac-specific problems harder). All bugreports (either to the list fortran@gcc.gnu.org, or to our bugzilla: http://gcc.gnu.org/bugzilla/) and general feedback are welcome!

Fri, 2007-06-29 10:14 — vgweirs

Scope of testing

I also think this is a great idea, and is a very good fit for MacResearch. I can propose that tests be classified by purpose (correctness of result, efficiency, others?), quality of reference solution (exact solution, community consensus, comparison of compilers), platforms (e.g. efficiency may depend on memory access speed, etc.)

As a side effect it would be useful to get information on how to set compiler flags - which are most effective and when, and when they change the results.

Not specific to fortran, I would also be interested in scaling tests (speedup vs cpus/cores) for representative scientific applications. I see the speedup numbers on Apple's site for various apps but can never really figure out if it is the code or the hardware that is the bottleneck...

V. Gregory Weirs
Sandia National Laboratories

Sun, 2007-07-01 08:07 — ghutchis

Speedups fall under "Benchmarks"

We have long had a section of benchmarks of scientific applications. Most recently, these were run to compare the release of the Intel-based Mac Pros to the old G5s.

If you have suggestions for new benchmarks, that's always useful. We do try to pick representative scientific applications and give real numbers.

Sun, 2007-07-01 08:17 — ghutchis

Comprehensive Test Suite?

One question that came up in our MacResearch discussions was how comprehensive the GCC Fortran test set is. Certainly if we build the gfortran sources and get identical test results, we have a good build of gfortran.

Our question is this -- are there weak areas in the gfortran test suite? If we compile a few well-known codes and verify numerical test results, this seems like good additional test coverage. (The GCC page indicates LAPACK is the only such Fortran package currently used, but C++ has 5.)

In other words, besides the test suite, are there other large-scale tests we can (or should) run?

Thu, 2007-07-05 18:00 — Asher Langton