Abstract
We argue that, in these cases, the field should take a leaf out of the practices of the SAT-solving community, and adopt systematic benchmarking, and benchmarking contests, as a way measuring (and stimulating) progress. This would involve a change of culture.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2018 |
| Publisher | IEEE |
| ISBN (Electronic) | 9781728106250 |
| ISBN (Print) | 9781728106267 |
| DOIs | |
| Publication status | Published - 2018 |
Keywords
- Computer algebra
- Benchmarking
- Performance
Cite this
The Role of Benchmarking in Symbolic Computation : (Position Paper). / Davenport, James.
Proceedings of the 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2018. IEEE, 2018.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - The Role of Benchmarking in Symbolic Computation
T2 - (Position Paper)
AU - Davenport, James
N1 - To be published, on past form, in 2019, but will display a 2018 publication date.
PY - 2018
Y1 - 2018
N2 - There is little doubt that, in the minds of most symbolic computation researchers, the ideal paper consists of a problem statement, a new algorithm, a complexity analysis and preferably a few validating examples. There are many such great papers. This paradigm has served computer algebra well for many years, and indeed continues to do so where it is applicable. However, it is much less applicable to sparse problems, where there are many NP-hardness results, or to many problems coming from algebraic geometry, where the worst-case complexity seems to be rare.We argue that, in these cases, the field should take a leaf out of the practices of the SAT-solving community, and adopt systematic benchmarking, and benchmarking contests, as a way measuring (and stimulating) progress. This would involve a change of culture.
AB - There is little doubt that, in the minds of most symbolic computation researchers, the ideal paper consists of a problem statement, a new algorithm, a complexity analysis and preferably a few validating examples. There are many such great papers. This paradigm has served computer algebra well for many years, and indeed continues to do so where it is applicable. However, it is much less applicable to sparse problems, where there are many NP-hardness results, or to many problems coming from algebraic geometry, where the worst-case complexity seems to be rare.We argue that, in these cases, the field should take a leaf out of the practices of the SAT-solving community, and adopt systematic benchmarking, and benchmarking contests, as a way measuring (and stimulating) progress. This would involve a change of culture.
KW - Computer algebra
KW - Benchmarking
KW - Performance
U2 - 10.1109/SYNASC.2018.00050
DO - 10.1109/SYNASC.2018.00050
M3 - Conference contribution
SN - 9781728106267
BT - Proceedings of the 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2018
PB - IEEE
ER -