This report describes a test of the large language model GPT-4 with the
the Wolfram Alpha and the Code Interpreter plug-ins on 105
original problems in science and math, at the high school and college
levels, carried out in June-August 2023.
Our tests suggest that the plug-ins significantly enhance GPT’s
ability to solve these
problems. Having said that, there are still often ”interface”
failures; that is, GPT often has trouble formulating problems in a way
that
elicits useful answers from the plug-ins. Fixing these interface
failures seems like a central challenge in making GPT a reliable tool
for college-level calculation problems.