\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Criteria** & \multicolumn{3}{c}{**Brier Score \(\downarrow\)**} & \multicolumn{3}{c}{**\% Accuracy \(\uparrow\)**} & \multicolumn{2}{c}{**\% Data Retained \(\uparrow\)**} \\ \cline{2-7}  & **Ours** & **Crowd** & **Aggregate** & **Ours** & **Crowd** & **Aggregate** & **Forecasts** & **Questions** \\ \hline
**All Questions** &.179,003 &.149,003 & **.146,002** & 71.5 & 77.0 & **77.8,6** & 100\% & 100\% \\
**Crowd Uncertain** & **.238,004** &.240,003 & **.233,002** & 58.1 & **58.3,3** & **60.2,12** & 51\% & 56\% \\
**Early Retrieval** &.186,003 &.162,004 & **.159,003** & 70.0 & 74.4 & **75.0,8** & 84\% & 100\% \\
**5+ Articles** &.175,003 &.142,003 & **.140,002** & 72.3 & 77.7 & **78.7,7** & 84\% & 94\% \\ \hline
**All Criteria** & **.240,005** &.247,004 & **.237,003** & **58.0,17** & 54.2,17 & **56.6,17** & 22\% & 43\% \\ \hline \hline \end{tabular}
\end{table}
Table 3: **System performance on the test set. “All Questions” shows the Brier score on the full test set. Other rows show selective evaluation when specified criteria are met, averaging over qualifying questions and retrieval dates. “Crowd Uncertain” refers to questions with crowd predictions between 0.3-0.7. “Early Retrieval” refers to the first 3 retrieval dates. “5+ Articles” refers to forecasting when at least 5 relevant articles are retrieved. Finally, “All Criteria” refers to forecasting when the 3 criteria are jointly met. Notably, in every setting the aggregate (average) of our system and crowd prediction is the best. Subscript numbers indicate 1 standard error. We bold entries that outperform the crowd aggregate, and underline the best entry in each category.*** Forecasting requires a holistic consideration of the possibilities (Tetlock and Gardner, 2015). We next prompt the model to leverage the retrieved information and its pre-training knowledge to produce arguments for why the outcome may or may not occur. * The model can potentially generate weak arguments. To avoid treating all considerations as equal, it is instructed to weigh them by importance and aggregate them accordingly into an initial forecast. * Finally, to prevent potential bias and miscalibration, the model is asked to check if it is over- or under-confident and consider historical base rates (Tetlock and Gardner, 2015), prompting it to calibrate and amend the prediction accordingly.