Is there evidence that open source research software produces more uptake of methods? In recent years there has been a drive to make science more open. This includes making the software used to perform research open source. The main argument in favour of this idea is that research should be reproducible, which has been addressed in other questions on this site.

I am more interested in the 'impact' that this produces. In particular, is there evidence that by publishing software alongside research papers means that more people use the methods described in the paper?

I am especially concerned with papers in applied mathematics but also more generally.
You ask "In particular, is there evidence that by publishing software alongside research papers means that more people use the methods described in the paper?"

I am not aware of any large scale empirical studies that have assessed this for software at present [1] however there is anecdotal evidence and related studies looking at the impact of open data.

One piece of related anecdotal evidence came from short survey I carried out in 2013 looking for people who had been "scooped" as a result of publishing their code openly (i.e. someone else had published a paper using that software to get to the same scientific results before the author of the software). Whilst there were very few examples of people getting scooped, there were many more examples of researchers who had gotten new collaborations, new citations, and new funding as a result of publishing their code openly. Many said that this was because others were more able to try out the methods because code was available.

More convincingly, there have been several good studies looking at the effects on citations (a proxy for the sort of impact you mention) from making data openly accessible [2-6]. Many of the reasons for the data citation benefit discussed in [7] appear qualitatively to be true for software as well [8-9].

Finally, in the area of Applied Mathematics, you might like to look at some of the outputs of the ICERM workshop on Reproducibility on Computational and Experimental Mathematics [10], as well as some of the publications of the participants. 



[1] One of the reasons for this is that up until very recently, it was difficult to conduct such a study - it was difficult to link software to a publication, few authors were publishing code, and it was difficult to data mine journals to assess impact. This is changing, an I expect studies to start emerging based on tools like ContentMine and ScienceToolbox.

[2] Piwowar, Day, Fridsma (2007). Sharing Detailed Research Data Is Associated with Increased Citation Rate. PLoS ONE.

[3] Gleditsch NP, Metelits C, Strand H. 2003. Posting your data: will you be scooped or will you be famous? International Studies Perspectives 4(1):89-97

[4] Pienta AM, Alter GC, Lyle JA. 2010. The enduring value of social science research: the use and reuse of primary research data. The Organisation, Economics and Policy of Scientific Research Workshop

[5] Henneken EA, Accomazzi A. 2011. Linking to data - effect on citation rates in astronomy.

[6] Dorch B. 2012. On the Citation Advantage of linking to data. hprints. 

[7] Piwowar H, Vision T. 2013. Data reuse and the open data citation advantage. PeerJ. PubMed 24109559

[8] Howison J, Herbsleb, J. 2013. Incentives and Integration In Scientific Software Production. CSCW 2013.

[9] Howison J, Bullard J. How is software visible in the scientific literature? Preprint available from https://github.com/jameshowison/softcite/blob/master/paper/HowisonBullard-SoftwareCitation-WorkingPaper.pdf?raw=true

[10] "Setting the Default to Reproducible: Reproducibility in Computational and Experimental Mathematics," ICERM Workshop report, with D. Bailey, J. Borwein, R. LeVeque, W. Rider, and W. Stein.