<commit_before>:markdown
	## Identifying data sets for public release

	> "Cities produce a great deal of data, all of which is probably interesting to somebody out there. Given our limited municipal data resources, how do we prioritize the datasets that we publish?"
	> 
	> -- [<NAME>](http://www.codeforamerica.org/blog/2012/07/05/filling-your-open-data-catalog/), 2012 Code for America Fellow (City of Santa Cruz)

	The first step in releasing open data is identifying which data sets or data resources to release. There are a number of easy ways to identify data that is suitable for release:

	### Review existing requests for data and information

	Take a look at requests for information received by your department or office - these may be formal Open Record requests or less formal requests sent via e-mail or some other channel. Does your department have a Facebook page or Twitter account? Check to see if there are comments or tweets about data that you might be able to make available. 

	### Review OpenDataPhilly.org

	Take a look at the [nomination](http://opendataphilly.org/opendata/nominate/?sort=rating_score&dir=desc) section of this site, where people can submit ideas for new data sets, or look at the results of the recent [Open Data Race](http://opendataphilly.org/contest/) for data sets that have lots of votes from non-profits and other data consumers. 

	### Look at what other cities are doing

	A number of other cities in the U.S. and around the world are releasing open data to external users. What are people in [other cities](http://civicdata.com) looking for? What kinds of data are [other governments](http://data.denvergov.org/) making available? Chances are, if there is a high demand for certain kinds of data in other cities there may be a high demand for it in the City of Philadelphia as well. This is true for a variety of different kinds of data, but is especially relevant for crime, property and financial data. 

	### Ask the public for ideas

	If you want to find out what data people are looking for from your department or agency, one of the best ways to find out is simply to ask. There is a [public forum](https://groups.google.com/forum/?fromgroups#!forum/opendataphilly) set up to allow city departments and others to interact with data users. Think about whether a posting to this - or some other - public forum would be a useful way to solicit feedback on data to release. 

	### Examine existing websites

	Look at the existing website for your agency or department. Do they contain documents with data in them or that are based on data, like annual reports, CompStat or CitiStat presentations, financial statements, etc.? Does your website have lists of service locations, or a search page that lets users find a location or service close to them? Are there any web-based application that have a database behind it - like a search feature - that is worth examining more closely for a data set that might be appropriate to release. 

	If you're already publishing on your government webiste that is formatted for consumption by human eyeballs, consider releasing it in a format that is appropriate for consumption by software and analytic tools.

	### Check for Scrapers

	Web scrapers are programs that are written by people that want to extract information - usually in bulk - from a website. There are many options for tools that can be used to conduct this type of extraction, but one of the most popular is ScraperWiki. Checking to see if there are [any ScraperWiki entries](https://scraperwiki.com/) for your websites is a great way to determine if some of your data has value to outside users. 

	The “Open Data Handbook” also has [some practical tips](http://opendatahandbook.org/en/how-to-open-up-data/choose-datasets.html) that can help identify data sets that might be suitable for release by your department.<commit_msg>fixed typo<commit_after>:markdown
	## Identifying data sets for public release

	> "Cities produce a great deal of data, all of which is probably interesting to somebody out there. Given our limited municipal data resources, how do we prioritize the datasets that we publish?"
	> 
	> -- [<NAME>](http://www.codeforamerica.org/blog/2012/07/05/filling-your-open-data-catalog/), 2012 Code for America Fellow (City of Santa Cruz)

	The first step in releasing open data is identifying which data sets or data resources to release. There are a number of easy ways to identify data that is suitable for release:

	### Review existing requests for data and information

	Take a look at requests for information received by your department or office - these may be formal Open Record requests or less formal requests sent via e-mail or some other channel. Does your department have a Facebook page or Twitter account? Check to see if there are comments or tweets about data that you might be able to make available. 

	### Review OpenDataPhilly.org

	Take a look at the [nomination](http://opendataphilly.org/opendata/nominate/?sort=rating_score&dir=desc) section of this site, where people can submit ideas for new data sets, or look at the results of the recent [Open Data Race](http://opendataphilly.org/contest/) for data sets that have lots of votes from non-profits and other data consumers. 

	### Look at what other cities are doing

	A number of other cities in the U.S. and around the world are releasing open data to external users. What are people in [other cities](http://civicdata.com) looking for? What kinds of data are [other governments](http://data.denvergov.org/) making available? Chances are, if there is a high demand for certain kinds of data in other cities there may be a high demand for it in the City of Philadelphia as well. This is true for a variety of different kinds of data, but is especially relevant for crime, property and financial data. 

	### Ask the public for ideas

	If you want to find out what data people are looking for from your department or agency, one of the best ways to find out is simply to ask. There is a [public forum](https://groups.google.com/forum/?fromgroups#!forum/opendataphilly) set up to allow city departments and others to interact with data users. Think about whether a posting to this - or some other - public forum would be a useful way to solicit feedback on data to release. 

	### Examine existing websites

	Look at the existing website for your agency or department. Do they contain documents with data in them or that are based on data, like annual reports, CompStat or CitiStat presentations, financial statements, etc.? Does your website have lists of service locations, or a search page that lets users find a location or service close to them? Are there any web-based application that have a database behind it - like a search feature - that is worth examining more closely for a data set that might be appropriate to release. 

	If you're already publishing data on your government webiste that is formatted for consumption by human eyeballs, consider releasing it in a format that is appropriate for consumption by software and analytic tools.

	### Check for Scrapers

	Web scrapers are programs that are written by people that want to extract information - usually in bulk - from a website. There are many options for tools that can be used to conduct this type of extraction, but one of the most popular is ScraperWiki. Checking to see if there are [any ScraperWiki entries](https://scraperwiki.com/) for your websites is a great way to determine if some of your data has value to outside users. 

	The “Open Data Handbook” also has [some practical tips](http://opendatahandbook.org/en/how-to-open-up-data/choose-datasets.html) that can help identify data sets that might be suitable for release by your department.
{'fraction_non_alphanumeric': 0.050372654844512976, 'fraction_numerical': 0.0030840400925212026, 'mean_word_length': 4.2766101694915255, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 11, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '485810', 'n_tokens_mistral': 1905, 'n_tokens_neox': 1806, 'n_words': 1175}