Semantic code searchis the task of retrieving relevant code given a
natural language query.
While related to other information retrieval tasks, it requires bridging the gap between
the language used in code (often abbreviated and highly technical)  and
natural language more suitable to describe vague concepts and ideas. To enable evaluation of progress on code search, we are releasing theCodeSearchNetCorpusand are presenting theCodeSearchNetChallenge, which consists of 99
natural language queries with about 4k expert relevance annotations
of likely results fromCodeSearchNetCorpus.
The corpus contains about 6 million functions from open-source code spanning
six programming languages (Go, Java, JavaScript, PHP,
Python, and Ruby) .
TheCodeSearchNetCorpusalso contains automatically generated query-like natural language
for 2 million functions, obtained from mechanically scraping and preprocessing
associated function documentation.
In this article, we describe the methodology used to obtain the corpus and
expert labels, as well as a number of simple baseline solutions for the task. We hope thatCodeSearchNetChallengeencourages researchers and practitioners to
study this interesting task further and will host a competition and leaderboard
to track the progress on the challenge.
We are also keen on extendingCodeSearchNetChallengeto more queries and programming
languages in the future.