<commit_before>{
	"comment" : "The main events of CS548." ,
	"course" : "CS548" ,
	"prerequisites" : [ "CS585", "CS561" ] ,
	"instructors" : [ "ambite" , "szekely" ] ,
	"teaching-assistants" : [ "slepicka" ] ,
	"graders" : [] ,
	"classroom" : "ZHS 163" ,
	"lectures" : 
	[

		{
			"id" : "l:intro" ,
			"title" : "Introduction and course overview" ,
			"description" : "Overview of issues that make information integration challenging and the techniques that we will cover in the course to address these challenges.  Disucssion of the grading policy, including the quizzes, homeworks and the final project." 
		} ,

		{
			"id" : "l:rdf" ,
			"title" : "RDF, graph data model" ,
			"description" : "Introduction to the Resource Description Framework (RDF), the langauage of the Semantic Web and the Linked Data Cloud. We will review basic concepts of XML and contrast XML with RDF. Will introduce the concept of URIs, the global graph of data that RDF supports, and the syntax to write RDF documents." 
		} ,

		{
			"id" : "l:rdfs" ,
			"title" : "RDF Schema and basic inference" ,
			"description" : "RDF Schema defines the semantics of RDF graphs. We will introduce the elements of the RDF Schema language and show how these elements enable an RDF database (triple store) to make inferences to derive new information." 
		} ,

		{
			"id" : "l:sparql" ,
			"title" : "SPARQL query language" ,
			"description" : "SPARQL is the query language of RDF databases. We will introduce the types of queries that can be expressed in SPARQL and the main elements of the language. We will show how one can query the Linked Data cloud using SPARQL." 
		} ,

		{
			"id" : "l:linked-data" ,
			"title" : "Linked Data, common vocabularies/ontologies" ,
			"description" : "<NAME> introduced the concept of Linked Data more than 10 years ago. Since then, the Linked Data cloud has grown to more than 30 billion facts covering pretty much any topic you can think of. We will review the principles of Linked Data and provide an overview of the main datasets and vocabularies available in the Linked Data cloud." 
		} ,

		{
			"id" : "l:record-linkage-string-matching" ,
			"title" : "Record linkage – string matching" ,
			"description" : "Record Linkage is the problem of identifying when two different records refer to the same real world entity (e.g., whether two records about restaurants refer to the same restaurant). Record Linkage is one of the most important problems in information integration. In this class we begin our study of Record Linkage by covering several techniques for fuzzy matching text strings." 
		} ,

		{
			"id" : "l:record-linkage-record-matching" ,
			"title" : "Record linkage – record matching" ,
			"description" : "We continue our study of Record Linkage by reviewing techniques for matching records consisting of multiple attributes. We will review the main algorithms for record linkage and cover optimization technques to avoid the N-squared comparison problem." 
		} ,

		{
			"id" : "l:entity-extraction" ,
			"title" : "Extracting entities and relations from text" ,
			"description" : "Many structured sources contain unstructured fields (e.g., the biography of a person). We will provide a brief overview of the main techniques to extract entities (people, places and organizations) from text and review several off-the-shelf tools for entity extraction." 
		} ,

		{
			"id" : "l:open-information-extraction" ,
			"title" : "Open information extraction" ,
			"description" : "New information extraction techniques are able to extract a wide variety of entities, relations and events from text. These techniques allow you to build large knowledge bases of a wide variety of topics (e.g., the places where famous people where born). Our guest lecturer is a leading expert in open information extrction and will present an overview of these techniques. " 
		} ,

		{
			"id" : "l:database-theory-basics" ,
			"title" : "Database theory basics: queries, query containment, Datalog" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:glav" ,
			"title" : "Answering queries using views (Global-as-View, Local-as-View)" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:data-cleaning" ,
			"title" : "Data cleaning" ,
			"description" : "Data cleaning is a major headache in information integration because most datasets contain errors that need to be corrected before the information is usable. We will cover the main types of errors and common techniques for detecting and correcting data errors. We will provide an overview of Google Refine, a popular tool for data cleaning." 
		} ,

		{
			"id" : "l:schema-mapping" ,
			"title" : "Schema Mapping" ,
			"description" : "Different datasets often use different schemas even when they contain information about the same topic. In order to integrate these datasets it is useful to identify a mapping between the attributes in these datasets so that one can combine them into one integrated dataset. In this class we cover techniques for automatically identifying mappings between schemas." 
		} ,

		{
			"id" : "l:rdf-mapping" ,
			"title" : "RDF mapping tools" ,
			"description" : "A popular and useful technique for integrating datasets (databases, delimited text files, XML or JSON) is to map them to RDF. Once the datasets are mapped to RDF it becomes possible to query them in an integrated way using SPARQL. In this class we cover techniques for mapping different types of datasets to RDF." 
		} ,

		{
			"id" : "l:automatic-modeling" ,
			"title" : "Automatic source modeling" ,
			"description" : "Mapping datasets to RDF can be difficult and time-consuming. In this class we cover different techniques for automatically mapping a dataset to a domain ontology (RDF Schema)." 
		} ,

		{
			"id" : "l:karma" ,
			"title" : "Karma, semi-automatic source modeling" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:owl2-basics" ,
			"title" : "OWL2: Description Logics, Inference" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:owl2-profiles" ,
			"title" : "OWL2 Profiles: QL, EL, RL" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:ontology-based-integration" ,
			"title" : "Ontology-based data integration" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:linked-services" ,
			"title" : "Linked Services" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:wrapper-generation" ,
			"title" : "Semi-structured Data: Wrapper Generation" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:wrapper-learning" ,
			"title" : "Semi-structured Data: Wrapper Learning" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:mashup-principles" ,
			"title" : "Mashups: principles" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:mashup-tools" ,
			"title" : "Mashups: Yahoo Pipes & YQL, REST" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:big-data" ,
			"title" : "Big Data" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:intellectual-property" ,
			"title" : "Intellectual Property" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:review" ,
			"title" : "Review" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:project-presentations-1" ,
			"title" : "Project Presentations" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:project-presentations-2" ,
			"title" : "Project Presentations" ,
			"description" : "TBD" 
		} 
	] ,
	"homeworks" :
	[
		{
			"id" : "hw:rdf" ,
			"title" : "RDF Graphs" ,
			"description" : "" 
		} ,

		{
			"id" : "hw:sparql" ,
			"title" : "SPARQL, the RDF Query Language" ,
			"description" : "" 
		} ,

		{
			"id" : "hw:karma" ,
			"title" : "Using Karma to Convert Datasets Into RDF" ,
			"description" : "" 
		} ,

		{
			"id" : "hw:triple-stores" ,
			"title" : "Working with Triple Stores" ,
			"description" : "" 
		} ,<commit_msg>Added descriptions for most lectures<commit_after>{
	"comment" : "The main events of CS548." ,
	"course" : "CS548" ,
	"prerequisites" : [ "CS585", "CS561" ] ,
	"instructors" : [ "ambite" , "szekely" ] ,
	"teaching-assistants" : [ "slepicka" ] ,
	"graders" : [] ,
	"classroom" : "ZHS 163" ,
	"lectures" : 
	[

		{
			"id" : "l:intro" ,
			"title" : "Introduction and course overview" ,
			"description" : "Overview of issues that make information integration challenging and the techniques that we will cover in the course to address these challenges.  Disucssion of the grading policy, including the quizzes, homeworks and the final project." 
		} ,

		{
			"id" : "l:rdf" ,
			"title" : "RDF, graph data model" ,
			"description" : "Introduction to the Resource Description Framework (RDF), the langauage of the Semantic Web and the Linked Data Cloud. We will review basic concepts of XML and contrast XML with RDF. Will introduce the concept of URIs, the global graph of data that RDF supports, and the syntax to write RDF documents." 
		} ,

		{
			"id" : "l:rdfs" ,
			"title" : "RDF Schema and basic inference" ,
			"description" : "RDF Schema defines the semantics of RDF graphs. We will introduce the elements of the RDF Schema language and show how these elements enable an RDF database (triple store) to make inferences to derive new information." 
		} ,

		{
			"id" : "l:sparql" ,
			"title" : "SPARQL query language" ,
			"description" : "SPARQL is the query language of RDF databases. We will introduce the types of queries that can be expressed in SPARQL and the main elements of the language. We will show how one can query the Linked Data cloud using SPARQL." 
		} ,

		{
			"id" : "l:linked-data" ,
			"title" : "Linked Data, common vocabularies/ontologies" ,
			"description" : "<NAME> introduced the concept of Linked Data more than 10 years ago. Since then, the Linked Data cloud has grown to more than 30 billion facts covering pretty much any topic you can think of. We will review the principles of Linked Data and provide an overview of the main datasets and vocabularies available in the Linked Data cloud." 
		} ,

		{
			"id" : "l:record-linkage-string-matching" ,
			"title" : "Record linkage – string matching" ,
			"description" : "Record Linkage is the problem of identifying when two different records refer to the same real world entity (e.g., whether two records about restaurants refer to the same restaurant). Record Linkage is one of the most important problems in information integration. In this class we begin our study of Record Linkage by covering several techniques for fuzzy matching text strings." 
		} ,

		{
			"id" : "l:record-linkage-record-matching" ,
			"title" : "Record linkage – record matching" ,
			"description" : "We continue our study of Record Linkage by reviewing techniques for matching records consisting of multiple attributes. We will review the main algorithms for record linkage and cover optimization technques to avoid the N-squared comparison problem." 
		} ,

		{
			"id" : "l:entity-extraction" ,
			"title" : "Extracting entities and relations from text" ,
			"description" : "Many structured sources contain unstructured fields (e.g., the biography of a person). We will provide a brief overview of the main techniques to extract entities (people, places and organizations) from text and review several off-the-shelf tools for entity extraction." 
		} ,

		{
			"id" : "l:open-information-extraction" ,
			"title" : "Open information extraction" ,
			"description" : "New information extraction techniques are able to extract a wide variety of entities, relations and events from text. These techniques allow you to build large knowledge bases of a wide variety of topics (e.g., the places where famous people where born). Our guest lecturer is a leading expert in open information extrction and will present an overview of these techniques. " 
		} ,

		{
			"id" : "l:database-theory-basics" ,
			"title" : "Database theory basics: queries, query containment, Datalog" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:glav" ,
			"title" : "Answering queries using views (Global-as-View, Local-as-View)" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:data-cleaning" ,
			"title" : "Data cleaning" ,
			"description" : "Data cleaning is a major headache in information integration because most datasets contain errors that need to be corrected before the information is usable. We will cover the main types of errors and common techniques for detecting and correcting data errors. We will provide an overview of Google Refine, a popular tool for data cleaning." 
		} ,

		{
			"id" : "l:schema-mapping" ,
			"title" : "Schema Mapping" ,
			"description" : "Different datasets often use different schemas even when they contain information about the same topic. In order to integrate these datasets it is useful to identify a mapping between the attributes in these datasets so that one can combine them into one integrated dataset. In this class we cover techniques for automatically identifying mappings between schemas." 
		} ,

		{
			"id" : "l:rdf-mapping" ,
			"title" : "RDF mapping tools" ,
			"description" : "A popular and useful technique for integrating datasets (databases, delimited text files, XML or JSON) is to map them to RDF. Once the datasets are mapped to RDF it becomes possible to query them in an integrated way using SPARQL. In this class we cover techniques for mapping different types of datasets to RDF." 
		} ,

		{
			"id" : "l:automatic-modeling" ,
			"title" : "Automatic source modeling" ,
			"description" : "Mapping datasets to RDF can be difficult and time-consuming. In this class we cover different techniques for automatically mapping a dataset to a domain ontology (RDF Schema)." 
		} ,

		{
			"id" : "l:karma" ,
			"title" : "Karma, semi-automatic source modeling" ,
			"description" : "Karma is our own tool for semi-automatically mapping a variety of sources to a domain model defined using an ontology or an RDF Schema. In this class we will cover the algorithms that Karma uses to learn from previous mappings and the algorithms to automatically suggest models for new sources." 
		} ,

		{
			"id" : "l:owl2-basics" ,
			"title" : "OWL2: Description Logics, Inference" ,
			"description" : "OWL, the Web Ontology Language is the standard language for defining ontologies in the Semantic Web. OWL ontologies define the semantics of classes and properties and allow inferences to be made to derive new facts. In this class we cover the main OWL constructs and how they allow a system to infer new facts." 
		} ,

		{
			"id" : "l:owl2-profiles" ,
			"title" : "OWL2 Profiles: QL, EL, RL" ,
			"description" : "Full OWL reasoning is computationally expensive (exponential) and thus impractical for most applications. The OWL profiles define subsets of OWL where the reasoning algorithms perform efficiently. In this class we introduce the most common OWL profiles, review the reasoning capabilities that they support and illustrate them in practical settings." 
		} ,

		{
			"id" : "l:ontology-based-integration" ,
			"title" : "Ontology-based data integration" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:linked-services" ,
			"title" : "Linked Services" ,
			"description" : "A significant amount of data on the Web is available via Web APIs. These APIs typically return their data in XML or JSON. In this class we review techniques for modeling Web APIs to make them easy to discover and to wrap them so that they can consume and produce RDF. This makes it possible to integrate them with other Linked Data sources." 
		} ,

		{
			"id" : "l:wrapper-generation" ,
			"title" : "Semi-structured Data: Wrapper Generation" ,
			"description" : "Most of the information on the Web is available on Web pages, and often is not available in a structured format or a Web API. Web wrappers are programs that extract data from Web pages and return it in a structured format, typically XML. Several libraries exist for programmers to develop such wrappers. There has been significant research on techniques to generate wrappers by example so that users who don't know how to program can generate wrappers. In this lecture we cover several systems that use this approch and discuss the algorithms that they use." 
		} ,

		{
			"id" : "l:wrapper-learning" ,
			"title" : "Semi-structured Data: Wrapper Learning" ,
			"description" : "An alternative approach to generating wrappers from examples is to automatically learn the wrappers given a set of Web pages. In this class we discuss techinques that enable a system to discover differences in pages and use this information to automatically generate a wrapper to extract information from Web pages." 
		} ,

		{
			"id" : "l:mashup-principles" ,
			"title" : "Mashups: principles" ,
			"description" : "Mashups are applications that combine information from multiple Web sites and Web APIs to build an interesting new application. In this calls we discuss the main techniques that have been used to create mashups and describe interesting research systemts that incorporate these techniques." 
		} ,

		{
			"id" : "l:mashup-tools" ,
			"title" : "Mashups: Yahoo Pipes & YQL, REST" ,
			"description" : "In this class we focus on techniques for building mashups using Web APIs. We begin by introducing the principles of RESTful APIs and illustrating these principles using examples. Then we cover two interesting systems that allow users to quickly and easily construct mashups using Web APIs." 
		} ,

		{
			"id" : "l:big-data" ,
			"title" : "Big Data" ,
			"description" : "Lately, there has been significant interest in NoSQL databases that support parallel distributed analysis of large datasets on commodity hardware. In this lecture we explore the latest ideas for using NoSQL databases for storing Linked Data at large scale." 
		} ,

		{
			"id" : "l:intellectual-property" ,
			"title" : "Intellectual Property" ,
			"description" : "TBD" 
		} ,

		{
			"id" : "l:review" ,
			"title" : "Review" ,
			"description" : "In this class we review all the topics covered during the semester highlighting the main ideas and techniques." 
		} ,

		{
			"id" : "l:project-presentations-1" ,
			"title" : "Project Presentations" ,
			"description" : "Students present their papers. The presentations will use the Pecha-Kucha format where students present the projects in 6 minutes in 40 seconds using 20 slides in auto-advance mode every 20 seconds. We used this technique for the first time in Fall 2013 and it was a huge success, so we will continue using this format." 
		} ,

		{
			"id" : "l:project-presentations-2" ,
			"title" : "Project Presentations" ,
			"description" : "Second day of project presentations using the Pech-Kucha style." 
		} 
	] ,
	"homeworks" :
	[
		{
			"id" : "hw:rdf" ,
			"title" : "RDF Graphs" ,
			"description" : "" 
		} ,

		{
			"id" : "hw:sparql" ,
			"title" : "SPARQL, the RDF Query Language" ,
			"description" : "" 
		} ,

		{
			"id" : "hw:karma" ,
			"title" : "Using Karma to Convert Datasets Into RDF" ,
			"description" : "" 
		} ,

		{
			"id" : "hw:triple-stores" ,
			"title" : "Working with Triple Stores" ,
			"description" : "" 
		} ,
{'fraction_non_alphanumeric': 0.09699769053117784, 'fraction_numerical': 0.0032017635943732943, 'mean_word_length': 3.550513494148555, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '89637', 'n_tokens_mistral': 6252, 'n_tokens_neox': 5398, 'n_words': 2367}