I have a large amount of CSV files, an example of which (for job titles) is listed below. The data is noisy (there are misspellings, difference in capitalisation, missing values, and they are not well-formed (some files have headers, some have not, and if headers are present, they don't always agree on name). I have gazetteers available....,IT Manager,......,Senior IT Manager,......,it manager,......,IT managre,......,junior IT managre,......,NULL,......,Business Consultant,......,Business consultent,...I have a finite set of entity types (First Name, Last Name, Location etc.), and the task is: Given the content of this column, which entity type, if any, does it represent? (It is the first step in a processing pipeline.)It is a form of named entity recognition [supervised learning classification] task, but all the papers I have read about named entity recognition uses conditional random fields or maximum entropy for natural language tasks, and I don't think this count as natural language.What would be the appropriate approach/ML algorithm for such a task? How should my training data be formatted?
ML algorithm for determining CSV file header names based on content
machine learning;named entity recognition;csv