BACKGROUND

Example embodiments are related to the field of data analytics, and specifically to generating action recommendations or action scripts for data sets. Specifically, example embodiments are directed to methods, devices, systems and computer readable media for identifying similarities in data sets from a plurality of different types of sources and generating a set of recommendations based on past user behavior on similar data sets.

Before “big data” systems can analyze data to provide useful results, the data needs to be added to the big data system and formatted such that it can be analyzed. This data onboarding presents a challenge for current cloud and “big data” systems. Typically, data being added to a big data system is noisy (e.g., the data is formatted incorrectly, erroneous, outdated, includes duplicates, etc.). When the data is analyzed (e.g., for reporting, predictive modeling, etc.) the poor signal to noise ratio of the data means the results are not useful. As a result, current solutions require substantial manual processes to clean and curate the data and/or the analyzed results. However, these manual processes cannot scale. As the amount of data being added and analyzed increases, the manual processes become difficult to implement.

The rapid proliferation of data from a variety of sources: internal and external, unstructured and structured, traditional and new data types, presents an enormous opportunity for businesses to gain valuable insights that can help them make improved and timely decisions that will win, serve, and retain customers. A key part of preparing these data sources for analysis is the ability to merge or join (e.g., blend) two or more data sets originating from different sources into a single file ready to be used for further processing, such as by a big data analytics system. The heterogeneity and size of data sets introduce tremendous challenges to understand the meaning of words, which could be used as a basis for downstream applications, such as identifying messages on a social media website or a communication feed about a particular product.

Further, homogeneous data from two or more sources may have different formats. For example, data from a first data source may be in a different format from data in a second data source. A data format can include an arrangement of content or a type of content. Therefore, data can be in diverse formats even though the data is representing a same type of information (e.g., street, ZIP code, city, data, etc.). For example, “avenue” can be abbreviated as “AV” or “Ave.” A date can be spelled out to include month, day and year, or a date can be represented by numbers separated by dashes or slashes. Therefore, it can be difficult to interpret all of the data if the data has a different format or includes different information.

An example data source can be a blog. Blogs can include demographic information. A user may desire to collect demographic information from blogs. In order to more easily manipulate the blog data, all of the demographic information can be entered in the spreadsheet that includes columns and rows of data. For example, a city, state, ZIP Code, and/or country could correspond to a column of data in the spreadsheet. Further, each of a plurality of blogs can correspond to a row in the spreadsheet. However, demographic information in a first blog can be in a different format or include different information from demographic information in a second blog. Therefore, it would not be easy for user to analyze the demographic information for the blogs unless the blogs were similarly formatted or included similar information. It would be very time-consuming to analyze each of the differently formatted demographic information.

Preparing data for performing analytics can be a very time-consuming process. Given the large amount of data that may be used in analytics, preparing the data can take a long time. Options that are currently available cannot scale when there is a large amount of data being processed. Since data sets are not necessarily in the same format with the same structure, handling the diverse data can be expensive and prone to errors.

BRIEF SUMMARY

In accordance with some example embodiments, similarities in data sets from two or more sources are identified. Regardless of the format or structure of each of the data sets, similarities can be identified. An example embodiment can generate a set of relevant recommendations based on past user behavior on similar data sets (e.g., column of data).

Example embodiments reduce an amount of time required to process large quantities of data by providing adaptive recommendations. The data is processed in order to perform data enrichment. Data enrichment can include enhancing, refining, and/or making improvements to data so that that data is in a form that can be more easily used by users. Data enrichment can include correcting typographical errors or misspellings, or adding additional information. For example, a business may have a database comprising customer address information (e.g., street number, apartment number, street name, city, ZIP Code, etc.). Address information is merely an example. Other types of information can include, for example, a telephone number, credit card information, employer, etc. However, customers may enter their address information in different formats. For example, a first user may enter “Street” in their address as “St.” and a second user may spell out “Street” in their address. Therefore, even though both the first user, and the second user have “Street” in their address, they have spelled it differently. This is merely an example and example embodiments apply to a plurality of different types of data which may be spelled differently or formatted differently. Therefore, it would be beneficial to enrich the data so that it is consistent and can be more easily processed by the user.

Data can be in the form of, for example, spreadsheet columns. A spreadsheet can be an electronic document in which data is arranged in rows and columns of a grid. A variety of actions can be provided to a user in order to enable them to prepare their data into a form that they would like their data to appear. A user can manually add an action to reformat the data. The actions can also be known as transform scripts. Actions can include, for example, obfuscate, date formatting, reformat, enrich columns, extract, split columns, etc. Actions can be done manually by a user using a graphical user interface, actions can be recommended to a user by the adaptive recommendation system, or actions can be based on a combination of manual inputs by the user and recommended actions. The actions that can be applied to the data can be based on the profile results of the data.

Over time, a user tends to perform the same actions on the same kinds of data. For example, for a column that is directed to credit card information, users may often obfuscate some of the numbers or the user may remove the credit card information. Therefore, in accordance with some example embodiments, actions are recommended to a user based on past behavior. This process of suggesting actions based on past behavior can be called adaptive recommendations. The system adapts to the user's use of the system. The system learns by the customer's previous actions and makes a recommendation based on previous actions on the data.

The foregoing, together with other features and embodiments will become more apparent upon referring to the following specification, claims, and accompanying drawings.

