The Data Mining Techniques In Business

Published Date: 02 Nov 2017

The evolution of information technology and communications resulted in the impending need to transform an increasingly larger volume of data produced by human society to conduct its activities into information and knowledge useful in applications ranging from analysis and production control to market analysis, fraud detection, scientific exploration, etc. In this context, industry database systems evolved towards developed advanced database systems, advanced data analysis (Data Warehousing and Data Mining) and Web-based databases. The existence of huge data volumes raised the question to redirect their use from a retrospective to a prospective operation. Given these considerations, in this paper we tackled Data Mining - a dedicated knowledge extraction technology - from a scientific literature point of view to highlight how it is used in the decision-making process.

Keywords-database; decisions; Data mining; integration; knowledge

Introduction

Databases and information technology systematically evolved from simple systems processing data collections to systems with powerful and sophisticated database that gives users direct access to stored data, which is convenient and advantageous. Effective methods used to process online transactions, OLTP (On-Line Transaction Processing) contributed significantly to the evolution and acceptance of relational database technology as a tool for efficient storage, retrieval and management of large amounts of data [10].

Rapid and continuous growth of data volumes that have exceeded human capacity to interpret required the emergence and use of Data Mining tools to extract useful information and knowledge from huge amounts of data collected and stored in large data warehouses [3].

Advances in digital technology has led to the emergence of powerful computers, data collection equipment and mass storage media that are the physical media for both databases of information and knowledge, as well as the applications that manage them [2]. Thus information technology and communications makes it possible to analyse vast amounts of information and data stored in different types of databases and information warehouses in order to retrieve useful knowledge and information [10].

In their essence, Data Mining tools performed data analysis and highlight important data models for directing the development of different areas (economic, scientific, medical, educational and so on) that economic organizations are involved in, for the purpose of achieving the objectives specified in the constituent documents.

Still, two important problems remain: data confidentiality can be compromised by the use of Data Mining tools [4] and also if these tools are, intentionally or unintentionally, misused, the results will not be an accurate prediction of the future of an activity [9]. This last observation is due to the fact that model generated by the Data Mining tools, if misused, will be erroneous.

This paper serves as a synthesis of the knowledge accumulated so far in applying Data Mining technologies to the field of business.

After we pointed out that knowledge and information uncovered in the Data Mining process is used in formulating decisions, in controlling an economic or industrial process, in information management and so on, we reviewed the main types of data and data models that can be "Mined", focusing on Data Mining technology in customer relationship management. We presented the architecture, the content and the data exploration stages and finished with a set of meaningful conclusions.

II. Knowledge discovery in databases

Any organization in order to make decisions needs a wealth of information obtained from the company’s widespread data processing IT systems, a process that is sometimes found to be slow. In this regard, business intelligence systems, as the basic platform for providing decision support, have gradually become competitive necessities in the arsenal of successful solutions in e-business. A system capable of discovering knowledge from large databases is called Knowledge Discovery in Database Systems - KDD.

KDD basically refers to the process of discovering useful knowledge from data, while Data Mining refers to a particular step in this process, namely, the application of specific algorithms for extracting patterns (models) of data. The existence of huge data volumes questioned the shifting of their use from a retrospective to a prospective operation. To do this, Data Mining technology is used, which is based on the development of models based on the processing of the previous period data (historical data), which is examined and are already known. Thus, the data is a collection of facts, and the model is a higher level of expression which describes the data or a subset of it. This model can be applied to new situations of the same type as those already known. What is operated through Data Mining are collections of data set up for other purposes (i.e. transactions conducted over a period of time), plus those from other sources such as official statistics concerning the evolution of the economy as a whole, data on competition or legislative action. It is desirable that the models discovered are understandable for further analysis, for the purpose of studying the causes and effects. Increasing competition and increasing market demands led firms to take into account the huge potential offered by data archives.

The role of Data Mining is extracting new knowledge, implicit and with direct action from large collections of data and discovering things that are not obvious from the data, which can not be extracted manually and represent useful information that can improve the current process of action.

The discovering of knowledge from large amounts of data is a complex process consisting mainly of simpler processes, which succeed each other in chronological order, as follows [3]: data cleaning (removing unnecessary and inconsistent data), data integration (combines data from several different data sources), data selection (extracting relevant data for analysis from available repositories), data transformation (putting data in common format, by conducting summary and/or aggregation operations), Data Mining (extracting data models based on the results obtained after the previous steps), pattern evaluation of extracted data models to identify the real knowledge that we are interest in), knowledge presentation, obtained to the users, using visualization techniques and adequate representation.

Information obtained by Data Mining is predictive or descriptive. Using predictive techniques (based on supervised learning) using a set of variables (predictors) through which ​​relative predictions are made to the values ​​(continuous or discrete) of other variables (decision variables), business processes that need improvement can be highlighted, leading to the adoption of better decisions in shorter time. Predictive models based on artificial intelligence are built in a training phase [5], in which the model learns to predict the decision, when various sets of predictors are entered. Thus, after this phase, the prediction model can be used to solve classification problems (if the decision variable is nominal or discrete) or regression problems (where the decision variable is continuous).

Descriptive techniques (based on unsupervised learning) are intended for the abstraction of patterns (understandable structure) of data. Bankcard fraud detection products are a typical problem of descriptive application. Unlike predictive models, descriptive methods uniformly treat variables without distinguishing between predictors and response (decision). Descriptive methods allow the describing and explaining of the studied system characteristic phenomenons based on the models found.

Data Mining uses sophisticated algorithms to analyse complex data and reveal interesting and necessary information for the analysis made by decision makers [8]. This is based on an integration of techniques from many fields: Databases, Data Warehouse, Digital technology, Information technology and communication and Statistics. The results are particularly relevant as they are based on large amounts of data which are used to obtain information through various techniques such as neural networks, decision trees, genetic algorithms, group analysis, case-based reasoning, and analysis ties.

Data Mining software can be divided into: tools that provide user specific Data Mining techniques that can be applied to any business problems, providing flexibility and accuracy in analysis and applications that incorporate Data Mining techniques within an application specially designed to address a specific business problem, leading to increased efficiency of Data Mining applications.

Knowledge and information found in Data Mining are used in formulating decisions, economic or industrial process control, management information etc.

It is important to note that Data Mining techniques are unable to solve any management problem; it can provide only a few actions such as classification, estimation, prediction, clustering analysis groups, which used at the right place may be useful for a lot of problems in the decision process.

By using Data Mining solutions, organizations can optimize customer interactions, improve management of high-risk activities, detect fraud etc. Whether people realize it or not, daily life is affected by an application of Data Mining. For example, almost any financial transaction is processed by an application of Data Mining to detect if there is any fraud. The potential offered by Data Mining is incorporated into the companies' business processes and information search does not become an end in itself but is useful only if it is converted to action. Firms can choose to respond or not to the various situations created (decreased number of customers, loss of sales, and loss of markets and so on).

Data Mining Categories and data models

In principle, the process of Data Mining can be applied to any type of data warehouse and data flows ("transient"). Of these the most common are: relational databases, transactional databases, Data storage - Data Warehouses, object databases, databases technologies.

Data Mining techniques can be common or different from one category to another category of data, as determined by the specific characteristics of each of these categories. The model is a simplification of reality, the elements and phenomena are reduced to their essential characteristics for a particular domain. In the era of digital communications and Internet, software models are built that lead to the development of flexible software systems that easily adapt to the rapidly changing technology and the always growing demands of the users.

Data models are grouped into categories determined by Data Mining system functions that extract them from user data, stored in data warehouses. In most cases, users do not know which data models are of interest and want to extract as many types of models, that are stored in data warehouses at their disposal (databases, Data Warehouse etc.), as possible. It is therefore important that a Data Mining system allows extraction of data models at different levels of abstraction, at appropriate levels of detail requested by the user and accepts user suggestions to direct searches to data models that interest them.

In a Data Mining system, data stored in databases, data warehouses or any other type of data stores, is grouped into classes or associated to concepts [1]. Class description/ concept are done by performing the following tasks: data characterization, data discrimination, data characterization and discrimination and consists in both summarizing the data from the target class, as well as in comparison of data from one or more classes to be compared.

Data characterization consists in summarizing, in terms as concise and precise as possible, of the general essential characteristics or traits, corresponding to the target class data. Target class data specified by the user can be collected typically through a database query. Effective characterization of the data in storage is accomplished by applying the following methods of data analysis: Simple summarization of the data based on statistical measurements and sampling; data cube-based OLAP (online analytical processing) roll-up and drill-down operations, which can be used to summarize the data, a process controlled by the user, along the dimensions specified by him; attribute-oriented induction technique that can be used to perform characterization and generalization of data without direct user interaction with the Data Mining.

Data discrimination is the comparison of the general essential characteristics, of the target-class data with the general essential characteristics, corresponding to the data of one or more classes to be compared. Target class and compared data specified by the user can be typically collected by a database query. Data analysis methods used for the stored data discrimination are similar to those used to characterize the data.

Data Mining System Architecture

The Data Mining system is a computer system consisting of all hardware and software components that interact and communicate with each other for discovery (extracting) of data models that represent knowledge of interest from a large amount of data stored in databases, data warehouses, or other types of data stores.

The Data Mining System Structure (principle model) represents its internal organization to fulfil its function, namely, the application of Data Mining techniques on data stored in databases, data warehouses or other types of data stores in order to uncover data patterns that are needed for managerial decisions (fig. 1). The structural components of the Data Mining system consist of sets of data, techniques and data models.

INPUTS

The data set to be "mining" (analyzed in detail for discovering knowledge of interest)

on which

to apply

PROCESSING

Data Mining techniques (operations that apply to the data set in the Data Mining)

results

OUTPUT

Data models (knowledge extracted by applying data mining techniques to analyze data set in the Data Mining)

Figure 1. Data Mining System Structure

The Data Mining system inputs specify the tasks that are executed to meet or achieve discovery functions (extracting) of the data patterns hidden in the user data stored in data stores, most often databases or data warehouses. These are reflected in Data Mining queries, because it is formulated in the form of questions either by using a suitable query language or a dedicated graphical interface. Virtually every question (query) is transformed during the Data Mining process into a set of operations that define a Data Mining task that is performed to meet a Data Mining system function.

A Data Mining query consists of primitives, which are fundamental elements necessary for specifying a Data Mining task as a query. These are:

relevant dataset for Data Mining, comprising: the name of the data store (database, data warehouse, etc.), which stores the data that is being "mined"; database tables or data warehouse cubes; database attributes or relevant data warehouse dimensions of interest for the user; data selection conditions; criteria for grouping the data;

category of knowledge to be "mined" -represent the class of data models of interest for the user that are extracted by Data Mining task execution by the Data Mining system (characterization / discrimination, association / correlation, classification / prediction, clustering, etc.);

basic knowledge of the domain to be "mined", necessary for directing the Data Mining process and for evaluating the data models; the most common form is basic knowledge hierarchy that allows "mining" of the data at several levels of abstraction corresponding to the levels of detail specified by the user;

the measure of interest that a data model represents for the user, is used to direct the Data Mining process to extract data patterns of interest for the user, and to assess patterns extracted;

representation of the extracted model for visualization, specifies the form in which the extracted data models are presented to the user; may include rules, tables, maps, charts, decision trees and data cubes.

Formulation of Data Mining queries is made with a Data Mining query language (DMQL) oriented to the user interaction with the Data Mining applications, allowing the user to make Data Mining requests in the form of questions (queries) for executing tasks specified through primitives. This language must provide commands for specifying each of the Data Mining primitives and allow the human user and applications - users from other systems - to query the Data Mining system in an eclectic and/or interactive way. Interactive communication of the user with the Data Mining system can be done either directly, by formulating Data Mining queries, or through a dedicated GUI (graphical user interface) built on a Data Mining query language. Basically, a friendly graphical interface can be built for each type of Data Mining query language defined. The constructive model of the Data Mining System mainly comprises the following components:

Database, Data Warehouse, World Wide Web or other repository of information;

The database or data warehouse server is the computing system responsible for extracting relevant data (significant), depending on the request made ​​by the Data Mining user;

Knowledge Database, represents the domain of knowledge used to direct the search for the data models or to assess their importance in relation to user requirements;

Data Mining engine - set of functional modules required to analyse the specific tasks of the Data Mining Process: association, correlation, characterization, classification, prediction, analysis groups (cluster analysis), exception analysis (outlier analysis), analysis of the evolution and so on;

The model assessment (both as interest and as a knowledge base) - is the component that interacts with the Data Mining modules for focusing the search on the patterns of interesting data; for an effective Data Mining process it is recommended to deepen the analysis of the importance of the data model in as much detail as possible so as to limit the search to the models that interest;

The user interface (graphics) is the component that provides communication between the Data Mining system and its users, allowing them to: (1) provide information needed to guide the search and the exploration of the data via a Data Mining query or a task;(2) to sweep database, data warehouses and other data structures schemes, to extract the dataset of interest in the Data Mining process;(3) evaluate the extracted data models ("Data mined") and view them in different forms, as user-friendly as possible.

A well-established architecture, since the design phase, leads to a Data Mining system able to use the performance computing environment in an optimal way, a necessity for an effective Data Mining process, which requires: ensuring an efficient exchange of data with other systems (e.g., database or data warehouse); developing and using Data Mining techniques imposed by the specificity of the data sets on which to apply the Data Mining process; adaptation of the functionality to the requirements of "mining" requested by users.

Exploring data

Data Mining techniques, in addition to implementing algorithms, require a constant updating of data, implying a process of trimming and smoothing them to allow accurate exploration, and their contents must be examined by specialists to identify the useful information they contain [2]. Given these features, it can be said that the data exploration activity has the following steps:

problem definition, requires notification of business opportunities or needs that will lead to the objectives and expected results by using Data Mining;

identification of data sources, involves delimitation of the data necessary to solve the problem, determining it’s structure, stages of formation and location;

collecting, selecting and storing data in a common database to be used when needed;

data preparation - because data is typically stored in data collections that were made for something else, there is a preliminary stage of preparation before extraction by Data Mining. Preparation consists in treating the extreme values, missing values, possible problems with text values, the development of summarization, or performing a recoding;

defining and building the model - is the core of Data Mining and it consists in creating a computer model that will perform the operation. This phase is accompanied by training or learning phase, depending on the Data Mining techniques used;

objective model evaluation aims to determine the fair values ​​of the model's ability to correctly determine the elements for new cases. The model will be applied to the last part of pre ranked dedicated evaluation data and the error rate determined will be accepted for the new data. Model performance can be appreciated by using the "confusion matrix" that compares the actual situation with that provided by the model;

integration of the model involves completing the process, by incorporating the model into the integrated decision support system, as a key element, or by inclusion in a general decision making process within the organization.

Application of Data Mining methods in Customer relationship management

The transition from transactional marketing to relationship marketing was done after the '90s marking the transition from focusing on the creation of a large number of transactions in a short term, to promoting medium and long term relationships with all types of people and organizations that have a direct or indirect interest in the organization ("stakeholders"). Affirmation of relationship marketing at the conceptual and operational level implied a reconsideration of relations with customers so that it can be said that customer relationship management is a business strategy not limited to the area of ​​marketing. The new type of marketing is based on selecting target groups of customers and establishing individual interactions with them across multiple communication channels.

Customer Relationship Management (CRM) covers strategies for attracting new customers, maintain existing ones and regaining those who migrated to other bidders. With CRM IT solutions, all data that exists in various departments of the organization and which has been collected through various communication channels, during interactions with clients (by staff "Front Office" and Web-based applications) is integrated for each client, providing not only efficient and effective management of relationships with customers, suppliers and other entities outside the organization, but also better communication between departments of the organization.

Given the objectives of CRM, many experts consider three levels of customer relationship management: strategic, operational and analytical.

The strategic level of CRM refers to the major objective of CRM and its position as the organization's strategy. Customer relationship management is based on a good knowledge of the customer and the specific characteristics of its demand and purchase behaviour. In this respect, developing, maintaining and updating databases on customers gains importance in the foundation of CRM strategies and programs, prioritizing customer relationships and the flow of net profits generated by the customer during the entire duration of the collaboration.

At the operational level, CRM includes all activities concerning direct contact with the consumer. At an analytical level, CRM provides a number of methods for analysing customer behaviour by analysing the data obtained from transaction processing systems.

Operational CRM - serves to coordinate and synchronize interactions with clients in marketing, sales and service in automated form.

Marketing automation tool takes into account customer segmentation, campaign management, communication and response to customer requests. Regarding sales force the main automated activities are management of opportunities and contacts, generating bids and shaping the solution for the client. Regarding services, CRM software solutions can coordinate communication flows that are specific to the different channels used by the organization: operations conducted by telemarketing centres and contact centres, services provided via the Internet and activities of partners.

Analytical customer relationship management has three major objectives:

Market segmentation, which is the process of dividing customers into groups as internally homogeneous as possible based on similarity manifested (habits, tastes, affinities), these groups being heterogeneous between each other. Thus, the company can custom handle various customer segments and can concentrate on specific target groups that correspond to specific criteria of profitability.

Consumer Profiling involves modelling consumer behaviour based on a wide range of attributes such as geographical, cultural and ethnic, economic conditions, frequency of purchase, frequency of claims and complaints, preferences and their level of satisfaction, age, education, lifestyle, the media used, the method of recruitment the client responded to.

Product positioning – in the preferences of potential customers is a marketing tool focused on identifying the most attractive features of a product so as to maximize the temptation of buying it. Hence the so-called problem of the shopping cart. The probability that certain products are purchased together is determined.

Analytical CRM - help firms to optimize data sources to better understand the behaviour of buyers. Essentially refers to the organization’s performance evaluation of customer relationships and grounding the strategies and tactics for creating and developing these relationships. Activities include the collection, storage, analysis, interpretation and use of customer information.

Collaborative CRM - enables companies to collaborate with suppliers, partners and customers in order to improve collaboration and to satisfy the needs and desires of customers.

Using Data Mining techniques, decision makers can monitor the performance variables of the business, with the ability to aggregate and summarize on specific and detailed categories at the same time, specific to a certain process or analysis, presenting accurate information and excluding extra items. For example, if we want to analyse sales across multiple regions, products, quarters, or rate of products return for whatever reason, or customer behaviour analysis based on context-specific default preferences, the following types of intelligent applications will be necessary:

Product analysis, which involves answering the following questions: What is the most profitable product? , What are the least profitable products? , What is the product preference of a market segment with certain monthly income?

Marketing analysis involves: demographic analysis using customer information and sales data, price sensitivity, preferences on products, response rates from the last marketing campaign. Using this information better marketing campaigns can be planned and their effects measured.

Sales Analysis includes: identifying the trends, seasonality analysis, associations between products; sales trend in X area stores that were opened in the last 2 years; products with sales rising and what type of customers buy them. With this information you can set sales targets and can measure the progress towards these goals.

The benefits of a well-designed CRM strategy and, at the same time, the objectives that a company has to consider when making the decision of implementation such a system are:

Improving the quality of customer support, measured in terms of fulfilment of their expectations (because quality criteria may be different in customer and supplier acceptance);

Increased efficiency of marketing processes by creating a relevant communication with the customer;

Increased efficiency of operational processes such as streamlining the activities of the Call Centre or at the points of sale by creating the possibility of accessing relevant customer information and also about its previous interactions with the company.

All these benefits finally translate into lower operational cost and potentially into an increase of spending that the customer is willing to do purchasing company products. These financial benefits for the company can be invested in creating memorable positive experiences for the customer, experiences that create his loyalty. Last but not least, the implementation of a CRM strategy will bring the company a competitive advantage difficult to achieve by other players in the market, even more since the little details matter in the consumer's mind and create an emotional bond between him and the company.

The organization should examine the ways in which customer information intervenes in business processes, where it is stored and how it is used. Companies have many ways to interact with customers, such as: mail campaigns, Web sites, call centres, marketing and advertising efforts. CRM systems make connections between all these channels and the data collected is shared by various operational and analytical systems that can search the database trying to find new models (templates). Company analysts can explore the data in detail to obtain an overview of customers and segments indicating where the services are disappointing.

Implementing a system for customer relationship management is one of the most important steps in the process of implementing a CRM strategy, because of the necessary budget and because of the substantial benefits that can be obtained from a successful implementation.

Most times, companies are looking for IT systems that are more configurable and adaptable to their business, but it is important to remember that implementing a CRM system can be a good opportunity to reassess and optimize existing processes and procedures, starting from the premise that CRM applications are built based on the "best practices" of the industry. The company's mission here is to obtain an optimal mix between the best-practices included in the software solution and their organizational processes. Many managers working with budget constraints accept the idea of ​​investing in IT solutions only if they are "tested" by others first and totally risk free.

CRM implications for decision making

Forecast activity - without CRM, estimating the revenues is difficult, time consuming and inaccurate. By periodic reporting of progress with customers and measuring relative progress at the sale stages defined by the company, the sales department provides the company management relevant information in real time, allowing systematic diagnosis and proactively adjustment of problems.

Conclusion and future work

The development of Data Mining techniques is explained by the accumulation of vast amounts of data, which economic organizations have developed over the years, increasingly fierce competition and the increase of market demands which have led firms to increasingly take into account the huge potential the data archives provide.

The potential offered by Data Mining is incorporated into the business processes of companies and the search for information and knowledge is not an end in itself but is useful only if it is converted into action. Firms can choose to respond or not to different situations created by reality (decreased number of customers, loss of sales, and loss of markets and so on). Often, the action of Data Mining can be a failure and not a success; it is possible that the measures taken are not adequate to the information obtained.

Through Data Mining, business people focus better on their best customers, detect and prevent fraud, discover influence characteristics that affect most of the key performance indicators of business or society and find hidden information. For example, by analysing the profiles of the best current customers through Data Mining, models and integrated applications can be built designed to identify customers who are most likely to become the best customers in the future, although they are currently not part of the portfolio of the company's best customers. Managers of economic organizations work like this in their decisions, with the "strategic value" of existing customers through which the business future itself is predicted.

Customer relationship management is a marketing imperative, derived not only from the need to improve the financial situation of the company, but also the indisputable advantages of retaining and stabilizing the customer base. Thus, marketing campaigns are focused on getting closer to potential customers through a variety of means (telephone, Internet, "face to face" contact), allowing customers to judge and make decisions and communicate suggestions on the company's activity. The means of approaching customers can get multiple valences: increased sense of importance for manufacturing company, which thus consider individual consumption behaviour of the individual concerned, but also establishing a sense of frustration and violation of privacy, generated by a veritable telemarketing "bombing", Internet surveys, personal contacts (sometimes certain approaches are at the limit of legality like spyware or monitoring, more or less agreed by the client, of its online activity).

Implementing an appropriate CRM system is a vital issue for a company that aims to establish long-term consumer relationships with clients. In firms with a small number of customers, but a sizeable volume traded, the relationship with them is extremely sensitive, but easy to monitor. The challenge arises when a vast clientele exists, which is difficult to segment and to watch closely.

The CRM system is a possibility to facilitate customer efforts by allowing it access to online distribution networks, by being able to see the opinions of other users, by receiving personalized proposals and offers from the company. By managing customer databases, companies can identify consumer behaviour, can categorize consumers and may foresee their expectations thus modelling offers by individual consumer’s consumption profile. CRM also creates a new range of products for management, new marketing entities (call centres) and superior capitalization of information. It reduces bureaucracy and redirects the flow of promotional efforts from attracting new customers to increased business volumes achieved with the current clients (such as facilitating and launching new products, whose first users will be the loyal customers).

Taking this into account, in this paper, we approached the use of Data Mining technology for management-level decision-making by using CRM systems.

Customer relationship management becomes, in the context of depreciation of the traditional means of promotion, the main way to strengthen the market position of a company, generating for it benefits such as efficient transmission of the message, proximity to target consumers and creating a trusting relationship with them.

In a greater overall perspective on this topic it would be very interesting to develop the material presented in this paper for a more detailed discussion on the possible implementations of Data Mining technologies for support in decision making by integrating these functions and capacities in Expert Systems designed for the field of business.