Recommendations For Instructor Based Learning

Published Date: 02 Nov 2017

Preface

This book has been inspired by both a passion for analytics, and a perceived gap between the need for having a better understanding of a business and the existing situation, where, except for large corporations, business analytics methods are mostly unknown.

Let’s talk a little bit about both. My passion for analytics stems for my major, Economics, a subject which encourages critical thinking and acceptance of a certain degree of ambiguity for conclusions and results. Some of you may know the joke ‘if there is an economist in the room, there will be at least two opinions’. This approach and way of thinking was, and is, an excellent fit for a practical statistician. Statistics, apart for the basic methods of producing tables and graphs and calculating indices, is an exploratory discipline, where relationships between variables are examined. This is both a drawback and an opportunity: a drawback that often it is hard to explain and sell your analysis to an audience that may not understand and accept a certain degree of ambiguity, and an opportunity to show that one is able to infer meaningful facts and relationships from data that does not reveal anything at a first glance.

Statistics, econometrics, data mining and machine learning methods have found wide applicability for solving business problems, helping businesses understand their customers and products better, and allowing them to take actions that would make them more profitable and outsmart their competitors. Thus, business analytics grew up as a distinct field, with a wide variety of tools, which are mostly known and used by specialists working for large corporations.

In fact, these methods have wide applicability and can be used by almost anyone who has a computer, an adequate level of numeric ability, and a critical an inquisitive mind. Then why we do not know about business analytics methods, and how can we benefit from using them?

Why we do know about them is rather due to their relatively complex and heterogeneous content. Formal education can face difficulties in addressing them, as this content does not fill well within the regular theoretical curriculum of statistics, econometrics and data mining or machine learning. These courses tend to have a high theoretical content, can be highly mathematical and abstract, which is a barrier for people who are not interested in becoming familiar with mathematics-like content and do not see their real-life relevance.

Another set of disciplines that address the issues that business analytics deals with belong to marketing. While marketing courses are more accessible and can provide a good background for getting to know and understand business analytics issues and basic measures, they tend to focus more on market research. This is a fact of life, driven by the fact that many major businesses have research departments. Thus, the focus is more towards solving research questions and do market intelligence, thus leaving out, or covering to a small extent more complex analytics tasks such as customer segmentation, market basket analysis, mining relationships between customers, etc. These other methods are used in very large corporations that have dedicated analytics, data mining or scoring department, employing highly specialized people and using expensive software. Thus, relevant courses get designed and are offered to such audiences, and remain largely unknown except for a small group.

All of the above have inspired me to write this book and design an introductory course on business analytics. First, I do believe that business analytics techniques are useful for most businesses. If large multinationals find them useful and employ people to do them, why a small or medium-sized business cannot do the same? There are two answers to this: having qualified people, and having the tools to do them.

The second one is not so much of an issue today. There are relatively unexpensive, and even free tools that can do a lot of what dedicated commercial software can. In this respect, R software can be a good competitor for other commercial analytics software. Being free, benefitting from the most advanced implementations of statistical, econometric, data mining and machine learning methods, being relatively easy to use due to an intuitive graphical user interface, and having relatively simple and understandable syntax for the more skilled, R is an excellent choice to get started and excel at business analytics.

The first one remains a hurdle. Standard business analytics materials assume a combination of business knowledge, a certain level of statistical, econometrics or data mining knowledge, and some proficiency in using the statistical software to solve some rather mundane data-processing tasks, which are not very easy to get. In some cases, certain knowledge is often confined to ones that actually work in a field, and does not get into textbooks.

This book is an attempt to put together all the needed ingredients for getting started with business analytics. Paced for someone that does not have an analysis, statistical or programming background, while allowing readers with some background on these to move fast to the topics that are new to them, this book attempts to provide a basic business analytics and data processing background knowledge, gets you started with learning R, one of the most advanced analytics software available, with the help of an intuitive graphical user interface, introduce basic statistical measures and data mining techniques with thorough explanations and practical examples, and give you the basics of some cumbersome data processing without which little or no analysis can be done.

All content is ordered in an increasing sequence of difficulty, and takes into account the learning curve experienced by an audience with little or no background on business analytics, or having the set of skills mentioned before.

And, as an essential ingredient for a well-rounded introductory book, the book will end with guidance and advice on how to design your research and present it in an efficient way, with a maximum impact.

I wish you a pleasant journey in the world of business analytics, and I hope you will benefit a lot from reading it and doing the exercises and applications inside.

January 2013 Adrian Otoiu

Contents

Preface

1. Introduction, purpose and focus

Foreword

The aim of this book

What is covered and what you should expect to learn

How to use this book

1.4.1 Guidelines for self-study

1.4.2 Recommendations for instructor-based learning

The focus: how to approach business analytics and a tentative recipe for success

Basic business analytics measures and getting started with R

Basic business analytics and data concepts

An introduction to R

Installing R and the Graphical User Interface

Basic applications with R

Importing data

Getting basic statistics with R

Merging data sets

Joins

Getting ready for work with R. Installing libraries and packages

Seasonal Adjustments and basics of time series manipulation

Exercises and questions for review

Basic statistics, econometrics and forecasting elements using R

Basic statistical measures and concepts

Basic data exploration

3.2.1 Correlation

Linear Regression

3.2.2.1 Regression Analysis. Features, Properties and Other Key Facts

3.2.2.2 Heteroskedasticity

3.2.2.3 Autocorrelation

3.2.2.4 Doing a basic forecast based on regression analysis results

3.3 Logistic regression

3.4 Exercises and questions for review

4. Data mining methods used for practical business analytics issues

4.1. Data mining and statistics. Focus, method and concepts

4.2. Customer segmentation using decision trees/classification trees

4.3. Clustering based on multiple criteria. The RFM framework

4.4. Market Basket/Association Analysis

4.5 Exercises

5. Selected business analytics and data manipulation topics; analysis design and presentation of models and results

5.1. Churn Rates

5.2. Price Elasticity

5.3 Price Points/Price Dispersion

5.4 Usual retail sales measures

5.5 Peak transaction periods

5.6 Detecting patterns in customer behavior

5.7 How to present your results and stand out in the eyes of your audience! or How to keep in mind the end result when doing the analysis work!

5.8. Exercises and questions for review

Appendix 1. Glossary of statistical terms

Appendix 2. Glossary of marketing terms

Index

Chapter 1

Introduction

1.1 Foreword

In almost any field, there are basic tools that are essential for doing the day-to-day job. A driver has a car to carry merchandise or passengers; an advertiser uses creativity to conceive ads and different media to convey them; a store owner needs merchandise to sell, and a location to display it, and examples could continue. These are the basic tools and requirements for doing business in a particular fields.

However, these would not ensure success by themselves. In an increasingly competitive and complex world, with a wide diversification of tastes, and with customization being the norm and available at relatively small costs, having the basic expertise to conduct a business is not enough. Increasingly, there is the need to know and master other tools that can ensure your business will survive and thrive, and tell you whether you are on a profitable track or not.

Knowing your activity and knowing your customers are essential ingredients for success. In an increasingly competitive world, with many companies ready to deliver the same product or service the winner is not necessarily the one with the highest quality or the lowest price. Figuring out who the customers are, and what they want, is key to meeting their needs, and secure business from them.

Also, knowledge about the size, scope, and occurrence of demand for products and services helps in making sure about the ability to meet it when it occurs. Dark spots, such as products with low profit margins, unpredictable demand, products with high rates of complaints and returns must also be spotted and dealt with, as they tie up resources and have negative impacts on the bottom line.

A lot of attention has been devoted to explore these issues and provide answers to basic question such as: who are our customers, what do they want, how much they can afford, what (else) influences sales, how did sales evolve over time, and so on. To this end, two disciplines have provided answers to these questions and tools to get them.

One of them is statistics. Dealing with data, statistics was already perceived as useful in the assessment of how a business is doing, and is summarizing the results. Its use is quite obvious when one looks at a company or product presentation. With the advent of computers, and with data becoming widely available from internal and external sources, statistics found new uses in helping a business. Measures used in describing the distribution of the data, and making inferences about its evolutions became more and more used and popular. Meanwhile, contributions from fields related to statistics have made their way into data analysis for business purposes. Econometrics proved useful through regression analysis for multivariate data analysis, while data mining and machine learning provided advanced methods for analyzing data and generating actionable recommendations.

Another one is marketing, with its stated purpose of creating and delivering products and services that are valuable to the customers, as defined by the American Marketing Association. Apart from a qualitative and management side, which helps us frame the specifics of delivering what has value to the customers, specific quantitative techniques were developed to help businesses define their value propositions and successfully reach out to customers and meet their needs. Market research methods has provided businesses with tools to classify customers according to particular business needs and preferences, and to examine features of the products and services offered on the market.

The evolution of both statistics and marketing towards providing tools to assess the activity of a company and its environment have eventually grown up into a distinctive field, named business analytics. This field has emerged from both quantitative disciplines (statistics, econometrics, data mining and machine learning) and business disciplines (marketing, market research), and structured itself so as to provide answers to most issues relating to assessing the performance of a business and knowing the essential quantitative characteristics of its customers and general environment.

1.2 The aim of this book

This is an introductory text which starts at a more basic level than any other similar book. Its purpose is to provide just the basic theoretical knowledge needed in doing practical business analysis work, and to give in a short time the basic tools needed to solve most of the basic business analytics issues likely to be encountered in everyday practice, to people with little or no prior relevant knowledge.

While the subject of the book is quite clear from its title, its purpose, aims, and target audience remain to be defined. And, following the discussion above, it is not only the essential tools and content that matter, but also the value that this book brings about to its readers.

The essential aim for this book is not merely to convey business analytics knowledge to the readers. There are other books that do this in different ways. Some business analytics books do this by building upon statistics and/or quantitative methods background of the readers, and provide them with a solid theoretical and/or applied background. They are appropriate for graduates of quantitative and analytical fields, and enable them to acquire the successful background needed for a career in business analytics with a major company or consultancy.

Other books are essentially management books that have a high-level approach of business analytics. Their purpose is to explain the use of business analytics tools and methods to managers and directors by covering their implementation requirements, their uses in different settings, and the benefits of implementing them into an organization.

While there are books available for both the audiences described above, there is a perceived gap that makes business analytics out of reach for a large segment of interested readers.

This segment is rather heterogeneous, made up of people with different levels of expertise and backgrounds, whose training and careers have not channeled them into the highly specialized tracks of business analytics practitioners, or medium or high-level managerial or directorial positions that make use of business analytics.

Thus, in most small and medium business environments, the workforce is fairly diverse and not well specialized, and managers and analysts may not have different backgrounds at all. In some cases, all of them have similar backgrounds (economics, sociology, psychology, political science, commerce) and do not necessarily fit the (junior) business analyst which is the target audience of some specialized books simply because these highly trained analysts work for large multinationals or consultancies for obvious reasons (pay, advancement opportunities, etc).

Also, managers and directors working for small and medium businesses (which are often the equivalent of seniors or team leaders in large corporations) are very likely to take on some of the analytics tasks (in fact they often do so in a less formal way when required by the business owner or a handful of board members to produce a report, or provide ideas, on how to steer the business into profitability). They are not that sophisticated and high level as their counterparts working in large corporations, and reading some of the classic high-level books may be useless to them since they may not have the people to perform the lower-level actual analysis tasks. Rather, they need to perform business analytics tasks themselves and, to some extent, provide some guidance to junior enthusiastic employees with rather sparse knowledge of statistics and marketing. This audience will likely see the benefits as it works though the content of the book, and needs not be told in a concise way the contribution to the bottom line of business analytics methods, as in a standard ‘book for managers’. Their business knowledge and first-hand experience will help them do so.

As for the junior analysts working in small or medium companies [1] , they are often confined to basic statistical analysis taught in school, and may never know whether most these methods exist. It is a fact that most small and medium businesses do not have the capability of doing basic forecasts, let alone segmenting customers and calculating metrics like churn rates. In fact, analysts here may never want to go into significantly more methodological detail unless they either develop a passion for analytics, or the company expands and they get promoted into a more senior role where they need to learn more to cope with increased requirements. Another fact already mentioned is the heterogeneity of their regular work tasks, which may prevent them in devoting more time to a more rigorous study of analytics.

This is the main idea of the book and the value it brings to the potential readers; that there is an audience with very little knowledge and awareness about business analytics, very little time, and limited resources to undertake a more formal training sequence, that would need to start somewhere. This "somewhere" is the selling proposition of this book and covers a large market essentially untapped by the more methodology-oriented books that do not necessarily show how to apply these methods, or by the high-level management texts whose content is quite esoteric for a hands-on small and medium business owner or manager.

Another advantage of this book is that it makes all the content easily accessible via the use of R. Most of the other high-level books only deal with theoretical content, or are geared towards using standard commercial software licensed to major companies, making the practice of business analytics be confined to major corporations and students of major universities. In order to actually be able to practice business analytics as instructed by these books, one needs to either work for a company that uses that software, be enrolled at an university and pay substantial tuition fees as an entrance pass to computing facilities and statistical software, or spend about $800 or more for a fully functional perpetual license fee [2] . This is often a major roadblock for individuals and small businesses alike, which have small budgets and for which the cost, doubled with the uncertainty of getting results using a rather novel set of methods, is a major barrier in undertaking business analytics tasks.

This book will teach business analytics together with how to install and use R, a free statistical software, which benefits from one of the most advanced implementations of statistical, econometric and data mining/machine learning models. While R is not specifically customized for business use as other commercial software is, this book shows you how to install and use an user-friendly interface that helps someone perform basic operations in R much like he/she would with a popular commercial software package (e.g. SPSS). Some calculations may require entering brief code sequences (not more than a line) which will help one achieve more control, and adjust the parameters of the analysis more easily than with a menu-based software. Besides, the interactive nature of the user interface enables the user to see the commands that perform different operations as the menu items are called in and used, thus easing their understanding and making customization of commands and their parameters accessible for less tech-savvy users.

In sum, the book aims at a less sophisticated audience of junior or aspiring analysts who do not have a specialized quantitative or marketing background, and low to mid-level managers working in small and medium-sized entities (that sometimes go by the name of seniors and team leaders) who are often doing operational tasks and/or supervise staff that perform them. In doing so, it provides them with the basic knowledge and real-world practical examples needed to start their own analysis, using a free software with state-of-the art implementations of statistical algorithms, and a user-friendly interface which makes its use comparable to the use of popular commercial software packages.

1.3 What is covered and what you should expect to learn

Business analytics is an evolving and rather heterogeneous field, which incorporates all the quantitative techniques and methods required to analyze the data relevant to a business and its environment.

Many of business analytics methods derive from marketing applications of quantitative methods. Regression analysis, basic forecasting, data mining, machine learning algorithms and techniques form the core knowledge of any business analytics course.

However, this is hardly covering all that is needed for an analyst to be able to independently carry out analysis. Basic statistics, and understanding key data issues is crucial in the appropriate application of the statistical tools mentioned above. Knowing the appropriate use of the medians, the importance of outliers and how to assess them, and basic data manipulation are the preliminary steps that ensure statistical algorithms are correctly applied and results are reliable. And presenting them with a practical focus to a novice audience, or one who has been exposed only to quantitative methods taught in a theoretical, academic ways, gives the reader a tremendous edge over the competition.

Manipulating data of different types, and converting it into the right format is a problem largely unknown until one actually gets to do real analytics work and has to use data that is not often put into the right format. In large corporations this is performed by specialized employees that have an IT or quantitative background, and often require programming skills and acumen needed to solve mundane problems. In some cases, these operations can take hours to get solved in a satisfactory way. Another frustrating thing is that solving these issues has nothing to do with the art of statistical analysis, the mastery of getting meaningful results from data, and connecting the dots from the findings. In this book, some of the most difficult but needed basic manipulations are going to be shown to you step by step, which will save you hours of trial-and-error painstaking manipulations, and enable you to be as efficient as professional analysts are.

The practical focus of this book is fostered by the presentation of some of the most basic calculations used in marketing and business analysis, and working through them using real-life examples. This later feature is fostered throughout the book, where atypical cases and data sets with atypical distributions are shown. This is meant to provide a strong advantage to the reader, which is not likely to acquire that knowledge, or the skills required to tackle these issues unless being exposed to them via practice, or guided by a seasoned professional.

This practical focus is also reinforced by walking the reader through some key marketing concepts and illustrates their application with real life examples. These are normally missing from an academic book, and come to a practitioner's knowledge only when actual work is being carried out. Most of this content is mistakenly left out from the introductory texts on the belief that this material is 'too advanced'. In fact, it is very simple and easy to grasp, and helps the reader develop the right way of thinking about business analytics issues as the essential methods and tools are studied and learned.

And, last but not least, this book also provides basic guidelines on how to design and conduct an analysis, present the results, and make them be appropriately perceived and value by their intended audience. While it does not have the ambition to substitute a communication course, it wants to compel the reader to go past the technical knowledge, structure the analytics work from start to finish, and turn it into a valuable service that can work towards advancing his/her career and provide genuine value-added to the employer and clients.

1.4 How to use this book

This introductory level book is to be used as an introduction to business analytics in a practical, down-to-earth manner. Ideally, the reader should work through its examples and exercises, and try to apply the knowledge to issues encountered in everyday life. Its introductory nature does not preclude the reader to use other references and foster knowledge on topics of interest by using more theoretical materials, and even consult marketing books or R forums. Its practical approach is designed to stimulate thinking and put life into theoretical concepts and definitions and make them part of the tools to be used in one's professional life. Its comprehensive nature makes it amenable to self study, as it is self-contained with respect to the topics covered for carrying out business analytics work, and can be used as supporting material for 8 to 10-week courses in business analytics or quantitative methods for business.

1.4.1 Guidelines for self-study

The self-contained nature of this book, who ambitiously packs quantitative knowledge and methods, marketing concepts, solutions to data processing issues, basic knowledge of the R statistical software, and some widely used basic quantitative business measures and concepts, does not come with an assurance that the reader will master business analytics. This is just the entrance ticket to using business analytics, and requires thorough work through the examples, a drive for understanding the basic marketing problems, and experimenting with the given solutions on real-life issues.

Given its basic level of theoretical knowledge and practical focus, this book is designed to encourage the reader to work through all examples and exercises, and compel the reader to go back and revise existing knowledge, reuse it during several applications spread throughout several chapters, and use concepts and methods that often employ heterogeneous knowledge originating from distinct fields. While this may be a bit frustrating in the beginning, it will prove to be very useful later and allow the reader to connect the dots and pave the way to becoming a well-rounded practitioner, something that is being achieved in a much harder way by those who pursue a defined education track and work for several years in analytic roles.

1.4.2 Recommendations for instructor-based learning

Its practical focus, clarity of exposition, and length makes this book a good candidate for being a course support for a quantitative-oriented business course.

This book can be used in teaching eight-to-ten-week courses for students enrolled in vocational schools, universities operating on quarter systems, MBA and executive education students. While these audiences are fairly diverse, this book can reasonably accommodate them. Its practical focus and self-contained nature can be very appealing for practice-oriented vocational students, or small business owners and aspiring analysts or junior managers looking for a comprehensive, well rounded education in many business subjects. The extensive use of the R graphical user interface, with items similar to those present in commercial statistical packages, will help the student become familiar with R very quickly, and help the students understand the code needed to run some commands. The code itself is kept to a minimum (not exceeding one line of code), and being thoroughly explained throughout the book helps in bringing the level of confidence and trust at high levels for less technical students [3] .

A possible course sequence could be as follows:

Lecture 1 Introduction to business analytics, backgrounder, setting up the course (Chapter 1, Section 2.1)

Lecture 2 Getting started with R (Sections 2.2.1, 2.2.2.1 and 2.2.2.2, 2.2.2.5)

Lecture 3 Essential data processing (Section 2.2.2.3, 2.2.2.4, 2.2.2.6)

Lecture 4 Basic statistics and econometrics (Chapter 3, up to 3.2.2.1)

Lecture 5 Statistics and Econometrics 2, intermediate topics: (The remainder of chapter 3)

Lecture 6 Machine learning I ( Sections 4.1-4.3)

Lecture 7 Association analysis and retail sales (Chapter 4.4, 5.6)

Lecture 8 Common business analytics calculations (Section 5.1, 5.2, 5.3 and 5.5)

Lecture 9 Designing business analytics tasks, and presenting results for the audience (5.7)

Variations to this schedule may occur depending on the pace of the class, and on the length of the term. For shorter terms, the first and second lecture should be combined. A longer sequence can insist more on Lecture 8 and ensure students master basic business analytics calculations.

1.5 The focus: how to approach business analytics and a tentative recipe for success

Business analytics is a rather heterogeneous field, a toolbox of quantitative methods applied to solve specific business issues. It is not meant to be a theoretical science, but rather a practical one where calculations and analysis are rooted into real-world basic marketing and business topics and issues.

Its specific nature poses particular challenges for those involved in practicing it. Success is not guaranteed by the mastery of statistical methods and sound marketing knowledge. Several industry and firm-specific issues require first-hand knowledge and grasp of the business issues that are being analyzed, and an open and creative mind is needed to tackle them and translate their requirements into practical problems and actionable solutions. Awareness of the analysis and solutions used by competition, and also by the business partners, can offer valuable hints in doing analysis work.

Also, knowing the audience and the customers of the analysis work is essential in achieving its added value, and ensures that appropriate courses of action are taken to utilize it to its full potential. It is not rare that close contact with bosses and clients ensures that the issues are appropriately clarified and addressed in a timely manner, and that solutions are effective and targeted towards cost-effective and achievable goals.

Even more than in other fields, being up-to date on tools and methods is essential given the highly dynamic nature of the field. Professional contacts may be especially valuable in exchanging professional opinions and maintaining competitiveness. Looking for professional experiences with renowned employers, with strong business analytics departments staffed with top professionals may be the best way towards a successful career.

On the rather technical aspects of business analytics, attention to detail and an acute sense for numbers are ingredients to success. But they should not come before having an analytical mind and imagination needed to understand and solve unique, off-the beaten track issues that often arise and need actionable solutions. Good communication skills in understanding the issues, keeping up-to date on the data and issues, and maintaining good relationships and trust with bosses and customers (internal or external) are major ingredients to success.