Monetising Data: How to Uplift Your Business

By Andrea Ahlemeyer-Stubbe and Shirley Coleman

Practical guide for deriving insight and commercial gain from data 

Monetising Data offers a practical guide for anyone working with commercial data but lacking deep knowledge of statistics or data mining. The authors — noted experts in the field — show how to generate extra benefit from data already collected and how to use it to solve business problems.  In accessible terms, the book details ways to extract data to enhance business practices and offers information on important topics such as data handling and management, statistical methods, graphics and business issues. The text presents a wide range of illustrative case studies and examples to demonstrate how to adapt the ideas towards monetisation, no matter the size or type of organisation.

The authors explain on a general level how data is cleaned and matched between data sets and how we learn from data analytics to address vital business issues. The book clearly shows how to analyse and organise data to identify people and follow and interact with them through the customer lifecycle. Monetising Data is an important resource:

• Focuses on different business scenarios and opportunities to turn data into value
• Gives an overview on how to store, manage and maintain data
• Presents mechanisms for using knowledge from data analytics to improve the business and increase profits
• Includes practical suggestions for identifying business issues from the data

Written for everyone engaged in improving the performance of a company, including managers and students, Monetising Data is an essential guide for understanding and using data to enrich business practice.

• Language: English
• Publisher: Wiley
• Release date: Feb 1, 2018
• ISBN: 9781119125150

Book preview

Preface

When we finished writing our Practical Guide to Data Mining for Business and Industry, we realised that there were still things to say. The growth of interest in data has been enormous and there are now even more opportunities than during the earlier years when there was a steady awakening to the importance of data for business and industry.

Data analytics appears on billboards in mainstream locations such as airports, and even mathematics is being coupled with adverts for cars in a positive way. Everyone is aware that they have data and has seen the graphs and predictions that analysis produces.

The book describes how any business can be uplifted by monetising data. We show how data is generated by sensors, smart homes, apps, website visits, social network usage, digital communication, purchase behaviour, credit card usage, connected car devices and self‐quantification. Enriched by integrating with official statistics, analysis of these datasets brings real business advantage.

The book invites the reader to think about their data resources and be creative in how they use them. The book is not organised as a technical text but includes many examples of innovative applications of statistical thinking and analytical approaches. It does not propose original statistical or machine learning methods but focuses on applications of data‐driven approaches. It is general in scope and can thus serve as an introductory text. It has a management focus and the reader can judge for themselves where they can use the ideas. The structure of the book aims to be logical and cover the whole loop of using data for business decisions. The idea of exploring and giving advice on how to convert data into money is really appealing.

Even after several years of excitement about big data, there are few practical case studies available. For this reason, we include 21 in the final chapter to give realistic suggestions for what to do. The other chapters of the book give necessary background and motivational content.

It is timely to publish this book now, as big data and data analytics have captured the imagination of business and public alike. Data can be seen as the most powerful resource of the future; we believe it has more influence on the wealth of companies and people than any other resource. The authors have long been proponents of data analysis for business advantage and so it is with delight that we can collate our experience and rationale and share it with other people.

The ideas in this book have arisen from many hours of fascinating consulting work. We have felt honoured to be allowed to immerse ourselves in the company culture and explore their data, and been able to present solutions that in many cases have brought great financial benefits.

We are grateful to all the business people we have worked with. Writing takes considerable time and our families and friends have been very accommodating. We thank them all very much.

1

The Opportunity

1.1 Introduction

Data awareness has swept across economic, political, occupational, social and personal life. Making sense of the fabulous opportunities afforded by such an abundance of data is the challenge of every business and each individual. The journey starts with understanding what data is, where it comes from, what insight it can give and how to extract it. These activities are sometimes referred to as descriptive analytics and predictive analytics. In descriptive analytics data is explored by looking at summary statistics and graphics, and the results are highly accessible and informative. Predictive analytics takes the analysis further and involves statistical approaches that utilise the full richness of the data and lead to predictive models to aid decision making.

This introductory chapter discusses the rise in data, changes in attitude to data and the advantages of getting to grips with accessing, analysing and utilising data. Definitions of concepts such as open data and big data are followed by guidance for reading the rest of the book.

1.2 The Rise of Data

There is much more data available and accessible than ever before.

Increasingly data is discussed in the popular press and, rather than shying away from figures, statistics and mathematics, advertisers are using these words more and more often. People are becoming more comfortable with data. This is clear from the increase in the use of self‐measurement and mapping facilities on personal devices such as mobile phones and tablets; people have a thirst for measuring everything in their daily life and like to try and control things to keep their life in good shape. Many people choose vehicles that are fitted with advanced digital measurement devices that manage engine performance and record fuel usage and location. All this is in addition to the increased automation of production lines and machinery, which have resulted in copious measurements being a familiar concept. A major contributor to the rise in importance of data is the impact of cheap data storage. For example, an external hard drive with terabytes of memory can be bought for the price of a visit to the hairdresser.

The common phrase to describe this changed world is ‘big data’ (Figure 1.1). A book on monetising data is inevitably about big data. We will interpret the term big data as data that is of a volume, variability and velocity that means common methods of appraisal are not appropriate. We need analytical methods to see the valuable patterns in it.

Diagram illustrating the worldwide volume of data with corresponding icons for Smartphones/mobile, Analytics, Production, Social networks, Internet browsing, RFID, Credit cards, Shopping, and Cars etc.

Figure 1.1 Where does big data come from?.

Since the early 2000s there has been a drive to make data more available, giving rise to the open data movement. This promotes sharing of data gathered with the benefit of public funding and includes most official statistics, academic research output and some market, product and service evaluation data. The opening up of data has led to a steep increase in requests for access to even more data; the result is a burgeoning interest in action learning and enthusiasm to understand the potential waiting to be uncovered from the data. The profession of data scientist has evolved and now encapsulates the skills and knowledge to handle and generate insights from this information.

Figure 1.2 shows how big data combined with analytics might empower different areas of any business. The aim of this book is to encourage people to use their big data to work out exciting business opportunities, make major changes and optimise the way things are run.

Diagram with a circle labeled big data and analytics, gear wheel labeled analytics, and gear wheel in a light bulb. Top: Chevron labeled potential, action insights, put into action, etc. Bottom: 6 Boxes with labels.

Figure 1.2 Big data empowers business.

1.3 Realising Data as an Opportunity

One of the key motivations for this book on monetising data is the sheer amount of under‐utilised data around. Hardly less important is the under‐achievement in terms of business benefit derived among those who do use their data. This suggests a two‐dimensional representation of the state of organisations, with one axis representing the usage of business data and the other axis representing the business benefit derived from it. Needless to say, the star performers are at the top right‐hand side of the resulting diagram in Figure 1.3. Being in the top and right‐hand corner is better than being at the top or at the right‐hand side of the axes because the two factors reinforce each other in a synergistic manner, giving greater benefits than either alone.

Graph of business benefit of data vs. usage of data displaying icons labeled “In need of more business focus”, “All to play for!”, and “In need of more analytics” with arrows pointing to “Top performers”.

Figure 1.3 Roadmap to success.

The marketplace is highly heterogeneous, with companies and institutions (all referred to as ‘organisations’ henceforth) differentiated in many ways, including:

sector

size of turnover

size in numbers of employees

maturity

research focus

product or service development.

The baseline against which organisations can benchmark themselves in Figure 1.3 is different for different types of organisation.

Familiar players using big data include retail, finance, automotive manufacturers, health providers and process industries. In addition, the following are some of the less familiar organisations likely to be in possession of big data:

Sports societies: these may have larger turnover than expected and hold vast data banks of members’ details and their sporting activities.

Museums and galleries: these may have loyalty cards and multiple entry passes that yield customer details, frequency of visits, distance travelled, inclination and time spent at the venue.

Theatres and entertainment venues: these have names, addresses and frequency of attendance of attendees, and can study their catchment area and the popularity of different acts.

Libraries: these have names and addresses and members’ interests and usage.

Small retailers: these have records of itemised sales by day of week, time of day and season plus amount spent.

Craft and niche experts: who are first aware of trends and may have a global outlook.

All these organisations can take advantage of their data but they start from different points with different resources and capabilities; with good ideas they may have the opportunity to become winners in their own areas. Experience suggests that organisations have a secret wish list for generating money out of their data. Figure 1.4 shows the ranking we observed from our clients. However, this is just a snapshot and does not include business enrichment and transformation, which are also possible.

Horizontal bar graph for generating money out of data illustrating cost savings in business processes (61%), cost savings in IT (57%), profits from the business model (35%), and competitive advantage (35%) etc.

Figure 1.4 Wish list for generating money out of data.

Figure 1.5 shows a very generalised process for monetising data. Data comes into the process and is first used for business monitoring, leading to business insights; these might generate business optimisation and might lead to monetisation and potential business transformation.

Image described by surrounding text.

Figure 1.5 Monetising data.

Despite differences in scale, the matrix in Figure 1.3 can help any organisation to map their current situation and plan their next steps to uplift their business.

1.4 Our Definition of Monetising Data

Data is the fundamental commodity, consisting of a representation of facts. However, when the data are summarised and illustrated they can lead to meaningful information, and assessing the meaningful information in context can lead to knowledge and wisdom.

Monetising data is more than just selling data and information. It includes everything where data is used in exchange for business advantage and supports business success. Large companies are often data rich and some have realised the advantage this gives them. Others consider themselves data rich but information poor because they have lots of data but it is not in a form that they can easily interpret or use to gain business insights. Statistical enthusiasm is a rare commodity but those businesses that pay attention to their data can find the answers to many of their policy and productivity questions. For example, scrutiny of data on sales easily yields information about seasonal trends: sales per customer might show shortfalls in maximising selling opportunities; total income might show overall success in attracting buyers, and so on.

Case studies and real data from our consulting practices are used throughout the book to illustrate the ideas, methods and techniques that are involved. As will be seen, most data can be monetised to bring benefit to the organisation. However, a lot of effort has to be expended to get the data into a suitable format for analysis. Data readiness can be assessed using tools that we will discuss. As analytics progresses, guidelines for data improvement become meaningful and we introduce the concept of the data improvement cycle to help organisations in continuous improvement and moving forward with their data analytics.

This book is aimed at managers in progressive organisations: managers who are keen to develop their own careers and who have the opportunity to suggest new ideas and innovative approaches for their organisation and influence how they are taken forward. The material requires background knowledge of dealing with numbers and spreadsheets and basic business principles. More specialised techniques, such as the use of decision tree analysis and predictive models, are fully explained. The main issue is the strength of desire to join the data revolution and hopefully after reading this book you will be an excited convert.

1.5 Guidance on the Rest of the Book

The rest of the book is planned as follows. Chapters 2 and 3 address data collection and preparation issues, including the use of mapping and meteorological data as well as official statistics. Chapter 4 looks at general issues around data mining: as a concept and a mechanism for gathering insights from data. Chapters 5 and 6 address technical methods; Chapter 5 looks at descriptive analytics, starting with statistical methods for summarising data and graphical presentations, and Chapter 6 moves on to statistical testing, modelling, segmentation, network analysis and predictive analytics.

Chapters 7 and 8 introduce the different strategies, motivations, modes and concepts for monetising data and examine barriers and enablers for organisations seeking to realise the full potential of their data, their valuable asset. Monetisation can be viewed strategically and operationally. Strategically we can look at new business directions, step changes in thinking, disruptive innovation and new income streams. Operationally we can consider optimising current business models, and making better use of customer targeting and segmentation. In Chapter 7 we focus on strategic issues, whilst operational improvements of the existing business will be explored in Chapter 8. In Chapter 9 we will consider the practicalities of implementation, such as issues of ethics, privacy and security; loss of cultural and technical learning due to staff turnover and the other dampers that have to be overcome before we can achieve strategic steps forward and improvement of the current situation.

The mutual importance of theory and practice has long been recognised. As Chebyshev, a founding father of much statistical theory, said back in the 19th century, ‘Progress in any discipline is most successful when theory and practice develop hand in hand’. Not only does practice benefit from theory but theory benefits from practice. So in Chapter 10 we describe a set of case studies in which monetisation has brought big gains and uplifted the business. Thus we will aim to end the book on a high note and provide inspiration to move forward.

If you locate yourself within the grid in Figure 1.3 you can see which parts of the book are most relevant for you. Those readers at the bottom left are probably at the beginning of their exploration of monetisation and could well jump to the case studies in Chapter 10 for motivation and then return to Chapter 2. Those at the bottom right have already gained substantial business advantages but could benefit from learning new statistical and data‐mining techniques to make deeper use of their data, as described in the more technical Chapters 3–6. Those at the top left already have experience of analysing data but need to realise a better business advantage and could go straight to Chapters 7–9. Those at the top right can read the whole book for revision purposes and further insights!

Note that we avoid naming specific companies. Instead we refer to them in a generic way and the reader is welcome to find example companies by searching online.

2

About Data and Data Science

2.1 Introduction

There is a pleasing increase in awareness of the importance of data. This extends across industry sectors and organisations of all sizes. Raising the profile of data means that there is more openness to exploring it and more determination to put it to good use. This chapter deals with aspects of data that are relevant to the practitioner wishing to apply data analytics to monetise data. We review the types of data that are available and how they are accessed. We consider the fast‐growing big data from internet exchanges and the attendant quality and storage issues, and consider which employees are best placed to maximise the value added from the data. We also consider the slower build‐up of transactional data from small traders and experiments on consumer behaviour. These can yield discrete collections of valuable figures ready to turn into information.

Internal company data arises as part of day‐to‐day business, and includes transactions, logistics, administration and financial data. This can be enriched by a variety of external data sources such as official statistics and open‐data sources. There is also a mass of useful data arising from social media. We define scales of measurement and terms commonly used to distinguish different types of data, the meaning and necessity of data quality, amounts of data and its storage, the skills needed for different data functions, and data readiness and how to assess where a company is on the cycle of data improvement.

2.2 Internal and External Sources of Data

Data to be used for enterprise information and knowledge creation can come from inside the company or from external sources. Integrating data from different sources is a powerful tactic in data mining for monetisation and gives the most scope for insights and innovation.

Naturally, the features of these different types of data vary and the costs associated with them range from very little to a lot. Internal data arise as part of the business and in principle they should be readily available for further analysis. In practice, the data are often difficult to access, belong to different systems and are owned by different stakeholders. A summary is given in Table 2.1.

Table 2.1 Typical internal and external data in information systems.

The issue of ownership is important because we may wish to use data and tables that are published but we don’t know to whom they belong, how accurate they are or how carefully they were obtained. The data may be available and easy to collect but we don’t know if there are any intellectual property rights that we may be inadvertently violating.

Data collected by ‘web scraping’ is an interesting case; the data here might be people’s online comments, obtained, for example, by text mining websites. The comments may be anonymous or attributed to a nickname, so that ownership is not clear. If the comments are attributed to someone then they are owned by a third party, but otherwise thought is required before using them.

Internal, operational information systems move large amounts of internally produced data through various processes and subsystems, such as payment control, warehouse, planning/forecasting, web servers, adserver technology systems and newsletter systems. One drawback with internal data is that it is used primarily to handle the daily business and operational systems may lack a facility for keeping a comprehensive history. However, at least the quality and reliability of internal data is in the control of the company. This is not the case for external data unless it has been generated under very strict guidelines, such as those of a research institute or government statistical service.

External data is generated outside the company’s own processes; it is often needed as a set of reference values. For example, a service provider can compare the characteristics of their customer base with those of the target population. Characteristics such as employment, housing and age distribution are available from national statistics institutions (NSIs). Official statistics are necessarily aggregated to conserve confidentiality. The level of granulation has to be such that people cannot identify individuals by triangulating knowledge from several sources.

Eurostat collects data from all European NSIs and has a very comprehensive website at www.eu.eurostat.org. Considerable effort has been invested by government statistical services to make their websites user‐friendly, not least because they are under pressure to show that they provide a useful service and are worth the public expense that they represent. Aggregated data are available as tables and graphics that can be animated, and there is a vast amount of detail available. However, it can take some patience to navigate to the data required and it is a good idea to make advance preparations against the possibility of needing the data in a hurry. An example of the use of NSI data is included in the case study in Section 10.6.

As well as providing reference information, external data is often also valuable for providing additional information about a customer. Analytically focused information systems such as marketing databases and customer relationship management (CRM) systems frequently add external data. This may be in the form of specifically purchased information about the customer, such as their address, peer group or segment, or their credit rating.

As an example, consider a company that has data about books bought in a certain geographical area over a period of time. The data is in time order for each sale and so is long and thin; an extract is shown in Table 2.2. Each row represents a sale and additional information is in each column. Sometimes the rows are referred to as ‘cases’.

Table 2.2 Extract of sales data.

The data is valuable even without further additions, but descriptive analytics may yield a wide range of important information as shown in Table 2.3.

Table 2.3 Company sales data analytics.

This data can be enriched by adding company‐owned information about the customer, including their address, date of first purchase, date of last purchase, and the frequency and monetary value of their purchases. These last factors feature in segmentation methods based on RFM: the recency, frequency and monetary value of purchases. Descriptive analytics of the data can now be enhanced to include statistics such as sales per customer segment.

The data can be further augmented by adding freely available open data collected by an NSI or by providing knowledge about the customer based on their location, such as the type of housing in the area, the population age range, socio‐economic activity, and so on. Other more specific data may be obtained about their peer group or segment from commercial sources such as www.caci.co.uk.

Descriptive analytics of the data can now be enhanced to include statistics such as sales per socio‐economic group. This could have implications for the effectiveness of promotional activities, or allow assessment of the impact of opening an outlet in an area or of increasing salesperson presence in an area (Table 2.4). Predictive analytics can address issues such as which factors are most related to sales quantities and values.

Table 2.4 Internal sales data enriched with external data.

In the example, the company now has more information about book sales and can use this in their promotions.

Combining data from different areas and plotting them as they change over time is the background to the ground breaking Gapminder website, www.gapminder.org, developed by Hans Rosling. For example, scatterplots of income per person against life expectancy at birth for each country plotted over time from 1809 to 2009 show the amazing changes that have taken place in different countries. Animated graphics are a powerful way to show the relative changes. Work by Stotesbury and Dorling has explored the relationships between country wealth and their waste production, water consumption, education levels and so on.

In a well‐organised, data‐aware company, the quality of internal data may be better than that from external resources, not least because the company can control exactly how and when the internal data is generated. External data may not match the internal data exactly in time (being contemporaneous) or location, but nevertheless the availability (often free of charge) and the extent of this data means that even poorly matched external data can be useful.

2.3 Scales of Measurement and Types of Data

Knowing about the different scales of measurement and types of data is important as it helps to determine how the data should be analysed. Measurements such as value of sales are quite different from counts of how many customers entered a retail outlet, or of the proportion of times sales exceeded a certain limit. Descriptive data, such as a location being ‘Rural’, ‘Coastal’, ‘Urban’, or ‘Suburban’, need to be treated quite differently from measurement data. ‘Frequency of occurrence’ can be evaluated for descriptive data but it does not make sense to calculate an average value (say, for location) unless some ordering is applied, for example a gradation between agricultural and industrial locations, so that an average has some sort of meaning.

Business information comes in many forms. Reports and opinions are qualitative in nature whereas sales figures and numbers of customers are clearly quantitative. Qualitative data can usefully be quantified into non‐numerical and numerical data. For example, theme analysis applied to reports gives a non‐numerical summary of the themes in their content and the frequency of occurrence of the themes gives a meaningful numerical summary.

There are different types of quantitative data, and they may be described in a number of ways. Table 2.5 contrasts some of the more common terms.

Table 2.5 Scales of measurement examples.

Data can be classified as continuous or categorical. Categories can be nominal or ordinal. The simplest level of measurement is nominal data, which indicates which named category is applicable. For example, a customer may live in an urban area, a rural area or a mixed area. In a dataset, this nominal data may be given in a column of values selected from urban/rural/mixed, with one row for each customer.

Once data has been identified as a useful analytical entity, it is often referred to as a ‘variable’. A data item such as income has a different value for each person and is called a variable because it varies across the sample of people being investigated. Note that being referred to as a ‘variable’ does not imply that the income of a particular person is uncertain, just that income varies across different people.

If a categorical variable has only two levels, for example ‘Male’ or ‘Female’, then the data is referred to as ‘binary’. Note that sex and gender refer to different concepts, with sex being biological and gender referring to the way the person sees themselves. Datasets can have several categories for gender. For example, one of the public datasets made available for data mining for the Knowledge, Discovery and Datamining Cup lists people who have lapsed from making donations to US veterans (see http://www.kdnuggets.com/meetings/kdd98/kdd‐cup‐98.html). The pivot table for gender has entries for ‘Male’, ‘Female’, ‘Missing’ and ‘Not known’ because the donation was from a joint account. In addition, some entries are blank and there is one case with the letter C, which does not have a defined meaning. There are six categories, some of which are only sparsely filled. If gender is used as a variable in analysis this sparseness may cause problems and the data should be pre‐processed before analysis. Note that there may also be additional accidental categories for ‘M’, ‘m’, ‘man’, and other erroneous entries.

If there is any order associated with the categories, then they are referred to as ‘ordinal’ data. Opinions can be captured as ordinal variables using questions, such as:

How was your experience today? Dreadful, poor, OK, good or very good

The responses usually need to be quantified if any meaningful analysis is to be carried out. In this example, it makes sense to code ‘Dreadful’ as −2, ‘Poor’ as −1, ‘OK’ as zero, ‘Good’ as +1 and ‘Very good’ as +2. The words can be replaced by pictures or emoticons as a more effective way of extracting opinion. Researchers have also investigated physical ways of gathering opinions; the engagement of a person can be evaluated by the length of time they keep eye contact and their certainty can be evaluated by the time they take to answer the question.

Variables that represent size are referred to as measures, measurements, scales or metrics. In data mining, the term ‘metric’ includes continuous measurements such as time spent, and counts such as the number of page views. Some statistical software packages, such as WEKA and SPSS, distinguish between scale and string variables, and will only allow certain actions with certain types of data. A string variable, such as ‘Male’ or ‘Female’ often needs to be recoded as a binary scale variable, taking values such as 1 or 2, as an additional alternative form, to ensure flexibility in the subsequent analysis. MINITAB distinguishes between quantitative variables and text variables and will not perform actions unless the appropriate data type is presented. Excel distinguishes between numbers and text. In R software, variables have to be specified as either numeric (numbers with decimal places), integers (whole numbers positive or negative), characters (string variables) or logical (true or false).

Many data items are measured on a continuous scale, for example the distance travelled to make a purchase. Continuous data does not need to be whole numbers like 4 km, but can be fractions of whole numbers, say 5.68 km. Continuous data may be of the interval type or the ratio type. Interval data has equal intervals between units but an arbitrary zero point. For example shoe or hat sizes. Ratio data is interval‐type data with the additional feature that zero is meaningful, for example a person’s salary. The fixed zero means that ratios are constant: €20,000 is twice as much as €10,000, and €6 is twice as much as €3.

Dates and times are interval data that have special treatment in statistical software because of their specific role in giving the timeline in any analysis. Usually a variety of formats are allowed. A numerical value can be extracted from the date as the number of days since a specified start date. The day of the week and the day of the month can be identified and both are useful depending on the analysis being carried out.

The different numbers of days in a month can sometimes cause problems (see Box).

Box Example of problems with days of the month.

Wet weather ‘behind drop in mortgages’

Metro newspaper, Tuesday 1 April 2014

The article states that:

The number of mortgages granted to home‐buyers fell to a four‐month low in February, Bank of England figures show. The drop to 70,309 from 76,753 in January was likely because of wet weather, analysts said. Ed Stansfield of Capital Economics said the temporary fall ‘should go some way towards calming fears the housing market recovery is rapidly spiralling out of control’.

76,753 mortgages in January equates to 2476 per day. At the same rate, February, with 28 days, should have 69,325 mortgages. The ‘drop’ is therefore actually an increase of 984.

Any comments?

The time variable can be represented by the number of minutes, hours, and so on since a start time. Time calculations can cause problems in practice, as some days start at 00:00, while others start at 06:00 or 07:00, say in Central European Time. These small discrepancies can have big implications in data analysis. For example, analysing the pattern of temperatures recorded across a geographical area quickly illustrated that some records were of mean temperature for the 24 hours from 00:00 and some were from 06:00.

Nominal and ordinal variables, referred to as categorical or classification variables, often represent dimensions, factors or custom variables that allow you to break down a metric by a particular value, for example screen views by screen name.

The measurement level can also be described as continuous or discrete. The number of occurrences, for example the number of times a customer returned an order, is discrete data. Measures such as the number of returns per unit of sales value are continuous in nature, as they can include decimal points. Another example is the measurement of interactions with the social web: alternative metrics, ‘altmetrics’, are measures of the impact of scholarly publications and research based on mentions on social media. This produces discrete variables such as the number of mentions across the web and continuous variables such as the citations per week since publication. The online research method, netnography, explores web activity in great detail.

Visual cues can be counted on to give useful numerical data. For example, we can count the number of exchanges between customer and salesperson, or record the time spent engaged in communication. Sounds can be quantified by their frequency or intensity.

Data items do not have to be single entities; combined data such as profiles, growth curves, or density and spatial distributions of product sales may be used as inputs to specialised algorithms. Such data have a connection in time or space, which is part of their description, and predictive analytics can attempt to determine important factors which affect them.To summarise, in data mining we consider:

classification or categorical variables, which can be nominal, binary and ordinal

scale or metric variables, which can be discrete, continuous, interval or ratio.

Qualitative data, such as pictures or text can be summarised into quantitative data by methods such as content analysis. More complex data structures, such as profiles and signatures, can be analysed but require more sophisticated techniques than those available in the more popular software packages.

The bibliography contains references to books and articles that cover data types, qualitative and quantitative data and information quality.

2.4 Data Dimensions

It is important for any organisation to evaluate their data and to determine who might value it. A first step is to audit the data and clarify its dimensions. Examples of data dimensions include customers, time (months, seasons, years), products, suppliers, applications and markets. The data can usefully be ‘sliced and diced’ according to these dimensions.

Each of these data dimensions can be explored using descriptive analytics – summary statistics, tables, graphs and charts – and predictive analytics examining the relationships between different parts of the data. Clearly there is value in having more dimensions to the data, greater quantity in terms of each dimension, more flexibility of presentation, greater depth and granularity.

A central feature of any data is how it relates to the economic and social environment as a whole. Therefore, it is important to enrich the database with relevant publicly available data, such as that from NSIs. This creates an integrated data resource.

2.5 Quality of Data

External data from NSIs is subject to tight regulation and its quality may be considered to be good. External data from other bodies has to be appraised on an individual basis. Guidelines covering definitions, regulations and quality requirements are readily available and a checklist may be devised. For example, the Organisation for Economic Co‐operation and Development (OECD) uses seven dimensions for quality assessment: relevance, accuracy, timeliness and punctuality, accessibility, interpretability, coherence, and credibility. Eurostat’s seven quality dimensions are relevance of statistical concept, accuracy of estimates, timeliness and punctuality in disseminating results, accessibility and clarity of the information, comparability, coherence, and completeness. The extensive work on quality standards carried out by institutions has created considerable intellectual capital available to everyone working with data.

Quality is evaluated, in our context, in terms of the usefulness of the data to support business opportunities. To ensure quality, data must be analysed to identify necessary repairs, additions and corrections. All information should be consolidated in terms of the business to provide a complete understanding of the relationship between all the data items. It is important to establish business rules for resolving conflicting data. For example, there may be conflicting information about a company’s size because the data has been