Neural Networks - Predicting Future Sales with Deep Learning

Forecasting is an area of interest for organisations. Taking past observations and using said observations to predict future outcomes has many practical applications, including better decisions made by decision-makers. Organisations often use sales forecasts to assist in strategic planning, using projections to better plan for the future, boost productivity, and change course when required. Another notable example of a forecast is Weather predictions which we all use daily.

Time-series analysis is a general field that aims to make predictions from time-series data using a series of time-indexed points. Traditionally, sales forecasting tasks have used simple linear regression models from the statistics field and, more recently, random forest models developed in the machine learning field. Artificial intelligence techniques are more accurate in certain situations, notably when the function lacks linearity.

This case study explores using a long-term, short-term memory (LTSM) artificial intelligence approach to sales forecasting. We demonstrate how the predicted values match very close to the actual values. LTSMs have also been used successfully in other areas such as natural language processing.

Solving the Problem via AI LTSM Deep Learning

Typically when visualising a time series, it is often easy to see a pattern and such a pattern continuing to repeat with slight variations. What is often far more difficult is finding a model that accurately fits such a pattern; the challenge compounds further by considering the ever so changing circumstances that underly future data points. LTSM has proven to work well in this area in providing closer to a generic solution to the problem of forecasting.

Probabilistic and statistical methods attempt to use probability distributions such as the normal distribution or mathematical functions such as linear or parabolic equations to fit a set of data points. Generally, these approaches can work well if the data aligns closely with the modelled function. However, these methods require expert analysis and are quite a manual process and are difficult to achieve in practice on a widescale given the necessary degree of specialisation. Machine learning methods such as Random Forest have also been proposed. A series of decision trees are constructed using pseudo-random sequences; a prediction is made by evaluating each subsequent prediction from a series of trees.

Random Forest has proven to work well and avoids overfitting, although the approach does not efficiently scale in making predictions as datasets become large and complex. Thus, difficult to implement in practical settings for all but a subset of problems with very limited datasets.

LTSM overcomes the limitations of previous approaches by training a variant of a neural network designed to train sequentially for each timestep and model the data directly. It achieves this via a series of gates: input, output, and forget gates. Values are remembered at each timestep, and the gate regulates the flow of information between states. Essentially, the network is training on the function of the data allowing for the AI to capture complex relationships. Consider the example below, the green line represents the actual data and the red line represents the forecasted data via LTSM, it can be seen that the forecast is very close to predicting the actual values.

Sales Report

Overview of the Organisational Challenge

Decision-making is an ongoing organisational process that typically requires consideration for future directions. Strategic decision-makers may consider where the market is moving, while operational decision-makers may consider supply and demand to ensure service deliverability.

Organisations, particularly as the size of an organisation increases, have numerous data preparation challenges and collating data for use in such analysis, given the vast quantities of data. We have discussed this at length in our Article “Preparing Organisational Data for Use in AI” (https://telemus.ai/preparing-organisational-data-for-use-with-ai.html) Irrespective of the type of decision, high-quality data helps drive better decisions. Taking into account the future is always a consideration in decision making. It is often easier to determine an organisation's current environment; looking into the future becomes more complex. Typical methods include analysing current trends and looking back to the same period in a previous year to ascertain what will happen, determine what innovations are on the horizon, and logically deducing how the landscape will change. Careful analysis of these data points can be highly accurate.

Assuming data is prepared and ready to be analysed, forecasting is a complex area that requires data analytics functionality embedded within the organisation to produce accurate and predictable reports that align closely with present benchmarks. Increasingly, more solutions are becoming available to help perform this function, although many still require programming skills. Tools such as Microsoft Excel can perform many statistical methods via a point-and-click interface, although ways to use machine learning and artificial intelligence are not generally available.

Another issue facing organisations is proceduralising and productionising the generation of forecasts to become part of the daily operations of an organisation. Much of these forecasts produced in the industry's current state are via static ad-hoc analysis. While the forecasts themselves tend to be accurate, arriving at them is highly dependent on the team of individuals tasked with putting them together. Documentation and step-by-step guides are possible methods that can help and allow for continuity as individuals and groups move into other areas. However, it does not entirely address the requisite skills required to run such processes.

Having the data forecasting and analytics processes embedded within the IT systems is a key step forward in allowing organisations to mature with their data strategy. Given the complexity of artificial intelligence as a field and in performing tasks that require artificial intelligence to be employed, organisations will need to adapt to allow for such a capability. Forecasting is an area that will, in time, rely on AI, and organisations that rely on traditional methods will begin to find themselves at a disadvantage. Telemus AI™ comes equipped to assist organisations in migration forecasts using the latest AI techniques.

Organisational Data Available as ML Input

Data sources available for use in AI forecasting are as follows:

Customer meta-data from CRM systems (i.e. Salesforce, Microsoft CRM).
Transaction timestamps and amounts (i.e. PoS Systems, Stripe, PayPal).
Inventory management systems.

Integration Methodology

The following is an overview of the process we would perform at a high level to analyse such feeds within an organisation:

Extract sales data for source systems such as Salesforce, Stripe, or raw banking transactions.
Visualise and validate the data to ensure it is correct and free from errors.
Run the training data through an LTSM AI and then evaluate it using test data, ensure that the forecast appears accurate via visualisation techniques and calculate the standard error.
Continue to update the forecast as time passes to consider present, actual data points.
Produce a report demonstrating the forecast and communicate it to the broader organisation, particularly key decision-makers.

Given Telemus AI™ has advanced AI-enabled forecasting ready out of the box, your organisation can focus on the business logic rather than the technical implementation.

Organisational Applications

The following lists potential applications for your organisation:

Forecasting sales and identifying patterns and trends.
Adjusting sales strategy based on forecasts to improve results.
Supply chain management to ensure efficient management of products.
Predicting employee churn.

Potential and Realised Benefits

The ability to predict gives organisations tremendous advantages in planning for the future, allowing for operations to run more efficiently; it also provides an edge for businesses that are competing for market share. Much of these benefits are already realised today with existing techniques, and forecasting is by no means new.

While Artificial Intelligence-based methods like LTSM are inherently more complex than statistical methods in how they work, their implementation for use in solving practical problems is where they bring tremendous benefits as they can fit many datasets without complex analysis and modelling that is typically required with traditional statistical-based methods, they also scale well unlike previously provided machine learning models. Thus, orgnisations can forecast and predict many more scenarios than they would otherwise have the resources to in previous settings.

Telemus AI™ is an Australian based artificial intelligence company providing advanced solutions to government and enterprise. Contact us today for a free consultation on how the Telemus AI™ can be integrated into your organisation.

References

[1] - Predicting Sales - Barış Karaman

Sales Forecasting