Custom Forecast Model for Commercial-Heavy Areas
Please note: Specific program names, internal tool references, and other potentially sensitive details mentioned in the original project document have been generalized or masked in this summary for confidentiality.
Introduction
This document outlines a successful initiative to significantly improve delivery volume forecasting accuracy during holiday periods, specifically for delivery stations serving a high percentage of commercial addresses. Inaccurate forecasts for these unique sites previously led to operational challenges during critical holiday weeks. By developing and implementing a customized Day-of-Week (DoW) curve model, we have achieved substantial improvements in forecast precision, paving the way for more efficient operations and better resource allocation. This new model demonstrated a remarkable reduction in forecast error, notably decreasing the Day-of-Week curve Mean Absolute Percentage Error (MAPE) by 395 bps for affected sites during initial testing.
The Challenge: Forecasting Inaccuracies for Commercial-Heavy Sites
Our delivery network serves a diverse customer base, primarily composed of residential addresses (approximately 90%), with commercial addresses making up a smaller but significant portion (around 7%). While residential deliveries generally occur seven days a week, including weekends and most holidays, commercial deliveries face constraints. Many businesses close on weekends and observe holidays, restricting delivery access and altering typical volume patterns.
The standard forecasting process historically applied a network-level DoW curve during holiday weeks. This approach aimed to align daily volume plans with overall holiday demand signals and promotional activities. However, while individual station forecasts are adjusted based on site-specific metrics, the network curve heavily influences the final daily plan. Consequently, stations with a high concentration of commercial customers often experienced significant deviations from the network DoW pattern during holidays, leading to forecasting inaccuracies and potential operational inefficiencies.
Understanding Holiday Impact Variations
Not all holidays impact delivery patterns equally. The discrepancy between commercial-heavy and standard sites depends largely on two factors: whether our network operates deliveries on the holiday itself, and whether a majority of businesses are closed for observance. Based on historical data analysis, holidays can be categorized into three groups, each with different implications for DoW planning.
Table 1: Holiday Classification and Planning Implications
| Group | Network Delivery Status | Typical Business Status | Example Holidays | Planning Implication |
|---|---|---|---|---|
| A | Non-Delivery | Closed | New Year’s Day, Independence Day, Thanksgiving, Christmas | Not suitable for this model; inconsistent patterns observed; includes peak periods prioritizing speed over accuracy. |
| B | Delivery | Closed | Presidents Day, Memorial Day, Labor Day, Columbus Day | Suitable for custom model; consistent lightness on the holiday and heaviness on subsequent days observed. |
| C | Delivery | Open | MLK Day, Juneteenth | Not suitable for this model; minimal DoW pattern deviations observed as businesses remain open. |
Analysis revealed that Group B holidays, where the network delivers but many businesses are closed, consistently showed volume lightness on the holiday itself ('Day-of') followed by heavier volume on subsequent days. This predictable pattern presented the clearest opportunity for improvement through a tailored forecasting model. Group A holidays showed inconsistent patterns, and Group C holidays exhibited minimal deviations, offering limited scope for this specific adjustment.
The Proposed Solution: A Customized DoW Curve Model
To address the unique patterns of commercial-heavy sites during Group B holidays, we developed a novel methodology. The core idea is to create a more accurate DoW forecast by blending the standard network curve with a curve that reflects the typical behavior of purely commercial delivery areas. This approach acknowledges that these sites exhibit characteristics of both residential and commercial delivery patterns.
The model requires downward adjustments to the standard holiday week forecast for weekends and the 'Day-of' holiday, reflecting reduced commercial activity. Conversely, upward corrections are applied to the days immediately following the holiday (D+1, D+2, D+3) to account for the catch-up volume.
How the Model Works
The calculation involves a weighted average approach:
- Network DoW Curve: This serves as the baseline, representing the expected pattern for a typical site within the network.
- Pseudo Commercial DoW Curve: A hypothetical curve is constructed to represent a 100% commercial delivery area. This curve features significantly lower volume percentages for weekends and the specific holiday 'Day-of', based on parameters derived from historical data analysis. The volume percentages for the remaining weekdays are then scaled up proportionally from the network curve to ensure the total weekly volume remains consistent. These parameters, initially set based on analysis (e.g., 7% for Sundays/Holidays, 8% for Saturdays), can be further optimized using techniques like Linear Programming.
- Station-Specific DoW Curve: The final forecast curve for a commercial-heavy station is calculated by taking a weighted average of the Network DoW Curve and the Pseudo Commercial DoW Curve. The weighting factor is determined by the station's specific percentage of commercial volume. For instance, a station with 41.2% commercial volume would have its final curve calculated as (Network Curve * (1 - 0.412)) + (Pseudo Commercial Curve * 0.412).
This weighted approach allows the model to dynamically adjust the forecast based on how heavily commercial a particular site is, providing a much more tailored and accurate prediction than the one-size-fits-all network curve.
Validation and Results: Significant Improvements Achieved
The proposed model underwent rigorous testing using historical data from 2023's Memorial Day and Labor Day weeks, focusing on stations identified as having high commercial volume (defined as 15% or more commercial share, based on analysis showing noticeable DoW gaps above this threshold). The results were highly encouraging.
For Memorial Day, applying the custom station curves resulted in an average DoW MAPE of 7.4%, a significant improvement compared to the 11.4% MAPE observed when using only the standard network curve – a reduction of 395 bps. Similarly, for Labor Day, the custom model achieved a MAPE of 5.9% versus the network curve's 8.7%, an improvement of 278 bps. These improvements stem directly from the model's ability to correctly anticipate lower volume on business closure days and higher volume on subsequent days, proportionate to each station's commercial volume share.
Further validation during the 2024 Memorial Day week deployment confirmed the model's effectiveness. Eleven sites utilized the special DoW model. Comparing performance to the previous year showed substantial enhancements in key metrics. Capacity Ask Weighted Average Percentage Error (WAPE), a critical measure of forecast accuracy impacting capacity planning, improved dramatically. For the forecast generated three weeks prior (W-3), WAPE improved by 397 bps (from 9.8% in 2023 to 5.8% in 2024). For the forecast one week prior (W-1), WAPE improved by 355 bps (from 9.1% to 5.6%). Much of this gain came from accurately reducing the forecast for Monday (Memorial Day) and increasing it for the following days. The input metric MAPE also showed strong results, with the custom override achieving 5.3% MAPE compared to the network curve's 11.1% MAPE for the same period, a 573 bps beat.
Implementation and Next Steps
Following successful validation and alignment with stakeholders, the model is targeted for broader implementation. The parameters used in the pseudo commercial curve (weekend and holiday DoW percentages) will be finalized after optimization using historical data.
The preferred deployment method is central application via an enhancement to the existing central forecasting scripts. This ensures consistency, allows for easier performance tracking, and reduces manual effort compared to individual planners applying the method locally. This centralized adjustment will take precedence over standard network curves for the designated commercial-heavy sites during applicable holiday weeks. The initial rollout targeted the Week 15 planning cycle.
This initiative represents a significant step forward in refining our forecasting capabilities. By recognizing and adapting to the distinct operational patterns of commercial-heavy delivery sites during holidays, we can improve planning accuracy, optimize resource utilization, and ultimately enhance our service delivery.