Member-only story
3 Types of Data Distribution Shifts in ML Systems
Why continuous monitoring in ML systems is of vital importance
My wife heard about a new cleaning product from her friend, who laid it on with a trowel. She praised the product as if it was the greatest invention of the century.
My wife ordered the product online and used it for cleaning a few different surfaces. I could immediately tell her friend was right about the product from my wife’s exaggerated “wow!”.
About a month later when we went to the grocery store, we noticed large amounts of that magnificent cleaning product in the detergent aisle.
The person who was responsible for the inventory management of the store seemed to make the right decision purchasing excess amounts of the product. Considering the popularity of the product and word of mouth, it wasn’t a difficult decision to make.
It might be an easy decision for a single product. However, a big retail store typically has over 10 thousand SKUs — a number retailers assign to products to keep track of stock levels. For such stores, it’s not possible to do inventory management by hand calculations, gut sense, or based on the word of mouth. They use forecasting models, which can be as simple as a moving average model or a highly complex XGBoost model with…
