[<Library component: Models|Core|etc...>] GreyKite/Silverkite API support

[sudhanshu-shukla-git]

Description

Description
Hi Team,
Is there any support for GeyKite/Silverkite model.

Any suggestion how can we integrate the external model with the Hierarchical Forecast

Use case

Currently using Hierarchical Forecasting , the results are not good with supported models. Tried separately with Greykite and it gives the best result. I want to integrate multilevel forecasting with Greykite.

_

[elephaint]

As long as you make the forecasts into the format that HierarchicalReconciliation expects you can use any model.

Basically I'd follow this tutorial, where you need to replace Y_hat_df and Y_fitted_df by something coming from your own model, but you need to make sure it's in the same format. This should be relatively straightforward, possibly with the help of the aggregate function.

If you provide a piece of code that I can run demonstrating your pipeline where you're struggling with a specific line I'm happy to help further.

[sudhanshu-shukla-git]

@elephaint Thanks for the suggestion. Below is the code we are using to create Y_hat_df and Y_fitted_df using Greykite/Silverkite.
Let me know if you feel the approach is fine or if something better can be done.

#Using specific version to make sure there are no conflicts
requirements.txt
greykite==1.0.0
scikit-learn==1.3.1
statsforecast==1.7.8
hierarchicalforecast==0.4.3

from hierarchicalforecast.core import HierarchicalReconciliation
from hierarchicalforecast.methods import  BottomUp, TopDown
from greykite.framework.templates.autogen.forecast_config import ForecastConfig
from greykite.framework.templates.autogen.forecast_config import MetadataParam
from greykite.framework.templates.forecaster import Forecaster
from greykite.framework.templates.model_templates import ModelTemplateEnum


def build_forecast(df, time_col, value_col, freq, forecast_horizon):
    """
    Build a forecast using Greykite's Forecaster.

    Parameters:
    - df: pd.DataFrame - The input DataFrame containing the time series data.
    - time_col: str - The name of the time column.
    - value_col: str - The name of the value column.
    - freq: str - Frequency of the time series (e.g., "B", "D", "W", "MS").
    - forecast_horizon: int - Number of steps to forecast ahead.

    Returns:
    - pd.DataFrame - DataFrame containing the forecasted values.
    """
    
    # Specifies dataset information
    metadata = MetadataParam(
        time_col=time_col,
        value_col=value_col,
        freq=freq
    )
    
    # Initialize Forecaster
    forecaster = Forecaster()

    # Run forecast configuration
    result = forecaster.run_forecast_config(
        df=df,
        config=ForecastConfig(
            model_template=ModelTemplateEnum.SILVERKITE.name,
            forecast_horizon=forecast_horizon,
            #error_score='raise',
            metadata_param=metadata
        )
    )

    # Return the forecasted DataFrame
    return result.forecast.df

#Generate forecasts for each unique_id
forecasts = {}


for uid in Y_train_df.index.unique():
    print(f"Processing UID: {uid}")
    try:
        loc = Y_train_df.index.get_loc(uid)
        subset = Y_train_df.iloc[loc] 
        print(subset.shape)# Using iloc to get the row by its position
        forecasts[uid] = build_forecast(subset, "ds", "y", "B", 11)
    except KeyError:
        print(f"Warning: {uid} not found in index.")

#combine forecasts into a single DataFrame
Y_fitted_df = pd.concat([forecasts[uid].assign(unique_id = uid) for uid in forecasts])
Y_fitted_df.columns= ["ds","y","Silverkite","unique_id"]
# Y_fitted_df
Y_hat_df = Y_fitted_df.drop(columns=["y"])

reconcilers = [BottomUp(),
               TopDown(method='forecast_proportions')]
            
rec_model = HierarchicalReconciliation(reconcilers=reconcilers)
Y_rec_df = hrec.reconcile(Y_hat_df=Y_hat_df, Y_df=Y_fitted_df, S=S_df, tags=tags)

[elephaint]

@elephaint Thanks for the suggestion. Below is the code we are using to create Y_hat_df and Y_fitted_df using Greykite/Silverkite. Let me know if you feel the approach is fine or if something better can be done.

#Using specific version to make sure there are no conflicts
requirements.txt
greykite==1.0.0
scikit-learn==1.3.1
statsforecast==1.7.8
hierarchicalforecast==0.4.3

from hierarchicalforecast.core import HierarchicalReconciliation
from hierarchicalforecast.methods import BottomUp, TopDown
from greykite.framework.templates.autogen.forecast_config import ForecastConfig
from greykite.framework.templates.autogen.forecast_config import MetadataParam
from greykite.framework.templates.forecaster import Forecaster
from greykite.framework.templates.model_templates import ModelTemplateEnum

def build_forecast(df, time_col, value_col, freq, forecast_horizon):
"""
Build a forecast using Greykite's Forecaster.

Parameters:
- df: pd.DataFrame - The input DataFrame containing the time series data.
- time_col: str - The name of the time column.
- value_col: str - The name of the value column.
- freq: str - Frequency of the time series (e.g., "B", "D", "W", "MS").
- forecast_horizon: int - Number of steps to forecast ahead.

Returns:
- pd.DataFrame - DataFrame containing the forecasted values.
"""

# Specifies dataset information
metadata = MetadataParam(
    time_col=time_col,
    value_col=value_col,
    freq=freq
)

# Initialize Forecaster
forecaster = Forecaster()

# Run forecast configuration
result = forecaster.run_forecast_config(
    df=df,
    config=ForecastConfig(
        model_template=ModelTemplateEnum.SILVERKITE.name,
        forecast_horizon=forecast_horizon,
        #error_score='raise',
        metadata_param=metadata
    )
)

# Return the forecasted DataFrame
return result.forecast.df

#Generate forecasts for each unique_id
forecasts = {}

for uid in Y_train_df.index.unique():
print(f"Processing UID: {uid}")
try:
loc = Y_train_df.index.get_loc(uid)
subset = Y_train_df.iloc[loc]
print(subset.shape)# Using iloc to get the row by its position
forecasts[uid] = build_forecast(subset, "ds", "y", "B", 11)
except KeyError:
print(f"Warning: {uid} not found in index.")

#combine forecasts into a single DataFrame
Y_fitted_df = pd.concat([forecasts[uid].assign(unique_id = uid) for uid in forecasts])
Y_fitted_df.columns= ["ds","y","Silverkite","unique_id"]

Y_fitted_df

Y_hat_df = Y_fitted_df.drop(columns=["y"])
reconcilers = [BottomUp(),
TopDown(method='forecast_proportions')]

rec_model = HierarchicalReconciliation(reconcilers=reconcilers)
Y_rec_df = hrec.reconcile(Y_hat_df=Y_hat_df, Y_df=Y_fitted_df, S=S_df, tags=tags)

This looks good to me - I think this could work, assuming your unique_id contains all the hierarchies you seek to reconcile.