Inference mode for time series models

[isaacmg]

Create a predict function which does inference for for time series models without requiring the target present. This module should initialize the model using the given configuration file (with a weight path). It should be able to consume a CSV file or query a SQL table #102 (thought this functionality is not required in the initial PR). It should ideally make use of the existing evaluator.py module but

Acceptance Criteria

• Passing tests

[isaacmg]

This can now consume a CSV file successfully. I will make a separate issue for the full SQL enhancement and such.

[pnmartinez]

Hi,

First, thanks for the great work! This is an amazing project 😄

I am trying to forecast using my already fitted models, having followed the main tutorial (Forecasting-Tutorial.ipynb).

For that, I am trying to follow your video on inference (Deep Learning For Time Series: Interpretability, and Deployment), but my WAND panel doesn't show any gcs_m_path_10_model or gcs_m_path_10_params fields.

I figured I'd need to re-run everything with GCS = True, but can you please confirm I am on the right direction?

---

UPDATE

I set up a GCS project correctly, uncommented the provided the environment variables (MODEL_BUCKET, ENVIRONMENT_GCP, GCP_PROJECT), added the variable os.environ["GOOGLE_APPLICATION_CREDENTIALS"] pointing to the JSON credentials for the project.

Everything runs well, without errors, WANDB panel shows GCS = true, **but still can't find on WANDB panel the variables gcs_m_path_10_model, gcs_m_path_10_params!

---

2nd UPDATE

My previous problem is solved. This morning has been a crash course about GCS. Now everything is clear and perfect to go.

However I would love to know your opinion about GCS alternatives. When performing inference: I could simply download the files from WANDB, and upload them manually to a Colab instance, right?

[pnmartinez]

I have more problems with the Inference notebook @isaacmg .

• For starters, the notebook has some typos that are not seen in the tutorial video (e.g. on section Initializing Inference, line after config_test variable definition modified the relevant_columns for a specific dataset), though I corrected those kinds of things,
• but I keep having errors calling InferenceMode(...) methods, see log below.

[In]: tensor, history, test, plt = d.make_plots(datetime(2016, 2, 15), new_water_data_path, wandb_plot_id="deploy_plot_777", save_name="preds2.csv")

[Out]: []
Now loading./train.csv
scaling now
CSV Path below
./train.csv

/content/flow-forecast/flood_forecast/preprocessing/pytorch_loaders.py:88: SettingWithCopyWarning:


A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

/usr/local/lib/python3.6/dist-packages/pandas/core/indexing.py:1734: SettingWithCopyWarning:


A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

---------------------------------------------------------------------------

IndexError                                Traceback (most recent call last)

<ipython-input-39-0e5281aeff8a> in <module>()
      1 from datetime import datetime
      2 # To do make path default
----> 3 tensor, history, test, plt = d.make_plots(datetime(2016, 2, 15), new_water_data_path, wandb_plot_id="deploy_plot_777", save_name="preds2.csv")

4 frames

/usr/local/lib/python3.6/dist-packages/pandas/core/indexes/base.py in __getitem__(self, key)
   4102         if is_scalar(key):
   4103             key = com.cast_scalar_indexer(key, warn_float=True)
-> 4104             return getitem(key)
   4105 
   4106         if isinstance(key, slice):

IndexError: index 0 is out of bounds for axis 0 with size 0

[isaacmg]

@pnmartinez That error is almost always due to the specified date time not being present in the CSV file you are using or the date time not being cast to the proper type. You should check that the date is there.

Regarding the files you could download them from wherever they stored. Wandb is not a file storage system it merely stores a record to the file path. Our framework automatically parses GCP paths to download files from GCP. If you had a file on AWS/Azure/IBM or wherever you would have to download it manually to Colab then you could use it.

[pnmartinez]

@isaacmg thanks for the input on the storage alternatives!

Regarding the error on the Inference method, that date sure is on the dataset: I've used the train.csv dataset used on the tutorial (link).

[isaacmg]

@pnmartinez Did you use a column called datetime like in the tutorial? At the moment evaluator/infer looks for a literal column called datetime. So if the df doesn't have it will fail. This is an issue #227

# We will preform very basic data techniques to just get the weekday.
import pandas as pd
import datetime as datetime
df = pd.read_csv("train.csv")
df["day_of_week"] = df["5 Minutes"].map(lambda x: datetime.datetime.strptime(x, '%d/%m/%Y %H:%M').weekday())
df["datetime"] = df['5 Minutes']

[pnmartinez]

@isaacmg yes, I copied that cell specifically. See below the df used:

• The date is on the df,
• the df has a datetime column.

Just in case you see anything else in my code, here's how I am initializing the Inference:

import json
!gsutil cp gs://transformers_tests/experiments/13_January_202103_16PM.json config.json
weight_path = "gs://transformers_tests/experiments/13_January_202103_16PM_model.pth"

# Original training dataset (for testing)
new_water_data_path = "./train.csv"

# To do add support for loading config file from path (local or GCS)
with open("config.json") as y:
  config_test = json.load(y)

# InferenceMode
d = InferenceMode(336, 30, config_test, new_water_data_path, weight_path, "uncategorized")

tensor, history, test, plt = d.make_plots(datetime(2016, 2, 15), new_water_data_path, wandb_plot_id="deploy_plot_777", save_name="preds2.csv")