Collect data on performance improvements to the dashboard

[shankari]

Plan discussed at meeting

We will use the following environments:

• stage
• ccebikes (large: 86k trips, 108 users)
• smart commute (medium, long-term: 18k trips, 21 users)
• STM community (small: 735 trips, 4 users)

We will access the environments at the following times:

• @JGreenlee 8am MT
• @JGreenlee 12pm MT
• @JGreenlee + @TeachMeTW + @shankari 4pm MT
• @TeachMeTW 8pm MT
• @TeachMeTW 12am MT

Steps to perform:

• Log in aka "cold boot" of the dashboard (initializes to "Overview")
• Data page, UUIDs table, Trip table, Trajectories table
• Maps page with no filters

We will collect data for one week for the baseline and one week for each change.
We will start with reverting the previous batching changes so that we can assess the impact before moving on.

Timeline:

• 21 Oct -25 Oct: baseline (Temporary Logging #141 + new PR for reversion)
• 28 Oct - 1 Nov: batching changes (Performance Improvements in User Stats Processing #124, Refactor and Improvements to #124 #127, Fix: Retain Current Page in UUID Table During Lazy Loading #142)
• 4 Nov - 8 Nov: overview changes (Performance Improvements in Home Page Processing #126 + any bugfixes to Performance Improvements in Home Page Processing #126)

[shankari]

to clarify, when I talk about "new PR for reversion", I expect the following

1. git revert b9b0c347a633a1f44bf697b4cd50e720a279ac09
2. commit (generate SHA1)
3. add timing (generate SHA2)
4. we determine baseline
5. then, we do git revert SHA2; commit
6. git revert SHA1; commit
7. then we can test the new code

[JGreenlee]

1. master has been rolled back to Sep 2, before @TeachMeTW started working on the dashboard
2. what used to be master is now called future
3. @TeachMeTW is going to add timing to the new master. Once that is done, it is considered the "baseline".
4. We will push the baseline to staging

We aim to have the baseline on staging and tested tonight. If good, we'll promote to the 3 prod environments we are testing on, and I will perform the first access time @ 8am MT tomorrow.

If any delay, we will push this back to 8am MT Wednesday.

[shankari]

I now see the float error again; we need to cherry-pick that change

ERROR:app_sidebar_collapsible:Exception on /_dash-update-component [POST]
Traceback (most recent call last):
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/flask/app.py", line 2529, in wsgi_app
    response = self.full_dispatch_request()
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/dash/dash.py", line 1310, in dispatch
    ctx.run(
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/dash/_callback.py", line 442, in add_context
    output_value = func(*func_args, **func_kwargs)  # %% callback invoked %%
  File "/usr/src/app/app_sidebar_collapsible.py", line 318, in update_store_trips
    df, user_input_cols = query_confirmed_trips(start_date, end_date, timezone)
  File "/usr/src/app/utils/db_utils.py", line 183, in query_confirmed_trips
    df["data.primary_ble_sensed_mode"] = df.ble_sensed_summary.apply(get_max_mode_from_summary)
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/pandas/core/series.py", line 4771, in apply
    return SeriesApply(self, func, convert_dtype, args, kwargs).apply()
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/pandas/core/apply.py", line 1123, in apply
    return self.apply_standard()
  File "/root/miniconda-23.5.2/envs/emission/lib/python3.9/site-packages/pandas/core/apply.py", line 1174, in apply_standard
    mapped = lib.map_infer(
  File "pandas/_libs/lib.pyx", line 2924, in pandas._libs.lib.map_infer
  File "/usr/src/app/utils/db_utils.py", line 179, in <lambda>
    get_max_mode_from_summary = lambda md: max(md["distance"], key=md["distance"].get) if len(md["distance"]) > 0 else "INVALID"
TypeError: 'float' object is not subscriptable

• Trips trend does not show up
• Trips table is still blank
• Contrary to @TeachMeTW's experience, the UUID table is not blank

[shankari]

Re-opening since this is not actually done.

[shankari]

While collecting data at 4pm MT (3pm PT) today, I ran into some blank tables, likely due to timeouts.

e.g.

Object { message: "Callback error updating tabs-content.children", html: "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n" }
[dash_renderer.v2_15_0m1729614367.min.js:2:95440](https://openpath-stage.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js)
    _o https://openpath-stage.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    ui https://openpath-stage.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    r https://openpath-stage.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    r https://openpath-stage.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    p https://openpath-stage.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    nt https://openpath-stage.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    zi https://openpath-stage.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2

Not sure if we are recording that anywhere or can record it given that it is an error in the plotly framework

Object { message: "Callback error updating card-active-users.children", html: "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n" }
[dash_renderer.v2_15_0m1729614367.min.js:2:95440](https://ccebikes-openpath.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js)
    _o https://ccebikes-openpath.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    ui https://ccebikes-openpath.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    r https://ccebikes-openpath.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    r https://ccebikes-openpath.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    p https://ccebikes-openpath.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    nt https://ccebikes-openpath.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2
    zi https://ccebikes-openpath.nrel.gov/admin/_dash-component-suites/dash/dash-renderer/build/dash_renderer.v2_15_0m1729614367.min.js:2

[TeachMeTW]

Unlike @shankari's findings; all STM, Ebike, Smart Commute, and Stage loaded fine for me

Tested: 4 PM MT - 4:47 MT

Load order speeds [STM, Smart Commute, Stage, ccebikes ]

-Initial Load

-From sidebar click "Data"

-Click "Trips" tab

-Click "Demographics" tab

-Click "Trajectories" tab

-From sidebar click "Map"

-Select "Density Heatmap"

-Select "Unlabeled"

Everything functioned normally for me

[JGreenlee]

As I reported on Teams this morning:

On ccebikes, UUIDs and Trajectories both timed out after several minutes and failed to display

This held true for the 12pm and 4pm accesses as well. No timeouts on smart commute or stm community.

[shankari]

@JGreenlee what about on stage? Are you testing that periodically as well?

[JGreenlee]

I tested Stage just now. UUIDs eventually loaded. Trips, Trajectories, and Heatmap were empty – I don't think it's due to a timeout, I think it is due to a lack of data in the last week

Starting tomorrow I'll access Stage along with the others.

[TeachMeTW]

8 PM MT:

• CCEbike did not load active users box
• Smart Commute did not load uuids
• Stage did not load uuids or trajectories or trips (might need to add date change to this test?)

[TeachMeTW]

12 AM MT:

• CCEbike did not load active users box
• Smart Commute now loads uuids
• CCEbike long load times in trajectories
• Stage loads uuids
• Stage no trips or trajectories