Repository navigation
fix(options): annualize pricing volatility on the run's bar cadence - #1753
Conversation
historical_volatility() scaled the rolling log-return standard deviation by a hardcoded sqrt(252), and run_options_backtest() never forwarded the bars_per_year the runner had already resolved for it, so every option leg was priced on 252 bars/year whatever the venue actually serves. The same run's metrics already annualize through effective_bars_per_year, whose contract is that the Sharpe, the annualised volatility and the validation Sharpe of one run card are annualised identically; pricing was the one consumer left out. Crypto sources map to 365 daily bars and okx hourly bars to 8760, and a cross-market basket passes None. On a deterministic daily series an at-the-money 30-day call was marked 16.02% below the price it should have filled at, with no warning, into trades.csv, greeks.csv, equity.csv, metrics.csv and the backtest_summary tool result. Forward the factor and resolve None through effective_bars_per_year, exactly as _calc_options_metrics does. The parameter defaults to 252, so daily-equity behaviour and the pinned warm-up regression are unchanged. Same convention as 8afa387, which resolved the risk x-ray's hardcoded 252 through calc_bars_per_year. Signed-off-by: Steven Huang <134844631+remote-controlled-man@users.noreply.github.com>
he-yufeng
left a comment
There was a problem hiding this comment.
Verified locally against current main (e532650).
Red/green: copied the new test file onto the parent tree, and all 5 tests fail there, the end-to-end one for exactly the right reason. On main the leg fills at 3.8135 where the 365-annualised vol says 4.5409, the 16% gap from your table. On the PR head all 5 pass, and the full option subset (369 passed, 6 skipped, including the pinned warm-up suite) stays green.
Mechanism checks, all hold:
effective_bars_per_yearwas already imported in options_portfolio.py, and_calc_options_metricsalready receives the factor at the line 615 call site, so pricing really was the one consumer left out of the run's cadence.- runner.py:1295-1314 confirms the None convention for cross-market baskets and that the resolved factor reaches
run_options_backtest. - 8afa387 is real and this follows the same convention.
effective_bars_per_yearfalls back to 252 on an empty or non-datetime index, so a None paired with an odd index degrades to the old behaviour instead of crashing.
One non-blocking thought: historical_volatility now accepts a bars_per_year of 0 or below, where _calc_options_metrics has an explicit guard with a warning for that case. Only reachable by a direct caller passing garbage, since every real call site resolves through calc/effective, so fine as is.
For the record, this touches the same file as my #1750 but disjoint regions (HV and the iv_map call site here, the maxdd seed inside _calc_options_metrics there), so either merge order should apply clean.
…lity on the run's bar cadence (cherry picked from commit 45d6713)
Summary
historical_volatility()annualized at a hardcodednp.sqrt(252), and the call site inrun_options_backtest()never forwarded thebars_per_yearthe run had already resolved — so every option leg was priced on 252 bars/year no matter what the venue actually serves.Nonethrougheffective_bars_per_yearexactly as_calc_options_metricsalready does. The parameter defaults to252, so existing daily-equity behaviour is unchanged.Why
backtest/metrics.py::calc_bars_per_yearmaps every crypto source to 365 daily bars and okx hourly bars to 8760, andrunner.pypassesbars_per_year=Nonefor a cross-market basket.run_options_backtest()receives that value and forwards it to_calc_options_metrics, which resolvesNonethrougheffective_bars_per_yearand honours it for CAGR, Sharpe and Sortino. It was not forwarded tohistorical_volatility, which produces the per-bar vol every leg is opened, marked and greeked at.Measured on a deterministic 60-bar daily series (alternating +2% / -1.5%), one at-the-money 30-day call,
risk_free_rate=0.05, flat smile:bars_per_year=252(current behaviour)bars_per_year=365(what an okx/binance daily run resolves to)The stale factor marked the leg 16.0% below the price it should have filled at, with
status: okand no warning, and that number propagated intoartifacts/trades.csv,greeks.csv,equity.csv,metrics.csvand thebacktest_summarytool result. Intraday is further off, since okx1Hresolves to 8760 bars/year.effective_bars_per_yeardocuments the intended invariant:The annualised volatility is exactly this function's output, and
backtest/validation.pyalready annualizes withsqrt(bars_per_year). Pricing was the one consumer left out. This is the options engine's copy of the bug fixed in8afa3879("fix(portfolio): annualize weekly and monthly risk bars correctly"), which resolved the risk x-ray's hardcoded 252 throughcalc_bars_per_year.Changes
agent/backtest/engines/options_portfolio.pyhistorical_volatility()takesbars_per_year: int | None = 252and scales bynp.sqrt(bars_per_year).Noneresolves througheffective_bars_per_year(close.index), the same helper the metrics path uses, so a leg is priced and graded on one cadence. Its Google-styleArgs:docstring documents the parameter, theNoneconvention and the 365/8760 venue factors — that docstring entry is the documentation change behind the checklist box below.iv_mapbuild inrun_options_backtest()forwards the factor it was already given. It resolves per code, because a cross-market basket has no single factor — which is why the runner passesNoneat all.agent/tests/test_options_hv_bars_per_year.py(new, offline): five regressions — a supplied cadence, the unchanged252default,Noneresolved from the observed span (the convention restated fromlen / elapsed yearsrather than by calling the helper), and two end-to-end runs throughrun_options_backtestthat assert the recorded fill price against an independent Black-Scholes oracle.Deliberately out of scope:
calc_bars_per_yearmapping table, to Sharpe/CAGR/Sortino, or tovalidation.py.options_portfolio.pyis already 876 lines onmainagainst the 800-line hard cap; splitting it is a separate change, and this PR adds a net 16 lines to it.effective_bars_per_yearcan return a small factor for a pathologically sparse span. That behaviour is shared with the metrics path and is unchanged here.Test Plan
pytest --ignore=agent/tests/e2e_backtest --tb=short -q)Focused:
test_options_hv_warmup.py::test_full_window_bars_are_unchanged_by_the_warmup_fixpinsrolling(30).std() * np.sqrt(252); it still passes because the new parameter defaults to252.Every test file referencing the changed symbols, the runner or the backtest tools (41 files):
The 3 failures are
OSError: [WinError 1314]symlink-privilege guards intest_backtest_summary.py. They are host artifacts, not regressions: reverting the changed file tomainand re-running the same 40 pre-existing files gives1226 passed, 3 failed, 1 skippedwith an identical failure node-ID set. The patch adds 5 passing tests and zero failures.Full offline selection on this Windows host (
pytest --ignore=agent/tests/e2e_backtest --ignore=agent/tests/test_e2e_harness_v2.py -q):16977 passed, 43 failed, 168 skipped, 4 errors. The same selection onmainwith this change reverted gives16971 passed, 44 failed, 168 skipped, 4 errors, and the patched failure node-ID set is a strict subset of the baseline's, so nothing new fails. The classes are Windows symlink-privilege guards (WinError 1314), packaging-metadata assertions that assume a non-editable install, one Unix-domain-socket test, console-encoding collection errors on very large unicode parametrizations, and assorted environment/registry assertions in unrelated modules. The single baseline-only failure,test_runtime_jobstore.py::test_concurrent_saves_use_distinct_temporary_files, is nondeterministic on unmodifiedmain— 2 failed / 4 passed across six identical runs. CI runs this selection on Ubuntu, where those classes do not apply.Manual reproduction:
run_options_backtestdriven directly by the two end-to-end tests above, reading the fill price back out ofartifacts/trades.csv. Synthetic deterministic closes only.ruff checkclean on both changed files.black --checkis not satisfied, but it is not satisfied bymaineither —pyproject.tomlhas no[tool.black]section, so black applies its 88-column default while the repo configures ruff atline-length = 120, and every pre-existing sibling options test failsblack --checkas committed. Per CONTRIBUTING's "avoid mixing unrelated formatting cleanup into a focused pull request", no reformatting was applied. Longest new line is 96 characters.Risk
No live, broker, MCP, network, secret or deployment risk: the change is pure arithmetic on an already-resolved integer, exercised offline against synthetic series. No order gate, mandate check, OAuth path, provider connector or external call is touched. No dependency, lockfile, CI or config change.
Rollback
Single commit;
git revertrestores the hardcodednp.sqrt(252)and drops the new test file. No schema, artifact-format or config change, so runs produced before and after remain readable either way.Checklist
src/agent/,src/session/,src/providers/) without prior discussion