v0.21.0

TLDR

• Torch version has been bumped to 2.7.0
• Support FSDP2 via a ENV VAR: FSDP_VERSION=2. Currently it only supports pretraining (with meta init). No yaml change is needed to enable FSDP2, the attrs that only apply to FSDP(1) will be ignored and raised as warnings. See composer release for more details

What's Changed

• Adding support for nope positional encoding in block overrides. by @ShashankMosaicML in #1794
• Bump foundry version to 0.21.0.dev0 by @dakinggg in #1812
• Adding temperature tuning in attention by @ShashankMosaicML in #1793
• Update foundry version in MCLI yamls by @dakinggg in #1813
• Upgrade yapf version by @dakinggg in #1814
• Allow subselecting the appropriate config for llama4 by @dakinggg in #1815
• Change RMSNorm to use PyTorch native implementation by @josejg in #1809
• Update datasets requirement from <3.6,>=3.3.2 to >=3.3.2,<3.7 by @dependabot in #1817
• Bump onnxruntime from 1.19.2 to 1.22.0 by @dependabot in #1819
• Update huggingface-hub[hf_xet] requirement from <0.31,>=0.30.0 to >=0.30.0,<0.32 by @dependabot in #1818
• Deprecate inference API wrappers by @dakinggg in #1821
• Fix Dtensor initialization by @bowenyang008 in #1820
• Update accelerate requirement from <1.7,>=0.25 to >=0.25,<1.8 by @dependabot in #1824
• Bump onnx from 1.17.0 to 1.18.0 by @dependabot in #1823
• Bump docformatter for python3.12 and change blank_line_before_module_docstring = false by @sashaDoubov in #1825
• Delete useless print("here") by @tsebaka in #1826
• Update ci-testing version to latest by @dakinggg in #1827
• Bump coverage[toml] from 7.8.0 to 7.8.2 by @dependabot in #1830
• Configurable shard size by @dakinggg in #1833
• Bump Composer 0.31.0 by @bowenyang008 in #1835
• Fix monolithic checkpointing against composer main by @dakinggg in #1836
• Bump torch version to 2.7 by @bowenyang008 in #1832
• bump huggingface-hub upper bound to 0.33 by @bowenyang008 in #1838

New Contributors

• @bowenyang008 made their first contribution in #1820
• @tsebaka made their first contribution in #1826

Full Changelog: v0.20.0...v0.21.0