Search results

Support ORPO/DPO Liger losses (and LigerORPOTrainer)

enhancementNew feature or requestNew feature or request

wip

Status: Open.

#2141 In axolotl-ai-cloud/axolotl;

· ccdv-ai opened on Dec 6, 2024

[docker] experiment with nvcr pytorch image for torch 2.5.1

wip

Status: Open (in progress).

axolotl-ai-cloud/axolotlnumber 2139

#2139 In axolotl-ai-cloud/axolotl;

· winglian opened on Dec 6, 2024

Deepspeed zero3 + LoRA: RuntimeError: Only Tensors of floating point and complex dtype can require gradients

bugSomething isn't workingSomething isn't working

waiting on upstream

wip

Status: Open.

#2068 In axolotl-ai-cloud/axolotl;

· bursteratom opened on Nov 16, 2024

Different training losses when flash_attention is on/off

bugSomething isn't workingSomething isn't working

wip

Status: Open.

#1918 In axolotl-ai-cloud/axolotl;

· zhangchen-xu opened on Sep 18, 2024

Integrate LLaVA for multimodal pre-training

enhancementNew feature or requestNew feature or request

wip

Status: Draft (not ready).

axolotl-ai-cloud/axolotlnumber 781

#781 In axolotl-ai-cloud/axolotl;

· winglian opened on Oct 24, 2023

[WIP][Feature] Tensor parallel

enhancementNew feature or requestNew feature or request

wip

Status: Open (in progress).

axolotl-ai-cloud/axolotlnumber 539

#539 In axolotl-ai-cloud/axolotl;

· winglian opened on Sep 8, 2023