Renaming and Organization of RL algorithms in preparation for Development

[jdchang1]

I apologize in advance since this shuffling/renaming will require a bit of updating of yamls and dependencies. My main goal is to make the organization of the code a bit more semantically meaningful and hopefully clearer to onboard. Here is a summary of the main changes

• Removed all algorithm specific references like DPO and PPO when we are using these as general purpose abstractions of families of algorithms
• Moved algorithm code to an algorithms directory
• shuffled some files to more meaningful directories. I.e. buffers.py => data rather than keep only in online
• unified between offline, online, and reward_modeling directories to use model_methods.py to implement forward and loss variants.
• Split online_rl_loss to policy_loss and critic_loss
• Added OnPolicyEnum and Algorithm_Type Enums for loss_type similar to PairwiseOfflineEnum
• Made online_rl_loss just return return_dict similar to other algorithm pipelines

MCLI run names for each pipeline:

Reward Training: reward-reorg-aWY7m3
Offline Training: rebel-reorg-nboVUO
Online Training: grpo-reorg-fK2HDm

MLFlow Link

[gupta-abhay]

looks good - will be good to maybe run a test for dpo / grpo with main and this branch with the appropriate yaml changes to show things look good.

[bcui-db]

LGTM! Thank you! As mentioned above, let's just test RMs, DPO, and an online RL training flow to make sure things still work.

[jdchang1]

MCLI run names for each pipeline:

Reward Training: reward-reorg-aWY7m3
Offline Training: rebel-reorg-nboVUO
Online Training: grpo-reorg-fK2HDm

MLFlow Link

[bcui-db]

Discussed offline, and overall LGTM! thank you for this! I'd also want @gupta-abhay or @dakinggg to stamp to just be sure, but from my end LGTM

[gupta-abhay]

LGTM! Please update yamls here (and let's figure out a path away from the bwd compatible stuff asap - unless TAO production is hinging on that old setup).

Huge thanks for doing this :)