Contact forces are zero in Ant-v2/v3

[pratikac]

The contact forces are all zero in the the MuJoCo Ant-v2/v3 environments. If one runs

import gym
import numpy as np

e = gym.make('Ant-v3')
for _ in range(100):
    e.reset()
    for i in range(1000):
        x=e.step(e.action_space.sample())[0]
        print(np.linalg.norm(x[27:]))

the last 84 dimensions in the state out of 111 dimensions are always zero. These dimensions are the contact forces in ant_v3.py (https://github.com/openai/gym/blob/master/gym/envs/mujoco/ant_v3.py#L124). Since the ant is in contact with the ground, I believe there should be some contact forces that are non-zero. Can you please clarify if this is the expected behavior?

[AlexanderGri]

The same goes for the Humanoid environment - contact forces are zero in the case of MuJoCo 2.0 . The environment uses (v2 and v3) the cfrc_ext field of PyMjData which is a proxy to the cfrc_ext field of MuJoCo C structure _mjData . But the changelog to MuJoCo 2.0 contains the following line:

The function mj_rnePostConstraint is no longer called by default. Instead, it is called only when the model contains force-related sensors which need the results of this function. The user can still call it directly in order to compute the mjData fields cacc, cfrc_ext, cfrc_int if desired.

Due to this Humanoid and Ant reward structure differs between MuJoCo 2.0 and <2.0 . Is it intentional?

[pshvechikov]

Also, found this very frustrating.
Algorithm benchmarks differ between versions of MuJoCo.
This can misguide researchers and invalidate comparison to previously published papers.
So, this seems like a severe bug.
Please, @christopherhesse @pzhokhov, led some light on the issue, as soon as it is possible for you.
Thanks in advance.

[christopherhesse]

Thanks for reporting this bug! It looks like this could have been introduced by #1401

Can you guys verify that using mujoco-py==1.50.1.68 fixes this issue?

[AlexanderGri]

Thanks for your response!

Using mujoco-py<2.0 will fix this issue since older versions enforce using MuJoCo<2.0 which computes contact forces by default.

I see two other options to not to lose MuJoCo updates:

• Use force-related sensors in .xml models as suggested in changelog

The function mj_rnePostConstraint is no longer called by default. Instead, it is called only when the model contains force-related sensors which need the results of this function. The user can still call it directly in order to compute the mjData fields cacc, cfrc_ext, cfrc_int if desired.

• Maybe it is possible to use mj_rnePostConstraint with mujoco-py API without modifying library. Here is an example of dirty solution requiring recompile

[christopherhesse]

I don't know if we are supporting two different versions of mujoco for any particular reason. I think I'd rather just stick to the old version of there is no compelling reason to upgrade. Any objections?

[AlexanderGri]

I don't know if we are supporting two different versions of mujoco for any particular reason. I think I'd rather just stick to the old version of there is no compelling reason to upgrade. Any objections?

I see several possible issues with this decision:

• mujoco<2.0 can't be used to simulate interactions with deformable objects
  I saw at least two RL papers of such kind last month (one uses Bullet, the other - mujoco 2.0)
• mujoco 2.0 contains several new features in its simulator */mujoco200/sample/simulate.cpp (but i'm not sure if it is just presented in code samples in 2.0 or it is new functionality)
• gym will restrict itself to the existing features being unable to incorporate new ones (in fact, the generalization of previous two issues)

[christopherhesse]

gym doesn't depend on any particular mujoco version, just the gym mujoco environments do, so this should not affect other uses of gym + mujoco 2.0.

[christopherhesse]

This means you can't use the gym-mujoco environments with mujoco 2.0 (unless you override the requirements) but they don't really work with mujoco 2.0 anyway. Users of pip install gym will not be affected, only users of pip install gym[mujoco]

Thanks for the bug report @pratikac!