building PX4 with colcon on ubuntu22 - ros2 humble: ROS2 interface proto can't find px4 module

[beniaminopozzan]

Describe the bug

• Starting from initial minimal PX4_ROS2 platform and px4_ros2_default build #20689 it should be possible to built px4 msgs directly from the main repo using colcon without need of the package px4_msgs. The package is indeed built, all PX4 msgs seems to be available, they can be listed by px4 interface package px4 and the message definition can be inspected by px4 interface show .... However, px4 interface proto fails and returns ModuleNotFoundError as in https://stackoverflow.com/questions/74859379/ros2-throws-modulenotfounderror-when-calling-interface-proto-for-custom-msg
  I got this error only on an Ubuntu 22.04 machine running ros humble. On another machine with Ubuntu 20.04 with ros foxy everything works smoothly.
• Secondary problem: the package name is px4 while the usual px4 messages are under the package px4_msgs. Assuming the the compilation works, adapting https://github.com/PX4/PX4-Autopilot/blob/main/src/modules/microdds_client/dds_topics.yaml is required.

To Reproduce

Steps to reproduce the behavior, assuming: ros2 humble installed from binaries, PX4 toolchain configured.

1. Create a new colcon workspace

   mkdir -p ~/px4_colcon_ws/src
   cd ~/px4_colcon_ws/src

2. Clone PX4-Autopilot

   git clone https://github.com/PX4/PX4-Autopilot.git --recursive

3. Modify at least one entry of https://github.com/PX4/PX4-Autopilot/blob/main/src/modules/microdds_client/dds_topics.yaml such that the required message type belongs to px4 package instead of px4_msgs, for example VehicleOdometry

   diff --git a/src/modules/microdds_client/dds_topics.yaml b/src/modules/microdds_client/dds_topics.yaml
   index 3a871848bb..90506a5821 100644
   --- a/src/modules/microdds_client/dds_topics.yaml
   +++ b/src/modules/microdds_client/dds_topics.yaml
   @@ -36,7 +36,7 @@ publications:
        type: px4_msgs::msg::VehicleLocalPosition
   
      - topic: /fmu/out/vehicle_odometry
   -    type: px4_msgs::msg::VehicleOdometry
   +    type: px4::msg::VehicleOdometry
   
      - topic: /fmu/out/vehicle_status
        type: px4_msgs::msg::VehicleStatus

   the diff file adapt-dds_topic.patch.txt can be applied with git apply << adapt-dds_topic.patch.txt
4. source the ros2 environment

   source /opt/ros/humble/setup.bash

5. compile the workspace

   colcon build --symlink-install

6. source the workspace

   source install/local_setup.bash

7. It is now possible to list all message associated to the newly compiled px4 package

   ros2 interface package px4

   which will return something like

   px4/msg/InputRc
   px4/msg/HeaterStatus
   px4/msg/QshellRetval
   px4/msg/VehicleAngularVelocity
   px4/msg/Airspeed
   px4/msg/SensorGps
   .
   .
   .

8. And it is possible to inspect the message definitions

   ros2 interface show px4/msg/Ping

   which will return

   uint64 timestamp			# time since system start (microseconds)
   uint64 ping_time			# Timestamp of the ping packet
   uint32 ping_sequence		# Sequence number of the ping packet
   uint32 dropped_packets		# Number of dropped ping packets
   float32 rtt_ms				# Round trip time (in ms)
   uint8 system_id				# System ID of the remote system
   uint8 component_id			# Component ID of the remote system

9. However, when trying to create a prototype of any message

   ros2 interface proto px4/msg/Ping

   you will get the error

   Traceback (most recent call last):
     File "/opt/ros/humble/bin/ros2", line 33, in <module>
       sys.exit(load_entry_point('ros2cli==0.18.5', 'console_scripts', 'ros2')())
     File "/opt/ros/humble/lib/python3.10/site-packages/ros2cli/cli.py", line 89, in main
       rc = extension.main(parser=parser, args=args)
     File "/opt/ros/humble/lib/python3.10/site-packages/ros2interface/command/interface.py", line 35, in main
       return extension.main(args=args)
     File "/opt/ros/humble/lib/python3.10/site-packages/ros2interface/verb/proto.py", line 33, in main
       yaml = interface_to_yaml(args.type)
     File "/opt/ros/humble/lib/python3.10/site-packages/ros2interface/api/__init__.py", line 46, in interface_to_yaml
       interface = utilities.get_interface(identifier)
     File "/opt/ros/humble/lib/python3.10/site-packages/rosidl_runtime_py/utilities.py", line 23, in get_interface
       return import_message_from_namespaced_type(get_namespaced_type(identifier))
     File "/opt/ros/humble/lib/python3.10/site-packages/rosidl_runtime_py/import_message.py", line 30, in import_message_from_namespaced_type
       module = importlib.import_module(
     File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
     File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
     File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
     File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
     File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
     File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
     File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
   ModuleNotFoundError: No module named 'px4'

10. At the same time, if a simulation is started make px4_sitl gazebo, the microDDS agent enabled MicroXRCEAgent udp4 -p 8888, then all topics are available but they cannot be echoed.

    ros2 topic echo /fmu/out/vehicle_odometry

    will return the error

    The message type 'px4/msg/VehicleOdometry' is invalid

Expected behavior

The expected behavior can be easily seen using px4_msgs packages are message container and repeating all the above test with it. In particular,

ros2 interface proto px4_msgs/msg/TaskStackInfo

gives

"timestamp: 0
ping_time: 0
ping_sequence: 0
dropped_packets: 0
rtt_ms: 0.0
system_id: 0
component_id: 0
"

while echoing a topic gives the expected echo.

[DronecodeBot]

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/problem-compiling-px4-v13-3-and-14-beta1-and-beta2-with-ros2-humble-and-ubuntu-22/32315/1

[beniaminopozzan]

I was able to track down the issue to rosidl_generate_interfaces.
If

PX4-Autopilot/platforms/ros2/CMakeLists.txt

Lines 63 to 65 in fc35792

	rosidl_generate_interfaces(${PROJECT_NAME}
	${ros_msg_files}
	)

is called from src/PX4-Autopilot/platforms/ros2/CMakeLists.txt, which in return is called by

PX4-Autopilot/CMakeLists.txt

Lines 447 to 448 in fc35792

	# firmware added last to generate the builtin for included modules
	add_subdirectory(platforms/${PX4_PLATFORM})

then the issue occurs.
As soon as the content of src/PX4-Autopilot/platforms/ros2/CMakeLists.txt is copied in the main CMakeLists.txt file in the root of PX4, then everything works.

I confirmed once again that the issue occurs on Ubuntu 22, ROS Humble while on Ubuntu 20, ROS Foxy it works.

At this point I don't know if this is a ROS issue, CMAKE issue, colcon issue.

My guess is that project definition, and message generation should be performed in the same CMakeLists file.
This kind of confirms the guess:
https://answers.ros.org/question/361289/ros2-components-registration-from-subdirectory-cmakelists-file/

This is probably because in CMake each call to add_subdirectory() creates a new scope, meaning CMake variables set in the subdirectory are not visible to the parent directory.

[DaesungAn]

I resolved same issue.
After applying ament/ament_cmake#431, re-build all and it worked.

[beniaminopozzan]

Hi @DaesungAn !
You are right, I installed ros rolling (base version) and made a clean build. It build successfully and ros2 interface proto px4_msgs/msg/TaskStackInfo gives the proper output.

At that point I tried to replace https://github.com/PX4/px4_msgs with this in tree package. I did the following test:

• ROS distro: rolling
• Workspace A: just https://github.com/PX4/px4_msgs
• Workspace B: just PX4-Autopilot with the package name changed to px4_msgs
• Workspace C: just https://github.com/PX4/px4_ros_com

I independently compiled workspace A and B, both of them extending only the bare ros2 rolling. The compilations were successful and I could get ros2 interface proto of the px4 messages from both workspaces. Then the real test:

• Compiling workspace C with Workspace A sourced (namely, using the standard https://github.com/PX4/px4_msgs) works just fine (as expected).
• Compiling workspace C with Workspace B sourced (namely, using the in tree PX4 package) gives me

  fatal error: px4_msgs/msg/offboard_control_mode.hpp: No such file or directory

  on all px4_msgs that I try to include and use.

@DaesungAn Did you try something similar?

At this point I went and see the content of the install directories of Workspace A and B, they should be the same, but they are not, in particular workspace B is missing files that I presume are related to the command ament_export_dependencies(rosidl_default_runtime). However, even adding it just before ament_package() in https://github.com/PX4/PX4-Autopilot/blob/main/platforms/ros2/cmake/finalize.cmake does not change the result.
I still think that part of the output of rosidl_generate_interfaces is hidden from ament_cmake due to the use of add_subdirectory

On the left workspace B, on the right workspace A

[GenCyber-Teacher]

"As soon as the content of src/PX4-Autopilot/platforms/ros2/CMakeLists.txt is copied in the main CMakeLists.txt file in the root of PX4, then everything works."
@beniaminopozzan: would you please show me which lines I need to copy to solve the problem?
Thanks

[aa-calvo]

I was getting the same exception from the terminal in which I was trying to call ros2 topic echo ... because I wasn't sourcing the install/setup.sh of the workspace in which I was building px4_msgs. I use a single workspace, so this probably doesn't apply to everyone, but hopefully it helps out a few.

[Cristian-wp]

Hi @beniaminopozzan , I am trying to repeat the change you made to avoid the
add_subdirectory(platforms/${PX4_PLATFORM})
problem in PX4-Autopilot/CMakeLists.txt.
But I can not understand what did you do...If I simply copy&paste the from PX4-Autopilot/platform/ros2/CMakeLists.txt I get an error about the px4 target already exists.
I have trying to resolve it, but I can not understand what I have to do...can you please help?