Error when using --symlink-install - ``error: option --install-layout not recognized``

[ijnek]

As reported in ament/ament_cmake#382 (comment) and ament/ament_lint#369, running

colcon build --merge-install for a workspace containing a python package results in the error:

usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --install-layout not recognized

This error happens when using colcon-core 0.8.2, although I have not confirmed that downgrading fixes the issue.

Can be reproduced with any python package, using a package from the tutorials for example,

mkdir -p test_ws/src
cd test_ws/src
git clone git@github.com:ros2/examples.git
cd ../
colcon build --symlink-install --packages-select examples_rclpy_minimal_publisher

[ijnek]

@sloretz #504 (comment) seems to be related to this issue, any clues?

[sloretz]

So far I'm unable to reproduce on Ubuntu Focal with a minimal Python package. I'll try Jammy with a ROS workspace next.

foo/
├── foo.py
└── setup.py

foo.py is empty.

setup.py distutils version:

from distutils.core import setup

setup(name='foo',
      version='0.1.0',
      py_modules=['foo'],
      )

Could not reproduce: distutils --symlink-install

$ colcon build --event-handlers console_direct+ --symlink-install
Starting >>> foo     
running develop                   
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running egg_info                  
creating foo.egg-info
writing foo.egg-info/PKG-INFO
writing dependency_links to foo.egg-info/dependency_links.txt
writing top-level names to foo.egg-info/top_level.txt
writing manifest file 'foo.egg-info/SOURCES.txt'
reading manifest file 'foo.egg-info/SOURCES.txt'
writing manifest file 'foo.egg-info/SOURCES.txt'
running build_ext                 
Creating /home/sloretz/ws/colcon-from-source/override_examples/test/install/foo/lib/python3.8/site-packages/foo.egg-link (link to .)

Installed /home/sloretz/ws/colcon-from-source/override_examples/test/build/foo
--- stderr: foo
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< foo [2.00s]

Summary: 1 package finished [2.67s]
  1 package had stderr output: foo

Could not reproduce: distutils --merge-install --symlink-install

$ colcon build --event-handlers console_direct+ --symlink-install --merge-install
Starting >>> foo     
running develop                   
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running egg_info                  
creating foo.egg-info
writing foo.egg-info/PKG-INFO
writing dependency_links to foo.egg-info/dependency_links.txt
writing top-level names to foo.egg-info/top_level.txt
writing manifest file 'foo.egg-info/SOURCES.txt'
reading manifest file 'foo.egg-info/SOURCES.txt'
writing manifest file 'foo.egg-info/SOURCES.txt'
running build_ext                 
Creating /home/sloretz/ws/colcon-from-source/override_examples/test/install/lib/python3.8/site-packages/foo.egg-link (link to .)

Installed /home/sloretz/ws/colcon-from-source/override_examples/test/build/foo
--- stderr: foo                   
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< foo [2.00s]

Summary: 1 package finished [2.67s]
  1 package had stderr output: foo

setup.py setuptools version:

from setuptools import setup

setup(name='foo',
      version='0.1.0',
      py_modules=['foo'],
      )

Could not reproduce: setuptools --symlink-install

$ colcon build --event-handlers console_direct+ --symlink-install                
Starting >>> foo     
running develop                   
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running egg_info                  
creating foo.egg-info
writing foo.egg-info/PKG-INFO     
writing dependency_links to foo.egg-info/dependency_links.txt
writing top-level names to foo.egg-info/top_level.txt
writing manifest file 'foo.egg-info/SOURCES.txt'
reading manifest file 'foo.egg-info/SOURCES.txt'
writing manifest file 'foo.egg-info/SOURCES.txt'
running build_ext
Creating /home/sloretz/ws/colcon-from-source/override_examples/test/install/foo/lib/python3.8/site-packages/foo.egg-link (link to .)

Installed /home/sloretz/ws/colcon-from-source/override_examples/test/build/foo
--- stderr: foo                   
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< foo [2.03s]

Summary: 1 package finished [2.70s]
  1 package had stderr output: foo

Could not reproduce: setuptools --symlink-install --merge-install

$ colcon build --event-handlers console_direct+ --symlink-install --merge-install
Starting >>> foo     
running develop                   
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running egg_info                  
creating foo.egg-info
writing foo.egg-info/PKG-INFO     
writing dependency_links to foo.egg-info/dependency_links.txt
writing top-level names to foo.egg-info/top_level.txt
writing manifest file 'foo.egg-info/SOURCES.txt'
reading manifest file 'foo.egg-info/SOURCES.txt'
writing manifest file 'foo.egg-info/SOURCES.txt'
running build_ext                 
Creating /home/sloretz/ws/colcon-from-source/override_examples/test/install/lib/python3.8/site-packages/foo.egg-link (link to .)

Installed /home/sloretz/ws/colcon-from-source/override_examples/test/build/foo
--- stderr: foo                   
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/sloretz/.local/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< foo [2.02s]

Summary: 1 package finished [2.72s]
  1 package had stderr output: foo

Colcon versions

$ vcs export --exact
repositories:
  colcon-argcomplete:
    type: git
    url: https://github.com/colcon/colcon-argcomplete.git
    version: 2662285c0884db9ecb3756ed06997a5c42b01f33
  colcon-bash:
    type: git
    url: https://github.com/colcon/colcon-bash.git
    version: c381f19535b4b1a9fe2ebb17c57b303d718f476f
  colcon-cd:
    type: git
    url: https://github.com/colcon/colcon-cd.git
    version: 555ea9f11082f95d24cd25db81e44bb4fee1e859
  colcon-cmake:
    type: git
    url: https://github.com/colcon/colcon-cmake.git
    version: ecf417d03b67c4ad4734f042cb73c42e61a70376
  colcon-core:
    type: git
    url: https://github.com/colcon/colcon-core.git
    version: b1e064aac42f6abd2fbc3d2fc06927fb12b5454c
  colcon-defaults:
    type: git
    url: https://github.com/colcon/colcon-defaults.git
    version: 831265b0d0ea9652e2b855d394b6d816e7d76b36
  colcon-devtools:
    type: git
    url: https://github.com/colcon/colcon-devtools.git
    version: 09e42d8b3abc2f8ffaa60a041a7227f7470603ad
  colcon-library-path:
    type: git
    url: https://github.com/colcon/colcon-library-path.git
    version: dabe0a4ae83f5d361a38be1e13cd6c30b9af1aea
  colcon-metadata:
    type: git
    url: https://github.com/colcon/colcon-metadata.git
    version: 480c4c52fb34869dab788641e7c0e33750e5b0d6
  colcon-mixin:
    type: git
    url: https://github.com/colcon/colcon-mixin.git
    version: a18e562fb07832d5d9869741f1a92d7eee9ec719
  colcon-notification:
    type: git
    url: https://github.com/colcon/colcon-notification.git
    version: 78ca714ecfe5b01e53f101c45901361944dccca4
  colcon-output:
    type: git
    url: https://github.com/colcon/colcon-output.git
    version: ab77ac7fc689f5ff0829456b53f17c35316fa763
  colcon-package-information:
    type: git
    url: https://github.com/colcon/colcon-package-information.git
    version: a9998b66a7f15933eb64145a0c4e6919a4487ed3
  colcon-package-selection:
    type: git
    url: https://github.com/colcon/colcon-package-selection.git
    version: bad9a0ac3ec6efc8330df2cd648f92b62a5ad344
  colcon-parallel-executor:
    type: git
    url: https://github.com/colcon/colcon-parallel-executor.git
    version: 09e805d47ac22e38c139251fb7cc83f91402f81e
  colcon-pkg-config:
    type: git
    url: https://github.com/colcon/colcon-pkg-config.git
    version: 14e20fadc185f99f074b885e1bd7f9525c39ee61
  colcon-powershell:
    type: git
    url: https://github.com/colcon/colcon-powershell.git
    version: b6e3656b47f601b54d5cc134432ff6a25076d412
  colcon-python-setup-py:
    type: git
    url: https://github.com/colcon/colcon-python-setup-py.git
    version: ed426b5f6bb741daf38b862d41172bec985dce4d
  colcon-recursive-crawl:
    type: git
    url: https://github.com/colcon/colcon-recursive-crawl.git
    version: 0cd20abe4b7d620a57a5995f034e8567b8414fdc
  colcon-ros:
    type: git
    url: https://github.com/colcon/colcon-ros.git
    version: d7a66047e5aac95ee31eee2ef837f76919e2be9e
  colcon-test-result:
    type: git
    url: https://github.com/colcon/colcon-test-result.git
    version: 1e724a34231e337b0103ef56bca36417a335ed2d
  colcon-zsh:
    type: git
    url: https://github.com/colcon/colcon-zsh.git
    version: fd835ad7fe43420decc2a940b215e02696c2b631

pip3 freeze

$ pip3 freeze       
argcomplete==2.0.0
attrs==21.4.0
catkin-pkg==0.4.24
colcon-argcomplete==0.3.3
colcon-bash==0.4.2
colcon-cd==0.1.1
colcon-cmake==0.2.26
colcon-core==0.8.2
colcon-defaults==0.2.6
colcon-devtools==0.2.3
colcon-library-path==0.2.1
colcon-metadata==0.2.5
colcon-mixin==0.2.2
colcon-notification==0.2.13
colcon-output==0.2.12
colcon-package-information==0.3.3
colcon-package-selection==0.2.10
colcon-parallel-executor==0.2.4
colcon-pkg-config==0.1.0
colcon-powershell==0.3.7
colcon-python-setup-py==0.2.7
colcon-recursive-crawl==0.2.1
colcon-ros==0.3.23
colcon-test-result==0.3.8
colcon-zsh==0.4.0
coverage==6.2
distlib==0.3.4
docutils==0.18.1
empy==3.3.4
iniconfig==1.1.1
notify2==0.3.1
packaging==21.3
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.7
pytest==6.2.5
pytest-cov==3.0.0
pytest-repeat==0.9.1
pytest-rerunfailures==10.2
pytest-runner==5.3.1
python-dateutil==2.8.2
PyYAML==6.0
six==1.16.0
toml==0.10.2
tomli==2.0.0

[sloretz]

Able to reproduce with ament_cppcheck and --symlink-install on Jammy using the same colcon sources from the comment above.

$ colcon build --event-handlers console_direct+ --symlink-install --packages-select ament_cppcheck
Starting >>> ament_cppcheck
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --install-layout not recognized
--- stderr: ament_cppcheck
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --install-layout not recognized
---
Failed   <<< ament_cppcheck [0.64s, exited with code 1]

Summary: 0 packages finished [0.91s]
  1 package failed: ament_cppcheck
  1 package had stderr output: ament_cppcheck

[sloretz]

The command being run:

['/usr/bin/python3', 'setup.py', 'develop', '--prefix', '/home/sloretz/ws/colcon-from-source/override_examples/ros/install/ament_cppcheck', '--editable', '--build-directory', '/home/sloretz/ws/colcon-from-source/override_examples/ros/build/ament_cppcheck/build', '--no-deps', 'install_data', '--install-dir', '/home/sloretz/ws/colcon-from-source/override_examples/ros/install/ament_cppcheck', '--install-layout', 'deb']

It's being run here:

colcon-core/colcon_core/task/python/build.py

Lines 114 to 115 in b1e064a

	completed = await run(
	self.context, cmd, cwd=args.build_base, env=env)

[sloretz]

Python3 setup.py develop does list an --install-layout option. I know this option is added by a Debian specific patch, but I forget where that is. I'll go look for it and see if that gives any clues.

$ /usr/bin/python3 setup.py develop --help
[...]
  --install-layout                           installation layout to choose
                                             (known values: deb)
[...]

[sloretz]

Backtrace from pdb

  /usr/lib/python3.10/bdb.py(597)run()
-> exec(cmd, globals, locals)
  <string>(1)<module>()
  /home/sloretz/ws/colcon-from-source/override_examples/ros/src/ament_lint/ament_cppcheck/setup.py(6)<module>()
-> setup(
  /usr/lib/python3/dist-packages/setuptools/__init__.py(153)setup()
-> return distutils.core.setup(**attrs)
  /usr/lib/python3.10/distutils/core.py(134)setup()
-> ok = dist.parse_command_line()
> /usr/lib/python3.10/distutils/dist.py(483)parse_command_line()
-> args = self._parse_command_opts(parser, args)
  /usr/lib/python3/dist-packages/setuptools/dist.py(1048)_parse_command_opts()
-> nargs = _Distribution._parse_command_opts(self, parser, args)
  /usr/lib/python3.10/distutils/dist.py(578)_parse_command_opts()
-> (args, opts) = parser.getopt(args[1:])
  /usr/lib/python3.10/distutils/fancy_getopt.py(235)getopt()
-> raise DistutilsArgError(msg)

Patch to python3-setuptools that adds the --install-layout option.

install-layout.diff

--- a/setuptools/command/easy_install.py
+++ b/setuptools/command/easy_install.py
@@ -138,6 +138,8 @@ class easy_install(Command):
         ('local-snapshots-ok', 'l',
          "allow building eggs from local checkouts"),
         ('version', None, "print version information and exit"),
+        ('install-layout=', None, "installation layout to choose (known values: deb)"),
+        ('force-installation-into-system-dir', '0', "force installation into /usr"),
         ('no-find-links', None,
          "Don't load find-links defined in packages being installed"),
         ('user', None, "install in user site-package '%s'" % site.USER_SITE)
@@ -145,7 +147,7 @@ class easy_install(Command):
     boolean_options = [
         'zip-ok', 'multi-version', 'exclude-scripts', 'upgrade', 'always-copy',
         'editable',
-        'no-deps', 'local-snapshots-ok', 'version',
+        'no-deps', 'local-snapshots-ok', 'version', 'force-installation-into-system-dir'
         'user'
     ]
 
@@ -188,6 +190,10 @@ class easy_install(Command):
         self.pth_file = self.always_copy_from = None
         self.site_dirs = None
         self.installed_projects = {}
+        # enable custom installation, known values: deb
+        self.install_layout = None
+        self.force_installation_into_system_dir = None
+
         # Always read easy_install options, even if we are subclassed, or have
         # an independent instance created.  This ensures that defaults will
         # always come from the standard configuration file(s)' "easy_install"
@@ -259,6 +265,11 @@ class easy_install(Command):
         self.expand_basedirs()
         self.expand_dirs()
 
+        if self.install_layout:
+            if not self.install_layout.lower() in ['deb']:
+                raise DistutilsOptionError("unknown value for --install-layout")
+            self.install_layout = self.install_layout.lower()
+
         self._expand(
             'install_dir', 'script_dir', 'build_directory',
             'site_dirs',
@@ -285,6 +296,15 @@ class easy_install(Command):
         if self.user and self.install_purelib:
             self.install_dir = self.install_purelib
             self.script_dir = self.install_scripts
+
+        if self.prefix == '/usr' and not self.force_installation_into_system_dir:
+            raise DistutilsOptionError("""installation into /usr
+
+Trying to install into the system managed parts of the file system. Please
+consider to install to another location, or use the option
+--force-installation-into-system-dir to overwrite this warning.
+""")
+
         # default --record from the install command
         self.set_undefined_options('install', ('record', 'record'))
         # Should this be moved to the if statement below? It's not used
@@ -1322,11 +1342,28 @@ class easy_install(Command):
                 self.debug_print("os.makedirs('%s', 0o700)" % path)
                 os.makedirs(path, 0o700)
 
+    if sys.version[:3] in ('2.3', '2.4', '2.5') or 'real_prefix' in sys.__dict__:
+        sitedir_name = 'site-packages'
+    else:
+        sitedir_name = 'dist-packages'
+
     INSTALL_SCHEMES = dict(
         posix=dict(
             install_dir='$base/lib/python$py_version_short/site-packages',
             script_dir='$base/bin',
         ),
+        unix_local = dict(
+            install_dir = '$base/local/lib/python$py_version_short/%s' % sitedir_name,
+            script_dir  = '$base/local/bin',
+        ),
+        posix_local = dict(
+            install_dir = '$base/local/lib/python$py_version_short/%s' % sitedir_name,
+            script_dir  = '$base/local/bin',
+        ),
+        deb_system = dict(
+            install_dir = '$base/lib/python3/%s' % sitedir_name,
+            script_dir  = '$base/bin',
+        ),
     )
 
     DEFAULT_SCHEME = dict(
@@ -1337,11 +1374,18 @@ class easy_install(Command):
     def _expand(self, *attrs):
         config_vars = self.get_finalized_command('install').config_vars
 
-        if self.prefix:
+        if self.prefix or self.install_layout:
+            if self.install_layout and self.install_layout in ['deb']:
+                    scheme_name = "deb_system"
+                    self.prefix = '/usr'
+            elif self.prefix or 'real_prefix' in sys.__dict__:
+                scheme_name = os.name
+            else:
+                scheme_name = "posix_local"
             # Set default install_dir/scripts from --prefix
             config_vars = config_vars.copy()
             config_vars['base'] = self.prefix
-            scheme = self.INSTALL_SCHEMES.get(os.name, self.DEFAULT_SCHEME)
+            scheme = self.INSTALL_SCHEMES.get(scheme_name,self.DEFAULT_SCHEME)
             for attr, val in scheme.items():
                 if getattr(self, attr, None) is None:
                     setattr(self, attr, val)
@@ -1385,9 +1429,15 @@ def get_site_dirs():
             sitedirs.extend([
                 os.path.join(
                     prefix,
+                    "local/lib",
+                    "python" + sys.version[:3],
+                    "dist-packages",
+                ),
+                os.path.join(
+                    prefix,
                     "lib",
                     "python{}.{}".format(*sys.version_info),
-                    "site-packages",
+                    "dist-packages",
                 ),
                 os.path.join(prefix, "lib", "site-python"),
             ])
--- a/setuptools/command/install_egg_info.py
+++ b/setuptools/command/install_egg_info.py
@@ -1,5 +1,5 @@
 from distutils import log, dir_util
-import os
+import os, sys
 
 from setuptools import Command
 from setuptools import namespaces
@@ -18,14 +18,31 @@ class install_egg_info(namespaces.Instal
 
     def initialize_options(self):
         self.install_dir = None
+        self.install_layout = None
+        self.prefix_option = None
 
     def finalize_options(self):
         self.set_undefined_options('install_lib',
                                    ('install_dir', 'install_dir'))
+        self.set_undefined_options('install',('install_layout','install_layout'))
+        if sys.hexversion > 0x2060000:
+            self.set_undefined_options('install',('prefix_option','prefix_option'))
         ei_cmd = self.get_finalized_command("egg_info")
         basename = pkg_resources.Distribution(
             None, None, ei_cmd.egg_name, ei_cmd.egg_version
         ).egg_name() + '.egg-info'
+
+        if self.install_layout:
+            if not self.install_layout.lower() in ['deb']:
+                raise DistutilsOptionError("unknown value for --install-layout")
+            self.install_layout = self.install_layout.lower()
+            basename = basename.replace('-py%s' % pkg_resources.PY_MAJOR, '')
+        elif self.prefix_option or 'real_prefix' in sys.__dict__:
+            # don't modify for virtualenv
+            pass
+        else:
+            basename = basename.replace('-py%s' % pkg_resources.PY_MAJOR, '')
+
         self.source = ei_cmd.egg_info
         self.target = os.path.join(self.install_dir, basename)
         self.outputs = []

[sloretz]

I strongly suspect colcon hasn't been using --install-layout deb before #504. Before Jammy, the logic to choose whether to use --install-layout deb or not would only trigger if the path returned by sysconfig included dist-packages. This itself is a Colcon bug. --install-layout deb is added in a Debian patch to setuptools, but there could be a newer version of setuptools in a home directory or a virtualenv regardless of what sysconfig returns.

I think we never saw it because before #504 the code deciding whether or not to use --install-layout always say a path with site-packages instead.

>>> sysconfig.get_path('purelib', vars={'base': '.'})
'lib/python3.8/site-packages

It wasn't until Jammy that Debian added a patch to sysconfig to make it return a path with dist-packages in it:

>>> sysconfig.get_path('purelib', vars={'base': '.'})
'local/lib/python3.10/dist-packages'

The cause of this issue is colcon is passing --install-layout deb to the install_data command, but it's actually an option on the develop command. That's easy to fix, bug if I fix it I run into a Debian bug. The Debian patch to python3-setuptools adds this error message:

+        if self.prefix == '/usr' and not self.force_installation_into_system_dir:
+            raise DistutilsOptionError("""installation into /usr
+
+Trying to install into the system managed parts of the file system. Please
+consider to install to another location, or use the option
+--force-installation-into-system-dir to overwrite this warning.
+""")

I can't avoid this error message because the Debian patch unconditionally sets self.prefix to /usr and ignores the one given by colcon. I think it should only set the prefix if it wasn't already set.

+        if self.prefix or self.install_layout:
+            if self.install_layout and self.install_layout in ['deb']:
+                    scheme_name = "deb_system"
+                    self.prefix = '/usr'

I looked at ros2-galactic-20210616-linux-focal-amd64.tar.bz2 from the Galactic packaging job because I know that was created with an older version of colcon. I see python packages are installed to lib/python3.8/site-packages, meaning Colcon didn't use --install-layout deb, and I strongly suspect it's never been used until #504.

I think a good solution is to make colcon avoid deb_system and --install-layout deb rather than embrace it, and in parallel submit a patch to Debian to fix the unconditional setting of prefix.