LaTeX: reST label before a ``module`` (or ``automodule``) sphinx.ext.autodoc directive causes pdf build to report multiply defined reference

[jfbu]

Describe the bug

As said in the title, the PDF build reports multiply-defined references.

How to Reproduce

Use

import os
import sys
sys.path.insert(0, os.path.abspath('.'))

extensions = ['sphinx.ext.autodoc']

in conf.py

file index.rst:

Welcome to FOO's documentation!
===============================

.. toctree::

   testfile

file testfile.rst:

.. _somelabel:

.. module:: testmodule

Some title
==========

Some text.

.. autofunction:: foo

file testmodule.py

"""not much here
"""

def foo():
    "I do not do much and simply return ``None``"

    return None

(the autofunction part is not part of the problem).

Then run make latexpdf.

Console output ends with

Latexmk: ====List of undefined refs and citations:
  Label `testfile:somelabel' multiply defined

This is not a serious problem (meaning it does not cause build failure).

But it happens in real life, see numpy/numpy#22930 (comment) and the

LaTeX Warning: Label `reference/maskedarray.generic:maskedarray-generic' multip
ly defined.

is an instance of the present issue. I have not yet checked but expect the others reported there as well.

Environment Information

Platform:              darwin
Python version:        3.10.9 (v3.10.9:1dd9be6584, Dec  6 2022, 14:37:36) [Clang 13.0.0 (clang-1300.0.29.30)])
Python implementation: CPython
Sphinx version:        6.2.0+/6e6b9ad45
Docutils version:      0.19
Jinja2 version:        3.1.2
Pygments version:      2.14.0

Sphinx extensions

``["sphinx.ext.autodoc"]``

Edit: all NumPy instances from building their numpy-ref.pdf are of this nature: numpy/numpy#22930 (comment)

[picnixz]

I investigated and probably located the issue in the visit_target method of the LaTeX builder:

sphinx/sphinx/writers/latex.py

Lines 1495 to 1503 in b6e6805

	if node.get('refid'):
	prev_node = get_prev_node(node)
	if isinstance(prev_node, nodes.reference) and node['refid'] == prev_node['refid']:
	# a target for a hyperlink reference having alias
	pass
	else:
	add_target(node['refid'])
	for id in node['ids']:
	add_target(id)

With your example, the document is:

<document source="/home/picnix/any/sphinx/doc/testfile.rst">
    <target refid="somelabel">
    <target ids="module-testmodule somelabel" ismod="True" names="somelabel">
    <index entries="('pair',\ 'module;\ testmodule',\ 'module-testmodule',\ '',\ None)">
    <section docname="testfile" ids="some-title :doc" names="some\ title">
        <title>
            Some title
        <paragraph>
            Some text.
        <index entries="('single',\ 'foo()\ (in\ module\ testmodule)',\ 'testmodule.foo',\ '',\ None)">
        <desc classes="py function" desctype="function" domain="py" nocontentsentry="False" noindex="False" noindexentry="False" objtype="function">
            <desc_signature _toc_name="foo()" _toc_parts="('testmodule', 'foo')" class="" classes="sig sig-object sig sig-object py" fullname="foo" ids="testmodule.foo" module="testmodule">
                <desc_addname classes="sig-prename descclassname sig-prename descclassname" xml:space="preserve">
                    testmodule.
                <desc_name classes="sig-name descname sig-name descname" xml:space="preserve">
                    foo
                <desc_parameterlist xml:space="preserve">
            <desc_content>
                <paragraph>
                    I do not do much and simply return 
                    <literal>
                        None

The only nodes that are of interest are:

<target refid="somelabel">
<target ids="module-testmodule somelabel" ismod="True" names="somelabel">

Those nodes are visited by visit_target but here is the issue:

• The first node branches on node.get('refid') and ignores the for-loop. The add_target then calls hypertarget(id, ...) which is the function that will add some \phantomsection\label{...}.

• The second node does not branch in node.get('refid'). However, all names in ids are used to create new \phantomsection\label{...} blocks. And this is the issue: because the second node also has somelabel in ids, then we re-add \phantomsection\label{...}. In particular, we end up with something like:

  %%% visiting: <target refid="somelabel">
  \phantomsection\label{\detokenize{testfile:somelabel}}
  %%%
  %%% visiting: <target ids="module-testmodule somelabel" ismod="True" names="somelabel"> 
  \phantomsection\label{\detokenize{testfile:module-testmodule}}
  \phantomsection\label{\detokenize{testfile:somelabel}} % <- the issue is here
  %%%

I don't know whether this could affect other labels or not, but I think that we could add the following check:

if node.get('refid'):
    prev_node = get_prev_node(node)
    if isinstance(prev_node, nodes.reference) and node['refid'] == prev_node['refid']:
        # a target for a hyperlink reference having alias
        pass
    else:
        add_target(node['refid'])

if node.get('ismod', False):
    # Detect if the previous node is a "target" label. If so, remove
    # the refid thereof from node['ids'] to avoid duplicated ids.
    prev_node = get_prev_node(node)
    if isinstance(prev_node, nodes.target) and prev_node.get('refid') in node['ids']:
      ids = node['ids'][:] # copy to avoid side-effects
      ids.remove(prev_node['refid'])
    else:
      ids = iter(node['ids']) # read-only iterator
else:
    ids = iter(node['ids']) # read-only iterator

for id in ids:
    add_target(id)

By tracing back all the code, I managed to find this:

• When parsing _somelabel, a nodes.target node with ids='somelabel' and names='somelabel' is created.

• If .. module follows just after, the Sphinx PyModule directive creates a nodes.target node with ids='module-testmodule' and ismod=True.

• In docutils.transforms.references, we have:

   next_node['ids'].extend(target['ids'])
   next_node['names'].extend(target['names'])

In particular, if a target follows another target node, then the previous target names ids and names are merged together (and this is why we end up with the module target node having somelabel as a valid ids.

So, one way to fix this behaviour is to avoid using nodes.target in PyModule:

sphinx/sphinx/domains/python.py

Lines 1044 to 1049 in b6e6805

	if not noindex:
	# note module to the domain
	node_id = make_id(self.env, self.state.document, 'module', modname)
	target = nodes.target('', '', ids=[node_id], ismod=True)
	self.set_source_info(target)
	self.state.document.note_explicit_target(target)

That way, we will not have this "merge" from docutils. Otherwise, we could use the fact that ismod=True in order to handle this case in the LaTeX writer as I did (although I am not sure that it covers all corner cases).

[jfbu]

@picnixz Thanks for this very useful analysis. In the case of html(5) it seems Sphinx on visiting target node simply uses original docutils.writers._html_base.visit_target() which will ignore those having a 'refid' attribute, but the target is not lost from the mechanism you explained which transferred the label name to the node created by the module directive. The Sphinx texinfo writer has its own visit_target and your analysis applies to it as well, but sphinx.writers.texinfo.TexinfoTranslator.add_anchor() filters out already encountered anchors:

sphinx/sphinx/writers/texinfo.py

Lines 529 to 538 in b6e6805

	def add_anchor(self, id: str, node: Node) -> None:
	if id.startswith('index-'):
	return
	id = self.curfilestack[-1] + ':' + id
	eid = self.escape_id(id)
	sid = self.get_short_id(id)
	for id in (eid, sid):
	if id not in self.written_ids:
	self.body.append('@anchor{%s}' % id)
	self.written_ids.add(id)

and there is no duplicate from a module directive following a label in source.

Your (first) work-around using ismod = True looks to me as it solves this issue (at least as it showed at numpy where I encountered it and in the minimal reproducer I am playing with) using a prudent approach minimizing potential changes. If you convert it into a PR, I think we can merge it for 6.2.0. The other approach looks good to but I need for this to analyse more, and as I (will) explain in replying to your other comment I will be mostly unavailable until mid-May (and should already have been for some weeks, but...)