Merge pull request #1929 from Exirel/flood-penalty-long-message

irc, config: config options for flood penalty on longer messages

Author: dgw

docs/source/configuration.rst

@@ -253,15 +253,26 @@ Flood Prevention
 ----------------
 
 In order to avoid flooding the server, Sopel has a built-in flood prevention
-mechanism. It can be controlled with several directives:
+mechanism. The flood burst limit can be controlled with these directives:
 
 * :attr:`~CoreSection.flood_burst_lines`: the number of messages
   that can be sent before triggering the throttle mechanism.
-* :attr:`~CoreSection.flood_empty_wait`: time to wait once burst limit has been
-  reached before sending a new message.
 * :attr:`~CoreSection.flood_refill_rate`: how much time (in seconds) must be
   spent before recovering flood limit.
 
+The wait time when the flood limit is reached can be controlled with these:
+
+* :attr:`~CoreSection.flood_empty_wait`: time to wait once burst limit has been
+  reached before sending a new message.
+* :attr:`~CoreSection.flood_max_wait`: absolute maximum time to wait before
+  sending a new message once the burst limit has been reached.
+
+And the extra wait penalty for longer messages can be controlled with these:
+
+* :attr:`~CoreSection.flood_text_length`: maximum size of messages before they
+  start getting an extra wait penalty.
+* :attr:`~CoreSection.flood_penalty_ratio`: ratio used to compute said penalty.
+
 For example this configuration::
 
     [core]
@@ -273,15 +284,50 @@ will allow 10 messages at once before triggering the throttle mechanism, then
 it'll wait 0.5s before sending a new message, and refill the burst limit every
 2 seconds.
 
+The wait time **cannot be longer** than :attr:`~CoreSection.flood_max_wait` (2s
+by default). This maximum wait time includes any potential extra penalty for
+longer messages.
+
+Messages that are longer than :attr:`~CoreSection.flood_text_length` get an
+extra wait penalty. The penalty is computed using a penalty ratio (controlled
+by :attr:`~CoreSection.flood_penalty_ratio`, which is 1.4 by default)::
+
+    length_overflow = max(0, (len(text) - flood_text_length))
+    extra_penalty = length_overflow / (flood_text_length * flood_penalty_ratio)
+
+For example with a message of 80 characters, the added extra penalty will be::
+
+    length_overflow = max(0, 80 - 50)  # == 30
+    extra_penalty = 30 / (50 * 1.4)  # == 0.428s (approximately)
+
+With the default configuration, it means a minimum wait time of 0.928s before
+sending any new message (0.5s + 0.428s).
+
+You can **deactivate** this extra wait penalty by setting
+:attr:`~CoreSection.flood_penalty_ratio` to 0.
+
 The default configuration works fine with most tested networks, but individual
 bots' owners are invited to tweak as necessary to respect their network's flood
 policy.
 
 .. versionadded:: 7.0
 
-   Flood prevention has been modified in Sopel 7.0 and these configuration
-   options have been added: ``flood_burst_lines``, ``flood_empty_wait``, and
-   ``flood_refill_rate``.
+    Additional configuration options: ``flood_burst_lines``, ``flood_empty_wait``,
+    and ``flood_refill_rate``.
+
+.. versionadded:: 7.1
+
+    Even more additional configuration options: ``flood_max_wait``,
+    ``flood_text_length``, and ``flood_penalty_ratio``.
+
+    It is now possible to deactivate the extra penalty for longer messages by
+    setting ``flood_penalty_ratio`` to 0.
+
+.. note::
+
+    ``@dgw`` said once about Sopel's flood protection logic:
+
+        *"It's some arcane magic from AT LEAST a decade ago."*
 
 Perform commands on connect
 ---------------------------

sopel/config/core_section.py

@@ -526,6 +526,57 @@ class CoreSection(StaticSection):
     .. versionadded:: 7.0
     """
 
+    flood_max_wait = ValidatedAttribute('flood_max_wait', float, default=2)
+    """How much time to wait at most when flood protection kicks in.
+
+    :default: ``2``
+
+    This is equivalent to the default value:
+
+    .. code-block:: ini
+
+        flood_max_wait = 2
+
+    .. seealso::
+
+        The :ref:`Flood Prevention` chapter to learn what each flood-related
+        setting does.
+
+    .. versionadded:: 7.1
+    """
+
+    flood_penalty_ratio = ValidatedAttribute('flood_penalty_ratio',
+                                             float,
+                                             default=1.4)
+    """Ratio of the message length used to compute the added wait penalty.
+
+    :default: ``1.4``
+
+    Messages longer than :attr:`flood_text_length` will get an added
+    wait penalty (in seconds) that will be computed like this::
+
+        overflow = max(0, (len(text) - flood_text_length))
+        rate = flood_text_length * flood_penalty_ratio
+        penalty = overflow / rate
+
+    .. note::
+
+        If the penalty ratio is 0, this penalty will be disabled.
+
+    This is equivalent to the default value:
+
+    .. code-block:: ini
+
+        flood_penalty_ratio = 1.4
+
+    .. seealso::
+
+        The :ref:`Flood Prevention` chapter to learn what each flood-related
+        setting does.
+
+    .. versionadded:: 7.1
+    """
+
     flood_refill_rate = ValidatedAttribute('flood_refill_rate', int, default=1)
     """How quickly burst mode recovers, in messages per second.
 
@@ -545,6 +596,28 @@ class CoreSection(StaticSection):
     .. versionadded:: 7.0
     """
 
+    flood_text_length = ValidatedAttribute('flood_text_length', int, default=50)
+    """Length of text at which an extra wait penalty is added.
+
+    :default: ``50``
+
+    Messages longer than this (in bytes) get an added wait penalty if the
+    flood protection limit is reached.
+
+    This is equivalent to the default value:
+
+    .. code-block:: ini
+
+        flood_text_length = 50
+
+    .. seealso::
+
+        The :ref:`Flood Prevention` chapter to learn what each flood-related
+        setting does.
+
+    .. versionadded:: 7.1
+    """
+
     help_prefix = ValidatedAttribute('help_prefix',
                                      default=COMMAND_DEFAULT_HELP_PREFIX)
     """The prefix to use in help output.

sopel/irc/__init__.py

@@ -566,11 +566,19 @@ def say(self, text, recipient, max_messages=1):
             # Manage multi-line only when needed
             text, excess = tools.get_sendable_message(text)
 
+        flood_max_wait = self.settings.core.flood_max_wait
+        flood_burst_lines = self.settings.core.flood_burst_lines
+        flood_refill_rate = self.settings.core.flood_refill_rate
+        flood_empty_wait = self.settings.core.flood_empty_wait
+
+        flood_text_length = self.settings.core.flood_text_length
+        flood_penalty_ratio = self.settings.core.flood_penalty_ratio
+
         with self.sending:
             recipient_id = tools.Identifier(recipient)
             recipient_stack = self.stack.setdefault(recipient_id, {
                 'messages': [],
-                'flood_left': self.config.core.flood_burst_lines,
+                'flood_left': flood_burst_lines,
             })
 
             if recipient_stack['messages']:
@@ -584,15 +592,36 @@ def say(self, text, recipient, max_messages=1):
             # based on how long it's been since our last message to recipient
             if not recipient_stack['flood_left']:
                 recipient_stack['flood_left'] = min(
-                    self.config.core.flood_burst_lines,
-                    int(elapsed) * self.config.core.flood_refill_rate)
+                    flood_burst_lines,
+                    int(elapsed) * flood_refill_rate)
 
             # If it's too soon to send another message, wait
             if not recipient_stack['flood_left']:
-                penalty = float(max(0, len(text) - 50)) / 70
-                wait = min(self.config.core.flood_empty_wait + penalty, 2)  # Maximum wait time is 2 sec
+                penalty = 0
+
+                if flood_penalty_ratio > 0:
+                    penalty_ratio = flood_text_length * flood_penalty_ratio
+                    text_length_overflow = float(
+                        max(0, len(text) - flood_text_length))
+                    penalty = text_length_overflow / penalty_ratio
+
+                # Maximum wait time is 2 sec by default
+                initial_wait_time = flood_empty_wait + penalty
+                wait = min(initial_wait_time, flood_max_wait)
                 if elapsed < wait:
-                    time.sleep(wait - elapsed)
+                    sleep_time = wait - elapsed
+                    LOGGER.debug(
+                        'Flood protection wait time: %.3fs; '
+                        'elapsed time: %.3fs; '
+                        'initial wait time (limited to %.3fs): %.3fs '
+                        '(including %.3fs of penalty).',
+                        sleep_time,
+                        elapsed,
+                        flood_max_wait,
+                        initial_wait_time,
+                        penalty,
+                    )
+                    time.sleep(sleep_time)
 
             # Loop detection
             messages = [m[1] for m in recipient_stack['messages'][-8:]]