instagram: oembed endpoint deprecated

[r4f4]

Description

When sopel tries to get info on an instagram link it fails with the message:

Unexpected error (Expecting value: line 1 column 1 (char 0)) from <user> at 2020-12-03 16:08:50.365477. Message was: [...]

That happens because, instead of a json, sopel gets the following message:

This endpoint is deprecated. Please read https://developers.facebook.com/docs/instagram/oembed  to learn how to migrate your endpoint

So not only has the endpoint changed, but it also requires an access token now.

Reproduction steps

1. Have the instagram plugin enabled
2. Connect to server, join a channel, ...?
3. Send the bot a message containing an instagram url
4. The bot displays an error message

Expected behavior

Sopel is able to display instagram metadata

Logs

[2020-12-03 16:08:50,365] sopel.bot            ERROR    - Unexpected error (Expecting value: line 1 column 1 (char 0)) from <user> at 2020-12-03 16:08:50.365477. Message was: https://www.instagram.com/p/CIV90SGp5uL/
Traceback (most recent call last):
  File "/home/sopel/sopel/bot.py", line 643, in call_rule
    rule.execute(sopel, trigger)
  File "/home/sopel/sopel/plugins/rules.py", line 921, in execute
    exit_code = self._handler(bot, trigger)
  File "/home/sopel/sopel/plugins/rules.py", line 1482, in execute_handler
    return handler(bot, trigger, match=trigger)
  File "/home/sopel/sopel/modules/instagram.py", line 51, in instaparse
    json = get_oembed_json(instagram_url)
  File "/home/sopel/sopel/modules/instagram.py", line 67, in get_oembed_json
    return response.json()
  File "/usr/lib/python3.6/site-packages/requests/models.py", line 850, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None

Environment

• Sopel .version: 82e3939a471e5f4d2909a8fb3cfbdad6d14388c9
• Sopel installed via: source
• Python version: 3.6.8
• Operating system: Centos 7
• Relevant plugins: instagram

Notes

Anything else we should know?

[dgw]

I don't think Facebook understands what "deprecated" means. Deprecated stuff should still work, just with a warning that it will stop working later. The state described here is "unavailable".

Self-assigning to move instagram.py into a standalone repo. Insta's clearly moving toward key-only access, which is incompatible with our core plugin policy (which is essentially "it has to Just Work").

[dgw]

Realized that sopel-irc/sopel#2000 obsoleted this issue in its original location, so it made sense to move it over to this standalone plugin repo.

[dgw]

Should anyone wish to work on this, using an InstaFix instance as the "backend" seems sanest.