Deprecation of HTTP Basic Auth and OAuth 1.0a

[grischard]

This is to track the discussion of the future deprecation of HTTP Basic Auth and OAuth 1.0a.

See pnorman's diary and zerebubuth/openstreetmap-cgimap#286

See the deprecation announcement for dates and details:

This change will have three key dates:

• March 1st, 2024: New OAuth 1.0a application registrations are disabled. Existing applications will not be impacted. HTTP Basic Auth will not be impacted.
• May 1st, 2024: Sysadmins will start brownouts to find applications that are still using OAuth 1.0a or HTTP Basic Auth
• June 1st, 2024: OAuth 1.0a and HTTP Basic Auth will be shut down.

[pnorman]

Over the last week 5% of API calls were HTTP Basic Auth, 23% OAuth 1.0a, and 73% OAuth 2.0. There's some daily jobs that are being run that call OAuth 1.0a starting at 01:15 UTC, 02:00 UTC, and one that calls HTTP Basic at 02:30 UTC, but even excluding those it's 18% OAuth 1.0a.

5 months ago, it was 70% OAuth 2.0, some traffic is moving to OAuth 2.0 but slowly.

[mmd-osm]

I think it would be helpful to have some stats highlighting the top OAuth 1.0a users. This way, we could raise follow up issue in the respective app repo. This would also help us to better track the overall progress.

The number of valid tokens per app id might be a good starting point:

SELECT app.id AS app_id,
       app.name AS app_name,
       count(tk.id) AS token_count
FROM client_applications AS app
LEFT JOIN oauth_tokens AS tk ON app.id = tk.client_application_id
AND tk.invalidated_at IS NULL
GROUP BY app.id,
         app.name
ORDER BY token_count DESC
LIMIT 100;

Possibly there are better ways to gather relevant insights on per-app OAuth 1.0a usage.

[tomhughes]

I doubt the token count is very useful because some apps save a token and use it persistently and some generate a new one on every use.

More useful might be the number of users with at least one token for an app though there is still the question of how recent it is.

[tomhughes]

Here's a complete list of UAs that used OAuth 1 yesterday, with a count of requests:

      1 FitoTrack
      2 CityZen
      2 Zend_Http_Client
      5 AtlasNavi
      5 viking
      6 Go-http-client
     14 Authlib
     17 maps.me
     20 Node
     22 PHPoAuthLib
     30 FireYak
     32 Locus
     43 OAuth
     50 OsmAnd
     57 MapRoulette
     73 Apache-HttpClient
    249 Python-urllib
    255 Python-httplib2
    301 GuzzleHttp
    338 PECL-OAuth
    370 osm-wikidata
    431 NewOSMContributorsInBelgium,
    450 Organic%20Maps
   1573 MAPS.ME
   2776 Vespucci
   3875 okhttp
   6254 Dalvik
  10015 StreetComplete_ee
  13827 python-requests
  17775 -
  45333 NewOSMContributorsInRussia,
  70304 Mozilla
  74752 Java
  87722 StreetComplete
 129793 JOSM

I've reduced it down to just the UA name to exclude version differences etc.

[tomhughes]

The number of generic frameworks in that list limits it's usefulness (:angry:) so here's some results from @mmd-osm's query but limited to tokens issued this year to try and exclude historic things, and everything over 1000 tokens as a start:

 app_id |                 app_name                 | token_count 
--------+------------------------------------------+-------------
   3998 | Geofabrik Downloads                      |      198430
    750 | uMap                                     |       88382
   1896 | Organic Maps                             |       43020
   3277 | HOT Export Tool                          |       22932
   2302 | New MapRoulette                          |       20081
  10793 | StreetComplete                           |       18758
   2985 | How did you contribute to OpenStreetMap? |       17070
   9790 | MR MKS                                   |       14896
   3817 | Osmcha                                   |       11493
   5742 | umap-de                                  |       10960
   8524 | Organic Maps                             |       10710
    946 | JOSM - Java OpenStreetMap Editor         |        9061
   5066 | OsmAnd                                   |        9050
   7526 | OSM-Boundaries                           |        7487
    821 | OsmHydrant                               |        5224
   1875 | uMap Framasoft                           |        5214
   8520 | OrganicMaps F-Droid                      |        4526
   9922 | tasking-manager-hyd.geo.apple.com        |        4478
   3175 | overpass turbo                           |        3583
   3551 | OSM Streak                               |        3489
   4188 | TeachOSM Tasking Manager                 |        3361
   5265 | OpenPlaceReviews                         |        3226
   2100 | KartaView                                |        3098
   7404 | MapComplete                              |        2888
  11156 | DEMR3                                    |        2773
  10210 | Vespucci 2022                            |        2621
  12240 | OSM Imagery                              |        2616
   1198 | umap.osm.ch                              |        1941
   1161 | Osmose                                   |        1586
   1917 | GNOME Maps                               |        1411
    952 | Vespucci                                 |        1376
   8490 | mr_mumbai                                |        1249
   3215 | Freemap Slovakia                         |        1200
   6020 | OSM Teams                                |        1070

I think there's plenty there we can target as a starting point...

[tsmock]

129793 JOSM

Do we have a better idea of when OAuth 1.0a/Basic Auth will be officially deprecated? Last I heard, it was still under consideration.

I've had a ticket open in JOSM Trac for hiding Basic Auth/OAuth 1.0a (see #22810) which I had scheduled to revisit in January. I don't (yet) want to remove OAuth 1.0/Basic Auth, since it still works, but I don't mind hiding it from new users if necessary.

[tomhughes]

I'm inclined to say they're deprecated right now - that's literally the point of this ticket that we're trying to get people to move to OAuth 2 instead.

The question is when we stop supporting them.

I wasn't actually bothered by the JOSM entries because I assumed it was just people that had configured it some time ago and that they could migrate when necessary.

[pnorman]

Yes, this ticket starts the deprecation process. We have no date for when we will shut down OAuth 1.0a or HTTP Basic Auth, but earlier stated that we did not plan to shut them off this year. I would expect sometime mid-next year but that depends entirely on how efforts to move people to OAuth 2.0 go.

[pnorman]

MapRoulette has moved, see maproulette/maproulette3#1876

I reached out to the TeachOSM and Apple Tasking Managers.