Change default auth for experimental backend to deny_all (#9611)

In a move that should surprise no one, a number of users do not read,
and leave the API wide open by default. Safe is better than powned

Author: ashb

UPDATING.md

@@ -1421,6 +1421,22 @@ Now the `dag_id` will not appear repeated in the payload, and the response forma
 }
 ```
 
+### Experimental API will deny all request by default.
+
+The previous default setting was to allow all API requests without authentication, but this poses security
+risks to users who miss this fact. This changes the default for new installs to deny all requests by default.
+
+**Note**: This will not change the behavior for existing installs, please update check your airflow.cfg
+
+If you wish to have the experimental API work, and aware of the risks of enabling this without authentication
+(or if you have your own authentication layer in front of Airflow) you can get
+the previous behaviour on a new install by setting this in your airflow.cfg:
+
+```
+[api]
+auth_backend = airflow.api.auth.backend.default
+```
+
 ## Airflow 1.10.10
 
 ### Setting Empty string to a Airflow Variable will return an empty string

airflow/config_templates/config.yml

@@ -574,11 +574,13 @@
   options:
     - name: auth_backend
       description: |
-        How to authenticate users of the API
+        How to authenticate users of the API. See
+        https://airflow.apache.org/docs/stable/security.html for possible values.
+        ("airflow.api.auth.backend.default" allows all requests for historic reasons)
       version_added: ~
       type: string
       example: ~
-      default: "airflow.api.auth.backend.default"
+      default: "airflow.api.auth.backend.deny_all"
 - name: lineage
   description: ~
   options:

airflow/config_templates/default_airflow.cfg

@@ -311,8 +311,10 @@ endpoint_url = http://localhost:8080
 fail_fast = False
 
 [api]
-# How to authenticate users of the API
-auth_backend = airflow.api.auth.backend.default
+# How to authenticate users of the API. See
+# https://airflow.apache.org/docs/stable/security.html for possible values.
+# ("airflow.api.auth.backend.default" allows all requests for historic reasons)
+auth_backend = airflow.api.auth.backend.deny_all
 
 [lineage]
 # what lineage backend to use