Move private headers to a private subdirectory

[gilles-peskine-arm]

In both TF-PSA-Crypto and mbedtls, move private headers from include/mbedtls to include/mbedtls/private. Adjust #include directives accordingly. The goal is that you have to type /private/ when #includeing a private header. See https://github.com/gilles-peskine-arm/TF-PSA-Crypto/blob/7cce26bc7993b7df39d9a43db75d02c4dde86511/docs/architecture/0e-plans.md#move-private-headers for more context.

This requires coordinated action in all three repositories, since Mbed TLS can include private TF-PSA-Crypto headers and mbedtls-framework can include private headers from both projects.

This is a single task, rather than done piece by piece, because each move of a header that is used in another project is an incompatible change. So we need to either prepare a transition period (which would be a lot more work), or deal with multiple small breaking changes (which is a lot of work because each breaking change is a hassle for the whole team, not just the task itself), or make a single breaking change (which is only a hassle once).

This task should be done with the help of a search-and-replace script. Include the script in the changelog message for the corresponding commit, which should consist purely of the changes made by the script.

This may require changes to build scripts. Keep them to a minimum, so that this task doesn't balloon with out-of-scope changes.

The following crypto headers are becoming private:

aes.h
aria.h
bignum.h
block_cipher.h
camellia.h
ccm.h
chacha20.h
chachapoly.h
cipher.h
cmac.h
config_adjust_legacy_from_psa.h
config_adjust_psa_superset_legacy.h
config_adjust_test_accelerators.h
config_psa.h
ctr_drbg.h
des.h
dhm.h
ecdh.h
ecdsa.h
ecjpake.h
ecp.h
entropy.h
error_common.h
gcm.h
hmac_drbg.h
md5.h
oid.h
pkcs12.h
pkcs5.h
poly1305.h
ripemd160.h
rsa.h
sha1.h
sha256.h
sha3.h
sha512.h

The following mbedtls headers are becoming private:

config_adjust_ssl.h
config_adjust_x509.h

Everest is out of scope here because it requires specific changes in build scripts. It's covered by Mbed-TLS/TF-PSA-Crypto#229.

[mpg]

This does not seem to have consensus (yet?), see Mbed-TLS/TF-PSA-Crypto#145 (comment) and Mbed-TLS/TF-PSA-Crypto#223 (comment) so I'm changing from "Implementation needed" to "Design needed" until we've reached a consensus.

[gilles-peskine-arm]

@mpg But it looks like we're going ahead with the Everest part of this task? Mbed-TLS/TF-PSA-Crypto#229 is well under way.

I still think we should do this. It's a simple way of conveying which parts of installed headers are public interfaces. Not all users figure out the APIs from the Doxygen rendered documentation.

Let's look at the alternatives. If we don't do this, how do we convey that all the content in e.g. cipher.h is private? We ensure that function calls won't build through Mbed-TLS/TF-PSA-Crypto#220 and friends, so technically we have conveyed it. But in a very unfriendly way: users will reasonably think that mbedtls_cipher_xxx() are public functions when they look at the header files (we can't expect readers to follow the structure of preprocessor conditionals, there are far too many of them), then they'll be puzzled when their code doesn't build.

At a very minimum, if we don't convey the non-public nature of these headers through the path, I think we should remove all Doxygen comments from them. Which is not that much less work.

[mpg]

At this point, I think just doing this will be less costly than debating its exact priority (I agree it's something we want ideally, I think the point being debated is just whether we need it) - especially considering part of it (Everest) has already been started and is in active development, as you correctly pointed out.