Add more pallet-psm tests

[rockbmb]

Description

This PR is a follow-up to #11068. It contains suggestions - for try_state checks and pallet unit tests - that were not made before it was merged.
It is also a companion to open-web3-stack/polkadot-ecosystem-tests#592 .

Any comments or suggestions appreciated (pinging @lrazovic since you authored #11068)..

Integration

N/A

Review Notes

• add more try_state checks to pallet-psm
• add a test that verifies expected behavior of fn do_try_state on valid state
• add unit tests that verify do_try_state fails appropriately on several different kinds of invalid pallet-psm states
• cover some additional edge cases regarding multi-asset ceiling boundaries
• add a basic generator-based fuzzer, and perform a short campaign to validate it.