feature: NETPLAY mouse support

[H4M5TER]

If there is no more memory for multiplayer mouse control
I hope players can use their mouse in turn and give the game ability to change the source.

[fejnartal]

Would people be interested in an opt-in way to support multi-touch/multiplayer without adding more memory?

I thought that "maybe" we could sacrifice a little bit of "responsiveness" and a maybe non-trivial amount of simplicity by treating screen position as a divide-and-conquer quadrant.

Wasm4 already uses 4bytes for positioning a single mouse. We could use 1byte for each player/finger.
With 8bits we can't map a fine-grained 160x160 space. At least NOT IN A SINGLE FRAME. But we could, in 2 or 3 frames (depending on some implementation details).

For example, top-left pixel could be expressed as follows:
FRAME 1
Bit 1. Select left or right halve of the screen (80x160)
Bit 2. Select top or bottom halve of the already selected area (80x80)
Bit 3. Select left or right halve... (40x80)
Bit 4. Select top or bottom halve (40x40)
Bit 5. Select left or right halve... (20x40)
Bit 6. Select top or bottom... (20x20)
Bit 7 and 8. We can use as a counter. That way we can distinguish frames: FIRST, SECOND THIRD...

SUBSEQUENT FRAMES will give a more fine-grained position of the touch (depending on the game, 20x20 might be good enough precision, if not, they can improve the precision waiting for the next frame).

I would love to hear your opinion on this or any other variation which better fits the design of the console, its philosophy, community, etc.

I have just learned about this fantasy console and haven't developed anything for it yet.
However, I already love the ideas behind it.
It's touch support and it's simple input system, specially when combined with the apparent simplicity of its multiplayer, being based in WebRTC and mostly transparent to the delevoper.

That's why I felt so sad when reading that mouse/touch is disabled in multiplayer. I kept reading trying to understand the underlying causes of this.

I found that there isn't enough memory to handle it gracefully. On top of that, one may think:
Will it downscale well to local multiplayer?
I believe there are many local mobile multi-touch games with a great 1-4 player experiences. Not so many for desktop multi-mouse (I only remember Hammerfight, and read somewhere which it was a pain to support multimouse as most operating systems will just list a single device).

I'm a software developer myself without much work experience in low-level but willing to learn and give a hand with the implementation if this proves interesting to you.

Thanks to all of you for this great fantasy console.
All in all, it looks amazing already.

P.S.:
I have just noticed that, in fact, Wasm4 uses 5 bytes for mouse handling (I forgot 1byte from MOUSE_BUTTONS). Some kind of multiframe logic might still be needed, but it was worth mentioning this extra byte.

[aduros]

Hi guys, I would also like to support mouse in netplay 🙂 The reason we don't is actually not because of memory limitations, but because the input prediction part of the rollback netcode doesn't work so well with mouse input.

Here's a quick overview of how our network synchronization works:

• Local peer executes a game frame. For the remote peers, we don't actually have their inputs because they haven't arrived yet. So we predict/guess their inputs this frame (eg: what buttons the remote players are pressing)
• Local peer continues executing and displaying frames this way as normal.
• Eventually we get a message from a remote peer containing their actual input history (what buttons they pressed) for the last few frames. We compare their actual input with the inputs we recently predicted, and where there was a divergence, we rewind (rollback!) to that frame and re-simulate all subsequent frames back up to the present with the corrected inputs.

"Input prediction" sounds fancy but it's really easy for button inputs: we simply assume remote players will be pressing the same buttons on frame N+1 as they did on frame N.

Mouse input is analog and harder to predict. If we were to plug it in as an input, we would basically always need to be rolling back when someone touches the mouse. Maybe that's actually fine, but it would be a little sad that a player waggling the mouse would trigger rollback spam on the other end, even for games that don't actually use the mouse (I don't think the runtime has a way of knowing whether the cart actually reads from the mouse).

If we can figure that out, I think it would also make sense to consider adding multiple mice to the system. WASM-4 only has a single mouse (MOUSE_X/MOUSE_Y/MOUSE_BUTTONS). Having players fight over who is moving the single mouse sounds like it would be really annoying 😆

[fejnartal]

Your explanation was extremely useful to make me realise I was not taking into account many other issues about multiplayer.

Still, I think the idea of adding a SYSTEM FLAG to enable "multiplayer_mouse_input" which instead of storing mouse coordinates, treats the screen as a quadrant could work with netplay.

For the quadrant we might us the strategy I mentioned in my previous comment to allow somewhat arbitrary precision.

I believe this would allow creating simple touch based games. Be it using multitouch in a single device or across the network.

The reason why I believe this might workaround the netplay challenges is because it seems to me that predicting quadrants is less prone to rollbacks than coordinates (more so the bigger the quadrants).

[aduros]

Lowering precision of the mouse would be one way to improve prediction, but I think we need full precision for most games to actually work.

Multi touch is an interesting point. If we add registers for MOUSE2 (and 3 and 4) it could maybe be used for local multi touch on mobile... but then people will ask for multi touch netplay support 😁