MeloNX/MeloNX
23
25
Fork 22
Code Issues Pull Requests 2 Actions Packages Projects Releases 12 Wiki Activity

Motion Control Implementation #33

Merged
stossy11 merged 6 commits from MediaMoots/MeloNX:motionDev into XC-ios-ht 2025-05-31 02:54:15 +00:00
Conversation 1 Commits 6 Files Changed 11 +569 -15
MediaMoots commented 2025-05-22 08:57:25 +00:00
Contributor
Copy Link

Overview

This pull request introduces motion control support to MeloNX by implementing a Cemuhook-compatible DSU (UDP) server.
This allows Ryujinx, running within MeloNX, to receive motion data from the iOS device's internal IMU or from connected game controllers that expose motion capabilities through Apple's GameController framework.

Key Changes:

  1. DSU Server Core (DSUServer.swift - New File):

    • Introduces a singleton DSUServer class responsible for handling DSU communications.
    • Utilizes CocoaAsyncSocket for efficient UDP packet handling on the standard DSU port 26760.
    • Implements the DSU protocol:
      • Responds to client discovery requests (0x100001 - DSUSPortInfo).
      • Broadcasts motion data (0x100002 - DSUSPadDataRsp) to subscribed clients (Ryujinx).
    • Manages a list of active DSUMotionProviders.

  2. Motion Providers (DSUMotionProviders.swift - New File):

    • Defines the DSUMotionProvider protocol for sources of motion data.
    • DeviceMotionProvider:
      • Captures motion data (accelerometer and gyroscope) from the iOS device's built-in IMU using CoreMotion.
      • Handles device orientation changes to transform sensor data into the coordinate system expected by Cemuhook (landscape-left as neutral).
      • Pushes DSUMotionSample data to the DSUServer.
    • ControllerMotionProvider:
      • Captures motion data from GCController instances (e.g., Switch Pro Controller, DualSense) that support GCMotion.
      • Transforms sensor data to the expected coordinate system.
      • Pushes DSUMotionSample data to the DSUServer.

  3. MeloNX Application Integration:

    • Ryujinx.swift (MeloNX/MeloNXApp/Core):
      • The Configuration struct now includes inputDSUServers: [String].
      • When launching Ryujinx, appropriate command-line arguments (--input-dsu-server-[slot] or --input-dsu-server-handheld) are appended, pointing to the local DSU server (127.0.0.1:26760).
    • ContentView.swift (MeloNX/MeloNXApp/View/Main):
      • The registerMotionForMatchingControllers() function is called before starting Ryujinx.
      • This function iterates through connected NativeControllers and the VirtualController (if active) to register corresponding motion providers with the shared DSUServer.
    • NativeController.swift & VirtualController.swift:
      • Added tryRegisterMotion(slot: UInt8) methods. These methods instantiate and register either a ControllerMotionProvider (for native controllers) or a DeviceMotionProvider (for the virtual/handheld controller) with the DSUServer.shared.
    • Package.resolved:
      • Updated to include/reflect the CocoaAsyncSocket dependency.

  4. Ryujinx Headless SDL Modifications:

    • Options.cs (src/Ryujinx.Headless.SDL2):
      • Added new command-line options: input-dsu-server-1 through input-dsu-server-8, and input-dsu-server-handheld. These options allow specifying the DSU server address (host:port) for each player and the handheld mode.
    • Program.cs (src/Ryujinx.Headless.SDL2):
      • HandlePlayerConfiguration: Modified to accept a new string inputDSUServer parameter. If this string is provided and not empty, it parses the host and port, then configures a CemuHookMotionController for the player's input configuration.
      • LoadPlayerConfiguration (and its callers): Updated to pass the option.InputDSUServerX or option.InputDSUServerHandheld value to HandlePlayerConfiguration.

How it Works:

  1. When a game is launched, MeloNX iterates through connected controllers and the device itself (for handheld mode) to register appropriate motion providers (DeviceMotionProvider or ControllerMotionProvider) with the DSUServer.
  2. Ryujinx is launched with command-line arguments telling it to connect to 127.0.0.1:26760 for DSU motion input for the relevant player slots.
  3. Ryujinx's internal CemuHookMotionController connects to this local DSU server.
  4. As motion occurs, the providers send DSUMotionSample data to the DSUServer.
  5. The DSUServer then broadcasts this data in the DSU protocol format to Ryujinx, enabling in-game motion controls.

Limitations & Considerations:

  • Joy-Cons: Motion controls are not supported for Nintendo Switch Joy-Cons. iOS does not currently expose gyro/motion data for Joy-Cons via the GameController framework.
  • PS5 DualSense / DualShock 4: Gyro controls for these controllers are untested but should theoretically work if iOS correctly exposes their motion data through GCMotion.
  • Controller Naming Conflicts: If multiple physical controllers of the same type are connected, they might share the same name (e.g., "Pro Controller"). This could lead to ambiguity in mapping motion to the correct player slot if names are the primary differentiator.
    • Workaround: Users can rename their controllers in iOS Bluetooth settings. After renaming and restarting the iOS device, the new unique names should be picked up, allowing for more reliable mapping.
  • There is a chance Knockoff Switch Pro Controllers will have bad motion data, only authentic Pro Controllers seem to work correctly.
  • DSU Server Port: The DSU server uses the standard port 26760.
# **Overview** This pull request introduces motion control support to MeloNX by implementing a Cemuhook-compatible DSU (UDP) server. This allows Ryujinx, running within MeloNX, to receive motion data from the iOS device's internal IMU or from connected game controllers that expose motion capabilities through Apple's GameController framework. ## **Key Changes:** 1. **DSU Server Core (`DSUServer.swift` - New File):** * Introduces a singleton `DSUServer` class responsible for handling DSU communications. * Utilizes `CocoaAsyncSocket` for efficient UDP packet handling on the standard DSU port `26760`. * Implements the DSU protocol: * Responds to client discovery requests (`0x100001 - DSUSPortInfo`). * Broadcasts motion data (`0x100002 - DSUSPadDataRsp`) to subscribed clients (Ryujinx). * Manages a list of active `DSUMotionProvider`s. 2. **Motion Providers (`DSUMotionProviders.swift` - New File):** * Defines the `DSUMotionProvider` protocol for sources of motion data. * `DeviceMotionProvider`: * Captures motion data (accelerometer and gyroscope) from the iOS device's built-in IMU using `CoreMotion`. * Handles device orientation changes to transform sensor data into the coordinate system expected by Cemuhook (landscape-left as neutral). * Pushes `DSUMotionSample` data to the `DSUServer`. * `ControllerMotionProvider`: * Captures motion data from `GCController` instances (e.g., Switch Pro Controller, DualSense) that support `GCMotion`. * Transforms sensor data to the expected coordinate system. * Pushes `DSUMotionSample` data to the `DSUServer`. 3. **MeloNX Application Integration:** * **`Ryujinx.swift` (MeloNX/MeloNXApp/Core):** * The `Configuration` struct now includes `inputDSUServers: [String]`. * When launching Ryujinx, appropriate command-line arguments (`--input-dsu-server-[slot]` or `--input-dsu-server-handheld`) are appended, pointing to the local DSU server (`127.0.0.1:26760`). * **`ContentView.swift` (MeloNX/MeloNXApp/View/Main):** * The `registerMotionForMatchingControllers()` function is called before starting Ryujinx. * This function iterates through connected `NativeController`s and the `VirtualController` (if active) to register corresponding motion providers with the shared `DSUServer`. * **`NativeController.swift` & `VirtualController.swift`:** * Added `tryRegisterMotion(slot: UInt8)` methods. These methods instantiate and register either a `ControllerMotionProvider` (for native controllers) or a `DeviceMotionProvider` (for the virtual/handheld controller) with the `DSUServer.shared`. * **`Package.resolved`:** * Updated to include/reflect the `CocoaAsyncSocket` dependency. 4. **Ryujinx Headless SDL Modifications:** * **`Options.cs` (src/Ryujinx.Headless.SDL2):** * Added new command-line options: `input-dsu-server-1` through `input-dsu-server-8`, and `input-dsu-server-handheld`. These options allow specifying the DSU server address (host:port) for each player and the handheld mode. * **`Program.cs` (src/Ryujinx.Headless.SDL2):** * `HandlePlayerConfiguration`: Modified to accept a new `string inputDSUServer` parameter. If this string is provided and not empty, it parses the host and port, then configures a `CemuHookMotionController` for the player's input configuration. * `LoadPlayerConfiguration` (and its callers): Updated to pass the `option.InputDSUServerX` or `option.InputDSUServerHandheld` value to `HandlePlayerConfiguration`. ## **How it Works:** 1. When a game is launched, MeloNX iterates through connected controllers and the device itself (for handheld mode) to register appropriate motion providers (`DeviceMotionProvider` or `ControllerMotionProvider`) with the `DSUServer`. 2. Ryujinx is launched with command-line arguments telling it to connect to `127.0.0.1:26760` for DSU motion input for the relevant player slots. 3. Ryujinx's internal `CemuHookMotionController` connects to this local DSU server. 4. As motion occurs, the providers send `DSUMotionSample` data to the `DSUServer`. 5. The `DSUServer` then broadcasts this data in the DSU protocol format to Ryujinx, enabling in-game motion controls. ## **Limitations & Considerations:** * **Joy-Cons:** Motion controls are **not** supported for Nintendo Switch Joy-Cons. iOS does not currently expose gyro/motion data for Joy-Cons via the GameController framework. * **PS5 DualSense / DualShock 4:** Gyro controls for these controllers are **untested** but *should* theoretically work if iOS correctly exposes their motion data through `GCMotion`. * **Controller Naming Conflicts:** If multiple physical controllers of the same type are connected, they might share the same name (e.g., "Pro Controller"). This could lead to ambiguity in mapping motion to the correct player slot if names are the primary differentiator. * **Workaround:** Users can rename their controllers in iOS Bluetooth settings. After renaming and restarting the iOS device, the new unique names should be picked up, allowing for more reliable mapping. * There is a chance **Knockoff** Switch Pro Controllers will have bad motion data, only authentic Pro Controllers seem to work correctly. * **DSU Server Port:** The DSU server uses the standard port `26760`.
MediaMoots added 4 commits 2025-05-22 08:57:26 +00:00
MediaMoots commented 2025-05-22 09:00:01 +00:00
Author
Contributor
Copy Link

Motion Control showcases:
https://discord.com/channels/1300369899704680479/1320760791087185930/1375007254486716497
https://discord.com/channels/1300369899704680479/1320760791087185930/1374793679503491146
https://discord.com/channels/1300369899704680479/1320760791087185930/1374793679503491146
https://discord.com/channels/1300369899704680479/1320760811622371544/1373227385138184214
https://discord.com/channels/1300369899704680479/1320760791087185930/1374759886814183604

**Motion Control showcases:** https://discord.com/channels/1300369899704680479/1320760791087185930/1375007254486716497 https://discord.com/channels/1300369899704680479/1320760791087185930/1374793679503491146 https://discord.com/channels/1300369899704680479/1320760791087185930/1374793679503491146 https://discord.com/channels/1300369899704680479/1320760811622371544/1373227385138184214 https://discord.com/channels/1300369899704680479/1320760791087185930/1374759886814183604
MediaMoots added 1 commit 2025-05-31 02:30:54 +00:00
MediaMoots added 1 commit 2025-05-31 02:48:10 +00:00
stossy11 merged commit f2cf4edb75 into XC-ios-ht 2025-05-31 02:54:15 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
bug
Something isn't working
documentation
Improvements or additions to documentation
duplicate
This issue or pull request already exists
enhancement
New feature or request
good first issue
Good for newcomers
help wanted
Extra attention is needed
invalid
This doesn't seem right
question
Further information is requested
wontfix
This will not be worked on
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
0-Blu Fen Lonkelle Propenchieferr Stern bella jkcoxson june ny spidy123222 stossy11
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: MeloNX/MeloNX#33
No description provided.
Delete Branch "MediaMoots/MeloNX:motionDev"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?