SM8550: add AYANEO Pocket EVO eight-zone RGB and effect support

[mrdidit]

Summary

• What is the goal of this PR? (e.g. Bump up an emulator version, implement a new feature. )

Implement eight-zone analogue-stick RGB control for the AYANEO Pocket EVO
through its serial MCU UART protocol.

This adds:

• A kernel serdev driver as the sole owner of the controller UART
• A generic ayaneo,serial-mcu compatible
• An optional ayaneo,xbox360-mode property for board-specific controller setup
• Eight multicolour LED zones: four physical quadrants on each stick ring
• Atomic whole-device layouts with independent colour and brightness per zone
• Fast matching-ring and independent per-stick Static control
• Static, Breath, RGB Breath, Rainbow, and Reactive effects
• Runtime colour calibration through sysfs for applications and manual control
• Exact output disable and state restoration
• Reboot, fake-suspend, and future real-suspend lifecycle handling
• Existing EmulationStation Static control without changing its helper contract

Testing

Hardware tested on an AYANEO Pocket EVO:

• Complete SM8550 images built successfully, installed, and booted
• Driver registered as ayaneo-serial-mcu with ABI version 3
• ayaneo,xbox360-mode enabled the board-specific controller bootstrap
• Gamepad Tester confirmed working input with the correct Xbox button layout
• Devices without the property do not enter the Xbox initialization path
• The former gamepad_mode sysfs control and userspace boot writer were removed
• EmulationStation applied matching and independent per-stick Static colours
• Per-stick changes used the faster broadcast/per-ring paths
• All eight renamed LED zones mapped to the correct physical quadrants
• Physical zone order verified as:
  left-zone-270 left-zone-0 left-zone-90 left-zone-180 right-zone-270 right-zone-0 right-zone-90 right-zone-180
• Individual-zone brightness control worked through the renamed LED paths
• Repeated mixed eight-zone layouts completed reliably in approximately
  4.0–4.2 seconds
• Final mixed-layout timing profile 40/350/40/350/350 ms passed repeated tests
• Lower post-initializer and inter-zone delays that failed intermittently were
  rejected
• Default 15 20, raw 100 100, zero-channel, and invalid calibration writes
  were tested
• Live calibration changes reapplied Static, Breath, and Reactive state while
  preserving semantic readback
• Breath operated smoothly and synchronously on both rings
• RGB Breath and Rainbow operated correctly and ignored calibration changes
  without restarting
• Reactive used the requested idle and active colours and responded
  independently to both physical sticks
• Disabling output turned both rings off while retaining the selected layout,
  effect, and calibration
• Re-enabling output restored the retained state
• Fake suspend blanked and restored Static layouts and animated effects
• Reboot restored the hardware-tested default calibration
• Final kernel log contained only the driver registration message, with no
  related UART, restore, effect, or locking errors
• Kernel driver object compiled with W=1 without warnings
• Driver source passed checkpatch without errors or warnings

Additional Context

• The Pocket EVO MCU is attached to a dedicated Qualcomm GENI UART. The serdev
  driver is its only owner.
• The compatible and driver use the generic ayaneo,serial-mcu name.
• The Pocket EVO Device Tree opts into its required controller setup with
  ayaneo,xbox360-mode.
• Xbox initialization frames are sent during probe and restored after a real
  resume only when that property is present.
• There is no runtime controller-mode sysfs attribute or separate raw-UART boot
  writer.
• EmulationStation remains Static-only and its existing helper contract is
  unchanged.
• Other userspace applications and scripts can use the device-wide effect ABI
  without changing existing EmulationStation behaviour.

The 3.87 seconds of programmed guard delay applies only to genuinely mixed
per-quadrant eight-zone Static transactions. Hardware testing observed complete
writes around 4.0–4.2 seconds, with the first frame visible immediately.
Matching-ring and independent per-stick Static layouts use faster
broadcast/per-ring paths. Breath, RGB Breath, Rainbow, and Reactive use
separate effect paths and apply almost instantly.

The calibration attribute allows applications or scripts to persist and
restore a user-selected green/blue pair:

• 15 20: hardware-tested Pocket EVO default/reset
• 100 100: raw RGB output
• Custom values: green and blue percentages from 0 through 100

Requested RGB values remain unchanged in layout and effect readback.
Calibration applies to mixed-red colours used by Static, single-colour Breath,
and Reactive. Pure red, green, blue, and cyan remain raw. RGB Breath and Rainbow
retain their captured hardware behaviour and are not recalculated when
calibration changes.

At the absolute minimum of the single-colour Breath curve, calibrated orange
can briefly appear green. The remainder of the cycle is correct. This is a
possible future brightness-dependent calibration improvement and does not
affect Static, RGB Breath, Rainbow, or Reactive operation.

The generic fake-suspend change only supplies a scoped suspend/resume phase to
the device LED helper. Other LED calls retain their existing behaviour.

Manual command usage

The device-wide ABI lives on the parent serial device. The direct Pocket EVO
path is:

RGB=/sys/bus/serial/devices/serial1-0/rgb

Applications should discover the bound device rather than hard-code its bus
name:

RGB=
for candidate in /sys/bus/serial/drivers/ayaneo-serial-mcu/*/rgb; do
  if [ -r "$candidate/abi_version" ]; then
    RGB=$candidate
    break
  fi
done
test -n "$RGB" || { echo "Pocket EVO RGB driver not found"; exit 1; }

Inspect the interface and current state:

cat "$RGB/abi_version"
cat "$RGB/available_effects"
cat "$RGB/effect"
cat "$RGB/zone_index"
cat "$RGB/zone_layout"
cat "$RGB/calibration"
cat "$RGB/enabled"

Set device-wide calibration:

printf '%s\n' '25 30' >"$RGB/calibration"    # Custom calibration
printf '%s\n' '15 20' >"$RGB/calibration"    # Pocket EVO default
printf '%s\n' '100 100' >"$RGB/calibration"  # Raw RGB

Applications or scripts can persist a user's custom pair and restore it after
boot. Invalid values are rejected without replacing the active calibration.

The existing helper provides matching or independent per-stick Static control.
Its argument order is brightness, right RGB, then left RGB:

analog_sticks_ledcontrol 180 0 0 255 255 0 0

That example makes the right ring blue and the left ring red at brightness 180.
The four-value compatibility form applies one colour to both rings:

analog_sticks_ledcontrol 180 255 0 0

A complete eight-zone layout contains one R G B brightness group for each
zone in the order reported by zone_index. This example makes the left ring
red at brightness 128 and the right ring blue at brightness 255:

printf '%s\n' \
  '255 0 0 128 255 0 0 128 255 0 0 128 255 0 0 128 0 0 255 255 0 0 255 255 0 0 255 255 0 0 255 255' \
  >"$RGB/zone_layout"

Select device-wide effects:

printf '%s\n' static >"$RGB/effect"
printf '%s\n' 'breath 180 255 128 0' >"$RGB/effect"
printf '%s\n' 'rgb-breath 180' >"$RGB/effect"
printf '%s\n' rainbow >"$RGB/effect"
printf '%s\n' 'reactive 180 255 128 0 0 0 255' >"$RGB/effect"

Reactive arguments are brightness, idle RGB, then active RGB. The example uses
calibrated orange while idle and blue during stick movement.

Blank and restore output without replacing the retained layout, effect, or
calibration:

printf '%s\n' 0 >"$RGB/enabled"
printf '%s\n' 1 >"$RGB/enabled"

These sysfs writes require the normal privileged access used by ROCKNIX
services.

---

AI Usage

While ROCKNIX doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? YES

[loki666]

I like the idea to expose the leds via a driver, but my issue with that is that it will be impossible to send frames to configure the gamepad mode...
@tiopex ?

[mrdidit]

Glad you like the idea. You’re right that once serdev owns the UART, /dev/ttyHS0 can’t be written directly anymore. For the Pocket EVO, I kept the existing gamepad setup inside the driver: the boot quirk writes xbox360 to gamepad_mode, and the driver sends the same binary and ASCII frames with the original 100 ms gaps, serialized with the RGB traffic. It also restores the mode after a real resume.
While checking this, I found and fixed a transcription error in the ASCII frame (3505010000000000 → 5505010000000000).
If there are any other Pocket EVO MCU configuration frames that still need to be sent over this UART, we can expose those through the same transport too.

[mrdidit]

Rebuilt and tested on the Pocket EVO. The driver registered cleanly, the boot quirk reported xbox360, and Gamepad Tester confirmed working input with the correct Xbox button layout. No related UART or driver errors appeared in dmesg.

[loki666]

ok, then a couple of changes, give a more generic name, like "ayaneo,serial-mcu" (I think it could work on other ayaneo)
and add a property to only send that xbox360 frame if present

[loki666]

and why "joystick-backlight-left-270" ?

[mrdidit]

Thanks loki,

The requested changes:

• Renamed the compatible and driver to ayaneo,serial-mcu.
• Added optional ayaneo,xbox360-mode; Xbox frames are only sent when the property is present.
• Removed the old boot writer and gamepad_mode sysfs control.
• Renamed the LEDs to ...-left-zone-270, etc. The angle is the physical position: 0 top, 90 right, 180 bottom, 270 left.
• Updated discovery to use the parent serial device directly; zone_index documents all eight zones.

Rebuilt and tested on the Pocket EVO:

• Xbox layout and all eight zone mappings are correct.
• Per-stick, per-zone, effects, output gating, and fake suspend work.
• No related driver errors appeared in the final kernel log.

[tiopex]

CONFIG_AYANEO_SERIAL_MCU

[tiopex]

rename all ayaneo-pocket-evo-rgb to /ayaneo-serial-mcu

[tiopex]

AYANEO Serial MCU control

[loki666]

@mrdidit
could you merge your led works with this Nova-Deck/os-build@b4ddf2d

it's a better implementation for the xbox mode and button layout. it also make sure the MCU is alive

ignore the inputplumber changes, but check the dts changes for the other devices ACE/DMG (you can let out the SM8650 changes for now)

[mrdidit]

Yep, I can merge the controller side into the current LED work. I’ll also complete the generic naming requested earlier: CONFIG_AYANEO_SERIAL_MCU, ayaneo-serial-mcu.c, and “AYANEO Serial MCU control”. The Pocket EVO RGB implementation will remain an explicitly gated device capability, not something exposed on every AYANEO MCU.
My understanding of the resulting structure is:

AYANEO Serial MCU driver
├── Shared AYANEO controller-MCU transport
│ ├── exclusive UART ownership and serialization
│ ├── MCU liveness detection
│ ├── generated binary/ASCII command frames
│ └── reliable Xbox initialization with bounded retries
└── AYANEO device-specific capabilities
├── Pocket EVO
│ ├── Xbox-labelled layout; no additional layout command
│ └── eight-zone RGB, effects and calibration
├── Pocket ACE
│ └── 0x1f positional button-layout configuration
└── Pocket DMG
├── 0x1f positional button-layout configuration
└── existing Static RGB path must be preserved or migrated

The common Device Tree node applies only to AYANEO Pocket devices inheriting qcs8550-ayaneo-pocket-common.dtsi; it does not affect other SM8550 manufacturers or devices.
The remaining AYANEO Pocket boards inheriting that node would use the shared Xbox initialization without exposing the Pocket EVO RGB capability. I’ll leave the InputPlumber and SM8650 changes out as requested.

One thing I want to confirm before I start: moving the MCU node into the AYANEO common DTS means serdev owns the UART on all those devices. ACE has no RGB as far as I know, so there should be nothing to migrate there. DMG currently uses /dev/ttyHS0 for its Static RGB helper, and that path disappears once serdev owns the port.
The current Japanese InputPlumber mapping also appears to perform the same button swap that the new 0x1f command makes positional, so leaving both active could double-swap the buttons.
Since the InputPlumber changes should remain out, is the DMG/InputPlumber side expected to be handled separately and land alongside this, or should I keep DMG out of the common binding until it is migrated? I can also move its existing Static RGB command into the driver if that belongs in this PR.

I have both an ACE and a DMG available, so I can hardware-validate the controller mode and button layout on both, as well as the DMG RGB path, before pushing anything.

[loki666]

let's ignore the Pcoket DMG leds stuff (don't think nobody really own a Pocket DMG), but you are right to push back on the InputPlumber.
I guess we can drop the japanse caps, and use the same for every sm8550 ayaneo (currently use use by EVO/S/S2K)
S Mini will also need the japanese dts property.

[loki666]

I think Pocket S/ Pocket S 2K also have the led stuff... it's difficult to know if they have the same protocol...

So I'm not really sure how to proceed.

I'm in favor let's change and see whose complaining... It's not a critical feature, worst case: leds don't work...

[mrdidit]

I have a DMG, so I traced stock GameWindow on Android and checked the command construction in its APK. This fills in the unresolved DMG part of my previous comment without changing the generic driver structure.
The DMG RGB path is now confirmed:

• Static uses the 27-byte broadcast frame with 58 08.
• Both 86 and 87 receive the same brightness value.
• Each completed change is sent three times with roughly 40 ms gaps.
• Off uses the all-zero broadcast frame.
• Default and Static use the same constant-light transport.
• Breath is host-rendered by repeatedly changing the Static brightness, so it does not require a separate hardware-effect command in the driver.

The current DMG helper does not match the present stock Android implementation: it uses 58 02, fixes 86 at ff, and sends one copy, while stock uses 58 08, writes brightness to both 86 and 87, and sends three copies.

Since serdev ownership removes the raw /dev/ttyHS0 path, I’d rather migrate the DMG’s validated Static support than leave it broken. I can test the complete implementation on the hardware before pushing it.

Also, was Pocket S2 left out only because this PR is currently scoped to SM8550? I implemented its working ROCKNIX RGB support a few weeks ago, and it uses the same 58 08 broadcast frame family. I’m fine leaving the actual SM8650 DTS and userspace migration for a follow-up, but the generic driver and variant model should be able to support it without another redesign.

The resulting structure would remain:

• Generic AYANEO Serial MCU control driver with shared UART ownership, serialization, MCU liveness detection, generated command frames and bounded Xbox initialization retries.
• Pocket EVO: Xbox-labelled layout with no additional layout command, plus eight-zone RGB, effects and calibration.
• Pocket DMG: 0x1f positional button layout plus validated broadcast Static RGB using selector 08.
• Pocket S and Pocket S 2K: shared controller initialization without exposing RGB until their protocol is verified.
• Pocket S2: compatible with the broadcast RGB variant, but its SM8650 integration can remain a follow-up.
  If the MCU node stays in the common AYANEO DTS, RGB registration should therefore be enabled by a device-specific compatible or property rather than exposed automatically by the generic compatible. The controller-layout and InputPlumber changes can still proceed as discussed.
  The previously agreed generic naming remains unchanged: CONFIG_AYANEO_SERIAL_MCU, ayaneo-serial-mcu.c, and “AYANEO Serial MCU control”.

[loki666]

sounds good, I'll check Pocket S2 after this

[mrdidit]

@loki666 one process question. The current AGENTS.md says not to carry direct kernel patches and to pull upstream sources dynamically. This PR carries the AYANEO serial-MCU driver as an SM8550 kernel patch, following the implementation requested in review. Is that approach acceptable here, or should the driver live elsewhere and be pulled in?

[loki666]

AYANEO serial-MCU driver is not an upstream patch, and never will