Xbox Wireless Headset disconnecting

Xbox Wireless Headset connected trough MS wireless dongle disconnects after a minute or two, with following output in dmesg

xone-gip-headset gip0.0: gip_auth_exchange_rsa: encrypt RSA failed: -17
rpm-ostree status
State: idle
Deployments:
ā— ostree-image-signed:docker://ghcr.io/ublue-os/bazzite-deck:testing
                   Digest: sha256:33d58ad0a578386dae5e66366ba0154446b87bdc0fe588bc8e64cd91c08f6df3
                  Version: testing-41.20250321 (2025-03-21T06:36:09Z)
          LayeredPackages: openrazer-daemon

  ostree-image-signed:docker://ghcr.io/ublue-os/bazzite-deck:stable
                   Digest: sha256:4004ffa193ffe84772204677ef7a8d90b66c1ac4027239445aa39d54c7c79e7e
                  Version: 41.20250317 (2025-03-17T05:10:33Z)
          LayeredPackages: openrazer-daemon