JMM’s notes on the

Hyper key

Because Control, Alt, and Super aren’t enough

Pretty early into using Emacs, I switched my Caps Lock key to a Control key. I used my Super key (a.k.a. the “Windows” key) to generally control my window manager. But I didn’t really use my left control key anymore. So now I’ve repurposed that as a Hyper key.

NixOS settings

In NixOS, I set up a Hyper key by setting the following (important setting highlighted): 

  services.xserver.xkbOptions = "grp:switch,ctrl:swapcaps_hyper,compose:rctrl";

What ctrl:swapcaps_hyper does is change your Caps Lock to a Control key and your Left Ctrl to a Hyper key.

To set this immediately under X11, you can do something like: 

setxkbmap -option "grp:alt_space_toggle,ctrl:swapcaps_hyper,compose:rctrl"

Under Wayland, though, your keyboard settings are controlled by your compositor. If you’re using GNOME, you can modify your preferences on the fly by editing the dconf setting for /org/gnome/desktop/input-sources/xkb-options. Normally you’d access these settings through gnome-tweaks and clicking some preferences under the Keyboard section. But you can also set this manually in a GUI with dconf-editor or through a command line with: 

dconf write /org/gnome/desktop/input-sources/xkb-options "['grp:switch', 'ctrl:swapcaps_hyper', 'compose:rctrl']"

Hyper key issue NixOS 23.11 Gnome 45

TL;DR: As of 2023-12-11 I solved this by patching xkeyboard-config to add an option where Hyper is bound to Mod3. Issues were caused by a change in xkeyboard-config where the semantics of the new ctrl:hyper_capscontrol option weren’t the same as the previous ctrl:swapcaps_hyper option. Previously it mapped Hyper to Mod3, but it was changed to Mod4 to be more consistent with other settings. This change seems good, but probably broke a tiny subset of people, and that included me.

My settings above worked for a while, but there seems to be some issue with it when I upgraded to NixOS 23.11, running GNOME 45. When I press Left Control right now, it seems to act like a Super key rather than a Hyper key. For example, pressing Left Control+A brings up GNOME Activites, just like Super+A would. I’m currently encountering this on 2023-12-10. So I’m going to try to pinpoint the problem.

ctrl:swapcaps_hyper definitely does something

Switching to ctrl:swapcaps by using dconf write /org/gnome/desktop/input-sources/xkb-options "['grp:switch', 'ctrl:swapcaps', 'compose:rctrl']", I check what happens under xev. When pressing Left_Control I see: 

KeyPress event, serial 40, synthetic NO, window 0x600001,
    root 0x248, subw 0x0, time 51029838, (162,-12), root:(589,213),
    state 0x2002, keycode 37 (keysym 0xffe5, Caps_Lock), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

Then when I switch to ctrl:swapcaps_hyper and press Left_Control under xev I see: 

KeyPress event, serial 38, synthetic NO, window 0x600001,
    root 0x248, subw 0x0, time 51295901, (160,-16), root:(540,347),
    state 0x2000, keycode 37 (keysym 0xffed, Hyper_L), same_screen YES,
    XLookupString gives 0 bytes:
    XmbLookupString gives 0 bytes:
    XFilterEvent returns: False

Which is good. The setting is being changed. And yet when I press Left_Control+A, I still get Activites popping up. And when I press Left_Control+Z (where Super+Z is unbound) in Emacs, I see: 

s-z is undefined

Which means Emacs sees this as a Super key. But I’m also running emacs-pgtk. I wonder if Emacs under X11 (or Xwayland at least) would register this differently.

Running emacs -nw under Alacritty (Wayland client) and pressing Left_Control+Z, Emacs doesn’t even show any modifier key is being pressed. The same thing happens with emacs -nw under xterm (X11 client).

I wonder if GTK or GDK is somehow filtering Hyper or Mod4. It might be that “Hyper” is recognized, but it isn’t recognized as “Mod4”. Looking at X keyboard extension - ArchWiki and trying more xev -event keyboard, I notice the following when I press different modifiers with Z:

Caps_Lock+Z
    state 0x2004, keycode 52 (keysym 0x7a, z), same_screen YES,
Left_Alt+Z
    state 0x2008, keycode 52 (keysym 0x7a, z), same_screen YES,
“Windows”/Super+L
    state 0x2040, keycode 52 (keysym 0x7a, z), same_screen YES,
Left_Control+Z
    state 0x2040, keycode 52 (keysym 0x7a, z), same_screen YES,

It looks like my Hyper is being mapped to the same modifier as Super.

Setting Hyper to be a different modifier

Yep, looking at the output of xmodmap -pm, I see: 

xmodmap:  up to 5 keys per modifier, (keycodes in parentheses):

shift       Shift_L (0x32),  Shift_R (0x3e)
lock
control     Control_L (0x42)
mod1        Alt_L (0x40),  Alt_L (0xcc),  Meta_L (0xcd)
mod2        Num_Lock (0x4d)
mod3        ISO_Level5_Shift (0xcb)
mod4        Hyper_L (0x25),  Super_L (0x85),  Super_R (0x86),  Super_L (0xce),  Hyper_L (0xcf)
mod5        ISO_Level3_Shift (0x5c)

So Hyper and Super are mapped to the same modifier. I try setting this with xmodmap: 

xmodmap -e "remove mod4 = Hyper_L" -e "add mod3 = Hyper_L"

And now I see 

xmodmap:  up to 3 keys per modifier, (keycodes in parentheses):

shift       Shift_L (0x32),  Shift_R (0x3e)
lock
control     Control_L (0x42)
mod1        Alt_L (0x40),  Alt_L (0xcc),  Meta_L (0xcd)
mod2        Num_Lock (0x4d)
mod3        Hyper_L (0x25),  ISO_Level5_Shift (0xcb),  Hyper_L (0xcf)
mod4        Super_L (0x85),  Super_R (0x86),  Super_L (0xce)
mod5        ISO_Level3_Shift (0x5c)

And yet that doesn’t seem to actually change anything with xev or other keys.

Trying out wev (for Wayland events), I see: 

[14:     wl_keyboard] key: serial: 50399; time: 55028622; key: 37; state: 1 (pressed)
                      sym: Hyper_L      (65517), utf8: ''
[14:     wl_keyboard] modifiers: serial: 1; group: 0
                      depressed: 00002040: Mod4 Super
                      latched: 00000000
                      locked: 00000000

One thing I noticed while looking at localectl list-x11-keymap-options is that it seems like the “ctrl:swapcaps_hyper” might have been renamed to “ctrl:hyper_capscontrol”.

The meaning of ctrl:swapcaps_hyper changed

Aha.

I found a libera.chat IRC log that pointed to xkeyboard-config issue #344, where they talk about Hyper inconsistently being Mod3 for ctrl:swapcaps_hyper where in other places it’s Mod4. Commit 7124dd51 changes hyper_capscontrol to be Mod4. Unfortunately this doesn’t work for me. But helpfully, the first comment of issue #344 points these few users like me to set a custom layout. So that I shall do.

Custom layout

NixOS has a services.xserver.xkb.extraLayouts setting that allows an override.

I try to make my own custom configuration: 

  services.xserver.xkb.extraLayouts = {
    jmmcustom = {
      description = "JMM’s custom xkb layout for the hyper key.";
      languages = [ "eng" ];
      symbolsFile = ./files/jmm_xkb_custom;
    };
  };

But the first build fails since there’s a conflict with my services.xserver.exportConfiguration = true; setting. So I comment that out.

Then my build fails with 

mesonConfigurePhase flags: --prefix=/nix/store/0sw5ma2syx2xmz5y92haay0dy3sabp8z-xkeyboard-config-2.40 --libdir=/nix/store/0sw5ma2syx2xmz5y92haay0dy3sabp8z-xkeyboard-config-2.40/lib --libexecdir=/nix/store/0sw5ma2syx2xmz5y92haay0dy3sabp8z-xkeyboard-config-2.40/libexec --bindir=/nix/store/0sw5ma2syx2xmz5y92haay0dy3sabp8z-xkeyboard-config-2.40/bin --sbindir=/nix/store/0sw5ma2syx2xmz5y92haay0dy3sabp8z-xkeyboard-config-2.40/sbin --includedir=/nix/store/0sw5ma2syx2xmz5y92haay0dy3sabp8z-xkeyboard-config-2.40/include --mandir=/nix/store/0sw5ma2syx2xmz5y92haay0dy3sabp8z-xkeyboard-config-2.40/share/man --infodir=/nix/store/0sw5ma2syx2xmz5y92haay0dy3sabp8z-xkeyboard-config-2.40/share/info --localedir=/nix/store/0sw5ma2syx2xmz5y92haay0dy3sabp8z-xkeyboard-config-2.40/share/locale -Dauto_features=enabled -Dwrap_mode=nodownload --buildtype=plain -Dxorg-rules-symlinks=true
/nix/store/qrmvf49dbs7yqshymnqdhaiig0mcljlz-meson-1.2.3/nix-support/setup-hook: line 3:    41 Bus error               (core dumped) meson setup build "${flagsArray[@]}"

Maybe this is related to NIX_HARDENING_ENABLE? Or it’s related to: 

      "mesonFlags": "-Dxorg-rules-symlinks=true",

It’s not that mesonFlags setting. I overrode it and it still failed to build.

It’s something with my “meson”, which is just getting a Bus error. 

$ nix-store --verify-path $(nix-store -qR /nix/store/qrmvf49dbs7yqshymnqdhaiig0mcljlz-meson-1.2.3/bin/meson)
path '/nix/store/qp5zys77biz7imbk6yy85q5pdv7qk84j-python3-3.11.6' was modified! expected hash 'sha256:1x4h04h2q757fqy786v78x1jbk6a7dcrbysa6zn00zzpzi1mz7yr', got 'sha256:0rb84gyvfpjhb2rgz2pzdnhj57ikqs8j9fk8k9gll0gd6yvwhwi8'

Trying to repair. 

$ nix-store --repair-path /nix/store/qp5zys77biz7imbk6yy85q5pdv7qk84j-python3-3.11.6
error: operation 'repairPath' is not supported by store 'daemon'
$ sudo nix-store --repair-path /nix/store/qp5zys77biz7imbk6yy85q5pdv7qk84j-python3-3.11.6
copying path '/nix/store/qp5zys77biz7imbk6yy85q5pdv7qk84j-python3-3.11.6' from 'https://cache.nixos.org'...
warning: removing corrupted link '/nix/store/.links/1gzxz7nqambfscpsyixp7kqkn5z0h6k8a9z9vxpz5fwimdjrdy9s'
warning: There may be more corrupted paths.
You should run `nix-store --verify --check-contents --repair` to fix them all

Building now works.

Patch overlay

Okay, rather than making my own layout (which didn’t work well), I just decided to add my own xkb option that’s a modified version of ctrl:hyper_capscontrol with Mod3 instead of Mod4. See xkeyboard-config-hyper.patch for a patch I made that defines an option ctrl:jmm_hyper_capscontrol.

I then have an overlay with a patched xkeyboardconfig: 

self: super:

{
  jmm_xkb_patched = self.pkgs.xorg.xkeyboardconfig.overrideAttrs (old: {
    patches = [ ./xkeyboard-config-hyper.patch ];
  });
}

Then I set the following two configuration settings in my configuration.nix: 

  services.xserver.xkb.dir = "${pkgs.jmm_xkb_patched}/etc/X11/xkb";
  environment.sessionVariables.XKB_CONFIG_ROOT = config.services.xserver.xkb.dir;

I had to restart display-manager.service, but after that everything worked as before. Emacs now recognizes Left_Control as Hyper, and it no longer triggers GNOME Activities.