Close picker with <ESC> but enter normal mode with <C-\><C-n>

[uloco]

I have configured the picker to close on esc key with the recommended config from the docs.
And I have set a keymap to enter normal mode everywhere. I prefer closing pickers, input fields etc with <ESC>
My keymap: keymap.set("t", "<A-e>", "<C-\\><C-n>", opts)

Pressing <ESC> closes the picker, but pressing <A-e> does so too. How can I enter normal mode without closing the picker and still have <ESC> close the picker? I also tried mapping <ESC> to "cancel" but had the same effect.

win = {
  -- input window
  input = {
    keys = {
      -- to close the picker on ESC instead of going to normal mode,
      -- add the following keymap to your config
      -- ["<Esc>"] = { "close", mode = { "n", "i" } },
      -- ...
    }
  }

[drowning-cat]

opts = {
  picker = {
    actions = {
      stopinsert = function(picker)
        picker.input:stopinsert() -- vim.cmd.stopinsert()
      end,
    },
    win = {
      input = {
        keys = {
          ['<Esc>'] = { 'close', mode = { 'i', 'n' } },
          ['<M-e>'] = { 'stopinsert', mode = { 'i', 'n' } },

[drowning-cat]

t is for terminal mode

vim.keymap.set("t", "<A-e>", "<C-\\><C-n>", opts)

[uloco]

oh thank you so much!