[GH-ISSUE #549] Using kitty terminal the kitty +kitten ssh machine@ip causes the shell to freeze during an asciinema rec session when asking for password input #301

Closed
opened 2026-02-25 20:33:16 +03:00 by kerem · 11 comments
Owner

Originally created by @AkechiShiro on GitHub (May 14, 2023).
Original GitHub issue: https://github.com/asciinema/asciinema/issues/549

Describe the bug
Using https://github.com/kovidgoyal/kitty as a terminal, running asciinema rec test.cast and then kitty +kitten ssh, if a password prompt (see the custom password prompt) :
image
is asked by the remote ssh server, the shell freezes up.

To be clear, what we see during an asciinema record session is that :

  • Running kitty +kitten ssh root@server.ssh, the shell freezes up.
  • Only running kill -9 on the asciinema process allows getting back a working shell, CTRL+C does not work, nor any other signal for that matter.

To Reproduce
Steps to reproduce the behavior:

  1. Launch kitty terminal
  2. asciinema rec test.cast
  3. kitty +kitten ssh root@test.machine
  4. See a/b cases
    4a. If a password input is asked, the shell will freeze, this happens probably because kitty uses a custom way to handle password input for ssh (read additional context).
    4b. If no password prompt is asked, the ssh access works all fine, and there is no freeze, the recording is all good as well.

Expected behavior
In case a password is asked, the shell should not freeze, but the password input should be asked by kitty and then ssh login should be made, so the user can record their session.
Versions:

  • OS: ArchLinux Linux 6.2.10-arch1-1
  • asciinema cli: 2.2.0

Additional context
This issue seems very specific to kitty terminal emulator, and more specifically to the use of https://sw.kovidgoyal.net/kitty/kittens/ssh/
I suppose the issue is due to a conflict between how the kitty native ssh password input, askpass handles the terminal.

According to this documentation part about askpass

Control the program SSH uses to ask for passwords or confirmation of host keys etc. The default is to use kitty’s native askpass, unless the SSH_ASKPASS environment variable is set. Set this option to ssh to not interfere with the normal ssh askpass mechanism at all, which typically means that ssh will prompt at the terminal. Set it to native to always use kitty’s native, built-in askpass implementation. Note that not using the kitty askpass implementation means that SSH might need to use the terminal before the connection is established, so the kitten cannot use the terminal to send data without an extra roundtrip, adding to initial connection latency.

Setting SSH_ASKPASS=ssh and trying to reproduce the bug fails, which further shows that the bug is due to a specific interaction between asciinema and kitty's native askpass.

Here is the process tree (when the shell is frozen :
image
On the long line beginning by ssh -t, we can see a script being eval that contains the copyright from the maintainer of Kitty.
image
This piece of code may or may not contain the call to askpass, I suppose.

Originally created by @AkechiShiro on GitHub (May 14, 2023). Original GitHub issue: https://github.com/asciinema/asciinema/issues/549 **Describe the bug** Using https://github.com/kovidgoyal/kitty as a terminal, running `asciinema rec test.cast` and then `kitty +kitten ssh`, if a password prompt (see the custom password prompt) : ![image](https://github.com/asciinema/asciinema/assets/14914796/c77d79e5-e308-4e03-8469-fe9597e91823) is asked by the remote ssh server, the shell freezes up. To be clear, what we see during an asciinema record session is that : - Running `kitty +kitten ssh root@server.ssh`, the shell freezes up. - Only running `kill -9` on the asciinema process allows getting back a working shell, CTRL+C does not work, nor any other signal for that matter. **To Reproduce** Steps to reproduce the behavior: 1. Launch `kitty` terminal 2. `asciinema rec test.cast` 3. `kitty +kitten ssh root@test.machine` 4. See a/b cases 4a. If a password input is asked, the shell will freeze, this happens probably because kitty uses a custom way to handle password input for ssh (read additional context). 4b. If no password prompt is asked, the ssh access works all fine, and there is no freeze, the recording is all good as well. **Expected behavior** In case a password is asked, the shell should not freeze, but the password input should be asked by kitty and then ssh login should be made, so the user can record their session. **Versions:** - OS: ArchLinux Linux 6.2.10-arch1-1 - asciinema cli: 2.2.0 **Additional context** This issue seems very specific to kitty terminal emulator, and more specifically to the use of https://sw.kovidgoyal.net/kitty/kittens/ssh/ I suppose the issue is due to a conflict between how the kitty native ssh password input, askpass handles the terminal. [According to this documentation part about askpass](https://sw.kovidgoyal.net/kitty/kittens/ssh/#opt-kitten-ssh.askpass) > Control the program SSH uses to ask for passwords or confirmation of host keys etc. The default is to use kitty’s native askpass, unless the [SSH_ASKPASS](https://sw.kovidgoyal.net/kitty/glossary/#envvar-SSH_ASKPASS) environment variable is set. Set this option to ssh to not interfere with the normal ssh askpass mechanism at all, which typically means that ssh will prompt at the terminal. Set it to native to always use kitty’s native, built-in askpass implementation. Note that not using the kitty askpass implementation means that SSH might need to use the terminal before the connection is established, so the kitten cannot use the terminal to send data without an extra roundtrip, adding to initial connection latency. Setting `SSH_ASKPASS=ssh` and trying to reproduce the bug fails, which further shows that the bug is due to a specific interaction between `asciinema` and kitty's native `askpass`. Here is the process tree (when the shell is frozen : ![image](https://github.com/asciinema/asciinema/assets/14914796/92c24f01-9a73-4b9b-8e55-4b5a22890c0e) On the long line beginning by `ssh -t`, we can see a script being `eval` that contains the copyright from the maintainer of Kitty. ![image](https://github.com/asciinema/asciinema/assets/14914796/fa270072-04d9-4b1e-8614-8a7d154ea4af) This piece of code may or may not contain the call to `askpass`, I suppose.
kerem closed this issue 2026-02-25 20:33:16 +03:00
Author
Owner

@AkechiShiro commented on GitHub (May 14, 2023):

Additional context 2, here is the code from Kitty that is called for askpass I believe : github.com/kovidgoyal/kitty@9a0068e318/kittens/ssh/askpass.go (L37)

<!-- gh-comment-id:1546985030 --> @AkechiShiro commented on GitHub (May 14, 2023): Additional context 2, here is the code from Kitty that is called for askpass I believe : https://github.com/kovidgoyal/kitty/blob/9a0068e3188d8aca9a5a041365b81cc26f27c8b8/kittens/ssh/askpass.go#L37
Author
Owner

@ku1ik commented on GitHub (May 15, 2023):

Thanks for comprehensive description of the problem 👌

I wonder if other terminal recording tools have similar issue when running under kitty/ssh. Are you able to check with script command or termrec?

<!-- gh-comment-id:1548364010 --> @ku1ik commented on GitHub (May 15, 2023): Thanks for comprehensive description of the problem :ok_hand: I wonder if other terminal recording tools have similar issue when running under kitty/ssh. Are you able to check with `script` command or [termrec](https://aur.archlinux.org/packages/termrec)?
Author
Owner

@AkechiShiro commented on GitHub (May 15, 2023):

Let me give it a try soon and report on it as soon as possible !

<!-- gh-comment-id:1548489809 --> @AkechiShiro commented on GitHub (May 15, 2023): Let me give it a try soon and report on it as soon as possible !
Author
Owner

@AkechiShiro commented on GitHub (May 15, 2023):

termrec

There is the same issue, but timing seems of essence indeed, waiting only a few seconds and then sending signals CTLR+C/CTRL+D/CTRL+Z seems to have crashed termrec somehow, there has been a lot of base64 (probably from kitty's custom way of sending the terminfo to the ssh machine) that was spit out inside the terminal ending with KITTY_DATA_END, trying to wait 30 sec to 1 min before sending out any signals, it seems the shell is frozen and unresponsive to any signals just like with asciinema only using kill -9 termrec unfreezes the shell and provokes the spit out of lots of base64 lines in the terminal input.

  • process tree :
    image

  • trying to replay the recording with termplay shows this :
    image
    The termrec play never ends by itself, it seems that playing it, unless I press CTRL+C.
    I'm not sure if the base64 is sensible or not, I've hidden most of it just in case.
    The terminal seems to have some borked some tty settings I suppose after playing that recording, a reset is needed so that ls's output isn't usual, it's not using 4 spaced tabs/spaces, fonts are all good, it's just spacing/formatting of output of usual commands that seems broken.


script

  • Same behavior as termrec, it freezes the shell indefinitely just like asciinema, killing the it spits out base64 lines in the terminal ending with KITTY_DATA_END.
  • See process tree before killing script :
    image
  • tty settings seems to be borked the same way as for termrec as well,

asciinema

The same behavior is exhibited, tty settings are somewhat modified, killing is obligatory and shell is frozen until asciinema is killed.

  • Replaying the asciinema record showed this :
    image

I wonder about what if I kill instead the kitten process, what would happen in that case 🤔

EDIT 2: asciinema leaves processes that hang the shell exit, termrec and script do not.

<!-- gh-comment-id:1548534117 --> @AkechiShiro commented on GitHub (May 15, 2023): ## termrec There is the same issue, but timing seems of essence indeed, waiting only a few seconds and then sending signals CTLR+C/CTRL+D/CTRL+Z seems to have crashed termrec somehow, there has been a lot of base64 (probably from kitty's custom way of sending the terminfo to the ssh machine) that was spit out inside the terminal ending with `KITTY_DATA_END`, trying to wait 30 sec to 1 min before sending out any signals, it seems the shell is frozen and **unresponsive to any signals** just like with `asciinema` only using `kill -9 termrec` unfreezes the shell and provokes the spit out of lots of base64 lines in the terminal input. - process tree : ![image](https://github.com/asciinema/asciinema/assets/14914796/8d372e7e-018e-4f0e-b498-e040e6fce94f) - trying to replay the recording with `termplay` shows this : ![image](https://github.com/asciinema/asciinema/assets/14914796/4532ebd3-9025-4fe4-b8b6-ad8fe722ca1c) The `termrec` play never ends by itself, it seems that playing it, unless I press CTRL+C. I'm not sure if the base64 is sensible or not, I've hidden most of it just in case. The terminal seems to have some borked some tty settings I suppose after playing that recording, a `reset` is needed so that `ls`'s output isn't usual, it's not using 4 spaced tabs/spaces, fonts are all good, it's just spacing/formatting of output of usual commands that seems broken. --- ## script - Same behavior as `termrec`, it freezes the shell indefinitely just like `asciinema`, killing the it spits out base64 lines in the terminal ending with `KITTY_DATA_END`. - See process tree before killing script : ![image](https://github.com/asciinema/asciinema/assets/14914796/b9b29ff9-3d56-4314-bda9-a5d5dd1f9676) - tty settings seems to be borked the same way as for `termrec` as well, ## asciinema The same behavior is exhibited, tty settings are somewhat modified, killing is obligatory and shell is frozen until asciinema is killed. - Replaying the asciinema record showed this : ![image](https://github.com/asciinema/asciinema/assets/14914796/5eba05c5-9f22-4588-86a7-3558c378170f) I wonder about what if I kill instead the kitten process, what would happen in that case :thinking: EDIT 2: `asciinema` leaves processes that hang the shell exit, `termrec` and `script` do not.
Author
Owner

@AkechiShiro commented on GitHub (May 15, 2023):

Just noticed one thing, killing asciinema using SIGTERM shows askpass's prompt !

EDIT: Actually, I killed all the kitten process before doing that, then send a SIGTERM to one of the asciinema process, the more in depth in the process tree, this showed askpass prompt.

Trying to do the same without killing the kitten processes doesn't show the same behavior, the main asciinema does not budge with a SIGTERM, only a SIGKILL (=> spits back base64 into the terminal)

<!-- gh-comment-id:1548549775 --> @AkechiShiro commented on GitHub (May 15, 2023): Just noticed one thing, killing `asciinema` using `SIGTERM` shows `askpass`'s prompt ! EDIT: Actually, I killed all the `kitten` process before doing that, then send a `SIGTERM` to one of the `asciinema` process, the more in depth in the process tree, this showed `askpass` prompt. Trying to do the same without killing the `kitten` processes doesn't show the same behavior, the main `asciinema` does not budge with a `SIGTERM`, only a `SIGKILL` (=> spits back base64 into the terminal)
Author
Owner

@AkechiShiro commented on GitHub (May 15, 2023):

So finally, what would you recommend in order to try and dive deeper at the root of this issue, is it worth going that far ?

Or should I just settle for a workaround such as setting SSH_ASKPASS=ssh as an env variable, or establishing the connection with a kitty shell, not running any recording, entering the password (then kitty should remember it for as long as it's running, I believe?), so I could open an ssh connection in the shell with the recording of the session.

<!-- gh-comment-id:1548561539 --> @AkechiShiro commented on GitHub (May 15, 2023): So finally, what would you recommend in order to try and dive deeper at the root of this issue, is it worth going that far ? Or should I just settle for a workaround such as setting `SSH_ASKPASS=ssh` as an env variable, or establishing the connection with a kitty shell, not running any recording, entering the password (then kitty should remember it for as long as it's running, I believe?), so I could open an ssh connection in the shell with the recording of the session.
Author
Owner

@ku1ik commented on GitHub (May 16, 2023):

Given this problem is not specific to asciinema (as you found similar thing happens with termrec, script) I'd say the problem lies more on the kitty side. I'm not quite sure what's happening so I don't have a workaround idea at the moment.

I'm not saying asciinema is bug free in this case. In fact it's not impossible that script and termrec have the same bug. However, I find it rather unlikely that 3 separate codebases would have the same problem. Especially that for example script has been out there for over 40 years - if it had a bug like that it would most likely surface already in other terminal emulators.

<!-- gh-comment-id:1549198209 --> @ku1ik commented on GitHub (May 16, 2023): Given this problem is not specific to asciinema (as you found similar thing happens with termrec, script) I'd say the problem lies more on the kitty side. I'm not quite sure what's happening so I don't have a workaround idea at the moment. I'm not saying asciinema is bug free in this case. In fact it's not impossible that script and termrec have the same bug. However, I find it rather unlikely that 3 separate codebases would have the same problem. Especially that for example script has been out there for over 40 years - if it had a bug like that it would most likely surface already in other terminal emulators.
Author
Owner

@ku1ik commented on GitHub (May 16, 2023):

For completeness you can also test with ttyrec, which records in the same way as asciinema, termrec, script.

<!-- gh-comment-id:1549202769 --> @ku1ik commented on GitHub (May 16, 2023): For completeness you can also test with [ttyrec](https://aur.archlinux.org/packages/ovh-ttyrec-git), which records in the same way as asciinema, termrec, script.
Author
Owner

@ku1ik commented on GitHub (May 16, 2023):

From the other issue:

I don't understand also why asciinema create 3 processes, forking 2 more, when most tools only create one ?

I'll reply here since it's not related to kitty:

There's main process where copying of data between recorded process and TTY happens (the code linked above).
There's another process dedicated for file writing, in order to not block process->TTY data copying on disk I/O.
And another one for notifier (which handles triggering desktop notification in certain situations), also to not block data copying by notifications.

<!-- gh-comment-id:1550211335 --> @ku1ik commented on GitHub (May 16, 2023): From the other issue: > I don't understand also why asciinema create 3 processes, forking 2 more, when most tools only create one ? I'll reply here since it's not related to kitty: There's main process where copying of data between recorded process and TTY happens (the code linked above). There's another process dedicated for file writing, in order to not block process->TTY data copying on disk I/O. And another one for notifier (which handles triggering desktop notification in certain situations), also to not block data copying by notifications.
Author
Owner

@ku1ik commented on GitHub (Jul 5, 2023):

This should be fixed in 2.3.0.

<!-- gh-comment-id:1621863729 --> @ku1ik commented on GitHub (Jul 5, 2023): This should be fixed in 2.3.0.
Author
Owner

@AkechiShiro commented on GitHub (Jul 5, 2023):

I'll test and then close the issue once I tested it, thanks !

<!-- gh-comment-id:1622302967 --> @AkechiShiro commented on GitHub (Jul 5, 2023): I'll test and then close the issue once I tested it, thanks !
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/asciinema#301
No description provided.