rlselect

Jan 19, 2025 ∞

Replacing Ctrl-R in Bash without TIOCSTI

I have previously written about how I use rlselect as a replacement for Ctrl+R in Bash.

It works by creating a key binding in Bash for Ctrl+R that invokes rlselect instead of the default Bash interactive history search command. rlselect looks something like this:

Screenshot of rlselect showing two entries, hello
and world, with hello selected.

If you press tab, the current selection is inserted at the prompt. If you press enter, the current selection is executed. This is the same behavior as the default Ctrl+R.

The mechanism for this stopped working in recent Linux kernel versions. I figured out how to solve it and in this blog post I explain how.

Old Mechanism

When rlselect is invoked from Ctrl+R, it is invoked with the --tab and --action flags. The first flag allows the tab key to be used to select a line and the second makes rlselect print the action taken on the first line before to the selection.

Here is an example where enter is pressed when “hello” is selected:

$ (echo hello; echo world) | rlselect --tab --action
enter
hello

Here is an example where tab is pressed when “world” is selected:

$ (echo hello; echo world) | rlselect --tab --action
tab
world

Here is an example where Ctrl+G is pressed:

$ (echo hello; echo world) | rlselect --tab --action
ctrl-g

Ctrl+G aborts, so no selection is printed on the second line.

To feed this output to the prompt, TIOCSTI is used. It simulates that you type characters in the terminal. The full script that Ctrl+R invokes looks like this:

set -e

result=$(tac ~/.bash_history | rlselect --tab --action -- "$@")

python - "$result" << EOF
import fcntl
import sys
import termios

action, selection = sys.argv[1].split("\n", 1)

if action != "tab":
    selection += "\n"

for ch in selection:
    fcntl.ioctl(sys.stdout.fileno(), termios.TIOCSTI, ch)
EOF

The last part is where TIOCSTI is used to simulate that you press the keys of the selection. Unless tab is pressed, it appends a newline to the selection to simulate that Enter is pressed.

The Bash configuration looks like this:

if [[ $- =~ .*i.* ]]; then bind '"\C-r": "\C-a rlselect-history \C-j"'; fi

Here is how it works:

Ctrl+R is bound to a series of keystrokes.
First Ctrl+A is simulated which takes the cursor to the beginning of the line.
Then <space>rlselect-history<space> is typed.
Then Ctrl+J is simulated which means accept the current line. Or execute it. The initial space entered in the previous step ensures that the rlselect-history command does not end up in the history. The moving of the cursor to the beginning of the line ensures that anything typed at the prompt is passed as an argument to rlselect-history.

(This configuration also makes the text rlselect-history ... appear in the terminal. The new mechanism makes that go away.)

This mechanism stopped working in recent Linux kernel versions because TIOCSTI can not be used like this. There is apparently security issues with TIOCSTI and it is now only allowed as root.

New Mechanism

The new Bash configuration for Ctrl+R behavior that I came up with looks like this:

rlselect-history() {
    local action
    local selection
    {
        read action
        read selection
    } < <(tac ~/.bash_history | rlselect --tab --action -- "${READLINE_LINE}")
    if [ "$action" = "tab" ]; then
        READLINE_LINE="${selection}"
        READLINE_POINT=${#READLINE_LINE}
        bind '"\C-x2":' # Bind Ctrl+x+2 to do nothing
    elif [ "$action" = "enter" ]; then
        READLINE_LINE="${selection}"
        READLINE_POINT=${#READLINE_LINE}
        bind '"\C-x2": accept-line' # Bind Ctrl+x+2 to accept line
    else
        bind '"\C-x2":' # Bind Ctrl+x+2 to do nothing
    fi
}

if [[ $- =~ .*i.* ]]; then
    # Bind history command to Ctrl+x+1 followed by Ctrl+x+2:
    bind '"\C-r": "\C-x1\C-x2"'
    # Bind Ctrl+x+1 to execute rlselect-history which does two things:
    # 1. Sets READLINE_*
    # 2. Binds Ctrl+x+2 to either accept line or do nothing.
    bind -x '"\C-x1": rlselect-history'
fi

Let’s break this down.

Ctrl+R is bound to a series of keystrokes.
First Ctrl+X+1 is simulated.
Then Ctrl+X+2 is simulated.
Ctrl+X+1 is bound to execute the command rlselect-history. The -x to bind ensures that the variables READLINE_* can be set. From man bash on set -x:

Cause shell-command to be executed whenever keyseq is entered. When shell-command is executed, the shell sets the READLINE_LINE variable to the contents of the readline line buffer and the READLINE_POINT and READLINE_MARK variables […] If the executed command changes the value of any of READLINE_LINE, READLINE_POINT, or READLINE_MARK, those new values will be reflected in the editing state.
rlselect-history is defined as a Bash function which allows it to reconfigure the key binding for Ctrl+X+2. Depending on if the current selection should be executed or not, it binds Ctrl+X+2 to either accept-line or nothing.

So the new mechanism relies on using two extra key bindings: Ctrl+X+1 and Ctrl+X+2. I chose them because I don’t use them otherwise. But they can be any two key bindings.

The trick to finding this solution for me was understanding Bash key bindings. This answer on StackOverflow writes the following:

With bind, you can bind keys to do one of three things, but no combination of them:

Execute a readline command: bind '"key": command'

Execute a series of keystrokes: bind '"key":"keystrokes"'

Execute a shell command: bind -x '"key": shell-command'

That made me understand that you can not call accept-line from within rlselect-history because it is executed in the context of bind -x, and readline commands can only be executed in the context of bind '"key": command'.

Resources

Here are some resources that talks about the problem with TIOCSTI that helped me:

hstr (the program that initially inspired me to write rlselect) had a similar problem and I found clues to my solution there.
The fzf-plugins repo and this dicussion provides a similar solution for fzf.
The article Readline and Fuzzy Finder helped me understand how to work with READLINE_* in Bash.

May 19, 2017 ∞

Evolution of recalling Bash history

This article is about how I’ve become more efficient at using Bash, the interactive UNIX shell.

When I work in Bash, I often want to execute a command again. In the beginning I re-typed the command and pressed enter. This worked fine for short commands, but became tedious for longer commands.

In some shells this is the only way to enter a new command. But Bash remembers the recently executed commands and provides ways to recall them.

Cycle with arrow keys

The first way I learned to recall history was with the arrow keys. If I pressed Up the previous command was inserted at the prompt. I could continue for as long as I wanted. If I pressed Down the next command was inserted at the prompt:

$ ls<Enter>
bin         ...

$ date<Enter>
Wed May 10 08:14:46 CEST 2017

$ <Up>

$ date<Up>

$ ls<Down>

$ date<Enter>
Wed May 10 08:14:59 CEST 2017

This worked fine for commands that I had executed recently, but tedious for commands that I had executed long ago because I had to press Up many times. I ended up pressing and holding Up so that history scrolled by and when I saw my command, I released the key and pressed Down until it appeared again.

Cycle with Ctrl-P/Ctrl-N

Later I learned that Ctrl-P (previous) had the same function as Up and that Ctrl-N (next) had the same function as Down.

These shortcuts were more comfortable for me because I like to keep my fingers as close to the home row as possible.

Searching with Ctrl-R

Then I learned about Bash’s interactive history search command. If I pressed Ctrl-R the prompt changed to this:

(reverse-i-search)`':

This special prompt allowed me to type parts of a command that I had executed previously. Say I wanted to execute the last find command again. I typed “find” and the prompt changed to this:

(reverse-i-search)`find': find -name '*.py' -a -type f

The text I typed, “find”, was present before the colon. After the colon the last command that I had executed that contained the string “find” was displayed. In this case I did a search for Python files. If this was not the match I was looking for, I could hit Ctrl-R again and the part to the right of the colon would change to the next command in the history that contained the string “find”. Once I found the command I was looking for I had two options: I could hit Tab to insert the command at the prompt:

$ find -name '*.py' -a -type f

This way I could edit the command before I executed it. Or I could hit Enter to execute the command directly.

Now I was able to recall commands that I had executed long ago. I almost replaced all my usages of Ctrl-P/Ctrl-N with Ctrl-R. Except for the cases where I knew that the command I wanted to recall was only a few entries back.

Frustrations with Ctrl-R

The interactive search worked great for me when I knew what I was looking for. It did not work so great when I was more uncertain or when I mistyped the name of a command.

The interactive search works by having a pointer to en entry in the history. When I typed a command it would move that pointer to the next item in the history that matched. But if I mistyped, the search might still match something further back in history. But when I erased a few characters to correct my mistake, the search would continue from there. Say this was my history:

tac ~/.bash_history
echo frustration
echo with
echo bash

I hit Ctrl-R to to begin searching for “bash”:

(reverse-i-search)`':

But I mistyped. Instead of “b” I typed “f”:

(reverse-i-search)`f': echo frustration

The search matched item 2. I erased the incorrectly typed character:

(reverse-i-search)`': echo frustration

The match remained. I typed bash correctly:

(reverse-i-search)`bash': tac ~/.bash_history

It now matched item 1 instead of item 4. The search continued from the previous match. I would have wanted the search to always show the most recent match from history. The easiest way I found to reset the search after a failure to find what I was looking for was to just execute a dummy command. Usually I selected ls because it was short to type and had no side effects.

Interactively filtering with external program

Then I was introduced to hstr by a colleague. It worked like a replacement for Ctrl-R. When I invoked it, it dropped into a text UI where my last history entries were shown. I could also type part of a command to narrow down the list. If I changed the search string, the narrowed down list changed accordingly. When I found a match I could similarly press Tab to insert the command at the prompt or press Enter to execute it immediately. It looked like this:

Demo of hstr (from their website)

This solved my frustrations with Bash’s interactive search. For me, this was a far easier way to find items from my history. The fact that it showed the last commands also helped me. I could visually inspect them, and they would guide my search.

hstr was so good that I wanted to use a similar selection mechanism for other things, but hstr was only for Bash history. I ended up writing my own selection program: rlselect. Partly because I wanted such a program, but also because it seemed like a fun program to write. The core selection program is called rlselect and then there are multiple programs that use it to allow selecting specific things. rlselect-history is a replacement for Ctrl-R/hstr:

Demo of rlselect

There are some differences between hstr and rlselect-history. I took only the parts I personally wanted from hstr and put them into rlselect-history.

If you want to improve your Bash usage, I suggest taking a look at hstr or rlselect.