Don't escape tildes that come from custom completions

A completion entry like «complete -a '\\~'» results in completions
that insert \~ into the command line.  However we usually want to
insert ~, but there is no way to do that.

There are a couple of longstanding issues about completion escaping
[1].  Until we fix those in a general way, fix the common case by
never escaping tildes when applying custom completions to the command
line. This is a hack but will probably work out fine because we don't
expect literal tildes in arguments.

The tilde is included in completions for cdh, or
__fish_complete_suffix, which simply forwards results from "complete
-C". Revert a workaround to cdh that expanded ~, because we can now
render that without escaping.

Closes #4570, #8441

[ja: tweak patch and commit message]

[1]: https://github.com/fish-shell/fish-shell/pull/8441#discussion_r748803338

CHANGELOG.rst

@@ -76,6 +76,7 @@ Scripting improvements
 - ``dirs`` always produces an exit status of 0, instead of sometimes returning 1 (:issue:`8211`).
 - ``cd ""`` no longer crashes fish (:issue:`8147`).
 - ``set --query`` can now query whether a variable is a pathvar via ``--path`` or ``--unpath`` (:issue:`8494`).
+- Tilde characters (``~``) produced by custom completions are no longer escaped when applied to the command line, making it easier to use the output of a recursive ``complete -C`` in completion scripts (:issue:`4570`).
 
 Interactive improvements
 ------------------------

share/completions/cdh.fish

@@ -11,17 +11,10 @@ function __fish_cdh_args
         end
     end
 
-    # Only shorten $HOME to "~" if the token starts with a "~",
-    # otherwise fish will escape it.
-    set -l shorten_tilde 0
-    string match -q '~*' -- (commandline -ct); and set shorten_tilde 1
-
     for dir in $uniq_dirs
-        if test $shorten_tilde -eq 1
+        set -l home_dir (string match -r "$HOME(/.*|\$)" "$dir")
-            set -l home_dir (string match -r "$HOME(/.*|\$)" "$dir")
+        if set -q home_dir[2]
-            if set -q home_dir[2]
+            set dir "~$home_dir[2]"
-                set dir "~$home_dir[2]"
-            end
         end
         echo $dir
     end

share/completions/obnam.fish

@@ -160,4 +160,4 @@ complete -c obnam --no-files -l no-pure-paramiko -d 'Use openssh if available'
 complete -c obnam -l ssh-command -d 'Executable to be used instead of "ssh"'
 complete -c obnam --no-files -l ssh-host-keys-check -a 'no yes ask ssh-config' -d 'ssh host key check'
 complete -c obnam -l ssh-key -d 'Use FILENAME as the ssh RSA key'
-complete -c obnam -l ssh-known-hosts -a '~/.ssh/known_hosts' -d 'FILENAME of the known_hosts file'
+complete -c obnam -l ssh-known-hosts -a '\\~/.ssh/known_hosts' -d 'FILENAME of the known_hosts file'

src/builtins/complete.cpp

@@ -418,6 +418,9 @@ maybe_t<int> builtin_complete(parser_t &parser, io_streams_t &streams, const wch
         }
     } else {
         int flags = COMPLETE_AUTO_SPACE;
+        // HACK: Don't escape tildes because at the beginning of a token they probably mean
+        // $HOME, for example as produced by a recursive call to "complete -C".
+        flags |= COMPLETE_DONT_ESCAPE_TILDES;
         if (preserve_order) {
             flags |= COMPLETE_DONT_SORT;
         }