Future of /usr/bin/which in Debian?

[Boyuan Yang]

Hi all,

I just noticed that the "which" command provided by debianutils has been
declared as deprecated in Debian Sid:

% /usr/bin/which test
/usr/bin/which: this version of 'which' is deprecated and should not be used.
/usr/bin/test

Obviously it was caused by the upload of debianutils/5.0-1 onto Unstable (see
https://salsa.debian.org/debian/debianutils/-/commit/3a8dd10b4502f7bae8fc6973c13ce23fc9da7efb
):

--------------------------------------
.SH DEPRECATION
Since type and command \-v were mandated by POSIX, this utility is no longer
useful for maintainer scripts and thus will be removed from debianutils.
--------------------------------------

I did some search and found related discussion thread at
https://lists.debian.org/debian-devel/2020/08/msg00081.html , which looks
pretty reasonable.

Now I'm just wondering what would be the good alternative that provides
/usr/bin/which. Obviously we had a round of discussion already (see
https://lists.debian.org/debian-devel/2020/08/msg00340.html ), but probably we
need a solution this time following debianutils which's deprecation. The most
reasonable choice might be GNU which anyway.

Besides, will the new "which" tool be installed in Debian by default? Since
debianutils is Essential:yes, not providing "which" tool by default could
probably break some existing packages.

Thanks,
Boyuan Yang

[Clint Adams]

Post by Boyuan Yang
Besides, will the new "which" tool be installed in Debian by default? Since
debianutils is Essential:yes, not providing "which" tool by default could
probably break some existing packages.

My personal opinion is that no one should be using `which` in maintainer
scripts and that no one should be using `which` in an interactive shell
unless it is a builtin of that shell.

There are a ton of postinst scripts relying on `which` right
now, which makes sense because `type` and `command -v` used to be
optional extensions to POSIX and not guaranteed to be provided
by /bin/sh.

[Gunnar Wolf]

Post by Clint Adams

Post by Boyuan Yang
Besides, will the new "which" tool be installed in Debian by default? Since
debianutils is Essential:yes, not providing "which" tool by default could
probably break some existing packages.

My personal opinion is that no one should be using `which` in maintainer
scripts and that no one should be using `which` in an interactive shell
unless it is a builtin of that shell.
There are a ton of postinst scripts relying on `which` right
now, which makes sense because `type` and `command -v` used to be
optional extensions to POSIX and not guaranteed to be provided
by /bin/sh.

I agree with you, maintainer scripts should not rely on 'which'
anymore. However, what about users? 'which' is a standard Unix tool
since forever, and I expect many users to experience head scratching
when told it's not cool to use it anymore.

[Andrey Rahmatullin]

Post by Gunnar Wolf
I agree with you, maintainer scripts should not rely on 'which'
anymore. However, what about users? 'which' is a standard Unix tool
since forever, and I expect many users to experience head scratching
when told it's not cool to use it anymore.

Why is a standard Unix tool since forever shipped by a package called
debianutils, that has a description saying "Miscellaneous utilities
specific to Debian" and "used primarily by the installation scripts of
Debian packages"? Surely a standard Unix tool would, or at least could, be
shipped somewhere else?

[Clint Adams]

<https://minnie.tuhs.org/cgi-bin/utree.pl?file=3BSD/usr/man/man1/which.1>
(1979). This was csh-specific and I think it later became built-into
csh.
<https://www.freebsd.org/cgi/man.cgi?query=which&apropos=0&sektion=0&manpath=FreeBSD+2.1.0-RELEASE&arch=default&format=html>
(1995). This was written in Perl and then rewritten in C a few years
later.
Debian's implementation started out in 1995 or 1996 as a shell script
calling 'type', and remains a shell script.
<https://www.freebsd.org/cgi/man.cgi?query=which&apropos=0&sektion=0&manpath=CentOS+7.1&arch=default&format=html>,
<https://carlowood.github.io/which/> (1999 or earlier). This is
written in C.
So 'which' has a very long history in csh, which was the default
interactive shell in many versions of Unix. Its availability to other
shells dates back to the 90s (at least) but is not portable due to
there being multiple very different implementations.

I'm happy to transition /usr/bin/which to alternatives if people are
interested in packaging all of these.

[Geert Stappers]

Post by Clint Adams
I'm happy to transition /usr/bin/which to alternatives

Which alternatives would that be?


| $ apt show alternatives
| N: Unable to locate package alternatives
| $

Post by Clint Adams
if people are interested in packaging all of these.

Groeten
Geert Stappers

[Clint Adams]

Post by Geert Stappers
Which alternatives would that be?

I meant

update-alternatives --install /usr/bin/which which /usr/bin/which.debianutils 0

in postinst and so on so that FreeBSD which and GNU which and friends could
take over.

[Geert Stappers]

} } I'm happy to transition /usr/bin/which to alternatives

Post by Geert Stappers
Which alternatives would that be?

I meant
update-alternatives --install /usr/bin/which which /usr/bin/which.debianutils 0
in postinst and so on so that FreeBSD which and GNU which and friends could
take over.

Please do. Make such take over possible.
It is what https://salsa.debian.org/debian/debianutils/-/merge_requests/6#note_242172
is asking for.



Groeten
Geert Stappers

[Clint Adams]

Post by Geert Stappers
Please do. Make such take over possible.
It is what https://salsa.debian.org/debian/debianutils/-/merge_requests/6#note_242172
is asking for.

Okay, you have it in debianutils 5.1-2.

[Boyuan Yang]

圚 2021-08-18星期䞉的 22:59 +0200Geert Stappers写道

} } I'm happy to transition /usr/bin/which to alternatives

Post by Geert Stappers
Which alternatives would that be?

I meant
update-alternatives --install /usr/bin/which which
/usr/bin/which.debianutils 0
in postinst and so on so that FreeBSD which and GNU which and friends could
take over.

Please do.  Make such take over possible.
It is what
https://salsa.debian.org/debian/debianutils/-/merge_requests/6#note_242172
is asking for.

So we will have https://salsa.debian.org/debian/which-gnu providing a binary
package with name "which". I will upload it to the NEW queue soon.

Still not sure about the the package's priority. Now I am using Priority:
optional, but we may raise it later if needed.

Thanks,
Boyuan Yang

[Paul Wise]

Post by Boyuan Yang
So we will have https://salsa.debian.org/debian/which-gnu providing a binary
package with name "which". I will upload it to the NEW queue soon.

Some folks on IRC wanted to package the FreeBSD which too, so I
suggest using which-gnu as the binary package name too.

[YunQiang Su]

Post by Paul Wise

Post by Boyuan Yang
So we will have https://salsa.debian.org/debian/which-gnu providing a binary
package with name "which". I will upload it to the NEW queue soon.

Some folks on IRC wanted to package the FreeBSD which too, so I
suggest using which-gnu as the binary package name too.

For such a simple tool, do we really need such many versions?

Post by Paul Wise
--
bye,
pabs
https://wiki.debian.org/PaulWise

[Calum McConnell]

Post by YunQiang Su

Post by Paul Wise

So we will have https://salsa.debian.org/debian/which-gnu providing
a binary
package with name "which". I will upload it to the NEW queue soon.

Some folks on IRC wanted to package the FreeBSD which too, so I
suggest using which-gnu as the binary package name too.

For such a simple tool, do we really need such many versions?

Which which witches wish to which with will wildly wander. We wish
welcome to witches which which with weird whichs, which will want whiches
which witches who were wasted winds when we were whelks used. Which which
any witch uses is a decision whence the heart, which we wish to watch each
which make.

[Jonathan Dowland]

Post by YunQiang Su
For such a simple tool, do we really need such many versions?

At least you've asked the question. I'd love to think that there was a
proper evaluation of BSD which versus GNU which prior to the latter
being uploaded to NEW, and there's a compelling reason that the GNU one
was chosen; but if so there's no evidence of that on -devel. :(

[Michael Stone]

Post by Jonathan Dowland

Post by YunQiang Su
For such a simple tool, do we really need such many versions?

At least you've asked the question. I'd love to think that there was a
proper evaluation of BSD which versus GNU which prior to the latter
being uploaded to NEW, and there's a compelling reason that the GNU one
was chosen; but if so there's no evidence of that on -devel. :(

It seems to install to /usr/bin/which.gnu, implying that you could
upload /usr/bin/which.bsd if you so desire; what's the issue?

[Jonathan Dowland]

Post by Michael Stone
It seems to install to /usr/bin/which.gnu, implying that you could
upload /usr/bin/which.bsd if you so desire; what's the issue?

I think we should have just one which implementation in the archive. We
should (have) pick(ed) the best one for Debian. I believe (perhaps
unfairly... I'd love to be proven wrong) that the GNU implementation was
uploaded very quickly, without the BSD implementation being considered.
Perhaps the GNU one is the best fit for our needs. It would have been
nice to see that there was an evaluation.

[Michael Stone]

Post by Jonathan Dowland

Post by Michael Stone
It seems to install to /usr/bin/which.gnu, implying that you could
upload /usr/bin/which.bsd if you so desire; what's the issue?

I think we should have just one which implementation in the archive. We
should (have) pick(ed) the best one for Debian. I believe (perhaps
unfairly... I'd love to be proven wrong) that the GNU implementation was
uploaded very quickly, without the BSD implementation being considered.
Perhaps the GNU one is the best fit for our needs. It would have been
nice to see that there was an evaluation.

I think it doesn't matter how many which implementations are in debian.
If you want something with specific portable semantics, just use command
-v. The remaining consumers of which are either programs that (ipso
facto) don't care about semantic corner cases or are humans who want to
use which just because, and probably have strong opinions on how it
should behave (as, apparently, you do). We can't satisfy everybody with
one implementation, and I see no technical reason that we'd even try to
do so.

[Nicholas D Steeves]

Post by Michael Stone

Post by Jonathan Dowland

Post by Michael Stone
It seems to install to /usr/bin/which.gnu, implying that you could
upload /usr/bin/which.bsd if you so desire; what's the issue?

I think we should have just one which implementation in the archive. We
should (have) pick(ed) the best one for Debian. I believe (perhaps
unfairly... I'd love to be proven wrong) that the GNU implementation was
uploaded very quickly, without the BSD implementation being considered.
Perhaps the GNU one is the best fit for our needs. It would have been
nice to see that there was an evaluation.

I think it doesn't matter how many which implementations are in debian.
If you want something with specific portable semantics, just use command
-v. The remaining consumers of which are either programs that (ipso
facto) don't care about semantic corner cases or are humans who want to
use which just because, and probably have strong opinions on how it
should behave (as, apparently, you do). We can't satisfy everybody with
one implementation, and I see no technical reason that we'd even try to
do so.

+1 for everything above. I also think it may be more reasonable to
prefer (by default, using the alternatives mechanism) the more LSBish
one (in this case GNU) rather than the potentially more simple, clean,
and full-featured one (BSD). For precedent see netcat-traditional vs
netcat-openbsd, and GNU tar vs bsdtar and pax--particularly during the
time when bsdtar was superior (iirc) to GNU tar.

I also acknowledge that this may entrench the precedent of preferring a
GNU/Linux-standard solution that may not be the technically best. For
the record, I seem to remember using bsdtar for the period of time when
GNU tar had some issue or another (or was it a missing feature? Maybe
xattr, sparse file support, or acl-related?), and I also have a strong
personal preference for netcat-openbsd.

Thankfully we have the /etc/alternatives and Provides mechanisms to
affirm user choice in such cases, and I think most of us will agree this
is a totally equitable and reasonable compromise :-)

Regards,
Nicholas

[Jonathan Dowland]

Post by Michael Stone
I think it doesn't matter how many which implementations are in debian.
If you want something with specific portable semantics, just use command
-v. The remaining consumers of which are either programs that (ipso
facto) don't care about semantic corner cases or are humans who want to
use which just because, and probably have strong opinions on how it
should behave (as, apparently, you do).

*I* don't; in Clint's categorization¹ I fall into (d) "wants there to be
exactly one version of `which` (except for all the shell builtins) so
that alternatives won't confuse and complicate things". The point I've
tried to make (too clumsily I guess) is the process of choosing one
should not be shoot-from-the-hip: there should be some consideration as
to which `which` would be the best fit for Debian. I hadn't seen any
evidence of that, until,

I also think it may be more reasonable to prefer (by default, using the
alternatives mechanism) the more LSBish one (in this case GNU) rather
than the potentially more simple, clean, and full-featured one (BSD).

^ this is an example of exactly what I would have hoped took place to
decide upon which `which` we provided.

Thankfully we have the /etc/alternatives and Provides mechanisms to
affirm user choice in such cases, and I think most of us will agree this
is a totally equitable and reasonable compromise :-)

Unless there's a really compelling reason for there to be more than one
`which`, we could avoid the burden of alternatives entirely.


I should get off my soapbox now.


¹ Message-ID: <***@scru.org>

[The Wanderer]

Post by Michael Stone

Post by Jonathan Dowland

Post by Michael Stone
It seems to install to /usr/bin/which.gnu, implying that you
could upload /usr/bin/which.bsd if you so desire; what's the
issue?

I think we should have just one which implementation in the
archive. We should (have) pick(ed) the best one for Debian. I
believe (perhaps unfairly... I'd love to be proven wrong) that the
GNU implementation was uploaded very quickly, without the BSD
implementation being considered. Perhaps the GNU one is the best
fit for our needs. It would have been nice to see that there was an
evaluation.

I think it doesn't matter how many which implementations are in
debian. If you want something with specific portable semantics, just
use command -v.

I think I've seen that suggested a lot as an alternative for 'which',
but it doesn't seem to be comparably reliable in all contexts.

The primary issue I've run across to date is with aliases.

For example, on my computer as I type this:

$ which ls
/usr/bin/ls
$ command -v ls
alias ls='ls -N --color=auto'
$ $(which ls) /
bin home lib32 media pulse srv var
boot initrd.img lib64 mnt root sys vmlinuz
dev initrd.img.old libx32 opt run tmp vmlinuz.old
etc lib lost+found proc sbin usr
$ $(command -v ls) /
bash: alias: -N: not found
bash: alias: /: not found

And then 'ls' is broken in that shell session; I haven't yet found a way
to get it working again, short of exiting and re-launching the shell.
(Though I also haven't tried *terribly* hard.)

This seems to demonstrate that you can't safely just use 'command -v'
wherever you would otherwise use 'which'.

[Russ Allbery]

Post by The Wanderer

Post by Michael Stone
I think it doesn't matter how many which implementations are in
debian. If you want something with specific portable semantics, just
use command -v.

I think I've seen that suggested a lot as an alternative for 'which',
but it doesn't seem to be comparably reliable in all contexts.

I don't think the point is that command -v is a drop-in replacement for
which (it definitely is not). I think the point is that command -v is a
standardized, portable interface. If you want portable semantics, the
standardized command is command -v, but it doesn't do quite the same thing
in quite the same way. If you want which, you have to live with the fact
that it's not portable and different which implementations behave in
different ways.

[Fabrice Bauzac-Stehly]

Post by The Wanderer

Post by Michael Stone
If you want something with specific portable semantics, just
use command -v.

I think I've seen that suggested a lot as an alternative for 'which',
but it doesn't seem to be comparably reliable in all contexts.
The primary issue I've run across to date is with aliases.

I don't know if it is always the case, especially with bizarre zsh
setups, but on my computers, aliases are not defined in shell scripts,
only in interactive shells.

If it's an issue, a shell script can do this beforehand:

\unalias -a

See https://pubs.opengroup.org/onlinepubs/009695399/utilities/command.html

[Andrej Shadura]

Hi,

Post by Boyuan Yang

Post by Clint Adams
/usr/bin/which.debianutils 0
in postinst and so on so that FreeBSD which and GNU which and friends could
take over.

Please do.  Make such take over possible.
It is what
https://salsa.debian.org/debian/debianutils/-/merge_requests/6#note_242172
is asking for.

So we will have https://salsa.debian.org/debian/which-gnu providing a binary
package with name "which". I will upload it to the NEW queue soon.

I'd rather suggest the FreeBSD which, which I'm mirroring here:
https://salsa.debian.org/andrewsh/freebsd-which/

I think it's much simpler than the GNU one.

[Boyuan Yang]

Hi,

圚 2021-08-22星期日的 17:36 +0200Andrej Shadura写道

Post by Andrej Shadura
Hi,

Post by Clint Adams
/usr/bin/which.debianutils 0
in postinst and so on so that FreeBSD which and GNU which and friends could
take over.

Please do.  Make such take over possible.
It is what
https://salsa.debian.org/debian/debianutils/-/merge_requests/6#note_242172
is asking for.

So we will have https://salsa.debian.org/debian/which-gnu providing a
binary
package with name "which". I will upload it to the NEW queue soon.

https://salsa.debian.org/andrewsh/freebsd-which/
I think it's much simpler than the GNU one.

The GNU which package is now in NEW queue:
https://ftp-master.debian.org/new/gnu-which_2.21-1.html

Having both freebsd which and gnu which in Debian archive is definitely ok. If
you would like, please also upload freebsd-which onto unstable.

Thanks,
Boyuan Yang

[Sean Whitton]

Hello Boyuan,

Post by Boyuan Yang
Hi,
圚 2021-08-22星期日的 17:36 +0200Andrej Shadura写道

Post by Andrej Shadura
Hi,

Post by Clint Adams
/usr/bin/which.debianutils 0
in postinst and so on so that FreeBSD which and GNU which and friends could
take over.

Please do.  Make such take over possible.
It is what
https://salsa.debian.org/debian/debianutils/-/merge_requests/6#note_242172
is asking for.

So we will have https://salsa.debian.org/debian/which-gnu providing a
binary
package with name "which". I will upload it to the NEW queue soon.

https://salsa.debian.org/andrewsh/freebsd-which/
I think it's much simpler than the GNU one.

https://ftp-master.debian.org/new/gnu-which_2.21-1.html
Having both freebsd which and gnu which in Debian archive is definitely ok. If
you would like, please also upload freebsd-which onto unstable.

It's okay, indeed, but please do consider NEW queue workload with things
like this -- upload it if you're sure it's going to get used, not just
for completeness.

[xiao sheng wen(肖盛文)]

Post by Sean Whitton

Post by Boyuan Yang

Post by Andrej Shadura
https://salsa.debian.org/andrewsh/freebsd-which/
I think it's much simpler than the GNU one.

https://ftp-master.debian.org/new/gnu-which_2.21-1.html
Having both freebsd which and gnu which in Debian archive is definitely ok. If
you would like, please also upload freebsd-which onto unstable.

It's okay, indeed, but please do consider NEW queue workload with things
like this -- upload it if you're sure it's going to get used, not just
for completeness.

If there are several different versions of which in Debian, this will
give the user that have a chance to choice one.

[Wookey]

Post by Boyuan Yang
Hi,
https://ftp-master.debian.org/new/gnu-which_2.21-1.html

Thanks for this. Missing which broke bazel (at least in tensorflow)
and the implementation made it fiddly to replace <command> with
<command+option> as recommended.

Have a plain which back is much easier than fixing the build mess.

I must admit that I have no idea why replacing such a longstanding
utility is deemed necessary.

Wookey

[Clint Adams]

Post by Wookey
I must admit that I have no idea why replacing such a longstanding
utility is deemed necessary.

Maybe this riddle will help.

Imagine that you are the product manager for Debian `which`. According
to the hatemail in my inbox, this is the most important utility in the
history of time, such that even printing a warning to stderr causes
global devastation, block hints, and calls for impeachment. So, it
makes sense for this to be a full-time job, though perhaps you manage
another, less significant utility as well.

You go on a Gemba walk, and discover you have several user personas
amongst your customers:

(a) wants GNU `which`, to have feature parity with Red Hat Enterprise Linux
(b) wants FreeBSD `which`, to have feature parity with FreeBSD
(c) wants nothing ever to change, and the xiafs removal from Linux 2.1.21
to be reverted
(d) wants there to be exactly one version of `which` (except for all the
shell builtins) so that alternatives won't confuse and complicate things

Wearing your customer-centricity hat, what is the optimal set of
personas to unperson so that you can implement a solution that works
for everyone who still matters?

[Wookey]

Post by Clint Adams

Post by Wookey
I must admit that I have no idea why replacing such a longstanding
utility is deemed necessary.

Maybe this riddle will help.
Imagine that you are the product manager for Debian `which`. According
to the hatemail in my inbox, this is the most important utility in the
history of time,

:-)

Post by Clint Adams
such that even printing a warning to stderr causes
global devastation, block hints, and calls for impeachment.

I do agree that it's pretty shoddy of bazel to break because a utility
several layers down issues stderr text whilst still returning a
success code. And yes I/someone should probably try to actually work
out why and fix it... But there is a lot of shit code in the world,
and only so many hours.

Post by Clint Adams
You go on a Gemba walk,

Never heard of a Gemba walk before, so I learned something today :-)

Post by Clint Adams
Wearing your customer-centricity hat, what is the optimal set of
personas to unperson so that you can implement a solution that works
for everyone who still matters?

I'm not fussy. Any of a,b,c or d would suit me fine. It's just e
(issue deprecation message), which caused me trouble.

Thanks for explaining what the actual problem is. You have my sympathy
trying to navigate the mess.

Wookey

[Ben Hutchings]

Post by Andrey Rahmatullin

Post by Gunnar Wolf
I agree with you, maintainer scripts should not rely on 'which'
anymore. However, what about users? 'which' is a standard Unix tool
since forever, and I expect many users to experience head scratching
when told it's not cool to use it anymore.

Why is a standard Unix tool since forever shipped by a package called
debianutils, that has a description saying "Miscellaneous utilities
specific to Debian" and "used primarily by the installation scripts of
Debian packages"? Surely a standard Unix tool would, or at least could, be
shipped somewhere else?

3BSD first implemented 'which':
<https://minnie.tuhs.org/cgi-bin/utree.pl?file=3BSD/usr/man/man1/which.1>
(1979). This was csh-specific and I think it later became built-into
csh.

FreeBSD 2.1 introduced a new non-built-in implementation:
<https://www.freebsd.org/cgi/man.cgi?query=which&apropos=0&sektion=0&manpath=FreeBSD+2.1.0-RELEASE&arch=default&format=html>
(1995). This was written in Perl and then rewritten in C a few years
later.

Debian's implementation started out in 1995 or 1996 as a shell script
calling 'type', and remains a shell script.

Red Hat has yet another implementation:
<https://www.freebsd.org/cgi/man.cgi?query=which&apropos=0&sektion=0&manpath=CentOS+7.1&arch=default&format=html>,
<https://carlowood.github.io/which/> (1999 or earlier). This is
written in C.

So 'which' has a very long history in csh, which was the default
interactive shell in many versions of Unix. Its availability to other
shells dates back to the 90s (at least) but is not portable due to
there being multiple very different implementations.

Ben.

[Colin Watson]

Debian's implementation started out in 1995 or 1996 as a shell script
calling 'type', and remains a shell script.

Not very important historical note: it's true that Debian had a "which"
command from 1995/1996 or so, but the current implementation doesn't
descend from that at all.

I wrote the earliest version of the current implementation from scratch,
as part of a job I held from 2000 to 2003; in that job, I worked on lots
of different Unix flavours, and we had NFS-mounted home directories so I
wanted a reasonably cross-platform ~/.bashrc. The spectacular lack of
consistent behaviour of "which" across those platforms got in my way, so
I wrote a shell script that I could put in my ~/bin and use everywhere.
I don't remember exactly when I wrote it, but it can't have been before
2000 and I think it was probably around 2002.

I contributed that script to Debian in 2002 in response to
https://bugs.debian.org/94507, where it became clear that the previous
implementation wasn't really salvageable; Clint merged that in
debianutils 1.16.5. It's been extended since then, but still has the
same basic approach.