Removing manpages from libpam-modules to improve multi-arch

[Sam Hartman]

TL;DR: I propose move man pages out of a multi-arch: same package into a
arch all package. Asking for any downsides and usrmerge review.

I'm in the process of packaging pam 1.7.0. Upstream has moved from
autotools to meson and in the process has streamlined their release
tarballs to remove all or almost all build artifacts such as generated
man pages.

Today, we include man pages for all the pam modules in libpam-modules.
Libpam-modules is multi-arch: Same, so all these man pages need to be
bit-for-bit identical across all architectures.
Originally that was easy: we just included unmodified upstream man
pages.
But various Debian patches change the man pages.
For a while we just built the man pages but if any of the docbook tools
changed between one arch build and another, we'd end up with m-a
uninstallable packages.

So, we ended up including patches to the man pages as well as their
source.
This was a real pain to maintain under the old build system, but it's
even more annoying now.

My proposal is to move the man pages into libpam-doc.
I'm not actually convinced that normal Debian users need man pages for
all the pam modules on all Debian systems, and a suggests relationship
should be sufficient.
If people really want to maintain the current level of man page
presence, we could move the manpages into libpam-modules-bin which is
M-A: foreign.

I think there are no usrmerge implications. While libpam-modules did
move files from /lib/multiarch_tripple/security to
/usr/lib/multiarch_tripple/security, the man pages have always been in
/usr/share/man, which lives on /usr.

Thoughts on this proposed action?

[Marvin Renich]

Post by Sam Hartman
TL;DR: I propose move man pages out of a multi-arch: same package into a
arch all package. Asking for any downsides and usrmerge review.
My proposal is to move the man pages into libpam-doc.
I'm not actually convinced that normal Debian users need man pages for
all the pam modules on all Debian systems, and a suggests relationship
should be sufficient.
If people really want to maintain the current level of man page
presence, we could move the manpages into libpam-modules-bin which is
M-A: foreign.

I have on a number of occasions used these man pages, and having them
installed locally is very helpful. I would rather have the man pages
installed without the additional documentation in libpam-doc. Why not
(other than a trip through NEW) put them in a new binary package
libpam-manpages (arch:all)? I would prefer recommends rather than
suggests.

...Marvin

[Sam Hartman]

Marvin> I have on a number of occasions used these man pages, and
Marvin> having them installed locally is very helpful. I would
Marvin> rather have the man pages installed without the additional
Marvin> documentation in libpam-doc. Why not (other than a trip
Marvin> through NEW) put them in a new binary package
Marvin> libpam-manpages (arch:all)? I would prefer recommends
Marvin> rather than suggests.

Do you actually have a system on which you want these man pages and on
which the extra space of libpam-doc would be a problem?

Unless there's a compelling need, my answer is that I don't understand
why manpages should be separated from other documentation in this instance.

--Sam

[G. Branden Robinson]

Post by Sam Hartman
Marvin> I have on a number of occasions used these man pages, and
Marvin> having them installed locally is very helpful. I would
Marvin> rather have the man pages installed without the additional
Marvin> documentation in libpam-doc. Why not (other than a trip
Marvin> through NEW) put them in a new binary package
Marvin> libpam-manpages (arch:all)? I would prefer recommends
Marvin> rather than suggests.
Do you actually have a system on which you want these man pages and on
which the extra space of libpam-doc would be a problem?
Unless there's a compelling need, my answer is that I don't understand
why manpages should be separated from other documentation in this instance.

Don't we have dpkg filters for this sort of use case?

dpkg(1):
--path-exclude=glob‐pattern
--path-include=glob‐pattern
Set glob‐pattern as a path filter, either by excluding or re‐
including previously excluded paths matching the specified
patterns during install (since dpkg 1.15.8).
...
This can be used to remove all paths except some particular
ones; a typical case is:

--path-exclude=/usr/share/doc/*
--path-include=/usr/share/doc/*/copyright
...

Are we missing some mechanism that would integrate this well with apt?
Possibly some curated set of "profiles" for common applications, like
localization file stripping?[1]

Elimination of those and man pages have been considered low-hanging
fruit by producers of "minimal" variants of Debian for about as long as
there have been such things _as_ variants of Debian.

Regards,
Branden

[1] I looked at the following page but the approaches listed seemed
mostly ad hoc, atomistic, and crude.

https://wiki.debian.org/ReduceDebian

[Sam Hartman]

G> At 2025-01-15T12:45:22-0700, Sam Hartman wrote:
Marvin> I have on a number of occasions used these man pages, and
Marvin> having them installed locally is very helpful. I would
Marvin> rather have the man pages installed without the additional
Marvin> documentation in libpam-doc. Why not (other than a trip
Marvin> through NEW) put them in a new binary package
Marvin> libpam-manpages (arch:all)? I would prefer recommends
Marvin> rather than suggests.

Post by Sam Hartman
Do you actually have a system on which you want these man pages
and on which the extra space of libpam-doc would be a problem?
Unless there's a compelling need, my answer is that I don't
understand why manpages should be separated from other
documentation in this instance.

G> Don't we have dpkg filters for this sort of use case?

I honestly can't tell from your message which position you are
supporting, which I do find somewhat frustrating when I'm trying to get
feedback to make a change.

I think that yes, because we have dpkg filters, there's not a compelling
argument to separate the man pages from the rest of the docs.

I think that given the multi-arch issues it still makes sense to
separate the man pages from the m-a: same binaries.
Were it not for multi-arch, I could see an argument that dpkg filters
were sufficient and the man pages could stay in libpam-modules.
But I could also see an argument for minimizing the essential set even
if someone does not use dpkg filters.

[G. Branden Robinson]

Post by Sam Hartman
G> Don't we have dpkg filters for this sort of use case?
I honestly can't tell from your message which position you are
supporting, which I do find somewhat frustrating when I'm trying to
get feedback to make a change.

I'm sorry to frustrate you--the reason you can't discern the direction
of my advocacy on this issue is that I don't have one!

I've been out of practice at Debian packaging for so long that I feel I
now know little, except for that I'd need to (1) spend long hours
perusing the Debian Policy Manual and (2) have an actual need to be
uploading packages on a regular basis to keep my skill current.

Post by Sam Hartman
I think that yes, because we have dpkg filters, there's not a
compelling argument to separate the man pages from the rest of the
docs.

One thing I didn't say out loud (or type in) is that I'm dubious of the
idea of adding more control fields. Doing that "feels" like an awfully
big hammer for this sort of problem. Having a "Documentation:" control
field as a sort of "Suggests:-for-certain-audiences" seems like it adds
another axis to the linear space of rank 1 upon which we organize
"Depends/Recommends/Suggests".

I seem to remember that Enrico Zini brought us, 20 years ago, a tag
system to attack that exact sort of problem, to keep control fields from
proliferating out of control. I also seem to remember that much more
recently he assessed that initiative as a failure. Even if it was,
that's not a reason to make the mistake he tried to prevent.

Post by Sam Hartman
I think that given the multi-arch issues it still makes sense to
separate the man pages from the m-a: same binaries.

Yes, I think that's a separate concern and much easier to defend.
Corralling architecture-independent data into their own binary packages
pays dividends in a few respects: less archive space consumed and easier
bootstrapping of new architectures (or just ABIs) are a couple that
spring to mind.

Post by Sam Hartman
Were it not for multi-arch, I could see an argument that dpkg filters
were sufficient and the man pages could stay in libpam-modules.

As someone who has a bit of a preoccupation with man pages, I feel
strongly that they should accompany the components of the system they
document. But "accompany" is a term that affords several
interpretations, and this feeling doesn't override my previous
paragraph.

Post by Sam Hartman
But I could also see an argument for minimizing the essential set even
if someone does not use dpkg filters.

I think we should default to shipping documentation, and let people opt
out of receiving it by some means. So if I were LCMNDFL[1], I'd decree
that documentation packages were always Recommended by their binary
counterpart(s).

One of the reasons it's hard to tell which side I've picked, so to
speak, is that I don't feel I'm current with the space of available
solutions in Debian in 2025. I therefore ask what they are.

Regards,
Branden

[1] Leisure-Class, Malignantly Narcissistic Dictator For Life

Too tautologous?

[Jonathan Dowland]

Post by G. Branden Robinson
As someone who has a bit of a preoccupation with man pages

You've reminded me that you presented 'Write the Fine Manual' in 2005,
(possibly at DebConf?). I thought it was a really good slide-deck, but
it's either unavailable or at least hard to find now. Would you be
prepared to put it back up somewhere? I have a copy of the PDF (which I
am happy to circulate) but not the LaTeX source.

(It's possible that modern tools to generate manpages from other sources
are better than they were in 2005, but I doubt it)

[G. Branden Robinson]

Hi Jonathan,

Post by Jonathan Dowland

Post by G. Branden Robinson
As someone who has a bit of a preoccupation with man pages

You've reminded me that you presented 'Write the Fine Manual' in 2005,
(possibly at DebConf?).

I did! When I spent some time in a semi-conscious self-exile from
Debian, I didn't look at it for many years. I stumbled across it again
after getting involved with groff, and I'm pleased to say that the thing
isn't _riddled_ with embarrassments.

I would also admit that it has had zero measurable effect in improving
man page composition practices.

Post by Jonathan Dowland
I thought it was a really good slide-deck, but it's either unavailable
or at least hard to find now. Would you be prepared to put it back up
somewhere? I have a copy of the PDF (which I am happy to circulate)
but not the LaTeX source.

I had a disgraceful episode of data loss, and cannot put my hands to it,
or I'd share it. Only _almost_ happily, because I'd feel morally
obliged to:

1. Bring it up to date, and
2. Rewrite it in groff, of course!

...moments later...

Ah, the Internet Archive to the rescue!

https://web.archive.org/web/20061206200701/http://people.debian.org/~branden/talks/wtfm/wtfm.tex

Man, that page brings back memories. You'd think I'd have bothered to
acquire an M1 helmet and a canteen cup by now. No microplastics!

Post by Jonathan Dowland
(It's possible that modern tools to generate manpages from other
sources are better than they were in 2005, but I doubt it)

More exist, and they are on the whole gradually getting better.
However, improvement seems to happen _only_ when Ingo Schwarze
(mandoc(1) maintainer) or I (or, God help them, both of us) show up on
their web-based development sites to offer advice, usually in response
to someone else's issue/MR/PR.

As a rule, developers of these generation tools (a) do not subscribe to
the groff mailing list, (b) do not file bug reports against groff, and
(c) do not otherwise contact groff developers nor, as far as I can tell,
experienced *roff users, to ask questions or seek guidance.

In my assessment there exists a Big Four of man(7) generators, and one
ignominious but noteworthy also-ran.

The Big Four:

1. podlators/Pod::Man -- a proud exception to some of the
generalizations above. First, it's been around probably longer than
any of these others, maybe by a margin of decades. Second, it was
initially developed by people who seemed to have a good grasp of the
man(7) macro language and the underlying roff(7) one--in other
words, old school Unix graybeards who knew their way around troff
sufficiently to submit a USENIX paper. A dying breed. Third, the
long-time maintainer is Russ Allbery, whom I trust needs no
introduction to this list's readers; Russ has approached the
***@gnu list with questions when he's had them (a rare occurrence,
because the guy seems to generally know what he's doing--somebody
should nominate him for TC or something[1]).

Ingo Schwarze--a difficult man to impress--and I agree that
Pod::Man's output is probably best-of-breed among the Big Four.

Downsides? Perl's sex appeal as an implementation language has
fallen precipitously over the past 20 years--in fact I think I saw
some "old and busted, seeking new hotness" sentiments expressed
about it recently on this very list--and absolutely no one selects a
new programming language on the basis of how well its support
infrastructure can generate high-quality man(7) documents. Pod::Man
also has some idiosyncrasies, like a smoldering hatred of adjusted
text in paragraphs, which presents some minor difficulties when it
collides with missing features in traditional troffs.

But as a rule you have to go out of your way to produce _bad_ output
with Pod::Man, and when it _is_ bad, it's likely bad in _all_ of its
output formats. That's enough to keep irascible types like Ingo and
me from griping very much.

Example of pod2man/groff interaction:
https://lists.gnu.org/archive/html/groff/2024-03/msg00078.html

Russ is the only man(7)-generator maintainer who has presented me
with a problem I haven't figured out how to solve nicely. I trust
he knows I'll have my revenge.

2. Pandoc. Haskell developers are seldom short on ambition and that
tendency shows up here. It's also an interesting case in that
its mission seems not to be as restricted to support infrastructure
for a programming language being promoted as the others are. It
also tries harder to be a _bidirectional_ converter than our other
specimens. If anyone can successfully come up with a metalanguage
to characterize every documentation markup language in the world,
it'll be a bunch of type theorists.

Example of pandoc/groff/mandoc interaction:
https://github.com/jgm/pandoc/issues/9020

3. Docutils. Python's documentation generator.

Example of docutils/groff interaction:
https://sourceforge.net/p/docutils/patches/205/

4. Asiidoctor. Ruby's documentation generator.

Example of asciidoctor/groff interaction:
https://github.com/asciidoctor/asciidoctor/issues/4430

I've saved the worst for last.

That is of course docbook-to-man. Ingo and I both find the quality of
its output to be execrable. It has been unmaintained for many years and
consistently seems to burn out and drive from FLOSS development anyone
who has gotten involved with it. Its brownfield status appears to be
widely if vaguely understood, and it is avoided--but only by its
would-be _maintainers_--like a toxic waste dump piled 100 meters deep
with burning tires, at the center of which a column of blue Cerenkov
light stabs the sky like a beacon of death.

It is consequently extremely popular and has enjoyed wide adoption, and
continues to be thought of as a viable tool even as the luster of XML
doctypes and DocBook in particular has worn off. Notoriously, all the
Git man pages are produced via this means. This is entirely in keeping
with prototypical Linux kernel developer behavior: identify a good idea
(like having a single master documentation format from which others are
generated), select the shittiest implementation thereof available, and
nail oneself to it for all eternity. And why change?

Yo, kernel hackers got better things to do than writing docs, bro. One
doesn't achieve world domination by telling other people how to do it.

Regards,
Branden

[1] Yes, I know.

[Russ Allbery]

Post by G. Branden Robinson
1. podlators/Pod::Man -- a proud exception to some of the
generalizations above. First, it's been around probably longer than
any of these others, maybe by a margin of decades. Second, it was
initially developed by people who seemed to have a good grasp of the
man(7) macro language and the underlying roff(7) one--in other
words, old school Unix graybeards who knew their way around troff
sufficiently to submit a USENIX paper. A dying breed. Third, the
long-time maintainer is Russ Allbery, whom I trust needs no
introduction to this list's readers; Russ has approached the
because the guy seems to generally know what he's doing--somebody
should nominate him for TC or something[1]).

In case anyone is curious about the history:

Back in the Perl 4 days, Larry Wall used to embed documentation in his
Perl scripts by putting carefully-chosen characters at the start of the
file so that it became both a valid Perl script and a valid roff document.
When I first learned UNIX properly, and Perl, and free software
development, around 1993 and 1994, it was by looking at Perl 4 scripts,
both those written by Larry Wall and by colleagues of mine that used the
same technique. That's what got me started on the design pattern of
embedding the documentation for a script in the script itself, which to
this day I still consider the killer feature of POD (you can never be
without the documentation of a Perl script), and I'm always sad that so
few other languages have adopted it.

When Perl 5 came along, and POD came along with it, Tom Christiansen wrote
a script called pod2man. Tom knew roff extremely well, for good or for
ill; the incredibly complex macros that attempted to manually format
various non-US-ASCII characters for troff output have provoked many
maintenance challenges over the years, although I think he largely cribbed
them from elsewhere. This was before I got involved in the Perl community
heavily. Most of the original design decisions are his.

I sadly forget what made me decide that I really wanted to work on POD
formatting in 1999, although for some years I'd maintained Stanford's TeX
installation and was generally interested in document formatting. I
started with Pod::Text, I think because I didn't like something about the
output of the Pod::Text that came with Perl. Shortly thereafter (and my
recollection was that it was at Tom's prompting), I took on pod2man as
well and rewrote it as a Perl module instead of a standalone script.

At the time that I started maintaining it, I did not know roff well. I had
written a few man pages directly in roff, but I always used LaTeX for
writing longer documents. (I was at Stanford University, after all, so
hopefully the roff partisans will forgive me.) [1] When I picked up
Pod::Man maintenance, I had no idea what I was doing, so I sat down and
read CSTR 54 (the AT&T Bell Labs troff user manual), and then I re-read it
multiple times while reading the pod2man source code until I (thought I)
understood every construction that was used in its output.

I still don't consider myself a roff expert of any kind, since I never
write roff directly, I only write Perl code to write roff. :) If it's not
in CSTR 54, I probably don't know it, and I usually have at least a few
dumb misunderstandings when any roff issue comes up. I try to make up for
it by asking questions and being extremely conservative about changes, in
the grounds that if people have been using a program for 25 years and very
rarely report bugs to me, that probably means it's mostly working and I
should try not to break it. But periodically some bug crops up, and I
usually go back and read CSTR 54 to try to figure out how to solve it.

Recently I've made quite a few major changes to try to update Pod::Man for
a world in which the correct solution to many problems it was attempting
to solve via awkward workarounds, such as trying to generate nice paired
quotes, is now "use Unicode if you want Unicode." So it now supports
Unicode input and output and I have deleted a lot of the pre-Unicode
troff-specific workarounds. Those changes largely made it into Perl 5.38
and later.

Any history would not be complete without mentioning Sean Burke, who
completely rewrote the POD parsing machinery for Perl and adapted both
Pod::Text and Pod::Man to the new parsing structure, which was an immense
amount of work that I am very grateful for not having to do.

I should probably note that while I was subscribed to the groff mailing
list for a while, I dropped off again in an attempt to reduce the amount
of stuff I was trying to keep track of. Absolutely nothing against the
great people on the list, just my own regrettable lack of time and
continued state of being behind on most non-work projects. But I will
certainly hop back on if something related to Pod::Man comes up.

[1] A very minor bit of personal trivia: Due to a kind of hilarious
combination of my dad's career path, my fascination with computers,
and my family's love of hobby projects, I wrote several middle school
papers in VAX DIGITAL Standard Runoff and in fact still have the
original source. DSR was one of the inspirations for roff, although
they don't have a lot common in terms of syntax other than the leading
period. roff felt very familiar when I encountered it some time later.

[Theodore Ts'o]

Post by G. Branden Robinson
I've saved the worst for last.
That is of course docbook-to-man. Ingo and I both find the quality of
its output to be execrable. It has been unmaintained for many years and
consistently seems to burn out and drive from FLOSS development anyone
who has gotten involved with it. Its brownfield status appears to be
widely if vaguely understood, and it is avoided--but only by its
would-be _maintainers_--like a toxic waste dump piled 100 meters deep
with burning tires, at the center of which a column of blue Cerenkov
light stabs the sky like a beacon of death.
It is consequently extremely popular and has enjoyed wide adoption, and
continues to be thought of as a viable tool even as the luster of XML
doctypes and DocBook in particular has worn off. Notoriously, all the
Git man pages are produced via this means. This is entirely in keeping
with prototypical Linux kernel developer behavior: identify a good idea
(like having a single master documentation format from which others are
generated), select the shittiest implementation thereof available, and
nail oneself to it for all eternity. And why change?
Yo, kernel hackers got better things to do than writing docs, bro. One
doesn't achieve world domination by telling other people how to do it.

Interestingly, in 2016 the Linux kernel switched from using DocBook to
reStructedText[1]. So the flaws of ASCII doc were relatively well
known as far back as eight years ago by the kernel community.

[1] https://lwn.net/Articles/692704/

The problem seems to be that Sphinx is really not great for creating
man pages, and very few people actually do roff these days. So the
man pages in the kernel (for example, for the perf tool) and the git
project are still implemented in AsciiDoc. That's probably because
it's good enough for the users of docbook-to-man, even if it's a
brownfield.

(For example, X11 still works better for me because Wayland still has
inconsistent support for cursor sizing across applications, sigh.

Post by G. Branden Robinson
From windowing system and desktop developers' perspective, most

probably wish X11 would die already, but users have these pesky
requirements, like not having their cursor effectively disappear when
it wanders into the wrong window if you are using a HiDPI display (at
least, if you are using KDE; I don't know if GNOME is any better at
this point, but GNOME has its own problems about listening to power
users' requirements...)

I'm sure that if someone wanted to help those consumers of DocBook to
switch to something better, by *listening* to their requirements and
then trying to meet them, and showing them how to do an automated job
of converting of their man pages from one source format to another,
most projects would be willing to switch. But that's a huge amount of
work for any volunteer or volunteer community to take on, whether it's
the git development community or some documentation tool community.

- Ted

[Marvin Renich]

[Please don't CC me]

Post by Sam Hartman
Do you actually have a system on which you want these man pages and on
which the extra space of libpam-doc would be a problem?

No.

Post by Sam Hartman
Unless there's a compelling need, my answer is that I don't understand
why manpages should be separated from other documentation in this instance.

I have a strong aversion to the mantras "disk space is cheap", "memory
is cheap", "network bandwidth is cheap", or any other denigration of
seemingly abundant resources. They are the enemy of good programming
principles.

In this particular case, I don't think putting the man pages in with the
other docs is a problem. But if every package did this, the bloat would
be significant. I think it sets a bad precedent.

Post by Sam Hartman
I'm not actually convinced that normal Debian users need man pages for
all the pam modules on all Debian systems, and a suggests relationship
should be sufficient.

I think you underestimate the amount these man pages are used and their
value.

The suggestion to create a new libpam-manpages was really just one way
to keep the status quo without bringing in the additional doc'n. With a
Recommends relationship, it would allow installing libpam-modules
without the man pages, if desired, but would include them by default
(except where the user had opted out of installing Recommends by
default).

I am fine with putting the man pages in with libpam-modules-bin.

...Marvin

[Russ Allbery]

My proposal is to move the man pages into libpam-doc. I'm not actually
convinced that normal Debian users need man pages for all the pam
modules on all Debian systems, and a suggests relationship should be
sufficient. If people really want to maintain the current level of man
page presence, we could move the manpages into libpam-modules-bin which
is M-A: foreign.

I am in favor of moving the man pages out of the multiarch library package
and into either libpam-doc or libpam-modules-bin. I don't have any strong
opinions between that choice of packages.

Putting anything other than the minimum required and ideally all
multiarch-aware files into the packages intended to be coinstalled in a
multiarch situation is asking for trouble, IMO. Those packages should be
as minimal as possible to deliver their functionality, for exactly the
reasons you describe.

[Simon Richter]

Hi,

Post by Sam Hartman
For a while we just built the man pages but if any of the docbook tools
changed between one arch build and another, we'd end up with m-a
uninstallable packages.

Can this be fixed by removing the "Generator:" comment in the generated
manpage, and possibly clamping the included date to SOURCE_DATE_EPOCH?

Simon

[Russ Allbery]

Post by Simon Richter

Post by Sam Hartman
For a while we just built the man pages but if any of the docbook tools
changed between one arch build and another, we'd end up with m-a
uninstallable packages.

Can this be fixed by removing the "Generator:" comment in the generated
manpage, and possibly clamping the included date to SOURCE_DATE_EPOCH?

There are various things one can do to try to make the output of a man
page generator like that more consistent, but they don't fix the problem,
just reduce its frequency, unless Debian sets up to do a fully
reproducible build with pinned versions of everything (which I don't think
we want to do).

Otherwise, the problem is that the documentation generation tools may have
changed versions, and therefore changed output in a way that's not really
avoidable, between the time the source package was uploaded and built and
the time a binNMU was scheduled for one of the architectures.

[Simon Richter]

Hi,

Post by Russ Allbery
There are various things one can do to try to make the output of a man
page generator like that more consistent, but they don't fix the problem,
just reduce its frequency, unless Debian sets up to do a fully
reproducible build with pinned versions of everything (which I don't think
we want to do).

Agreed, it's not a complete fix, but I'd expect the frequency of changes
in the output besides the version number to be low enough for this to be
the least-effort solution.

If it means we need to trigger a rebuild of a few packages every few
years, then this thread has already used more time.

Simon

[Bálint Réczey]

Hi,

Post by Simon Richter
Hi,

Post by Russ Allbery
There are various things one can do to try to make the output of a man
page generator like that more consistent, but they don't fix the problem,
just reduce its frequency, unless Debian sets up to do a fully
reproducible build with pinned versions of everything (which I don't

think

Post by Russ Allbery
we want to do).

Agreed, it's not a complete fix, but I'd expect the frequency of changes
in the output besides the version number to be low enough for this to be
the least-effort solution.
If it means we need to trigger a rebuild of a few packages every few
years, then this thread has already used more time.

I agree. It is very easy to detect file differences between multiarch
packages and scheduling binNMUs.

Since the described problem potentially affects all packages shipping man
pages with the binaries - which is the best practice - splitting man pages
from a single package to solve that particular problem sounds misdirected
effort.

Cheers,
Balint

Post by Simon Richter
Simon

[Sam Hartman]

Simon> Hi,

Post by Sam Hartman
For a while we just built the man pages but if any of the docbook
tools changed between one arch build and another, we'd end up
with m-a uninstallable packages.

Simon> Can this be fixed by removing the "Generator:" comment in the
Simon> generated manpage, and possibly clamping the included date to
Simon> SOURCE_DATE_EPOCH?

Patches welcome for clamping the included date if that's not already
being done by something.
I agree with Russ that I want to move the man pages out of multi-arch:
same packages, but would welcome patches to make the build more
reproducible.

You raised the issue that we want man pages to accompany their binary.
It's fairly rare for binaries in /usr/bin or /usr/sbin to be in m-a:
same packages.
I think for libraries, we want the man page to be in the -dev package,
and that's where I think we really start running into trouble.
Cross building encourages us to make those -dev packages M-A: same.
I think that I might generally agree with you that scheduling bin NMUs
when -dev packages get out of sync might be sufficient.
If you can give me clean patches to make libpam0g-dev's man pages
reproducible (or spend the effort to confirm they already are), I'll
leave the dev man pages there even though it is M-A: same.

pages

[Helmut Grohne]

Hi Sam,

Post by Sam Hartman
My proposal is to move the man pages into libpam-doc.
I'm not actually convinced that normal Debian users need man pages for
all the pam modules on all Debian systems, and a suggests relationship
should be sufficient.

I've actually used those pages more than once and appreciated their
presence by default, but I also appreciate the ability to not install
them. Sounds like I'll end up installing libpam-doc where I need them.

Post by Sam Hartman
From a package building pov, I'd appreciate if you could also move the

tools for building the manual pages to Build-Depends-Indep while you
move them.

Post by Sam Hartman
I think there are no usrmerge implications. While libpam-modules did
move files from /lib/multiarch_tripple/security to
/usr/lib/multiarch_tripple/security, the man pages have always been in
/usr/share/man, which lives on /usr.

I confirm. Their presence inside the /usr/share hierarchy makes them
fully independent of any /usr-merge or /usr-move concerns.

Helmut

[Sam Hartman]

package: pam
version: 1.5.3-1
severity: wishlist
tags: help
[talking about pam manpages]

Helmut> From a package building pov, I'd appreciate if you could
Helmut> also move the tools for building the manual pages to
Helmut> Build-Depends-Indep while you move them.

I think we still end up with a bunch of manpages in libpam0g-dev.
Although that's also M-A: same, so probably has similar issues, although
probably they come up less frequently than libpam-modules.

But let's say we move those man pages too.

I'd love to minimize build-depends-arch, but I don't know how.
In my meson setup call, I want to either enable or disable the docs
feature.
I realize I could leave it auto, but that has been too fragile in the
past as upstream changes their doc dependencies. So it is too easy to
get into a situation where upstream builds none of the docs because some
tool is not present.

But the meson setup call is in override_dh_auto_configure.
I don't know at that point how to figure out of I am building arch all
packages.

If someone can either give me some hints here or provide a patch, I'd
love to review it.

[Simon McVittie]

Post by Sam Hartman
But the meson setup call is in override_dh_auto_configure.
I don't know at that point how to figure out of I am building arch all
packages.

I find that it's often better to do this in terms of "am I building
package X?" instead of "am I building arch: all packages?" because that
way, you will also get the correct answer when dealing with build-profiles.

We do this a lot in GNOME and GNOME-adjacent packages. Something like this
(in this case taken from gobject-introspection, and enabling/disabling
the gtk_doc option depending on whether we are building the -doc package):

built_binaries := $(shell dh_listpackages)

configure_options = ... options you use unconditionally ...

ifeq ($(filter %-doc,$(built_binaries)),)
configure_options += -Dgtk_doc=false
else
configure_options += -Dgtk_doc=true
endif

override_dh_auto_configure:
dh_auto_configure -- $(configure_options)

[Sam Hartman]

But the meson setup call is in override_dh_auto_configure. I
don't know at that point how to figure out of I am building arch
all packages.

Simon> I find that it's often better to do this in terms of "am I
Simon> building package X?" instead of "am I building arch: all
Simon> packages?" because that way, you will also get the correct
Simon> answer when dealing with build-profiles.

Simon> We do this a lot in GNOME and GNOME-adjacent
Simon> packages. Something like this (in this case taken from
Simon> gobject-introspection, and enabling/disabling the gtk_doc
Simon> option depending on whether we are building the -doc
Simon> package):

Simon> built_binaries := $(shell dh_listpackages)

Okay, so dh_listpackages is smart enough to know what kind of build it
is in even when called from override_dh_auto_configure?
I didn't know that, and yes it makes this easy.

I also had somehow missed the Build-Profiles binary package control
field.
To confirm, it would be reasonable to mark libpam-doc with

Build-Profiles: <!nodoc>

[Niels Thykier]

Post by Sam Hartman

But the meson setup call is in override_dh_auto_configure. I
don't know at that point how to figure out of I am building arch
all packages.

Simon> I find that it's often better to do this in terms of "am I
Simon> building package X?" instead of "am I building arch: all
Simon> packages?" because that way, you will also get the correct
Simon> answer when dealing with build-profiles.
Simon> We do this a lot in GNOME and GNOME-adjacent
Simon> packages. Something like this (in this case taken from
Simon> gobject-introspection, and enabling/disabling the gtk_doc
Simon> option depending on whether we are building the -doc
Simon> built_binaries := $(shell dh_listpackages)
Okay, so dh_listpackages is smart enough to know what kind of build it
is in even when called from override_dh_auto_configure?
I didn't know that, and yes it makes this easy.
I also had somehow missed the Build-Profiles binary package control
field.
To confirm, it would be reasonable to mark libpam-doc with
Build-Profiles: <!nodoc>

With some limitations, yes. The limitations for `dh_listpackages` are
not applicable to this case, but it is a "here by dragons" area that can
be confusing when you accidentally run into the dragon's nest. This mail
is mostly to serve as a "caveat" for other participants, since the
concrete case is fine.


The caveat:

When used **inside** an override or hook target as in this case, Yes.
When used **outside** of a override or hook target, you might end up
with surprising result.

See "Caveats with hook targets and makefile conditionals" from "man dh",
which applies equally to variables used outside hook targets.

The implementation details is that `dh` exposes an internal ENV variable
that any `Debian::Debhelper::Dh_Lib` based `dh_*` command reacts to that
makes it pick up the `-a` or `-i` implied by `build-arch` or
`build-indep` that triggered the build for `dh`.

However, `debian/rules` is run once **outside** a hook target (when `dh`
scans for hook targets) in dry-run mode. During this parsing,
`dh_listpackages` does not know which kind of build it is (`-b`, `-A`,
or `-B`) and it therefore assumes a `-b` build listing all binary
packages not excluded by Build-Profiles. The conditional must not cause
any harm like shadowing a hook target during this dry-run, because then
`dh` does not see the hook target and you will end up very confused by
why `debhelper` is ignoring your override.

As said in the beginning, the example Simon gave does not have this
problem, so that example should be fine. But be careful with
`dh_listpackages` based conditionals outside override/hook targets. it
is probably not what you want to spend your Debian time debugging. :)

Best regards,
Niels

[Guillem Jover]

Hi!

Post by Sam Hartman
My proposal is to move the man pages into libpam-doc.
I'm not actually convinced that normal Debian users need man pages for
all the pam modules on all Debian systems, and a suggests relationship
should be sufficient.
If people really want to maintain the current level of man page
presence, we could move the manpages into libpam-modules-bin which is
M-A: foreign.

ISTM that moving them to libpam-modules-bin would be the better
path forward, as it would not regress with missing man pages. I
think having no man pages by default when installing programs,
config or other such content, that previously had them would be
rather unexpected (I certainly miss them when packages have no man
pages at all or even provide no man pages by default). I don't think
size should be considered an issue here given that the man pages were
already shipped (and we expect them to be installed as per policy),
and as mentioned in the thread people can filter them out if desired.

(Perhaps if you are shuffling files around you could also consider
whether moving /etc/security and /usr/share/pam-configs into
libpam-modules-bin as well also makes sense, to avoid potentially
weird semantics with refcounted conffiles and M-A:same? Not a blocker
though, just a thought, while checking the contents.)

Thanks,
Guillem

[Sam Hartman]

Guillem> Hi!

My proposal is to move the man pages into libpam-doc. I'm not
actually convinced that normal Debian users need man pages for
all the pam modules on all Debian systems, and a suggests
relationship should be sufficient. If people really want to
maintain the current level of man page presence, we could move
the manpages into libpam-modules-bin which is M-A: foreign.

Guillem> ISTM that moving them to libpam-modules-bin would be the
Guillem> better path forward, as it would not regress with missing
Guillem> man pages. I think having no man pages by default when
Guillem> installing programs, config or other such content, that
Guillem> previously had them would be rather unexpected (I certainly
Guillem> miss them when packages have no man pages at all or even
Guillem> provide no man pages by default). I don't think size should
Guillem> be considered an issue here given that the man pages were
Guillem> already shipped (and we expect them to be installed as per
Guillem> policy), and as mentioned in the thread people can filter
Guillem> them out if desired.

Helmut has argued for moving the docs into arch: all packages to
hopefully be able to reduce arch all build dependencies.
That sounds like a good argument to me.
(We'd also need to do something about libpam0g-dev man pages).

Guillem> (Perhaps if you are shuffling files around you could also
Guillem> consider whether moving /etc/security and
Guillem> /usr/share/pam-configs into libpam-modules-bin as well also
Guillem> makes sense, to avoid potentially weird semantics with
Guillem> refcounted conffiles and M-A:same? Not a blocker though,
Guillem> just a thought, while checking the contents.)

Will moving conffiles around like /etc/security just work even if they
are modified, or will that trigger a bunch of what to do about modified
conffile at install time warnings?

[Simon McVittie]

Post by Sam Hartman
(We'd also need to do something about libpam0g-dev man pages).

Moving user-facing documentation from libpam0g into either
libpam-modules-bin or libpam-doc (depending how often you expect users to
need it), and developer documentation from libpam0g-dev into libpam-doc,
seems like it would make sense to me?

API reference material for C libraries is often in a -doc package, whether
its format is HTML, PDF or man pages.

smcv

[Sam Hartman]

Post by Sam Hartman
(We'd also need to do something about libpam0g-dev man pages).

Simon> Moving user-facing documentation from libpam0g into either
Simon> libpam-modules-bin or libpam-doc (depending how often you
Simon> expect users to need it), and developer documentation from
Simon> libpam0g-dev into libpam-doc, seems like it would make sense
Simon> to me?

I got enough requests from people who want the sysadmin-facing
documentation installed by default (including a mild preference from
Helmut who I generally align with on issues like this).

I got a request from Helmut to minimize build dependencies.

With the exception of Simon Richter, we appear to be agreed that
avoiding man pages in m-a: same packages is good.

So what I'm going to do is move the developer man pages into libpam-doc
and the sysadmin man pages into libpam-runtime.
I'll be open to requests from people who want to minimize essential set
to move the sysadmin man pages to libpam-doc, but so far I'm not seeing
that.

I think this means:

* build-arch will not need the doc tools

* nodoc alone isn't enough to avoid the doc tools because of
libpam-runtime

* bootstrapping can reuse libpam-runtime so pam at least won't bring
the xml docbook stack into bootstrapping essential.

* I will recommend libpam-doc from libpam0g-dev

[Simon Richter]

Hi,

Post by Sam Hartman
With the exception of Simon Richter, we appear to be agreed that
avoiding man pages in m-a: same packages is good.

I mean, this is specifically about the manpages included in
libpam-modules, which are at the intersection of

- likely to be useful when the full -doc package is not installed
- in a m-a:same package
- built with a tool that insists on inserting a comment with its own
version number into output files

So generalizing this to a rule to not ship manpages in m-a:same packages
is overly broad -- basically we'd have to give every -dev package
containing manual pages a -doc package, which is not really economical,
especially when it's only a problem for generated manpages, and the
differences between versions of the file are in comment lines.

For libpam-modules itself, because there is a -runtime package already,
moving the manpages there is a good solution, but it's not something I
believe should become a general rule, because there absolutely are use
cases for user facing documentation shipped with library packages.

Simon

[Simon McVittie]

basically we'd have to give every -dev package containing
manual pages a -doc package

In many cases I think this is best-practice anyway, because it takes
the documentation generation toolchain out of the critical path for
bootstrapping, cross-compiling, and slow architectures. Many libraries
have their API reference as HTML or even PDF, generated via something
like Doxygen, gtk-doc or Pandoc, and kicking out that documentation into
a separate package significantly reduces what needs to be built to get
a minimal version of the library that is enough to continue to build
dependent packages. For existing libraries that have not done this,
I can see the argument for not introducing a -doc package to avoid NEW,
but when packaging a new library or doing a SONAME bump I would nearly
always split out the API reference into a -doc package.

The nodoc build option and build profile go some way towards resolving
this, but need to be invoked explicitly every time and alter the
contents of built binary packages (not reproducible), whereas a separate
Architecture: all documentation package "just works" and is easy to
acceptance-test (if it builds successfully on the official buildds and
has the desired contents, then you have succeeded).

I can see that if the library of interest only has man pages (no HTML)
and they're hand-written *roff (not generated from a more author-friendly
source format with some tool) this might be considered unnecessary,
because in that case there is no documentation toolchain, but that's
uncommon in the ecosystems I usually work with.

Similarly if a library has manual or automated as-installed tests, I've
taken to always splitting them into an -examples or -tests package to
avoid multiarch issues (and if it doesn't have even a manual smoke-test,
I've started considering that to be an upstream bug that should be resolved
before packaging, because that's how we get packages that nobody wants to
NMU or team-upload because we don't know how to test them).

smcv

[Jonas Smedegaard]

Quoting Simon McVittie (2025-01-17 10:19:47)

Many libraries have their API reference as HTML or even PDF, generated
via something like Doxygen, gtk-doc or Pandoc,

Those packages currently using pandoc are recommended to check if either
of the much lighter cmark or cmark-gfm is sufficient.

- Jonas

[Ben Collins]

Post by Sam Hartman
My proposal is to move the man pages into libpam-doc.
I'm not actually convinced that normal Debian users need man pages for
all the pam modules on all Debian systems, and a suggests relationship
should be sufficient.

Unrelated to the question at hand, but I was wondering: why doesn't
d/control contain a "Documentation:" field?

Is there a proposal on this field at all? I ask because I'm interested
in knowning more about what you're suggesting. I can think of some syntax
that would be helpful. One that comes to mind is:

Package: libfoo-dev
Documentation: any(foo), rfc(2549)

Package: libfoo-doc
Provides-Documentation: doc-base(foo), manpages(foo)

It just seems to me that documentation would be topical and media-
grouped rather than a package naming paradigm. The RFC type could be
expanded to https://www.rfc-editor.org/rfc/rfc{ID}, for example.