Packages with a history of security issues and whose packaged version is not up to date

[Santiago Ruano Rincón]

Dear Debian fellows,

I am writing this email under the hypothesis that having the latest (or
longest supported) upstream version in the next release will: 1. make it
easier to provide security support during the whole release lifecycle,
and 2. it will be useful for users, as they could have the latest
features and bugfixes, in case of minor point release updates.

Here attached you can find a list of packages that have ever had a
security issue **and** whose packaged version is not "up to date",
according to the uscan results. It is sorted by the number of currently
open CVEs in sid (the first "column"), and by the number of security
issues ever (second "column").

So, this is a call for comments: is this kind of package list useful?
I'd say that the CVEs open in sid are not critical nor have a
high-severity, but it would be nice to have them fixed, as soon as
possible. If having this list available somewhere is a good idea, could
it be "integrated" into UDD somehow? As a cgi-bin that outputs a json
file?

This is also a call for action/help proposal*: I would like to invite
the related maintainers and teams to evaluate if it is worth it to
package the latest upstream version of the listed packages, and try to
make it for trixie. I know that the time is really short, and this kind
of call could be improved and made it earlier for the next releases.

* I am personally willing to work on a subset of the listed packages.

I also know there could be "false positives", such as nginx or openjdk*,
but please take this as a draft list (and a draft script).

Attached you can find the script that produces this list.

Any thoughts?

Cheers,

-- Santiago

[Jonas Smedegaard]

Hi Santiago,

Quoting Santiago Ruano Rincón (2025-02-13 20:21:10)

Post by Santiago Ruano Rincón
Here attached you can find a list of packages that have ever had a
security issue **and** whose packaged version is not "up to date",
according to the uscan results. It is sorted by the number of currently
open CVEs in sid (the first "column"), and by the number of security
issues ever (second "column").
So, this is a call for comments: is this kind of package list useful?
I'd say that the CVEs open in sid are not critical nor have a
high-severity, but it would be nice to have them fixed, as soon as
possible. If having this list available somewhere is a good idea, could
it be "integrated" into UDD somehow? As a cgi-bin that outputs a json
file?
This is also a call for action/help proposal*: I would like to invite
the related maintainers and teams to evaluate if it is worth it to
package the latest upstream version of the listed packages, and try to
make it for trixie. I know that the time is really short, and this kind
of call could be improved and made it earlier for the next releases.

It would probably be helpful to also share the result of somehow running
the compiled list through dd-list, to raise attention for involved
maintainers.

- Jonas

[Holger Levsen]

Post by Jonas Smedegaard
Hi Santiago,
It would probably be helpful to also share the result of somehow running
the compiled list through dd-list, to raise attention for involved
maintainers.

I want to say: yes. :) please do.
--
cheers,
Holger

⢀⣎⠟⠻⢶⣊⠀
⣟⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄

Dat gifft in PlattdÌÌtschen keen Woort för „FlÃŒchtlinge”. Dat sÃŒnd allens LÌÌd,
Mischen, Kinners, Olle, Froons, MannslÌÌd, so as Du un Ick.

[Jeremy Stanley]

On 2025-02-13 16:21:10 -0300 (-0300), Santiago Ruano Rincón wrote:
[...]

Post by Santiago Ruano Rincón
So, this is a call for comments: is this kind of package list useful?

[...]

The main problem I see is that the list includes projects who
backport security fixes to stable branches, so for example
python-keystonemiddleware 10.7.1 is the latest point release on
their stable/2024.2 branch, while 10.8.0 is a development release on
their master branch. Both include the same security fixes,
therefore 10.7.1 isn't outdated even though there is a higher
version number available.

I see at least a few projects in your list I recognize as being in
the same situation, but no idea how many more there might be.

[Paul Gevers]

Hi,

Post by Santiago Ruano Rincón
Any thoughts?

You might also want to somehow take activity on the package into
account. E.g. cacti (that I am nearly the only uploader for) has seen an
update for CVE's only last week. I don't think I need (nor would I
appreciate) more naging.

How do you envision *using* this list, except having this discussion and
sharing a dd-list as suggested by Jonas? File wishlist bugs against the
packages if they don't exist yet (taking the first paragraph into
account too)?

Paul

[Jonathan Dowland]

Post by Paul Gevers
You might also want to somehow take activity on the package into
account.

Absolutely. E.g. the new OpenJDK 11 package came out a week ago. It
would be interesting to see which packages in the list have a much
larger gap, such as years.

[Santiago Ruano Rincón]

Hi,

Post by Santiago Ruano Rincón
Any thoughts?

You might also want to somehow take activity on the package into account.
E.g. cacti (that I am nearly the only uploader for) has seen an update for
CVE's only last week. I don't think I need (nor would I appreciate) more
naging.

Thanks for this comment, acknowledged.

How do you envision *using* this list, except having this discussion and
sharing a dd-list as suggested by Jonas? File wishlist bugs against the
packages if they don't exist yet (taking the first paragraph into account
too)?

I think that reporting bugs would make sense, and it is the first idea
that naturally came up to my mind. In any case, I was not thinking in
blindly running mass-bug filing.

Other than to take into account recent activity, I think it is important
to look for possible reasons for not packaging the version reported by
uscan, before filing a bug report.
Maintainers may have different methods to keep a package up-to-date,
e.g. (again) nginx, for which keeping ABI/API stable is also important.
So, I was *not* planing to file bugs for all of the packages listed
there.

To answer other comments in this thread, yes, more filtering would
enhance this list. I acknowledge that the current script and list is not
perfect.

Cheers,

-- Santiago

[Chris Hofstaedtler]

Post by Santiago Ruano Rincón
Here attached you can find a list of packages that have ever had a
security issue **and** whose packaged version is not "up to date",
according to the uscan results. It is sorted by the number of currently
open CVEs in sid (the first "column"), and by the number of security
issues ever (second "column").
So, this is a call for comments: is this kind of package list useful?

Just having the list does not add anything new. All software can
have security bugs, so this list devolves to "packages that are not
uptodate wrt to upstream".

ddpo already has my list of packages that I should be updating.

Chris

[Marc Haber]

On Fri, 14 Feb 2025 14:44:47 +0100, Chris Hofstaedtler

Post by Chris Hofstaedtler

Post by Santiago Ruano Rincón
Here attached you can find a list of packages that have ever had a
security issue **and** whose packaged version is not "up to date",
according to the uscan results. It is sorted by the number of currently
open CVEs in sid (the first "column"), and by the number of security
issues ever (second "column").
So, this is a call for comments: is this kind of package list useful?

Just having the list does not add anything new. All software can
have security bugs, so this list devolves to "packages that are not
uptodate wrt to upstream".

Especially if the list just goes the (wrong) way of so many commercial
security tools and/or consultants who just compare version numbers and
flag our stable versions as vulnerable regardless whether we have
patched vulnerabilities or not.

Just parsing version numbers is easy, it has been done hundreds of
times, and each one of those times is wrong and a waste of resources,
both of the instance who compiles and compares, and on our side to
verify the suggestion.

Greetings
Marc

[Colin Watson]

Post by Marc Haber
Especially if the list just goes the (wrong) way of so many commercial
security tools and/or consultants who just compare version numbers and
flag our stable versions as vulnerable regardless whether we have
patched vulnerabilities or not.

But it doesn't. Santiago's using the data from the security tracker to
determine whether CVEs are open.

[Marc Haber]

Post by Colin Watson

Post by Marc Haber
Especially if the list just goes the (wrong) way of so many commercial
security tools and/or consultants who just compare version numbers and
flag our stable versions as vulnerable regardless whether we have
patched vulnerabilities or not.

But it doesn't. Santiago's using the data from the security tracker to
determine whether CVEs are open.

Good. Don't we have debsecan for that? Or the security tracker itself?

Greetings
Marc

[Chris Hofstaedtler]

Post by Colin Watson

Post by Marc Haber
Especially if the list just goes the (wrong) way of so many commercial
security tools and/or consultants who just compare version numbers and
flag our stable versions as vulnerable regardless whether we have
patched vulnerabilities or not.

But it doesn't. Santiago's using the data from the security tracker to
determine whether CVEs are open.

I understood Santiago looked at all packages that ever had a
security issue reported. The two of my packages in the list would
support this interpretation.

I don't see how this is a meaningful prioritization.

Chris

[Santiago Ruano Rincón]

Post by Chris Hofstaedtler

Post by Colin Watson

Post by Marc Haber
Especially if the list just goes the (wrong) way of so many commercial
security tools and/or consultants who just compare version numbers and
flag our stable versions as vulnerable regardless whether we have
patched vulnerabilities or not.

But it doesn't. Santiago's using the data from the security tracker to
determine whether CVEs are open.

I understood Santiago looked at all packages that ever had a
security issue reported. The two of my packages in the list would
support this interpretation.
I don't see how this is a meaningful prioritization.

There are two numbers accompanying the source packages: the amount of
currently open security issues in sid, and the number of security issues
that have been present in Debian ever (as you mention).

One of my main motivations for creating this list is the *hypothesis*
that reducing the gap with upstream will make the work easier for the
people backporting patches during the Debian release life cycle.
Let's take paramiko as example (which was on the list of packages with
no open issues, but with some cve history, and because the version in
sid was 3.5.0 while upstream had released 3.5.1), and let's suppose that
during the time trixie is supported, there is a new CVE about how
paramiko handles private keys. Suppose that to fix that very
hypothetical issue it would be needed to backport/cherry-pick the
padding-related changes made in 3.5.1:
https://github.com/paramiko/paramiko/commit/3acc7906d25189b11219a7472e1532fa3ad9a1d9
If 3.5.1 is shipped in trixie, the related code wouldn't have to be
backported, so the patch would be smaller (and hopefully easier). And
the code would be exposed more time via the unstable to testing
migration than with a regular security update, which would reduce the
probability of introducing regressions.
Of course, I would expect that similar examples would make much more
sense for packages with a more important gap between sid and unstable.

If I have a limited amount of time to spend on Debian, I would like to
prioritize (thanks Colin for communicating this better than me) the
things to do where I could offer my help. I don't have any way to tell
which packages will have new security issues during trixie, so I assume
that packages with a history of security issues have more chances to
need fixes. And so, I prioritize my Debian time looking at them.

Cheers,

-- Santiago

[Paul R. Tagliamonte]

Post by Santiago Ruano Rincón
There are two numbers accompanying the source packages: the amount of
currently open security issues in sid, and the number of security issues
that have been present in Debian ever (as you mention).

I've been biting my tongue a bit here but there's an implicit third number
(which no one is able to actually compute) here: number of actual security
issues that need attention.

CVEs are not perfect. CVE count is, charitably, a proxy for how much
security attention / research it gets (hopefully that is, in turn, a proxy
for how important a package is). Not so charitably, it's perhaps a proxy
for how many people who want to build a reputation as an expert have spent
time finding something that would pass minimal scrutiny as a security issue.

There are plenty of security issues that are solved via normal bugfixes by
people who never realize the security implications of their bugfixes. In
important security sensitive places, too!

Updating to the latest upstreams is a good idea for lots of reasons, but I
don't totally understand the nexus to CVE here. Don't let me dissuade you
from doing good work here, but I reckon CVE counting is likely going to
lead to a lot of very weird non-security related biases which you may or
may not actually want.

FWIW this will solve one real problem: Lots of companies complain endlessly
and mindlessly about CVEs based on package version(s) without regards to
the issue being exploitable or even reachable (or built into the binary, in
some cases!). Closing CVEs out will no doubt make them complain less, which
sounds nice.

Paul

[Roberto C. Sánchez]

Post by Paul R. Tagliamonte
CVEs are not perfect. CVE count is, charitably, a proxy for how much
security attention / research it gets (hopefully that is, in turn, a proxy
for how important a package is). Not so charitably, it's perhaps a proxy
for how many people who want to build a reputation as an expert have spent
time finding something that would pass minimal scrutiny as a security
issue.

This is really the central point of the issue. In the instances I have
observed, we are usually talking about a *real* issue, but most often
not one that deserves to be considered a CVE, and even less deserves
some arbitrarily inflated CVSS score. Fixing the issue is good, but
dealing with all the CVE and CVSS noise is a pain.

Post by Paul R. Tagliamonte
There are plenty of security issues that are solved via normal bugfixes by
people who never realize the security implications of their bugfixes. In
important security sensitive places, too!

In theory, any abnormal program behavior has the potential to carry a
security implication. And in one project I have actually seen where
there was a push to retroactively designate CVEs for past bug fixes that
it turned out had some kind of specific security vulnerability
associated with them. It was very bizarre, and I took it as a sign that
the "everything has to have a CVE and every CVE must be fixed" mentality
is infecting more and more parts of the software development world.

Post by Paul R. Tagliamonte
Updating to the latest upstreams is a good idea for lots of reasons, but I
don't totally understand the nexus to CVE here. Don't let me dissuade you
from doing good work here, but I reckon CVE counting is likely going to
lead to a lot of very weird non-security related biases which you may or
may not actually want.
FWIW this will solve one real problem: Lots of companies complain
endlessly and mindlessly about CVEs based on package version(s) without
regards to the issue being exploitable or even reachable (or built into
the binary, in some cases!). Closing CVEs out will no doubt make them
complain less, which sounds nice.

I have seen lots of mindless complaining based on package versions, and
I agree that something like this effort is likely to reduce the
occurence of that sort of thing. And, yes, CVE is probably not a great
proxy. But Santiago has discussed this with quite a few of us on the LTS
team at various points along the way, and a better proxy hasn't been
found.

Regards,

-Robeto

[Jeremy Stanley]

On 2025-02-18 08:55:55 -0500 (-0500), Roberto C. Sánchez wrote:
[...]

Post by Roberto C. Sánchez
In theory, any abnormal program behavior has the potential to carry a
security implication. And in one project I have actually seen where
there was a push to retroactively designate CVEs for past bug fixes that
it turned out had some kind of specific security vulnerability
associated with them. It was very bizarre, and I took it as a sign that
the "everything has to have a CVE and every CVE must be fixed" mentality
is infecting more and more parts of the software development world.

[...]

Agreed. In projects where I do vulnerability management work, we've
mostly given up pushing back on things others want to file CVEs
about in our projects. Our view is that we decide what does and
doesn't get an official advisory published, and we'll request a CVE
assignment for that ourselves if one hasn't already been issued, but
if anyone else wants a CVE for something in our software they can
get one themselves and we generally don't have the time or energy to
burn on disputing those. If someone comes to us about one of those
third-party-issued CVEs not being fixed or fixes for it not getting
backported, we remind them that we don't consider it a vulnerability
and they're free to take it up with whoever requested or issued it.

[Paul R. Tagliamonte]

Post by Roberto C. Sánchez
I agree that something like this effort is likely to reduce the
occurence of that sort of thing. And, yes, CVE is probably not a great
proxy. But Santiago has discussed this with quite a few of us on the LTS
team at various points along the way, and a better proxy hasn't been
found.

Fair enough - any prioritization is better than "bump everything" - this is
at least actionable. I reckon as long as we all know (as I suspect we do at
least in this thread) this may just be a list of "important packages plus
packages someone looking to find a nail matching their hammer collection
decided was high return on effort to find a bug" it's still totally
worthwhile to do imvho.

I can't think of any situation except the exceptional where we'd want to
intentionally stay out of date with latest stable release of upstreams, so
kudos for that work!

Hopefully it also leads to the long tail of attack surface (that doesn't
know it and never had a CVE) in Debian similarly being motivated to follow
this example and update too.

Thanks for all your hard work!
Paul

[Marco d'Itri]

Post by Colin Watson
But it doesn't. Santiago's using the data from the security tracker to
determine whether CVEs are open.

And in the case of one of my own packages these CVEs have not yet been
fixed upstream, not even in an unreleased branch.

[Santiago Ruano Rincón]

Post by Marco d'Itri

Post by Colin Watson
But it doesn't. Santiago's using the data from the security tracker to
determine whether CVEs are open.

And in the case of one of my own packages these CVEs have not yet been
fixed upstream, not even in an unreleased branch.

Yes, and those are examples of the "false positive" cases I was
referring to originally. I am not aware of any way to automatically
filter them, as of today.

[Colin Watson]

Post by Chris Hofstaedtler
Just having the list does not add anything new. All software can
have security bugs, so this list devolves to "packages that are not
uptodate wrt to upstream".

I'm not sure that's quite right. It's a _prioritized_ list of packages
that are not up-to-date with upstream. When I look at team pages such
as
https://udd.debian.org/dmd/?email1=team%2Bpython%40tracker.debian.org&nosponsor1=on&email2=&email3=&packages=&ignpackages=&format=html&onlytesting=on#todo,
some kind of prioritization of which packages out of the huge pile are
worth paying more attention to than others is useful to me.

[Philip Hands]

Post by Santiago Ruano Rincón
Any thoughts?

I'm sure there are things up near the top of the list that do need a

Post by Santiago Ruano Rincón
0, 1, openqa, (4.6.1732034221.ae34b08ff -> 4.6.1739296030.77d38ef),

by the time you get down that far, it's probably just pure noise.

In the case of openqa, a) I uploaded that on Jan 23rd, and b) upstream
treats every commit to their master branch as a release, so there is
basically no way of ever satisfying your up-to-dateness test, because
there will generally have been another commit before a package makes it
through the buildds let alone making it into testing.

Having said that, I applaud the effort, but a few more filters would
appear to be needed for it to become properly useful.

Cheers, Phil.

[Sam Hartman]

Santiago> Dear Debian fellows, I am writing this email under the
Santiago> hypothesis that having the latest (or longest supported)
Santiago> upstream version in the next release will: 1. make it
Santiago> easier to provide security support during the whole
Santiago> release lifecycle, and 2. it will be useful for users, as
Santiago> they could have the latest features and bugfixes, in case
Santiago> of minor point release updates.

I found this really cool and thanks for doing it.
I think some of the issues others have raised are valid, and I don't
actually know what next steps ought to be with this data.
I know that for myself, I found reading through the first few hundred
entries in your list was interesting and gave me a snapshot of Debian I
was not getting elsewhere.

Thanks for doing the work and please continue to think about how this
could evolve.