Remove ancient uploads from experimental (and later unstable)

[Ansgar 🐱]

Hi,

I'm wondering how we can clean up suites like experimental and
unstable. They tend to slowly accumulate cruft that nobody cleans up,
including no longer installable packages.

As a very simple start, I would like to remove packages from
experimental that haven't seen an upload for a long time (arbitrarily
chosen as before 2020-01-01 for the list below).

What do people think about this?

I would also like to do something similar to unstable; maybe start with
packages uploaded before some arbitrary date that are also not included
in any of oldstable/stable/testing. These can cause problems like
wasting time to investigate cruft removals, build failures, ...

Does that seem reasonable as well?

Ansgar

[Matthias Geiger]

Date: Sun, 29 Dec 2024 13:37:50 +0100
From: Matthias Geiger <***@riseup.net>
To: <***@debian.org>, <debian-***@lists.debian.org>
Cc:
Bcc:
Subject: Re: Remove ancient uploads from experimental (and later
unstable)
In-Reply-To: <***@43-1.org>

Hi,

Post by Ansgar 🐱
As a very simple start, I would like to remove packages from
experimental that haven't seen an upload for a long time (arbitrarily
chosen as before 2020-01-01 for the list below).
What do people think about this?

Sure, seems like a sane choice. If a package has been in exp for 2+
years and never has been uploaded to unstable it's likely been "forgot".
As experimental is a staging ground for packages I agree it should be
periodically be checked for outdated cruft(where the date is > 2 years).

best,

werdahias

Please CC me as I am not subscribed to the list.

[Andrey Rakhmatullin]

Post by Ansgar 🐱
I'm wondering how we can clean up suites like experimental and
unstable. They tend to slowly accumulate cruft that nobody cleans up,
including no longer installable packages.
As a very simple start, I would like to remove packages from
experimental that haven't seen an upload for a long time (arbitrarily
chosen as before 2020-01-01 for the list below).
What do people think about this?

This is a very good idea.

Post by Ansgar 🐱
I would also like to do something similar to unstable; maybe start with
packages uploaded before some arbitrary date that are also not included
in any of oldstable/stable/testing. These can cause problems like
wasting time to investigate cruft removals, build failures, ...
Does that seem reasonable as well?

This one deserves some discussion but is also a good idea. It was also
proposed recently in
https://lists.debian.org/debian-devel/2024/08/msg00298.html

[David Prévot]

Hi,

On 29/12/2024 13:08, Ansgar 🐱 wrote:
[…]

Post by Ansgar 🐱
As a very simple start, I would like to remove packages from
experimental that haven't seen an upload for a long time (arbitrarily
chosen as before 2020-01-01 for the list below).

Yes please.

Post by Ansgar 🐱
php-sabre-event (U)
php-sabre-vobject (U)

Ouch, you have my consent on these.

Regards,

taffit

[Thorsten Glaser]

musescore-snapshot | 2019-07-05 00:19:11.735843+00

I do have plans to pick that up once I find the tuits for it
and have finished the preceding musescore{2,3} uploads. (Lots
to do there.) So please keep that for now.

Thanks,
//mirabilos

[Helmut Grohne]

Hi Ansgar,

Post by Ansgar 🐱
I'm wondering how we can clean up suites like experimental and
unstable. They tend to slowly accumulate cruft that nobody cleans up,
including no longer installable packages.
As a very simple start, I would like to remove packages from
experimental that haven't seen an upload for a long time (arbitrarily
chosen as before 2020-01-01 for the list below).
What do people think about this?

Thanks for cleaning up the archive. The proposed criteria vaguely makes
sense to me for experimental, but not as much for unstable.

Post by Ansgar 🐱
I would also like to do something similar to unstable; maybe start with
packages uploaded before some arbitrary date that are also not included
in any of oldstable/stable/testing. These can cause problems like
wasting time to investigate cruft removals, build failures, ...
Does that seem reasonable as well?

I earlier proposed unstable removals and am now operating a
semi-automatic unstable auto remover. Its semantics are as follows.

Consider packages that have an rc bug in unstable that hasn't been
interacted with within one year and where the package in question is not
part of stable nor testing and is not a key package. For each of those
packages file an important removal suggestion bug. Reassign such bugs to
ftp.d.o as RM/RoQA bugs to ftp after a month of idleness. This seems to
work out quite well and I removed more than 100 source packages this
way.
https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=helmutg%40debian.org&tag=sidremove
Feedback welcome.

For unstable, I think there is more reason to keep packages that have
not been uploaded in a while - so long as they do not accumulate rc
bugs.

Post by Ansgar 🐱
monkeysphere | 2019-05-19 23:33:52.925219+00

I removed it from unstable, and missed experimental. Do you have any
hints on how to improve the RM bug #1085868 such that experimental is
also covered?

Post by Ansgar 🐱
php-sabre-event | 2015-11-06 01:20:46.165676+00

I think I also removed this from unstable. Same question.

Helmut

[Andreas Metzler]

[...]

Post by Helmut Grohne

Post by Ansgar 🐱
I would also like to do something similar to unstable; maybe start with
packages uploaded before some arbitrary date that are also not included
in any of oldstable/stable/testing. These can cause problems like
wasting time to investigate cruft removals, build failures, ...
Does that seem reasonable as well?

I earlier proposed unstable removals and am now operating a
semi-automatic unstable auto remover. Its semantics are as follows.
Consider packages that have an rc bug in unstable that hasn't been
interacted with within one year and where the package in question is not
part of stable nor testing and is not a key package. For each of those
packages file an important removal suggestion bug. Reassign such bugs to
ftp.d.o as RM/RoQA bugs to ftp after a month of idleness. This seems to
work out quite well and I removed more than 100 source packages this
way.
https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=helmutg%40debian.org&tag=sidremove
Feedback welcome.

[...]

Good morning,

I think this needs a mechanism fo excempt packages which are kept
out of testing *intentionally* with a dummy rc bug (like firefox's
#817954).

cu Andreas

[Chris Hofstaedtler]

Post by Andreas Metzler
[...]

Post by Helmut Grohne

Post by Ansgar 🐱
I would also like to do something similar to unstable; maybe start with
packages uploaded before some arbitrary date that are also not included
in any of oldstable/stable/testing. These can cause problems like
wasting time to investigate cruft removals, build failures, ...
Does that seem reasonable as well?

I earlier proposed unstable removals and am now operating a
semi-automatic unstable auto remover. Its semantics are as follows.
Consider packages that have an rc bug in unstable that hasn't been
interacted with within one year and where the package in question is not
part of stable nor testing and is not a key package. For each of those
packages file an important removal suggestion bug. Reassign such bugs to
ftp.d.o as RM/RoQA bugs to ftp after a month of idleness. This seems to
work out quite well and I removed more than 100 source packages this
way.
https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=helmutg%40debian.org&tag=sidremove
Feedback welcome.

[...]
Good morning,
I think this needs a mechanism fo excempt packages which are kept
out of testing *intentionally* with a dummy rc bug (like firefox's
#817954).

Thats this usertag:

https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=***@debian.org&tag=sidremove-ignore

C.

[Bill Allombert]

Post by Ansgar 🐱
Hi,
I'm wondering how we can clean up suites like experimental and
unstable. They tend to slowly accumulate cruft that nobody cleans up,
including no longer installable packages.
As a very simple start, I would like to remove packages from
experimental that haven't seen an upload for a long time (arbitrarily
chosen as before 2020-01-01 for the list below).
What do people think about this?

Only if that does not lead to a subsequent upload of the package to be
directed to the NEW queue again. This would waste way too much time.

Post by Ansgar 🐱
I would also like to do something similar to unstable; maybe start with
packages uploaded before some arbitrary date that are also not included
in any of oldstable/stable/testing. These can cause problems like
wasting time to investigate cruft removals, build failures, ...
Does that seem reasonable as well?

Not for some of my packages at least. As a rule do not remove packages
that do not have RC bugs, even if they are barred of entering testing.

Cheers,
Bill.

[Holger Levsen]

Post by Ansgar 🐱
I'm wondering how we can clean up suites like experimental and
unstable. They tend to slowly accumulate cruft that nobody cleans up,
including no longer installable packages.

for experimental: yes, please!
for unstable: what Helmut said & does.

Post by Ansgar 🐱
As a very simple start, I would like to remove packages from
experimental that haven't seen an upload for a long time (arbitrarily
chosen as before 2020-01-01 for the list below).

I'd propose to be more "aggressive" and use 2021-08-14 as the date, IOW
the bullseye release date. 2023-06-10 (=bookworm release date) would IMO
be too agressive *now*, but fine after the trixie release.

Thanks for bringing this up; cruft like this wastes human and computing
ressources regularily, so we regularily need to reduce it, as it accumulates
all the time.
--
cheers,
Holger

⢀⣎⠟⠻⢶⣊⠀
⣟⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄

Bananas are berries.

[Chris Hofstaedtler]

Post by Holger Levsen

Post by Ansgar 🐱
I'm wondering how we can clean up suites like experimental and
unstable. They tend to slowly accumulate cruft that nobody cleans up,
including no longer installable packages.

for experimental: yes, please!
for unstable: what Helmut said & does.

My thoughts too.

Cutoff date could be more aggressive, but I don't have a strong
opinion on it.

Chris