Towards DEP-14 acceptance and recently proposed changes

[Raphael Hertzog]

Hello,

DEP-14 is 10 years old and while it's far from being ubiquitous, it
seems to have gained enough traction (in particular in git-buildpackage
thanks to the work of Otto KekÀlÀinen, and in quite some packaging teams
thanks to the respective team members) that it would be worth marking
it as ACCEPTED.

https://dep-team.pages.debian.net/deps/dep14

But before this, there are a couple of outstanding improvements that
have been suggested where I'd like peer review.

1/ https://salsa.debian.org/dep-team/deps/-/merge_requests/9
(see mr9.patch attached if you don't want to look up on the web)

We had a couple of revisions already and it seems fine to me to merge
this, if anyone has valid objections (i.e. with good rationale), now is
the time to raise them. You might want to look the closed comments
on the above merge request to see whether your concerns have been
discussed already.

This change basically adds the recommendation to use "upstreamvcs" as the
name of the "git remote" to access the upstream repository and it also
documents the possibility to merge the upstream commits in the
"upstream/latest" branch (as proposed by gbp import-orig
--upstream-vcs-tag) when your workflow is to import the upstream tarball.

2/ https://salsa.debian.org/dep-team/deps/-/merge_requests/16
(see mr16.patch attached if you don't want to look up on the web)

This change is still under discussion (no consensus has been reached yet)
but it would be nice to have some broader participation to help bring the
discussion to some conclusion. Basically the topic is how to name the
branches when the package maintainer wants to support multiple upstream
releases for a given target Debian release.

We solved this for the "upstream/*" branches but we never specified this for
the "debian/*" branches partly because Debian doesn't have PPA so it's
rather unusual to maintain multiple upstream releases for the same target
Debian release... but the use case seems like worth supporting and having
a good recommendation for it.

Ideally join the discussion on salsa, but I'll take into account comments
shared here as well. Please cc me preferrably as I don't read debian-devel
daily.

Cheers,

[Julien Plissonneau Duquène]

Post by Raphael Hertzog
2/ https://salsa.debian.org/dep-team/deps/-/merge_requests/16

To expand a bit on this, proposals so far for naming versioned branches
are:
1. <vendor>/<version>/<suite> e.g. debian/6.12/trixie which (I think)
matches what the kernel team is already doing [1]
2. <vendor>/<version.x>/<suite> e.g. debian/6.12.x/trixie
3. <src:pkg>/<vendor>/<suite> when the version is embedded in source
package name
4. [<src:pkg>/][version/]<vendor>/<suite>

The MR above has additional links and descriptions of current practices.
I do not have a strong preference for a choice yet so feedback would be
appreciated.

Cheers,

[1]: https://salsa.debian.org/kernel-team/linux/-/branches/active

[Marco d'Itri]

Post by Raphael Hertzog
This change basically adds the recommendation to use "upstreamvcs" as the
name of the "git remote" to access the upstream repository and it also

Like many others, this looks like a gratuitous change for change's sake.
Countless packages have been using "upstream" for years, but apparently
it is not perfect enough and somebody had to invent a new name.

I still believe that this is trying to codify a lot of pratices (e.g.
the obsession with pristine-tar) which are not widely accepted.
This DEP is the result of an echo chamber and is the consensus of the
few people debating it on salsa.

Post by Raphael Hertzog
+The codename can also be prefixed with the version so that branch names
+are correctly sorted by age of the target release in the list of available
+branches. Examples: `debian/12-bookworm`, `debian/11-bullseye`,
+`ubuntu/24.04-noble`, `ubuntu/22.04-jammy`.

So to gain optional sorting now you lose the benefit of being able to
mechanically determine the branch name? This is beyond silly.

Post by Raphael Hertzog
+unlikely to clash with the packaging branches. Despite this, it is good
+practice to not push any upstream branch to the packaging repository so as
+to not confuse anyone about the purpose of the Git repository.

WTF? I say instead that in the modern worlds it is a best practice to
have the packaging repository as a branch of the upstream repository.
Again, trying to promote personal preferences to a standard.

[Julien Plissonneau Duquène]

Post by Marco d'Itri

Post by Raphael Hertzog
This change basically adds the recommendation to use "upstreamvcs" as the
name of the "git remote" to access the upstream repository and it also

Like many others, this looks like a gratuitous change for change's sake.
Countless packages have been using "upstream" for years, but apparently
it is not perfect enough and somebody had to invent a new name.

Let's say upstream name their release tags like "1.2.3". That's valid
and that happens.

In your Debian packaging repository, after importing that release you
now have a tag named "upstream/1.2.3" if you follow the proposed
convention or use some existing tooling with its default configuration.

Let's now assume that your upstream remote is named "upstream", as is
common practice (I do that too).

git checkout upstream/1.2.3

--> the tag from upstream or the tag from your import? Nominally that
should be the same content, excepted that there are many projects that
repack sources, and the tooling will always generate a different commit
anyway. The different name is to resolve this ambiguity.

Post by Marco d'Itri
I still believe that this is trying to codify a lot of pratices (e.g.
the obsession with pristine-tar) which are not widely accepted.

This DEP only codifies the branch name (and btw forgets to mention
pristine-lfs) if you choose to use it, and it's not even pushing you to
use it in any way.

Post by Marco d'Itri
This DEP is the result of an echo chamber and is the consensus of the
few people debating it on salsa.

You are free to join or discuss it here. That DEP has been online for
some years now, it's not exactly happening in the hiding.

Post by Marco d'Itri

Post by Raphael Hertzog
+The codename can also be prefixed with the version so that branch names
+are correctly sorted by age of the target release in the list of available
+branches. Examples: `debian/12-bookworm`, `debian/11-bullseye`,
+`ubuntu/24.04-noble`, `ubuntu/22.04-jammy`.

So to gain optional sorting now you lose the benefit of being able to
mechanically determine the branch name? This is beyond silly.

This ability is not lost, it just needs a bit more logic or input data
to match both naked codenames and versioned codenames. The tradeoff
seems reasonable.

Post by Marco d'Itri
WTF? I say instead that in the modern worlds it is a best practice to
have the packaging repository as a branch of the upstream repository.

Could you please give us a few examples of projects where that already
works out-of-the-box, ie the package is built and uploaded to the
archive exactly as provided by upstream, debian/changelog and all?

Post by Marco d'Itri
Again, trying to promote personal preferences to a standard.

That's just a set of recommendations, not policy. You are free not to
follow them. In the meantime many packages already adopted some of the
proposed conventions so maybe just expect to be asked questions more
often about your own personal preferences, but if they are reasonable
that should not be much of an issue, and if that happens often enough
you may want to publicly document them somewhere to avoid repeating
yourself.

Cheers,

[Andrey Rakhmatullin]

Post by Marco d'Itri

Post by Raphael Hertzog
This change basically adds the recommendation to use "upstreamvcs" as the
name of the "git remote" to access the upstream repository and it also

Like many others, this looks like a gratuitous change for change's sake.
Countless packages have been using "upstream" for years, but apparently
it is not perfect enough and somebody had to invent a new name.

Let's say upstream name their release tags like "1.2.3". That's valid and
that happens.
In your Debian packaging repository, after importing that release you now
have a tag named "upstream/1.2.3" if you follow the proposed convention or
use some existing tooling with its default configuration.
Let's now assume that your upstream remote is named "upstream", as is common
practice (I do that too).
git checkout upstream/1.2.3
--> the tag from upstream or the tag from your import? Nominally that should
be the same content, excepted that there are many projects that repack
sources, and the tooling will always generate a different commit anyway. The
different name is to resolve this ambiguity.

Tags in git are not prefixed with the remote name or anything else. A tag
named upstream/1.2.3 is never from the upstream. A tag named 1.2.3 should
always be from the upstream.

Branches, on the other hand, indeed clash between "upstream/foo branch
from the maintainer" and "foo branch from the upstream".

[Julien Plissonneau Duquène]

Post by Andrey Rakhmatullin
Tags in git are not prefixed with the remote name or anything else. A tag
named upstream/1.2.3 is never from the upstream. A tag named 1.2.3 should
always be from the upstream.
Branches, on the other hand, indeed clash between "upstream/foo branch
from the maintainer" and "foo branch from the upstream".

Exact for the release-tag-from-upstream-repo, sorry for the confusion.
The tag-generated-from-import though is actually named "upstream/x.y.z".

So the possible clash is between a local tag and a remote branch.
Branches named after version numbers also happen quite often upstream.

[Marco d'Itri]

You are free to join or discuss it here. That DEP has been online for some
years now, it's not exactly happening in the hiding.

Nobody mentioned hiding anything, but it's still just a few people
chatting in a gitlab issue.

Post by Marco d'Itri
WTF? I say instead that in the modern worlds it is a best practice to
have the packaging repository as a branch of the upstream repository.

Could you please give us a few examples of projects where that already works
out-of-the-box, ie the package is built and uploaded to the archive exactly
as provided by upstream, debian/changelog and all?

I mean having the packaging branch in the Debian repository, not in the
upstream repository.

Post by Marco d'Itri
Again, trying to promote personal preferences to a standard.

That's just a set of recommendations, not policy. You are free not to follow

With the obvious goal of becoming a standard.
It's the whole point of DEPs.

[Xiyue Deng]

[..snip..]
Could you please give us a few examples of projects where that already
works out-of-the-box, ie the package is built and uploaded to the
archive exactly as provided by upstream, debian/changelog and all?

I'm not Marco, but I happen to come across some packages that has
"debian/" maintained in upstream repo because the Debian maintainer is
also upstream maintainer, e.g. notmuch[1].

[1] https://git.notmuchmail.org/git/notmuch

[gregor herrmann]

Post by Marco d'Itri

Post by Raphael Hertzog
This change basically adds the recommendation to use "upstreamvcs" as the
name of the "git remote" to access the upstream repository and it also

Like many others, this looks like a gratuitous change for change's sake.
Countless packages have been using "upstream" for years, but apparently
it is not perfect enough and somebody had to invent a new name.

As a data point, the Debian Perl Group is using "upstream-repo" for
this additional remote in its tools since December 2013.


Cheers,
gregor

[Raphael Hertzog]

Hi,

Post by Marco d'Itri
Like many others, this looks like a gratuitous change for change's sake.
Countless packages have been using "upstream" for years, but apparently
it is not perfect enough and somebody had to invent a new name.

It's difficult to have figures for this since the name of the remote
is a local choice. There have been cases where I used "upstream" too
but indeed it clashes with the namespace that the DEP recommeds for
local branches and tags for the upstream import. So "upstream" is not a
good choice.

Given that the "upstream" branch name cames from the git-buildpackage and
that it's git-buildpackage which introduced "upstreamvcs", it seems fair
to me to standardize on this name.

Post by Marco d'Itri
I still believe that this is trying to codify a lot of pratices (e.g.
the obsession with pristine-tar) which are not widely accepted.
This DEP is the result of an echo chamber and is the consensus of the
few people debating it on salsa.

I never pushed further earlier because I had that feeling in the early
years but I believe this has changed lately. That said I don't have
figures to back it up and I agree it would be nice to have some concrete
figures of adoption (as Lucas suggested).

Post by Marco d'Itri

Post by Raphael Hertzog
+The codename can also be prefixed with the version so that branch names
+are correctly sorted by age of the target release in the list of available
+branches. Examples: `debian/12-bookworm`, `debian/11-bullseye`,
+`ubuntu/24.04-noble`, `ubuntu/22.04-jammy`.

So to gain optional sorting now you lose the benefit of being able to
mechanically determine the branch name? This is beyond silly.

This "mechanical determination" has never been true unfortunately,
the DEP allows the use of "codename" as well as "suite" as well as
"latest" depending on the different use cases.

So the reason why I accepted this is that we're not going backwards
in the fact that you need to list all the available branches to figure
out which is the correct one that you are looking for.

Post by Marco d'Itri

Post by Raphael Hertzog
+unlikely to clash with the packaging branches. Despite this, it is good
+practice to not push any upstream branch to the packaging repository so as
+to not confuse anyone about the purpose of the Git repository.

WTF? I say instead that in the modern worlds it is a best practice to
have the packaging repository as a branch of the upstream repository.
Again, trying to promote personal preferences to a standard.

It can certainly be a branch of the upstream repository (and the DEP does
document that workflow as one of the recommended ones), but that's not a
reason to have more than the upstream tag in your packaging repository
since that's what you merge in your packaging branch.

Cheers,

[gregor herrmann]

Post by Raphael Hertzog
Given that the "upstream" branch name cames from the git-buildpackage and
that it's git-buildpackage which introduced "upstreamvcs", it seems fair
to me to standardize on this name.

In theory, yes. In pratice, quoting myself:

| As a data point, the Debian Perl Group is using "upstream-repo" for
| this additional remote in its tools since December 2013.

Not that one is better than the other, and although I like to follow
git-buildpackage in general, having to change the names on 4000+
remotes on dozens of developers' machine is annoying.


Cheers,
gregor

[Chris Hofstaedtler]

Post by Raphael Hertzog
1/ https://salsa.debian.org/dep-team/deps/-/merge_requests/9
(see mr9.patch attached if you don't want to look up on the web)
We had a couple of revisions already and it seems fine to me to merge
this, if anyone has valid objections (i.e. with good rationale), now is
the time to raise them. You might want to look the closed comments
on the above merge request to see whether your concerns have been
discussed already.
This change basically adds the recommendation to use "upstreamvcs" as the
name of the "git remote" to access the upstream repository and it also
documents the possibility to merge the upstream commits in the
"upstream/latest" branch (as proposed by gbp import-orig
--upstream-vcs-tag) when your workflow is to import the upstream tarball.

The name for the remote could easily have been "upstream" like
various packages are already set up and documented today. But
apparently gbp picked the IMO unwieldly name in the meantime?

Meh. But what's done is done, I guess. We'll see who will adopt that
name.

Chris

[Lucas Nussbaum]

Post by Chris Hofstaedtler

Post by Raphael Hertzog
1/ https://salsa.debian.org/dep-team/deps/-/merge_requests/9
(see mr9.patch attached if you don't want to look up on the web)
We had a couple of revisions already and it seems fine to me to merge
this, if anyone has valid objections (i.e. with good rationale), now is
the time to raise them. You might want to look the closed comments
on the above merge request to see whether your concerns have been
discussed already.
This change basically adds the recommendation to use "upstreamvcs" as the
name of the "git remote" to access the upstream repository and it also
documents the possibility to merge the upstream commits in the
"upstream/latest" branch (as proposed by gbp import-orig
--upstream-vcs-tag) when your workflow is to import the upstream tarball.

The name for the remote could easily have been "upstream" like
various packages are already set up and documented today. But
apparently gbp picked the IMO unwieldly name in the meantime?
Meh. But what's done is done, I guess. We'll see who will adopt that
name.

Maybe before moving it to ACCEPTED, it would be useful to design a
dashboard of some kind to track adoption, not just in tooling, but in
actual packages?

This could probably be built as an extension to vcswatch.

Lucas

[Gioele Barabucci]

Post by Chris Hofstaedtler

Post by Raphael Hertzog
This change basically adds the recommendation to use "upstreamvcs" as the
name of the "git remote" to access the upstream repository and it also
documents the possibility to merge the upstream commits in the
"upstream/latest" branch (as proposed by gbp import-orig
--upstream-vcs-tag) when your workflow is to import the upstream tarball.

The name for the remote could easily have been "upstream" like
various packages are already set up and documented today. But
apparently gbp picked the IMO unwieldly name in the meantime?
Meh. But what's done is done, I guess. We'll see who will adopt that
name.

This is what git-buildpackage already has. Nothing stops people from
suggesting other names. I remember Guido mentioning at one point he is
open to change it or to make it configurable.
Personally I feel that just picking a reasonably good name and
sticking to it for now is the best return on investment for Debian.
The 'upstreamvcs' is reasonably good and unambiguous on what it means,
and it won't get mixed up with `upstream/*` tag names.

Maybe I'm missing something, but why is the name of the upstream remote
such a big deal?

The names of the remotes are limited to the local repository and git
allows for multiple remotes pointing to the same URL. Everybody can keep
their favorite remote names and add one extra `upstreamvcs` remote.

One can simply state: "some scripts need to know the name of the remote.
Make sure that the upstreamvcs remote exists or configure another name
in gbp.conf [when this option will exist]".

Personally, I use the following scheme for remotes. For repos on salsa,
I name the remote after the user/team name. For repos on GitHub I name
them "{user}-gh", for those on freedesktop "{user}-fd" and so on.

A few examples:

* https://salsa.debian.org/utopia-team/avahi.git ⇒ utopia-team
* ***@salsa.debian.org:gioele/avahi.git ⇒ gioele
* https://github.com/avahi/avahi.git ⇒ avahi-gh

Becoming DEP-14 compatible would be a matter of typing once `git remote
add upstreamvcs https://github.com/avahi/avahi.git`. Or (once the gbp
option will exist) adding "upstream-remote = avahi-gh" to gbp.conf.

Regards,