DEP-18: Git and GitLab usage in other Linux distros (Re: Representing Debian Metadata in Git)

[Otto Kekäläinen]

Hi!

Considering what other Linux distros are doing and what is popular
among software developers today, Debian will probably gain more
contributors and be more approachable if we don't reinvent too many
custom things, and hence...

..

In the current DEP14/DEP18 discussions a lot of discussion was had
about how we should represent Debian things in git; your mail also
goes into this direction.
My *feeling* is we should do the opposite - that is, represent less
Debian stuff in git, and especially do it in less Debian-specific
ways. IOW, no git extensions, no setup with multiple branches that
contain more or less unrelated things, etc.
I think we should move more towards a setup that is easily
understood by people not closely following our Debian-specific
things. We should avoid surprising things, again that would include
the multiple branches and any git extensions.
Before pushing for new ways of representing Debian stuff in git, I
think it would be a good idea to learn from all the other distros
and distro-like systems successfully using git [1]. Debian is not
the only distro that wants to use git to capture changes and
encourage contributions to its packages.
Chris
[1] alpine, homebrew, freebsd ports come to mind immediately. nixos
and others too.

…this is the right attitude and I wanted to cater to it and summarize
how packaging sources look in various distros.

I am using MariaDB in the examples as it is a big and common package
that can be found everywhere.

## Alpine
Example: https://git.alpinelinux.org/aports/tree/main/mariadb
Package defined in a single APKBUILD file somewhat similar to a .spec
file. Patches are plain patches in the same directory. References to
upstream source packages by SHA256SUMs. In my view it is not
effortless nor transparent to track code changes. Hosting is a
monorepo on cgit. The official way to contribute is on their GitLab
instance using Merge Requests at
https://gitlab.alpinelinux.org/alpine/aports

## Arch
Example: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=android-x86-64-mariadb
Very similar to Alpine, package defined in a single PKGBUILD file.
Patches as plain files in the same directory. No upstream git or
files, just MD5SUM references to upstream tarballs. One git repository
per package. Authoritative git repo on cgit. Collaboration features
via https://gitlab.archlinux.org/archlinux.

## Fedora
Example: https://src.fedoraproject.org/rpms/mariadb10.11/tree/rawhide
Fully custom code hosting, building and testing system due to long
history building up their own infra, which includes a facility to make
pull requests. Base of packaging is a .spec file plus a bunch of
included extra files. Patches as plain files.

## CentOS Stream
Example: https://gitlab.com/redhat/centos-stream/rpms/mariadb/-/tree/c9s
Basically, downstream from Fedora with some modifications. Hosting and
collaboration on custom GitLab instance.

## Rocky Linux
Example: https://git.rockylinux.org/staging/rpms/mariadb/-/tree/r9
This is one of the CentOS clones that came about after Red Hat
acquired CentOS. Inherits the .spec file from Fedora/CentOS. Custom
GitLab instance for all collaboration.

## OpenSUSE
Example: https://build.opensuse.org/package/show/openSUSE:Factory/mariadb
Packaging based on .spec file and individual patch files. The .spec
file format is custom and not fully the same as in the Fedora
ecosystem. Hosting and building on their custom-build platform (the
Open Build System) which includes graphical merge requests under name
"Requests".

## NixOS
Example: https://github.com/NixOS/nixpkgs/tree/nixos-unstable/pkgs/servers/sql/mariadb
Custom single-file packaging format in a .nix file. No upstream source
code hosting, just reference by SHA256 to upstream tarballs. Monorepo
with all packaging files. Uses GitHub. Currently, 5k+ open Pull
Requests and almost 300k (!) closed ones.

## Gentoo
Custom single-file .ebuild files, with some patches as individual
files under files/. Authoritative hosting on cgit, but
https://packages.gentoo.org/packages/dev-db/mariadb links to GitHub
for Pull Requests as the recommended way to contribute.

## Deepin
Example: https://github.com/deepin-community/mariadb/tree/master/debian/deepin
The most popular Debian variant in China. Packaging done in
subdirectory `debian/deepin/`. Hosted on GitHub. Imports seem to be
one commit per upstream release, not sure if the content is from git
or tarball, but it isn't upstream git commits. They seem to have some
automation that files Pull Requests automatically when new upstream
import is pending.

## OpenMandriva
Example: https://github.com/OpenMandrivaAssociation/mariadb/tree/rolling
Again a .spec file user, patches as individual files. Hosting and Pull
Requests on GitHub.

## FreeBSD ports
Example: https://www.freshports.org/databases/mariadb114-server/ ->
https://cgit.freebsd.org/ports/tree/databases/mariadb114-server
Custom ports directory, package source builds based on Makefiles.
Authoritative hosting on cgit, but they have also Codeberg, GitHub and
GitLab mirrors. Most contributions seem to be patch files attached to
bug reports (e.g.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274494) but there
are also some GitHub PRs
(https://github.com/freebsd/freebsd-ports/pulls). The GitLab mirror
had zero Merge Requests.

## OpenBSD ports
Example: https://openports.pl/path/databases/mariadb
Custom ports directory. Source code hosted on both csvweb and GitHub
mirror. Monorepo with only packaging files (Makefile + patch files and
more). Instead of upstream sources there are files named 'distinfo'
that contain the upstream tar.gz name and SHA256.

## MacPorts
Example: https://ports.macports.org/port/mariadb-10.11/details/ ->
https://github.com/macports/macports-ports/blob/master/databases/mariadb-10.11/Portfile
Custom ports directory software. Packages in custom Portfile in
monorepo with SHA256 references to upstream tarballs.

## Homebrew
Example: https://formulae.brew.sh/formula/mariadb ->
https://github.com/Homebrew/homebrew-core/blob/master/Formula/m/mariadb.rb
Again, custom ports directory, custom packaging format, files on
GitHub in a monorepo and upstream tarballs referenced by sha256.

## Chocolatey
Example: https://community.chocolatey.org/packages/mariadb ->
https://github.com/mkevenaar/chocolatey-packages/tree/master/automatic/mariadb
This is the Windows equivalent of Homebrew. Packaging in .nuspec
files, fully custom thing. A centralized directory and seems a lot of
the packages are in a single monorepo on GitHub, not sure if that is
all, though.


Conclusions & personal comments:

- Everyone uses git. Sentiments that the Debian repository is enough
of version control, or that git would somehow not be suitable for
Debian packaging, seem detached from practical reality.

- Everyone uses normal git. The fact that Debian has two or three
branches (debian/latest + upstream/latest + pristine-tar) is a bit
special. I would still consider it warranted, as we have higher
standards regarding hosting the source code and having a full audit
trail of changes. If this is ever simplified, the somewhat obvious
solution would be to introduce a new debian/latest branch structure
that is just the current 'debian/*' contents plus a file, perhaps
called 'upstream-source', with url and sha256 sum of upstream tarball.

- GitHub and GitLab are quite common, people know how to use them. The
fact that we have salsa.debian.org is an asset for us and helps
attract new people. Thanks to various cli-tools people preferring to
do all work from the command-line (or from Emacs) should not
inherently be worse off if we have more contributors joining the
project who use Salsa.

- Pull Requests and Merge Requests are also very common. I think the
best course forward would be to have an open mind in accepting
contributions both as git pull requests as well as patch files in
email.

- The number of contributors/maintainers is low everywhere. Ending
single-person maintainership is not going to happen any soon, but
hopefully, we can work towards first increasing the pool of
contributors who participate, and then expand on practices around
Merge Requests and reviews and maybe have some kind of formal
sign-offs from at least two people before upload. Initially, perhaps
only for the top-150 packages. But before we can institute review
workflows, we need to have more unification around the version control
and basic packaging workflows.

- Otto

[Johannes Schauer Marin Rodrigues]

Hi,

Quoting Otto KekÀlÀinen (2024-08-27 08:42:53)

Before pushing for new ways of representing Debian stuff in git, I think it
would be a good idea to learn from all the other distros and distro-like
systems successfully using git [1]. Debian is not the only distro that
wants to use git to capture changes and encourage contributions to its
packages.
Chris
[1] alpine, homebrew, freebsd ports come to mind immediately. nixos
and others too.


this is the right attitude and I wanted to cater to it and summarize
how packaging sources look in various distros.

thank you for your investigations.

- The number of contributors/maintainers is low everywhere. Ending
single-person maintainership is not going to happen any soon, but hopefully,
we can work towards first increasing the pool of contributors who
participate, and then expand on practices around Merge Requests and reviews
and maybe have some kind of formal sign-offs from at least two people before
upload. Initially, perhaps only for the top-150 packages. But before we can
institute review workflows, we need to have more unification around the
version control and basic packaging workflows.

I'm still dubious any "2 people sign-off" can work [1]. In your investigations,
did you find other distributions which implemented this successfully?

I think "work towards easier collaboration" and "require more than one person
for every commit/upload" are two very different things which should be
discussed independently.

Thanks!

cheers, josch

[1] My own experience with this comes from my contributions to devscripts which
is in the debian group, thus "team" maintained and probably all of you have it
installed and should feel responsible for it (right?). Nevertheless, my MRs
mostly get zero replies, so I usually just merge them after waiting a couple of
months. The situation is a bit better for sbuild but not by much.

[Otto Kekäläinen]

Hi!

...

Post by Johannes Schauer Marin Rodrigues

Post by Otto Kekäläinen
- The number of contributors/maintainers is low everywhere. Ending
single-person maintainership is not going to happen any soon, but hopefully,
we can work towards first increasing the pool of contributors who
participate, and then expand on practices around Merge Requests and reviews
and maybe have some kind of formal sign-offs from at least two people before
upload. Initially, perhaps only for the top-150 packages. But before we can
institute review workflows, we need to have more unification around the
version control and basic packaging workflows.

I'm still dubious any "2 people sign-off" can work [1]. In your investigations,
did you find other distributions which implemented this successfully?
I think "work towards easier collaboration" and "require more than one person
for every commit/upload" are two very different things which should be
discussed independently.
Thanks!
cheers, josch
[1] My own experience with this comes from my contributions to devscripts which
is in the debian group, thus "team" maintained and probably all of you have it
installed and should feel responsible for it (right?). Nevertheless, my MRs
mostly get zero replies, so I usually just merge them after waiting a couple of
months. The situation is a bit better for sbuild but not by much.

I see you have 41 MRs at
https://salsa.debian.org/debian/devscripts/-/merge_requests?scope=all&state=all&author_username=josch
Some of them were approved by Niels, but indeed most go without
approval nor even comments.

I Salsa CI and in Lintian for example I do see pretty good review
activity. Perhaps we just need to have more visibility and reminders
for people to check out
https://salsa.debian.org/debian/devscripts/-/merge_requests or even
https://salsa.debian.org/groups/debian/-/merge_requests which has now
993 MRs open (and nearly 10k MRs in total).

I wonder if it would make sense to organize some kind of "office
hours" for code reviews and code testing best practices and
guidance...