Version Control system

[bb_]

This thread is meant to gather and streamline the discussion regarding our choice of version control system. Discussion appeared to be in several places. I did my best to gather all relevant material, but feel free to add any missing information.

We currently use svn, this has been working ever since 2004.

Some earlier discussion about the lobby bots:

Spoiler

Currently the svn bot code is hosted in the svn repo (source/tools/lobbybots/). @Dunedan imported the code in the 0ad github instance at https://github.com/0ad/lobby-bots and proposes to remove the code from SVN in https://code.wildfiregames.com/D4155. Below the discussion in the revision:

bb said:

Don't understand why we would want to move our code from our self-hosted server to any other random platform. What is wrong with having it in svn? If someone wishes to use the the git-mirror, feel free.
Search the forums for discussion but couldn't find.

Notice that only few devs have access the the new git and the code gets spread around more, making it harder to track all changes. Also there is no phabricator/trac or the like instance on it. Seems like it is making deving only harder.

Why is this tool special so it gets an own git, shouldn't we do it with every tool then? Makes the mess even bigger.

stan said:

Indeed it was not discussed publicly I believe, as I had a really hard time reaching to @user1

It was discussed with @implodedok and a bit with @Itms. The idea was to not put more strain on the server as the CI is already bringing it to its knees and instead rely on a third party (here github for the CI)

First reason is that while development of the lobby is linked, it actually lives its own life and evolves at a different pace and to my despair it's not actually completely versioned

This is actually a good step towards the engine split.

bb said:

There is no trac phabricator/instance

If you mean to track tickets/ make comments on pull requests you can do that too on Github.

Unlike all the other tools this one is actually used in production .

bb said:

stan said:

The idea was to not put more strain on the server as the CI is already bringing it to its knees and instead rely on a third party (here github for the CI)

How is this required to reduce the stress on the CI? Keeping 20-odd files in our svn repo won't hurt that much. If we want the bots to ignore these files, let them (though we have the bots for a reason so we better use them when we need to).
Also what happens if github goes offline/kicks us (maybe not likely, neither impossible), we loose our code? What if gitHub puts restrictions on what we do, do we then abide? Why give away the sovereignty we have? Why let the codebase fall apart, literally?
Now some third party CI might be free, probably many proper things we currently host ourselves are going to be expensive (now the SPI reports says we don't have to worry to much about our finances, I think we can spend our money better)

stan said:

First reason is that while development of the lobby is linked, it actually lives its own life and evolves at a different pace and to my despair it's not actually completely versioned

Say if I were to implement a new system of ratings, I did need to change the bots (as I need to compute the ratings) as well as some gui (i.e. I need to show the new rating). Having several repo's makes developing such a feature much harder, so is not wanted.
We have files in our repo which haven't been touched in the last 10 years or so and files which are changed every month or so. That is completely fine. So if these files are developed evolve at a different pace: fine. What is the problem? I suppose during CF we shouldn't change the lobby bots either (if there are bugs with them, we need to lift CF for them: fine). Even the more reason to keep them in the same repo.

stan said:

This is actually a good step towards the engine split.

Splitting the engine doesn't mean having several repo's. In fact we shouldn't have the engine in another repo than the vanilla game or anything else we develop related to 0ad. If we want to split stuff we need to work with directories within the repo: having a separate directory for tools like this within our repo is perfectly fine with me, storing it in source might indeed not be ideal.
Even if we need several repo's, we should host them ourselves.

bb said:

There is no trac phabricator/instance

stan said:

If you mean to track tickets/ make comments on pull requests you can do that too on Github.

So we as developers need to look at yet another place for patches, instead of keeping everything in one place.

stan said:

Unlike all the other tools this one is actually used in production .

So are the translation scripts and probably others too.

wraitii said:

 

bb said:

Also what happens if github goes offline/kicks us (maybe not likely, neither impossible), we loose our code?

I think this is absurdly unlikely at this point, but even if GitHub deletes the repose unilaterally, git makes it so that we'd very likely have recent-is mirrors on several dev computers & on the lobby server itself. I don't think this should be a concern.

bb said:

Why give away the sovereignty we have?

'Cause Phabricator is dying, see https://wildfiregames.com/forum/topic/42181-phabricator-is-no-longer-being-maintained-upstream/
Also sovereignty is essentially kept per the first point.

bb said:

Why let the codebase fall apart, literally?

Other arguments:

• It's a good way to gain knowledge about GitHub usage, which will help should we decide someday to migrate somewhere else.
• the person who wants to work on this code wants to work on GitHub because it's easier for them

bb said:

Say if I were to implement a new system of ratings, I did need to change the bots (as I need to compute the ratings) as well as some gui (i.e. I need to show the new rating). Having several repo's makes developing such a feature much harder, so is not wanted.

It's a quite realistic prospect to change the rating calculation without changing the rating display, and vice-versa.
The actual hard problem is deployment, and since we don't actually deploy the bots synchronously with SVN, we'd already have a lot of trouble if you wanted to implement a new system of rating.

bb said:

Even if we need several repo's, we should host them ourselves.

The logic of 'we should do it ourselves' just creates more work for everyone and means we have a laggy Phabricator CI that's not maintained instead of a fast, easy-for-beginners GitHub CI, to be honest.

bb said:

wraitii said:

bb said:

Also what happens if github goes offline/kicks us (maybe not likely, neither impossible), we loose our code?

I think this is absurdly unlikely at this point, but even if GitHub deletes the repose unilaterally, git makes it so that we'd very likely have recent-is mirrors on several dev computers & on the lobby server itself. I don't think this should be a concern.

Even if unlikely not impossible. So we will be dependent on a few persons who have the code. Also this doesn't save all comments and discussions.

wraitii said:

bb said:

Why give away the sovereignty we have?

'Cause Phabricator is dying, see https://wildfiregames.com/forum/topic/42181-phabricator-is-no-longer-being-maintained-upstream/

Phab will keep running as long as we keep it running on our servers. Also I saw there were efforts to fork it.

wraitii said:

Also sovereignty is essentially kept per the first point.

The first point doesn't keep our sovereignty at all.

wraitii said:

bb said:

Why let the codebase fall apart, literally?

Other arguments:

It's a good way to gain knowledge about GitHub usage, which will help should we decide someday to migrate somewhere else.

If we were migrating (which I would oppose, enfin), we did be migrating to a self hosted git, at the least. With moving to github we do NOT gain any experience on this.

wraitii said:

the person who wants to work on this code wants to work on GitHub because it's easier for them

So he can use the git mirror, fine, just commit the patches in the SVN. If the mirror bot needs to run more often than once a day, that shouldn't be much of an issue. There is also the option of git-svn. And if one wants an own mirror on github of this piece of the code, I won't object.

wraitii said:

bb said:

Say if I were to implement a new system of ratings, I did need to change the bots (as I need to compute the ratings) as well as some gui (i.e. I need to show the new rating). Having several repo's makes developing such a feature much harder, so is not wanted.

It's a quite realistic prospect to change the rating calculation without changing the rating display, and vice-versa.

Even more realistic is to change both (e.g. team rating etc.).
Compare it to the following case: our art and code mostly behave independently. Artists can do their thing, coders do their thing. However in the secondary attack patch, I need a lot of code AND some new cursors. Having them together makes it possible to have one patch only for testers to test it. So even though most of the time the art and code are independent, since there are cases they are not, we should keep it in the same repo as it makes things easier.

wraitii said:

The actual hard problem is deployment, and since we don't actually deploy the bots synchronously with SVN, we'd already have a lot of trouble if you wanted to implement a new system of rating.

Moving to git doesn't help even a 1% on this problem. We probably need more devs who have access and more lobby mods and stuff like that. Anyway out of scope.

wraitii said:

bb said:

Even if we need several repo's, we should host them ourselves.

The logic of 'we should do it ourselves' just creates more work for everyone and means we have a laggy Phabricator CI that's not maintained instead of a fast, easy-for-beginners GitHub CI, to be honest.

So just keep it in the same repo??? Sounds like the easiest solution for all.

wraitii said:

 

bb said:

Even if unlikely not impossible. So we will be dependent on a few persons who have the code. Also this doesn't save all comments and discussions.

Er, we technically depend on a few persons now. If the servers were to crash irrecoverable and backups didn't work, we'd lose everything as well. Point is, I don't think this is much of an argument either way, though there are pros & cons for both.

bb said:

Phab will keep running as long as we keep it running on our servers. Also I saw there were efforts to fork it.

AFAIK none support SVN, unfortunately. And well, it's true, but it'll also keep all its security flaws (if any) & its general slowness & never get any new feature.

bb said:

If we were migrating (which I would oppose, enfin), we did be migrating to a self hosted git, at the least. With moving to github we do NOT gain any experience on this.

That's if we migrate to a self-hosted git, which is not necessarily the best option.

---

Anyway, I don't have a very strong opinion either way on this particular diff.

 

Below an excerpt from an e-mail Stan has send round a while ago to all staff members:

Spoiler

I am finding myself in front of an issue with the lobby. Despite all my efforts to improve communication and contribution to the lobby, it seems to remain the same blackbox it has been for a long while now at least since elexis stopped working on it.

• The code source in the repository is not representative of the production code.
• The offense reporting is completely transparent which made a few players mad.
• We have a very long time of response when issues arise in the lobby. For instance the Ratings bot was kicked from the lobby because it said some offensive player name and it took at least a day to report it. The certificate almost expired recently, and the vm configuration was blocking the automatic renewal.
• People are reporting being harassed on the lobby
• There are lot of smurf accounts with are in violation of a ToS we can not enforce.
• The knowledge of how the VM is setup is not to be found anywhere;
• You need admin access to moderate the lobby.

To remedy this I tried to open the discussion with user1, migrated the repository to github to separate the workflow and to alleviate some stress on the servers had we been to

As bb pointed out in this diff I have not communicated well on the lobby matters and a few things happened behind the scenes as I was trying to focus the discussion into the biggest actors of the lobby.

As some of you may know the lobby underwent a huge rewrite for A24, and it took a while to get the code committed to the repository. It was only committed after A24 came out.

To try to make the contribution to the lobby easier we migrated the code to Github, after discussing the possibility of self-hosting it with Itms and implodedok and rejecting it because we did not want to put more strain on the server with extra CI which is currently still hogging the server's resources and forcing us to change all the disks every year.

Dunedan took on to write pipelines and tests to automate everything so we would only have limited maintenance to do. But so far none of it has been reviewed and the code on the lobby VM might be really different. This has been a terrible contribution experience, and still is.

In January we wanted to extend the lobby VM disk, to be able to log more things and to stop it from constantly running out of space. It took about three whole months to get the necessary access to do it. After doing the lobby did not restart properly, and players didn't have any ranked games for two days.

A bit later after Dunedan was given access to the lobby VM he was able to renew the certificate in extremis before it would fail all clients. Now that his access is limited again to the scope of the of bots, he won't be able to do anything when, in 13 days the TLS-certificate of the lobby is expires again.

I'd like to hear your thoughts about how we can improve the current situation.

A relevant related topic is

On November 24 2021, a discussion on IRC:

Spoiler

(06:12:06 PM) molgrum: the main thing for me is you are running Trac instead of Github.
(06:12:12 PM) molgrum: I can use Trac with LibreJS.
(06:12:23 PM) molgrum: I'm originally a dev for Quakeworld.
(06:12:35 PM) molgrum: but they use Github so I can't dev for them.
(06:12:36 PM) Freagarach: We're using Phabricator now.
(06:12:51 PM) Freagarach: @wiki Phabricator
(06:12:51 PM) WildfireBot`: http://trac.wildfiregames.com/wiki/Phabricator
(06:13:34 PM) Freagarach: Why can't you use GitHub?
(06:13:39 PM) Freagarach: If I may ask.
(06:14:18 PM) molgrum: Freagarach: it's broken with LibreJS unfortunately.
(06:14:39 PM) molgrum: I can't run the code and Github heavily relies on non free JS.
(06:14:56 PM) molgrum: of course, I could send patches outside of Github.
(06:15:02 PM) Freagarach: Ah.
(06:15:09 PM) Freagarach: Let me search for LibreJS.
(06:15:10 PM) vv221: Stan, I am around
(06:16:35 PM) vv221: Freagarach, I think LibreJS is a Web browser extension preventing the execution of JavaScript unless said scripts are published under a libre/free license.
(06:16:47 PM) vv221: molgrum, is that right? ↑
(06:16:55 PM) Freagarach: Yep, I read it, thanks.
(06:17:02 PM) molgrum: vv221: yes.
(06:17:13 PM) Freagarach: Can you use code.wildfiregames.com?
(06:17:27 PM) Stan: vv221, imploded1k: is asking me to choose between a rpm based distro
(06:17:42 PM) Stan: And a debian based distro for setupping gitlab
(06:17:57 PM) Stan: With a preference for rpm
(06:17:58 PM) molgrum: Freagarach: it looks promising.
(06:18:08 PM) vv221: Freagarach, it seems that code.wildfiregames.com is usable even with no JS at all.
(06:18:15 PM) vv221: (at the cost of a couple features of course)
(06:18:21 PM) Stan: vv221: Might not pass the registration phase
(06:18:28 PM) Stan: I think there is a dumb recaptcha
(06:18:49 PM) Stan: But once you're registered it's good
(06:19:19 PM) Stan: Also if you have any recommandation for the vm specs for gitlab
(06:19:38 PM) vv221: Stan, the only GitLab setup I have experience with is specific to Debian. I don’t know how it is distributed for rpm-based distris, but I suspect it is using the Omnibus all-in-one "package".
(06:20:02 PM) Stan: (Initially I wanted to use the same vm as trac and phabricator but seeing it keeps failing kinda makes me scared)
(06:20:11 PM) vv221: I can check the specs of the VPS that used to host ./play.it GitLab forge, but right now it is running on my day-to-day computer
(06:20:24 PM) Stan: vv221: And that's bad because it bundles too much stuff?
(06:20:45 PM) Freagarach: https://about.gitlab.com/devops-tools/gitea-vs-gitlab/
(06:20:59 PM) Freagarach: Says you need .5Core 4GB memory.
(06:21:25 PM) vv221: That’s bad (to me) because I do not trust the GitLab team to maintain GitLab + Postgresql + nginx + etc. And trust the maintainer of all the distinct JS packages and Ruby gems it relies one.
(06:22:12 PM) vv221: With the Debian-provided packaging, I’m putting most of my trust into an organization and an infrastructure I already worked with for years.
(06:22:27 PM) vv221: Of course, it works because I trust Debian in the first place
(06:23:45 PM) vv221: Stan, my VPS is 4-core / 8GB RAM / 160 GB storage. But in my experience it is ~twice as big as what I really needed.
(06:24:08 PM) vv221: Keeping in mind that ./play.it is a low-traffic software project, with very lightweight repositories.
(06:25:19 PM) vv221: Right now ./play.it forge is on my gaming computer, so 8-core / 18GB RAM / 42 gazillion bytes of storage ;P
(06:26:14 PM) Stan: We have a very heavy svn repository which I will have to migrate to git
(06:26:31 PM) Stan: I was thinking of starting by building the fcollada library
(06:26:45 PM) JCWasmx86: So if I understand you correctly, you are migrating from Phabricator to Gitlab
(06:26:48 PM) Stan: Using gitlab ci to get familiar with it and artifacts
(06:26:48 PM) vv221: I did a quick check, 4GB RAM might be too little. It is already using almost that here.
(06:26:55 PM) Stan: JCWasmx86: That's my plan
(06:27:01 PM) Stan: But I might hit a wall
(06:27:36 PM) JCWasmx86: Is there anything I can help with? (I use linux on the desktop daily, but I have no experience for linux on servers)
(06:28:28 PM) Stan: Well my biggest problem is migrating everything to gitlab
(06:28:37 PM) Stan: There seem to be tools for phabricator
(06:28:47 PM) Stan: But nothing we actually use on phabricator
(06:28:55 PM) Stan: Not sure about trac
(06:29:13 PM) Stan: And i know you can run a jenkins instance on gitlab but i don't want to do that
(06:29:36 PM) Stan: And the final problem is migrating the repo to git
(06:29:41 PM) Stan: In a not dirty way
(06:30:00 PM) Stan: Aka not like the github and gitlab mirrors we have
(06:30:16 PM) Stan: We would have no windows binaries for linux users for instance
(06:30:26 PM) Stan: They'd be an artifact from gitlqb
(06:30:33 PM) Stan: Which should be easy to download
(06:30:46 PM) Stan: Because the goal is not to make life harder on windows
(06:31:48 PM) JCWasmx86: Ah well, I understand. Converting the tickets from trac should be able using the trac API to extract and the gitlab API to make issues
(06:32:22 PM) JCWasmx86: The wiki could be more difficult with all the history/diffs
(06:33:48 PM) vv221: I’m not sure if this info could help: GitLab wikis are actual git repos.
(06:36:45 PM) Freagarach: So we're going from SVN to GitLab, because there are many potential contributors on GitHub? Sorry for asking nasty questions. ^^
(06:38:30 PM) Stan: Freagarach: No we're going from SVN to Git because Phab won't support svn anymore doesn't get updates, and is hard to maintain
(06:38:41 PM) Stan: And we're going to a self hosted gitlab
(06:39:17 PM) Stan: Not gitlab.com nor github.com because apparently it's important that we keep our sovereignty in case big companies collapse
(06:42:43 PM) Stan: Tell me if I sound insane
(06:43:42 PM) Stan: Also I'm migrating everything to gitlab (jenkins and trac) because experience shows that development time is more important than sysadmin time given the current state of the team
(06:44:00 PM) Stan: And if we're (or I) gonna have to maintain something I'd rather have people I can ask like vv221
(06:44:17 PM) Stan: And also limit the maintaining to installing updates
(06:46:17 PM) vv221: Well, we probably won’t have less maintenance issues this way… but at least we can hope to have them only once (on ./play.it side or 0 A.D. side) and can warn the other or share the resolution.
(06:49:21 PM) Stan: I mean if I ask you why my gitlab is acting out vs why is my phabricator acting out you might have more insights ^^
(06:53:39 PM) vv221: No doubt about that
(07:08:10 PM) Stan: But yeah the migration is scaringly huge
(07:08:18 PM) Stan: And I'm not in the best state

[cut]

(07:23:22 PM) ***bb still owes you guys a forum thread on VCS choice
(07:25:08 PM) Stan: Indeed although I guess it will mostly end in dead silence
(07:25:50 PM) bb: could be true, still any choice whatever it is should be documented

What I did like to hear from everyone (staff, contributors, svn testers etc.): What is required from the version control system and CI? Which features are required? And what is only nice to have? Please list your wishes in this topic, preferably ordered in priority.

What are the current limitations and what are possible solutions for that? What are the pro's and cons for them?

[bb_]

I try to keep track here of all feature requests. Please yell if I wrongdo anything.

List of feauture request fro VCS

• Low sysadmin efforts (at most as high as phabricator currently) (Stan, wraitii)
• Security (dave_k)
• Proper binary handling (Freagarach, dave_k)
• All code art etc. in one place (bb, dave_k, Freagarach)
• Longevity (dave_k)
• Used in mods and other FLOSS projects (Stan, Locyneah, samulis)
• Companies involved are FLOSS minded (dave_k)
• Partial checkouts (Freagarach)
• Possibility to self-host (locyneah, bb)
• Reliable (dave_k)
• No branching (Freagarach)
• Branching (locyneah, maroder)
• Possible to work completely remotely (locyneah)
• Back up on devs machines (Freagarach)
• Public in production third party back up using different vcs (bb)
• Possibility for bots (elexis)
• Easy export of data (dave_k)
• Autobuild in repo (maroder, bb)
• Easy rebasing (maroder)
• Incorporate gitlab PR, github PR and phab Diffs in one system (wraitii)
• Usable by everyone from anywhere (bb)
• Simple straightforward workflow (bb)
• Low threshold for new contributers (Stan, bb, dave_k)
• Supported in maintaned platforms (Stan)
• 2FA (dave_k)
• Including (parts of) other (third party) repo's in ours (submodules) (Stan)
• Easily properly credit authors of patches, even if they don't have direct commit access (Stan)
• GPG signing of patches, tags and pull requests (dave_k)
• Support for GUI based workflow (samulis)

 

Features not directly related to VCS

• Low number of different systems used (maroder)

[bb_]

Some current limitation:

• Phabricator not being maintained anymore
• To much sysadmin time
• Many different systems used
• No full GUI workflow
• Highish threshold for new contributers

 

Post reserved to gather pro's and cons of different solutions.

Edited December 7, 2021 by bb_
Add a list of current limitations

[Freagarach]

Thanks for making this topic, @bb_!

So I guess the first step is the difference of SVN vs git. There are several websites discussing that (e.g. https://svnvsgit.com/), but I think the main conclusions are:

Pro SVN:

• Partial checkouts. (And access control.)
• Handles binaries better (our art).
• No branching (https://www.freecodecamp.org/news/why-you-should-not-use-feature-branches-a86950126124/).
• Easy to copy, delete or rename files.

Pro git:

• Backup on every dev machine.
• Pull requests?

Note that since we have git mirrors, one can still use branches on personal development copies and work offline (SVN needs connection with the server).

Needs for a VCS for me:

1. Handle binaries properly.
2. Support refactorings (if I rename and change a file, I don't want it to show up as a new file, that totally breaks blaming me).

[Stan`]

Pro Git

• Industry standard
• Co-authoring of commits
• Submodules? With different access
• Supported by maintained platforms (Gitlab, Gitea, Github, Bitbucket, Azure Devops)
• Already used by most (if not all) mods

59 minutes ago, Freagarach said:

Note that since we have git mirrors, one can still use branches on personal development copies and work offline (SVN needs connection with the server).

Assuming you want to extract diffs from branches once we have to drop Phabricator because it becomes a security flaw (or doesn't work anymore, currently we need to use hacks on the macOS and Linux VMs for certificates to work.

Needs for a VCS for me:

• Require the least sysadmin possible
• Be common enough to lower the entry barrier by having resources to use

[artoo]

If you go git, I would probably use (a) dedicated art repo(s), ie separate art from code.

Keeps the code repo small in terms of history compared to art repos containing images.

Git has two builtin options to handle possible external repo inclusion, subtree and submodules, but in my view, both are a PITA. There is also a 3rd party solution that works, git-subrepo.

[Stan`]

1 minute ago, artoo said:

Git has two builtin options to handle possible external repo inclusion, subtree and submodules, but in my view, both are a PITA. There is also a 3rd party solution that works, git-subrepo.

Yeah submodules are generally a PITA. Mentioned them for the sake of the argument, because I believe more things should be split from the repo, which is not a consensus inside the team see the spoiler in the first post.

Relevant links

• https://wildfiregames.com/forum/topic/16149-git-migration/
• [Git] [Step 1] Update autobuild utility for git workflow
• [Git] [Step 3] Create .gitattributes for Git code repo
• [Git] [Step 2] Create update.bat to sync windows binaries

[artoo]

Just now, Stan` said:

Yeah submodules are generally a PITA. Mentioned them for the sake of the argument, because I believe more things should be split from the repo, which is not a consensus inside the team see the spoiler in the first post.

 

Looking just from outside at the current project structure, if you migrate to git, reorganizing the project in git should be chosen wisely in my view in respect to the future repo sizes, commit history etc.

I think it would be no fun to put the current source tree in git as is, accumulating tiny problems over time.

I would probably consider an image repo for skins, textures, icons, portraits. If you wanted these included in source tree, probably git-subrepo would be the most painless solution. But even submodules might work for the purpose.

[Stan`]

1 minute ago, artoo said:

I think it would be no fun to put the current source tree in git as is, accumulating tiny problems over time.

https://github.com/0ad/0ad

https://gitlab.com/0ad/0ad

1 minute ago, artoo said:

I would probably consider an image repo for skins, textures, icons, portraits. If you wanted these included in source tree, probably git-subrepo would be the most painless solution. But even submodules might work for the purpose.

You need some of it for the engine to run.  See https://trac.wildfiregames.com/ticket/5366 (For another topic)

 

[artoo]

6 minutes ago, Stan` said:

https://github.com/0ad/0ad

https://gitlab.com/0ad/0ad

 

You don't work in the mirror repos.

The problems would stack if you used git with current structure, would work, but you might come to the conclusion in 3 years time and 1000s commits later "if we had only split it when we migrated"

Edited November 25, 2021 by artoo

[Stan`]

Just now, artoo said:

You don't work in the mirror repos.

Some people do, then you upload the patches to Phabricator.

Re binary files support could use this https://git-lfs.github.com/

1 minute ago, artoo said:

The problems would stack if you used git with current structure, would work, but you might come to the conclusion in 3 years time and 1000s commits later "if we had only split it when we migrated"

Well I can't do any of this if we don't agree migrating to git is necessary.

[artoo]

1 minute ago, Stan` said:

Well I can't do any of this if we don't agree migrating to git is necessary.

That's a decision I can't help with.

But I can give my experience with git, having used svn as default project vcs many years ago.

The big pro of git is the remotes in my opinion, the fork concept merging branches.

 

[maroder]

5 hours ago, Freagarach said:

No branching (https://www.freecodecamp.org/news/why-you-should-not-use-feature-branches-a86950126124/).

I've read thew article, but that is still not a pro for me, that's a con

With branches you don't have to stop development of new features just because there is a release. You can split the release branch of, feature freeze it and still continue other work.

See the blender release process: https://wiki.blender.org/wiki/Process/Release_Cycle

_____________________________________

What I generally want from a version control system:

Since patches tend to stay uncommitted for quite some time, it should be easy to rebase them. And maybe its my fault, but I never found out how to do that easily in svn. The branching system of git on the other hand does that quite well in my experience.

As a windows user I would like to keep the autobuild.

___________________________________

General thoughts not specific to only version control:

It hugely annoys me that there are at the moment so many different places (trac, forum, irc, phab) where stuff is discussed and where information is scattered (and more or less outdated). Whatever option is chosen should combine them (probably still +forum +irc).

 

4 hours ago, Stan` said:

• Already used by most (if not all) mods

^ this. huge pro for git. It is just the standard.

____________________________________

General thoughts on self hosting vs relying on third parties:

I get the whole point about reducing the dependency on big companies and honestly, at the end of the day I don't really care as I won't be the one maintaining it, but there are a few things to consider imo.

Yes the companies may shutdown their platform without any warning and we couldn't do anything about it. But how realistic is that?

Self-hosting means that there still needs to be someone in the team who puts in the work maintaining the server, getting new updates, doing other sysadmin stuff....

Sure, this reduces the potential risk, but it binds development resources that are from what I know very sparse right now. So is it worth it? Imo no.

Edited November 25, 2021 by maroder
typo

[Locynaeh]

Agreed for the git pros:

• clearer use of branch (the huge one for me) and release tags
• remote
• being THE main standard
• The submodules possibility

1 hour ago, maroder said:

Yes the companies may shutdown their platform without any warning and we couldn't do anything about it. But how realistic is that?

Self-hosting means that there still needs to be someone in the team who puts in the work maintaining the server, getting new updates, doing other sysadmin stuff....

Sure, this reduces the potential risk, but it binds development resources that are from what I know very sparse right now. So is it worth it? Imo no.

I think the shutdown argument is a strawman. The problems are rather in my view:

• the use conditions of the platform
• the users'personnal data management by the company
• how much you're stuck with the proprietary solution

So my view is that self-hosting is a good solution over these points but using, for example, an instance of the community (FLOSS) version of GitLab from a provider is OK :

• Migration is possible in another instance of GitLab Community, self-hosted or not
• You're not stuck with the proprietary management of issues and all on GitHub for example.

I said it for GitLab but the same goes for any git management FLOSS solution (Gitea, Tuleap, etc.).

Edited November 25, 2021 by Locynaeh

[Feldfeld]

2 hours ago, Freagarach said:

No branching (https://www.freecodecamp.org/news/why-you-should-not-use-feature-branches-a86950126124/).

I only read the article in diagonal but from my small experience, the current svn workflow with patches which are sometimes not trivial and get merged at the end, is not a solution to the problem this article mentions. I also think the current svn workflow is no different than doing git branch and rebasing at the end before merging (with amending every commit if we want stricter equivalence)

[nani]

4 hours ago, Freagarach said:

• Partial checkouts. (And access control.)

https://git-scm.com/docs/git-sparse-checkout

[Stan`]

Quote

THIS COMMAND IS EXPERIMENTAL. ITS BEHAVIOR, AND THE BEHAVIOR OF OTHER COMMANDS IN THE PRESENCE OF SPARSE-CHECKOUTS, WILL LIKELY CHANGE IN THE FUTURE.

This is fun   But I didn't know it existed so thanks

Quote

From IRC today Freagarach asked if we could edit fork pull requests to make minor changes to be possible. It apparently works with Gitlab.

https://docs.gitlab.com/ee/user/project/merge_requests/allow_collaboration.html

[artoo]

3 hours ago, Locynaeh said:

the use conditions of the platform

That's what I find one worrying aspect.

The other would be you depend on 3rd party services, GL or GH, if its down, its bad for the project, and there is nothing you can do about it, but wait.

[Freagarach]

5 hours ago, maroder said:

Yes the companies may shutdown their platform without any warning and we couldn't do anything about it. But how realistic is that?

It is mostly this kind of stuff:

 

Thanks for reacting on my personal preferences. Please don't think I hate git, or admire SVN. I just want an educated decision to be made and not follow along with whatever is popular.

If we're good with the binaries (I have no experience in that) I don't mind (too much) a change in workflow.

Regarding the supported platforms: Redmine supports SVN? (As do RhodeCode, OpenProject, pach.no.)

[smiley]

10 hours ago, Freagarach said:

No branching (https://www.freecodecamp.org/news/why-you-should-not-use-feature-branches-a86950126124/).

If a team tried making different changes to overlapping code, they would know that trunk based development, especially the model this article suggests, which is pushing to main often won't do. Instead of ugly merge graphs, you get a ugly history. Large changes take weeks and months, not 3 or 2 days. Try keeping track of all that without using branches.

When I wrote patches here. I had to clutter the whole workspace with multiple patch files, and svn revert, patch apply, repeat. good luck if someone wants to run an svn match while you were working on something. Instead of branches, everyone had a dozen copies which is clearly not better. This is a strong con of svn in my opinion.

Also being able to rewrite history is underrated. Along with git stash.

Hosting providers are not relevant to this discussion, you change remotes and push to another place if "Company Hosting Git Repo" shut downs.

Edited November 25, 2021 by smiley

[Lion.Kanzen]

to me the dependence on a platform that could close projects because they do not adhere to its policies.

Policies that change according to the convenience of states and companies at the cost of individual freedom.

[bb_]

I distilled a feature list from the posts above and will continue doing so. Please yell if I wrongdo anything here.

Some questions from my side:

8 hours ago, Stan` said:

Re binary files support could use this https://git-lfs.github.com/

How does that work with versioning of the binaries? And with clones?

8 hours ago, Stan` said:

• Co-authoring of commits
• Submodules? With different access

Can someone elaborate on these? what these are? why we would care? etc

10 hours ago, Freagarach said:

Pull requests?

I am not sure if I understand the exact feature one is after here. "Being able to say 'this patch is ready to be merged'"?

 

Lastly my own requested features:

First and foremost, I find that all code, art and anything else must be in one and the same repo. Obviously there must be a clear directory structure. But for some patches (the secondary attack patch #252, is just one example) it is required to change both code and art at the same time. Having to propose different patches for different repo's, will make life a lot harder.

We have been around a long time, the world around us will change  and we don't know what will happen. Therefore I will strongly plea for keeping sovereignty. Make sure everything is available in a self-hosted variant. Using third-party services is perfectly fine with me, however we should depend minimally on them. Whether it is to avoid a total collapse of the service, changes in the service itself, or changes in the terms, it shouldn't matter us. We should be able to continue, and at any point be able to decide to stop using any service.

To make things secure and reliable, I would recommend to have a third party back up which is usable as is, preferably using a different vcs. This will make sure that if our one server or vcs collapses, we can continue, using the other at will.

A VCS should work, for everyone from anywhere. There should be a minimum of potential restrictions on joining deving. This means that the vcs should work on many platforms, be free of (third party) account registrations, have a simple workflow with straightforward commands (also understandable for ppl who never used a vcs before).

Dev version testers should be made easy to test: we need them and they might not be as experienced as devs when it comes to handling code bases. Therefore precompiled binaries (autobuilds) should be directly available in the repo. This might also help devs on some platforms.

[artoo]

30 minutes ago, bb_ said:

Can someone elaborate on these? what these are? why we would care? etc

I can try to break it down.

 

Co authoring commits, there is an upstream repo, and there is a fork of that repo.

Anyone can fork a repo with git, which is easy to keep uptodate with the upstream repo that is forked, the remotes. You can author a patch in a branch, and send a request to the upstream repo, chose a target branch, master or some feature whatever. Then the merge or pull request can be reviewed, and merged according a selected merge strategy. Or the review determines the PR requires some more changes in order to merge.

This also works from a branch of the upstream repo, same concept, you can merge that branch into master branch or any other branch.

 

What all these git servers provide is more or less just a web UI to git. It can all be done locally too and then pushed, ie merge branch 'super_feature' into master, or merge branch 'hotfix-123' into master.

 

 submodules are git's native way to include another git repo. Many people, including myself find these a PITA, but there are different ways, including a native git only solution of just using branches.

In summary, it is a way to make from more than one repo one unified source tree.

Edited November 25, 2021 by artoo

[maroder]

@bb_ I agree with many of your points. Sovereignty and the option to be able to switch the platform if anything with a certain provider goes wrong, are important things. Also the stuff @Freagarach and @Lion.Kanzen mentioned about people loosing access due to police changes. That sucks.

But then again, what are the odds that actually happening/ having an impact on the currently active team members and also as @smiley said:

54 minutes ago, smiley said:

Hosting providers are not relevant to this discussion, you change remotes and push to another place if "Company Hosting Git Repo" shut downs.

 

Despite this point, I think we need a more differentiated discussion on the different aspects that influence the choice of a version control system. (Or create different threads, whatever is preferred).

Because it is not only a question of: What system has which features. Other options / question are bound to the choice of hosting solution/ provider:

Should the code hosting, documentation/ wiki and project/ feature planning be on one platform?

Here I think Gitlab would be the best option. It is actually quite decent in the whole project managing department as well as code hosting and CI.

I heard good stuff about OpenProject @Freagarach mentioned, but their focus is more on project management and less on code hosting/ review.

How do we want to handle account registration/ third party data collection and stuff you mentioned?

I can not argue with that. If that is a heavy concern than there is no way around self hosting. Personally I wouldn't care that much. Just wanted to mention that keeping control means putting in maintenance work and binding human resources.

What does the choice of hosting provider mean for the barrier of entry and the possible pool of contributors.

This would be the biggest argument for GitHub. It has an extremely large and active community of developers.

_________________________________

Something to the cons of svn: to move / rename files is majorly annoying (to me). Git seems to handle that way better.

 

[Samulis]

Some input as a non-programmer:

My main experience is with Git, primarily using GUI-based tools like Github Desktop. I have not used much of any SVN.

I generally think a lot of this comes down to a question of accessibility and features. Obviously with an open source project it is important that it is accessible, even to novices and less-technical people.

I think anything that requires command line/text-based use is too advanced for many non-programmer contributors (art/audio), so I would encourage that we consider an option that has some kind of at least somewhat functional GUI-based option. Yes, command line is not really that scary and even a few hours of digging around the file system and doing basic stuff in there is enough for it to become second nature and very simple in retrospect. However, there are many people who are truly afraid of using it or have such a limited understanding of basic command line use that it is genuinely dangerous for them to be poking around on their own. I think the last thing we want is to have to spend a hour or more giving each new contributor a tutorial on how to submit changes.

On the other hand, if something is so easy that even a total beginner can contribute to it without any kind of technical hurdle, it may lead to a risk of lower quality code or assets being contributed. In the current state, I feel like the use of SVN acts as a very minor gatekeeper and encourages less experienced potential contributors to spend more time on the forum. I have had occasional issues where people have submitted unusable content in merge requests to projects I have run on GitHub and had to decline them.

Regarding sovereignty, I definitely agree that we should maintain some kind of local version, if not the main working version, on servers we control. Although I do not think a place like Github or similar is going to go 'poof!' one day, I have had occasional problems in the past with companies deciding to stop running services I rely on as essential and do not think it is healthy to keep something this large and important in one place, especially one controlled by another party.

I can also see how using the same version control system as most mods might be advantageous as well. It might make it easier for those who have just started to get the hang of modding the game to then contribute directly to the game.

I'll leave the actual deciding to those of you with more experience, but these are just some thoughts which may help provide a different perspective.