Planned Disruption - Migration to git and Gitea

[real_tabasco_sauce]

14 minutes ago, Stan` said:

I don't think it's possible but maybe as they say here with github actions https://forum.gitea.com/t/automatically-update-forks/7046

Hmm, I ran across that earlier as well. However, I did find some nice bells and whistles in VS that seem to streamline the workflow well! So I'll be able to start adding my existing patches as PRs.

[hyperion]

Add 0ad/0ad repo  as a (default) remote and you never have to update your fork

[Itms]

Hello there! After fixing a very nasty network issue yesterday, I was able to publish the first nightly build this morning!

This officially concludes the migration, all outstanding issues have been fixed. There will be more discovered I'm sure, please report them here along the way. It will also take some time for the development flow to adapt to the new environment. I will improve some documentation and prepare posts to reach a wider audience about the migration, over the weekend.

In a nutshell:

• If you are a regular player just regularly testing the latest development version: you should ditch your svn copy and instead clone the new "nightly-build" svn repository at https://svn.wildfiregames.com/nightly-build/trunk. Use "svn update" as usual to get the latest changes daily.
• If you are used to compiling the game (either because you are a Unix user, a maintainer, or interested in contributing): please note that the build process has changed a bit, you now have to get/build libraries using the correct script under libraries/, before the usual update-workspaces and subsequent build.
• If you were working on something on Phabricator: please port your work to a new PR on Gitea, post a short message on the diff you ported, and close it.
• If you have some work in a github/gitlab clone: the new git repo is not compatible and can't be added as a remote. You should export your work (for instance with git-format-patch) and reimport it in a fresh clone of the new repo. You can then ditch your clones of the github/gitlab repo, which will not receive updates anymore, and will be archived soon.

[Dunedan]

10 hours ago, real_tabasco_sauce said:

So we will fetch upstream and rebase each time we want to make a change? It seems like a bit of a chore compared to svn and I'm sure I will mess that up frequently. Although, maybe i'll get used to it.

You don't have to fetch upstream for every change. Just do it as often as you'd do "svn up". The only situation where you will need to do it is when you want to merge a pull request and there are conflicts between the current status of the upstream main branch and your feature branch. If there are no conflicts and you merge using the Gitea web interface, you don't even have to rebase manually, as Gitea will handle it transparently for you.

[Feldfeld]

Is this Gitea instance suitable to host mods?

[real_tabasco_sauce]

6 hours ago, Feldfeld said:

Is this Gitea instance suitable to host mods?

Well it should probably need to be a separate instance though right? You don't want a lot of unnecessary files. I am working on a couple of mods and two of them use GitLab, which i've found pretty intuitive and easy to get started on.

The nice thing is you can just clone the repository you are working on right into your mods folder. If you host feldmap for future development, i'd be happy to contribute!

Edited August 25 by real_tabasco_sauce

[Stan`]

I've ported the community mod over as it is a bit of special case. But for other mods I'll let @Itms decide. Generally speaking I think you have a max repo limit at 0 by default.

[real_tabasco_sauce]

4 hours ago, Stan` said:

I've ported the community mod over as it is a bit of special case. But for other mods I'll let @Itms decide. Generally speaking I think you have a max repo limit at 0 by default.

Really? Nice work! Where/When did you do that?

https://gitea.wildfiregames.com/0ad/0ad-community-mod-a26

cool! its nice that these will be kind of centralized. Maybe we can make a branch a27, make it compatible with 0addev and merge it over at some point?

That would be pretty slick.

^Although there would be a lot of file overlap, as in some cases, community mod features are just a27 stuff added for testing.

Edited August 25 by real_tabasco_sauce

[Stan`]

1 hour ago, real_tabasco_sauce said:

cool! its nice that these will be kind of centralized. Maybe we can make a branch a27, make it compatible with 0addev and merge it over at some point?

Yeah possible. We could also have an ongoing PR to the main repo which mirrors this repo in the future. Then when it time to merge you don't have to worry too much since you rebased.

 

[real_tabasco_sauce]

@Stan` how will the gitea version be set up for releases? Will there be some actions related to signing and uploading to modio? There are a handful of things I could put together into a new community mod release.

[Stan`]

34 minutes ago, real_tabasco_sauce said:

@Stan` how will the gitea version be set up for releases? Will there be some actions related to signing and uploading to modio? There are a handful of things I could put together into a new community mod release.

Works the same. PR does XML validation for changed files, push on main does XML validation for all the files, push on signed pushes to mod.io.

The only thing I could not replicate was the release creation. For that it seems Gitea needs a tag to be created as trigger.

I also added a few linters following @Dunedan's example. The last thing missing would be checking for missing files using checkrefs.

Another improvement is using the correct mods to package the community mod so images are now compressed correctly.

[andy5995]

On 22/08/2024 at 1:51 AM, real_tabasco_sauce said:

So we will fetch upstream and rebase each time we want to make a change? It seems like a bit of a chore compared to svn and I'm sure I will mess that up frequently. Although, maybe i'll get used to it.

Is there really no way to just automatically update the our fork with 0ad/0ad?

@real_tabasco_sauceI think I can offer some useful tips about this. I haven't used git much the last few months so I'll do my best. I hope someone will correct me if I'm mistaken about anything or omit anything.

I don't think you'll have to fetch and rebase each time you want a make a change. While working on a separate branch:

1. If no new commits have been made to the main branch, you'll be able to just continue working on your branch

2. If you know that any new commits to main do not include any files you are working on. If a file has been edited that you're working on, rebasing will help prevent merge conflicts later. You may find it's not always strictly necessary though.

3. If you know your code changes won't affect or break any new code changes that have been committed (or vice versa), then your branch doesn't need to be up to date with what's in the main branch.

Sometimes you won't know for certain or it may take time to check, and you'll just want to rebase, or someone may suggest that you to "rebase your branch" if they're aware of a branch you're working on (for example, if you have an open PR that you're still pushing to and isn't ready for merging yet).

 

On 22/08/2024 at 3:20 AM, hyperion said:

Add 0ad/0ad repo  as a (default) remote and you never have to update your fork

In this case, you'll be able to skip the "fetching and merging into your local fork", but you'll need to switch to the main branch, pull, checkout your branch, then rebase (unless you run into the conditions or similar ones like I mentioned above.

[real_tabasco_sauce]

I found a workflow that works pretty well for me. I can do a 'git remote update' to get the new changes to the master repository and then make my branches based directly on the remote. 

[wowgetoffyourcellphone]

Really trusting everyone knew what they were doing when the decision to go this route was made. Seems twice as complex as before, although I've managed to find an okay workflow today. Editing files in the SVN repo for testing, then having to copy them over to my Git fork, then pushing those changes to Gitea is a pain. And like R_T_S, I'm not sure how to keep my fork utd with the 0ad/0ad main. And I've already edited meshes in the Git repo accidentally, thinking I was editing the SVN mesh, and then testing the mesh in Atlas and being confused as to why it's not working as expected, oh yeah, I need to copy the file over from Git to SVN to test it (lolz! mah bad). It's alright, I think I'll eventually be able to keep it all straight, especially with Stan's help.  

[hyperion]

4 hours ago, andy5995 said:

In this case, you'll be able to skip the "fetching and merging into your local fork", but you'll need to switch to the main branch, pull, checkout your branch, then rebase (unless you run into the conditions or similar ones like I mentioned above.

There are many ways to setup remotes. As for rebase-ing the active branch a simple:

git fetch
git rebase origin/main

will do. Here I obviously assumed origin points to 0ad/0ad. Also demonstrates why you probably never will use pull. Actually you will even find plenty on "git pull evil" on the net.

The term local fork can also be misleading/confusing. There are three repositories in the pr workflow: 0ad/0ad, <user>/0ad, and local. The term fork I'd only ever use for meaning <user>/0ad, just to avoid possible conceptual issues.

A simple setup could be having only one remote configured in your local repo but setting url to 0ad/0ad and pushurl <user>/0ad. Let's assume the sole configured remote is origin.

git remote set-url origin https://gitea.wildfiregames.com/0ad/0ad.git
git remote set-url --push origin https://gitea.wildfiregames.com/<user>/0ad.git

This makes commands simple stupid, but has it's own drawbacks.

Recommended reading https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes

[real_tabasco_sauce]

What's a good way to test someone's PR? Do I check out my own branch based off of their fork's branch?

[andy5995]

On GitHub, I save my work (if I'm on a different branch) and checkout main, then

git fetch origin pull/216/head:pr216

which creates branch 'pr216' (based on changes from pr 216), then I check that out.

 

Edited September 1 by andy5995

[hyperion]

5 minutes ago, andy5995 said:

On GitHub, I save my work (if I'm on a different branch) and checkout main, then

git fetch origin pull/216/head:pr216

which creates branch 'pr216' (based on changes from pr 216), then I check that out.

 

"pull/216/head:pr216" is a called a refspec https://git-scm.com/book/en/v2/Git-Internals-The-Refspec while you don't have to checkout main first, a dirty branch won't work indeed, either commit or stash https://git-scm.com/book/en/v2/Git-Tools-Stashing-and-Cleaning or reset https://git-scm.com/book/en/v2/Git-Tools-Reset-Demystified first.

[real_tabasco_sauce]

Hmm so i've found an unfortunate property of VS:

VS automatically hides things from the gitignore in the file explorer. Now this is mostly fine when it's just files, I can click "show all files" and all the hidden files, including ones needed to build the game, are revealed.

I can then build the game. However, the pyrogenesis.sln is hidden in the solutions page:

which means one must navigate to build/workspaces/vs2017/pyrogenesis.sln each time to build the game.

If I comment out "build/workspaces/vs2017" from the gitignore, pyrogenesis.sln appears in this menu.

Its not a huge annoyance, but it would make things a lot smoother if I could just turn off "hide gitignore files" like it seems can be done in VS code:

https://stackoverflow.com/questions/77939177/how-to-prevent-visual-studio-from-hiding-files-and-folders-in-gitignored

Edited September 1 by real_tabasco_sauce

[Stan`]

That should have happened with SVN but did not because VS doesn't support SVN

 

[real_tabasco_sauce]

1 hour ago, Stan&#x60; said:

That should have happened with SVN but did not because VS doesn't support SVN

 

What should have happened? pyrogenesis.sln being hidden? Frankly, I liked that it was so readily accessible there.

[Stan`]

Yeah it was also svn ignored. Well it's parent folder was.

Honestly once I opened it in Visual Studio, it was always in my recent projects and solutions. Are you saying it doesn't show up there ?

I almost never use the folder view only the solution explorer.

[real_tabasco_sauce]

47 minutes ago, Stan&#x60; said:

Yeah it was also svn ignored. Well it's parent folder was.

Honestly once I opened it in Visual Studio, it was always in my recent projects and solutions. Are you saying it doesn't show up there ?

I almost never use the folder view only the solution explorer.

Im not sure what part is "recent projects and solutions"

Just to be sure, I am using Visual Studio not VS code.  I can select it from the startup dropdown:

It doesn't really do anything though, as I still need to open it. Its not that big a deal, I guess I was just used to switching between solutions and available views.

[Stan`]

Oh I've never used it that way that's why. When Visual Studio starts I only open the pyrogenesis.sln file since I'm never gonna need any of the others.

Recent projects and solutions is in the File dropdown

 

For JS i usually place myself at the mod json level. (and use vs code)

[Itms]

On 01/09/2024 at 6:19 AM, wowgetoffyourcellphone said:

Really trusting everyone knew what they were doing when the decision to go this route was made. Seems twice as complex as before, although I've managed to find an okay workflow today. Editing files in the SVN repo for testing, then having to copy them over to my Git fork, then pushing those changes to Gitea is a pain. And like R_T_S, I'm not sure how to keep my fork utd with the 0ad/0ad main. And I've already edited meshes in the Git repo accidentally, thinking I was editing the SVN mesh, and then testing the mesh in Atlas and being confused as to why it's not working as expected, oh yeah, I need to copy the file over from Git to SVN to test it (lolz! mah bad). It's alright, I think I'll eventually be able to keep it all straight, especially with Stan's help.  

I think you'll grow used to the new fork/update workflow after a few tries. That part is not really more difficult than what we had with SVN+Phabricator, it's just very different.

However, editing/testing across the SVN and the git repo does sound like a pain, I still think it's a valid workflow for small one-time contributions, but it's too impractical for team artists. And we're not going to require you to learn and build the game.

I'm going to add a script for copying the nightly windows prebuilt files into the git repo. That way you would be able to modify in the git repo and test there without back and forth. I already did that for pulling translations and spirv shaders, I could also do that for the built game (it would obviously be Windows-only, but that would cover team artists).