Jump to content

A new git-based development environment


Recommended Posts

On 11/07/2024 at 3:41 PM, Itms said:

Yes that's one of the things I want to setup next week :secret:

Did you get around doing that? :)

Btw what's the plan for the two mirrors on Github and Gitlab ? Will you just force push on their respective repos ? We should also probably migrate all the repos there to Gitea.

Does that work with LFS?

Should we provide a Codeberg mirror ?

 

Link to comment
Share on other sites

8 hours ago, Stan` said:

Did you get around doing that? :)

Ha, with the translations thing, past week became this week ;)

8 hours ago, Stan` said:

Btw what's the plan for the two mirrors on Github and Gitlab ? Will you just force push on their respective repos ? We should also probably migrate all the repos there to Gitea.

No, I think it's better to rename the current repos to something like old_git-svn_mirror in order not to break forks, archive them, and but up a big bold ugly notice about the migration and the new repo.

People will be able to migrate their repos when they wish to Gitea, I'm not doing it as part of migration. By the way I think the mirroring to github/gitlab should wait for a couple days after the migration.

8 hours ago, Stan` said:

Does that work with LFS?

Yes, I'm currently testing a mirror-push to Github, and seeing how it takes times, it's definitely uploading the LFS contents.

8 hours ago, Stan` said:

Should we provide a Codeberg mirror ?

Great idea! I just reserved the 0ad name (y)  I can add you as owner if you PM me your Codeberg username.

Link to comment
Share on other sites

1 hour ago, Itms said:
10 hours ago, Stan` said:

Does that work with LFS?

Yes, I'm currently testing a mirror-push to Github, and seeing how it takes times, it's definitely uploading the LFS contents.

Oh... GitHub and GitLab both have a low LFS usage limit on the free tier (1 GiB and 5 GiB respectively). It is one more proof that we needed self-hosting. However that creates issues with mirroring, that's bad. :wacko:

Link to comment
Share on other sites

1 hour ago, Itms said:

No, I think it's better to rename the current repos to something like old_git-svn_mirror in order not to break forks, archive them, and but up a big bold ugly notice about the migration and the new repo.

Good thought about forks.
Maybe we should add a Contributing.MD file on those mirrors, so that people know they should contribute on Gitea?

1 hour ago, Itms said:

People will be able to migrate their repos when they wish to Gitea, I'm not doing it as part of migration. By the way I think the mirroring to github/gitlab should wait for a couple days after the migration.

I mostly meant community mod, https://gitlab.com/0ad/0ad-community-mod-a26
This one is no longer used https://gitlab.com/0ad/lobby-moderation

And the following

https://github.com/0ad/lobby-infrastructure
https://github.com/0ad/lobby-bots
https://github.com/0ad/play0ad.com
https://github.com/0ad/wildfiregames.com

 

12 minutes ago, Itms said:

Oh... GitHub and GitLab both have a low LFS usage limit on the free tier (1 GiB and 5 GiB respectively). It is one more proof that we needed self-hosting. However that creates issues with mirroring, that's bad. :wacko:

I know Gitlab has an opensource tier we are not subscribed to, not sure about Github.

Link to comment
Share on other sites

12 hours ago, Stan` said:

Btw what's the plan for the two mirrors on Github and Gitlab ?

I'd prefer if we don't continue mirroring after the migration. With Gitea we'll have an accessible platform for contributors and I'd argue the disadvantages of mirrors outweigh their benefits.

  • Like 1
Link to comment
Share on other sites

3 minutes ago, Dunedan said:

I'd prefer if we don't continue mirroring after the migration. With Gitea we'll have an accessible platform for contributors and I'd argue the disadvantages of mirrors outweigh their benefits.

Since we won't be able to actually LFS-mirror to Github and Gitlab, it sounds like stopping mirroring altogether is the best solution. I would then just archive the historical repos without renaming them.

Codeberg can allocate storage upon request, so I would still like to contact them after the migration, asking if we could have a mirror there, in order to show mutual support. Having a mirror on that Forgejo instance would also be a good way to keep in touch with the Forgejo community in case we want to replace Gitea or to stop self-hosting.

Link to comment
Share on other sites

1 hour ago, Dunedan said:

I'd prefer if we don't continue mirroring after the migration. With Gitea we'll have an accessible platform for contributors and I'd argue the disadvantages of mirrors outweigh their benefits.

That's fair. I hoped we could leverage some visibility. But it's hard to say whether we got none or people got put off by Phab / SVN until now.

 

 

Link to comment
Share on other sites

Here is a changelog of my work since last time:

  • numerous infrastructure tweaks for the new server
  • SPIR-V shaders are generated in the nightly build and can be downloaded in a git clone - this removes the need for a spirv mod entirely
  • added git hooks to prevent upload of binary files in case a user hasn't installed git-lfs; to enforce a linear commit history by default; added size limits to git uploads
  • finished all the CI pipelines for Linux, Windows, and macOS
  • finished the workflow refactoring for macOS
  • updated the wiki page for the CI/CD setup
  • fixed references to dropped commits in the history
  • addressed some remarks above as commented, looked into the translation alternative
  • disabled localization of Gitea, since English is supposed to be used for collaboration, and the contents of Gitea (main page, wiki, policies, ..) are not localizable anyway

I tested everything I planned to test on the server we will use, so I am ready to perform the migration on the 19th and 20th of August.

I have a few nice improvements I will implement in the next couple days (issue templates, code owners, and more).

On 21/07/2024 at 2:55 PM, hyperion said:

NIST banned the use of sha1 and I expect a strong push to migrate to sha256 object format the next couple years. The git object format is no longer labeled experimental and gitea supports it properly for all I know. Suggest to at least consider it.

I made sure that sha256 repos are available in Gitea after the migration (they are not available on my PoC due to an older git version). Unfortunately, reposurgeon, which is used to convert the SVN repo into a git repo, does not seem to support SHA-256. Reading the documentation, it looks like the tool internally relies a lot on SHA-1 to work. So I cannot generate a SHA-256 repo. git-filter-repo, with which I rewrite history afterwards, does not propose conversion from SHA-1 to SHA-256 either.

  • Like 1
Link to comment
Share on other sites

8 hours ago, Stan` said:

Bad news it seems wiki isn't supported as a rss feed :( 

https://github.com/go-gitea/gitea/issues/19071

It'a a git repo though so there might be something to do there.

Oh snap. I was wondering about that recently and forgot to check. I think it would be necessary to create a new issue that is clearly titled "Add RSS feed for wiki updates", because that meta-issue you link doesn't seem to get a lot of traction, and only that comment from a random user hints at our use case.

8 hours ago, Stan` said:

By the way what about the On groups ?

The what?

Link to comment
Share on other sites

3 minutes ago, Itms said:

The what?

O12 Art O7 catchall etc :)

3 minutes ago, Itms said:

Oh snap. I was wondering about that recently and forgot to check. I think it would be necessary to create a new issue that is clearly titled "Add RSS feed for wiki updates", because that meta-issue you link doesn't seem to get a lot of traction, and only that comment from a random user hints at our use case.

Okay will do :)

Link to comment
Share on other sites

3 minutes ago, Stan` said:

O12 Art O7 catchall etc :)

10 hours ago, Itms said:

I have a few nice improvements I will implement in the next couple days (issue templates, code owners, and more).

  • Like 1
Link to comment
Share on other sites

  • 2 weeks later...

Nothing has thwarted the migration project, so it will happen as planned during the upcoming two days :)

Here are the final improvements performed over the past weeks:

  • Added missing "Patch By" credits in a number of commits
  • Set up issue templates
  • Set up code owners
  • Devised an alternative way of tracking a release process (using issue templates)
  • fixed issue search after the Trac import
  • Like 4
Link to comment
Share on other sites

  • 2 weeks later...
On 21/7/2024 at 6:04 PM, Stan` said:

Right so you can only point to a specific submodule git repo a specific commit and a specific branch.

But you could have more branches in the containing repo. Anyway sounds like a mess ^^

branch is simply a commit in git. Super lightweight. Branch or tag just optional. Works with eg origin/main instead of maintaining your local “main”. git push origin <ref>:main.  Often ref is HEAD


Advantage of submodule is testability, repeatability. In master branch “main” or in release branch eg 0.1.0 whoever updates the pointer to the translation or lib ensures it is tested, working. Simplifying CI, CD / nightly

Other than this added transparency submodule can be extra maintenance, noise, confuse potential contributors

 

Con allowing forks. Not needed. Branch suffices (fork is redundant, adds unneeded complexity due to many remotes)

 

Pro extra translation git repo

 

Pro one repo per source lib

 

Pro no duplication (rather split functionality of i18helper if possible)

 

Can use param --pretty for short hash on git log

 

Overall very logical deeds and thoughts Itms.

Maybe no more svn valid wish to reduce deps. Move all repos to git later? Not very important but would be consistent.

Nice work anyway. Finally

And Stan: phoenix 

  • Like 2
Link to comment
Share on other sites

  • 1 month later...

I have a very limited bandwidth right now. Is there a possibility to just update my clone from the latest svn r28209 to the latest nightly without having to checkout the whole data again?

I am using Win 11 and when selecting the context menu of my folder C:\Users\JohnDoe\0ad_svn and navigating to TortoiseSVN > Switch... i can only switch to folders behind /public*. But I need https://svn.wildfiregames.com/nightly-build/trunk.

*https://svn.wildfiregames.com/public/ps/trunk/

Link to comment
Share on other sites

@Obelix Unfortunately, I confirm it is not possible :(  Those are two different repositories, with extremely different histories, even though they contain roughly the same files in the HEAD revision. When a repository is moved to another address, the "SVN Relocate" command will save you from having to re-clone, but this is not the case here.

On 29/09/2024 at 5:57 PM, Obelix said:

i can only switch to folders behind /public

Even if I had decided to create the nightly repo inside the "public" set of repos, using "SVN Switch" would have re-downloaded everything, as public/nightly wouldn't have any common history with public/ps.

  • Thanks 1
Link to comment
Share on other sites

3 hours ago, kkslidermight said:

You may want to read this:

https://forgejo.org/compare-to-gitea/

Hey thanks for reaching out. We're aware of forgejo, but the plugin we use for Jenkins was made for Gitea, not forgejo, and forgejo runners did not seem ready for use the last time we check. We're also following Blender, which as far as I know hasn't switched yet.

Link to comment
Share on other sites

Can someone reset password for "Dizaka" on https://gitea.wildfiregames.com/

I can make separate account with user "Akazid" if "Dizaka" can't be reset but would prefer a reset if possible.  I have no idea which email I used but if a password reset was forced to registered email I'd be able to respond.

@Stan`I remember a while back you were developing a webpage for sharing stuff.  Is that under development still / being worked on?

Looking for places/stuff to contribute to.

Edited by Dizaka
Link to comment
Share on other sites

Only @Itms can reset passwords as admin. You should be able to from the UI though.

4 hours ago, Dizaka said:

 

@Stan`I remember a while back you were developing a webpage for sharing stuff.  Is that under development still / being worked on?

It's still up on https://replay-pallas.wildfiregames.ovh/

It's still missing a piece with the last graphics of local ratings. But what I'm missing most is people uploading replay to make the ratings more accurate.

I haven't ported them to Gitea but you can see the code here:

https://github.com/StanleySweet/replay-pallas

And

https://github.com/StanleySweet/replay-pallas-api

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...