Itms

Hello, what is your main account? I can see and check if your IPs match. Or else you'll have to find another way to prove that you are behind the other account. If we have proof, we can merge your accounts without problem, but else we obviously won't do it

@Dunedan Thinking about it, if you want to give a hand, you could try and take a look at creating that translations repo. You should have access to create repositories on Gitea under the 0ad/ organization. Right now I'm focused on shaders and other Jenkins improvements, I'm not going to look at translations until next week.

If the exported translations are not committed, that feedback does not actually happen... Do you think it could create actual issues? Yes, the first one isn't possible. The second one I hadn't considered but maybe it would be nice. I am thinking this "translations" repository could even allow us to store the translations of official/semi-official mods. Those would not be pulled into the git and the nightly-build of 0ad, but they could be synced with Transifex, allowing our translator community to translate them. The modders would then pull translations into their mods when they release them, and push pot files to the repository whenever they wish. For the third one, I answer you further below. Well it wouldn't be possible to review changes to the libraries themselves, but changes to the 0 A.D. code would still be reviewed and even tested by CI. This is indeed problematic but it is the same situation as today. Take for instance D5002: the contributor cannot publish the SM upgrade itself inside that diff, the diff only consists of the changes in the game. Right now that situation is too complicated for our CI pipelines to handle it (arc patch doesn't work well with binaries) thus the build for those diffs always fail and make much noise. With git, it would be possible for a contributor to send big binaries in their PR for initial review of the whole. Then we could commit the update to source-libs and have a final round of CI on the final PR that would just update the SVN revision in libraries/ scripts, before committing. So that addresses your concern. windows-libs changes are currently unreviewable, as they are just me or Stan sending new binaries on SVN. At least with the new system we could send the new binaries, then create a PR that just updates the SVN revision in libraries/ scripts, so that CI can run on it. So it's still a bit unreviewable, but it will actually go through CI before commit. ah that's right! Well, it is arbitrary, or at least based on feelings (but I think it's important to consider those). One of my issues is: those commits would lack justification. The history of the repo would contain a lot of commits that do not address an issue. Those would just be commits happening because "translations are updated all Mondays and Fridays". Moreover, those commits have the big downside of highlighting low development activity. It feels very bad to see that autobuild commits more than the team in times of reduced activity. This is awful for morale. My proposal allows us to have daily updated translations, which is better for testing, but if we keep translations in the git repo, having daily updates would dwarf actual human contributions. Hence, if my current system really irks you, your second proposal above is the only one I would find acceptable. Basically I think that the issue with spir-v and with translations is the same: we have files that are needed for the end user (including nightly testers) but which cannot be committed to the git repo (for translations, replace "cannot be" by "I would rather have not"). I figured that if I generated shaders in the nightly and provided a way to export them into the git repo if developers needed them, I could just have the translations follow suit, and fix two problems with the same consistent solution. You can contribute PRs to the private dev-migration repo on Gitea. But I agree it's a difficult project to collaborate on Your thoughtful feedback is much appreciated though By "large" I was thinking about translations (and "very" with shaders on top) but you're right that going from 100MiB to 700MiB would probably not bother users too much (I, on the other hand, would be sad to miss that opportunity to reduce the size by such a factor!) and I hadn't indeed realized that increasing the frequency of translation commits would not increase size that much. Thanks again for your feedback I'll try and think about the idea of a cleaner, separate repo for the translations.

By the way, since I also read about that on IRC: It is very likely that my urgent work after the migration will be a (partial or full) SpiderMonkey upgrade. Having handled several of those now, I am thrilled (that's not an exaggeration) about performing that future upgrade under the new git setup I designed, mostly because CI will finally be able to handle it. I think I'm a good judge of whether the setup is adapted to performing library upgrades. If I stumble on rough edges, that will be the opportunity to address them, and if I don't, it will be a good example in the repo history of how to perform such an upgrade in the new setup. To reiterate, maybe it would be even better with submodules, but I don't know enough about them to design a submodule-based setup myself. I'm not against using them in the future.

Thanks for the detailed point of view, I'll try to address that from my perspective. (I read on IRC about feeling sorry about giving me more work, don't be! I was aware of that possibility since I worked on my own on this migration. Some parts of it are a bit opinionated: I tried to setup my own ideal setup. That said, I am not against anything in principle, but I'm not going to overwork myself: whenever I am convinced by a proposal, I'll perform it with pleasure, but if I'm not, I'll let someone else do it) My number one expectation is: ditch Phabricator (well, they ditched us first, as implodedok put it). Stan hinted at that above. We need to continue having a collaboration platform, and Phabricator was the only big one that provided support for SVN. But apart from that, I have nothing against SVN. It's still a very usable tool, with many upsides, and as long as we had a forge allowing us to collaborate, we could do without branches and PRs. But it's not the case anymore, so we have to switch to git. You (and others) dream of stopping using SVN entirely, but I also had to convince people who don't want to use git at all. I tried to find a middle ground here and tried to exploit the main advantages of both tools, and I'm quite happy with the result. I disagree with that part. You'd need to have subversion installed on your system (just like now), but you wouldn't have to interact with it anymore. You'd just run the libraries/build-source-libs.sh script and be done with it. No, I added a source/tools/i18n/get-nightly-translations.sh script to export the latest translations into your git clone. It was a valid concern though, bb was the one to bring it to my attention first. That is true, but the majority of devs will not have to interact with those repositories. The upgrade of libraries is fundamentally non-collaborative: me and Stan are the only ones who may upgrade Windows libs these days; you can add wraitii and s0600204 to the list for SpiderMonkey upgrades and that's all. Being one of the involved devs, I agree that the new setup is not a huge improvement over what we did on SVN, but it feels way more organized and streamlined to me. I never used git submodules, so basically: I wouldn't dare trying to setup that I have nothing against it in principle, but I had not thought of submodules before yesterday. If I'm not mistaken, this is a change that could be performed in the future without rewriting history, right? It could be an incremental improvement over the future setup, that could be performed by a knowledgeable contributor. Would that be acceptable to you? same as above, but I believe the directory structure of the repository would make that much harder it would create similar problems to the engine/mods split I think, I'm not sure. That, on the other hand, I am against. The history of po/pot files in the git repo, when I started my experiment, was HUGE. It was several times larger than the current size (LFS blobs excluded) that I got to. (I can't remember how much, but off the top of my head 600MiB) And even that is not satisfactory because you only get new translations twice a week. With the nightly-build system, testers obtain everything "as in a release" daily, which is a bit improvement! Of course, committing translations daily to the git repo would be possible, but the size would skyrocket. Generally speaking, I'm also a bit adverse to letting a bot commit to the source repo. It makes more sense to me to have a git repo managed entirely by humans, and then a repo generated by the CD system, which provides a ready-to-play state to testers. Let me link once again to the glorious diagram I'm really proud of in which it looks like concerns are cleanly separated. But the main thing that convinced me to strip the translations from the git repo is the SPIR-V shaders for Vulkan. Right now we are blocked with a Vulkan support that depends on a mod which cannot be committed because of its size, the generated nature of its contents, and the huge amount of time/computing needed to generate it (and I believe that's part of the reasons for that stalled A27, as including the mod would complicate the release process). Vulkan cannot be tested out of the box by users without some knowledge. I want to demonstrate next week (unfortunately I can't show that to support my claim yet) that this can be fixed by my nightly-build approach. The git repo does not have to contain those large and ever-changing shader files. Instead, the nightly build generation can provide those files daily, with the CD system taking care of generating them, and we'd have them ready for releases in advance. I only have to make sure that the generation can be efficient (we mustn't spend 5 hours generating them every night, but only when needed), and that will be my task for the next week. --- In a nutshell: 1. I don't think SVN is so bad that we should avoid using it entirely. 2. I am open to replacing my source-libs and windows-libs SVN repos with git submodules, but that can be done in the future, and I'd be happy if someone else would do it. 3. I believe that a nightly build for testers and for generating release bundles, generated from a lightweight git repo for the developers, is a superior approach than a 1:1 SVN mirror of a (very?) large git repo.

Since that is for the lobby of the 0ad game, it would go to 0ad/lobby-infrastructure which would be consistent with GitHub.

Having a git forge would be a good opportunity to start versioning some of our staff information, so I went ahead and made the change: that way, repositories associated with the development of the game go to 0ad, while we have the wfg organization for actual organizational matters. For instance, infrastructure documentation for the server as asked by Dunedan. SPI-related stuff should also go under wfg/ We should also really look into improving our websites, and thus we should have, when this project starts, a wfg/website repo for wildfiregames.com and a 0ad/website repo for play0ad.com. Does that sound logical?

Yes that's one of the things I want to setup next week

While we are breaking paths: I'm not sure wfg/0ad is a good repo path on Gitea. It's inconsistent with github and gitlab (where it's 0ad/0ad). I see that blender made the same decision and called the organization "blender" on their own gitea. So I'm pondering whether to rename the "wfg" organization to "0ad" (I'll probably keep "Wildfire Games" as the real name). Does that sound good? I don't like it much but it may objectively be better.

Thanks for the feedback! It's in the staff forums. I am ready to update those threads as soon as changes happen. It would indeed be nice to store them in a repo, for instance a private repo on Gitea after the migration I do like originality, but this is a very good point. I'll change trunk to main. Indeed. I suppose they found a way to comply though, or else the service wouldn't work anymore. I'll disable Gravatar then. I'm not going to perform extra work for that. Users will have to fill their profile themselves on Gitea. Yes, that's nice to have! However I would like to avoid adding and maintaining too many devtools in the future If Gitea could become our one and only centralized platform for development that would be great.

Thank you for your interest! I sent you a PM with login information and some pointers. Please go through the wiki and tickets, test and update stuff as you wish and let me know if you run into an issue.

Hello folks! I have been hard at work on this, and things are starting to look usable. Here is a (probably incomplete) changelog: The CI works! It is still a bit raw and I deactivated it to save space on Jenkins, but I am now working on improving it. It is the last big chunk of work I have to perform, and then we will be ready to migrate. The nightly build now exists! Get it via SVN at https://svn.itms.ovh/nightly-build/trunk. It is generated from a month ago but I'll update it through Jenkins in the upcoming days. All the documentation was updated and improved during the JDLL event in May where I presented the project and received some feedback. I also improved the Privacy Policy based on feedback. Please head over to the wiki especially BuildAndDeploymentEnvironment. I need help! The glorious FAQ has been defaced by the conversion to Gitea, which uses Github-flavored Markdown. Is anyone interested in working on manually restoring the appearance of the FAQ, adapting it to its new home on Gitea? Please let me know if you wish to help. Basically it would be necessary to 1) cleanup the structure of the page, fix the tables, the raw HTML and hard links and 2) improve the appearance just like it was done on Trac, but using Markdown features instead of Trac features. The git repository now has a script for getting translations straight from the nightly build (the same will be done for Vulkan shaders) Links to changesets in the format [25001] on Trac are correctly converted to 04ec75ed7e on Gitea (and I fixed the commit text in that specific revision) Important Trac keywords (regression, pathfinding, design, ...) now have a Gitea label, but I can't easily automatically convert keywords to labels, so I will add the labels manually after the migration. Thanks in advance for your feedback!

Not implemented yet, but close. Sent. I'm almost ready to start CI (hopefully) and I'll use the checkrefs script as my main test avenue. It will allow me to check that Git LFS interacts correctly with the CI. Once I have set things up, I will be very interested in having those changes in a pull request on gitea. This is automatically covered by the Gitea Jenkins plugin. Jenkins uses the Jenkinsfile from the upstream branch when the pull request author is not a recognized contributor. If it works as advertised, this will remove the need to have access to the Jenkins instance.

Hello @plusmid! I know about Forgejo. It's a great project and you're right to promote it. Basically we want to use and support open-source software, but we'd also like to use widely-used, even if it's commercial, software in order to save energy in maintaining our tools. A lot of members of the community have been advocating for using Github, which everyone uses, and which is "too big to fail". Obviously we're reluctant to use Github/Microsoft closed software, but maybe we'll end up there anyway if it proves too difficult to self-host a forge. Gitlab is too heavy for us to self-host, Gitlab.com is another alternative possibility for the future. I think Gitea is a very good middle-ground, both open-source, self-hosted and Github-like. It has official maintained support from the Jenkins developers (we use that for CI/CD). We're very happy to know that there is a non-profit alternative in case the Gitea firm starts doing shady stuff, but for now, the only thing we want from upstream is stability and ease-of-use. Forgejo is simply not big enough yet, to put it bluntly.

I regenerated the repo and associated data (that broke some links, I can't do much except rewiping the databases and users, which I'll try to avoid doing too frequently). I'm going to stop changing/fixing bugs in the PoC for now, thanks for all the reports and testing. I am now going to focus on the Unix build environment and the setting up of CI/CD. Changelog: git repository: - used short hashes in messages (I didn't have to choose the length, git automatically chose 10 chars) - wrapped the content of commit messages to 72 chars, ignoring the first line - grouped metadata lines in commit messages as much as possible (my regexp detects "[...] by:" and "Differential Revision:" but not for instance Trac tickets:, I can't cover and test all cases) - handle phab: links and [Pp]hab: links to non-commits - added a Bugs herder team Imported issues: - updated label display (typography/color), removed activity labels - disabled time tracking - do not publish a Patch change if the new value is empty wiki documentation: - added notices to install git LFS - adapted contents to label improvement Cosmetics: - added link to docs.wildfiregames.com in top menu Future branch: - merged Stan's update to .gitignore In gitea 1.22 (currently RC, will upgrade when it's released): - command-line instructions to merge will be adapted to our merge strategies - username will be correctly used instead of real name in commit feeds (including the RSS consumed by the IRC bot) Didn't do: - rename TracUser, I really think it's important to be explicit about the fact content comes from Trac - keep the Trac registration date of users... because Trac doesn't record that at all (only the last activity date) - I cannot reproduce @hyperion problem with "issues created by me"

Why not! But that would have to link with the migration. They can be moved now or later, in both cases I'll preserve them. No need at all! Images can be retrieved from a remote URL, for instance on play0ad.com, or from the nightly-build SVN. It's okay, it's still kinda maintained, and as long as it doesn't break, we don't need to fix it. But there are indeed more popular choices for C++ test frameworks nowadays. That's also unrelated to the migration. My changes to the build environment will probably supersede those interesting diffs you linked.

Oh I misunderstood your previous message, sorry! I thought you were suggesting doing things, but you were questioning/proposing improvements on what I did. Please go ahead and make a PR The bat files are used in the Windows installer I think, no need to make sh equivalents when we have a .desktop file. Converting to Markdown is a must, but it would indeed be great to deduplicate things that go to the wiki. Instead it would be awesome to have a readme.md which is actually appealing, with screenshots and a user-orientated presentation of the project. I have not worked in that direction but I'd love to see it done. I have already planned to remove svn-related binaries, but I have decided not to remove premake and cxxtestgen. Those are very lightweight binaries, and keeping them really simplifies the Windows build process in the design I am proposing.

@Stan` Everything you listed in the latest message is either already covered or still WIP but planned in https://gitea.itms.ovh/Itms/0ad in what I called the "future" branch. Please test and propose alternative coloring if you have a strong opinion on the matter. That was not at all the aims I had in mind, I see how my wording was confusing. Please avoid deeming others' work "pointless" and being rude for no reason. I see that there is indeed overlap with other Gitea features. I wanted to make sure PRs without any reviewers would stand out. Instead of adding a "reviewer-needed" label, we could make a rule that reviewers assign themselves to PRs (which I hadn't realized is different from assigning one to an issue). That way, orphan PRs could be found with the "No assignees" filter. "under-review" is then already covered by the presence of assignees, and "work-in-progress" is covered by the "changes requested" display. This is a very good point, it would be great to be able to mark an issue as waiting on info without closing it. The "Due Date" feature would here be useful to keep track of when to close it. (still no found use for the Time Tracker which is a different feature) Well I could call that "closed". Order doesn't matter here since closed tickets should just have a "closed/resolved" and "theme" label. I disagree, there is an actual difference between "nice to have" (this would provide an actual benefit to the user or the devs, but low priority) and "if time permits" (this would be nice, but it either wouldn't change anything from the user pov, or the cost of doing it would counterbalance the small benefits). Then again, I just don't want to lose any information from Trac. Merging the labels can be done in the future if the team wishes.

Will take a look Another thing worth mentioning: I plan to delete all Trac accounts without content as part of the migration (using the Trac spam filter). I did not perform that in my PoC because I haven't set up the spam filter (useless in a read-only instance). As a consequence, I will only create Gitea users for active Trac accounts in the actual migration.

I would go with 10 to match the Gitea display, but maybe 12 would reassure me more. Scratch that, the Linux kernel uses 12, but that's overkill for us. I'll use 10. I am not planning to touch gitconfig in the repo. I specifically chose "theme" so that it is after activity/difficulty/priority (alphabetically). "topic" would also work. It is a less important label than those. I could prefix the label types with A/B/C like some projects do on Github, but I suspect you won't like it That's not how labels work on Gitea. I suggest you make your own label set in your test repository, so that you can see what the constraints are. What you call "key/value" pair labels are the "exclusive" labels, which is not the case for the theme. But reflecting on this now, Trac didn't leave the possibility to set multiple components for a ticket, so maybe I'm going too far by allowing multiple theme labels. Oh that's on me, I assumed that white space wasn't allowed. Thanks, I'll improve that then! (there is a possibility that the trac2gitea tool doesn't support it though, we will see) I kinda agree but right now all the labels are used, removing some would mean losing information from Trac. And I strongly believe we need the "activity" label or an equivalent, to triage PRs (and it needs to be the first label type, and to stand out) Interesting. That's stronger than preventing you to post. Oh that's easy to do, and I did it almost daily when testing my migration scripts I deleted your top-secret account. Also I unrestricted you, but I also have the option to prevent you from logging in, so I'm testing that. Can you confirm you can't login at all?

Yes I just restricted PoppaSmurf, I don't know how it appears to you. I can't contact users but I do have access to their email addresses (pretty much like in Trac). It is actually a good thing in my opinion that Gitea doesn't have a PM system. I would be unhappy so see communication even more split between platforms. Yes, I tested a while ago, it works. In the worst case I can debug that after the migration, emails can wait for a few days. I really don't want to test now and spam a wide public. I really had a hard time to decide between an organization and a user. Users need email addresses so I didn't really want that. Also I figured it might be handy to be able to add people to the organization if there is a need to manually edit imported Trac data... but that is realistically not going to happen. What do you mean about surprising? In terms of privacy you mean? (if yes, I tried to be precise in the privacy policy, and people who don't use Gravatar won't have their data sent there, so they are not supposed to complain unless I missed something). For the PoC, getting the profiles from Gravatar allows one to have a sense of the look and feel of the platform and of the community interactions. I wouldn't like to see everyone with generated avatars... For the actual migration, active users will be able to populate their profile, but IMO it would be a bit of a shame to leave all the old members of the community without a face. I'm very happy to see a profile pic associated with janwas or k776 for instance. Oh, that works for me, so that's a bug indeed. I'll track it and let you know Yeah I figured that out, now are you advocating for 10-char hashes? 7-char? Something else? I used the default color palette*, and since I'm not a designer, I try to trust the Gitea devs to have done their UX work. This can be changed at any time anyway, so I'm going to stay with those for the migration unless an artist offers insight. *apart for the new "activity" labels, so if your grudge was specifically against them, your concern is valid and I'll change those colors Can you be more specific, which alternative would you propose? I disagree, colors convey meaning for priorities or difficulties. I kept a single color for resolution, components, and ticket types. I don't have a strong opinion for the colors of the new "activity" label. I can try pascal case, but that wouldn't work for themes, so meh for the consistency Yes it is, labels are sorted alphanumerically (both in menus and in issue lists and previews), I need to group them and order them correctly for the users. I think you're correct. I'll check whether it's possible to disable that feature

Going through points I didn't address. As an admin, I see all repos including private ones. I'll try and write a regex for identifying metadata and grouping them. It is unfortunate that the empty line was mandatory for Phabricator to parse correctly the closing of differentials. The first one I agree. Long lines should be wrapped. Also added to my TODO board. For the second one, it looks perfect when viewed in gitea, because the web view reduces the hash to a short version. But I don't know if it's a good practice to use short hashes in the actual message. If yes, I suppose that 10-char hashes are unique enough for our use. But I haven't made up my mind. I can see that activity on that page but you are probably taking about a different one, can you send the URL? Yes indeed, ariadne is for that. I kinda documented it at HowToUseTrac. But there is no exact plan, basically there is no reason to shut down Trac (unlike Phabricator, it is still maintained) but no reason to keep it up either (since all of its contents can be migrated). I would advise caution and avoid stopping it right at the migration (I may have forgotten to migrate something, or I may have made a mistake in ariadne...). So considering the high amount of places redirecting to our Trac instance, I propose to keep it online for a couple months, maybe a year, before making the URLs point to the redirect tool. Is it the one at docs.wildfiregames.com? I also used it at svn.itms.ovh, but I had to fix a bit the CSS, which makes the width overflow, at least on Firefox. (width:100% is not compatible with having a padding).

A bare clone of my repo is 5.9 GiB. Unfortunately a checkout will take twice as much space (.git is 5.9GiB, binaries/ is the same size when LFS files are downloaded). On the other hand, on Gitea, the entire history of LFS files is stored, and that takes 13GiB, which is what is displayed in the web interface. Maybe I should make a change in the documentation, to avoid confusion, and write "12GiB" for the size of a checkout. So that people don't mix that up with the 13GiB of stored LFS files in Gitea. It's a coincidence the values are similar: it basically means that each asset of the game has on average two versions in history. If it was, say, three, the size of the LFS repo would be 18GiB while the checkout would still be 12. I think The SVN repos are not supposed to be web-visited so maybe not clutter the top bar, however adding the docs is a fantastic idea. I saw the huge progress on that front a couple days ago, congrats!

Oh @hyperion I hadn't realized that you had created an account instead of using your migrated Trac account. Thanks so much for your enthusiasm! I added you. Can you check that you can edit the wiki but didn't get any extra permission for the repo nor the issues? By the way, did uploading the Ubuntu ISO correctly fail?

It seems unrealistic to me to automatically convert SVN patches on arbitrary revisions to branches of git commits. Importing the discussions could probably be done but it's a huge amount of work I'm not interested in doing. The path is documented on the page Phabricator. Yes, I already created a Docs team with wiki edit access. But so far this post doesn't have a huge success, as almost nobody has asked for the password to their gitea account. So it hasn't been tested yet.