Itms

I regenerated the repo and associated data (that broke some links, I can't do much except rewiping the databases and users, which I'll try to avoid doing too frequently). I'm going to stop changing/fixing bugs in the PoC for now, thanks for all the reports and testing. I am now going to focus on the Unix build environment and the setting up of CI/CD. Changelog: git repository: - used short hashes in messages (I didn't have to choose the length, git automatically chose 10 chars) - wrapped the content of commit messages to 72 chars, ignoring the first line - grouped metadata lines in commit messages as much as possible (my regexp detects "[...] by:" and "Differential Revision:" but not for instance Trac tickets:, I can't cover and test all cases) - handle phab: links and [Pp]hab: links to non-commits - added a Bugs herder team Imported issues: - updated label display (typography/color), removed activity labels - disabled time tracking - do not publish a Patch change if the new value is empty wiki documentation: - added notices to install git LFS - adapted contents to label improvement Cosmetics: - added link to docs.wildfiregames.com in top menu Future branch: - merged Stan's update to .gitignore In gitea 1.22 (currently RC, will upgrade when it's released): - command-line instructions to merge will be adapted to our merge strategies - username will be correctly used instead of real name in commit feeds (including the RSS consumed by the IRC bot) Didn't do: - rename TracUser, I really think it's important to be explicit about the fact content comes from Trac - keep the Trac registration date of users... because Trac doesn't record that at all (only the last activity date) - I cannot reproduce @hyperion problem with "issues created by me"

Why not! But that would have to link with the migration. They can be moved now or later, in both cases I'll preserve them. No need at all! Images can be retrieved from a remote URL, for instance on play0ad.com, or from the nightly-build SVN. It's okay, it's still kinda maintained, and as long as it doesn't break, we don't need to fix it. But there are indeed more popular choices for C++ test frameworks nowadays. That's also unrelated to the migration. My changes to the build environment will probably supersede those interesting diffs you linked.

Oh I misunderstood your previous message, sorry! I thought you were suggesting doing things, but you were questioning/proposing improvements on what I did. Please go ahead and make a PR The bat files are used in the Windows installer I think, no need to make sh equivalents when we have a .desktop file. Converting to Markdown is a must, but it would indeed be great to deduplicate things that go to the wiki. Instead it would be awesome to have a readme.md which is actually appealing, with screenshots and a user-orientated presentation of the project. I have not worked in that direction but I'd love to see it done. I have already planned to remove svn-related binaries, but I have decided not to remove premake and cxxtestgen. Those are very lightweight binaries, and keeping them really simplifies the Windows build process in the design I am proposing.

@Stan` Everything you listed in the latest message is either already covered or still WIP but planned in https://gitea.itms.ovh/Itms/0ad in what I called the "future" branch. Please test and propose alternative coloring if you have a strong opinion on the matter. That was not at all the aims I had in mind, I see how my wording was confusing. Please avoid deeming others' work "pointless" and being rude for no reason. I see that there is indeed overlap with other Gitea features. I wanted to make sure PRs without any reviewers would stand out. Instead of adding a "reviewer-needed" label, we could make a rule that reviewers assign themselves to PRs (which I hadn't realized is different from assigning one to an issue). That way, orphan PRs could be found with the "No assignees" filter. "under-review" is then already covered by the presence of assignees, and "work-in-progress" is covered by the "changes requested" display. This is a very good point, it would be great to be able to mark an issue as waiting on info without closing it. The "Due Date" feature would here be useful to keep track of when to close it. (still no found use for the Time Tracker which is a different feature) Well I could call that "closed". Order doesn't matter here since closed tickets should just have a "closed/resolved" and "theme" label. I disagree, there is an actual difference between "nice to have" (this would provide an actual benefit to the user or the devs, but low priority) and "if time permits" (this would be nice, but it either wouldn't change anything from the user pov, or the cost of doing it would counterbalance the small benefits). Then again, I just don't want to lose any information from Trac. Merging the labels can be done in the future if the team wishes.

Will take a look Another thing worth mentioning: I plan to delete all Trac accounts without content as part of the migration (using the Trac spam filter). I did not perform that in my PoC because I haven't set up the spam filter (useless in a read-only instance). As a consequence, I will only create Gitea users for active Trac accounts in the actual migration.

I would go with 10 to match the Gitea display, but maybe 12 would reassure me more. Scratch that, the Linux kernel uses 12, but that's overkill for us. I'll use 10. I am not planning to touch gitconfig in the repo. I specifically chose "theme" so that it is after activity/difficulty/priority (alphabetically). "topic" would also work. It is a less important label than those. I could prefix the label types with A/B/C like some projects do on Github, but I suspect you won't like it That's not how labels work on Gitea. I suggest you make your own label set in your test repository, so that you can see what the constraints are. What you call "key/value" pair labels are the "exclusive" labels, which is not the case for the theme. But reflecting on this now, Trac didn't leave the possibility to set multiple components for a ticket, so maybe I'm going too far by allowing multiple theme labels. Oh that's on me, I assumed that white space wasn't allowed. Thanks, I'll improve that then! (there is a possibility that the trac2gitea tool doesn't support it though, we will see) I kinda agree but right now all the labels are used, removing some would mean losing information from Trac. And I strongly believe we need the "activity" label or an equivalent, to triage PRs (and it needs to be the first label type, and to stand out) Interesting. That's stronger than preventing you to post. Oh that's easy to do, and I did it almost daily when testing my migration scripts I deleted your top-secret account. Also I unrestricted you, but I also have the option to prevent you from logging in, so I'm testing that. Can you confirm you can't login at all?

Yes I just restricted PoppaSmurf, I don't know how it appears to you. I can't contact users but I do have access to their email addresses (pretty much like in Trac). It is actually a good thing in my opinion that Gitea doesn't have a PM system. I would be unhappy so see communication even more split between platforms. Yes, I tested a while ago, it works. In the worst case I can debug that after the migration, emails can wait for a few days. I really don't want to test now and spam a wide public. I really had a hard time to decide between an organization and a user. Users need email addresses so I didn't really want that. Also I figured it might be handy to be able to add people to the organization if there is a need to manually edit imported Trac data... but that is realistically not going to happen. What do you mean about surprising? In terms of privacy you mean? (if yes, I tried to be precise in the privacy policy, and people who don't use Gravatar won't have their data sent there, so they are not supposed to complain unless I missed something). For the PoC, getting the profiles from Gravatar allows one to have a sense of the look and feel of the platform and of the community interactions. I wouldn't like to see everyone with generated avatars... For the actual migration, active users will be able to populate their profile, but IMO it would be a bit of a shame to leave all the old members of the community without a face. I'm very happy to see a profile pic associated with janwas or k776 for instance. Oh, that works for me, so that's a bug indeed. I'll track it and let you know Yeah I figured that out, now are you advocating for 10-char hashes? 7-char? Something else? I used the default color palette*, and since I'm not a designer, I try to trust the Gitea devs to have done their UX work. This can be changed at any time anyway, so I'm going to stay with those for the migration unless an artist offers insight. *apart for the new "activity" labels, so if your grudge was specifically against them, your concern is valid and I'll change those colors Can you be more specific, which alternative would you propose? I disagree, colors convey meaning for priorities or difficulties. I kept a single color for resolution, components, and ticket types. I don't have a strong opinion for the colors of the new "activity" label. I can try pascal case, but that wouldn't work for themes, so meh for the consistency Yes it is, labels are sorted alphanumerically (both in menus and in issue lists and previews), I need to group them and order them correctly for the users. I think you're correct. I'll check whether it's possible to disable that feature

Going through points I didn't address. As an admin, I see all repos including private ones. I'll try and write a regex for identifying metadata and grouping them. It is unfortunate that the empty line was mandatory for Phabricator to parse correctly the closing of differentials. The first one I agree. Long lines should be wrapped. Also added to my TODO board. For the second one, it looks perfect when viewed in gitea, because the web view reduces the hash to a short version. But I don't know if it's a good practice to use short hashes in the actual message. If yes, I suppose that 10-char hashes are unique enough for our use. But I haven't made up my mind. I can see that activity on that page but you are probably taking about a different one, can you send the URL? Yes indeed, ariadne is for that. I kinda documented it at HowToUseTrac. But there is no exact plan, basically there is no reason to shut down Trac (unlike Phabricator, it is still maintained) but no reason to keep it up either (since all of its contents can be migrated). I would advise caution and avoid stopping it right at the migration (I may have forgotten to migrate something, or I may have made a mistake in ariadne...). So considering the high amount of places redirecting to our Trac instance, I propose to keep it online for a couple months, maybe a year, before making the URLs point to the redirect tool. Is it the one at docs.wildfiregames.com? I also used it at svn.itms.ovh, but I had to fix a bit the CSS, which makes the width overflow, at least on Firefox. (width:100% is not compatible with having a padding).

A bare clone of my repo is 5.9 GiB. Unfortunately a checkout will take twice as much space (.git is 5.9GiB, binaries/ is the same size when LFS files are downloaded). On the other hand, on Gitea, the entire history of LFS files is stored, and that takes 13GiB, which is what is displayed in the web interface. Maybe I should make a change in the documentation, to avoid confusion, and write "12GiB" for the size of a checkout. So that people don't mix that up with the 13GiB of stored LFS files in Gitea. It's a coincidence the values are similar: it basically means that each asset of the game has on average two versions in history. If it was, say, three, the size of the LFS repo would be 18GiB while the checkout would still be 12. I think The SVN repos are not supposed to be web-visited so maybe not clutter the top bar, however adding the docs is a fantastic idea. I saw the huge progress on that front a couple days ago, congrats!

Oh @hyperion I hadn't realized that you had created an account instead of using your migrated Trac account. Thanks so much for your enthusiasm! I added you. Can you check that you can edit the wiki but didn't get any extra permission for the repo nor the issues? By the way, did uploading the Ubuntu ISO correctly fail?

It seems unrealistic to me to automatically convert SVN patches on arbitrary revisions to branches of git commits. Importing the discussions could probably be done but it's a huge amount of work I'm not interested in doing. The path is documented on the page Phabricator. Yes, I already created a Docs team with wiki edit access. But so far this post doesn't have a huge success, as almost nobody has asked for the password to their gitea account. So it hasn't been tested yet.

They are still served over SVN (svn.itms.ovh), with audio and art_source public; art private, I didn't change anything. None of that is in the git repo. I had to overcome a few issues arising from "svn copy" being used between repos, and I also had to expunge a few sensitive files from ps, that are private in SVN, those are not in the git repo either.

That's out of scope for the migration (and milestones are one of the rare features that perfectly match between Trac and Gitea...) . And I've been inactive for some time but the milestones were necessary to plan releases. That's true, but I can't drop those tickets during the migration. Maybe they can be moved to a different website repo if we start seriously versioning our website theme. Or maybe we should have a Wfg infrastructure repo. That will have to wait, and the corresponding label will stay in the meantime. Uuuuh once again that final slash looks highly suspicious. Not having ".git" is unusual but treating that as a directory seems wrong. I will try to upload things and trigger your errors during the weekend. But it would help if you'd send the commands you tried to run, and also the full output from git, because for an authentication issue I'd like to know if you are using a password manager, whether git prompted you for identifiers, etc. For the record, I uploaded the 0ad repo through ssh and the commits on the "future" branch with http,so both methods work, at least in some contexts.

I didn't look into that! You can test for sure, especially since I don't have access to the ircbots machine. Please test. It is supposed to fail/deny the upload. If it breaks I'll fix and learn a lesson, and the costs for this server are fixed, so no worries about that.

Well I suppose it should happen at the split, not before. Currently this repository is 0ad, since it contains the game assets. Also it is important to have a memorable and easy to type address. I am already reluctant to have called the repo wfg/0ad instead of 0ad/0ad, but the organization must be named wfg, so I settled with that. Don't you need ".git" at the end of the remote url? If not, that's an actual bug I have to take a look at. Please don't restrain yourself, test away. Better detect bottlenecks now.

Oh I thought that was for his encyclopedia repo test... Using git clone would be more straightforward and would rely less on assumptions. It's a good idea to use a more widely recognized name, yes. (However I, for one, didn't know about Ghost) Yes, but in a private repo. Oh yes, it is undesirable. I thought you were talking about wrapping the first line, and that is difficult to do. The second line of a git commit message must be empty, and I certainly don't want to cut an informative commit message in half. Can you point out a few instances of commit messages where something else than the first line would benefit from being wrapped?

I'd like to keep trunk for historical/nostalgic reasons, and also because I'd like us to explicitly follow a trunk-based development model. I am open to discussion about that. No, that's too difficult. But you can fix your message on Trac by adding a ! before the name to prevent link generation, and thus the migration to gitea will keep it as a non-link. TracUser is a Gitea account I created specifically for messages originating from people who have no account on Trac, and thus no account from migration on Gitea. This covers two cases: people who posted once on Trac without registering with an email address; and people who deleted their Trac account. This has no relationship with CI/CD. Mh I can probably group the Phabricator-related lines indeed, but I'm not sure it makes much of a difference. If others want me to do that, please weigh in. I think that the reference to the SVN rev must be separated from the rest by a blank line though. Could you come up with a script that performs that rewrap? I am not sure how to perform this in a way that provides a satisfying result for all commit messages. Indeed, it is.

Reminder: registration is now disabled on Phabricator. Only currently active users can post those updates. And indeed I forgot to write this in the top message: all the accounts without content were deleted on Phabricator. If you login to code.itms.ovh, you'll see that every possible dormant spam account was deleted. Of course, if the spammer has recently posted, my script will not delete them... But we can get rid of them manually and then we'll be fine.

No, I want everyone to be able to keep commenting on open diffs, for instance for posting updates such as "I opened a PR for this diff at the following address..". It is important not to break current work when launching the migration. I believe my fix will consist of blocking the diff upload URL directly in the web server config, but I'll have to test and see whether that blocks arcanist upload through the API... Well maybe I'll just block the API for security reasons. No! That's the beauty of it! (yes I'm going to blow my own horn quite a bit in this thread, sorry). Forks do not duplicate LFS storage. So every fork of 0 A.D. only weighs 100MiB. Yes, if forks don't count as we seem to have discovered, I'll keep the limit at 0 repos, it's better. I'm more worried about malicious software than forks... (also we need to stop using the word "fork" in that sense if we start using gitea... I saw that but I think you didn't use the correct tools at all. I'll get back to you when I actually start working on this.

Yes, I am hesitating between letting new accounts create garbage repositories, and having to manually approve any new contributor. I chose to set the default repo limit to 1. But it looks like forks don't count? If that's the case, I could set the default to 0 to let new contributors fork and contribute: testing this now. In any case, I set the limit to 5 on both your accounts. What is the output of git remote -v? This is because the commit author's email does not match the PoppaSmurf account email. You can use secondary email addresses in your account for that. I wrote down that I have to document it. No, please post everything here, I'm following. And this is just an unofficial PoC for now, please don't report bugs on Trac.

Hi everyone, For a few years now we have pondered whether git would be the right VCS for the development of 0 A.D. One of the main selling points of git is the ability to create numerous branches for parallel development of features and bugfixes. Several popular platforms (Github, Gitlab.com, ...) leverage this to provide a feature-packed development environment, where people can contribute, discuss, review, and test patches. Historically, the development of 0 A.D. has been performed using SVN, which is a bit old... but suits our needs in a number of ways, already discussed elsewhere. We are using Phabricator to supercharge SVN with a review and CI/CD platform, but alas, Phabricator has been abandoned by its developers. In any case, a lot of contributors in the past couple of years have expressed disappointment with the peculiarities of Phabricator. Thus, we have discussed finally moving to git, without any strong urgency or hard opinions on the matter. I have proposed myself to set up a migration path and a new development environment for 0 A.D., and I am proud to present it to you folks today. I have started working on this "for real" more than a year ago, and started dedicating all my "0 A.D. time" (which has been very scarce) on this project at the end of Spring 2023. It is still, unfortunately, very much a work-in-progress, but now is the time to reveal it and receive feedback. Ideally, I would like to collect feedback from the community, finish the CI/CD system, and fix bugs, over the course of the spring, with the aim of performing the actual migration, if the community wishes to perform it, during the summer. As always, delays should be expected. It is worth noting that personal changes in my life/work balance will give me more free time after the summer. Thanks in advance for the feedback you will provide Presentation of the Proof of Concept I am self-hosting the services for the duration of the tests, so that I have full control and do not divert resources from WFG. All URLs are under itms.ovh, which should eventually become wildfiregames.com Email does not work on this Proof of Concept! I have no will to setup email and I certainly don't want to spam anyone with tests. The services are not updated automatically: the full migration takes around 48h and necessitates some human input. The PoC I am presenting was generated from scratch the past two days. I will update stuff from time to time, when I fix bugs or when I'm bored, but don't expect the PoC to follow the actual development of the WFG services. Please test as many things as you want and don't hesitate to try to break things! Now is the time to make sure I didn't forget anything. All the data sent to the PoC will be wiped out whenever I re-migrate the services, and of course whenever the actual migration happens. Please feel free to play around all this. The services (this will be regularly updated) - gitea.itms.ovh The development platform. This is an instance of Gitea, a self-hosted lightweight alternative to Gitlab/Github. If, in the far future, we want to stop self-hosting stuff, Gitea is compatible with Github features to allow for easy migration. The git repository is at wfg/0ad. Commit messages were reformatted, contain the original SVN revision, links to other commits are preserved as much as possible. The HEAD revision is the HEAD of the SVN repo. A currently WIP of the "future" branch of the git repository, adding commits needed for development with git, is at my fork Itms/0ad. All Gitea users were imported from Trac. Since email does not work, if you want to login, shoot me a DM and I'll send you a password. All Trac tickets are imported at wfg/0ad/issues. The Trac wiki is imported at wfg/0ad/wiki. I tried to self-document the PoC as much as possible, so please read the updated pages, especially BuildInstructionsGettingTheCode and BuildAndDeploymentEnvironment. The FAQ is defaced beyond readability because of its formatting peculiarities, work is needed, my apologies to everyone who worked hard on the visuals of the Trac page... - trac.itms.ovh A read-only copy of Trac, upgraded to the latest release. This serves as a reference, and can stay online as long as needed. See below for the redirect tool. - code.itms.ovh A registration-disabled copy of Phabricator, upgraded to the latest version (RIP). All the inactive accounts were deleted, especially the dormant spam accounts. This should stay up as much as possible. We need to keep all the discussions on patches and commits. TODO: I need to find a (possibly ugly) way to disable the upload of new diffs. - svn.itms.ovh The old and new SVN repositories. This holds the current ps repo (which will stop receiving commits after the migration), the audio and art sources (which will keep working as-is), the new libraries repos for precompiled Windows and for bundled libs, and most importantly the new nightly-build repo WIP: This is currently the biggest missing piece, as I am now going to start working on CI/CD, and the generation of the nightly build. Please read NightlyBuild if you want to know how this will work in the (hopefully close) future. - ariadne.itms.ovh Ariadne is a small redirect tool I wrote as a drop-in replacement for Trac or Phabricator when/if we decide to drop them. A lot of links to Trac exist on the Internet. You can follow Trac-like paths on Ariadne to find the relevant content. Each Trac page displays a link to the associated Ariadne path. For instance : A ticket https://ariadne.itms.ovh/ticket/666 A wiki page https://ariadne.itms.ovh/wiki/Manual_Settings A SVN changeset https://ariadne.itms.ovh/changeset/28056 A Phabricator commit page https://ariadne.itms.ovh/rP28056 A file in the Trac browser https://ariadne.itms.ovh/browser/ps/trunk/source/test_setup.cpp Ariadne also provides a nice page for knowing commit correspondence, just use `rXXXX` as path: https://ariadne.itms.ovh/r28056 https://ariadne.itms.ovh/r28000 Enjoy! And see you soon for regular updates

Hello! I have just set up a mitigation attempt against the DoS attacks. Let us see if it holds over time, if it does, I'll close the ticket. Thanks for your patience PS. News are coming soon about the migration project, but it's unrelated to the attacks on Trac (the mitigation I am setting up will have to be kept in the future)

Done, with some delay. Sorry for the inconvenience!

Oh, it's entirely my fault. I'll restore those. Thanks for the ping

Hello Stan. I know all too well the feeling of not being able to carry on anymore, despite wanting to. You have been an extraordinary leader and the best successor I could have had when I decided to step down. You should indeed have absolutely no regrets, including in this decision of stopping and focusing on yourself and your health. Others have worded it perfectly during these ten past days: thank you for making 0 A.D. the best game that exists. I am certain that, thanks to your work, the project will endure.