A new git-based development environment

[Stan`]

This website? https://replay-pallas.wildfiregames.ovh/

Interesting I always thought this ticket had some insight about trac https://gitea.itms.ovh/wfg/0ad/issues/1819 but it seems the discussions happened somewhere else maybe IRL.

 

[hyperion]

13 minutes ago, Stan` said:

This website? https://replay-pallas.wildfiregames.ovh/

Similar but better looking if I'm not completely wrong

I meant the gitea instance hosted at git.wildfiregames.com at some point. I think it was based off of the one on play0ad.com

[ShadowOfHassen]

So, anything else I need to be testing? I think I've exhausted most of the random things I can think of.

 

 

[Itms]

Going through points I didn't address.

On 05/04/2024 at 8:53 AM, Stan` said:

Someone ill intentioned could host an entire fork of the game free of charge on the server.

As an admin, I see all repos including private ones.

On 05/04/2024 at 12:02 PM, hyperion said:

It helps distinguishing metadata from commit message, the linux kernel and many other projects do this as well, guess it would also be good to follow this convention in the future.

I'll try and write a regex for identifying metadata and grouping them. It is unfortunate that the empty line was mandatory for Phabricator to parse correctly the closing of differentials.

On 05/04/2024 at 12:42 PM, hyperion said:

Guess there are many, one with questionable wrapping is fc5518a22b15827c7f4d4410c27dad4c628af8a4 and on that got worse with rev to sha is 9f85b097e11ffac38df23815a62b14bd25925d2d

The first one I agree. Long lines should be wrapped. Also added to my TODO board.

For the second one, it looks perfect when viewed in gitea, because the web view reduces the hash to a short version. But I don't know if it's a good practice to use short hashes in the actual message. If yes, I suppose that 10-char hashes are unique enough for our use. But I haven't made up my mind.

On 05/04/2024 at 9:46 PM, hyperion said:

Opened a bug with a dummy account, but none are listed in the created by you section of the user

I can see that activity on that page but you are probably taking about a different one, can you send the URL?

On 06/04/2024 at 7:44 PM, hyperion said:

Guess an acceptable compromise. Is there a similar document for trac becoming read-only?

On 06/04/2024 at 9:12 PM, Stan` said:

Irc the goal was to get rid of trac since it gets completely absorbed in gitea. Then we can just make the domain point to the gitea for some time I guess ?

On 06/04/2024 at 9:42 PM, hyperion said:

I guess that is what the adriane tool mentioned is meant for but a draft document outlining the plan would allow us to actually comment on not just guesses

Yes indeed, ariadne is for that. I kinda documented it at HowToUseTrac. But there is no exact plan, basically there is no reason to shut down Trac (unlike Phabricator, it is still maintained) but no reason to keep it up either (since all of its contents can be migrated). I would advise caution and avoid stopping it right at the migration (I may have forgotten to migrate something, or I may have made a mistake in ariadne...). So considering the high amount of places redirecting to our Trac instance, I propose to keep it online for a couple months, maybe a year, before making the URLs point to the redirect tool.

On 06/04/2024 at 10:28 PM, hyperion said:

Similar but better looking if I'm not completely wrong

Is it the one at docs.wildfiregames.com? I also used it at svn.itms.ovh, but I had to fix a bit the CSS, which makes the width overflow, at least on Firefox. (width:100% is not compatible with having a padding).

[ShadowOfHassen]

33 minutes ago, Itms said:

As an admin, I see all repos including private ones.

Just to confirm that, can you see Poppa Smurf's secret repo?

Another question I have is moderation. Can you put restrictions on a user, or somehow contact them to say they are breaking guidelines?

[Stan`]

1 hour ago, Itms said:

As an admin, I see all repos including private ones.

Ah good, on Phab it was a bit messy to see all revisions, even the admin user didn't see everything (and had tbh really limited rights for an admin user)

 

Did you look into sending e-mails ? I imagine it should be as easy as Phab, but might be worth testing.

EDIT: Any reason trac user appears as an organization https://gitea.itms.ovh/explore/organizations

EDIT2: I don't have a strong opinion on it, but the fact Gitea is using gravatar might be suprising to some, IIRC the forum used to do that as well.

[hyperion]

35 minutes ago, Itms said:

I can see that activity on that page but you are probably taking about a different one, can you send the URL?

https://gitea.itms.ovh/issues should list a logged in users issues instance wide.

 

48 minutes ago, Itms said:

because the web view reduces the hash to a short version

I almost always use git log from cli.

 

37 minutes ago, Itms said:

Is it the one at docs.wildfiregames.com?

That one works, tho the one I had in mind was a tweaked version of the one on play0ad IIRC. Use what you deem fit.

 

While we are discussing UI, the labels are somewhat messy. The color scheme makes it a bit hard to read them, maybe a bit more muted would be better. Those theme/.. labels should be changed to some more meaningful text, theme/core-engine feels just wrong. The type portion of labels should probably have same color for labels of the same type. The priority labels using kebab case feels meh, also starting with a number is not needed.

 

The time tracking feature for issues probably should be disabled, don't think we will use it.

[Itms]

4 hours ago, ShadowOfHassen said:

Just to confirm that, can you see Poppa Smurf's secret repo?

Yes

4 hours ago, ShadowOfHassen said:

Another question I have is moderation. Can you put restrictions on a user, or somehow contact them to say they are breaking guidelines?

I just restricted PoppaSmurf, I don't know how it appears to you. I can't contact users but I do have access to their email addresses (pretty much like in Trac).

It is actually a good thing in my opinion that Gitea doesn't have a PM system. I would be unhappy so see communication even more split between platforms.

4 hours ago, Stan` said:

Did you look into sending e-mails ? I imagine it should be as easy as Phab, but might be worth testing.

Yes, I tested a while ago, it works. In the worst case I can debug that after the migration, emails can wait for a few days. I really don't want to test now and spam a wide public.

4 hours ago, Stan` said:

Any reason trac user appears as an organization https://gitea.itms.ovh/explore/organizations

I really had a hard time to decide between an organization and a user. Users need email addresses so I didn't really want that. Also I figured it might be handy to be able to add people to the organization if there is a need to manually edit imported Trac data... but that is realistically not going to happen.

4 hours ago, Stan` said:

EDIT2: I don't have a strong opinion on it, but the fact Gitea is using gravatar might be suprising to some, IIRC the forum used to do that as well.

What do you mean about surprising? In terms of privacy you mean? (if yes, I tried to be precise in the privacy policy, and people who don't use Gravatar won't have their data sent there, so they are not supposed to complain unless I missed something).

For the PoC, getting the profiles from Gravatar allows one to have a sense of the look and feel of the platform and of the community interactions. I wouldn't like to see everyone with generated avatars...

For the actual migration, active users will be able to populate their profile, but IMO it would be a bit of a shame to leave all the old members of the community without a face. I'm very happy to see a profile pic associated with janwas or k776 for instance.

4 hours ago, hyperion said:

https://gitea.itms.ovh/issues should list a logged in users issues instance wide.

Oh, that works for me, so that's a bug indeed. I'll track it and let you know

4 hours ago, hyperion said:

I almost always use git log from cli.

Yeah I figured that out, now are you advocating for 10-char hashes? 7-char? Something else?

4 hours ago, hyperion said:

While we are discussing UI, the labels are somewhat messy. The color scheme makes it a bit hard to read them, maybe a bit more muted would be better.

I used the default color palette*, and since I'm not a designer, I try to trust the Gitea devs to have done their UX work. This can be changed at any time anyway, so I'm going to stay with those for the migration unless an artist offers insight.

*apart for the new "activity" labels, so if your grudge was specifically against them, your concern is valid and I'll change those colors

4 hours ago, hyperion said:

Those theme/.. labels should be changed to some more meaningful text, theme/core-engine feels just wrong.

Can you be more specific, which alternative would you propose?

4 hours ago, hyperion said:

The type portion of labels should probably have same color for labels of the same type.

I disagree, colors convey meaning for priorities or difficulties. I kept a single color for resolution, components, and ticket types. I don't have a strong opinion for the colors of the new "activity" label.

4 hours ago, hyperion said:

The priority labels using kebab case feels meh

I can try pascal case, but that wouldn't work for themes, so meh for the consistency

4 hours ago, hyperion said:

also starting with a number is not needed.

Yes it is, labels are sorted alphanumerically (both in menus and in issue lists and previews), I need to group them and order them correctly for the users.

4 hours ago, hyperion said:

The time tracking feature for issues probably should be disabled, don't think we will use it.

I think you're correct. I'll check whether it's possible to disable that feature

[hyperion]

2 hours ago, Itms said:

Yeah I figured that out, now are you advocating for 10-char hashes? 7-char? Something else?

7-char will blow up on us, 10-char is just about acceptable, rough approximation:

$ bc <<< "scale=15; commits=25000; digits=10; commits^2 / (2*(16^digits))"
.000284217094304

Whatever should be added to gitconfig, also remember commands like revert will pre-populate the commit message with full hash.

 

3 hours ago, Itms said:

I try to trust the Gitea devs to have done their UX work. This can be changed at any time anyway, so I'm going to stay with those for the migration unless an artist offers insight.

Well, white on light gray I have trouble reading.

 

3 hours ago, Itms said:

and since I'm not a designer

Help! 

 

2 hours ago, Itms said:

Can you be more specific, which alternative would you propose?

Component instead of theme maybe, also make them key value pair labels?

 

3 hours ago, Itms said:

I disagree, colors convey meaning for priorities or difficulties. I kept a single color for resolution, components, and ticket types. I don't have a strong opinion for the colors of the new "activity" label.

Coloring the value portion of the label is separate from coloring the key portion, I only suggest the key portion to be the same across a label type.

 

3 hours ago, Itms said:

can try pascal case, but that wouldn't work for themes, so meh for the consistency

White space would be fine me thinks.

 

3 hours ago, Itms said:

Yes it is, labels are sorted alphanumerically (both in menus and in issue lists and previews), I need to group them and order them correctly for the users.

An indicator that there are just to many. Let's say open/closed for status is enough *runs*

[ShadowOfHassen]

3 hours ago, Itms said:

I just restricted PoppaSmurf, I don't know how it appears to you. I can't contact users but I do have access to their email addresses (pretty much like in Trac).

It is actually a good thing in my opinion that Gitea doesn't have a PM system. I would be unhappy so see communication even more split between platforms.

I think an email is good enough to contact someone. We just need to be certain that mods email offending people using a wildfire game account.

The ban makes me un able to see any repository that I hadn't' created, so that's something.  See if you can un restrict Poppa Smurf, and then I'd suggest attempting to remove repositories and/or users.

[Itms]

7 hours ago, hyperion said:

7-char will blow up on us, 10-char is just about acceptable, rough approximation:

Whatever should be added to gitconfig, also remember commands like revert will pre-populate the commit message with full hash.

I would go with 10 to match the Gitea display, but maybe 12 would reassure me more.  Scratch that, the Linux kernel uses 12, but that's overkill for us. I'll use 10. I am not planning to touch gitconfig in the repo.

7 hours ago, hyperion said:

Component instead of theme maybe

I specifically chose "theme" so that it is after activity/difficulty/priority (alphabetically). "topic" would also work. It is a less important label than those. I could prefix the label types with A/B/C like some projects do on Github, but I suspect you won't like it

7 hours ago, hyperion said:

also make them key value pair labels?

7 hours ago, hyperion said:

Coloring the value portion of the label is separate from coloring the key portion, I only suggest the key portion to be the same across a label type.

That's not how labels work on Gitea. I suggest you make your own label set in your test repository, so that you can see what the constraints are. What you call "key/value" pair labels are the "exclusive" labels, which is not the case for the theme. But reflecting on this now, Trac didn't leave the possibility to set multiple components for a ticket, so maybe I'm going too far by allowing multiple theme labels.

7 hours ago, hyperion said:

White space would be fine me thinks.

Oh that's on me, I assumed that white space wasn't allowed. Thanks, I'll improve that then! (there is a possibility that the trac2gitea tool doesn't support it though, we will see)

7 hours ago, hyperion said:

An indicator that there are just to many.

I kinda agree but right now all the labels are used, removing some would mean losing information from Trac. And I strongly believe we need the "activity" label or an equivalent, to triage PRs (and it needs to be the first label type, and to stand out)

7 hours ago, ShadowOfHassen said:

The ban makes me un able to see any repository that I hadn't' created, so that's something.

Interesting. That's stronger than preventing you to post.

7 hours ago, ShadowOfHassen said:

See if you can un restrict Poppa Smurf, and then I'd suggest attempting to remove repositories and/or users.

Oh that's easy to do, and I did it almost daily when testing my migration scripts I deleted your top-secret account. Also I unrestricted you, but I also have the option to prevent you from logging in, so I'm testing that. Can you confirm you can't login at all?

[Stan`]

11 hours ago, Itms said:

For the actual migration, active users will be able to populate their profile, but IMO it would be a bit of a shame to leave all the old members of the community without a face. I'm very happy to see a profile pic associated with janwas or k776 for instance.

Sounds fair.

 

11 hours ago, Itms said:

 

It is actually a good thing in my opinion that Gitea doesn't have a PM system. I would be unhappy so see communication even more split between platforms.

+1 on this. The least media of communication the better. Would really help to have a keycloak or equivalent though so having one account on Gitea means you have a forum account as well that would be the recipient of messages if they need to be private for some reason. Would avoid the my repo got deleted and I don't know why situation. Although a deletion process (Eviction issue creation, two weeks repo deletion could work too)

Can you send me my Gitea password ?

Is it possible to keep the actual registration dates if Trac has them ?

Some accounts may not look legit if they have a very recent date of registration I suppose.

 

 

[Itms]

1 hour ago, Stan` said:

Is it possible to keep the actual registration dates if Trac has them ?

Will take a look

Another thing worth mentioning: I plan to delete all Trac accounts without content as part of the migration (using the Trac spam filter). I did not perform that in my PoC because I haven't set up the spam filter (useless in a read-only instance). As a consequence, I will only create Gitea users for active Trac accounts in the actual migration.

[Stan`]

Sounds good for data minimization. Since Gitea is a bit more popular than trac these days, we'll also have to monitor whether spambots register and how to deal with them. I know @vv221 simply banned all gmail addresses but it's not really an option for us  

[ShadowOfHassen]

5 hours ago, Itms said:

Oh that's easy to do, and I did it almost daily when testing my migration scripts I deleted your top-secret account. Also I unrestricted you, but I also have the option to prevent you from logging in, so I'm testing that. Can you confirm you can't login at all?

I can't login at all. I get a "sign in prohibited" message.

[Stan`]

Another interesting thing that comes to mind for the CI is the tarball for spidermonkey/nvtt/fcollada location, do we prefer having them managed on http://releases.wildfiregames.com/libs/ or hosted on Gitea releases like https://github.com/wraitii/spidermonkey-tarballs/releases/download/${FOLDER}/${FOLDER}.tar.xz

(Fromhttps://code.wildfiregames.com/source/0ad/browse/ps/trunk/libraries/source/spidermonkey/build.sh$122 )

EDIT1: The git repo could probably benefit from ignoring the following exe|dll|lib|pdb|zip|exp|ilk|so|a|obj|raw|tar|bz2|gz|xz|suo|bmp|jpg|psd

obj,raw,bmp,jpg and psd were never supported by the engine and can probably be nuked alltogether from history

There is also *.thumbs.db which is like dstore (windows miniature cache)

 

Any reason we're keeping the bat files ? Shouldn't we provide sh files as well then ?

 

Does it make sense to convert the binaries/readme.txt to readme.md ?
Should it be part of the wiki instead ? Then people would stop making PRs to translate it
https://github.com/0ad/0ad/pull/37/files

https://github.com/0ad/0ad/pull/35

https://github.com/0ad/0ad/pull/27

[hyperion]

6 hours ago, Itms said:

That's not how labels work on Gitea.

Which doesn't make sense to me, also I have seen it in the wild. To render scoped labels as they call it as scoped labels only if they are also exclusive is counterintuitive. I'm not alone thinking such https://github.com/go-gitea/gitea/pull/23164/commits/c278def8e3441783b1c07c67bd4bf208d487950b

 

There are open bugs for broken/bad contrast algorithm, so for now choosing colors that work is what one can do.

 

The activity labels are pointless. "reviewer-needed" is true for any open pull request by virtue of being open. Does "under-review" have any connotation other than I grabbed that PR and anyone else should stfu? "work-in-progress" is just the unsafe way to mark a PR as WIP.

 

The "resolved" are also not ideal as it stands for closed, however needs-info could just as well be kept open and only if we don't hear from a user over a certain time close it. Also fixed is closed without any tag.

 

I think it would also make sense to merge priority labels 4 and 5. Basically low medium high and bricked are enough.

[Itms]

@Stan` Everything you listed in the latest message is either already covered or still WIP but planned in https://gitea.itms.ovh/Itms/0ad in what I called the "future" branch.

4 hours ago, hyperion said:

There are open bugs for broken/bad contrast algorithm, so for now choosing colors that work is what one can do.

Please test and propose alternative coloring if you have a strong opinion on the matter.

4 hours ago, hyperion said:

The activity labels are pointless. "reviewer-needed" is true for any open pull request by virtue of being open. Does "under-review" have any connotation other than I grabbed that PR and anyone else should stfu? "work-in-progress" is just the unsafe way to mark a PR as WIP.

That was not at all the aims I had in mind, I see how my wording was confusing. Please avoid deeming others' work "pointless" and being rude for no reason.

I see that there is indeed overlap with other Gitea features. I wanted to make sure PRs without any reviewers would stand out. Instead of adding a "reviewer-needed" label, we could make a rule that reviewers assign themselves to PRs (which I hadn't realized is different from assigning one to an issue). That way, orphan PRs could be found with the "No assignees" filter. "under-review" is then already covered by the presence of assignees, and "work-in-progress" is covered by the "changes requested" display.

5 hours ago, hyperion said:

needs-info could just as well be kept open and only if we don't hear from a user over a certain time close it

This is a very good point, it would be great to be able to mark an issue as waiting on info without closing it. The "Due Date" feature would here be useful to keep track of when to close it. (still no found use for the Time Tracker which is a different feature)

5 hours ago, hyperion said:

The "resolved" are also not ideal as it stands for closed

Well I could call that "closed". Order doesn't matter here since closed tickets should just have a "closed/resolved" and "theme" label.

5 hours ago, hyperion said:

I think it would also make sense to merge priority labels 4 and 5. Basically low medium high and bricked are enough.

I disagree, there is an actual difference between "nice to have" (this would provide an actual benefit to the user or the devs, but low priority) and "if time permits" (this would be nice, but it either wouldn't change anything from the user pov, or the cost of doing it would counterbalance the small benefits).

Then again, I just don't want to lose any information from Trac. Merging the labels can be done in the future if the team wishes.

[Stan`]

1 hour ago, Itms said:

Everything you listed in the latest message is either already covered or still WIP but planned in https://gitea.itms.ovh/Itms/0ad in what I called the "future" branch.

Okay Since the files aren't ignored yet do you want a PR on the dev repo ? (Also aren't ignored by the surgeon tool)

[s0600204]

Firstly, thanks for working on this.

If we're excluding Windows-only binary files from the gitea repo, would the contents of ./build/bin be considered candidates for removal? And ./build/premake/premake5/bin/release/premake.exe?

• For Linux and macOS, the premake binary is (re)built as part of update-workspaces.sh, and I don't immediately see why we couldn't (eventually) do the same on Windows;
• The cxxtestgen executable only seems to exist to get around the need to install python, (which we require on *nix to also build spidermonkey - for Windows, python is bundled inside mozbuild, a required prerequisite on Windows-systems only);
• The svnversion executable
  • looks to only be used during the creation of a release (and so would only need to be installed on whatever machine was assembling a Windows release), and
  • likely wouldn't work if we start cutting releases from gitea anyhow.

[Itms]

Oh I misunderstood your previous message, sorry! I thought you were suggesting doing things, but you were questioning/proposing improvements on what I did.

Please go ahead and make a PR

8 hours ago, Stan` said:

Any reason we're keeping the bat files ? Shouldn't we provide sh files as well then ?

The bat files are used in the Windows installer I think, no need to make sh equivalents when we have a .desktop file.

8 hours ago, Stan` said:

Does it make sense to convert the binaries/readme.txt to readme.md ?
Should it be part of the wiki instead ? Then people would stop making PRs to translate it

Converting to Markdown is a must, but it would indeed be great to deduplicate things that go to the wiki. Instead it would be awesome to have a readme.md which is actually appealing, with screenshots and a user-orientated presentation of the project. I have not worked in that direction but I'd love to see it done.

14 minutes ago, s0600204 said:

If we're excluding Windows-only binary files from the gitea repo, would the contents of ./build/bin be considered candidates for removal? And ./build/premake/premake5/bin/release/premake.exe?

I have already planned to remove svn-related binaries, but I have decided not to remove premake and cxxtestgen. Those are very lightweight binaries, and keeping them really simplifies the Windows build process in the design I am proposing.

[ShadowOfHassen]

3 hours ago, Itms said:

Converting to Markdown is a must, but it would indeed be great to deduplicate things that go to the wiki. Instead it would be awesome to have a readme.md which is actually appealing, with screenshots and a user-orientated presentation of the project. I have not worked in that direction but I'd love to see it done.

Ooh that would be nice.

[Stan`]

18 hours ago, Itms said:

The bat files are used in the Windows installer I think, no need to make sh equivalents when we have a .desktop file.

Shouldn't they be in build/dist/ then ? I don't mind having them here, but it seems unfair considering no other platforms have them.

18 hours ago, Itms said:

 

Please go ahead and make a PR

Okay will take a look.

 

18 hours ago, Itms said:

Converting to Markdown is a must, but it would indeed be great to deduplicate things that go to the wiki. Instead it would be awesome to have a readme.md which is actually appealing, with screenshots and a user-orientated presentation of the project. I have not worked in that direction but I'd love to see it done.

Well to be fair it's also duplicated in the code

https://code.wildfiregames.com/D2432

MD with images would be nice but it could probably clutter a bit the binaries folder

18 hours ago, Itms said:

I have already planned to remove svn-related binaries, but I have decided not to remove premake and cxxtestgen. Those are very lightweight binaries, and keeping them really simplifies the Windows build process in the design I am proposing.

Yeah at some point I went full crazy on it  https://code.wildfiregames.com/D4894

TBH though I don't know if requiring people to have them in the PATH would be such a stretch. I guess the fact there is no official release of premake doesn't help.

Also sucks that CXXtest seems to be no longer maintained (https://github.com/CxxTest/cxxtest)...

https://code.wildfiregames.com/D3119

 

[Itms]

3 hours ago, Stan` said:

Shouldn't they be in build/dist/ then ? I don't mind having them here, but it seems unfair considering no other platforms have them.

Why not! But that would have to link with the migration. They can be moved now or later, in both cases I'll preserve them.

3 hours ago, Stan` said:

MD with images would be nice but it could probably clutter a bit the binaries folder

No need at all! Images can be retrieved from a remote URL, for instance on play0ad.com, or from the nightly-build SVN.

3 hours ago, Stan` said:

Also sucks that CXXtest seems to be no longer maintained (https://github.com/CxxTest/cxxtest)

It's okay, it's still kinda maintained, and as long as it doesn't break, we don't need to fix it. But there are indeed more popular choices for C++ test frameworks nowadays. That's also unrelated to the migration.

My changes to the build environment will probably supersede those interesting diffs you linked.

[Itms]

I regenerated the repo and associated data (that broke some links, I can't do much except rewiping the databases and users, which I'll try to avoid doing too frequently).

I'm going to stop changing/fixing bugs in the PoC for now, thanks for all the reports and testing. I am now going to focus on the Unix build environment and the setting up of CI/CD.

Changelog:

git repository:
- used short hashes in messages (I didn't have to choose the length, git automatically chose 10 chars)
- wrapped the content of commit messages to 72 chars, ignoring the first line
- grouped metadata lines in commit messages as much as possible (my regexp detects "[...] by:" and "Differential Revision:" but not for instance Trac tickets:, I can't cover and test all cases)
- handle phab: links and [Pp]hab: links to non-commits
- added a Bugs herder team

Imported issues:
- updated label display (typography/color), removed activity labels
- disabled time tracking
- do not publish a Patch change if the new value is empty

wiki documentation:
- added notices to install git LFS
- adapted contents to label improvement

Cosmetics:
- added link to docs.wildfiregames.com in top menu

Future branch:
- merged Stan's update to .gitignore

In gitea 1.22 (currently RC, will upgrade when it's released):
- command-line instructions to merge will be adapted to our merge strategies
- username will be correctly used instead of real name in commit feeds (including the RSS consumed by the IRC bot)

Didn't do:
- rename TracUser, I really think it's important to be explicit about the fact content comes from Trac
- keep the Trac registration date of users... because Trac doesn't record that at all (only the last activity date)
- I cannot reproduce @hyperion problem with "issues created by me"