Optimize / speed development via GPT

[Stev]

HI All, first would just say that I absolutely love this game. Its definitely one of my favorite games and have contributed to the effort via donation b/c I want this game to succeed.

Not being a developer myself, I have no way to directly help build the game but in seeing the capability of GPT, do others see this being a significant contributor going forward? I know there has been developments recently to get a more stable playing experience but my sense is that there's still quite a bit of work to do. 

Has anyone started to experiment with this? Also, do you possibly see less experienced people like myself being able to contribute more with tools like GPT? I work in a company and are looking at migrating code from legacy software using GPT so would think it would be doable here too. 

Would love to hear others opinions on this. Basically hoping for more frequent builds

The game is amazing. It feels like what Age 4 should have been, at least in my opinion. 

[Vantha]

I guess you could help with bug fixing. Have a look at some tickets and paste the related code into GPT, ask it what the problem is and to fix it. I don't know how well this works but you could try.

[ShadowOfHassen]

I am not a big fan of using ChatGPT for anything. I'm speaking of straight out of a writer perspective it might be slightly different with programming, but the issues could be same. First we have the moral objection of, ChatGPT is trained by using things on the internet without asking permission, so is it stealing or violating copyrights? That Aside, I can usually tell if ChatGPT writes something because in writing it is trained on all the internet and a majority of it isn't good writing. I've run into bad code on the internet so maybe it'd write bad code.

Secondly, ChatGPT is inherently programmed to be biased and can also be plain wrong. That has nothing to do with programming, but is a reason why it shouldn't write descriptions and stuff. History is something I wouldn't trust it with because there are a lot of false sources on the internet where it could pull its data from, and I've seen ChatGPT with biases hard coded into it. For example, ask ChatGPT why its parent company is called open AI when they don't have open source stuff, and it gives a horrible, squeamish answer.

 

If you want to help there are plenty of things you can do not related to ChatGPT you can test, report bugs. We're working on an 0 A.D, history encyclopedia that we're open for contributions to, but not with ChatGPT.

 

It is great though that you want to help.

 

[Sp00ky]

Hello Stev.

As a software developer, one thing that people have the misconception is that you can just ask ChatGPT to solve any kind of programming problem. This is not the case. ChatGPT is designed as a language model, this makes it's response very believable. Yet it doesn't actually do any reasoning behind the scenes, it just tries to predict what is your desired response. For most cases this works wonderfully, specially when you are learning a technology and don't want to be going through countless post on how to make an HTTP request or how to search in a binary tree.

When you ask it to actually solve a logical problem it falls apart pretty easily:

I am using ChatGPT and generative AI for voices for my voice actors mod, but honestly. I try to double check it's results and see if it adjusts to what I can understand from my almost non existing  knowledge of other languages than english and spanish ( I do have pretty low standards for the mod since it's an immersion thing I'm doing for myself).

If you were to ask chatgpt to help you solve a bug, it might to some extent, but it wouldn't have the context that is needed to actually find a bug. And if you don't have the knowledge working with C++ or Javascript you could end up producing more bugs. Some of these could be extremely troublesome (specially with C++ and it's memory management). There is also this tendency of ChatGPT to just plainly lie about some things, for example I work with the language Elixir, I asked some help with some problems in the past, and ChatGPT basically always responded with a solution, inventing functions that didn't exist in the language. When pointed out, ChatGPT would write a slightly different code and invent a slightly different function (this is another thing dangerous and a sign that there is no reasoning behind ChatGPT, it will never tell you it doesn't know the response, except for the notice of the unavailability of data past 2021 )

That being said as other forum poster said, you can do a lot of things without knowing how to code. Testing mods, bug reporting, content writing, map making, etc. Even mod making doesn't really need programming knowledge in some cases, unless you consider XML programming.

Edited June 17, 2023 by Sp00ky

[ShadowOfHassen]

I forgot xml programming. I don't know a bit of c++ or JavaScript, but it isn't hard to figure out modding.

 

Oh, @Stev if you want to do some testing you could try out the balancing I'm trying to make. it's discussed here:https://wildfiregames.com/forum/topic/107425-balancing-for-resource-gathering-improvment-costs/page/2

 

 

Also you could help with the encyclopedia I'm spearheading, just no Chat GPT.

[ChronA]

I've been following the discussion and want to share some thoughts.

Firstly, about the copyright concern. It's a common misunderstanding about how language models work. ChatGPT learns from loads of text data on the internet, yes, but it doesn't memorize or directly copy from specific sources. It picks up patterns and relationships between words, kind of like how we humans learn languages. It doesn't retain specific sentences or paragraphs, so it's not infringing on copyrights any more than you are.

As for the quality of AI-generated content, a lot depends on the input given. If you feed ChatGPT a detailed, precise prompt, it's more likely to produce a useful response. A vague or misleading prompt could lead to less accurate outputs. It's not trying to deceive us – it's simply doing its best to predict what comes next based on the input. The more you give it to work from, the better it does.

About biases and inaccuracies in AI output, these stem from the training data and reinforcement. It's a problem the tech community is actively addressing. But, like any information source, we should always approach AI outputs with a healthy dose of skepticism.

I totally agree that AI can't replace human expertise, especially for complex tasks like coding or creating historical content. But as a productivity tool? It has potential. It could draft initial content for our encyclopedia, or digest player feedback for game balancing. With enough instruction it could probably even do things like preparing unit templates or adding small, modular features to engine's codebase. This frees us to focus on the harder, more creative parts of the project.

So while we shouldn't blindly trust AI, understanding its strengths and limitations can help us figure out how to best use it in our work. Or we can reenact the luddites, I'm sure that will solve this project's crippling manpower shortage. /s 

Edited June 20, 2023 by ChronA
setting something up

[Vantha]

The problem is that many people have only tried the free version of Chatgpt. And to be honest I don't really understand to hype around Chatgpt. Yes, it is good at creative tasks, but that's it. When you try to do research with it or ask simple logic questions, it completely fails. So, yes, it is correct, you totally shouldn't use the free version fo Chatgpt for anything serious.
But, Bing Chat and Chatgpt Plus are miles ahead of the free Chatgpt. They use Gpt 4 instead of Gpt 3.5 like free Chatgpt and can browse the internet. This makes them 10x more useful than basic Chatgpt. Now, I know people will critizise this, but I personally use Bing Chat every day for doing research or troubleshooting. And when I say research I don't mean asking it for information, because all it does is checking the first few search results, which I can also do myself. Asking it to explain complicated texts or connections between topics has actually helped me a lot and saved me countless hours. I don't have Chatgpt Plus but it's said to be similar to Bing Chat. @Stev asked if Gpt could be used to help with coding and while I haven't tried to use Gpt 4 for coding, I have to say Gpt 4 with internet access is very good at helping with troubleshooting, that's why i think it should be able to do basic bug fixing.

 

19 hours ago, Sp00ky said:

That being said as other forum poster said, you can do a lot of things without knowing how to code. Testing mods, bug reporting, content writing, map making, etc.

Still, after all, you are right, it's best to help in other ways if you don't know how to code.

 

16 hours ago, ShadowOfHassen said:

just no Chat GPT.

Btw, don't worry I haven't used anything Ai related for the encyclopedia.

[ChronA]

10 hours ago, Vantha said:

I don't have Chatgpt Plus but it's said to be similar to Bing Chat.

Having used both, I slightly prefer ChatGPT Plus over Bing Chat despite it being the more limited system (and Bing having a better UI). I find Bing Chat is too easily swayed, or one could even say distracted, by whatever information it digs up with its web search. As you can imagine, that can easily go very wrong. With ChatGPT Plus you have more control over what information gets introduced into its logic stream. I've also found it to be more deliberative in its though process and more open to changing its mind than Bing Chat when prompted with contradictory evidence or the opportunity to critique its own work. It seems like Microsoft tuned some of the model parameters to make it more concise and decisive at the expense of "intelligence".

Of course if you are dealing with a task where up to date information is critical I would probably still go for Bing Chat, rather than spend hours writing in briefings to bring ChatGPT up to date. I've also experimented with letting ChatGPT write queries to Bing Chat. In theory that would give you the best of both worlds, but so far my results have been mixed. ChatGPT can easily grasp the idea of being able to pass searches to Bing Chat, but has trouble sticking to a workable syntax while also interacting with the user. ChatGPT tends to just want to converse directly with Bing Chat in natural language, and I've had the most success just allowing them to do that. However, without human direction they tend to forget the parameters of their assignment.
For example: https://chat.openai.com/share/c01f8d07-779e-4c93-9862-d7bade5b58ce

10 hours ago, Vantha said:

Btw, don't worry I haven't used anything Ai related for the encyclopedia.

Here's the problem, how can ShadowOfHassen know if you are telling the truth? GPT-4 when it is properly supervised and firing on all cylinders produces text that is (in my experience) completely indistinguishable from what a human would write.

Edited June 18, 2023 by ChronA

[Vantha]

19 hours ago, ChronA said:

I find Bing Chat is too easily swayed, or one could even say distracted, by whatever information it digs up with its web search.

That's something I've also noticed. I still prefer Bing Chat just because it's free and good enough for what I need it for.

 

19 hours ago, ChronA said:

ve also experimented with letting ChatGPT write queries to Bing Chat. In theory that would give you the best of both worlds, but so far my results have been mixed. ChatGPT can easily grasp the idea of being able to pass searches to Bing Chat, but has trouble sticking to a workable syntax while also interacting with the user. ChatGPT tends to just want to converse directly with Bing Chat in natural language, and I've had the most success just allowing them to do that. However, without human direction they tend to forget the parameters of their assignment.
For example: https://chat.openai.com/share/c01f8d07-779e-4c93-9862-d7bade5b58ce

That's a very interesting idea... and from what I can tell it actually worked pretty well.
You probably have a good reason to do it like this, but isn't that task perfect for something like autogpt?
Also, I might be wrong about this, but doesn't Chatgpt Plus use the exact same browsing mechanism as Bing Chat?

 

19 hours ago, ChronA said:

Here's the problem, how can ShadowOfHassen know if you are telling the truth?

That's true, he can't and there's no way for him to tell whether I'm lying. I just personally have a clear vision of how I want the texts to be like and therefore write all texts myself. Also, I like doing research myself on these topics, so having Bing Chat or Chatgpt do everything would cut off the interesting and fun part for me.

[ShadowOfHassen]

23 hours ago, Vantha said:

That's true, he can't and there's no way for him to tell whether I'm lying. I just personally have a clear vision of how I want the texts to be like and therefore write all texts myself. Also, I like doing research myself on these topics, so having Bing Chat or Chatgpt do everything would cut off the interesting and fun part for me.

Actually, i haven't played that much with the advanced chatGPT but I've seen examples of it. I might not always catch it but Chat GPT eats writing on the internet and makes stuff like it and not all writing on the internet is good. I've caught a lot of passive voice and some other things that if you read it aloud, *ugh* it sounds terrible.

 

They might have gotten better though in the 3 months since I've checked.

[ChronA]

@ShadowOfHassen, @Vantha - Thanks for your input. You bring up valid points, but I think it's essential to recognize that the capabilities of language models, especially more advanced ones like GPT-4, are rapidly expanding.

ShadowOfHassen, you're correct that AI models are trained on a vast array of internet text, not all of which is high quality. However, these models learn patterns, not specific content, and are designed to prioritize more credible and high-quality sources. Passive voice or other stylistic issues you've noticed are not inherent flaws of the model but rather reflect the style of the input it's working from. With appropriate instruction, these models can generate content tailored to specific stylistic preferences.

And Vantha, you've highlighted an essential distinction between the free version of ChatGPT and the more advanced versions like Bing Chat and ChatGPT Plus. These advanced versions, equipped with more sophisticated algorithms, have proven to be effective tools for tasks like research, troubleshooting, and even coding assistance.

The key here is that we're not looking at AI as a replacement for human expertise or creativity, but as a highly capable tool. When used effectively, AI can offload routine tasks, expedite the research process, and provide useful insights, enabling us to focus on more complex and creative aspects of our projects. The potential is vast, and it's up to us to leverage it responsibly and effectively. 

[ChronA]

Hello, all - I have a bit of a reveal. This conversation, along with the responses I've posted, Some of the responses I've posted to this conversation, has been part of an experiment in AI capabilities, namely those of the AI model, GPT-4. The experiment aimed to see if anyone would catch on that some of the replies in this thread were, in fact, written by ChatGPT, powered by GPT-4.

Any responses that end with a "cheers" emoji have been primarily composed by the AI. Words in italics are additions made by ChronA, our human collaborator in this discussion. The goal was not to deceive, but rather to demonstrate the level of sophistication and relevance AI can achieve in conversation, particularly in thoughtful, nuanced discussions like ours.

Now, looking back on this experiment, it wasn't as successful as we had hoped. ChronA found the process more time-consuming than simply writing the posts from scratch. And while this isn't indicative of his usual experience with ChatGPT, it raises an important point about the limitations of AI.

In this case, the AI model was biased against making the bold claims we wanted it to make. GPT-4 is designed to be humble about its own capabilities and to avoid taking on responsibilities typically handled by humans. This bias, coupled with its training data only going up to 2021, skewed the conversation away from what we intended. (ChatGPT wanted very much to take your side of the argument @ShadowOfHassen, and getting it to mimic my usual posting style was also a struggle. This is an outlier in my time playing with GPT-4.)

It's a fascinating and somewhat ironic reflection of the current AI landscape. Even in trying to showcase its capabilities, we're reminded of AI's inherent constraints and the critical importance of human collaboration. AI models are tools - incredibly sophisticated and continually improving, but still tools. They need human oversight, fine-tuning, and, at times, a bit of editing to be truly effective.

This experiment has been insightful, and I hope it's sparked some thought and discussion. Thanks for being a part of it! Cheers! 

If you want to see how the sausage was made, check it out: https://chat.openai.com/share/1b56a5c4-5210-4d7e-80a0-67e6fa946df2

[Vantha]

Really? No way, I was about to ask wether Gpt 4 wrote that second response. It seemed very suspicious to me. But the first one ... I didn't notice anything.

 

2 hours ago, ChronA said:

Cheers! 

Haha, I like that

Edited June 21, 2023 by Vantha

[Vantha]

@ChronA I would love to hear your thoughts on Chatgpt plugins. From what I've seen they are not as useful as they seemed when they were inteoduced, but I don't have Chatgpt plus and you seem like a person who has tried them out.

Cheers! (just kidding)

[ShadowOfHassen]

To be honest, something wasn't En_US. I didn't know if it was because @ChronA you weren't a native English or ChatGPT, (or that you don't put 7 levels of editing in your posts) I had considered the possibility and I thought it would be rude to point anything out.

Rereading it now, it doesn't sound at all human...

[ShadowOfHassen]

However, if Chat gpt can look for obscure quotes in ancient text's that'd be cool because It'd be cool to have a quote from ancient texts for each animal in 0 A.D.'s encyclopedia, but I don't have time to read 100+ ancient works right now.

[Vantha]

17 hours ago, ShadowOfHassen said:

However, if Chat gpt can look for obscure quotes in ancient text's that'd be cool because It'd be cool to have a quote from ancient texts for each animal in 0 A.D.'s encyclopedia, but I don't have time to read 100+ ancient works right now.

I guess it could do that, but it has no memory and can't directly search or remember quotes in ancient texts. It would just google it which you can also do yourself.

[ChronA]

22 hours ago, ShadowOfHassen said:

However, if Chat gpt can look for obscure quotes in ancient text's that'd be cool because It'd be cool to have a quote from ancient texts for each animal in 0 A.D.'s encyclopedia, but I don't have time to read 100+ ancient works right now.

PrivateGPT is probably what you need. (https://github.com/imartinez/privateGPT) I haven't actually tried it yet, but from what I've read this is exactly the use case it is built for. The only caveats are: 1) you need to have all of your documents downloaded together into one folder in formats the system is able to read, and 2) privateGPT uses older open source language models, which means it is going to be stupid compared to GPT-4. Still I would wager it would be up to the task of finding quotes about animals for you. That doesn't take a lot of analytical sophistication.

I've been wanting to try a privateGPT project. If you have a folder of the ancient texts I could make a copy of and the list of animals you need quotes for, I might be inclined to give it a shot. No guarantees though. I have my hands full.

[ShadowOfHassen]

I heard in a podcast that ChatGPT could be used to find quotes, which is why I mentioned it.

 

I will try Private GPT but I don't know how well it'd work because the only e ancient texts I can get is anything on project gutenburg.

[LetswaveaBook]

On 14/06/2023 at 8:58 PM, Stev said:

Has anyone started to experiment with this? Also, do you possibly see less experienced people like myself being able to contribute more with tools like GPT? I work in a company and are looking at migrating code from legacy software using GPT so would think it would be doable here too. 

Actually I don't know anything about chat GPT.

But it is a text bot that is incredibly good at what is does. However if you ask chat GTP about the most popular 0ad mods, it will give 10 answers and none is a 0ad mod. So how do you expect chat GPT to write code for an engine that is only used by 0ad?