New Sound Manager svn patch

[stwf]

Hello,

I have attached patches that were generated from by git branch. Sorry there are 5 (the command I used treats each commit separately). They should work with svn although you may need to apply them with the command 'patch -p1 -i 0001-new-soundmanager-code-release-1.patch', etc. Let me know if they don't work......

As mentioned in this blog post:

http://codeprairie.n...s-with-git.aspx

So while the underlying code has been completely rewritten, the only changes to the javascript side were the additions of the classes MusicSound, and AmbientSound to the previous Sound. MusicSound and AmbientSound each play in their own tracks, at their own volumes, and only one can play at a time. When you set a new one it will automatically fade out and release the old one. Regular Sounds, as well as the SoundGroups support for the SoundGroup xml remain the same.

So it seems like a good place to do some testing and firm up the new underlying SoundManager code. There are alot of numbers for things like buffer sizes that can be tweaked for better performance if need be. I can say it works perfectly on my blazingly fast iMac with lots of RAM and an SSD.....

I've tried to keep to the projects coding conventions where I could, any hints to improve the code more than welcome! Good Luck!

PS, still missing a lot of functionality. Position isn't set for the action sounds, many of the parameters in the SoundGroup files are currently ignored, fading throuigh the javascript side is currently ignored as well as setting many of the parameters from javascript (gain is supported however) but in my examples most of these aren't used yet. Anyway there are other things too I'm sure. Enjoy,

NewSoundManager.zip

Edited April 27, 2012 by stwf

[k776]

Thanks stwf. I'm giving this a go now.

For those with git repos of the project, here is stwf's work: https://github.com/stwf/0ad/commits/new-sound-manager

[feneur]

I'll leave it to the programmers to comment on the programming related stuff, but I'll say as a general comment that this sounds very promising Thanks for working on this!

[stwf]

No problem, its been fun!

I do have a new release should be pushed to github, the url is:

https://github.com/stwf/0ad/tree/new-sound-manager

Also there are these svn patches that git created. I attached them here. I'm interested to see if they now work.

This version cleans up the logs, fixes the crash on exit, and a few minor tweaks. Also the parameters for sound buffer size, and the number of buffers to keep in memory per item can now be set through the config file. So we can start to tweak these things if the values I'm using don't work out in the wild... Enjoy.

4_28_12_sndCode.zip

[k776]

Excellent work. Your latest changes work flawlessly. It looks like this commit (https://github.com/stwf/0ad/commit/e118770c61f84b8c4c14fbecd79a0cb7d9f43077#diff-0) stopped actionGain from being set though.

Other than that, it's working great on Mac. Have you been able to try this on Linux and Windows?

If it works on all 3 operating systems, then we can start moving ahead to include it into Alpha 10 hopefully :-)

Anything I've missed from the TODO list?

* Sounds intensity based on position

* Sounds intensity based on zoom level

[feneur]

Excellent work. Your latest changes work flawlessly. It looks like this commit (https://github.com/stwf/0ad/commit/e118770c61f84b8c4c14fbecd79a0cb7d9f43077#diff-0) stopped actionGain from being set though.

Other than that, it's working great on Mac. Have you been able to try this on Linux and Windows?

If it works on all 3 operating systems, then we can start moving ahead to include it into Alpha 10 hopefully :-)

Anything I've missed from the TODO list?

* Sounds intensity based on position

* Sounds intensity based on zoom level

Please be very sure there are no issues before including it. Unless it's extremely stable I would wait until after Alpha 10 is released, it is after all just over a week until the feature freeze, so while it sure would be nice to have sound on Mac as well I definitely don't think we should do anything that would lessen the experience on the other platforms. Even if everything works at least as well on Win/linux I'm still a bit wary about including something as significant this close to a release. Better to include it right after the release and make sure everything works well for Alpha 11 imho.

[stwf]

Thanks! You are right about this commit, my mistake SoundGroups have their own volume, so once that is supported(soon) ths won't be used.but I. Didn't mean to leave it out of this build... My mistake... I will fix that tomorrow...

I don't have a windows build nvironmentnalthough i can run win and Linux under VMware. Is it easy to set one up? Anyone have a good link explaining it?

[historic_bruno]

VMWare is really easy to setup (or you can use VirtualBox as a free alternative, but VMWare is slightly more polished), you just create a new virtual machine, insert the CD or ISO of the install disc, it will normally detect the OS automatically, then you create a virtual hard drive of whatever size you want, then it will boot the installer disc and away you go Afterwards you'll want to install the guest support tools if available, that gives you copy/paste between guest and host, 3D acceleration, etc.

[WhiteTreePaladin]

Virtualbox is pretty easy to work with. I'd recommend it. Free is always nice

[stwf]

Hello all,

Thanks for the help, I got Ubuntu running in VMWare, not too bad! Why is Copy/Paste different in different apps? lol

I posted a diff file to trac:

http://trac.wildfiregames.com/ticket/1223

This version builds on mac and linux

Fully supports the SoundGroup.xml as well as allowing for control of the Sound item from Javascript.

The missing feature is for position. It seems like the implementation of this merits some discussions here.

Bugs carried over from the previous code include the playing of sounds that aren't on the screen and special intensity levels which also probably needs some discussion. I think the code is pretty solid, but probably wouldn't recommend Alpha 10 unless you plan on holding it off a bit. Of course its up to you either way.

[Pureon]

Do you need us to decide on how the camera position affects sound before you can progress further?

Excellent work

[stwf]

Thanks... After much thought I think i have an idea, but any feedback is appreciated. Time spent discussing now can save a lot of coding time later!

I was thinking that the real 3d is very tricky and probably not of much help to the player. Since the only representation they have is the 2D screen. My idea was to map the location of the sounds to the location of the image on the screen. Just ignore everything about the location except for its right<-> left positioning (screenwise), assume the left side of the screen is 45 degrees left of center, the right side of the screen 45 degrees to the right, and place the various sounds along that range. Then put offscreen sounds that are set to play anyway a bit further out than that than that. This would put everything at the same distance from the listener. It could be a nice touch to have far distant sounds be quieter (in a low shot), but in most game situations the distances aren't different enough to convey useful information to the player.

Even if someone has a full surround sound system I don't think that playing a sound behind them would prompt them to scroll to the bottom of the screen. But a sound full in the left ear might get them to look to the left hand side of the screen.

Feel free to tell me if this is totally off-base. Between learning the system and writing the code I've actually played very few games :-(...

[Pureon]

I'd be interested to hear what others think, but I don't see a problem with sound working on a 2d plane as you describe. I haven't tried your upload because I don't know how that all works, but I know a lot of Mac users are looking forward to hearing in-game sounds

[stwf]

Hello Gang, I have a new version of the sound code. Mostly just adds the 0AD boilerplate at the top of each file and some minor code cleanup. I've started working on the code for locating sounds and should have that soon...

BTW the new forum wouldn't let me upload a .diff file. I had to add a .txt suffix to get it to go...

new_sound_manager_2.diff.txt

[WhiteTreePaladin]

Good to hear you're still working on this.

[feneur]

The forum didn't allow .diff before either, previously you've either been attaching .zips or at Trac I've added the .diff attachment type now though, so you should be able to do things either way And again, thanks for working on this, it's greatly needed

[quantumstate]

I was thinking that the real 3d is very tricky and probably not of much help to the player. Since the only representation they have is the 2D screen. My idea was to map the location of the sounds to the location of the image on the screen. Just ignore everything about the location except for its right<-> left positioning (screenwise), assume the left side of the screen is 45 degrees left of center, the right side of the screen 45 degrees to the right, and place the various sounds along that range. Then put offscreen sounds that are set to play anyway a bit further out than that than that. This would put everything at the same distance from the listener. It could be a nice touch to have far distant sounds be quieter (in a low shot), but in most game situations the distances aren't different enough to convey useful information to the player.

The idea that I have had is to first take the 2D positions of all the units and then do the following transforms:

1. The visible area on screen is distorted by perspective, first make it nice and rectangular. The player is positioned so that the left and right sides of the screen are at about 45 degrees as you said. The green dot shows where I think the player should be in terms of height.

2. Take everything that falls outside of the box and scale it by some large factor, the factor should be determined by what type of sound it is (defined in some config files) So for woodcutters it would be really big so sound falls off really fast. For battle noise it should lower so I can hear a battle in the distance.

Hopefully these diagrams will help show what I mean.

[khegarty]

Thanks... After much thought I think i have an idea, but any feedback is appreciated. Time spent discussing now can save a lot of coding time later!

I was thinking that the real 3d is very tricky and probably not of much help to the player. Since the only representation they have is the 2D screen. My idea was to map the location of the sounds to the location of the image on the screen. Just ignore everything about the location except for its right<-> left positioning (screenwise), assume the left side of the screen is 45 degrees left of center, the right side of the screen 45 degrees to the right, and place the various sounds along that range. Then put offscreen sounds that are set to play anyway a bit further out than that than that. This would put everything at the same distance from the listener. It could be a nice touch to have far distant sounds be quieter (in a low shot), but in most game situations the distances aren't different enough to convey useful information to the player.

Even if someone has a full surround sound system I don't think that playing a sound behind them would prompt them to scroll to the bottom of the screen. But a sound full in the left ear might get them to look to the left hand side of the screen.

Feel free to tell me if this is totally off-base. Between learning the system and writing the code I've actually played very few games :-(...

I agree stwf on the 45 degree thing for the sounds, but have one thing to add. To keep consistent with how audio panning is usually set up I would suggest we have hard left/right be at 60 degrees left and right of center. (For audio usually -63, +63 are used, but I don't think 126 wouldn't be a simple number to work with coding wise so 120 instead)? If this doesn't make sense in coding then 45 will work fine.

I also agree that putting a sound behind someone will not make them look down, using left/right and increasing/descreasing volume I feel could solve the up down issue. We could make distant sounds quieter and I don't know what the options are in coding, but possibly have a little more reverb on a sound the farther away it is, to help a player preceive distance. I understand this may not be possible in code.

quantumstate, I am still a little confused by what you mean, could you describe this again? Sorry!

[historic_bruno]

Surely it won't be hard to tweak the exact behavior? Compared to say, the challenge of rewriting the entire sound system Honestly I have no idea how the above suggestions would sound in practice, I would rather hear them for myself.

[stwf]

Surely it won't be hard to tweak the exact behavior? Compared to say, the challenge of rewriting the entire sound system Honestly I have no idea how the above suggestions would sound in practice, I would rather hear them for myself.

This is actually the correct answer OpenAL is pretty flexible, so things like custom volume rolloff, doppler effect, and applying reverb to single sources is straightforward. So getting these details right can be an iterative process.

The current system only provides a flag, eOmnipresent that is defined as playing a sound at the listeners location. It does make sense to add one for a sounds with direction that should be played even when off screen. All of the sounds are specified in xml files. So a custom rolloff for each could be added to that pretty easily.

I do wonder how much you want to hear things that are offscreen. Of course you want to be notified of attacks as they start, but actually hearing all of the sounds of battle while you are trying to do something on the other side of the screen could be distracting.

[Pureon]

I do wonder how much you want to hear things that are offscreen. Of course you want to be notified of attacks as they start, but actually hearing all of the sounds of battle while you are trying to do something on the other side of the screen could be distracting.

I agree, I'd like to hear a warning sound when one of my units is under attack, and maybe see a ping on the minimap to show where it is happening. Battle sounds could probably travel further than other less critical sounds, but not to the other side of the map.

[Sonarpulse]

I'm not sure whether your system allows for this, but it would be fun to have a mode were all sound is done realistically, a la 3D simulation with camera position as ear. Maybe it wouldn't be to practical for gameplay purposes, but it would be kinda neat. And hey, it's sort of a trade off, the more of the map you see, the more your hearing is impaired.

[quantumstate]

I'm not sure whether your system allows for this, but it would be fun to have a mode were all sound is done realistically, a la 3D simulation with camera position as ear. Maybe it wouldn't be to practical for gameplay purposes, but it would be kinda neat. And hey, it's sort of a trade off, the more of the map you see, the more your hearing is impaired.

That sounds like how the current sound system works.

[Ykkrosh]

it would be fun to have a mode were all sound is done realistically, a la 3D simulation with camera position as ear.

That's what the current implementation does. It's only 'realistic' if you imagine you're in a (silent) helicopter high above the ground watching through binoculars, i.e. not very realistic at all . (And it's not good for playability since off-screen objects will usually sound about the same as on-screen objects, which is confusing and unhelpful and unintuitive.)

[historic_bruno]

BTW, I tested the patch on Windows and left some comments in the chat log