Ykkrosh

Thanks for starting the work on this! I haven't looked at it in detail yet, but I think it's what we want so we can stop relying on the bundled copy of the library. A few random points about our usage: * We don't need 64-bit Windows support. (We use a load of other libraries and it'd be a pain to recompile them all for 64-bit, and 32-bit is perfectly good). * Windows binaries should be compilable with MSVC (not e.g. MinGW), probably 2005 (for compatibility when people compile the game with that version). Currently we use the .vcproj files, I think; I don't know how it'd work with an autotools-based build. * On Linux / OS X we need to be able to optionally build from a bundled copy instead of an installed system copy, because it'll be years before fcolladaCE is available on all Linux distros. That should work without needing to be root and installing things in /usr etc. We can point at whatever relative paths are needed. * Currently we compile separate Debug and Release versions of FCollada (to different filenames), and use the appropriate one when the game is compiled in Debug or Release mode. I'm unsure whether that's necessary - we don't really need it for debugging, but I don't know if the library has problems when it's compiled in a different mode than the game. (Maybe on Windows, where debug vs non-debug CRT DLLs cause conflicts?) * For testing FCollada in the game, running binaries/system/test (or test_dbg) after building the game will test it a little. To use it in the actual game, first delete ~/.cache/0ad/ then launch the game. (We only convert a model once using FCollada, and then cache the result, because conversion is slow.)

I'd prefer not to make the page too 0 A.D.-specific - most people reading it probably won't care about our game. But I'll try to set up a page on our wiki documenting what extensions we use (as required or optional), since currently that information is scattered throughout the code and it'd be nice to get a coherent view. (Got 918 distinct device reports now, by the way, stretching back to GeForce2 MX. Not bad for a week's data collection )

Water effects require the scene to be rendered three times (once for reflections, once for refractions, once normal) so they make it much slower, and the renderer is a bit slow anyway. We're working on optimisations that should make it much better in future releases

Hrm, okay, that's the shadow bug - should be fixed in the SVN version, but the only way to avoid it in the alpha release is to disable shadows

Outlines in 3D are a standard cel-shading effect, but as far as I'm aware there's no very fast way to do them. In particular you need to compute and store more information with every vertex in order to render a decent-quality expanded version of the model, and our models are animated so we'd have to compute it all every frame (and we might have hundreds of models). You also render a second copy of every model to act as a stencil to cut out the middle - the outline gets coloured where the expanded model is drawn and the non-expanded model is not drawn. So it's more complexity and slightly worse performance, and probably not worthwhile unless people think it'd be a hugely better effect. Toggling can be added easily when we have a better graphics options system. The difficulty is what to do with overlapping silhouettes, of the same colour or of different colours. We can't control the order in which models get drawn (because of performance concerns), so they're effectively random (though consistent from frame to frame). If we could use additive blending, like this, the order doesn't matter since it'll just add all overlapping polygons. If we want standard blending, like this, then whichever model randomly happens to be rendered first will be the one you see, which goes a bit weird for units consisting of multiple models (see the archer's vanished heads (or, rather, don't see them)). But the latter case will be fine if units are all the same colour, and probably players won't really notice when multiple colours are overlapping a little weirdly, so we can do that if people don't like additive blending. Maybe someone could make a nicer screenshot first?

Aken is just a high-res timer, and it should be independent of water/shadows, so I don't think it should affect this. Does the game run okay if you keep water enabled and just disable shadows? Does it show any error messages in the top-left corner of the screen when running? (There's some shadow bug in Alpha 4 that affects some graphics devices, but I wouldn't expect it to affect this one too.)

Either sounds fine to me. That is what I want (in an ideal world where we have time to not just kludge it ). If we want to append/split/etc paths from the OS, we don't have to know the encoding, since we still know e.g. the code unit '/' is the directory separator regardless of encoding and regardless of code unit size (given that Linux will only use ASCII-compatible encodings). If we want to append paths which come partially from our own code (mod names, saved game names, etc) it's best to just require our path fragments to be ASCII (we won't show the filenames to users - for i18n we need to store names in multiple languages in some metadata file anyway, and also people might want names with '/' and ':' etc so we couldn't represent the full names in the filename anyway), which means we can append byte strings and not care about the encoding. When someone is running the game from /home/josé (last component is 4 bytes) with locale es_ES.iso-8859-1, decoding and encoding as UTF-8 would break (either with a decoding error, or with a 3 or 5 byte re-encoded output). Boost Filesystem V2 at least decoded/encoded using the current locale, so it would work as long as the locale was configured correctly; V3 on POSIX uses byte strings instead so it should work even in misconfigured environments. (OS X often has misconfigured environments, it seems.) We need some platform-dependent conversion functions anyway, because std::ofstream wants char* on Linux. Otherwise it seems the most common situation is Join(NativePath, std::wstring) which can assume the second argument is ASCIIish and convert to native automatically, and LOGMESSAGE(L"Loaded %ls", NativePath.string().c_str()) which can convert to wstring by trying UTF-8 and falling back to ISO-8859-1 (it doesn't matter if that's lossy since it's just a log message).(If there's anything I can do to help with this then I'd be willing - I don't mean to sound like I expect you to do everything here . If there was some NativePath class/typedef that was implemented as std::wstring using UTF-8 everywhere, I think I should be able to change that to std::string on Linux/OS X and fix up all the conversions and operations and whatnot, if that would help.)

A thing.

Getting rid of Boost Filesystem means we won't have to worry about the V2 API getting dropped in future releases, which is a nice side-effect of this. Boost doesn't use the namespace "fs", so I don't think it'd be confusing to have something silently Boost-like (not advertised as a drop-in replacement, but preferring compatibility when possible) reusing that namespace. Anybody new to the code would have no expectation that it's like Boost, and anybody old will be aware of this change. I think a namespace is important because otherwise the function names are misleading, e.g. I'd probably expect Join("a", "b") to be equivalent to "a"+"b" and wouldn't expect it to be path-related. SHARED_PTR looks very ugly. Can't it just be shared_ptr? (and include using 'using' or 'typedef' instead of macro) Since you're assuming shared_ptr exists either in std:: or std::tr1:: (if Boost is disabled), could you also assume (for your set of compilers) that unordered_map exists there too, instead of using STL_HASH_MAP which has much less predictable behaviour? (Regardless of that, outside of lib we can rely on Boost and should probably get rid of STL_HASH_MAP entirely and use unordered_map, for modernity and consistency. That's probably separate from this patch, though.) I don't remember ever accidentally mixing VFS and non-VFS paths, such that the type system would help. But I do find VfsPath very useful as documentation, since it stops me getting to the stage of accidentally mixing paths, so it sounds good to keep it. Attached a patch that fixes a few issues on Linux, but some errors remain, of the forms: error: ‘path_from_wpath’ was not declared in this scope error: no matching function for call to ‘std::basic_ofstream<char, std::char_traits<char> >::basic_ofstream(const wchar_t*, std::_Ios_Openmode)’ This brings up the old encoding problem - paths on Windows are natively 16-bit strings, paths on Linux are natively 8-bit, and conversion can be lossy. With fs::wpath, Boost decoded 8-bit paths with the current locale (which caused this since people have misconfigured locales, but it mostly worked). We used external_file_string for std::ofstream, which returns 8-bit or 16-bit depending on platform. If we want to fix things properly, I think instead of replacing the non-VFS fs::wpath with std::wstring we should replace it with a platform-dependent path type which uses std::string on Linux and std::wstring on Windows, so no lossy conversions are needed, with suitable handling of string literals etc (which we can probably assume are ASCII). (VfsPath would remain std::wstring and we could require the VFS directory contents are encoded as UTF-8 on Linux regardless of locale, probably). That's basically what Boost Filesystem V3 does. If we want to fix things quickly, we can assume most people on Linux have UTF-8-compatible paths (which we mostly assume already), so path_from_wpath just does UTF-8 encoding, and for std::ofstream we can add a native_path_from_wpath that encodes (on Linux) or returns the std::wstring (on Windows), and it probably won't break for many people. In the latter case, maybe we should at least use some typedef of std::wstring for non-VFS paths (NativePath? PhysicalPath?), so we can relatively easily change it to a native-sized path in the future if the UTF-8 assumption is sufficiently invalid. reduceBoost2.patch

Got a lot more data now, so it became very slow, so I made some changes to get the GL report page adequately fast again. (There's a batch process that extracts the relevant data into a few SQL tables, instead of having to parse a hundred megabytes of JSON every time someone loads a page.)

Run "make CONFIG=Release clean" before running the normal "make ..." command.

In AoM it looks like the terrain texture gets changed once the building reaches 100% completion (and stays permanently changed). When you destroy/delete a completed or under-construction building, it creates a pile of smoking rubble underneath the building's old location, which disappears after tens of seconds. Looks like AoK is similar - change texture at 100%, display temporary rubble on destruction. It only seems to support about one angle of slope, so it doesn't have to care about steep terrain, and there's no flattening. Not quite - I think what we could/should do is allow units to walk across tiles that were not steep at the start of the match. We can still have units and buildings be blocked by steepness if we want (unlike AoM's approach of ignoring slope and just using a special impassable terrain texture for mountainsides/cliffs), but the computation of steepness will ignore dynamic changes caused by building placement. If a unit's limit is 30 degrees, and a building flattens some terrain and causes a nearby tile to tilt from 29 degrees to 31 degrees, the unit will still be able to walk on it. If buildings are not allowed on very steep terrain, they'll never cause a huge amount of tilting, so we won't have a tile tilting from 29 to 89 degrees and nobody will really notice the discrepancy. (The fundamental issue I want to avoid is a unit being on passable terrain, then constructing a nearby building, and suddenly finding itself on impassable terrain, because that messes up the pathfinding code quite badly - it doesn't work when a unit's path starts on an invalid tile. We still could hypothetically have other effects that leave permanent gameplay-affecting changes to the terrain, e.g. a meteor crater or a rising water level, as long as we added special code for handling units which would suddenly find themselves on invalid tiles, e.g. by crushing them or drowning them; but it's not fair to kill units just for being on steep terrain, and I can't think of any other good ways to handle that case, so it seems best to simply never make a tile become invalid in that way.) My concern applies to non-excessive flattening too (e.g. a tile tilting from 29 degrees to 31 degrees) since that can still cross the threshold of passability vs non-passability. Given that we seemingly want some non-zero flattening in certain cases, I'd like a robust solution to that, rather than one that's based on carefully balancing constraints. But once we have a solution I don't care how much it's used - if we restrict buildings to fairly flat land, or make farms not flatten terrain, or give all buildings one-metre-deep foundations so we can visually tolerate terrain height variations of one metre and only flatten any variations stronger than that, that's all fine and it's just an art design issue. (Personally I'd agree it's probably best to minimise the use of flattening, since it's kind of an ugly solution to the problem, and there are better solutions for farms at least.)

(Why "CONFIG=Debug make"? Debug mode will make the game extremely slow, and it's usually not useful except for developers, so I'd expect packages to use CONFIG=Release instead.) I imagine that might be causing problems, since SpiderMonkey may get confused and fail to compile some necessary code. The "x86_64-chakra-gnu-linux" comes from your CHOST, but where does that come from? The only reference on the web seems to be this post, so I suspect you have a peculiar configuration . It should be like "x86_64-pc-linux-gnu", or maybe "x86_64-chakra-linux-gnu" if you want - note the order of the last two words.

Hmm... In AoM, it looks like steepness is ignored entirely - units can walk up near-vertical slopes, and buildings can be placed on them. When a building is placed, the construction-site mesh appears immediately (possibly partially floating in air or stuck into the ground); once a builder has reached it, and other units have moved out of the way, the terrain instantly gets flattened. Units will never get stranded by the flattening, because they don't care about the steepness. Building near water can cause land tiles to become submerged, and can cause water tiles to become dry. Pathfinding seems to ignore that entirely - units can still walk across the newly-submerged areas but not the dry areas, while ships can still move along the dry areas and not the newly-submerged. That seems like cheating and/or a bug, but I suppose it's a reasonable way to avoid overcomplicating the issue. Determine all pathfinding based on the initial state of the terrain, and let the building flattening be a purely cosmetic effect that the gameplay code never even notices. Then there's no way a player could exploit it to trap enemy units, or to get across an impassable mountain, etc, and it'd be easy enough to implement that way.

As soon as you click to place the building? (even if e.g. animals or friendly/enemy units are currently standing in the way?) (even if you immediately delete the foundation and get all your resources back, so it's kind of a free terraforming tool?) (I'm not familiar enough with other RTS games to know whether this kind of thing would be exploitable for unfair advantages.) No. A totally different approach could be used for that, though - probably some kind of flag in the actor which means it takes the (x,y,z) of the parent actor's attachpoint and then replaces the y coordinate [the vertical axis] with the terrain height (or water height). I think that'd be easy to add - is it something you'd find useful?

You probably need to install an updated version of nasm (via MacPorts or whatever).

I added some like this, based on data files like this - you just set an actor to use a decal instead of a mesh, and it turns into a textured rectangle that follows the terrain. This also avoids z-fighting problems (the ugly flickering that sometimes occurred with the old decal meshes when they precisely overlapped, visible in e.g. the Pathfinding Demo map with the maze of corrals). With the foundations, the mesh is still hanging out into the air. What should we do about that? (Flatten the land immediately when the foundation is placed, so the mesh is never on uneven ground? Change the mesh to have some parts going vertically downwards below the zero plane, so it looks like it's still got support if the terrain is lowered? Do some fancy dynamic mesh modification to conform to the terrain?) Some things that still need to be added: (How important are these to people?) * Actor Editor support. * Floating decals (e.g. for lily pads, where using decals would probably be good to prevent z-fighting). * Support for large numbers of special-effect decals, e.g. shadows or footprints or pools of blood. (Currently it's just designed for buildings, which are few and large and stationary.) (Also someone probably needs to update all the actors to use the new decals.)

Are you checking out from http://svn.wildfiregames.com/public/ps/trunk/ ? That seems to work for me without asking for any authentication.

On this subject, I just saw something suggesting per-vertex lighting values as a way to do ambient occlusion, which would be much simpler since there's no need for texture unwrapping and would have no rendering cost. I'd guess our buildings have too few vertexes / too many large polygons to make that look decent, though.

The biggest problem with multiple planes is the cost of rendering reflection/refraction textures for each one. We could add a much simpler water shader that just reflects the skybox, instead of computing dynamic reflections - that would mean multiple planes have the same cost as a single plane (and a low enough cost we could probably enable it on low-end graphics hardware) and might look fine in most cases. Then the issue is just editing tools, which probably shouldn't be too problematic (e.g. we could represent water as a heightmap instead of a plane, and then have 'fill' and 'empty' tools that do a kind of flood-fill to let you easily produce flat bodies of water, with independent rendering settings for each body, or whatever). Yeah, there's been many iterations . I think the current one is needlessly inefficient and could be improved significantly, and it needs to be integrated with actors, and ideally it'd be integrated with Atlas somehow, but otherwise it should be okay.

They're not really separate renderers, they're just separate modes within the renderer, and scaling down quality inevitably needs multiple modes; but I believe we could remove some of the existing modes and make the new ones more consistent with each other, so they could share most of their code and be a bit simpler. But I think this is what I mean about not getting too distracted - I could get tempted to redesign lots of stuff, but it probably wouldn't be very well designed and it wouldn't really help the game by itself, so it's probably better to focus on real features and only redesign things when they're relevant to making those features work. (That has to be balanced with the long-term cost of hacking new stuff on top of an existing design that can't support the complexity, but I don't really worry about that since I'm in much greater danger of going too far the other way and redesigning things that don't need to be )

About the attack/defense stats: In theory, we could just make a giant table of every unit type vs every other unit type, with two numbers in each cell giving the frequency of attacks and the probability of reducing LP on each attack. Any system of skills, shields, bonuses, etc, is just a shortcut for defining that table, to make it easier for us to design and easier for the player to understand the relationships between units - I think that's an independent issue and less fundamental, gameplay-wise, than the concept of making randomness significant and the consequences that the statistics will have on how players micromanage units and perceive risk. About directions: Currently units do have directions but they can turn instantly - you can only hit them from behind once before they'll turn to face their attacker. We can't get rid of instant turning, because it would make pathfinding way too complex (since units would move in curves instead of straight lines). But we could do things like calculate a flanking bonus that is triggered when attacking from a certain direction and then remains for 5 seconds (maybe decaying down to 0% bonus over that time) so it's more persistent. Also we could reduce the value of direction-micromanagement by calculating how recently the unit turned to face its attacker (if it turned 0.5 seconds before the attack then it reduces the flanking bonus by 25%; if 1 second before then reduce by 50%; if 2 seconds before then no flanking bonus at all; so you don't gain anything by rushing to turn all your units a millisecond before they're attacked). Alternatively, if we had rigid formations then we could calculate damage based on the position in the formation rather than based on the individual direction of each unit.

I think the difficulty with particles is mostly the artistic aspect - there's a huge variety of possible effects, so the problem is working out which ones to implement and how to expose them to artists in a usable form. The code itself should be fairly straightforward. That's what I meant by the term "billboard" . Either make them directly face the camera, or rotate them just around their vertical axis. That's easy to do, and it's necessary for particles too.

Hmm, interesting suggestion. In terms of details, I think it's probably useful to approach it from the other direction: What is the desired behaviour of battles? Considering the simplest case of two identical units fighting each other, under the current system the fight will always take (for example) 10 seconds and the unit that initiated the attack will win but have 1HP left. If it's 2v1, it'll take 5 seconds and the 2 will win and one will have full HP while the other still has half HP, so you're significantly better off (especially if you have healers). The consequence is that you win by micromanaging all your units to focus on a single enemy at a time (because the sooner you kill an enemy, the sooner you'll reduce their total damage-per-second and shift the balance in your favour), and by withdrawing heavily injured units. If two players micromanage equally then whoever has most units wins and the outcome is entirely predictable. I think your suggestion makes a fundamental change: there is very little value (actually negative value) in focusing all attacks on a single enemy. (Warning: maths ahead). Say each unit rolls the dice once per second, and there's a 10% chance of killing the enemy each time. If two units target separate enemies, there's a 1% chance of both getting a kill, 81% chance of both missing, and 18% chance of getting exactly one kill, so you'll get an average of 20% of an enemy casualty per second. If two units target the same enemy, there's a 1% chance of both killing that enemy simultaneously (so one of their kills is wasted), 81% both missing, 18% one hitting, so you'll average 19% of an enemy casualty per second. Similarly, with 10 units attacking different enemies you'd get an average 100% of a casualty per second; with 10 attacking the same enemy you'd get 65%. So it's better to attack more enemy units at once (the opposite of the current system), but it doesn't make such a huge difference anyway. Since standard units can't be in a partially-injured state, you can't withdraw them from battle to heal. So it seems this would greatly reduce the effect of micromanagement on combat. Is that a goal or an antigoal? It may be useful to compare graphs of the probability that a unit kills its target after a given time, in a 1v1 battle. In the current system there's 0% chance of killing it before (for example) 10 seconds, and 100% after 10 seconds, so you get a shape like this. With a system where there's a 10% chance per second of killing it, you get a shape somewhat like this: there's a 65% chance of killing before 10 seconds, 88% before 20 seconds, 96% before 30 seconds, 99.8% before 60 seconds, etc. (In theory they could carry on forever). It's a far more gradual curve than the 10-second-step, so the length and result of the battle is far more unpredictable. But in larger battles, the randomness gets averaged out. In 10v10, every second there's a 35% chance of killing nothing, 39% of killing 1 unit, 19% of killing 2, 6% of killing 3, so there's quite a bit of variation. In 100v100, there's a 90% chance of killing somewhere between 5 and 15 - the variation is much lower. Total War games presumably have large numbers of units (does its dice-rolling match the number of graphical units, or is it an independent number?) so most of the randomness gets averaged out and the battles tend to go the way you expect. The danger of using the same system with a much smaller number of units is that the outcome depends much more heavily on fate-whims, and players will be unhappy if the game feels totally unpredictable and uncontrollable. Increasing each unit's Life Points is a lot like increasing the total number of units: the randomness will get averaged out. The extreme case is when a unit has maybe 100LP, i.e. exactly like our current system of hitpoints, where there's basically no randomness left. Similarly, increasing the rate of attacks (say a 1% chance of killing the target every 0.1 seconds, vs 10% every 1 second, vs 100% every 10 seconds) will reduce the random variation. So I think the critical questions are: How many units do we want in battles, and how much randomness do we want? If I put up 5 units against 10 equivalent enemy units, is it acceptable if I have a 30% chance of winning, or a 2% chance of winning, or what? If I put 50 up against 100, what chances are acceptable? If we have answers to that, then we could calculate whether giving every unit 1LP would be acceptable, or if giving them all 2LP or 10LP etc would be much closer to the desired behaviour. I think the maths is a fairly straightforward use of statistics - the important thing is work out the game design first (to a rough level - details can be tweaked later), and then we can design the computations to implement that. How much combat micromanagement do we want to be possible, and how big do we want battles to be, and how random should they be?

As a continuation of various earlier ramblings: I've been thinking a bit about graphics recently, and I'd like to do some work on that for alpha 5 before I get bored of graphics again, but probably need to work out what's most important to focus on so I don't get too distracted. Performance is always nice but it doesn't really help get the game complete - simple optimisations should still be done, but at this stage it's probably better to be driven by user-visible features more than by squeezing out every last performance improvement, since optimisations often limit flexibility and will make features harder to add later. The most relevant things I currently see are: Display obscured units (draw player-coloured silhouettes when they're behind buildings/hills/etc) - necessary because currently it's too easy to lose units. Terrain decals (arbitrarily-rotated flat textured rectangles that lie on the terrain/water) - needed for building foundation textures if they're not on perfectly flat land (or hang out over the edge of a flattened area, etc), and probably for nice farms. Also useful for lots of decorations and graphical effects (lily pads, leaves, blood, footprints, nicer unit selection circles, etc), so they work on non-flat terrain and so they don't suffer from z-fighting when overlapping. Billboard sprites - would artists want to use these for decorative objects (tufts of grass etc)? They wouldn't have the problem 3D meshes have when you see one polygon at a steep angle and it vanishes and looks odd, and they'd need fewer polygons, but they would have the problem of maybe looking odd when rotating the camera (though players usually won't rotate that often). Nicer FoWed-unit rendering. Distance fog. Fix broken shadow/reflection culling. Particles - there's some code but it probably needs lots of fixing to be usable. Weather effects. Nicer shadows. Specular maps. Normal maps. Bloom filters. Are there things I'm missing that would be worth adding to the renderer eventually? What features do people consider higher priority to work on first?