janwas
-
Posts
2.733 -
Joined
-
Last visited
Posts posted by janwas
-
-
Thanks for tackling this
I am very busy ATM, but wrote a good deal of the existing scripts and might be able to help out if you get stuck somewhere (or are wondering 'why did they do it this way') - just send an e-mail.
-
Sounds like the main menu (which is fairly simple graphics-wise) is rendered quickly enough and limited by vsync. However, the in-game performance is far too low, as expected. That explains mouse lag, since on Linux we render the cursor via OpenGL (not via hardware mouse cursor).
What you need to do is get proper hardware accelerated drivers (maybe the proprietary ones).
-
Without a real hardware-accelerated graphics driver (which would probably also support S3TC), the framerate is expected to be really low, thus explaining the mouse lag.
-
Thanks for paying attention to these warnings!
I think it's a general sign of shoddy quality when you compile other code and see TONS of warnings. We've generally been pretty clean, and that makes the remaining warnings stick out (as they should).
What remains are warnings about wdll_delay_load and the archive-member-inaccessible thing.
As with Philip, I think it is useful to keep these because they should eventually be fixed
The delay-loader stuff is from sample code that is pretty crappy and should be overhauled or removed (since most of its raison d'etre is now gone).
Fixing the archive-member-inaccessible things would have a slight cost of its own: if those source files are removed and we end up needing to move something from .h into .cpp or just add them, that would require another svn add+rebuild workspace. I think it is easier to do that once as soon as a new header is added, and just tolerate those warnings.
It's probably good to leave them enabled, though, because if the source file isn't empty, it is good to know (e.g. os_cpu - that's telling us that the automatic error name association thing is broken due to MSVC's library linking scheme. it used to work before we had separate static libs).
I'm not seeing the delay-load warning in VC2010. Can you confirm that the linker command line includes /DELAYLOAD:zlib1.dll and that you were compiling in Release mode?
-
Very much like this idea! At one point, there was work on an online error reporting system, but the cart apparently got stuck in the mud. I still think even that would be helpful, and the other information we could collect/tell users even more so.
Libcurl and simply sending all previously existing logfiles whenever the executable is started sounds good. (When our process has crashed, it's better to minimize activity, and transmitting immediately is a bit riskier than writing to an (ideally previously created) file and transmitting next time.)
-
Greetings! Any chance you could compile the game in debug mode and run in the debugger? Unfortunately those error messages by themselves do not tell us much (the wutil one is just telling us something else failed).
The second one in whrt.cpp requires debug output to diagnose. Even if you're not running Visual Studio, you could get at it with a little app: http://technet.microsoft.com/en-us/sysinte...s/bb896647.aspx
Could you please post the line that begins with HRT?
-
The reason it's called RangeOp, by the way, is because the "operation" we're performing on ranges (sum in this case) could be other operations too. However, there's probably a better name for this kind of datastructure. This is just what I've seen it called before.
Yeah, I've seen this data structure used for "Range Minimum Queries", but since it generalizes to any semigroup, RangeOp seems a decent name.
-
Interesting idea. Just for comparison, Vietcong had this feature. If you got shot up, a medpack would only heal half the damage, and the other half was marked a darker shade of red in the HP bar (as you suggest).
-
Works fine after a minor change
-
(By the way, it looks like DST stops around the end of this month in most places, so we need to be careful about that when suggesting times.)
That always causes problems, heh.
It's a great opportunity to change the meeting time, though, because we'll need to adjust anyway.
I find it very valuable to have your contributions whenever you're able to have time, so it'd be good for me for the time to be convenient for youGlad to hear it, I will continue to try and make it. Was just saying that there's not a huge difference whether it's early or late, and DST switch will help as well.
-
I can definitely understand that time being too early on the US west coast.
2000 GMT translates to 2200 local here, which makes it unlikely that I can stay for longer than 1.5 hours and also makes it harder to dodge other evening activities. That's probably not critical, though, it is more important that Philip and other contributors find the time convenient.
-
Thanks for the report. Successfully duplicated and fixed - it was just a matter of the (32-bit in 32-bit builds) statistics counters overflowing.
-
We are using 1.35 in the Windows build. Newer versions will probably have a few conflicts, because they changed some function names in boost::filesystem, but nothing major. If you grab a newer version, a patch fixing these differences would be appreciated
-
(Threaded compression is fun - everything is nice and smooth (at least on dual core) and the textures slowly but magically pop into existence.
heh, sounds nice
I don't think I can even use WriteBuffer since it can call io_Allocate which can use the global AllocatorChecker, which is unsafe by itself and also calls debug_assert which is unsafe.yep, there's all sorts of stuff to check - but that's also something I can do at work later this year, since the proportion of parallel sections / threaded code is always increasing.
Sync loads are needed about once in the whole engine, so they're not very interesting. Prefetching won't call Bind (since the whole point of prefetching is to load textures before the renderer first needs to bind them and render them). So Bind will always be associated with the load-synchronously-unless-it's-going-to-take-too-long behaviour.OK, makes sense. As long as that is mentioned in the doxygen, it should be fine
Looks like the design here is pretty much hashed out, which is good because I won't have any time tomorrow and will fly out on Thursday.
-
(Hmm, compression is irritatingly slow - I'll probably see if I can move just that part into a thread for now (but not any of the IO code or anything), so the game stays playable while it's busy compressing (e.g. when you change the compression settings file and it hotloads all the textures).)
Roger that. It'll be awhile before we can reliably do threaded IO, anyway.
It doesn't seem that surprising to me - these are load-on-demand texture objects, and Bind() is the demand, so it should load them.The thing is, which of these three methods would Bind call, if it wasn't done already?
assert failures wouldn't be trivial to debug (they might occur in rarely-used codepaths, and if a texture is shared by two different systems then it might depend randomly on the order in which those systems run), so it doesn't sound like an improvement.At least you get a warning of incorrect use. If someone accidentally forgets their call to Load or RequestPrefetch, then they get different behavior (-> with different and probably undesirable performance characteristics), and you wouldn't even know that anything is wrong (try debugging "why is the texture loader so laggy", or "why is it doing so much unnecessary work?").
-
hm, that's the first time I've heard of such a problem.
Do you also have lots of free space on the drive containing %temp%? The installer might also be writing there.
Finally, I'd check if your SMART values are OK - write failures might be a sign of trouble for the HD.
-
Uh, I thought the main point of doing this stuff asynchronously was to avoid variation in latency and keep a consistent framerate even when loading. Optimising just the worst case (dying hard disks etc), at the expense of the best/typical case, sounds counterproductive.
Vastly improving the worst case at the expense of a slight slowdown for the average case is a good engineering principle IMO - consider Quicksort vs. Mergesort.
To elaborate: I think it's acceptable to drop a frame here and then if that is the price of avoiding potentially unbounded freezing of the main loop while converting the input files (e.g. in the case of modders).
Maybe we could do both if it was multithreaded, though: LoadTexture can start a background load, wait up to 0.5 sec for a reply, then fall back to the placeholder texture if it takes longer than that, so it'll never freeze for long.Yep, sounds good. Win32 async IO does the same (but I would shorten the timeout to say 150 ms to ensure responsiveness).
Hmm... Another quick measurement: with cold cache, loading and uploading terrain/model (not GUI) textures for the initial view on Arcadia takes 2 secs; CObjectEntry::BuildVariation (which I think loads prop actors and meshes and idle animations) cumulatively takes 8 secs. (With hot cache it's about 0.1 secs / 0.2 secs).Good to know! That is surprisingly long. However, rather than invalidating the case for placeholders, I would say that's an additional incentive for optimizing BuildVariation and thinking about similar 'placeholders' there (e.g. reducing the amount of variations until later in the game).
Also I should probably give each texture a pointer to the texture manager, so I can just call m_Texture->Load().Yep, that's good, would also avoid shutdown issues when the texture manager gets wiped out before the last texture is destroyed.
And then make Bind() implicitly do the StartLoading thing, so callers don't have to remember to load it first.)Ugh, that sounds surprising (i.e. bad) and could cause unnoticed weird behavior (e.g. if callers actually wanted sync loads, or only wanted prefetching+placeholder). Why not just assert() that callers have called one of the three APIs before Bind()?
-
I'm glad you're now able to compile, but I have no idea why make clean didn't work.
About the linker error: we can't do anything other than google the error message either.
http://old.nabble.com/-brlcad-tracker----b...td27003602.html
There, they mention this is a compiler bug and you need to compile with -O2, not -O3. Maybe that helps?
(PS: will probably not be able to answer during the next 2 days, got crunch time and then a flight to a conference going on.)
-
I think then it would fetch just the assets from the selected factions?
Yep, that's correct
Just wanting to clarify what "all the terrain and all the units in the world" meant.(Networking ought to be handled in a separate thread anyway, so that the communication latency isn't hurt by the framerate. I think that shouldn't be too hard to add to the current network code, except for some issues with needing to deserialize JS objects.)That'd be nice.
Continue rendering as normal and just skip the SDL_GL_SwapBuffers() at the end, perhaps? (I have no idea if that would really work - sounds a bit dodgy to completely stop repainting the screen. Also it means the best case would be a 1 frame delay (perhaps 50ms), even if the texture would load in under 1ms (which it does with a hot disk cache).)That's perfectly legit. Each Render() does glClear and renders into the back buffer. We can do that arbitrarily often; only when the buffers are swapped does it become visible.
Also, latency is entirely acceptable. Consider that some TFT monitors add consistent 70 ms delay, and the network latency is probably 200..300 ms anyway.
Textures aren't the whole problem, though - actors and meshes and animations take time to load (much more? much less? I don't have any measurements now), and it'd be nice to handle them similarly, and there isn't any quick low-detail version we could swap in for them. So we need to handle them with some combination of prefetching and render-delaying, and fancy placeholders just for textures probably wouldn't add much benefit.Quick unscientific measurement: DDS files are 265 MB, PMD 11.5 MB and the animation folder is 40 MB. There would be a large benefit to placeholder textures after all, and combining them into one large file is a really good idea. I was thinking about just loading one of the higher mipmap levels, but a texture atlas would avoid the disk seek/IO.
Your API sounds reasonable
I'd be happier if it really were in a thread, but that's not going to be possible within the near future. As long as there is a queue and a spot in the main loop where updates are triggered, that should be easy to retrofit.One thing that strikes me is the LoadTexture name - that is only fitting for the IMMEDIATE case. Seems better to provide several routines (RequestPrefetch, StartLoading, Load) instead of overloading the single entry point. Those names make clear what is/could be going on under the hood.
-
I have a surprisingly strong feeling that the synchronous API would be a mistake - perhaps because I've always found this topic interesting and have been thinking about it off and on.
Is it feasible to load files and textures in a background thread with the current lib code? I thought that was all non-threadsafe and probably very hard to fix. We could do the texture compression in a thread but that wouldn't be used by normal players and doesn't seem worth the complexity.It's certainly feasible. The biggest culprit is probably the file cache, that'd blow up fairly quickly.
It'll need to be locked, but that's certainly doable and I don't foresee too many problems.
I think rendering the placeholder texture (currently solid black) for a few frames would be ugly and disturbing, and worse than just pausing for a fraction of a second. We don't know every single texture that a frame will use until we actually render that frame, so the renderer will have to ask for the texture then synchronously wait if it's not loaded yet.Let's take a step back here. The problem is that the renderer suddenly decides it needs a texture.
If we have a fairly studly prefetcher (a non-trivial task), it might reduce the load, but surely it won't always keep up (either due to high system load, e.g. antivirus, or lots of in-game activity, e.g. a transport unloading new stuff or lots of training or viewing a new base for the first time, etc.).
Therefore, we have to be prepared for the worst case, i.e. wait for everything to load, or provide for placeholders. What happens if the load takes a while (HD is busy, conversion to do, temporary read failure while the HD allocates a replacement sector, ...)? If the API is synchronous, the main loop (renderer) is frozen and we're unreactive to input and also network messages (and other players may think we lagged out).
Let's instead consider an asynchronous API that provides placeholders and swaps them out transparently, and provides a notification that the loader is still working on stuff that's needed now (as opposed to just prefetching). With this notification, you could achieve the same perceived effect as the synchronous API (graphics are paused for a bit), but you're still running the main loop and are responsive to network/input events (boss key
).When the prefetcher can't keep up, we could reduce the wait time by leaning a little bit on the placeholders. I agree that black stuff flickering in would be distracting, but it needn't be that bad. A nice example is Jak&Dexter, which makes the player stumble if the loader isn't keeping up (silly idea for us: cover the terrain with clouds
). Instead, we could go with a more neutral color, or a rough approximation of the actual hue. That of course suggests a level-of-detail scheme, where a really low-res texture might be loaded first and replaced by the full version when available.Let's forget about fancy placeholders not being discernable, that's probably too much work. The important point is that an async API allows the same behavior as the sync version, but avoids blocking and has better information (how many files remain, vs. always having to wait until the next texture has been read). It seems a shame to have this nice centralized loader, only to waste it via an API that just has the renderer call SyncLoadTexture().
Disagree - only one of several terrain biomes would be active at a time, non-water maps skip all the naval stuff, and we have a rather large number of units/buildings, of which 50% are probably unused (think back to our game a few weeks back - did we really see even a quarter of ALL units and buildings?).Players are very likely to see almost all the terrain and all the units in the world, by the end of a game, so we'll have to load it eventually and design it to fit in their memory. -
hm, I don't know of a good way to determine the previous version you had.
If a copy of the EXE is still lying around somewhere, it will display the version in the main menu screen.
Otherwise, the best that can be done is guess when you got the previous version and look in the TortoiseSVN log of the source\graphics and source\renderer directories to see what changed.
While there were a few graphics-related changes (mostly Alt+Tab support), I don't recall anything major that might explain this, but that would depend on exactly how old your version was.
-
Impossible
r8080 is the newer version of that file, yes. However, the offending function call moved down a few lines, it's no longer on line 51. You must be compiling an older version still - did you do make clean here as well?The ld warnings also indicate something is very wrong with your build system. Unfortunately I can't help there, never done any dev work on OS X :/
-
hm, interesting. Line 51 is the new location of the cpu_CAS64 call, so it looks like those casserts are not failing. Apparently what is happening is that intptr_t and i64 aren't actually the same type, so GCC refuses to cast.
I have now removed the cpu_CAS64 wrapper. Code that always wants to write 64 bits (regardless of pointer size) will now need to call ia32_asm_CAS64 #if ARCH_IA32, which isn't too bad since it's only 2 spots.
That should fix it - can you confirm?
(BTW, mention of wxJS is unrelated - it was probably compiling that in parallel.)
-
hm, that driver looks quite recent.
You mentioned you'd "upgraded" to the alpha - did it work before? What SVN revision was that?
Networking Error | Windows 7 x64
in Bug reports
Posted
hm, we're not all too familiar with the Windows cryptography API (used in that function) and might be doing something wrong. First, can you indicate whether this was before or after SVN revision 8869?