Home » MODDING HQ 1.13 » v1.13 Coding Talk » Improving loading times
Improving loading times[message #335045] Sun, 17 August 2014 22:23 Go to next message
Flugente

 
Messages:3339
Registered:April 2009
Location: Germany
Loading times are getting worse and worse it seems. I wanted to know how bad it currently is. I find the results interesting, thus I'll share it with you here.

For this test, I updated the increased teamsize exe to the current trunk. Then, in debug, I added various timers in the loading function, which then prints out how long loading takes. I used three savegames here, and got these logs:
  • a savegame of a new game started with one merc, saved upon cleaning Omerta
    [491.981] : Setting up savegame:   0.6322
    [491.983] : Inventory maintenance:   0.0023
    [492.809] : A9 ENTER SECTOR TIME:  1.19 seconds.
    [492.809] : See JA2\Data\TimeResults.txt for more detailed timings.
    [493.752] : Before Soldier Structure:   1.7695
    [493.884] : After Soldier Structure:   0.1312
    [493.896] : Before E-Mail:   0.0122
    [493.909] : Before Strategic Movement Groups:   0.0129
    [493.913] : Before Map Temp files:   0.0042
    [494.433] : SynchronizeItemTempFile()  Error!  Reported 0, should be 3
    [494.568] : After Map Temp files:   0.6547
    [494.96] : Arms Dealers:   0.3922
    [495.24] : Before Misc Info:   0.2800
    [495.58] : Before Town Loyalty:   0.3402
    [495.581] : Before Vehicle Information:   0.0012
    [495.587] : Before Physisc table:   0.0054
    [495.59] : Before Air Raid:   0.0033
    [495.593] : After Air Raid:   0.0029
    [495.597] : Before Creature Spreading:   0.0039
    [495.6] : Before Strategic Status:   0.0029
    [495.603] : Before Strategic AI:   0.0034
    [495.641] : Before New Vehicles:   0.0379
    [495.647] : Before Briefing Room:   0.0063
    [495.65] : After Dynamic Dialogue:   0.0024
    [495.65] : Before Applying temp files:   0.0004
    [496.392] : After Applying temp files:   0.7424
    [496.42] : Before Selecting Soldier:   0.0279
    [496.428] : At Done:   0.0072
    [496.429] : The end of it:   0.0019
    [496.43] : Total loading time:   5.0810
    
  • a savegame of an an ongoing campaign with increased teamsize on INSANE. Currently has 47 mercs, loads of militia and troops everywhere, 4 cities liberated, thousands upon thousands of items everywhere - basically a 'heavy load savegame'.
    [30.396] : Setting up savegame:   0.0082
    [30.3987] : Inventory maintenance:   0.0024
    [31.4899] : C5 ENTER SECTOR TIME:  1.61 seconds.
    [31.4906] : See JA2\Data\TimeResults.txt for more detailed timings.
    [31.8164] : Before Soldier Structure:   1.4180
    [34.6214] : After Soldier Structure:   2.8050
    [34.6335] : Before E-Mail:   0.0121
    [34.6481] : Before Strategic Movement Groups:   0.0146
    [34.6549] : Before Map Temp files:   0.0069
    [124.989] : SynchronizeItemTempFile()  Error!  Reported 0, should be 429
    [388.373] : After Map Temp files: 353.7181
    [388.743] : Arms Dealers:   0.3698
    [390.952] : Before Misc Info:   2.2086
    [391.321] : Before Town Loyalty:   0.3697
    [391.322] : Before Vehicle Information:   0.0012
    [391.328] : Before Physisc table:   0.0051
    [391.331] : Before Air Raid:   0.0031
    [391.333] : After Air Raid:   0.0025
    [391.337] : Before Creature Spreading:   0.0035
    [391.339] : Before Strategic Status:   0.0026
    [391.342] : Before Strategic AI:   0.0026
    [391.437] : Before New Vehicles:   0.0951
    [391.443] : Before Briefing Room:   0.0063
    [391.457] : After Dynamic Dialogue:   0.0137
    [391.457] : Before Applying temp files:   0.0006
    [398.032] : After Applying temp files:   6.5748
    [398.207] : Before Selecting Soldier:   0.1752
    [398.215] : At Done:   0.0071
    [398.218] : The end of it:   0.0038
    [398.219] : Total loading time: 367.8305
    
  • A savegame similar to #2, but saved in an older savegame format (the savegame version is often increased when we add features that add new variables to data structures we save).
    [206.474] : Setting up savegame:   0.0079
    [206.477] : Inventory maintenance:   0.0024
    [207.67] : C5 ENTER SECTOR TIME:  1.76 seconds.
    [207.671] : See JA2\Data\TimeResults.txt for more detailed timings.
    [208.022] : Before Soldier Structure:   1.5447
    [210.632] : After Soldier Structure:   2.6105
    [210.645] : Before E-Mail:   0.0125
    [210.661] : Before Strategic Movement Groups:   0.0161
    [210.666] : Before Map Temp files:   0.0057
    [361.023] : SynchronizeItemTempFile()  Error!  Reported 0, should be 429
    [842.298] : After Map Temp files: 631.6316
    [842.705] : Arms Dealers:   0.4065
    [844.165] : Before Misc Info:   1.4605
    [844.482] : Before Town Loyalty:   0.3169
    [844.483] : Before Vehicle Information:   0.0018
    [844.49] : Before Physisc table:   0.0063
    [844.493] : Before Air Raid:   0.0037
    [844.498] : After Air Raid:   0.0038
    [844.502] : Before Creature Spreading:   0.0046
    [844.506] : Before Strategic Status:   0.0036
    [844.509] : Before Strategic AI:   0.0033
    [844.614] : Before New Vehicles:   0.1048
    [844.622] : Before Briefing Room:   0.0078
    [844.636] : After Dynamic Dialogue:   0.0141
    [844.637] : Before Applying temp files:   0.0010
    [867.35] : After Applying temp files:  22.7131
    [867.54] : Before Selecting Soldier:   0.1901
    [867.547] : At Done:   0.0071
    [867.55] : The end of it:   0.0033
    [867.551] : Total loading time: 661.0838
    
Keep in mind that I used a debug exe for this. Loading with a debugger takes a lot more time than with a normal release exe (which you get from SCIs for example). The loading times a normal player experiences are much, much lower, so there is no need to go to rage mode for you here Smile

Now, what can we learn from this?

First of all, it seems that the increased teamsize has only a very marginal effect on loading times - the entire loading of the soldiers takes about ~3 seconds with many mercs, which is negligible compared to the rest.

It also seems other new features don't significantly increase loading times either, which is good.

However: The bulk of loading is spent on Map Temp files. A very, very huge amount in fact. The reason is that this is the part where corpses, structure changes etc. are loaded - and items. In fact, in #2, > 300 seconds are just for items! Loading of items is what makes loading so long! For this reason, loading times increase during the campaign, as more an more items are created and have to be stored. If anyone wants to improve loading times, this is where you should start.

Loading a savegame of and older version takes even longer. The reason is that while loading, we actually save all items again in the new savegame version, so we don't have to do that again in that campaign. This is pretty bad (I already committed a few improvements here - without those, you can add another 200 seconds for that part). Luckily, the normal player will only rarely, if ever, load such a savegame, so this is more of an annoyance, as it should be a one-time event.



So that is that. Any useful code additions to item loading will thus be of great use.


Re: Improving loading times[message #335059] Mon, 18 August 2014 21:24 Go to previous messageGo to next message
RoWa21

 
Messages:2039
Registered:October 2005
Location: Austria
good catch flugente.

maybe we can improve the loading speed by checking all those loops and break out as early as possible. I assume some loops are looped till the end even if that is not needed.
I haven't take a look at the specific code-parts, it is just a suggestion...


Re: Improving loading times[message #335062] Mon, 18 August 2014 22:48 Go to previous message
Flugente

 
Messages:3339
Registered:April 2009
Location: Germany
The function we spend all that time on is
BOOLEAN	LoadMapTempFilesFromSavedGameFile( HWFILE hFile )
in Tactical Save.cpp, and in there, it's happening mostly in
void SynchronizeItemTempFileVisbleItemsToSectorInfoVisbleItems( INT16 sMapX, INT16 sMapY, INT8 bMapZ, BOOLEAN fLoadingGame )
Let's see...


Previous Topic: New Tactical AI - Functional Considerations
Next Topic: JA2 Roguelike mod
Goto Forum:
  


Current Time: Mon Jun 25 16:43:01 EEST 2018

Total time taken to generate the page: 0.01400 seconds