Hammerwatch Forum

Technical Support => Technical Support => Topic started by: Heretic on November 15, 2013, 03:12:51 PM

Title: Editor Out of Memory
Post by: Heretic on November 15, 2013, 03:12:51 PM
Soooo I've possibly pushed this editor a bit too hard, and have a fairly large level with tons of new assets - It's to the point now where when I work for 2-5 minutes it crashes due to memory. For future levels I'll try to keep them smaller but this level has a small ways to go to be finished and I'm wondering if there is a way to increase the amount of memory the editor can use or reserve? Or are there any tips to get the editor to crash less?
Title: Re: Editor Out of Memory
Post by: NekoBaron on November 15, 2013, 08:06:37 PM
You could try merging some art assets to one file if your using a separate file for each one.
Title: Re: Editor Out of Memory
Post by: Heretic on November 15, 2013, 08:21:58 PM
With the amount of assets I have in this level that would probably require redoing the entire level :/

I couldnt find a config file anywhere for the editor that might designate more memory or something - as it seems to be a limit of the program itself - my system has plenty of memory to spare.
Title: Re: Editor Out of Memory
Post by: NekoBaron on November 16, 2013, 12:38:55 AM
I put a lot of my art assets into a single file, you can have multiple xmls reference the same image file.

How many xmls and images we talking about? I'm curious what the limits are or if they're just something your hitting,
Title: Re: Editor Out of Memory
Post by: Heretic on November 16, 2013, 03:19:50 AM
Well most of them are in the same image already, it's not referencing different images for each asset - they are mostly part of a large palette. As far as doodads, about half my list of doodads is vanilla hammerwatch and the other half is stuff I've imported - actually a bit less than half. The difference would be in the scaling of the assets - some 16, some 32, some 64, etc. Also the amount of doodads I have placed and such seems to be quite a lot - they are most times smaller and require more of them to fill the same space. Lots of doodads overlapping, etc.

Crashing occurs most when changing doodad layers, copying doodads, and (almost everytime) reloading resources in the editor.

I checked the .xml file size and it is just under 6MB - I thought maybe the file size was some indicator but after looking at the file sizes of the vanilla levels some are bigger.
Title: Re: Editor Out of Memory
Post by: Heretic on November 16, 2013, 06:35:01 PM
After some tinkering and removing over half of the assets placed in this large level - it still seems to crash often...

Could one of the devs comment on what takes up resources the most, or some best practices for conserving memory?

Any plans on increasing the available memory for the editor in the future?
Title: Re: Editor Out of Memory
Post by: NekoBaron on November 17, 2013, 01:37:59 AM
I don't think its the level having too many assets, but that you have a lot of custom assets which are all loaded into the editor even if they're not used.

It might be all the 64 scale images taking up too much for the editor, from what I remember the default general scale is 16 so 32 is 4x as much needed and 64 is 16x as much being used by images.
Title: Re: Editor Out of Memory
Post by: Hipshot on November 17, 2013, 12:04:46 PM
May I ask, does the level have more assets and are larger than level 9 or 11 in the campaign?
Title: Re: Editor Out of Memory
Post by: Heretic on November 17, 2013, 03:30:09 PM
May I ask, does the level have more assets and are larger than level 9 or 11 in the campaign?

It's just under the size of level 9, but I cut it basically in half to make it two different levels but the crashing still happens. Before cutting in half the assets placed might be a little more than the assets placed in 9 and 11.
Title: Re: Editor Out of Memory
Post by: Hipshot on November 17, 2013, 09:47:38 PM
If you work a lot with tilemaps, a lot, like I did with level 6, all those small borders, I had crashes all the time because the editor ran out of memory. Myran fixed this by flushing the history data when the memory peaks at 1GB (I think), I can ask him if we can make an option to remove the history states completely, that would probably increase the performance, espec on slower machines.
Title: Re: Editor Out of Memory
Post by: Heretic on November 18, 2013, 01:01:31 AM
I don't actually work with tilemaps that much - at this point its really just grass and a dirt path. In fact, i laid down the tilemap first and mostly didn't touch it very much since. The crashing occurs when i copy a doodad, change the doodad layer (via ctrl+#) and I do notice the history list on the left 'refresh' itself by completely deleting itself - which happens literally every minute.

I should also note that my system isn't very low end, it's kinda a beast. I have a quad 3.6ghz and 8 gigs of ddr3 RAM.

In some cases I have a very large image palette that I only use a small item or two from - but that item is used many times. Would part of the memory issue have something to do with that? I suppose it might be best practice to make those images as small as possible anyways - its just more work for me and I'm lazy :D
Title: Re: Editor Out of Memory
Post by: Hipshot on November 18, 2013, 08:33:55 AM
Would it be possible for you to share the map to me, either here or via PM, skype or whatever? I can just see how it handles on my computers, this one here at home or the one at work?

The amount of memory doesn't seem to matter when you go above a certain point, I'm sure the editor only uses 1GB anyway. For some reason... I've asked myran about this before, but I remember it being something complicated.
Title: Re: Editor Out of Memory
Post by: Heretic on November 18, 2013, 03:09:11 PM
Yes I can! I'd have to package all of the new assets into a zip too, right? Let me see if I can get it all together correctly and I'll PM it to you.
Title: Re: Editor Out of Memory
Post by: Hipshot on November 20, 2013, 11:42:22 AM
I tried the level in question, I don't get the lag or any sort of performance issues, not at home and not here at work.

However, there's a lot of different technical errors here, both with tilemaps that shares the same layer and most important, you have textures that dosn't use the power of 2. You should really try and fix the latter issue and see if it fixes your performance.

Also, what kind of graphics card do you have?
Title: Re: Editor Out of Memory
Post by: Heretic on November 20, 2013, 02:17:08 PM
It's just crashing of the editor - but after I removed the doubles of the vanilla assets in the hammerwatch/assets folder it kinda stopped crashing overall. I think maybe because there were so many duplicates it was having some kind of problem.

I have radeon 6850 GT

I plan on going through and changing everything to the 'powers of 2' rule, but before I had noticed or even cared about that I already had most assets imported as they are. Hopefully it will work alright long enough for me to work on the levels and such, and I'll add the powers of 2 rule to my list for optimization near the end.

If you notice, the tilemap conflict is saying "shares same level as tilemaps/a_dirt.xml and tilemaps/a_dirt.xml - both of which are YOUR tilemap and I'm not sure how I'm supposed to fix this :/ Is it because I ahve the assets dumped into my editor folder?
Title: Re: Editor Out of Memory
Post by: Heretic on November 20, 2013, 03:16:11 PM
Also, I've tried after a certain point to keep my textures in the power of 2 rule, and according to that error list my hand_spikes trap 'needs to be in the power of 2'. After going through some of these .xmls I'm noticing that i AM keeping to that rule. Here is an example of one of the ones that it is saying is NOT in the powers of 2:

Code: [Select]
<doodad defaultlayer="0">

<sprite scale="16" name="closed">
<texture>doodads/trap_hand_spikes_yellow.png</texture>
<origin>16 16</origin>
<frame>0 0 32 32</frame>
</sprite>

<sprite scale="16" name="open">
<texture>doodads/trap_hand_spikes_yellow.png</texture>
<origin>16 16</origin>
<frame time="100">0 32 32 32</frame>
<frame time="100">32 32 32 32</frame>
<frame time="100">64 32 32 32</frame>
<frame time="100">96 32 32 32</frame>
<frame time="100">128 32 32 32</frame>
<frame time="100">0 64 32 32</frame>
<frame time="100">32 64 32 32</frame>
<frame time="100">64 64 32 32</frame>
</sprite>

<sprite scale="16" name="activate">
<texture>doodads/trap_hand_spikes_yellow.png</texture>
<origin>16 16</origin>
<frame time="100">0 0 32 32</frame>
<frame time="100">32 0 32 32</frame>
<frame time="100">64 0 32 32</frame>
<frame time="100">96 0 32 32</frame>
<frame time="100">128 0 32 32</frame>
</sprite>

<sprite scale="16" name="deactivate">
<texture>doodads/trap_hand_spikes_yellow.png</texture>
<origin>16 16</origin>
<frame time="100">128 0 32 32</frame>
<frame time="100">96 0 32 32</frame>
<frame time="100">64 0 32 32</frame>
<frame time="100">32 0 32 32</frame>
<frame time="100">0 0 32 32</frame>
</sprite>

<states default="open">
<transition from="deactivate" to="closed" />
<transition from="activate" to="open" />
</states>

</doodad>

I can't see where I'm going wrong with it?
Title: Re: Editor Out of Memory
Post by: NekoBaron on November 20, 2013, 08:18:52 PM
If you notice, the tilemap conflict is saying "shares same level as tilemaps/a_dirt.xml and tilemaps/a_dirt.xml - both of which are YOUR tilemap and I'm not sure how I'm supposed to fix this :/ Is it because I ahve the assets dumped into my editor folder?
Uh yeah I can imagine that would annoy the editor greatly having every asset double loaded.

I have the unpacked assets in my editor (IE editor/assets) folder which wont load them up in the editor or game.
Title: Re: Editor Out of Memory
Post by: Heretic on November 21, 2013, 02:39:00 PM
I had mine in the editor/mod/assets folder so that when I wanted to directly edit the .xml through the editor it would work. I use modified versions of some original .xmls so it was much easier to have my own set of assets all in one place.

Also - the real issue at this point is the Rule of 2 which my .xmls seem to be following but the editor is flagging as an error. Does anyone know whats wrong with that xml I just pasted?

Additional question for the devs: Most of the assets listed in that error list that claim it needs to follow the rule of 2 are not even placed in the level that crashes constantly - does that matter?
Title: Re: Editor Out of Memory
Post by: Hipshot on November 21, 2013, 03:21:42 PM
The rule of two doesn't have anything to do with the xmls, it's the pngs themselves =/
You need to resize yours to become that, the best way to do that and not cause any strange errors would be to extend them to the right and down so they becomes the right size.

16x16
32x32
64x64
128x128
256x256
512x512
1024x1024

Or 512x32 perhaps...



About all assets, there's no need to ship the level with all our assets if they are not changed, it will most likely cause issues, you extract them only to be able to see how they work and copy what we've done =)
Title: Re: Editor Out of Memory
Post by: Heretic on November 21, 2013, 08:41:45 PM
The rule of two doesn't have anything to do with the xmls, it's the pngs themselves =/
You need to resize yours to become that, the best way to do that and not cause any strange errors would be to extend them to the right and down so they becomes the right size.


Okay awesome I'll start changing the images then!
Title: Re: Editor Out of Memory
Post by: Hipshot on November 21, 2013, 09:58:52 PM
Yea, try that and don't mix your assets with the default ones extracted, that's most likely a bad idea, you don't even need to have them extracted to be honest.
Title: Re: Editor Out of Memory
Post by: Heretic on November 22, 2013, 01:27:23 AM
The editor is much more stable after changing all the resource files to powers of 2. I also removed the default assets from the editor folder and any assets that I wish to reuse I will rename so that they don't conflict. The editor will still crash, but nowhere near as often and this is much more managable! :D