Map unpacker

[admalledd]

So, recently I got a little bored and wanted to see how a bit more of the internal map structure worked in the .hwm file format. As a result I was quickly able to make a unpacker that can unpack everything but some of the levels/*.xml.bin files. (still working on reversing those for fun)

My question about this though is that is there a particular reason to not have a levelUnpacker program? If there is valid interest and no good reasons against having a unpacker I would be willing to release it, but I don't want to step on anyones toes.

As a side note: chances of a recompile of the editor for linux?

[Hipshot]

I don't know about your last linux question, but you can download the campaign levels somewhere in the resource section of this forum =)

[NekoBaron]

Its generally seen as frowned upon to decompile or depack other user content for most games depending on what you intend to do with it.

And as hipshot said you can get the games level resources already, and if you actually need help working something out your better off asking the person who made it first.

[admalledd]

Ok then, I will stay away from releasing my unpacker.

PS: love the level format for the most part, only question is why only store the file count in the pre-header and not filecount+sizes so that decompressing single files would be easier? (or filecount+offsets?)

[Myran]

The file format is just pretty much as simple as I could make it, so since the game always loads all the resources and never loads or seeks them individually, I didn't add any extra info to help with that. You can actually get the editor to run on Linux if you want, but it has a few dependencies. You will need X and mono-complete, then you can just run the .exe file through mono. It's made in WinForms though so it doesn't look great, but everything should work fine.

As far as the unpacker goes..  well, we do have an unpacker, but I'm just not sure if releasing it is a good idea..  like you can see the .hwm format is trivially simple and contains no protection, but at the same time people who make maps might want to keep their content private, so I just don't know.

If you're curious or want to finish your unpacker the .xml.bin files are pretty much the same format as the .xml except they are packed as binaries for slightly faster loading and smaller filesizes. Basically we have an internal structure where we map simple data types in a hierarchy and then we can serialize that hierarchy in a few different ways (xml and binary), this is what we use for save files, level files, some of the more complex network commands, stuff like that. So when packing the .hwm files we open all the levels and reserialize them as binaries, here is the source code for the class: http://hammerwatch.com/SValue.txt
So if you're using C# you can just use that class and do a LoadStream() then SaveXML() to get the .xml files back (that also works for the .hws save files, in case you wanna edit those manually), if you're not using C# the format is still simple enough for you to just reimplement that loading.

[admalledd]

Thanks! Because of how close I was, just for the fun of it I finished the unpacker (and won't release it as was asked though). Considering you posted the raw code I thought it would be fun to at least finish my work and have those also converted. For anyone who wants to play around with the decompresser I wrote in python for the binary stream I have put it here on gist.github.com. Fair warning that its not recurse-safe and probably never will be. (your data structure uses nesting and I don't bother to prevent recursion issues. never had one be to large yet.)

Not going to bother with a compressor unless asked (and really anyone who knows enough code to use svalue.py should be able to add it easily...)

Its been far too long since I reverse engineered something! I was actually a little ways into figuring it out when you had posted. Mostly that you had a one byte "type" then the data, then next byte "type", and had got it reading the first thing it had found of a dict of <key,val>. I had abused your levelPacker.exe and the mono debugger to narrow down to only compressing specific things. As well as your TiltedEngine.dll was exposing all of its internal function names so I knew mostly what might be inside. Thanks for indulging in my fun, my brother bought it for me as a gift with the tag "we must storm the castle!" two days of play later I find myself poking around the level files... Hoping to see people with talent at map making create some awesome ones!

If for funsies you want a full copy of my (poorly written in a day or so) python unpacker I can PM you the info.

Also heads up for the level editor on why it wasn't working for me: apparently a conflict with a really old mono build I made long ago was mucking things up. mono-complete and libsdl2-2.0.0 was all I needed after fixing that.

[Zerel96]

¿Can you give me the unpacker in private? i can use it in my diablo campaign to take the levels of the dungeon generator and put it in my campaign

[shiftyBOT]

I would love the editor and all the others programs in the editor folder to have native linux versions if possible:P
it would make my life so much easier:P
EDIT:
what's this X thingy you speak of?

[Myran]

Technically they're not even native on Windows so native Linux version would be hard but installing mono-complete should be pretty easy with the Linux package management systems. X is a windowing system, unless you know you're using something else (Wayland?) I would guess that you have X.

[admalledd]

Just confirming, yes installing mono-complete and libsdl2 then just running "mono .local/share/Steam/SteamApps/common/Hammerwatch/editor/HammerEditor.exe" (or whatever the path is for your copy) to launch the map editor. Works fine here.

[shiftyBOT]

tried to start the HammerEditor with mono,it spewed out this in the terminal

Unhandled Exception:
System.DllNotFoundException: libSDL2-2.0.so.0
  at (wrapper managed-to-native) SDL2.SDL:SDL_Init (uint)
  at ARPGEditor.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.DllNotFoundException: libSDL2-2.0.so.0
  at (wrapper managed-to-native) SDL2.SDL:SDL_Init (uint)
  at ARPGEditor.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0

and also I cant unpack the assets

[admalledd]

You need to install libsdl2. Depending on your distro the commands are different,for me (currently ubuntu 14.04 custom) it was "sudo apt-get install libsdl2-2.0-0". (if you are not a debian-stable (or higher) based distribution, let me know and I can look up the command and version info for you)

[shiftyBOT]

thanks dude:)
now I'm working to make a custom map:D
I cant wait to add content to the community
(if some one wants to help test I would be happy to send you a version of it later)