Having encountered some odd behaviour while trying to record Hammerwatch myself, I can confirm that it seems to play much better with FRAPS. Thankfully I still had it knocking about from the early days of my LPing, though I was a little dissapointed to have to return to using it, if only because I've become more comfortable with other software since.
Still, the game is certainly worth the effort, and FRAPS seemed to record it quite well
I am curious why it seems other software, such as XSPLIT and OBS have an issue with Hammerwatch though. From what I could observe, when trying to record Hammerwatch using software other than FRAPS it caused the first core on my CPU to max out, while the other 5 cores hung around the 30 / 40 percent mark, however when using FRAPS (which is a significantly more resource hungry pieces of software) all cores seemed to balance around the 60 / 70 percent range.
I didn't do any thorough troubleshooting though, so it could very well be a coincidence at this point, however if anyone else has had similar experiences while trying to record I'd be interested in hearing about them. It may lead to finding a solution