swbf-unmunge v1.1.0

Releasing the source files for your mod or map? Post em' here. (Applies to both SWBF1 & SWBF2)

Moderator: Moderators

Second Lance Corporal
Posts: 119
Joined: Thu Mar 03, 2011 5:08 pm
xbox live or psn: N/A

swbf-unmunge v1.1.0

Postby SleepKiller » Sat Oct 14, 2017 1:48 am

This is a tool that isn't actually that new and a fair few of you likely already know about. Anyway now that it has already been out in the open for a while and is now "finished" I feel compelled to post it here for the sake of completeness.

Anyway this tool as you might tell from the name is designed to undo the munge process. And it does quite a good job at it most of the time (for a mostly complete listing of what it can do see here) of extracting and "unmunging" an .lvl's contents.

Now let me just say I made this in my spare time because I found it fun and interesting. I did not make with the intent of using it (or enabling other people) to get access to another modder's assets without their permission. If you want to use it on a map another modder made to learn how they did something then that is probably fine, but if you want to use it so you can use something of their's without asking then that probably isn't okay. But I digress a tool is a tool, it isn't inherently evil or good. I know I would have loved to have this a few years ago when I lost one of my maps to the abyss and only had the .lvl of it. (Also it's because of making this I was able to know how to make this, so it's not all bad.)

It's a commandline application so if you struggle with those you can find a guide here.

You can find the source code on GitHub.
https://github.com/SleepKiller/swbf-unmunge
And hey would you look at that you can always find the most recent release on GitHub as well, isn't GitHub amazing.
https://github.com/SleepKiller/swbf-unmunge/releases

Be nice, do good not evil with it and all that.
Last edited by SleepKiller on Wed Feb 14, 2018 10:36 pm, edited 1 time in total.

Resistance Leader
User avatar
Posts: 4995
Joined: Tue Feb 09, 2010 8:43 pm
Location: Los Angeles, CA
Projects :: EVERYWHERE
Games I'm Playing :: The Division + DEHR
xbox live or psn: Marth8880

Re: swbf-unmunge v1.0.1

Postby Marth8880 » Sun Oct 15, 2017 5:27 pm

Excellent!

Due to how insanely useful it'd be (especially for rebuilding the 1.3 patch), I've submitted a requisition ticket for adding Lua script support here: https://github.com/SleepKiller/swbf-unmunge/issues/10

Second Lance Corporal
Posts: 119
Joined: Thu Mar 03, 2011 5:08 pm
xbox live or psn: N/A

Re: swbf-unmunge v1.0.1

Postby SleepKiller » Sun Oct 15, 2017 10:09 pm

I'll try to make it the next feature I implement. I can't promise perfect output but I feel I should know enough to get reasonable output. Any updates I have on it's support I'll post on the GitHub issue. (My usual disclaimer of this is obviously a hobby project of mine so I don't offer a concrete promise or time frame but I'll aim for having something within the next two weeks.)

Master of the Force
User avatar
Posts: 4695
Joined: Sat Sep 19, 2009 11:37 am
Location: Mos Espa (germany)
Projects :: Republic Commando Side Mod

Re: swbf-unmunge v1.0.1

Postby Anakin » Sun Nov 26, 2017 6:39 am

Marth8880 wrote:Excellent!

Due to how insanely useful it'd be (especially for rebuilding the 1.3 patch), I've submitted a requisition ticket for adding Lua script support here: https://github.com/SleepKiller/swbf-unmunge/issues/10



Lua support would be MUCH more useful then the 3d models :D


==EDIT==

I just tested your tool and the msh export does not work correctly. Seams that there is a problem with the UV

Hidden/Spoiler:
Image


==EDIT2==

i tested a little bit and it seams that the UVs are mirrored horizontal

Second Lance Corporal
Posts: 119
Joined: Thu Mar 03, 2011 5:08 pm
xbox live or psn: N/A

Re: swbf-unmunge v1.0.1

Postby SleepKiller » Sun Nov 26, 2017 8:27 am

Yeah that's been reoccurring problem for the tool. If I remember correctly the munger flips some model's texture coordinates and not others; I never sat down and worked out when and why it does this. The tool does try to flip the texture coordinate intelligently. It appears it is getting it wrong though. A hack solution is to just flip the texture to match the incorrect UV coordinates.

When/if I like work on this again I'll try to make it one of things I fix. The for loops in the read_vbuf_span overloads in this file control which vertex buffer types do and don't have their V texture coordinate flipped. If I don't end up fixing it myself and in the unlikely event someone in the future really wants it fixed that is where to start.

Droid Pilot Assassin
User avatar
Posts: 2080
Joined: Mon Jan 19, 2009 5:58 pm
Projects :: Star Wars Battlefront Legacy

Re: swbf-unmunge v1.0.1

Postby ggctuk » Sun Nov 26, 2017 2:07 pm

The same bug exists with some of the stock models and the solution is to simply resave the TGA file. That should fix the issue.

Master of the Force
User avatar
Posts: 4695
Joined: Sat Sep 19, 2009 11:37 am
Location: Mos Espa (germany)
Projects :: Republic Commando Side Mod

Re: swbf-unmunge v1.0.1

Postby Anakin » Fri Dec 15, 2017 12:28 pm

i noticed that mirroring the texture does not work. You need just to open and resave the tga file and it works. No idea how that could actually could be. For some reason the unmunge saves the tga in a "broken" format, that makes the texture mirrored in the viewer, but PSE, P.net,... can open those files and on resave they do everything correct, so the viewer works again.

Private
User avatar
Posts: 41
Joined: Sun Jan 18, 2015 5:23 pm
Projects :: SWBF2 XBOX Mod Environment
Games I'm Playing :: SW Battlefront 2
xbox live or psn: No gamertag set

Re: swbf-unmunge v1.0.1

Postby cbadal » Wed Jan 10, 2018 5:27 pm

Nice!
Happy that this exists :)

It would be helpful if you print your help message (from your .md ) when the other commands aren't recognized. Or when the user uses one of the more common help arguments [/? /h -h --help]

Code: Select all
swbf-unmunge <options>

Options:
 -file <filepath> Specify an input file to operate on.
 -files <files> Specify a list of input files to operate, delimited by ';'.
   Example: "-files foo.lvl;bar.lvl"
 -version <version> Set the game version of the input file. Can be 'swbf_ii' or 'swbf. Default is 'swbf_ii'.
 -outversion <version> Set the game version the output files will target. Can be 'swbf_ii' or 'swbf. Default is 'swbf_ii'.
 -imgfmt <format> Set the output image format for textures. Can be 'tga', 'png' or 'dds'. Default is 'tga'.
 -platform <platform> Set the platform the input file was munged for. Can be 'pc', 'ps2' or 'xbox'. Default is 'pc'.
 -verbose Enable verbose output.
 -mode <mode> Set the mode of operation for the tool. Can be 'extract', 'explode' or 'assemble'.
   'extract' (default) - Extract and "unmunge" the contents of the file.
   'explode' - Recursively explode the file's chunks into their hierarchies.
   'assemble' - Recursively assemble a previously exploded file. Input files will be treated as directories.

Droid Pilot Assassin
User avatar
Posts: 2080
Joined: Mon Jan 19, 2009 5:58 pm
Projects :: Star Wars Battlefront Legacy

Re: swbf-unmunge v1.0.1

Postby ggctuk » Wed Feb 14, 2018 11:48 am

This version can now unmunge console version files. Sleepkiller tested it on the XBOX DLC and it worked well there.

Master of the Force
User avatar
Posts: 4695
Joined: Sat Sep 19, 2009 11:37 am
Location: Mos Espa (germany)
Projects :: Republic Commando Side Mod

Re: swbf-unmunge v1.0.1

Postby Anakin » Wed Feb 14, 2018 11:58 am

nice, but i'm still waiting for the script unmunging. Any ideas what is necessary for doing that??

Private
User avatar
Posts: 41
Joined: Sun Jan 18, 2015 5:23 pm
Projects :: SWBF2 XBOX Mod Environment
Games I'm Playing :: SW Battlefront 2
xbox live or psn: No gamertag set

Re: swbf-unmunge v1.0.1

Postby cbadal » Fri Feb 16, 2018 10:26 pm

ggctuk wrote:This version can now unmunge console version files. Sleepkiller tested it on the XBOX DLC and it worked well there.

That's AWESOME!!!

Could it be used to unmunge a PC lvl and re-munge it for a Console version?

EDIT:
Anakin wrote:nice, but i'm still waiting for the script unmunging. Any ideas what is necessary for doing that??

For the compiled Lua (with -s option), a lot of the operations are straight forward to decompile. I have a tool that does a decent job on most of the mission lua files in 'mission.lvl', but it has been tricky to get good results with 'and' & 'or' logic so far.

Master of the Force
User avatar
Posts: 4695
Joined: Sat Sep 19, 2009 11:37 am
Location: Mos Espa (germany)
Projects :: Republic Commando Side Mod

Re: swbf-unmunge v1.1.0

Postby Anakin » Sat Feb 17, 2018 6:05 am

well i don't need mission stuff, i want to unmunge the shell lvl so i can see the changes Zerted did in his 1.3 patch

Droid Pilot Assassin
User avatar
Posts: 2080
Joined: Mon Jan 19, 2009 5:58 pm
Projects :: Star Wars Battlefront Legacy

Re: swbf-unmunge v1.0.1

Postby ggctuk » Sat Feb 17, 2018 6:06 am

cbadal wrote:
ggctuk wrote:This version can now unmunge console version files. Sleepkiller tested it on the XBOX DLC and it worked well there.

That's AWESOME!!!

Could it be used to unmunge a PC lvl and re-munge it for a Console version?


Most likely, yes.

Jedi Admin
User avatar
Posts: 10927
Joined: Wed Sep 07, 2005 11:42 pm
Location: Indiana
Projects :: No Mod project currently.
Games I'm Playing :: Destiny
xbox live or psn: No gamertag set

Re: swbf-unmunge v1.1.0

Postby Teancum » Sat Feb 17, 2018 9:41 am

Some of them anyway. I've had some work out such as Rattatak and Nal Hutta to the point where I can do the rest by hand and get it back into ZeroEdit. A few favorites (Concord Dawn being one) don't unmunge everything necessary to do the rest by hand. I've been meaning to contact SleepKiller with some examples but I've been focused on other things.

Private
User avatar
Posts: 41
Joined: Sun Jan 18, 2015 5:23 pm
Projects :: SWBF2 XBOX Mod Environment
Games I'm Playing :: SW Battlefront 2
xbox live or psn: No gamertag set

Re: swbf-unmunge v1.1.0

Postby cbadal » Sat Feb 17, 2018 4:05 pm

Anakin wrote:well i don't need mission stuff, i want to unmunge the shell lvl so i can see the changes Zerted did in his 1.3 patch


Looking at how much that patch seems to be used, Zerted really should release the source to the SWBF2 community.
Anyone ask him yet?

Jedi Admin
User avatar
Posts: 10927
Joined: Wed Sep 07, 2005 11:42 pm
Location: Indiana
Projects :: No Mod project currently.
Games I'm Playing :: Destiny
xbox live or psn: No gamertag set

Re: swbf-unmunge v1.1.0

Postby Teancum » Sat Feb 17, 2018 4:13 pm

cbadal wrote:
Anakin wrote:well i don't need mission stuff, i want to unmunge the shell lvl so i can see the changes Zerted did in his 1.3 patch


Looking at how much that patch seems to be used, Zerted really should release the source to the SWBF2 community.
Anyone ask him yet?

It's out there. I can't find a download link this second, but the source code to the luas were all released anyway. What's a bigger bummer is that the custom gametype luas he used to host on his old site are gone now.

Master of the Force
User avatar
Posts: 4695
Joined: Sat Sep 19, 2009 11:37 am
Location: Mos Espa (germany)
Projects :: Republic Commando Side Mod

Re: swbf-unmunge v1.1.0

Postby Anakin » Sat Feb 17, 2018 5:26 pm

Teancum wrote:
cbadal wrote:
Anakin wrote:well i don't need mission stuff, i want to unmunge the shell lvl so i can see the changes Zerted did in his 1.3 patch


Looking at how much that patch seems to be used, Zerted really should release the source to the SWBF2 community.
Anyone ask him yet?

It's out there. I can't find a download link this second, but the source code to the luas were all released anyway. What's a bigger bummer is that the custom gametype luas he used to host on his old site are gone now.


yeah the additional luas are released and i have them, but the problem is, that we don't know what he did to the original luas.
I unmunged the 1.3 and the stock shell and there are multiple files edited.

Private
User avatar
Posts: 41
Joined: Sun Jan 18, 2015 5:23 pm
Projects :: SWBF2 XBOX Mod Environment
Games I'm Playing :: SW Battlefront 2
xbox live or psn: No gamertag set

Re: swbf-unmunge v1.1.0

Postby cbadal » Sun Feb 18, 2018 7:47 pm

Anakin wrote:yeah the additional luas are released and i have them, but the problem is, that we don't know what he did to the original luas.
I unmunged the 1.3 and the stock shell and there are multiple files edited.

If you have the additional Luas that he release, then please upload them to a Github account and post the link to the 1.3 patch thread (if not already done).

I'm not sure if the decompiler will ever to decompile all Lua perfectly.

If you really, really want to see the differences you can compare the luac -l listings of the files (I'd suggest using the 'BeyondCompare' program).
You'll need to read this document to understand the listing syntax:
https://docs.google.com/viewer?a=v&pid= ... g0ZGFlMWI4

One of the files from his 1.3 patch I looked at was 'ifs_mpgs_pclogin'.
Very little of it had changed.
The only difference was at the beginning:
Hidden/Spoiler:
Code: Select all
   1   [-]   GETGLOBAL   0 0   ; ScriptCB_GetOnlineService
   2   [-]   CALL        0 1 2
   3   [-]   EQ          0 0 129   ; - "Galaxy"
   4   [-]   JMP         0 13   ; to 18
   5   [-]   GETGLOBAL   0 2   ; ScriptCB_DoFile
   6   [-]   LOADK       1 3   ; "ifs_mpgs_galaxylogin"
   7   [-]   CALL        0 2 1
   8   [-]   GETGLOBAL   0 2   ; ScriptCB_DoFile
   9   [-]   LOADK       1 4   ; "ifs_mp_leaderboard"
   10   [-]   CALL        0 2 1
   11   [-]   GETGLOBAL   0 2   ; ScriptCB_DoFile
   12   [-]   LOADK       1 5   ; "ifs_mp_leaderboarddetails"
   13   [-]   CALL        0 2 1
   14   [-]   GETGLOBAL   0 2   ; ScriptCB_DoFile
   15   [-]   LOADK       1 6   ; "ifs_mpgs_friends"
   16   [-]   CALL        0 2 1
   17   [-]   JMP         0 149   ; to 167

Which would resolve to this:
Hidden/Spoiler:
Code: Select all
if ScriptCB_GetOnlineService() == "Galaxy" then
   ScriptCB_DoFile("ifs_mpgs_galaxylogin")
   ScriptCB_DoFile("ifs_mp_leaderboard")
   ScriptCB_DoFile("ifs_mp_leaderboarddetails")
   ScriptCB_DoFile("ifs_mpgs_friends")
else   -- the corresponding 'end' is at the very end of the file.

So if you really, really want to. You would be able to construct the differences (but it would take some effort).

Master of the Force
User avatar
Posts: 4695
Joined: Sat Sep 19, 2009 11:37 am
Location: Mos Espa (germany)
Projects :: Republic Commando Side Mod

Re: swbf-unmunge v1.1.0

Postby Anakin » Mon Feb 19, 2018 4:22 am

If you are looking for the recovered source, here they are: viewtopic.php?f=25&t=27604&p=473223#p473223.

Gonna try those unmunge things when i find no other way the next days

Return to Released Assets

Who is online

Users browsing this forum: Bing [Bot] and 1 guest