Building New Level Geometry and Lightmaps Shadowmaps in UDK: First Steps [EXPERIMENTAL]
Disclaimer at start: I'm new to Mass Effect modding as well as Unreal Engine 3/UDK so please forgive me if you notice any glaring beginner mistakes in this tutorial, regarding either ME modding tricks or using UDK itself, I'm learning all of this myself as I go. The concept is fresh and experimental (as in "let's see if this silly idea works"), I'm sure there's still a ton of things to discover along with some new/alternate/simpler methods to utilize and expand on the level creation process. I will be updating this tutorial and/or writing new chapters regularly in the coming months as I learn and uncover new possibilites, especially on how to improve the workflow. Feel absolutely free to add to this tutotial if you know how some aspects can be improved or simplified or if you see any mistakes I made. The ultimate goal is being able to create the majority of a level's world design in UDK, as easily as possible and this is just a first but key step in the process.
TOOLS NEEDED:
- the latest version of ME3Tweaks' Legendary Explorer
- Unreal Developement Kit 2015
- (optional) Glidor's uModel, Glidor's ActorX Import/FBX Batch Export plugin, 3ds Mas 2017+
- patience, determination, masochism
INTRODUCTION:
In this tutotial I will show & explain my recently discovered "backdoor" method of building new levels for ME3/LE3 using Unreal Developement Kit (2015) and setting up Static Lighting.
Mass Effect series was built on a customized version of Unreal Engine 3. While obviously we do not have access to Bioware's internal edition, the publicly available Unreal Developement Kit is close enough or idential in many aspects to be used in level creation process as well as many other modding ventures.
The levels in the series are almost exclusively built using Static Meshes (which don't move compared to Dynamic Skeletal Meshes such as NPCs or moving objects), even including "naturalistic" terrain areas like Palaven or Rannoch - the rocks and hills are blocks of Static Meshes. The structure is modular, with multiple meshes arranged and used multiple times to create the world geomtery.
As part of the PS3/360 era, Mass Effect games utilize next to none real-time lighting for Static Meshes/un-moveable level geometry. Instead, the games use Lightmaps/Shadowmaps which in essence are "photos" of the real lighting set up and prepared in UDK Editor, "baked" in a form of textures which are then applied over the Static Meshes as a sort of a mask. As you can guess, the Lightmap textures in this case are responsible for the "lightness/brightness mask" (and color in case of colorful lights) of the objects and the Shadowmap textures add the shadows over the objects, as they were seen in UDK Editor.
The "baking" is achived thanks to each and every StaticMesh used by the game having a carefully pre-prepared unique UV channel reserved exslucively for Lightmaps/Shadowmaps baking purpouses. In Mass Effect's case it's always channel 1 (with channel 0 reserved for the object's base textures like Diffuse/Normal/Emissive with a comletly different set of UVs). Once the level gemotry is created (as in - once you build your world from StaticMesh blocks) the unique UV sets are then used by baking process to create an atlas of light/dark textures for every static object within the level and every side of it. Once in-game, these "baked" Lightmap/Shadowmap textures get overlayed "on top" of the StaticMesh geometry and you see the final pretty result - a level that looks realistic with realistic lighting yet utilies no actual performance-heavy real-time lighting. We will build our new levels using some of the same methods Bioware used in essence, just with extra steps to account for differences between UDK and ME3/LE3 engine version. The three main steps will involve:
- Exporting Static Meshes from the base game, importing into UDK and preparing for use
- Building our new level with pre-prepared Static Meshes, Setting up basic Lighting, Baking Lightmaps/Shadowmaps
- Transfering our level geomtery and lightmap/shadowmap textures into PCC format
- Playing our first shockingly-beautiful level in game!
Let's begin!
1a) Exporting Static Meshes from the base game
Open the Legendary Explorer -> Meshplorer. Select which PCC has the StaticMeshes you want to extract and use in your level. Most meshes are located inside BioA_MapName_SubFolder packages, although sometimes the main "parent" BioA_MapName package has them as well. Right click and select "Export Mesh to UDK. See Figure 1 below:
You should save your mesh in a newly-created unique sub-folder in UDK's Content Folder (usually it's something like C:/Program Files/UDK/UDK-2015-01/UDKGame/Content), so in this example we will create Guidetest folder and save our stuff there.
Save the Mesh in your newly created Guidetest Folder under the same name as its parent sub-package (in my case the parent package of Helipad_Floor is BioS_Arctic, see Figure 1 again. This is very important and will prove to be a huge help at the end of our level building process. If the parent sub-package contains multiple meshes causing the name you would be saving the exported package as to be already taken, just add 02 at the end (so i.e. BioD_Arctic02), or anything else as long as you can later recognize that 2 or more meshes came from the same package.
1b) Importing Static Meshes into UDK
Inside UDK, open 'Content Browser (View-> Browser Windows -> Content Browser). Check the folders list in the bottom-left corner, you should see previously created Guidetest Folder under UDK's Content and your saved Meshes, under the names of their parent packages. Right Click on your Guidetest folder and select Fully Load.
Few things to fix before the meshes are ready for use - first, if you've exported multiple meshes from the same parent package (BioS_Arctic for example had multiple meshes as you remember), you should have multiple UPK packages with the same name and a 02/03 /04appendix at the end. We need to combine these into a single package. Select BioS_Arctic02 package and Right-Click on it -> select Move or Rename. Rename the Package as the base name where you will move the mesh to. See Figure 2 below:
As Meshplorer currently doesn't export Lightmap coordinate Index nor Lightmap resolution, we will have to set that up ourselves. Pick your first UPK package with meshes and Right-Click on the first Mesh -> select Edit Using Static Mesh Editor. On the right side, set the Lightmap Coordinate Index to 1 (that's the UV channel that contains Lightmap UVs, channel 0 contains UVs for Diffuse/Normal/Emmisve textures) and then set the resolution. See Figure 3 below:
Most objects in the game use resolution of 32 but this it NOT the universal rule, some objects have higher resolution. You will have to consult with Meshplorer and check each object you exported earlier. If there's no resolution listed for an object in Meshplorer, set it as 32 OR leave the Lightmap resolution in UDK as 0 (in many cases it means the mesh uses Vertex lighting instead of traditional Lightmaps as with glass objects for example, that will cast shadow on the surface beneath/next to it but should not have visible shadow lines on themselves from other objects). Repeat the process for all meshes. After that, your meshes are ready for use in creating level geomtery.
The meshes are texture-less as Meshplorer currently doesn't export Material Info along with the Mesh, which prevents applying new materials in the editor. Textureless meshes are fine for bulding the structure of the world and creating Lightmaps & Shadowmaps though, many professional lighting artists often work on building level lighting without any textures, either by choice or because the texture art departement has not delivered the assests yet.
Additionally, some meshes exported to UDK format in Meshplorer don't export correctly. If you're really not comfortable with working with textureless meshes or have a particular mesh that doesn't export correctly to UDK format in Meshplorer, I describe an alternate (but much slower and troublesome) method below, using Glidor's umodel exportet, ActorX plugin and 3ds Max.
alternate 1a) Exporting Static Meshes from the base game using uModel
This alternate method will ensure an export of every mesh from the game and allow applying textures in UDK. It is however much slower and requires additional tools. You can mix&match the 2 methods as you see fit, until better/easier solutions are developed and the guide is updated.
You will need to install 3ds Max 2017+ (free student license, paid sub, or... you know), Glidor's uModel and Glidor's ActorX Import/FBX Batch Export plugin for 3ds Max. These tools will be used to export the meshes into individual upk files and convert them into FBX format readable to UDK Editor.
Open Glidor's uModel and click the button next to 'Path to game files' to select your CookedPCConsole folder inside ME3/LE3 installation folder. Select which PCC you want to export Meshes from and hit the Export button, see Figure 4 below:
Select Export settings as in Figure 5:
It doesn't matter where you chose to save the export for now, a new folder will be created with the main PCC package name you picked to extract meshes from.
After exporting, open 3ds Max 2017+. Previously-installed ActorX Import and FBX Batch Export plugins should already be visible. If not, click Scripting -> Run Script. Do not import/export anything for now yet though! We need to set the settings for proper FBX export first! In 3ds, click on the main Menu button -> Export -> Game Exporter. A new window will appear. Check the settings as in Figure 6 below:
Then click Advanced Settings (cog icon in bottom-right corner) and check the following settings as in Figure 7:
And hit OK. Now SAVE your settings Preset by clicking the Save Icon in the main Game Exporter window (near top-right corner). This preset will be used by Glidor's FBX Exporter and you will have to load it each time you launch 3ds Max and aim to export a new batch of meshes.
After saving your preset and having it active and loaded, it's time to use the FBX Batch Export tool. Check if the settings are as in Figure 8 below and click on the button next to "Path to PSK" (that you exported with uModel earlier), then select the package you want to convert to FBX. Select the Folder with the mesh(es) inside, not the mesh itself.
After selecting the package click the Export Meshes button. This will batch-export all the meshes inside the folder you selected into FBX 2013 format thanks to the settings you've previously applied in Game Exporter menu. After that, you're ready to bring the FBX meshes into UDK.
alternate 1b) Importing FBX Static Meshes into UDK
Inside UDK, open 'Content Browser (View-> Browser Windows -> Content Browser). In the bottom-left corner click the Import button next to New button. Go to the folder that contains the meshes you want to import and then into the FBX sub-folder (which was autocreated by 3ds). Select the meshes and hit Import. A new window will appear, apply the following settings as in Figure 9 below. As the name of the package, use the same name as the Folder you imported the meshes from (the main folder, not the "FBX" subfolder name).
Hit "OK to all". You should now see your new package somewhere in the Content Browser folder list (most likely under "NewPackages") with an asteriks next to it (so for example BioS_Arctic*). The asteriks means the package is not saved yet - Right-Click and save your package in a newly-created unique sub-folder in UDK's Content Folder (usually it's something like C:/Program Files/UDK/UDK-2015-01/UDKGame/Content), so in this example we will create Guidetest folder and save our stuff there (as in the Meshplorer exporting method earlier). Save the package under the same name you created it.
Repeat the process for each exported Folder wich meshes (which you then re-exported from inside 3ds as FBX).
alternate 1c) Adding textures to your meshes
If you want to add textures to your meshes to have a better visual view when building your level geometry, you will have to import them into your packages first. Select one of the packages you've just created (for example BioS_Arctic) and import the textures you need for your objects. Some of them may already be in the same folder where the meshes got exported to from uModel, but often the textures are stored in different subfolders. For meshes in BioS_Arctic for example, the textures are originally in a separate BioT_Arctic folder. You will have to check each mesh, if you're having trouble identifying where a texture you need is located, refer back to Meshplorer - the materials menu will tell you which textures in which folders are assigned to your selected Mesh.
UDK works differently than Maya, 3ds or Blender so you can't just "Drag&Drop" the texture you've imported onto your mesh. You have to open the Material assigned to the mesh (it should already be in the package, auto-imported with the FBX file you did earlier) and drag&drop the texture inside the Material. See Figure 10:
Once you've added the texture, click and Hold the Texture's left Black Square Box and link it with the Diffuse channel (assuming the first and probably only texture you will want to apply is the basic Diffuse). If the blank empty Color texture is blocking your view of the "Diffuse" link, click on it and move it away while holding Ctrl. Now you can drag and drop this material onto your mesh.
2a) Using Static Meshes to build your level
I will try to include as much UDK "basics" in describing the steps here as possible, however if you're new to the whole thing (as I am) and still having major issues moving around UDK, which is definitely not as intuitive as modern user interface experience, I recommend World Of Level Design YouTube channel with fantastic videos for beginners, showing all the basics of UDK.
Create a New Blank Level (File-> New Level-> Blank Map) and save it as Leveltest.udk in our previously created Guidetest Folder inside UDK's Content folder. The same folder that has all our mesh packages. Turn the Unlit button on. It's necessary for now as we're working without any lighting placed yet. Once you place Lights (later on that in a second) you can turn the Light button on right next to button on. See Figure 11 below:
Place your painfully prepared & selected Meshes by drag&dropping them into the level right from the Content Browser. You can also Right Click anywhere in the map view and select Add Recent which will add another copy of the previously added Mesh. Use the Move/Rotate/Scale buttons to maniulate the objects. See Figure 12 below:
For any floor/ground meshes, make sure "BlockNonZeroExtenct" is set to true, Right-Click on the mesh in the map view -> StaticMeshActor Properties -> Collision. This will prevent the player character from falling through the object.
2b) Setting up basic lighing
After you've added some objects, it's time for lighting.
- In general the indoor areas with little-to-none outdoor lighting are lit by using Point Light & Spotlight Lights. Using these will not even create Shadowmaps most of the time, just Lightmaps.
- The outdoor areas though, affcted by sun which creates highly visible shadow lines are lit by using Directional Light and these will be the main source of shading in Shadowmaps. It doesn't matter where you place the Directional Light, it will always act in a similar way to the sun and shade your geomtry in the same general direction in every spot on the map. Note - Directional Light will NOT produce shadows (or any lighing in general) if there's anything in its way - you can't light a room with directional light from above if there's a roof blocking the light's way. You can have the directional light pour thorugh windows from the side or have the Point Light lighting up the room the inside tho.
In our Leveltest.udk we will set up a general "sunny day" outdoor lighting. Right-Click anywhere in your map and select Add Actor->Add Light (Point). Then Right click on the light and select Convert Light-> Dominant Light -> DominantDirectionalLight. See Figure 13 below:
This will give us our "Sun" for our level. It doesn't matter where the icon of this DirectionalLight is placed, it covers the entire "infinite" area of the map anyway. Rotate the light as you see fit until you get the shadows result you want.
You should now have a result similar to Figure 14 below:
A level constructed with Static Meshes with Lighting set in place. The final step before "baking" out new lighting into textures is adding a LightmassImportanceVolume. This "box" will contain the computing of our light within its boundaries. Click on Brush in the Main Menu, Volume -> add LightmassImportanceVolume. See Figure 15 below:
The volume box should hold your entire level inside. Move around and scale it appropirately so that your level is within its inner boundaries.
2c) Baking the Lightmaps & Shadowmaps
Select Lighting bake quality (Production) and click Build Lighting (this may take a while). See Figure 16 below:
Once done, save your level. If you check Content Browser and your Leveltest.udk map inside it, you should see it now contains your Lightmap and Shadowmap textures. We are ready to bring this level into the game hurray!*
*This was of course a VERY basic and brief guide on how to use UDK to build worlds and set up static lighting. Level design is far more complex than that, as is lighting, but it is beyond the scope of this tutorial and requires extended knowledge of UDK and Mass Effect level structure itself.
3a) Preparing new empty ME3/LE3 level
With your new level geometry created and lighting baked into Lightmap/Shadowmap textures, we are now ready to bring that into the game. But first we have to prepare an (almost) empty new ME3/LE3 level.
Select which level you want to "hollow out" - in this case we will be using Jacob's Mission as it already has an important DirectionalLight inside of it. Copy BioP_CerJcb, BioSnd_CerJcb, BioA_CerJcb and BioA_CerJcb_000LevelTrans files from CookedPCConsole into a temporary working folder. Rename the files you've just copied - BioP_CerJcb, BioSnd_CerJcb, BioA_CerJcb and BioA_CerJcb_000LevelTrans to BioP_Artest, BioSnd_Artest, BioA_Artest and BioA_Artest_000LevelTrans. Just renaming the files themselves is not enough, we need to open each of the newly-renamed files and change the name of main self-export package within them. Open BioA_CerJcb and locate the main self-export package BioA_CerJcb. Click on Metadata and change the object name into BioA_Artest, hit Enter. You will be asked about adding new name to the list, confirm and then click on "Save all changes" button at the bottom of the Metadata. You should now see the name of self-export package changed. Save the changes in the files (File->Save). See figure 17 below for visual guide:
Repeat the process for the remaining BioP_Artest, BioSnd_Artest, BioA_Artest_000LevelTrans files.
Now open the Package Editor, click File -> Create New -> New Level File and save it as BioD_Artest in your temporary folder. We want the Design section to be an empty level.
Create another New Level File and save it as BioA_Artest_500Landing.
We now have all the files we need. Now open BioA_Artest - lots of modifications needed here. Under BioA_Artest's PersistentLevel section, trash all LevelStreamingKismets except the two for BioA_CerJcb_000LevelTrans and BioA_CerJcb_500Landing. Also trash all BioTriggerStreams except BioTriggerStream_8 (export 18). See Figure 18 below:
Now click on BioWorld (still inside PersistentLevel) and then its StreamingLevels property. Trash all no-longer-relevant references. See Figure 19 below:
Now we need to rename the LevelStreamingKismets that are still there. Click on the first one and inside its Properties click on the PackageName. Change the name to BioA_Artest_000LevelTrans, hit Set to confirm changes and accept adding a new Name to the package. Repeat the same provess for BioA_Artest_500Landing Kismet. See Figure 20 below:
Now go to the sole remaining BioTriggerStream and delete all Items under StreamingStates except for LevelTransArt and LevelTransArtDone. In the remaining items set the InChunkName to "None" instead of any BioA name. Remove all the Load/VisibleSoon chunks (by using the minus "-" button, between + and up/down arrows) and add BioA_Artest_000LevelTrans and BioA_Artest_500Landing into Visible sections. Basically you should transform the Kismet into something like this, Figure 21:
Save end exit. We now have all the files needed, our new level "base", time to transfer stuff we've created in UDK.
3b) Transferring assets between packages
Open the empty level file BioA_Artest_500Landing that you've created in a temporary folder . First you have to move all the mesh assets you're using in your level from other PCC packages from the actual game. So if you're using meshes located in BioS_Arctic, you need to clone the entire package from one of the vanilla packages into you empty BioA_Artest_500Landing file. In our example we will clone BioS_Arctic from the original BioA_CerJcb_500Landing (located in CookedPCConsole folder) into our new empty level BioA_Artest_500Landing file. Drag&drop onto the top package file name, and select "Clone All References". See Figure 22:
Cloning with all referencs should bring all the necessary textures and materials as well. Once you make sure you have all the original meshes in your new level file (in their parent package names!), open the Leveltest.udk file you've created in UDK Editor. Set the two side-by-side, we have to move the assets between the 2 packages, from UDK-readable package into PCC ME3/LE3 readable format.
Move all the Lightmaps (DirectionalMaxComponent, NormalizedAverageColor) and Shadowmaps into your BioA_Artest_500Landing by drag&dropping them onto the top package file name, clone them with "Add only" option. See Figure 23 below:
For each of Shadowmaps transferred into your BioA_Artest_500Landing, you have to additionaly add "NeverStream" property and set it to "True". See figure 24 below:
Now we have to Drag&Drop all of the StaticMeshActors from Leveltest.udk into our BioA_Artest_500Landing. Drag & drop each StaticMeshActor onto the "PersistentLevel" tag, not the top package name this time! Select "Clone" option (not Clone All References!) See Figure 25 below:
This is where all the hardwork naming your mesh packages for UDK usage pays off. StaticMeshComponents call up the general address of a mesh within a local Unreal environment, NOT where the files are on the disc. By having you pre-prepared Meshes in correctly named packages in UDK and your map level in the same folder, the StaticMeshComponent address is the same in UDK as it is now inside Mass Effect's PCC.
Save your file. Now expand one of your newly added StaticMeshActors -> StaticMeshComponent -> ShadowMap2D. Check the LightGuid property and write down the ABCD numbers somewhere (like in Notepad). See Figure 26 below:
Exit and Open BioA_Artest file once again. Move to PersistentLevel and find StaticLightCollectionActor, select the first DirectionalLight and click on its LightGuid. Now you need to change the ABCD values to the same values you've just wrote down in Notepad (the value that Shadowmaps use). See Figure 27 below:
This is necessary to have the SHadowmaps actually appear in the game. Unlike Lightmaps, Shadowmaps still need a technically-active light to show up, regardless if the light doesn't actually do any real-time lighting. The Shadowmaps are still linked to it.
Save your BioA_Artest file and exit. Now copy all of your files from temporary working folder (BioP_Artest, BioD_Artest, BioSndArtest, BioA_Artest, BioA_Artest_000LevelTrans and BioA_Artest_500Landing) into ME3's CookedPCConsole folder*. Launch the game.
*This of course it NOT a ready-to-release modding format, we're just doing it the quickest way for testing.
3c) Launching the level in-game
Once in-game (usually you would start at Normandy), open the Command Console and type "at BioP_Artest". You should spawn inside the Kodiak shuttle. Now open the Console Command again and type "ghost". You can now move through walls and avoid falling down. Move around and "find" your level geometry. Once you see it, position yourself above it and type "walk" in Command Console. Shepard should drop down onto your new geomtry.
If you fall through the floor, check whether your floor/ground StaticMeshComponents have "BlockNonZeroExtenct" property set to true. If not you will have to add it earlier either inside UDK or inside PackageEditor later on.
Congratulations, you're playing your brand new level!
As I said at the beginning, the whole process is experimental and I'm learning ME modding, UDK and all of this myself too and discover things as I go. I will be updating the guide regularly, this is the first step in creating a workflow for the whole level-building idea. Once again I apologize for any mistakes in the guide, feel free to point them out mercilessly, add new info and discoveries to the guide and join up on the level-building&lighting masochistic fun so more stuff can be uncovered and the workflow improved.