Maya Threaded Server

I spent some time this weekend trying to figure out how to run an external process and send any updates back to Maya. I'm not too familiar with network programming but with some scrounging around on blogs and stack-overflow of course, I started to get a reasonable understanding of socket communication. After writing and debugging many examples I have a working server, an external process updating that server and several clients that receive the process updates from the server.

My last goal was to have this external process inform Maya and the tool that resides in Maya, of any updates that may need to be made. If you've written any Python and user interfaces with Python you are likely familiar with GIL (global interpreter lock). In a simplified explanation you could being running a process that will lock up your tool and any parent of the tool, in this case Maya. By using threads you can run your processes in the background and not lock up your tool or Maya. Well, it's less likely to happen. Now that I can communicate with Maya from an external source, I should be able to update any tools within as needed.

Many thanks to the TDAnon guys, including @DhruvGovil for pointing me in the right direction.


Here is an example of me sending external messages to Maya. The scripts I use can be found below.

Maya Create Bone at Custom Pivot

When rigging weapons or hard surface meshes with specific angles or orientations, I want to be able to quickly create a bone with the same orient of a selected mesh component. I want to ensure I have as precise a position and orientation as possible.

With the latest versions of Maya creating and manipulating pivots have become far easier with native tools. Though, I've yet to see a process to do exactly what I've been looking for. I've written a fairly simple script to achieve what I've found to be the most intuitive approach to creating a bone with the position and orientation that I desire.

Here is a brief walk through of how I create bones using a custom pivot. The script I use can be found below.
    
Character Model by Chris Wells

Maya Override Drag & Drop Behavior

A friend of mine recently pointed out that you can change the default behavior of Maya's drag and drop files. I generally have not used this workflow but upon changing how it handles the open conditions, I have found drag and dropping far more useful recently.




I have changed the local Maya file performFileDropAction.mel, from exclusively choosing to do a file import, to giving the user the options they wish to perform.

You can find the original file in  C:\Program Files\Autodesk\Maya20XX\scripts\others\performFileAction.mel

Proper Distribution:

The best implementation of this updated procedure would of course be to package the modified code into a module. This is a far more reasonable approach for distribution if you need to roll out to a team. If you need more information on this process, check out Steve Theodore's article on Maya modules.

http://techartsurvival.blogspot.com/2014/01/mayas-mildy-magical-modules.html

Here is a brief walk through of how the new behavior works & the updated code can be found below.

Customizing Radical Heights - Part 3

In this post I will go over the process of importing and implementing the customization item into game using Unreal Engine 4.

Part 2 can be found here:
https://techanimator.blogspot.com/2018/04/customizing-radical-heights-part-2.html

Import the mesh

The mesh we are importing must have the same skeletal mesh hierarchy as the base body. So, when importing we will choose the base_body_skeleton. We share skeletons in UE4 so that the skeletal meshes can share any animation transformations that may occur on the given skeleton.

Imported Skeletal Mesh

Customization Data

There are several data files that the customization system in Radical Heights needs to populate items on the character. The first is the PlayerPawn, this is the main data container that represents the player character in game, the components it needs to function and of course the skeletal meshes for customization purposes. When the meshes are chosen for customization in the game front end, they are slotted into the named mesh components seen below.

Player Character Pawn

Customization Data Assets

The data assets here mostly represent each "slot" or body part on the character that can be customized. The main customization data asset references the specific body part customization data files. Each body part data file will hold all of the items that the character can wear on that area of the body. For example, the CustomizationUpperBody data file will contain reference & markup data for all shirts, coats, etc that can be worn on the UpperBody location of the character.

Customization Data Assets

UpperBody Customization Data

Customization Body Vids

In Part 1, it was mentioned that we use UV channel 2 on the body mesh to define face group sections that will be used for hiding. Inside the CustomizationBody data file, we define the mapping of UV coordinates to indices. If an item blueprint has parameters that flag certain PartVidsToHide indices, then the associated faces for the body mesh will hide in game. Some examples can be seen below.

Customization Blueprint

After the mesh has been imported it can be assigned to a new customization blueprint. The blueprint is a visual container for the artist to reference the meshes and materials that will define a customization asset. Here the user can also mark up specific parameters for controlling how the asset interacts and functions in relation to other items within the customization system. Below is an UpperBody specific blueprint that has mappings to inform the customization system this item can only be applied to the UpperBody section of the character. When the user saves this file it will populate the associated customization data part file, seen above, with all the references and modified parameters in the found in the item config.

Item Customization Blueprint

Item Config Data

Config Data

1. Incompatible Parts - Given body part item will be hidden when this item is active.
2. Part VIDs to Hide    - Mesh face groups on the body part that will be hidden when this item is active.
3. Part Morphs    - Given body part item will activate a morph target with a given value when this item is active.
4. Bodies  - These are the bodies and part references that this data container represents.

Below are some examples of item config data to make sure items will work, look visually correct, in relation to other items that may be active.

When a full helmet is active we hide the hair part

 

Helmet item hides the Hair part

When a hat is active a morph is applied to the hair part

Hair item is morphed to scrunch inside the Hat

Face groups flagged to hide on the base body

Body face groups for the feet are hidden

Conclusion

At a pretty high level this covers the customization pipeline and process we used for Radical Heights. It should be noted that the core of the editor implementation and runtime system was engineered by Jim Ashcraft. Many systems like these require a solid engineer to get it going and Jim is one of the best!

I hope this was an informative and helpful look into a our customization process.
If you have any questions or comments don't hesitate to post them here hit me up on Twitter.
@Meatplowz

Customizing Radical Heights - Part 2

In this post I will step over the process the artist will run through to get the assets ready for game. The artists can import the mesh from any DCC they choose to create the assets within. The setup process itself is handled entirely in Maya.

Part 1 can be found here:
https://techanimator.blogspot.com/2018/04/customizing-radical-heights-part-1.html

Setting up the item

We need to first import the artist created meshes into a clean Maya scene. As mentioned previously the clothing is created for specific sections of the character. In this scene we have a shirt/upperbody and pants/lowerbody.

Assign the mesh
The artist runs the Customization Builder tool, selects the mesh and plugs the selection into the gender specific field. This field is used to determine which rig/skinned mesh the item will be constructed upon. Upon assigning the mesh it is evaluated for poly count, number of uv channels and number of material assignments. It also makes sure the transforms are correct on the mesh before attempting to bind to a skeleton.

Name Item & Associate Slot
Next we set a unique item name and then we assign the body slot/section for the clothing item. This will rename the mesh accordingly, create an export hierarchy and apply attributes to the groups and meshes used to rig and export the item. These attributes assist the tool when opened later and will also inform the export process how to handle the current asset.

       

• Assign the selected mesh
• Create a unique item name
• Choose the body part section

Rigging the mesh

In this stage we import a clean skinned base body mesh. We locate the flagged gender specific body mesh that was imported and copy the skinweights from the body to the clothing mesh. Additionally, the skinweights and bones are pruned based on the body part chosen, to use the least required amount possible.

The base body mesh is flagged for skinweight copying

Rigged body mesh is versioned to handle updates

Importing not Referencing

It's important to note that I am not referencing the base body mesh here. While there are benefits to references they don't apply very well here. If I update the body mesh or skeleton, I want to know how to propagate those changes to the meshes in the customization files. If I allow referencing to just update naturally I cannot control the changes. After opening an existing customization file a callback will check the local mesh/rig version and compare to the latest one in the depot. I use the version numbers to inform how to process any updates. Some updates happen without user intervention while others require notification. The callback code for the tool also has versioning so I can run code specific to the version number where applicable. Once the scene version is up to date I can tick the local version attributes to the proper index. 

Rules for version updates:

• Major: This requires a complete replacement of the rig in the scene & skin copy update.
• Minor: The tool can update specific aspects of the mesh/rig without requiring a full update.
• Patch: These small changes can generally be ignored and likely require no updates.

Range of Motion

The artist can optionally import the skeleton range of motion to check how the skinning worked out. A rom skeleton with baked animation is imported into the scene and the scene skeleton is constrained to the that skeleton. The artist can then apply some quick skinning fixes to the mesh or let the Character TD address any issues later in a polish pass.

View mesh skinning with Range of Motion

Exporting the mesh
Once everything looks good to go we can move forward with exporting the item. As was mentioned before the exporter handles all of the file naming and determines where to export relative to the project and the item slot. The file must also be saved and added to the depot in a proper location before exporting will occur. This ensures an artist that is putting assets into game will have the source file available as well.

On export the item FBX file is written out, a post-process is run on the FBX file to remove unnecessary nodes and correct the hierarchy where needed. A thumbnail is generated for our asset tool and all of the files are checked out and/or added to perforce automatically.

Exporting the mesh

The clean processed FBX file

Completed Process

Below is a video showing how easy the entire process is for the artist setting up the asset for use in game. The artist can further edit the existing meshes or update with another internal tool that will preserve all of the settings and replace the mesh, uv's, etc, all with updated skinning as needed.

RandallHess - Customization Builder from Randall Hess on Vimeo.

Coming up Next

In the next and final part I will go over how the customization items are implemented in game using Unreal Engine 4.

Part 3

https://techanimator.blogspot.com/2018/05/customizing-radical-heights-part-3.html

Customizing Radical Heights - Part 1

My work on Boss Key Productions most recent project, Radical Heights, focused a lot on character customization. In this post I'm going to discuss the process and creation of a customizable character for Radical Heights.

Having previously worked on Lawbreakers and the Saints Row franchise, much of my time has been focused on creating processes for constructing customization for game characters. The methods described below were no doubt influenced by my past experiences and lessons learned working on these titles.

For customization purposes it is important that your visual goals are outlined initially and the "known" technicalities are addressed before jumping in and creating clothing assets. If you change any of the base character assets during production, you will lose a great deal of time re-working any of the existing clothing assets you may have already created. Understanding the problems up front is key to defining the customization process. You want to minimize the work on the art team and maximize the quality and quantity of assets that can be created.

Character Scale

First and foremost the Character Artist and the Character TD will need to work with the level and environment team to ensure you character height will work properly in the scale of the world. In most cases it is a good idea to keep close to real world units for scale. It's likely easier to create real world weapons that work correctly with your character, if you follow this rule as well.

Play-testing your character in game running around, even early in a rough state will help you identify issues with character and world scale. It's important to nail this down early in your process.

Character Mesh Topology

We started with a fairly standard male mesh until we fully determined how we wanted to approach the semi-stylistic look we wanted the characters to have. Here you can see some stages the proportions of the character went through before we came to our current version. The overall height and proportions didn't have to change drastically from our initial construction. However, some aspects changed enough for us to have to modify the skeleton and in turn adjust and re-export animations.You can also see some of the topology was reduced to minimize the work that it would take to project onto future clothing assets and to lower the polycount overall.

From my past experiences, I found its best to have the character mesh topology to be created in a uniform manner. As you can see here, it's almost a grid-like pattern. This actually makes it easy to define notable landmarks, where clothing assets will generally start and end. If you know the type of clothing and outfits the character may wear, its usually a good idea to make sure edge loops can border these common areas. This will be necessary for hiding the geo of the underlying character mesh at run-time, using a technique I will describe below.

When optimizing the mesh, it's important to remove unnecessary edge loops where the detail doesn't need to project onto clothing assets or the detail doesn't support deformation. You may notice the nipples were removed from the topology as they may only be seen in very few assets and projecting that topology onto most of the clothing would just be too time consuming and wasteful. Only add edge loops to areas that need to support deformation and where the silhouettes of clothing would benefit from that detail.

Character model by Chris Wells

Customization Asset Topology

In the following image you will see how the clothing topology mimics that of the body mesh. This is important for deformation consistency. If the body can deform fine with the existing skeleton then the overlapping clothing assets with the same edge loops, will usually deform the same. You don't want to have issues with the body tearing through the clothing and sticking out. If the topology was different between the meshes then you would likely have to add bones to correct for the tearing and intersecting issues as a result. This is not ideal when you are creating a game that needs to run as optimal as possible and extra bones increases processing time. Fixing visual issues with solutions that would hinder performance is not the best route to take.

Clothing meshes mimic the body topology

Character UVs & Mesh Hiding

The first uv channel is fairly standard as you can see here. The second uv channel is what we use to hide the faces of the body mesh. We first identified the common clothing areas and broke them down into subsections depending on the types of clothing. Next we generated uv shells based on those subsections and smashed the coordinates of those uvs to a singular value in the UV space. This coordinate can then be mapped to a game-side data file and referenced to hide when marked up with corresponding clothing assets. The material on the body skeletal mesh will have a technique that can then hide the triangles, or set them to not render when the coordinates have been flagged to be hidden.

For example, if the character is wearing a shirt, we would flag the first couple of coordinates being selected below to hide, ( U 0.1, 0.2, 0.3 ) Keep in mind the mesh assets need to be created so that the overlying mesh crosses over the edge boundaries, so that holes in the body will not be visible, when the body mesh faces are hidden.

UV Channel 1

UV channel 2 coordinates used to hide body mesh faces in game.

Customization Parts

When determining how complex you want your customization system to be you have to understand the amount of work you will be taking on. In Lawbreakers we did full mesh customization. This is essentially a mesh swap for each version of customization you want to have for the character. The creation is straight-forward to build as long as the meshes follow the conventions of the original character. The in game system is not complex at all, as it is just a mesh reference change, so the programming effort to implement this system was minimal.

For Radical Heights we decided to do a relatively basic "Paper Doll" setup. We researched the type of clothing we expected the character to wear and defined the most notable areas that we want to be able to customize so we could plan our asset creation accordingly. On the surface this looks easy to create for but it can spiral out of control very quickly from asset creation to in game implementation.

Clothing Boundaries

The things to be aware when creating the clothing, are asset parts that cross the boundaries of other parts and the volume/thickness of those assets. For example, if you create a shirt and pants with the same thickness and the shirt crosses over the boundary of the top of the pants, you will get an unwanted intersection. Determining how to create these assets consistently while avoiding these issues need to be decided up front.

Clothing Layers

Layered clothing, such as the UpperBodyArmor here, has to work relative to varying levels of volume/thickness of the upperbody assets. To correct for this instance, we have a morph target on the bulletproof vest mesh, that can be adjusted per upperbody asset at run-time. Addressing a singular asset in this case to correct an issue, works out far better than generating morph targets for each and every shirt asset. Always look for the cheapest way to solve a problem.

Clothing Materials & Draw Calls

Radical Heights can have up to 100 characters and the body mesh for the character already has three draw calls ( head, body and the eyes). For customization purposes we have separated out the clothing to specific parts, and each part on its own becomes a singular draw call, this means by default a character can be upwards of ten draw calls. This is not a great start, which means we need to keep the material count low for each clothing asset. We try to make sure they sure each asset has only a single material and two in the worst case.

Customization Fixes

When you have assets crossing boundaries or overlapping other parts you also have to handle how you want to deal with other crashing issues. In extreme cases we will hide the offending assets, large helmets will hide the hair parts. In other situations, in which we identify a common issue, we will create named morph targets to fix problems. Similar to what we did with the armor we will create a common morph target for each hair asset when a standard hat is enabled. This morph target will scrunch the hair down to work better with the hat. We cannot create a custom morph target for every hat asset, that amount of work and data loading would be ridiculous. Therefore, when we create the hat model, they fit to a predefined location on the head, so that a singular hair morph will work with every hat.

We did later determine that we wanted to have headbands for our characters to customize, as they fit the time period and aesthetic we were going for. This meant we had to circle back and create new morph targets for each of the hair assets to accommodate the headband location. This was an unknown initially but it was important for us to implement this specific data fix for each asset.

These rules for how hats, headbands and every other customization part can work together are important in defining creation guidelines. These guidelines are necessary to prevent data correction bloat from permutation issues. Understanding the volumes and boundaries for each asset is helpful to reducing these data fixes and ensuring compatibility with future assets.

  Character Skeleton

Radical Heights is an online Multiplayer game with up to 100 players. This means the character itself has to be fairly simplistic in terms of run-time complexity. The number of processing skeleton joints should be fairly small even at the lowest Level of Detail (LOD). The deforming corrective deforming joints start to drop off quickly as the character starts to LOD out. The twist and corrective joints are usually the first to go, ultimately leaving only the primary body joints, when deforming characters in the furthest distances on screen.

The supplemental joints here are comprised of joints used for IK hands and feet as well as joints only used when the character is in First Person/Aim Down Sights (ADS) mode.

  Character Rig

We continued to use Epic's ART tools, created by Jeremy Ernst, to construct the player character rig for animation. The tool set allows you team to hit the ground running with a fully animation friendly rig and accompanying animation tools. I use most of the rig construction out of the box with additional custom rigging on top to satisfy our specific needs for animation and deformation.

Shared Skeletal Mesh

On Lawbreakers we chose to have two separate skeletons for First and Third Person. This required us to duplicate the customization assets work done by the character artists and double the amount of data that needed to be loaded by a character in game. The fidelity of the assets had also changed enough to necessitate this choice for that project. However, the speed at which we needed to create assets and the bandwidth we had on Radical Heights, meant we needed to be a bit more conservative and efficient with our time in asset creation. By having a single skeleton for first and third person it meant we should only have to create the clothing assets once and the same assets could be visible for both perspectives in game.
I modified the rig and skeleton hierarchy to be able to handle both aspects for animation purposes. The core of the third person skeleton remains untouched aside from various supplemental joints for controlling the camera and upperbody pitching in first person. This is handled by the rig using a single switch that modifies the visibility of various animation controls and changes multiple constraints that affect the behavior of the rig.

  Character Animation

In First Person, the camera is positioned relative to the head location, so we need to move the head out of the way for animation purposes. In the animation file we just rotate it backwards. In game we also hide the faces on the head and parts of the torso, using the UV method mentioned above, to prevent the them from rendering in the players view.

Animation by Ryan Palser

Checking Animation with Clothing Assets

The pipeline and tool set I created to work with all of the assets on Radical Heights, allows the animator to view customization items in their scene and correct poses where needed. Not every item will work perfectly with every pose, but this gives the animators the opportunity to easily address some of the worst cases scenarios as they are bugged by our QA team.

Animation by Nick Maw-Naing

Coming up Next

In the next post I will be going over the customization tool used by the character artists to setup assets for export and implementation in game.

Part 2

https://techanimator.blogspot.com/2018/04/customizing-radical-heights-part-2.html

Fixing Mechanical joints For Rigging

For Lawbreakers we had a juggernaut class that featured two bipedal robots. The robots had a shared base skeleton hierarchy for animation purposes. However, they had custom joints to handle the visual differences in their armor and armor plating. In most cases the base bipedal skeleton worked just fine, but there were issues when the mechanical mesh joint visual pivots and the actual skeleton joints were inconsistent. We had a couple levels of customization and we had to make sure the joints were modeled consistently to make sure they would work with the rig and animation properly.

In the video below, I go over a process on how to correct the meshes for these issues specifically on the fingers joints.

Fixing Mechanical Joints for Rigging from Randall Hess on Vimeo.

The only thing missing in the video is the clean up of the temporary joints and freezing the mesh to preserve its new modified transform. Once that is done, you can bind the meshes back to the original skeletal hierarchy.