Lots of Things Before the Big Day!

The day before the final exhibit! We still had a few more things to adjust in the project, but otherwise, I had thought we were in a good spot already. As usual, I had multiple tasks to help in different areas of the project.

I started by fully integrating all of the particle effects along with Vivian. In addition to the sockets I'd used on our puppet skeleton last time for testing, I created more sockets for every touch area. We had 12 of them in total: 2 on each arm, 2 on each leg, 2 on the front/chest, and 2 on the back.

(Some of the sockets on the puppet skeleton to which the emitters will be attached.)

For some sockets that were created inside a larger part of the mesh, like chest/back, I moved them slightly outside the body so that the particles wouldn't disappear inside the mesh. After deciding, which sockets were going to be attached to which touch areas and trigger what kind of particle fx, I made all the necessary connections inside the blueprint.

(All touch-socket-particle connections in the master blueprint.)

(Close-up of socket location-to-particle connections.)

As we tested the particle fx, Vivian made changes to the emitters (size, color, velocity, etc.) and I edited the socket locations as needed. Finally, we were able to get 3 different effects to trigger along the puppet's body based on touch sensor data: warm yellow sparks, blue electric effect, and red destruction effect.

(Blue electric sparks triggered when the back of the puppet is touched.)


As Yash and I were setting up the Kinects and iPi software, we came across the biggest issue of the week yet: our free license for the mocap software had ended!!!

I had been under the impression that the software licenses were given on an email account basis, so I quickly tried creating a new account to get another 30 days of trial. But that didn't work! We tried uninstalling the software, restarting the desktop, and reinstalling it, but nothing seemed to help. The software must have been assigning licenses based on the computer's IP address. This was horrible news because all other aspects of our project depended on the specific machine we were using and we couldn't easily change it!

(Buying the software wasn't even an option when it cost somewhere around $600.)

While trying to fix the issue and not panic on the side, I suddenly had the idea to use a different machine for the iPi software. A new computer would allow us to download a new free license and in theory, as long as the new computer was connected to the same network, we should be able to send the mocap data through Live Link into Unreal in the original computer.

None of our laptops would work for this idea. The only laptop powerful enough to handle the data was Yash's but he had already used up his free trial a few weeks ago. Thus, I rolled out the second computer cart from the side room into the studio.

Setting up the network, managing connections, creating the project drive, installing the software... and we came across another issue.

When both Kinects were connected to the new computer, the software would register both but only use live data from one. No matter what we did, I couldn't get two Kinect cameras to work at the same time in the new machine. I'm not too knowledgeable in computer hardware, but Yash guessed that it was due to the motherboard inside the machine being too old to handle both usb connections... something we definitely couldn't fix ourselves.

At this point, I came up with my wildest but successful idea yet: the ping-pong system.

Now, iPi mocap consists of 2 software: iPi Recorder and iPi Mocap Studio.

  • Recorder is totally free, and this is where we connect the two Kinects and get live data from calibrated cameras which is sent to Mocap Studio.
  • Recorder does not work on the new computer.

  • Mocap Studio is the software for which our free license ran out, and where the camera data is used to retarget animations onto Unreal mannequin. Mocap Studio live streams the skeletal animation into Unreal Engine.
  • Mocap Studio can only work on the new computer.
My grand idea was to connect Kinects and use Recorder in the main computer(1) and broadcast the Kinect data to the new machine(2).
Then, we would use Mocap Studio in the new machine(2) to retarget our live data and stream it back into Unreal in the main machine(1).
Live Link in Unreal in the main machine(1) would receive the data and apply it to the puppet as needed.
Hence... the ping-pong system :D

A little tedious, and probably not optimized at all, we gave this idea a go, and it worked!!! We finally had our Kinects and the mocap running as usual. And the data flow was much smoother than I had imagined, to be honest.

We now had to use two different computers in our final exhibit, which added a tiny bit more work during setup, but as long as everything worked as intended, we didn't mind!

(My new behind-the-scenes setup.)

(iPi Recorder and iPi Mocap Studio running on two different machines.)


Something else I also wanted to look into today was the physics and collisions in our final scene. Last time in the studio, I had briefly tried to get the puppet to interact with some balls in the scene, but couldn't get them to work. Though I wasn't the person who was responsible for looking into collisions in our group, I wanted to play around with some settings in the brief time I had left.

I created some spheres in the scene and made sure collisions and physics were turned on for all of them. After changing some settings in the mattress' collisions, I was able to get the spheres to interact with the mattress and roll off from the sides when dropped from some height.

However, the puppet was still clipping through the spheres instead of interacting with them. Looking back to the default third-person project where I had successfully made the character interact with a ball, I realized that our puppet didn't have its own collision capsule by default. I was not able to create one through the skeleton and skeletal mesh assets, so I created a new blueprint for our puppet.


Inside this blueprint, the puppet gained a collision capsule automatically. On the right tab, I made sure that the correct skeletal mesh was selected and compiled the blueprint to test.

Unfortunately, it seemed that even the capsule didn't help the puppet collide with the spheres in the scene. This might have been because some details in the blueprint were not set correctly. However, I didn't have enough knowledge or time left in the studio to fix the issue, so I had to call it a day and start packing up with everyone else.

If we have time tomorrow before the big show, I want to get the collisions working between my other tasks for sure!

Comments

Post a Comment

Popular posts from this blog

DMX in Unreal Engine

Image
Though I dived into Unreal DMX setup without confirming if my lights were properly connected, I eventually figured out how to get them to work through the engine! My main sources were the Unreal documentation and this specific  tutorial on Youtube. There are many tutorials that show cool DMX stuff that can be done in Unreal, but I've realized that only a limited few actually show how to pipe in the equipment into Unreal Engine or set up the network connections. Here are the steps I followed to get the lighting to work with Unreal: Creating a project and enabling plug-ins. I chose to create a DMX template project, which already came with all necessary plugins enabled. Changing project settings. I created a new "Output Port" and adjusted the settings based on the hardware and network I used. (The first 3 ports are the pre-existing ones from the project template.) - Net3 Gateway uses sACN protocol instead of Art-Net. - "Network Interface IP Address" is the main ...
Read more

Final Tests on CRMX

Image
Putting together everything I've researched and learned so far, created a mini set with DMX lighting on finals week. I've started by gathering all the equipment and setting up all the connections. After a few iterations, my final set included a camera, 5 Aputure lights (controlled via CRMX), and a big screen displaying the Unreal Engine scene for background. For my first test, I wanted to use a scene of a city with multiple, moving light sources. Looking through the Epic store, I found the free Soul City sample and decided to use it for its night scene. I've set up my scene in Unreal Engine and parented a Cine Camera and Scene Capture 2D camera to a locator. These cameras were facing the opposite way from each other. This way, the Cine Camera would render the scene behind the actor and put it on the display screen, while the Scene Capture 2D camera would calculate lighting information from the scene in front of the actor and send it to Pixel Mapping. To make the world move...
Read more

Pixel Mapping

Image
Here is one of the ideas that got stuck with me since the first week of class: taking the light information from a camera or a specific point in space in Unreal and doing some calculations to send the average color into a DMX light in the studio. While looking for information or ideas on how to achieve this, I've come across an Unreal feature called Pixel Mapping. And yes... this feature was exactly what I had been thinking of creating all this time, so turns out I don't have to reinvent the wheel! Here is the Unreal Pixel Mapping documentation with more detailed information. Since this feature didn't seem too complicated to set up, I wanted to give it a try and see how customizable it can be. This process simply has 3 main actors... 2D Scene Capture Camera: This camera is how the artist can decide which area in the scene will be captured in view and used for color averaging. Render Target: This actor takes the output from the camera and turns it into a texture. Pixel Mapp...
Read more