Finalizing Motion Sensors and Configuration

Another Friday in the studio, and we started off by testing different configurations with 3 Kinects this time. Only to realize after spending quite a bit of time that the live broadcasting of the software we use does not support more than 2 Kinects! So we had to take a step back to our front-back configuration of 2 Kinects.

Next, we tested a few more calibration objects. Yash tried one more time to get a Move controller working, but it was mostly in vain. I think the software is too old and undocumented to use with an independent computer. We had to get creative again and combined our phone light + bottle cap idea with the silicon ball on top of the controller.

(Our final makeshift calibration object)

This method led to the best results we got from different options and we decided to finalize it as our calibration object. Despite multiple tests, we still weren't able to get a "perfect" calibration in iPi, but we got the closest we could. The only data that's preventing us from achieving a perfect calibration is "average position error". We've tried calibrating with different lighting setups in the room and different lengths of calibration video, so I'm guessing the 0.071cm error has to do with our makeshift calibration object.
(The best calibration we got with 2 Kinect setup and the final calibration object)

In the last few minutes of our testing day, we realized that if the Kinects were positioned on two sides of the actor, the live tracking was as good as if they were positioned at the front and back! The side-to-side setup could track a full turn of the performer around herself and it would limit the audience interference with the cameras' views! Ignoring the snapping joints that could be further refined, the below video shows that the Kinects could track pretty well when they can see the performer's sides.


Another thing I wanted to get going this week was the projection screen and the Unreal nDisplay. I've had previous experience working with the 2 rear projectors in the studio, so getting them calibrated with the Scalable Desktop software went pretty smoothly.

(Rear projector setup in the studio and the calibration process)

For setting up nDisplay, I tried to follow the quick start tutorial on Unreal Engine's documentation. However, the tutorial is set up for multiple LED walls. I tried making changes to the configuration based on our single-screen setup, but for some reason, Unreal would crash every time I deleted an extra screen :(

I didn't have much time in the studio to keep troubleshooting for the day, so I'm planning to go home and test a nDisplay setup from scratch on my own laptop. If that works, I should have a pretty good idea on how to do it again in the studio on Monday.

Comments

Post a Comment

Popular posts from this blog

DMX in Unreal Engine

Image
Though I dived into Unreal DMX setup without confirming if my lights were properly connected, I eventually figured out how to get them to work through the engine! My main sources were the Unreal documentation and this specific  tutorial on Youtube. There are many tutorials that show cool DMX stuff that can be done in Unreal, but I've realized that only a limited few actually show how to pipe in the equipment into Unreal Engine or set up the network connections. Here are the steps I followed to get the lighting to work with Unreal: Creating a project and enabling plug-ins. I chose to create a DMX template project, which already came with all necessary plugins enabled. Changing project settings. I created a new "Output Port" and adjusted the settings based on the hardware and network I used. (The first 3 ports are the pre-existing ones from the project template.) - Net3 Gateway uses sACN protocol instead of Art-Net. - "Network Interface IP Address" is the main ...
Read more

Final Tests on CRMX

Image
Putting together everything I've researched and learned so far, created a mini set with DMX lighting on finals week. I've started by gathering all the equipment and setting up all the connections. After a few iterations, my final set included a camera, 5 Aputure lights (controlled via CRMX), and a big screen displaying the Unreal Engine scene for background. For my first test, I wanted to use a scene of a city with multiple, moving light sources. Looking through the Epic store, I found the free Soul City sample and decided to use it for its night scene. I've set up my scene in Unreal Engine and parented a Cine Camera and Scene Capture 2D camera to a locator. These cameras were facing the opposite way from each other. This way, the Cine Camera would render the scene behind the actor and put it on the display screen, while the Scene Capture 2D camera would calculate lighting information from the scene in front of the actor and send it to Pixel Mapping. To make the world move...
Read more

Pixel Mapping

Image
Here is one of the ideas that got stuck with me since the first week of class: taking the light information from a camera or a specific point in space in Unreal and doing some calculations to send the average color into a DMX light in the studio. While looking for information or ideas on how to achieve this, I've come across an Unreal feature called Pixel Mapping. And yes... this feature was exactly what I had been thinking of creating all this time, so turns out I don't have to reinvent the wheel! Here is the Unreal Pixel Mapping documentation with more detailed information. Since this feature didn't seem too complicated to set up, I wanted to give it a try and see how customizable it can be. This process simply has 3 main actors... 2D Scene Capture Camera: This camera is how the artist can decide which area in the scene will be captured in view and used for color averaging. Render Target: This actor takes the output from the camera and turns it into a texture. Pixel Mapp...
Read more