Asterism

Full Moon
Dev Blog #17

29 September 2023

Green screen effects

For one of the music videos in Asterism I've been experimenting with recorded footage from my phone to see what interesting effects I can make. I thought it would be fun to see what I could do with a green screen approach, where part of the video would be replaced with a different texture or video. Last week I showed some of these tests to some of Biome Collective at one of our weekly 'sharings' chats, and have uploaded the video of us discussing them for you to see. I've included screenshots and timestamps from the video along the way if there's anything specific you want to see more about, otherwise the full video is at the bottom of this page. I'll be describing the approach I took and then going through some of the Unity implementation, I'm always happy for anyone to reach out to me if there's any details I've missed that you'd be interested in though, as it won't be a step by step guide in this blog.

How do green screens work?

The main purpose of green screens is to remove backgrounds or replace part of an image. Whatever you want to be replaced should be made a specific colour (the standard for this is bright super-saturated green, which is unlikley to be anywhere else in your footage. But it doesn't need to be green, you could use any colour in theory), which will then be replaced in post-production with another texture or video. Often a physical screen will be used for this, which you might have seen in behind the scenes footage for films that use a lot of CG or locations that would be expensive to film at. Sometimes it's the actors themselves who are green screened by wearing morph suits, which may be used for sci-fi/fantasy/superhero-type characters. Other times it might be a specific screen, window, portal to another dimension, etc. that utilises the green screen effect.

During post-production, a method called Chroma Key is used to select the desired colour range and replace it with the new footage. A lot of video editors will be able to add a chroma key filter. For my testing I used OBS, which is primarily a screen-capturing software, but has a number of filters including chroma key that can be applied to your videos. I'll go into details about how to do this below.

What was my goal?

I had imagined a scene where a person was walking, but instead of their body you could see another scene. Either you could move the person themselves to look around a static image, or you could move the scene that was replacing them. Either way you'd be exploring this alternative scene through the silhouette of the person.

How did I test the idea?

I wanted to do a quick and cheap test before I invested in anything like a morph suit (not that they're expensive, it's just quite a niche thing to own if you don't have a purpose for it). I enlisted the help of James and a friend of mine who was visiting for the weekend to help me film some test footage in a forest nearby. We used a purple towel and purple blanket to cover ourselves like capes and walked around in front of James as he slowly walked forwards. The main reason I chose purple here was mostly down to what large fabric I had available at home, and since I would just be using natural lighting I thought using anything actually green might be a mistake in the setting of a forest, where there is a lot of green.

Video timestamp: 01:44



I also recorded some other footage of me walking away from the camera in various locations. These were for another video test, which I won't go into detail of here. I used this video for the background video of the green screen test though.

Video timestamp: 01:53



I then used OBS's Chroma Key filter to replace a specific purple colour range with the background video.

Video timestamp: 02:05



To use Chroma Key in OBS, add your video as a source and right-click it to add a filter. Then under the 'Effects Filters' you can select 'Chroma Key' and adjust the parameters. These are the parameters I used in my test, but it will be different for each video.

Video timestamp: 03:26



This was an interesting test, and it worked well considering I hadn't put much thought into the colour/lighting etc. OBS had felt like a good starting point to try this out quickly, but seeing the ouptut confirmed for me that I definitely wanted more interactivity in the green screen effect. So next I moved into Unity and tried to recreate the effect there. I followed this tutorial for a chroma key shader, which worked perfectly. I edited it to have multiple colour pickers to broaden the colour range, but otherwise kept it the same. The scene is set up with a 2D plane with the background texture, and another 2D plane with the chroma key shader, using a render texture from a video player as the main texture.

Video timestamp: 04:50



Next, I did another test to see how it would look to have a separate render texture as the background texture. The goal here was to give the player the ability to freely look around a separate space through the replaced image, rather than just to display a static image or video. I began by creating green screen versions of some of the player character sprites from my other levels, and used those as the top 'green screen' layer of the scene (rather than the purple cape video). Now I had a character who could walk around the scene and render another texture wherever they moved. Next, I replaced that background texture with a camera that had it's own render texture, which could then display anything else from the 3D Unity scene.

Video timestamp: 11:46



I stopped at this point, because it felt like I'd tested everything I'd needed to know and would be able to make something cool from this! The last screenshot I want to show is when I replaced the beach background with the video of me walking, which had the unintentional effect of looking like the character was following me.

Video timestamp: 15:56



That in itself could probably lead to lots of different mechanics! It feels like there's a LOT of possibilities with this effect, so it will be fun deciding where to go next with it! I hope this has been useful for anyone out there who also wants to make weird green screen videos! The full video:





Thanks for reading this moon's blog :)

Next Full Moon Dev Blog on 28 October 2023

Previous Full Moon Dev Blog
Next Full Moon Dev Blog

Sign up to my newsletter to get updates about the project!