Feeding Live Video into Adobe After Effects - Frischluft Lensfeed for Windows

After Effects Tutorial from The Creative COW Magazine

Barend Onneweer Rotterdam , Netherlands ©2006 Barend Onneweer and CreativeCOW.net. All rights reserved.

Article Focus: Frischluft Lensfeed for Windows - Rough but ready to do the job, this unique AE plug-in fills a need for After Effects users wanting live video to test and set-up shots for later compositing.

Every now and then when I am on the set of a visual effects shoot, I wish I could load the footage being shot into my laptop and do a quick composite to see if I'm getting what I need. Sometimes I'd like to do a quick chroma-key test to see if the foreground separates nicely from the background. Other times, I'd like to see how the set lighting works against a pre-created background.

In very rare cases, I've had the time to actually grab a frame or two from the camera and load them into a laptop. I might suggest a couple of changes to the blocking of the scene or the lighting, based on the result, but that's about it.

So I've had this secret fantasy of being able to take a live camera feed into my laptop and run the video into my compositing software continuously. I've actually done a couple of quick experiments with VJ software packages like Arkaos and Resolume - and although it works, it didn't allow me to use my preferred keying tools and also took me out of my compositing software.

Fortunately, a short while ago I stumbled upon the Frischluft Lensfeed plug-in for After Effects. Frischluft is Philipp Spoeth's oneman company that is probably best known for Lenscare, a high quality lens blur that has been quite popular in visual effects studios for a couple of years now.


USING LENSFEED

Lensfeed is a download-only application that comes with a PDF manual. The introduction in the manual says: "Lensfeed extends After Effects with the possibility to view live feeds from a camera (or any other video device) directly inside a composition." After going through the functions of the software, the manual concludes with a warning: "Since After Effects was not built with plug-ins like Lensfeed in mind, there are a few side effects and problems that occur when Lensfeed is active." The following review outlines my experience with Lensfeed.

Once the software is installed in the After Effects plug-ins folder, it shows up in the Filters menu. Actually, the Lensfeed folder contains 5 effects, named Lensfeed 1 to 5. This will allow for five separate video signals to be fed into After Effects simultaneously, and apparently if you need more, you can drop an email to Frischluft.



To get the video feed going, you simply apply one of the Lensfeed filters to a Solid layer.

The user interface that shows up in the Effect Controls viewer is very rudimentary. There are four buttons, that are easily mistaken for checkboxes. The first button says: "next cam ->". By clicking this button you can cycle through the video inputs that are available on your system.

Lensfeed recognizes almost any video device, such as Firewire, USB video devices and on my video workstation it even recognized my Decklink HD Pro card.

For this test I only had a DV camera available. After connecting the camera to the Firewire port of my laptop, I clicked the ‘next cam' button and miraculously the video signal from the camera appeared in the composition window.

The video can be paused by clicking the ‘toggle feed on/ off' button. When paused, the current source frame is frozen in the Comp viewer. Unfortunately, there's no feedback about this in the user interface, but in the Comp viewer you get a message that says "lensfeed:paused".

When no other things are going on in the composition except the video feed, the frame rate is around 2fps on my 2GHz P4 laptop. On my dual AMD Opteron workstation, the results seem similar. As Philipp suggests in the manual, Adobe After Effects isn't really designed for this sort of thing, and this may well cause the slow refresh on the video feed.



So once you've applied Lensfeed to a layer and selected the right video input, the Comp viewer automatically updates at roughly 2fps. You don't have to run a RAM preview, the Lensfeed layer automatically forces rendering of the frames.

The most obvious use for Lensfeed is at a chromakey shoot. It allows you to feed the actual video into Adobe After Effects, and apply effects and transformations as you would normally do. This makes it very easy to see if there's enough separation between the background and foreground to pull a good key. If you have an image of the final background available, you can bring that into the composition for a preview of how the final composite might look. Matching lighting and camera angles becomes a lot easier this way.

Another way to use the direct video feed into After Effects is do color correction on-set, to see if the raw footage has everything needed when doing final color correction in post production.


PERFORMANCE & QUIRKS

Although the slow frame rate may rule out some situations, I feel that it's not much of an issue when using Lensfeed for on-set supervising of chroma-key shoots. At roughly 2fps it's still interactive enough to get good feedback while adjusting camera angles or set lighting.

When shooting interlaced video, you get raw interlaced video in the composition. To remove the interlacing before compositing you need to apply a de-interlacing filter. Unfortunately Re:VisionFX's FieldsKit only works when it's the first effect in the stack, which makes it all but useless for this purpose. It works when applied to an adjustment layer on top of the Lensfeed layer, but when I also want to apply a keying filter, this quickly makes my composition more complicated than I really need .

As a quick workaround I used "Reduce Interlace Flicker", which subtly softens up the image. Not perfect, but good enough for previz use. I would have liked to see a very simple de-interlacer built into Lensfeed.

With only Keylight applied, the frame rate stays around 2fps at SD resolution. When I applied ‘Reduce Interlace Flicker' and a DV deartifactor, the frame rate dropped to around 1fps.

Another weird quirk is the user interface. Like I mentioned earlier, it consists of four buttons that look like checkboxes. The first button cycles through the available video inputs on your machine. The second toggles "pause". The other two are to increase or reduce resolution, but were only available when I had the Decklink input selected, but I couldn't test whether they worked. This makes Lensfeed feel like it's not quite finished. The manual also mentions some other issues when running Lensfeed, such as the snapshot doesn't work anymore, until you pause the feed.


FINAL VERDICT

On the other hand, when I saw that Lensfeed also recognizes the Decklink HD board in my workstation, I couldn't help thinking of building a small shuttle PC with a Decklink board in it, to bring to the set. By using a laptop I'd be pretty much stuck with a DV signal over Firewire. A Shuttle PC with a Blackmagic Decklink board in it would allow me to take an SDI or even HD-SDI signal directly into After Effects.

The functionality in Lensfeed is unique in the world of Adobe After Effects. There is no other tool or plug-in that will let you feed live video into After Effects. Some of us have been wanting this functionality for a long time. If you do a lot of chroma-key shoots, and use After Effects for compositing, Lensfeed can be extremely helpful in doing quick compositing tests on-set. It is pretty exciting to be able to use After Effects to do a live previz of the composite.

The performance is nothing special, but at the same time it's all I need for this type of use. Lensfeed doesn't make After Effects a live real-time video application, but at around one frame per second for DV footage, it still does the job.

So, do I recommend this plugin? That's a tough question. Lensfeed sells for USD 199, which seems a little steep for an application that looks so rough. The price can probably be explained by the small market for this type of plug-in. But for that price I would expect the user interface to be slightly more polished and user friendly. At the same time, the quirky buttons hardly limit the functionality.

If you are regularly doing chroma-key shoots and run into poorly lit footage, this is a very useful tool. Even at USD 199 you should be able to quickly recoup the investment because it would help reduce the time spent fixing problems in post production.

Lensfeed 1.01 (Windows only): $199 A watermarked demo can be downloaded from the Frischluft website www.frischluft.com


Barend Onneweer
Gouda, The Netherlands

Barend Onneweer is an instructor at the Art Academy in Rotterdam, The Netherlands. He has been a leader in Creative Cow since the day it went online and is one of our After Effects leaders. Barend speaks at seminars and is a contributor to books, magazines and other training tools.

Find more great Creative COW Magazine articles by signing up for the complementary Creative COW Magazine.