Match moving

The match moving is a technique used in the field of the special Effets and bound to the Motion captures. This term is employed to refer to the various techniques making it possible to extract information from movement since a video sequence, and more particularly the movements of camera. Is also known under the name of motion alignment .

In this article, match moving will be defined as being art to extract information from movement since a single video sequence. The use of additional cameras, of sensors of movement, a camera of the type motion control , or of another device are possible as a supplement, but nonnecessary.

The technique of match moving is mainly used for tracker the movement of a camera during a catch of kind so that this movement can be reproduced with identical on a virtual camera in a computer program. Thus, when the real and virtual scenes are amalgamated together (operation called Compositing), they will give the vraisemblante impression to be filmed same point of view.

There are two types of techniques of match moving:

the programs of compositing such as Adobe After Effects, Discreet Combustion and Shake, can apply a technique of two-dimensional match moving . This functionality is able to carry out translations of objects in space image and to add effects such as blur in order to mask the errors of trajectories between two objects moving. This technique is sufficient to create an impression of probability if there are no changes important of the parameters of the camera. For example, a billboard placed in background of a video sequence can be easily tracked in dimension two, then replaced by an image contained in memory.

the three-dimensional tools of match moving can extrapolate three-dimensional information starting from two-dimensional photographs. Among the software able to carry out a three-dimensional match moving one can quote:

2d3 Boujou
Icarus (freeware, always used in spite of a stop of the development)
Maya Live (Module of Maya Unlimited)
PixelFarm PFTrack (commercial reincarnation of Icarus)
Realviz MatchMover
Ssontech SynthEyes
Sciene.D.Visions 3DEqualizer (which gained an academic reward for accomplissment technical)
Voodoo (free software)

These programs make it possible to the users to deduce the movements from cameras as well as other relative movements starting from an unspecified measuring. Information of alignment can then be transferred on a software from computer graphics such as Blender, Lightwave or Maya in order to be used to animate virtual cameras and objects in synthesized image.

The first, and one of best, example where the technique of match moving was used is the film Jurassic Park . The realizers placed balls of tennis coloured in the scene as markers. They then used these markers to track the movement of the camera during the various scenes. This allowed many virtual objects, such as dinosaurs in synthesized images, to be added to scenes having complex movements of camera and even of the camera-shoulder. The balls of tennis were thereafter painted numerically in order to exclude them from the final assembly.

The match moving is right now a tool recognized in the medium of the special effects.

How the match moving functions

The process of match moving can be separate in two parts.

Alignment

The first stage consists in identifying and with tracker targets. A target is a specific point of the image which an algorithm of alignment can lock (in the military direction) and follow on several images. The choice of these targets depends on the algorithm of alignment, but are often luminous/dark places, stop or of the corners. The important thing is that each target represents a specific point of the surface of a real object. When it is trackée, a target becomes a succession of two-dimensional coordinates representing the position of the target through the sequence of image. This continuation is called track . Once these tracks was calculated, they can either be used immediately to make match moving 2D, or to be used to calculate information 3D.

Calibration

The second stage requires a resolution to obtain the movement 3D. The goal is to deduce the movement from the camera by solving an opposite projection of the ways 2D for the position of the camera. This process is called Calibration.

More precisely: when a point of the surface of a three-dimensional object is photographed, its position in the image 2D can be calculated by a function of the type projection 3D. One can consider that a camera is an abstraction which contains all the parameters necessary to the modeling of a camera in a real or virtual universe. Thus, a camera is a vector which contains like elements: the position of the camera, its orientation, its focal distance, and other parameters possible which define how the camera focuses the light on the film. The way in which this vector is built imports little as long as there exists a function of compatible projection P .

The function of projection P takes as entry a vector of camera (noted camera ) and another vector representing the position of a point 3D in space (noted xyz ), and turns over a point 2D which is projected point 3D on a plan says image and located in front of the camera (noted XY ) One has the following expression then:

XY = P ( camera , xyz )

The function of projection transforms a point 3D in particular by removing the component of depth. Without knowing the depth, an opposite projection can only turn over one whole of points 3D solutions. This unit is a line on the basis of the optical center of the camera and passing by the projected point 2D. One can express projection reverses by:

xyz ∈ P' ( camera , XY )

{ xyz :P ( camera , xyz ) = XY }

Let us suppose that we are if target that we are tracking are on the surface of a rigid obejt, for example a building. As we know that the real point xyz will remain at the same place in the space (except if the building becomes deformed) of an image on the other, one can force this point to be constant when well even his position is not known. Thus:

xyz _I = xyz _J

where the indices I and J are arbitrary numbers of images of the scene which we are analyzing. That enables us to affirm that:

P' ( camera _I, XY _I) ∩ P' ( camera _J, XY _J) ≠ {}

Owing to the fact that the value XY _I was given for all the images where the target was tracked by the program, one can solve opposite projection between two images as long as P' ( camera _I, XY _I) ∩ P' ( camera _J, XY _J) is a restricted unit. The whole of the vectors possible camera which are solutions of the equation at moments I and J (noted C_ij).

C_ij = {( camera _I, camera _J): P' ( camera _I, XY _I) ∩ P' ( camera _J, XY _J) ≠ {})

There is thus a whole of pars of vectors camera C_ij for which the intersection of the opposite projection of two points XY _I and XY _J is not-vacuum, preferably small, and is centered around the theoretically stationary point xyz .

In other words, imagine a floating black spot in a white space and a camera. For each position of the space where the camera is placed, there is a whole of corresponding parameters (orientation, focal distance, etc) which will photograph this point exactly same manner. Like a.c. a number of infinite members, only one point is insufficient to determine the current location of the camera.

By increasing the number of targeted points, one can restrict the whole of the possible positions for the camera. For example, if one has a whole of points { xyz _{I, 0},…, xyz _{I, N}} and { xyz _{J, 0},…, xyz _{J, N}} I and J being always indication of image and N are an index representing each target. One can then obtain a whole of pairs of vector-camera {C_{I, J, 0},…, C_{I, J, N}}.

In this manner, one restricts the whole of the possible parameters of camera. The whole of the possible parameters which are appropriate for the camera, F, is the intersection of all the units:

C_{I, J, 0} ∩… ∩ C_{I, J, N}

Smaller is this unit, more it is easy to approach the vector-camera solution. However in practice, of the errors introduced by the phase of follow-up forces a statistical approach to determine the solution, algorithms of optimization are often used. Unfortunately, there are so much parameters in a vector-camera that when each one of these parameters is independent of different, one can be unable to restrict F with a single possibility it does not matter the number of points which one tries to follow. The larger the number of parameters which one can restrict at the time of a catch is (in particular the focal distance), the more it is easy to determine the solution.

One calls the treatment consisting in restricting the possible number of solutions of the movement of the camera in order to reach only one possibility which are appropriate for the phase of compositing: phase of resolution 3D.

Projection of the group of dots

Once the position of the camera was given for each image, it then becomes possible to estimate the position of each target in real space by opposite projection. The resulting whole of points is often named group of dots because of its apparance Nébuleuse. As the group of dots often reveals part of the shape of the scene 3D, it can be used as reference to place objects in synthesized image or, using a program of rebuilding, to create a virtual version of the real scene.

Determination of the plan representing the ground

The camera and the group of dots require to be directed in space. Thus, once the finished calibration, it is necessary to define the plan representing the ground. Normally, there is a unit plan which determines the scale, the orientation and the origin of projected space. Certain programs try to make it automatically however, generally, it is the user who defines this plan. As modification of this plan results only one simple transformation on all the points, the position of such a plan is really only one question of suitability.

Rebuilding

The rebuilding is the interactive process which consists in recreating an object photographed by using the data of alignment. This technique is related to the Photogrammétrie. In this particular case, it is about utitiliser the software of match moving with an aim of reconstruir the scene since an adequate catch.

A program of rebuilding can create three-dimensional objects representatives truths objects of the photgraphiée scene. By using the data of the group of dots and the estimate of the user, the program can create a virtual object and to extract a texture since the video which will be projected on the virtual object like textures surface.

Automatic alignment vs. interactive Alignment

There exist two methods by which the information of movement can be extracted since an image. The interactive alignment rests on the capacities of the user to follow the targets during a sequence. The points tracked by the user are then used to calculate the movement of the camera. The automatic alignment rests on algorithms to identify and follow the targets during the sequence.

The advantage of the interactive alignment is that human can follow a target during a whole sequence without being disorientated by the targets which would not be rigid. The defect is that the user inevitably will introduce small errors which go, while following the objects, to inevitably lead towards a drift .

The advantage of the automatic alignment is that the computer can create much more points than can it human. A greater number of points can ête analyzed for statistically determining which are the most reliable data. The disadvantage of the alignment autoimatic is that, according to the algorithm, the computer can easily be muddled and lose the targets.

The professional software of match moving generally uses a combination interactive alignment automatic alignment. An artist can remove the clearly abnormal points and use one or more alignment mattes in order to block parasitic information out of the process of alignment.

Matte alignment

A " matte tracking" is a concept similar to the Matte painting. However the goal of a matte alignment is to prevent the algorithm of alignment from using not-reliable, without report/ratio or not-rigid data. For example, in a scene where an actor walks in front of a background, the match mover (that which deals with the match moving) will want to use only this background to obtain the movement of its camera knowing that the movement of the actor will interfere in calculations. In this case, the artist will build a matte alignment to follow the actor in the scene and to block this information in the process of alignment.

Refining

As there are often multiple solutions possibiles with the process of calibration and that a significant quantity of error can accumulate, the final stage to obtain the movement often implies a refining (English refining) of the manual solution. That means to modify oneself the movement of the camera by giving indices to the engine of calibration. This calibration, more interactive is sometimes called refining of the calibration.

Material approach

In certain cases where:

a character must interact with an environment created by computer
the combination of a shift phase and of a zoom the way ambiguous
makes the resolution required by the placement of the camera is higher than that which one can obtain by treatment of a video.
the positions of the components the ones compared to the others are not constant (the scene is a deformable solid)

a material approach is necessary. In these cases, of DELs visible or infra-red can be fixed on objects such as supports or on the cameras and an optical system of alignment can be used to track the cameras, the actors and the supports.

This method is preferred only when the material is already necessary to track the actors or the supports, the software approach functioning sufficiently well and not requiring any material. A system of active markers such as that of PhaseSpace makes it possible to integrate these markers into the interiors of the objects of the scene and provided in real-time the relative coordinates of the authorizing system of the complex interactions. Embarked processors modulate the luminosity of DELs in order to differentiate each marker and thus of the hundreds of objects can be tracked.

The Councils for the " match movers"

Memorize tout.
As in any photographic visual effect, take note of each aspect of the catch. That will help during the estimate.
* Memorize the focal distance.
* Know the size of the filmback/sensor.
* Measure the height of the lens compared to the ground.
* Measure the distance between the optical center and the obvious targets.
* Measure the distance between the obvious targets.
* Measure the distance between the positions of beginning and end of movement of the camera.
Créez targets if there are not the good ones.
Avoid broad surfaces which have very small or very repetitive textures. Made marks or adds objects in the scene which could be easily tracked. It will be enough as of repainting them numerically to remove them scene. Use coloured balls, coloured points, or a grid of points on a blue screen or green the spheres is what functions best because it is easy to determine their center can imports the visual angle.
Force as many parameters as possible.
Less there will be variable parameters, easier will be the resolution of the movement of the camera.
* Use a constant focal distance. Do not zoomez.
* Remain on a foot. Of course will not do it to you, but that would be easier.
* Made effects such as the panoramic ones while turning around the optical center. It is about the theoretical point around which one can carry out a rotation without that not modifying the prospect. Even at the time of a catch on a carriage, that will largely simplify the general movement of the camera.
* Remain on a carriage. Certain algorithms of calibration can force the movement 3D with a line or a curve.
Introduisez side movements into your prises.
If you relocate horizontally or vertically, you will introduce parallaxes into your scene. That can improve the precision of your calibration and projections of group of dots.
Evitez the effects of blur.
the blur can increase the error in the perception of the hiring of the target. Great quantities of blur can create losses of follow-up in the process of alignment what results by discontinuities in the tracks. Keep the catches with the most stable possible camera-shoulder and use broad lenses.

See too

Internal bonds

Compositing
Photogrammetry
Realviz: software publisher for the special effects.
Rotoscopie
Virtual studio

External bonds

3D Estimate and Applications to Match Move an article which explains in-depth the mathematical theory which is behind the match moving.

Software

SynthEyes Camera tracker Tools for the mocap, the match moving and other applications for the special effects.
Voodoo a program of match moving free and functional for Linux and Microsoft Windows.
2d3 boujou
Realviz Matchmover
Pixel Farm PFtrack

Material

PhaseSpace - real Solution of mocap time based on a system of DELs for virtual reality, increased reality, application medical,…

Random links: