Learn the skills and knowledge required to include virtual production in the VFX creation area of work.
The virtual art department (VAD) is the emerging terminology for the team creating the environments used in virtual production – typically a VFX team. This is a new function, sitting somewhere between traditional VFX and the art department. All the VFX creation discussed in this page mainly relates to that used for the LED volume, which is built and overseen by the VAD.
These tasks are divided into three main functions: designing and articulating VFX ideas, creating VFX components and combining VFX components.
These complement the wider competencies required within each of these functions to undertake more traditional VFX projects, which are detailed in the full National Occupational Standards.
Download a PDF version of these competencies.
Designing and articulating VFX ideas
1. Design and create drawings to meet production requirements
- Design and draw concepts for virtual environments to be used in VP based on input from production design and other key creatives
- Select methods, media, and the relevant scale for producing suitable drawings
- Ensure that drawings clearly show the visual effect at key stages intended by the decision makers for the production (static artwork vs motion)
- When specialist skills are required, consult with appropriate experts, and ensure the relevant technical information is clearly expressed in the drawings and other visual materials
- Use immersive technology such as virtual scouting to present ideas when working on a VP project
- Be flexible and iterative in the design process through regular consultation with other departments in pre-production
- How to use light, colour, and shape to convey structural information, narrative, mood
- How to interpret what is required for presenting in an immersive way for VP
- How to use the LED wall to tell the story in VP
- The importance of creating both visual and technical drawings for VP
- The different types of method and medium – for producing drawings and associated information
- Common challenges when filming on VP stages and how environment design may help mitigate these
2. Create storyboards, animatics or other preparatory visualisations
- Collaborate with the director and other colleagues to facilitate the creative process and delivery of required outcomes
- Create outputs that provides high quality visualisation that will appeal to the intended audience
- Use appropriate media to tell a story, such as storyboards, 2D animatics or 3D pre-vis using a real-time game engine
- Make use of existing resources and know where to obtain these to accelerate the development process
- Take into account shot composition, timings, transitions, camera moves
- Take into consideration the stage size and LED wall size and shape
- Make sure that your visual story meets the proposed design constraints, technical requirements, facilities, and production process for the production
- Ensure visual, narrative and performance continuity for VP
- The importance and benefits of collaborative working for the creative process in VP
- The technical requirements of the production, such as screen ratio as dictated by pre-visualisation
- The intended production process including outsourcing for VP
- The facilities that are available for production such as the volume of the space, equipment/tech the production already has versus what needs to be provided (including but not limited to the screens), personnel etc.
- Any budgetary or design constraints applying to the production
- The expected format for visual storytelling
- The relationship between visualisation and production design
- Film structure and conventions for editing and composition
- The conventions for producing outputs that visually tell stories
- How to time action for performance and narrative
- How to use relevant computer software and programmes, if required, to follow an established design
Creating VFX components
1. Evolve the look for computer generated assets
Asset creation will remain the same process and skills but for virtual production it is essential to know how these assets will be used in the pipeline and what end goal is – such as for real-time use on an LED volume or pre-rendered.
- Analyse briefs, scripts, visual references, and technical and production parameters to determine the required visual style
- Suggest viable looks when working with a vague brief or when no brief is available
- Liaise with those responsible for the visual style to ensure the look meets requirements
- Create the material properties for characters, props or environments that conform to the required visual style
- Create set-ups which test that look development meets requirements
- Identify and implement any necessary changes to improve the look
- Define how standardised or flexible the look needs to be
- Ensure the look maintains continuity with the look of other parts of the production
- Ensure the look being developed is as render efficient as possible
- Design pipelines to ensure the look is correctly applied to different instances in different shots.
- Liaise with those responsible for implementing the look to identify any issues
- Proactively identify and resolve problems that may emerge as the look is implemented by self or others
- Think ahead how your assets will be used and rendered for use in VP
- Create photoreal environments which are effectively optimised for real-time
- Ensuring practical lighting and virtual lighting are appropriately matched
- Generate ideas on how to use lighting to meet the design brief for each shot and confirm the lighting plot
- Set up lighting to meet the creative requirements, technical parameters and production schedules
- Light characters, props and environments to create the required moods and effects to enhance the narrative
- Create lighting tests as necessary to check that lighting meets requirements
- Adjust lighting according to the needs of specific shots and sequences throughout the production to meet production and aesthetic requirements
- Respond positively to feedback from others and changing requirements and make refinements as needed
- Make sure lighting activity will lead to effective rendering
- How to identify the established design theme for a sequence or a project
- The role of the art department, concept artist or production designer in determining visual style
- When it is appropriate to use hero shots and turntables in look development and how to use them
- The other parts of the production that the look has to comply with including on set versions and existing assets
- The physics of motion and resistance
- Surface properties of materials, different types of textures, lighting and other effects that can be applied to 3D objects and environments
- The effects of camera positions, angles, lens types and lighting in relation to objects and environments
- The theory and importance of colour, lighting and cinematography
- The purpose of shaders and how they are developed
- How lights or shaders can be used to standardise looks
- How to exploit the potential of industry-standard software
- The role of modellers, texture artists and others in implementing the look
- The design, style and look of the production and how lighting impacts on it the technical parameters, such as aspect ratio, colour space and format of final deliverables
- Image capture equipment
- Principles of optimisation for real-time engines and apply these to your assets and environment builds
- Theory and techniques and the limitations of real camera gear
- Lighting theory and techniques, real world lighting, physics and vector maths sufficient to carry out the role
- Colour theory and how lighting works in films, painting and photography
- How to set up creative lighting and match real lighting from plate
- How to create different moods and effects such as times of day and seasons through lighting and camera angles and how it is used in story and character
- The importance of light, shade and intensity, the impact these have on colour and how shadows behave and can be used
- The way materials react to light including how shading and reflections behave and can be used
- The effects of camera positions, angles, lens types and lighting in relation to characters, props and environments
- How to exploit the potential of industry-standard software related to lighting
- The requirements of the rendering package that will be used
- The entire production process and the impact your work has on others in the pipeline
- How to effectively work alongside key people
- How to use scripting to streamline tasks
2. Model, texture or paint characters, creatures, props, environments, or backgrounds for visual effects
- Create models, textures or paintings that conform to the developed look and technical requirements and are fit for purpose
- Check and plan for continuity of models, textures or paintings and make sure they are fit for purpose for all required camera positions and angles
- Evaluate the quality of assets you have ensuring they match any on-set versions
- Respond positively and in a timely manner to feedback about work and changing requirements and make refinements promptly as needed
- The importance of understanding the usage of assets
- The importance of prioritising time and efforts when creating assets for VP
- The importance of acquisition of data sets
- The benefits of keeping or obtaining a library of assets for use in VP
3. Create animation rigs
- Create systems and structures to drive, control and deform assets in ways that recreate the desired movement, behaviour and characteristics and meet the brief
- Ensure rigs have a version that can be used for real-time feedback
- The motion characteristics that rigs need to achieve to meet the brief
- Real-time feedback and its importance in the use of VP
- The importance of providing rigs for real-time feedback
- How to exploit the potential of industry-standard software to solve rigging problems
- Awareness of technological developments
- Efficiency of data usage
- The value of seeking early and continuous feedback
4. Create and match live action camera movements in 3D, including use of real-time tracking solutions
- Interpret information from background plates, floor sheets and camera data to identify match moving and 3D tracking requirements
- Calibrate each lens to identify and remove distortion
- Draw vectors to determine access planes within the plate
- Set up tracking scenes which comply with on-set camera data
- Make sure tracking points stick to appropriate points on plates for the length of the shot
- Check for errors and refine as necessary
- Align camera solve to improve match-moving and 3D tracking
- Composite tracking sequence for review
- Share work with others when appropriate and constructively use feedback to revise tracking when required
- Convert plates into various formats for use with inhouse software
- Follow process for the planning, setup and calibration of real-time tracking solutions – for example placing markers, software configuration, hardware mounting and maintenance during filming
- How and where to obtain up-to-date floor sheets and camera data
- The differences between parallax, nodal, non-nodal and mixed motion camera movement and how parallax helps camera tracking software calculate an accurate track
- How to use tracking software
- Real-time tracking solutions
- How to create and use distortion lens grids
- The principles of line up geometry and how to use it for match moving and 3D tracking
- How to measure and recreate focal length
- How to identify and correct errors in tracking
- How to render out geometry
5. Create 3D digital animation
Virtual production may make use of animated objects rendered in real-time in the engine used to project the virtual environment onto the LED wall.
- Analyse briefs, specifications, visual references, technical and production parameters to determine what is needed
- Create and agree with the director an appropriate series of stepped key frames to structure the animation
- Select the appropriate method to create your animation (mo-cap as one)
- Work with performers to get a performance that you are looking for
- Create the movement and performance required by the production to in-between the animation, adjusting the curves and adding extras keys as appropriate
- Make sure rigging techniques and references create credible animation with movements that are realistic for the type of object being modelled and the style of animation required
- Make sure that animations are in sync with the soundtrack
- Make effective use of the software you are using
- Create animations using given camera angles and according to the production demands and schedule
- Review output with relevant people and offer suggestions to assist others with the production
- Respond positively to feedback about the animations you create, making refinements as needed
- Remain constantly flexible and adaptable to new directions, creative requirements and software developments
- Prepare and store files to enable the next stage of production to run efficiently
- The creative style, overall concept and level of animation required for the production
- Technical and production parameters for the project, such as the schedule, timelines, software program, frame count, field size, aspect ratio and format
- In-house pipelines and tools
- Using animation in a real-time engine
- How to read and interpret the relevant sources of information about the production, such as the script, animatic, x-sheet or dope-sheet, character and colour model references, soundtrack
- The context within the production of the scene you are working on
- The principles of 3D animation
- The principles of anatomy and how these affect movement
- The physics of motion and resistance
- How live action reference can influence your ability to animate characters
- How to observe the world around you and find appropriate visual, written, empirical and physical references in order to create credible animation
- The techniques, issues, costs and output of motion capture, and when it is appropriate to use it
- Shot breakdown and continuity
- Animation conventions and techniques for the type of production
- How to use industry-standard 3D animation software
- The possibilities and constraints offered by the software you are using
- The importance of maintaining data security and following your organisation’s guidelines and file structures
Combining VFX components
1. Render visual effects
The render approach for creating environments used in virtual production will likely make more use of a real-time engine, introducing this into the pipeline. Renders may run in real-time on an LED wall or be pre-rendered from software.
- Analyse briefs, scripts, visual references, technical and production parameters to determine what is needed
- Provide information to others on render times and any likely bottlenecks or conflicting priorities in ways that enable solutions to be found
- Establish what render passes are required by artists later down the workflow and accommodate any valid requests to factor these into calculations and schedules.
- Identify and use the technique that is most appropriate to the production
- Undertake test renders to determine the length of time required for rendering and check for errors
- Establish the render settings needed to gain the required appearance, whilst creating sufficient flexibility in the compositing stage
- Apply appropriate render settings to render the required degree of realism
- Prioritise renders in accordance with production priorities and compositing requirements
- Fix bugs and resolve problems that arise with rendering
- Respond positively to feedback and changing requirements and make refinements as needed
- The creative style, overall concept and render quality and compositing requirements
- The technical requirements of the project, such as the aspect ratio, frame-rate, format, film ASPC (advanced signal processing communications), image resolution and colour space
- The rendering requirements for the production
- Who to collect information from and pass information to about render requirements, render times and render schedules
- The intended appearance and required degree of realism of the finished image you are working on
- The computer processing power and storage space available for rendering
- Factors affecting render speed, such as size of texture map, ray and reflection depth, global illumination, ambient occlusion, anti-aliasing, blurry reflections or area shadows
- Rendering techniques, such as ray tracing; texture mapping to define the colour, texture and reflectance of 3D objects and environments; buffering techniques to simulate a sense of perspective to describe the distance between 3D objects and environments; exposure depth of field to alter the sense of depth or focus on 3D objects and environments; creative blurring and transforms to give the appearance of live-action, toon rendering and stereo rendering
- How to save and duplicate render settings across multiple files
- Surface properties and how shading models can be applied to represent variations in different materials
- How to calculate render times and storage space required
Explore the competencies:
Visual effects (VFX) competencies for virtual production - production
Visual effects (VFX) competencies for virtual production - quality monitoring