ThinkBox Deadline Progress
- Oscar Reyes (Unlicensed)
This template is brought to you by Mural, a visual collaboration app.
Previous Pipeline
Previous Pipeline documentation on the Wiki for the pipeline on Github. The code is also available there.
Unmet needs
The problem with this is that it is ran by an old system that needs to be revamped. There is a lot of old code that makes software work together. This could all be replaced by Thinkbox Deadline thanks to the integrations and multiple pieces of software that it comes with.
Objectives
Set up Deadline Monitor to work on Windows and MacOS
Set up Deadline Worker on Windows Machines and MacOS
Set up Deadline Pulse on Windows Machines and MacOS
Set up Deadline Licensing
Set up shared storage for Pipeline
Set up integrations for software that will be used with Deadline (Maya, Houdini, RenderMan, etc.)
User personas
Visualization department
Students of the visualization department
Maybe in the future all of the College of Architecture
Jobs we want to cover
When I send a render to the farm
I want to it to render across all the machines that are part of the pipeline
So I can get my projects rendered across all machines quickly and efficiently
Some history
The Department of Visualization currently has an animation pipeline that relies on multiple pieces to allow students to have a rendering farm that allows them to have distributed compute across the Macs and Linux computers that are in the department. This allows them to render their projects quicker across all the machines.
Constraints
Storage is one of the constraints for this whole project. The storage has to be served in a way that it can work mainly on Windows but later also hopefully work with Mac. Justin has been working on architecting the storage for this project
Another constraint is Centrify. We are trying to get out of using Centrify so the storage is a big key to make sure that we are able to stop using Centrify. The current storage served in the animation pipeline uses Centrify to figure out group permissions and project permissions.
Where we are at:
Alec has built a windows license server for Deadline and setup the Repository (You will have to read deadline documentation for more detailed information.)
We can install the Deadline Client on Windows through Chocolatey and Gorilla
We can install Maya 2020 through Chocolatey and Gorilla
We can install RenderMan 24.1 and license it through Chocolatey and Gorilla. We also move the RenderMan descriptor into the right spot through the installation of one of the RenderMan packages
rendermanRenderer Descriptor:
Move file:
C:\Program Files\Pixar\RenderManForMaya-24.1\etc\rendermanRenderer.xml
To:
C:\Program Files\Autodesk\Maya2020\bin\rendererDesc
Installation Command for Deadline Package for Windows:
***With service user unattended***
***This has already been packaged into chocolatey***
.\DeadlineClient-10.1.17.4-windows-installer.exe --mode unattended --connectiontype Remote --proxyrootdir "deadline-rcs.arch.tamu.edu:4433" --tlsport 4433 --enabletls true --proxycertificate "C:\Users\Public\Deadline10RemoteClient.pfx" --licenseserver "@license.arch.tamu.edu" --launcherservice true --serviceuser "coa\deadline-user" --servicepassword "D0lly-l@te-ensign" --killprocesses true
Installing Deadline (And RenderMan and Maya)
First we need to add the pipeline source for chocolatey because that is where the packages for Deadline. You should be safe to assume that Chocolatey and Gorilla are present in any Windows computer that has been imaged by us. If it does not, then there is a problem and that computer has to be looked at by IT.
The order is important for Maya and RenderMan. You want to install Maya before RenderMan because you need to move a file from the RenderMan directories to the maya directories which is explained above for the RenderMan descriptor.
choco source add --name=pipeline --source=https://proget.arch.tamu.edu/nuget/pipeline/ -u='chocodownload' -p='Aggies2020!' -r
***In theory if you declare the source at the end of the command you should not have to add the source like in the command before this. However, we specified it for documentation purposes.***
choco install its_maya2020 --version 2020.4 --source https://proget.arch.tamu.edu/nuget/Pipeline/
choco install its_rendermanproserver --version 24.1 --source https://proget.arch.tamu.edu/nuget/Pipeline/
choco install its_rendermanformaya --version 24.1 --source https://proget.arch.tamu.edu/nuget/Pipeline/
choco install its_deadlineclient --version 10.1.17.4 --source https://proget.arch.tamu.edu/nuget/Pipeline/
Deadline Software
The Deadline set of software is made up of the Deadline Balancer, Deadline Launcher, Deadline Monitor, Deadline Pulse and finally the Deadline Worker. The main pieces we have already worked with have been the Deadline Monitor and Deadline Worker. The Deadline Monitor is a substitute for the Tractor Monitor.
The function this serves is to monitor the jobs you have submitted and make sure they are rendering correctly. It also lets you check on all your nodes to make sure they are able rendering correctly and not erroring out. From here you can also edit the settings of the nodes to your liking. You can make groups of computers as well as groups of users.
One of the menus that will come in handy us the Tools menu where you can enter super user mode and have total control over Deadline and its settings.
The rest of the documentation for the details you will have to look at the Deadline Documentation.
The packages for Deadline Client, RenderMan and Maya 2020 are on Chocolatey. There is an article to learn to make Chocolatey packages.
Other settings you should be aware of are the Mapped Paths settings. This is the tool that Deadline has in order to path replacement in jobs so that they can render across Windows, Macs and Linux if necessary. These paths will have to be changed once the full production solution gets created.
Another setting is the Mapped Drives setting. This helps Deadline mount the necessary drive if it is not mounted. This will have to be changed once the final production storage solution gets created.
Finally, each software that works with Deadline has a plugin that has to be installed. There are different instructions for each and different requirements to make things work. You also have to configure the correct paths required for rendering for each application. In our case, Maya and RenderMan have been configured already. There are examples of the paths that you need to add for each application and version. You will have to go to Tools > Configure Plugins
Next steps
Explanation: The storage component of the pipeline is very important. It has to be setup correctly to be able to serve projects and storage for students. The storage is a part of the pipeline that has to always be accessible for Deadline in order for them to render because all machines will need to have access to the files in order to render the job that was sent to the Deadline Client. We have been considering to make the storage in a way that it attaches the student’s specific shared drive. It would do that for every single person that logs in but it also unmounts it once they log in. Additionally, when nobody is logged in the share should be available but Deadline has functionality for that.
Other documents