7 Download Services 3.2.4 serial key or number

Next: Configuration Up:3. Administrators' Manual Previous: Introduction   Contents   Index

Subsections

---

This section contains the instructions for installing Condor. The installation will have a default configuration that can be customized. Sections of the manual that follow this one explain customization.

Read this entire section before starting installation.

Please read the copyright and disclaimer information in section  on page  of the manual. Installation and use of Condor is acknowledgment that you have read and agree to the terms.

Obtaining Condor

The first step to installing Condor is to download it from the Condor web site, cromwellpsi.com The downloads are available from the downloads page, at cromwellpsi.com

The platform-dependent Condor files are currently available from two sites. The main site is at the University of Wisconsin-Madison, Madison, Wisconsin, USA. A second site is the Istituto Nazionale di Fisica Nucleare Sezione di Bologna, Bologna, Italy. Please choose the site nearest to you.

Make note of the location of where you download the binary into.

The Condor binary distribution is packaged in the following files and directories:

directions on where to find Condor documentation

these installation directions

the licensing agreement. By installing Condor, you agree to the contents of this file

general information

the Perl script used to install and configure Condor

the Perl script used to install Condor

directory containing C, Fortran and C++ example programs to run with Condor

directory which contains the distribution Condor user programs.

directory which contains the distribution Condor system programs.

directory which contains the distribution Condor configuration data.

directory which contains the distribution Condor libraries.

directory which contains the distribution Condor programs that are only used internally by Condor.

directory which contains the distribution Condor manual pages.

directory which contains the distribution Condor source code for CHIRP and DRMAA.

Before you install, please consider joining the condor-world mailing list. Traffic on this list is kept to an absolute minimum. It is only used to announce new releases of Condor. To subscribe, send a message to majordomo@cromwellpsi.com with the body:

subscribe condor-world

Preparation

Before installation, make a few important decisions about the basic layout of your pool. The decisions answer the questions:

1. What machine will be the central manager?
2. What machines should be allowed to submit jobs?
3. Will Condor run as root or not?
4. Who will be administering Condor on the machines in your pool?
5. Will you have a Unix user named condor and will its home directory be shared?
6. Where should the machine-specific directories for Condor go?
7. Where should the parts of the Condor system be installed?
   • Configuration files
   • Release directory
     • user binaries
     • system binaries
     • directory
     • directory
   • Documentation
8. Am I using AFS?
9. Do I have enough disk space for Condor?

1. What machine will be the central manager?

One machine in your pool must be the central manager. Install Condor on this machine first. This is the centralized information repository for the Condor pool, and it is also the machine that does match-making between available machines and submitted jobs. If the central manager machine crashes, any currently active matches in the system will keep running, but no new matches will be made. Moreover, most Condor tools will stop working. Because of the importance of this machine for the proper functioning of Condor, install the central manager on a machine that is likely to stay up all the time, or on one that will be rebooted quickly if it does crash.

Also consider network traffic and your network layout when choosing your central manager. All the daemons send updates (by default, every 5 minutes) to this machine. Memory requirements for the central manager differ by the number of machines in the pool. A pool with up to about machines will require approximately 25 Mbytes of memory for the central manager's tasks. A pool with about machines will require approximately Mbytes of memory for the central manager's tasks.

A faster CPU will improve the time to do matchmaking.

2. Which machines should be allowed to submit jobs?

Condor can restrict the machines allowed to submit jobs. Alternatively, it can allow any machine the network allows to connect to a submit machine to submit jobs. If the Condor pool is behind a firewall, and all machines inside the firewall are trusted, the configuration entry can be set to *. Otherwise, it should be set to reflect the set of machines permitted to submit jobs to this pool. Condor tries to be secure by default, so out of the box, the configuration file ships with an invalid definition for this configuration variable. This invalid value allows no machine to connect and submit jobs, so after installation, change this entry. Look for the entry defined with the value .

3. Will Condor run as root or not?

Start up the Condor daemons as the Unix user root. Without this, Condor can do very little to enforce security and policy decisions. You can install Condor as any user, however there are both serious security and performance consequences. Please see section  on page  in the manual for the details and ramifications of running Condor as a Unix user other than root.

4. Who will administer Condor?

Either root will be administering Condor directly, or someone else would be acting as the Condor administrator. If root has delegated the responsibility to another person, keep in mind that as long as Condor is started up as root, it should be clearly understood that whoever has the ability to edit the condor configuration files can effectively run arbitrary programs as root.

5. Will you have a Unix user named condor, and will its home directory be shared?

To simplify installation of Condor, create a Unix user named condor on all machines in the pool. The Condor daemons will create files (such as the log files) owned by this user, and the home directory can be used to specify the location of files and directories needed by Condor. The home directory of this user can either be shared among all machines in your pool, or could be a separate home directory on the local partition of each machine. Both approaches have advantages and disadvantages. Having the directories centralized can make administration easier, but also concentrates the resource usage such that you potentially need a lot of space for a single shared home directory. See the section below on machine-specific directories for more details.

Note that the user condor must not be an account into which a person can log in. If a person can log in as user condor, it permits a major security breach, in that the user condor could submit jobs that run as any other user, providing complete access to the user's data by the jobs. A standard way of not allowing log in to an account on Unix platforms is to enter an invalid shell in the password file.

If you choose not to create a user named condor, then you must specify either via the environment variable or the config file setting which cromwellpsi.com pair should be used for the ownership of various Condor files. See section  on UIDs in Condor on page  in the Administrator's Manual for details.

6. Where should the machine-specific directories for Condor go?

Condor needs a few directories that are unique on every machine in your pool. These are , , and . Generally, all three are subdirectories of a single machine specific directory called the local directory (specified by the macro in the configuration file). Each should be owned by the user that Condor is to be run as.

If you have a Unix user named condor with a local home directory on each machine, the could just be user condor's home directory ( = in the configuration file). If this user's home directory is shared among all machines in your pool, you would want to create a directory for each host (named by host name) for the local directory (for example, = /hosts/). If you do not have a condor account on your machines, you can put these directories wherever you'd like. However, where to place them will require some thought, as each one has its own resource needs:

This is the directory that acts as the current working directory for any Condor jobs that run on a given execute machine. The binary for the remote job is copied into this directory, so there must be enough space for it. (Condor will not send a job to a machine that does not have enough disk space to hold the initial binary). In addition, if the remote job dumps core for some reason, it is first dumped to the execute directory before it is sent back to the submit machine. So, put the execute directory on a partition with enough space to hold a possible core file from the jobs submitted to your pool.

The directory holds the job queue and history files, and the checkpoint files for all jobs submitted from a given machine. As a result, disk space requirements for the directory can be quite large, particularly if users are submitting jobs with very large executables or image sizes. By using a checkpoint server (see section  on Installing a Checkpoint Server on page  for details), you can ease the disk space requirements, since all checkpoint files are stored on the server instead of the spool directories for each machine. However, the initial checkpoint files (the executables for all the clusters you submit) are still stored in the spool directory, so you will need some space, even with a checkpoint server.

Each Condor daemon writes its own log file, and each log file is placed in the directory. You can specify what size you want these files to grow to before they are rotated, so the disk space requirements of the directory are configurable. The larger the log files, the more historical information they will hold if there is a problem, but the more disk space they use up. If you have a network file system installed at your pool, you might want to place the log directories in a shared location (such as ), so that you can view the log files from all your machines in a single location. However, if you take this approach, you will have to specify a local partition for the directory (see below).

lock

Condor uses a small number of lock files to synchronize access to certain files that are shared between multiple daemons. Because of problems encountered with file locking and network file systems (particularly NFS), these lock files should be placed on a local partition on each machine. By default, they are placed in the directory. If you place your directory on a network file system partition, specify a local partition for the lock files with the parameter in the configuration file (such as ).

Generally speaking, it is recommended that you do not put these directories (except ) on the same partition as , since if the partition fills up, you will fill up as well. This will cause lots of problems for your machines. Ideally, you will have a separate partition for the Condor directories. Then, the only consequence of filling up the directories will be Condor's malfunction, not your whole machine.

7. Where should the parts of the Condor system be installed?

• Configuration Files
• Release directory
  • User Binaries
  • System Binaries
  • Directory
  • Directory
• Documentation

Configuration Files

There are a number of configuration files that allow you different levels of control over how Condor is configured at each machine in your pool. The global configuration file is shared by all machines in the pool. For ease of administration, this file should be located on a shared file system, if possible. In addition, there is a local configuration file for each machine, where you can override settings in the global file. This allows you to have different daemons running, different policies for when to start and stop Condor jobs, and so on. You can also have configuration files specific to each platform in your pool. See section  on page  about Configuring Condor for Multiple Platforms for details.

In general, there are a number of places that Condor will look to find its configuration files. The first file it looks for is the global configuration file. These locations are searched in order until a configuration file is found. If none contain a valid configuration file, Condor will print an error message and exit:

1. File specified in the environment variable

If you specify a file in the environment variable and there's a problem reading that file, Condor will print an error message and exit right away, instead of continuing to search the other options. However, if no environment variable is set, Condor will search through the other options.

Next, Condor tries to load the local configuration file(s). The only way to specify the local configuration file(s) is in the global configuration file, with the macro. If that macro is not set, no local configuration file is used. This macro can be a list of files or a single file.

Release Directory

Every binary distribution contains a contains five subdirectories: , , , , and . Wherever you choose to install these five directories we call the release directory (specified by the macro in the configuration file). Each release directory contains platform-dependent binaries and libraries, so you will need to install a separate one for each kind of machine in your pool. For ease of administration, these directories should be located on a shared file system, if possible.

• User Binaries:

  All of the files in the directory are programs the end Condor users should expect to have in their path. You could either put them in a well known location (such as ) which you have Condor users add to their environment variable, or copy those files directly into a well known place already in the user's PATHs (such as ). With the above examples, you could also leave the binaries in and put in soft links from to point to each program.

• System Binaries:

  All of the files in the directory are Condor daemons and agents, or programs that only the Condor administrator would need to run. Therefore, add these programs only to the of the Condor administrator.

• Private Condor Binaries:

  All of the files in the directory are Condor programs that should never be run by hand, but are only used internally by Condor.

• Directory:

  The files in the directory are the Condor libraries that must be linked in with user jobs for all of Condor's checkpointing and migration features to be used. also contains scripts used by the condor_compile program to help re-link jobs with the Condor libraries. These files should be placed in a location that is world-readable, but they do not need to be placed in anyone's . The condor_compile script checks the configuration file for the location of the directory.

• Directory:

  contains an subdirectory which holds various example configuration files and other files used for installing Condor. is the recommended location to keep the master copy of your configuration files. You can put in soft links from one of the places mentioned above that Condor checks automatically to find its global configuration file.

Documentation

The documentation provided with Condor is currently available in HTML, Postscript and PDF (Adobe Acrobat). It can be locally installed wherever is customary at your site. You can also find the Condor documentation on the web at: cromwellpsi.com

7. Am I using AFS?

If you are using AFS at your site, be sure to read the section  on page  in the manual. Condor does not currently have a way to authenticate itself to AFS. A solution is not ready for Version This implies that you are probably not going to want to have the for Condor on AFS. However, you can (and probably should) have the Condor on AFS, so that you can share one copy of those files and upgrade them in a centralized location. You will also have to do something special if you submit jobs to Condor from a directory on AFS. Again, read manual section  for all the details.

8. Do I have enough disk space for Condor?

Condor takes up a fair amount of space. This is another reason why it is a good idea to have it on a shared file system. The compressed downloads currently range from a low of about Mbytes for Windows to about Mbytes for Linux. The compressed source code takes approximately 16 Mbytes.

In addition, you will need a lot of disk space in the local directory of any machines that are submitting jobs to Condor. See question 6 above for details on this.

Newer Unix Installation Procedure

The Perl script condor_configure installs Condor. Command-line arguments specify all needed information to this script. The script can be executed multiple times, to modify or further set the configuration. condor_configure has been tested using Perl Use this or a more recent version of Perl.

After download, all the files are in a compressed, tar format. They need to be untarred, as

tar xzf cromwellpsi.com After untarring, the directory will have the Perl scripts condor_configure and condor_install, as well as a ``bin'', ``etc'', ``examples'', ``include'', ``lib'', ``libexec'', ``man'', ``sbin'', ``sql'' and ``src'' subdirectories.

condor_configure and condor_install are the same program, but have different default behaviors. condor_install is identical to running ``condor_configure -install=.''. condor_configure and condor_install work on above directories (``sbin'', etc.). As the names imply, condor_install is used to install Condor, whereas condor_configure is used to modify the configuration of an existing Condor install.

condor_configure and condor_install are completely command-line driven; it is not interactive. Several command-line arguments are always needed with condor_configure and condor_install. The argument

--install=/path/to/release. specifies the path to the Condor release directories (see above). The default for condor_install is ``-install=.''. The argument --install-dir=directory or --prefix=directory specifies the path to the install directory.

The argument

--local-dir=directory specifies the path to the local directory.

The -type option to condor_configure specifies one or more of the roles that a machine may take on within the Condor pool: central manager, submit or execute. These options are given in a comma separated list. So, if a machine is both a submit and execute machine, the proper command-line option is

--type=manager,execute

Install Condor on the central manager machine first. If Condor will run as root in this pool (Item 3 above), run condor_install as root, and it will install and set the file permissions correctly. On the central manager machine, run condor_install as follows.

% condor_install --prefix=~condor \ --local-dir=/scratch/condor --type=manager

To update the above Condor installation, for example, to also be submit machine:

% condor_configure --prefix=~condor \ --local-dir=/scratch/condor --type=manager,submit

As in the above example, the central manager can also be a submit point or and execute machine, but this is only recommended for very small pools. If this is the case, the -type option changes to or or .

After the central manager is installed, the execute and submit machines should then be configured. Decisions about whether to run Condor as root should be consistent throughout the pool. For each machine in the pool, run

% condor_install --prefix=~condor \ --local-dir=/scratch/condor --type=execute,submit

See the condor_configure manual page in section 10 on page  for details.

Starting Condor Under Unix After Installation

Now that Condor has been installed on the machine(s), there are a few things to check before starting up Condor.

1. Read through the file. There are a lot of possible settings and you should at least take a look at the first two main sections to make sure everything looks okay. In particular, you might want to set up security for Condor. See the section  on page  to learn how to do this.
2. For Linux platforms, run the condor_kbdd to monitor keyboard and mouse activity on all machines within the pool that will run a condor_startd; these are machines that execute jobs. To do this, the subsystem will need to be added to the configuration variable definition.

   For Unix platforms other than Linux, Condor can monitor the activity of your mouse and keyboard, provided that you tell it where to look. You do this with the entry in the condor_startd section of the configuration file. On most platforms, reasonable defaults are provided. For example, the default device for the mouse is 'mouse', since most installations have a soft link from that points to the right device (such as if you have a serial mouse, if you have a PS/2 bus mouse, etc). If you do not have a link, you should either create one (you will be glad you did), or change the entry in Condor's configuration file. This entry is a comma separated list, so you can have any devices in count as 'console devices' and activity will be reported in the condor_startd's ClassAd as .

3. (Linux only) Condor needs to be able to find the file. According to the Linux File System Standard, this file should be . If Condor cannot find it there, it looks in . If it still cannot find it, it gives up. So, if your Linux distribution places this file somewhere else, be sure to put a soft link from to point to the real location.

To start up the Condor daemons, execute . This is the Condor master, whose only job in life is to make sure the other Condor daemons are running. The master keeps track of the daemons, restarts them if they crash, and periodically checks to see if you have installed new binaries (and if so, restarts the affected daemons).

If you are setting up your own pool, you should start Condor on your central manager machine first. If you have done a submit-only installation and are adding machines to an existing pool, the start order does not matter.

To ensure that Condor is running, you can run either:

ps -ef | egrep condor_ or ps -aux | egrep condor_ depending on your flavor of Unix. On a central manager machine that can submit jobs as well as execute them, there will be processes for:

• condor_master
• condor_collector
• condor_negotiator
• condor_startd
• condor_schedd

On a central manager machine that does not submit jobs nor execute them, there will be processes for:

• condor_master
• condor_collector
• condor_negotiator

For a machine that only submits jobs, there will be processes for:

• condor_master
• condor_schedd

For a machine that only executes jobs, there will be processes for:

• condor_master
• condor_startd

Once you are sure the Condor daemons are running, check to make sure that they are communicating with each other. You can run condor_status to get a one line summary of the status of each machine in your pool.

Once you are sure Condor is working properly, you should add condor_master into your startup/bootup scripts (i.e. ) so that your machine runs condor_master upon bootup. condor_master will then fire up the necessary Condor daemons whenever your machine is rebooted.

If your system uses System-V style init scripts, you can look in for a script that can be used to start and stop Condor automatically by init. Normally, you would install this script as and put in soft link from various directories (for example, ) that point back to . The exact location of these scripts and links will vary on different platforms.

If your system uses BSD style boot scripts, you probably have an file. Add a line to start up .

Now that the Condor daemons are running, there are a few things you can and should do:

1. (Optional) Do a full install for the condor_compile script. condor_compile assists in linking jobs with the Condor libraries to take advantage of all of Condor's features. As it is currently installed, it will work by placing it in front of any of the following commands that you would normally use to link your code: gcc, g++, g77, cc, acc, c89, CC, f77, fort77 and ld. If you complete the full install, you will be able to use condor_compile with any command whatsoever, in particular, make. See section  on page  in the manual for directions.
2. Try building and submitting some test jobs. See for details.
3. If your site uses the AFS network file system, see section  on page  in the manual.
4. We strongly recommend that you start up Condor (run the condor_master daemon) as user root. If you must start Condor as some user other than root, see section  on page .

Installation on Windows

This section contains the instructions for installing the Windows version of Condor. The install program will set up a slightly customized configuration file that may be further customized after the installation has completed.

Please read the copyright and disclaimer information in section  on page  of the manual. Installation and use of Condor is acknowledgment that you have read and agree to the terms.

Be sure that the Condor tools are of the same version as the daemons installed. The Condor executable for distribution is packaged in a single file named similar to:

condorwinntxmsi This file is approximately 80 Mbytes in size, and it may be removed once Condor is fully installed.

Before installing Condor, please consider joining the condor-world mailing list. Traffic on this list is kept to an absolute minimum. It is only used to announce new releases of Condor. To subscribe, follow the directions given at cromwellpsi.com

For any installation, Condor services are installed and run as the Local System account. Running the Condor services as any other account (such as a domain user) is not supported and could be problematic.

Installation Requirements

• Condor for Windows requires Windows SP4, Windows XP SP2, or a more recent version.
• megabytes of free disk space is recommended. Significantly more disk space could be desired to be able to run jobs with large data files.
• Condor for Windows will operate on either an NTFS or FAT file system. However, for security purposes, NTFS is preferred.
• Condor for Windows requires the Visual C++ C runtime library.

Preparing to Install Condor under Windows

Before installing the Windows version of Condor, there are two major decisions to make about the basic layout of the pool.

1. What machine will be the central manager?
2. Is there enough disk space for Condor?

If the answers to these questions are already known, skip to the Windows Installation Procedure section below, section  on page . If unsure, read on.

• What machine will be the central manager?

  One machine in your pool must be the central manager. This is the centralized information repository for the Condor pool and is also the machine that matches available machines with waiting jobs. If the central manager machine crashes, any currently active matches in the system will keep running, but no new matches will be made. Moreover, most Condor tools will stop working. Because of the importance of this machine for the proper functioning of Condor, we recommend installing it on a machine that is likely to stay up all the time, or at the very least, one that will be rebooted quickly if it does crash. Also, because all the services will send updates (by default every 5 minutes) to this machine, it is advisable to consider network traffic and network layout when choosing the central manager.

  Install Condor on the central manager before installing on the other machines within the pool.

• Is there enough disk space for Condor?

  The Condor release directory takes up a fair amount of space. The size requirement for the release directory is approximately Mbytes. Condor itself, however, needs space to store all of the jobs and their input files. If there will be large numbers of jobs, consider installing Condor on a volume with a large amount of free space.

Installation Procedure Using the MSI Program

Installation of Condor must be done by a user with administrator privileges. After installation, the Condor services will be run under the local system account. When Condor is running a user job, however, it will run that user job with normal user permissions.

Download Condor, and start the installation process by running the installer. The Condor installation is completed by answering questions and choosing options within the following steps.

If Condor is already installed.

If Condor has been previously installed, a dialog box will appear before the installation of Condor proceeds. The question asks if you wish to preserve your current Condor configuration files. Answer yes or no, as appropriate.

If you answer yes, your configuration files will not be changed, and you will proceed to the point where the new binaries will be installed.

If you answer no, then there will be a second question that asks if you want to use answers given during the previous installation as default answers.

STEP 1: License Agreement.

The first step in installing Condor is a welcome screen and license agreement. You are reminded that it is best to run the installation when no other Windows programs are running. If you need to close other Windows programs, it is safe to cancel the installation and close them. You are asked to agree to the license. Answer yes or no. If you should disagree with the License, the installation will not continue.

Also fill in name and company information, or use the defaults as given.

STEP 2: Condor Pool Configuration.

The Condor configuration needs to be set based upon if this is a new pool or to join an existing one. Choose the appropriate radio button.

For a new pool, enter a chosen name for the pool. To join an existing pool, enter the host name of the central manager of the pool.

STEP 3: This Machine's Roles.

Each machine within a Condor pool may either submit jobs or execute submitted jobs, or both submit and execute jobs. A check box determines if this machine will be a submit point for the pool.

A set of radio buttons determines the ability and configuration of the ability to execute jobs. There are four choices:

Do not run jobs on this machine.

This machine will not execute Condor jobs.

Always run jobs and never suspend them.

Run jobs when the keyboard has been idle for 15 minutes.

Run jobs when the keyboard has been idle for 15 minutes, and the CPU is idle.

For testing purposes, it is often helpful to use the always run Condor jobs option.

For a machine that is to execute jobs and the choice is one of the last two in the list, Condor needs to further know what to do with the currently running jobs. There are two choices:

Keep the job in memory and continue when the machine meets the condition chosen for when to run jobs.

Restart the job on a different machine.

This choice involves a trade off. Restarting the job on a different machine is less intrusive on the workstation owner than leaving the job in memory for a later time. A suspended job left in memory will require swap space, which could be a scarce resource. Leaving a job in memory, however, has the benefit that accumulated run time is not lost for a partially completed job.

STEP 4: The Account Domain.

Enter the machine's accounting (or UID) domain. On this version of Condor for Windows, this setting is only used for user priorities (see section  on page ) and to form a default e-mail address for the user.

STEP 5: E-mail Settings.

Various parts of Condor will send e-mail to a Condor administrator if something goes wrong and requires human attention. Specify the e-mail address and the SMTP relay host of this administrator. Please pay close attention to this e-mail, since it will indicate problems in the Condor pool.

STEP 6: Java Settings.

In order to run jobs in the java universe, Condor must have the path to the jvm executable on the machine. The installer will search for and list the jvm path, if it finds one. If not, enter the path. To disable use of the java universe, leave the field blank.

STEP 7: Host Permission Settings.

Machines within the Condor pool will need various types of access permission. The three categories of permission are read, write, and administrator. Enter the machines or domain to be given access permissions, or use the defaults provided. Wild cards and macros are permitted.

Read

Read access allows a machine to obtain information about Condor such as the status of machines in the pool and the job queues. All machines in the pool should be given read access. In addition, giving read access to *cromwellpsi.com will allow the Condor team to obtain information about the Condor pool, in the event that debugging is needed.

Write

All machines in the pool should be given write access. It allows the machines you specify to send information to your local Condor daemons, for example, to start a Condor job. Note that for a machine to join the Condor pool, it must have both read and write access to all of the machines in the pool.

Administrator

A machine with administrator access will be allowed more extended permission to do things such as change other user's priorities, modify the job queue, turn Condor services on and off, and restart Condor. The central manager should be given administrator access and is the default listed. This setting is granted to the entire machine, so care should be taken not to make this too open.

For more details on these access permissions, and others that can be manually changed in your configuration file, please see the section titled Setting Up IP/Host-Based Security in Condor in section section  on page .

STEP 8: VM Universe Setting.

A radio button determines whether this machine will be configured to run vm universe jobs utilizing VMware. In addition to having the VMware Server installed, Condor also needs Perl installed. The resources available for vm universe jobs can be tuned with these settings, or the defaults listed may be used.

Version

Use the default value, as only one version is currently supported.

Maximum Memory

The maximum memory that each virtual machine is permitted to use on the target machine.

Maximum Number of VMs

The number of virtual machines that can be run in parallel on the target machine.

Networking Support

The VMware instances can be configured to use network support. There are four options in the pull-down menu.

• None: No networking support.
• NAT: Network address translation.
• Bridged: Bridged mode.
• NAT and Bridged: Allow both methods.

Path to Perl Executable

The path to the Perl executable.

STEP 9: HDFS Settings.

A radio button enables support for the Hadoop Distributed File System (HDFS). When enabled, a further radio button specifies either name node or data node mode.

Running HDFS requires Java to be installed, and Condor must know where the installation is. Running HDFS in data node mode also requires the installation of Cygwin, and the path to the Cygwin directory must be added to the global environment variable.

HDFS has several configuration options that must be filled in to be used.

Primary Name Node

The full host name of the primary name node.

Name Node Port

The port that the name node is listening on.

Name Node Web Port

The port the name node's web interface is bound to. It should be different from the name node's main port.

STEP Choose Setup Type

The next step is where the destination of the Condor files will be decided. We recommend that Condor be installed in the location shown as the default in the install choice: . This is due to several hard coded paths in scripts and configuration files. Clicking on the Custom choice permits changing the installation directory.

Installation on the local disk is chosen for several reasons. The Condor services run as local system, and within Microsoft Windows, local system has no network privileges. Therefore, for Condor to operate, Condor should be installed on a local hard drive, as opposed to a network drive (file server).

The second reason for installation on the local disk is that the Windows usage of drive letters has implications for where Condor is placed. The drive letter used must be not change, even when different users are logged in. Local drive letters do not change under normal operation of Windows.

While it is strongly discouraged, it may be possible to place Condor on a hard drive that is not local, if a dependency is added to the service control manager such that Condor starts after the required file services are available.

Unattended Installation Procedure Using the Included Set Up Program

This section details how to run the Condor for Windows installer in an unattended batch mode. This mode is one that occurs completely from the command prompt, without the GUI interface.

The Condor for Windows installer uses the Microsoft Installer (MSI) technology, and it can be configured for unattended installs analogous to any other ordinary MSI installer.

The following is a sample batch file that is used to set all the properties necessary for an unattended install.

@echo on set ARGS= set ARGS=NEWPOOL="N" set ARGS=%ARGS% POOLNAME="" set ARGS=%ARGS% RUNJOBS="C" set ARGS=%ARGS% VACATEJOBS="Y" set ARGS=%ARGS% SUBMITJOBS="Y" set ARGS=%ARGS% CONDOREMAIL="you@cromwellpsi.com" set ARGS=%ARGS% SMTPSERVER="cromwellpsi.comost" set ARGS=%ARGS% HOSTALLOWREAD="*" set ARGS=%ARGS% HOSTALLOWWRITE="*" set ARGS=%ARGS% HOSTALLOWADMINISTRATOR="$(IP_ADDRESS)" set ARGS=%ARGS% INSTALLDIR="C:\Condor" set ARGS=%ARGS% POOLHOSTNAME="$(IP_ADDRESS)" set ARGS=%ARGS% ACCOUNTINGDOMAIN="none" set ARGS=%ARGS% JVMLOCATION="C:\Windows\system32\cromwellpsi.com" set ARGS=%ARGS% USEVMUNIVERSE="N"set ARGS=%ARGS% VMMEMORY="" set ARGS=%ARGS% VMMAXNUMBER="$(NUM_CPUS)" set ARGS=%ARGS% VMNETWORKING="N" set ARGS=%ARGS% USEHDFS="N" set ARGS=%ARGS% NAMENODE="" set ARGS=%ARGS% HDFSMODE="HDFS_NAMENODE" set ARGS=%ARGS% HDFSPORT="" set ARGS=%ARGS% HDFSWEBPORT="" msiexec /qb /l* cromwellpsi.com /i condorwinntxmsi %ARGS%

Each property corresponds to answers that would have been supplied while running an interactive installer. The following is a brief explanation of each property as it applies to unattended installations:

NEWPOOL = < Y | N >

determines whether the installer will create a new pool with the target machine as the central manager.

POOLNAME

sets the name of the pool, if a new pool is to be created. Possible values are either the name or the empty string .

RUNJOBS = < N | A | I | C >

determines when Condor will run jobs. This can be set to:

• Never run jobs (N)
• Always run jobs (A)
• Only run jobs when the keyboard and mouse are Idle (I)
• Only run jobs when the keyboard and mouse are idle and the CPU usage is low (C)

VACATEJOBS = < Y | N >

determines what Condor should do when it has to stop the execution of a user job. When set to Y, Condor will vacate the job and start it somewhere else if possible. When set to N, Condor will merely suspend the job in memory and wait for the machine to become available again.

SUBMITJOBS = < Y | N >

will cause the installer to configure the machine as a submit node when set to Y.

CONDOREMAIL

sets the e-mail address of the Condor administrator. Possible values are an e-mail address or the empty string .

HOSTALLOWREAD

is a list of host names that are allowed to issue READ commands to Condor daemons. This value should be set in accordance with the setting in the configuration file, as described in section  on page .

HOSTALLOWWRITE

is a list of host names that are allowed to issue WRITE commands to Condor daemons. This value should be set in accordance with the setting in the configuration file, as described in section  on page .

HOSTALLOWADMINISTRATOR

is a list of host names that are allowed to issue ADMINISTRATOR commands to Condor daemons. This value should be set in accordance with the setting in the configuration file, as described in section  on page .

INSTALLDIR

defines the path to the directory where Condor will be installed.

POOLHOSTNAME

defines the host name of the pool's central manager.

ACCOUNTINGDOMAIN

defines the accounting (or UID) domain the target machine will be in.

JVMLOCATION

defines the path to Java virtual machine on the target machine.

SMTPSERVER

defines the host name of the SMTP server that the target machine is to use to send e-mail.

VMMEMORY

an integer value that defines the maximum memory each VM run on the target machine.

VMMAXNUMBER

an integer value that defines the number of VMs that can be run in parallel on the target machine.

VMNETWORKING = < N | A | B | C >

determines if VM Universe can use networking. This can be set to:

• None (N)
• NAT (A)
• Bridged (B)
• NAT and Bridged (C)

USEVMUNIVERSE = < Y | N >

will cause the installer to enable VM Universe jobs on the target machine.

PERLLOCATION

defines the path to Perl on the target machine. This is required in order to use the vm universe.

USEHDFS = < Y | N >

determines if HDFS is run.

HDFSMODE < HDFS_DATANODE | HDFS_NAMENODE >

sets the mode HDFS runs in.

NAMENODE

sets the host name of the primary name node.

HDFSPORT

sets the port number that the primary name node listens to.

HDFSWEBPORT

sets the port number that the name node web interface is bound to.

After defining each of these properties for the MSI installer, the installer can be started with the msiexec command. The following command starts the installer in unattended mode, and it dumps a journal of the installer's progress to a log file:

msiexec /qb /lxv* cromwellpsi.com /i condorwinntxmsi [property=value]

More information on the features of msiexec can be found at Microsoft's website at cromwellpsi.com

Manual Installation Condor on Windows

If you are to install Condor on many different machines, you may wish to use some other mechanism to install Condor on additional machines rather than running the Setup program described above on each machine.

WARNING: This is for advanced users only! All others should use the Setup program described above.

Here is a brief overview of how to install Condor manually without using the provided GUI-based setup program:

The Service

The service that Condor will install is called "Condor". The Startup Type is Automatic. The service should log on as System Account, but do not enable "Allow Service to Interact with Desktop". The program that is run is condor_cromwellpsi.com.

The Condor service can be installed and removed using the tool, which is included in Windows XP and Windows Server. The tool is also available as part of the Windows Resource Kit.

Installation can be done as follows:

sc create Condor binpath= c:\condor\bin\condor_cromwellpsi.com

To remove the service, use:

sc delete Condor

The Registry

Condor uses a few registry entries in its operation. The key that Condor uses is HKEY_LOCAL_MACHINE/Software/Condor. The values that Condor puts in this registry key serve two purposes.

1. The values of CONDOR_CONFIG and RELEASE_DIR are used for Condor to start its service.

   CONDOR_CONFIG should point to the file. In this version of Condor, it must reside on the local disk.

   RELEASE_DIR should point to the directory where Condor is installed. This is typically , and again, this must reside on the local disk.

2. The other purpose is storing the entries from the last installation so that they can be used for the next one.

The File System

The files that are needed for Condor to operate are identical to the Unix version of Condor, except that executable files end in . For example the on Unix one of the files is and on Condor the corresponding file is .

These files currently must reside on the local disk for a variety of reasons. Advanced Windows users might be able to put the files on remote resources. The main concern is twofold. First, the files must be there when the service is started. Second, the files must always be in the same spot (including drive letter), no matter who is logged into the machine.

Note also that when installing manually, you will need to create the directories that Condor will expect to be present given your configuration. This normally is simply a matter of creating the , , and directories.

Starting Condor Under Windows After Installation

After the installation of Condor is completed, the Condor service must be started. If you used the GUI-based setup program to install Condor, the Condor service should already be started. If you installed manually, Condor must be started by hand, or you can simply reboot. NOTE: The Condor service will start automatically whenever you reboot your machine.

To start Condor by hand:

1. From the Start menu, choose Settings.
2. From the Settings menu, choose Control Panel.
3. From the Control Panel, choose Services.
4. From Services, choose Condor, and Start.

Or, alternatively you can enter the following command from a command prompt:

net start condor

Run the Task Manager (Control-Shift-Escape) to check that Condor services are running. The following tasks should be running:

• condor_cromwellpsi.com
• condor_cromwellpsi.com, if this machine is a central manager.
• condor_cromwellpsi.com, if this machine is a central manager.
• condor_cromwellpsi.com, if you indicated that this Condor node should start jobs
• condor_cromwellpsi.com, if you indicated that this Condor node should submit jobs to the Condor pool.

Also, you should now be able to open up a new cmd (DOS prompt) window, and the Condor bin directory should be in your path, so you can issue the normal Condor commands, such as condor_q and condor_status.

Condor is Running Under Windows Now What?

Once Condor services are running, try submitting test jobs. Example 2 within section  on page  presents a vanilla universe job.

RPMs

RPMs are available in Condor Version We provide a Yum repository, as well as installation and configuration in one easy step. This RPM installation is currently available for Red Hat-compatible systems only. As of Condor version , the Condor RPM installs into FHS locations.

Yum repositories are at cromwellpsi.com . The repositories are named to distinguish stable releases from development releases and by Red Hat version number. The 4 repositories are:

Here are an ordered set of steps that get Condor running using the RPM.

1. The Condor package will automatically add a user/group, if it does not exist already. Sites wishing to control the attributes of this user/group should add the user/group manually before installation.
2. Download and install the meta-data that describes the appropriate YUM repository. This example is for the stable series, on RHEL 5. cd /etc/cromwellpsi.comd wget cromwellpsi.com Note that this step need be done only once; do not get the same repository more than once.
3. Install Condor. For bit machines: yum install condor For bit machines: yum install condor.x86_64
4. As needed, edit the Condor configuration files to customize. The configuration files are in the directory . Do not use condor_configure or condor_install for configuration. The installation will be able to find configuration files without additional administrative intervention, as the configuration files are placed in , and Condor searches this directory.
5. Start Condor daemons: /sbin/service condor start

Debian Packages

Debian packages are available in Condor Version We provide an APT repository, as well as installation and configuration in one easy step. These Debian packages of Condor are currently available for Debian 4 (Etch) and Debian 5 (Lenny). As of Condor version , the Condor Debian package installs into FHS locations.

The Condor APT repositories are specified at cromwellpsi.com . See this web page for repository information.

Here are an ordered set of steps that get Condor running.

1. The Condor package will automatically add a user/group, if it does not exist already. Sites wishing to control the attributes of this user/group should add the user/group manually before installation.
2. If not already present, set up access to the appropriate APT repository; they are distinguished as stable or development release, and by operating system. Ensure that the correct one of the following release and operating system-specific lines is in the file . deb cromwellpsi.com etch contrib deb cromwellpsi.com etch contrib deb cromwellpsi.com lenny contrib deb cromwellpsi.com lenny contrib Note that this step need be done only once; do not add the same repository more than once.
3. Install and start Condor services: apt-get update apt-get install condor
4. As needed, edit the Condor configuration files to customize. The configuration files are in the directory . Do not use condor_configure or condor_install for configuration. The installation will be able to find configuration files without additional administrative intervention, as the configuration files are placed in , and Condor searches this directory.

   Then, if any configuration changes are made, restart Condor with

   /etc/init.d/condor restart

Upgrading - Installing a Newer Version of Condor

Section  on page  within the section on Pool Management describes strategies for doing an upgrade: changing the running version of Condor from the current installation to a newer version.

Dynamic Deployment

Dynamic deployment is a mechanism that allows rapid, automated installation and start up of Condor resources on a given machine. In this way any machine can be added to a Condor pool. The dynamic deployment tool set also provides tools to remove a machine from the pool, without leaving residual effects on the machine such as leftover installations, log files, and working directories.

Installation and start up is provided by condor_cold_start. The condor_cold_start program determines the operating system and architecture of the target machine, and transfers the correct installation package from an ftp, http, or grid ftp site. After transfer, it installs Condor and creates a local working directory for Condor to run in. As a last step, condor_cold_start begins running Condor in a manner which allows for later easy and reliable shut down.

The program that reliably shuts down and uninstalls a previously dynamically installed Condor instance is condor_cold_stop. condor_cold_stop begins by safely and reliably shutting off the running Condor installation. It ensures that Condor has completely shut down before continuing, and optionally ensures that there are no queued jobs at the site. Next, condor_cold_stop removes and optionally archives the Condor working directories, including the directory. These archives can be stored to a mounted file system or to a grid ftp site. As a last step, condor_cold_stop uninstalls the Condor executables and libraries. The end result is that the machine resources are left unchanged after a dynamic deployment of Condor leaves.

Configuration and Usage

Dynamic deployment is designed for the expert Condor user and administrator. Tool design choices were made for functionality, not ease-of-use.

Like every installation of Condor, a dynamically deployed installation relies on a configuration. To add a target machine to a previously created Condor pool, the global configuration file for that pool is a good starting point. Modifications to that configuration can be made in a separate, local configuration file used in the dynamic deployment. The global configuration file must be placed on an ftp, http, grid ftp, or file server accessible by condor_cold_start. The local configuration file is to be on a file system accessible by the target machine. There are some specific configuration variables that may be set for dynamic deployment. A list of executables and directories which must be present for Condor to start on the target machine may be set with the configuration variables and . If defined and the comma-separated list of executables or directories are not present, then condor_cold_start exits with error. Note this does not affect what is installed, only whether start up is successful.

A list of executables and directories which are recommended to be present for Condor to start on the target machine may be set with the configuration variables and . If defined and the comma-separated lists of executables or directories are not present, then condor_cold_start prints a warning message and continues. Here is a portion of the configuration relevant to a dynamic deployment of a Condor submit node:

DEPLOYMENT_REQUIRED_EXECS = MASTER, SCHEDD, PREEN, STARTER, \ STARTER_STANDARD, SHADOW, \ SHADOW_STANDARD, GRIDMANAGER, GAHP, CONDOR_GAHP DEPLOYMENT_REQUIRED_DIRS = SPOOL, LOG, EXECUTE DEPLOYMENT_RECOMMENDED_EXECS = CREDD DEPLOYMENT_RECOMMENDED_DIRS = LIB, LIBEXEC

Additionally, the user must specify which Condor services will be started. This is done through the configuration variable. Another excerpt from a dynamic submit node deployment configuration:

DAEMON_LIST = MASTER, SCHEDD

Finally, the location of the dynamically installed Condor executables is tricky to set, since the location is unknown before installation. Therefore, the variable is defined in the environment. It corresponds to the location of the dynamic Condor installation. If, as is often the case, the configuration file specifies the location of Condor executables in relation to the variable, the configuration can be made dynamically deployable by setting to as

RELEASE_DIR = $(DEPLOYMENT_RELEASE_DIR)

In addition to setting up the configuration, the user must also determine where the installation package will reside. The installation package can be in either tar or gzipped tar form, and may reside on a ftp, http, grid ftp, or file server. Create this installation package by tar'ing up the binaries and libraries needed, and place them on the appropriate server. The binaries can be tar'ed in a flat structure or within and . Here is a list of files to give an example structure for a dynamic deployment of the condor_schedd daemon.

% tar tfz cromwellpsi.com bin/ bin/condor_config_val bin/condor_q sbin/ sbin/condor_preen sbin/condor_cromwellpsi.com sbin/condor_cromwellpsi.com sbin/condor_schedd sbin/condor_master sbin/condor_gridmanager sbin/gahp_server sbin/condor_starter sbin/condor_shadow sbin/condor_c-gahp sbin/condor_off

---

Next: Configuration Up:3. Administrators' Manual Previous: Introduction   Contents   Index condor-admin@cromwellpsi.com

To clear the issue, try the following points one by one:

Check your License Type

Make sure the correct license type is in use. See  Find License Information - Download & Install.

Update your software

Check your internet connection

Make sure the internet connection is not filtered by a proxy/firewall (see  Establish an Internet connection to continue - when launching Autodesk software in a proxy-secured environment).

Reset license and clear the cromwellpsi.com file

• For versions and later, install the latest Autodesk Licensing Service Update and Autodesk Single Sign On Component (AdSSO). Find them with the Autodesk Desktop App or from your Autodesk Account.
  • After installing the Autodesk Licensing Service Update, reset the program license type by running following commands: 

cd C:\Program Files (x86)\Common Files\Autodesk Shared\AdskLicensing\Current\helper\ AdskLicensingInstHelper change -pk L1 -pv F -lm ""

Note: The above command is specific to AutoCAD , change product key (L1) to your Autodesk (or higher) software product key.

• For versions and earlier go to C:\ProgramData\Autodesk\CLM\LGS and delete the folder specific to the product <ProductKey_Version> and delete cromwellpsi.com file if exists.

For example, for AutoCAD , the path would be 

C:\ProgramData\Autodesk\CLM\LGS\K1_F\cromwellpsi.com

• Delete cromwellpsi.com from %LocalAppData%\Autodesk\Web Services and launch the product again.
• Launch your Autodesk software and on Lets Get Started dialog window, select the correct license type.

Switch graphic libraries used by the Licensing/Login Component (Windows 7 only)

Licensing/Login Component by default uses Qt. Qt checks the OpenGL drivers or later and loads/uses the native OpenGL driver. If OpenGL driver is lower than , Qt loads/uses the ANGLE library instead. Try entering the following system variables:

QT_OPENGL= angle 
QT_ANGLE_PLATFORM= warp
 

Note: Other possible entered values for QT_ANGLE_PLATFORM are d3d9and d3d11. Here is an example on how to set up a system variable on Windows.

Set up Desktop App with your proxy details

If you are behind a proxy, make sure to set up the proxy IP address and port inside the Autodesk Desktop App. See:  To Specify Proxy Settings.

Set up Antivirus exclusions

Make sure that your antivirus is not blocking the software. See:  How to set up antivirus (AV) exclusions to improve stability and performance of Autodesk software.

Repair the Autodesk License Service for serial number based software (Windows)

1. Navigate to to Windows Control Panel > Date & Time.
2. Verify that the date and time are set correctly for the time zone.
3. Open the Task Manager, and select Autodesk Desktop App​. Select "End Task."
4. Also, end the following processes in the Task Manager if they are running:

• cromwellpsi.com*32
• ​cromwellpsi.com*32

1. Close all any other Autodesk software.
2. Navigate to the Control Panel and select "Programs and Features."
3. Right click on Autodesk License Service, and select "Repair."
4. Reset the Autodesk license information cache:

Note: Hidden folders must be revealed to proceed:  show hidden files and folders. 

Note: If running, cromwellpsi.com process may need to be ended in the Windows Task Manager.

1. Go to C:\Users\<username>\AppData\Local\Autodesk\Web Services.
2. Rename Web Services folder to "old_Web Services."
3. Go to C:\Users\<username>\AppData\Roaming\Autodesk\.
4. If exists, rename the ADUT folder to ADUT_old.
   • Note: A new ADUT folder will be created automatically during next license activation.
5. If using Flex Licensing, navigate to C:\ProgramData\FLEXnet and delete all files that start with adskflex.

1. Start the Autodesk software and sign in with Autodesk account credentials when asked.

Confirm the correct time zone

1. Right-click in the Windows logo. 
2. Select "Settings," then "Time & Language". 
3. If not already done, enable the following:
   • Set time automatically.
   • Set time zone automatically.
   • Adjust for daylight saving time automatically.

Verify that the licensing service is turned on

1. Click Start (or right-click Start on Windows 10) and choose Run.
2. In the Run box, type cromwellpsi.com
3. In the Name column of the Services console window, find FlexNet Licensing 64 and check that the status states "Running."
   1. If FlexNet Licensing 64 is running, if it is then this solution does not apply.
   2. If FlexNet Licensing 64 is not running, go to step 4.
4. In the Startup Type column, right-click "Manual and select Properties."
   • Note: if the service cannot be started, make sure the Windows profile being used has enough permissions. Consult with an IT administrator if necessary.)
5. In the Properties dialog, use the drop-down menu to change Startup Type to "Automatic" and click OK.
6. To change the status of FlexNet to Running, right-click its empty Status cell and select "Start" from the context menu.

Perform a clean uninstall

For Windows, use the following guide:  Perform a Clean Uninstall - Download & Install.

For Mac, use the following process:

1. Close any running Autodesk programs (make sure A is closed from the notification area on the right side of the Finder bar).
2. Go to /User/<username>/Library/Application Support/Autodesk/ADUT and remove all files.
3. Go to /User/<username>/Library/Application Support/Autodesk/Web Services and delete the file "cromwellpsi.com"
4. Go to /Library/Application Support/Autodesk/CLM and delete the folder "LGS."
5. Go to /Library/Preferences/FLEXnet Publisher folder and delete all files that start with "adsk."

Note: Do not remove any non-Autodesk files. Other products, such as Adobe, might use this licensing technology.

1. Launch the Autodesk product to initiate the re-activation.

Refresh the Date & Time

1. Choose Apple menu > System Preferences, then click Date & Time.
2. Click Date & Time, then set the date and time automatically or manually:
   • Automatically: Select “Set date and time automatically,” then choose the network time server for the region.
   • Manually: Deselect “Set date and time automatically,” click today’s date on the calendar, drag the clock’s hands to show the correct time (or enter the time), then click Save.
3. Click Time Zone, then set the time zone automatically or manually.
   • Automatically: Select “Set time zone automatically using current location.”

• Manually: Deselect “Set time zone automatically using current location,” click a location on the map, click the Closest City pop-up menu, then choose the city closest to the location.

Assign the end user to the license

Confirm that the license has been assigned to the end user. Use the following guide to do so:  Assign Products and Services to Users in the New View - Account Management