Managing jobs
This section describes how you can reconfigure, move, schedule, and otherwise manage the jobs that can run on your system.
Configuring where jobs run
The Hitachi Content Intelligence system lets you limit the system instances that jobs are allowed to run on. You can specify lists of allowed instances for all jobs, for jobs of a specific type, or for each individual job.
For example, if your system includes instances numbered 101 through 104, you could specify that:
- Only instances 101, 102, and 103 can run jobs
- Only instances 101 and 102 can run Example-type jobs
- Only instance 101 can run the Example-type job named Example1
Procedure
Click Jobs.
Select the job you want and click Configure.
On the Scale tab, specify which instances can run jobs:
From the Available Instances list, select the instances where you want jobs to run and then click Add Instances.
Optionally, enable the option Add newly selected instances to existing jobs.
With this option enabled, any new instances that you selected in the previous step will automatically be added as selected instances to all existing jobs. When disabled, existing jobs of this type keep their current lists of selected instances.
From the Selected Instances list, select any instances where you don't want jobs to run an then click Remove Instances.
Note- If you remove instances where jobs are already running, any existing jobs remain running on those instances, but new jobs will not run on the removed instances.
- You cannot remove all selected instances from the All Jobs menu. If you don't want jobs to run anywhere, you need to remove the selected instances for each job type
Click Update.
Specify which instances can run jobs of a particular type:
Select a job type and then click Configure.
On the Scale tab, from the Eligible Instances list, select the instances where you want this type of job to run and then click Add Instances.
Optionally, enable the option Add newly selected instances to existing jobs.
With this option enabled, any new instances that you selected in the previous step will automatically be added as selected instances to all existing jobs of this type. When disabled, existing jobs of this type keep their current lists of selected instances.From the Selected Instances list, select any instances where you don't want this type of job to run and then click Remove Instances.
NoteIf you remove instances where any jobs of this type are already running, the existing jobs remain running on those instances, but new jobs of this type will not run on the removed instances.Click Update.
Repeat this step for any other job types you want to configure.
Specify which instances can run a particular job
On the Scale tab, select the instances you want from the Available Instances list and then click Add Instances. Here, the Available Instances list displays all instances where jobs of this type are allowed to run.
Click Update.
Repeat this step for any other jobs you want to configure.
Related CLI commands
editJob
editJobSettings
editJobType
Related REST API methods
PUT /jobs/{uuid}
PUT /jobs/settings
PUT /jobs/types/{name}
You can get help on specific REST API methods for the Admin App at REST API - Admin.
Configuring job settings
You can configure these Docker container settings for all job types and individual jobs:
- Container Memory: The hard memory limit for a job's Docker container, in megabytes (MB).
- CPU: The relative CPU usage weight for a job's Docker container. Generally, a higher value means that the container receives more CPU resources than other processes (including other service Docker containers) running on the instance.
Valid values are decimal numbers.
NoteFor Workflow-Agent jobs, avoid making changes to the settings. Instead, use the workflow's Task Settings page to edit the comparable settings for the workflow task.For the Workflow-Agent type and individual Workflow-Agent jobs, you can also configure the ports for the job to communicate on:
- Driver Debug Port: The port to use for Java debugging of the job driver. Only accessible if
debugwas set to true in thecluster.configfile during system installation. - Executor Debug Port: The port to use for Java debugging of the job executor. Only used if
debugwas set to true in thecluster.configfile during system installation. - Default UI Port: The default port used for the spark UI. Spark will automatically pick another port if this one is in use.
When you configure settings for a job type, those settings are inherited by created jobs, not by the jobs that exist at the time you make the configuration change.
Procedure
Click Jobs.
Select a job type
On the Configuration tab, configure the settings you want.
Click Update.
Scheduling jobs
You can configure a schedule for both job types and individual jobs.
When you configure a schedule for a job type, the schedule is inherited by newly created jobs, not by the jobs that exist at the time you make the configuration change.
When you configure a schedule for a Workflow-Agent job, identical changes are made to the schedule for the corresponding workflow task.
If a job has a schedule, the job enters the Running state automatically when a scheduled time period begins, regardless of the job's current state. A job can also be started by a service outside of the job's scheduled time periods.
When the end of a scheduled time period arrives:
- If the job is still running, the job stops and enters the Idle state.
- If the job is not running (for example, it's in the Completed or Failed state), the job remains in its current state.
Procedure
Click Jobs.
Select a job type or individual job.
On the Schedule tab, use the calendar tool to specify when the job or job type should run.
In this tool, click a day to add a block of time. Click and drag the block to cover the hours you want the job or jobs to run.
To remove a block, right-click it.
Click Update.