Creating and modifying processing pipelines
When you create a processing pipeline, you can add any combination of stages and conditional statements to it.
You can link multiple pipelines together by adding them to the same workflow pipeline.
- HCI_id
- HCI_URI
- HCI_dataSourceUuid
- HCI_displayName
- HCI_doc_version
Creating a new pipeline
To create a processing pipeline, you first create the processing pipeline and then add stages to it.
When using the command line or REST API, you can both create a processing pipeline and add stages to it in a single step.
Procedure
Click the Processing Pipelines window.
Click Create Pipeline.
Enter a name and, optionally, a description for the pipeline.
Click Create.
Results
Adding stages to a pipeline
You can add any number of stages to a pipeline. For best practices on arranging the built-in stages, see Best practices for configuring stages and pipelines.
Procedure
Click the Processing Pipelines window.
Select the processing pipeline that you want.
Click Add Stages.
Browse or search for the stage you want:
To browse, click the category tabs and view the stages within each:
- Analyze: Includes stages that identify certain characteristics about your data.
- Extract: Includes stages that examine your documents and obtain fields from them.
- Transform: Includes stages that make changes to documents and their fields.
- Filter: Includes stages that remove documents or fields from the pipeline.
- Enrich: Includes stages that add fields to documents.
- Store: Includes stages that store documents in locations outside of your system.
- Other: Includes stages that do not fit in any of the other categories.
Do one of these:
- Select a stage and drag it to the bottom of the page.
- Mouse over a stage and click the add icon (
).
Configure the stage.
For information on the configuration settings for each built-in stage, see Built-in stages.
Click Update. The stage is added to the pipeline.
To reorder stages, click and drag them to new positions in the pipeline.
Configuring stages
For information on the settings that you can configure for the built-in stages, see the individual topics under Built-in stages.
If you've written custom stage plugins, your plugin determines what settings you can configure for the stage.
Procedure
Click the Processing Pipelines window.
Select the processing pipeline that you want.
Click the edit icon (
) for the stage that you want to configure.Configure the stage.
Adding conditional statements to a pipeline
You can surround any number of stages with conditional statements. This lets you filter the documents that can pass through the conditional statement to the stages it encloses.
You can nest conditional statements within other conditional statements to form more complex conditions. You can nest conditional statements up to four levels deep.
For information on conditional statements and how you can configure them, see Conditional statements.
Procedure
Click the Processing Pipelines window.
Select the pipeline you want.
Do one of the following:
- Click the conditional icon (
) for a stage in the pipeline.The stage is surrounded by a conditional statement.
- Click Add Stages > Add Conditional.
A conditional statement is added to the end of the pipeline. You can click the conditional statement and drag it to where you want it in the pipeline.
To add stages within the conditional statement, click a stage and drag it into the conditional statement.
- Click the conditional icon (
To add another conditional statement within a conditional statement, click on the conditional icon (
) for a conditional statement.
Adding actions to a pipeline
You can use the Execute Action stage to perform component actions in a pipeline. For example:
- Use the HCP data connection's Write File action to write documents to an HCP system.
- Use an Index action to index documents at any point within a pipeline. For an example, see Use the Index action in your pipeline to test the effects of pipeline stages on your index.
Instead, you need to delete the stage and add a new one for the action you want.
Procedure
Click the Processing Pipelines window.
Select the pipeline that you want.
Click Add Stages > Action.
Drag the Execute Action stage into the pipeline.
On the Configure Action page:
Select a component.
Select an action.
Specify any configuration settings required for the action you selected.
The configuration settings for data connection actions differ depending on the data connection. For information on these settings, see the topic for the applicable data connection under Data connection types and settings.
Click Update.
Removing stages from a pipeline
Click the Processing Pipelines window.
Click the pipeline that you want.
Click the remove icon (
) for the stage you want to remove.