Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Creating and modifying processing pipelines

When you create a processing pipeline, you can add any combination of stages and conditional statements to it.

You can link multiple pipelines together by adding them to the same workflow pipeline.

Important Hitachi Content Intelligence requires that every document it processes include these single-valued fields:
  • HCI_id
  • HCI_URI
  • HCI_dataSourceUuid
  • HCI_displayName
  • HCI_doc_version
These fields are added automatically to all documents.
NoteDo not configure your pipeline stages to edit either the names or values for these fields.

Creating a new pipeline

To create a processing pipeline, you first create the processing pipeline and then add stages to it.

When using the command line or REST API, you can both create a processing pipeline and add stages to it in a single step.

Procedure

  1. Click the Processing Pipelines window.

  2. Click Create Pipeline.

  3. Enter a name and, optionally, a description for the pipeline.

  4. Click Create.

Results

The new pipeline appears.

Adding stages to a pipeline

You can add any number of stages to a pipeline. For best practices on arranging the built-in stages, see Best practices for configuring stages and pipelines.

Procedure

  1. Click the Processing Pipelines window.

  2. Select the processing pipeline that you want.

  3. Click Add Stages.

  4. Browse or search for the stage you want:

    To browse, click the category tabs and view the stages within each:

    • Analyze: Includes stages that identify certain characteristics about your data.
    • Extract: Includes stages that examine your documents and obtain fields from them.
    • Transform: Includes stages that make changes to documents and their fields.
    • Filter: Includes stages that remove documents or fields from the pipeline.
    • Enrich: Includes stages that add fields to documents.
    • Store: Includes stages that store documents in locations outside of your system.
    • Other: Includes stages that do not fit in any of the other categories.
  5. Do one of these:

    • Select a stage and drag it to the bottom of the page.
    • Mouse over a stage and click the add icon (GUID-B08F264D-C430-4C0F-B9E8-847F59551FE6-low.png).
  6. Configure the stage.

    For information on the configuration settings for each built-in stage, see Built-in stages.

  7. Click Update. The stage is added to the pipeline.

  8. To reorder stages, click and drag them to new positions in the pipeline.

Configuring stages

For information on the settings that you can configure for the built-in stages, see the individual topics under Built-in stages.

If you've written custom stage plugins, your plugin determines what settings you can configure for the stage.

Procedure

  1. Click the Processing Pipelines window.

  2. Select the processing pipeline that you want.

  3. Click the edit icon (GUID-2548EA14-5968-4706-89BC-165684BD24D9-low.png) for the stage that you want to configure.

  4. Configure the stage.

Adding conditional statements to a pipeline

You can surround any number of stages with conditional statements. This lets you filter the documents that can pass through the conditional statement to the stages it encloses.

You can nest conditional statements within other conditional statements to form more complex conditions. You can nest conditional statements up to four levels deep.

For information on conditional statements and how you can configure them, see Conditional statements.

Procedure

  1. Click the Processing Pipelines window.

  2. Select the pipeline you want.

  3. Do one of the following:

    • Click the conditional icon (GUID-FD84B3E8-9E70-4A09-9F05-8A7C63E82DC6-low.png) for a stage in the pipeline.

      The stage is surrounded by a conditional statement.

    • Click Add Stages > Add Conditional.

      A conditional statement is added to the end of the pipeline. You can click the conditional statement and drag it to where you want it in the pipeline.

      To add stages within the conditional statement, click a stage and drag it into the conditional statement.

  4. To add another conditional statement within a conditional statement, click on the conditional icon (GUID-FD84B3E8-9E70-4A09-9F05-8A7C63E82DC6-low.png) for a conditional statement.

Adding actions to a pipeline

You can use the Execute Action stage to perform component actions in a pipeline. For example:

Note After you add an Execute Action stage to a pipeline, you cannot change the type of action it performs.

Instead, you need to delete the stage and add a new one for the action you want.

Procedure

  1. Click the Processing Pipelines window.

  2. Select the pipeline that you want.

  3. Click Add Stages > Action.

  4. Drag the Execute Action stage into the pipeline.

  5. On the Configure Action page:

    1. Select a component.

    2. Select an action.

    3. Specify any configuration settings required for the action you selected.

      The configuration settings for data connection actions differ depending on the data connection. For information on these settings, see the topic for the applicable data connection under Data connection types and settings.

  6. Click Update.

Removing stages from a pipeline

  1. Click the Processing Pipelines window.

  2. Click the pipeline that you want.

  3. Click the remove icon (GUID-B250EA11-E38E-4687-8084-F4FB2448E4AB-low.png) for the stage you want to remove.

 

  • Was this article helpful?