Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Built-in pipelines

Hitachi Content Intelligence includes a number of built-in pipelines that you can use in production or as examples for the pipelines you build yourself.

These built-in pipelines cannot be deleted or edited, but you can clone them.

Default pipeline

You can use the built-in default pipeline:

  • As a basis for creating other, less general processing pipelines.
  • As the first pipeline in your workflow pipeline.
  • As a test pipeline for quickly evaluating your data.
NoteThe default pipeline is a combination of the Basic Preprocessing pipeline and Basic Extraction pipeline, so you shouldn't use it in a workflow that uses either of those other pipelines.

This illustration describes the stages in this pipeline and how the pipeline functions when run as part of a workflow task.

GUID-E87C164F-5B29-4995-9A5B-26B49D8BC951-low.pngGUID-51323F09-FCDC-4917-AB2D-D51A7092E5ED-low.pngGUID-F638084F-FA6B-4CAC-AB1B-69DCFC6897CB-low.pngGUID-B49DA8A5-3D86-4BA0-B9DD-FC6FA7399F32-low.png

Basic Preprocessing pipeline

This built-in pipeline discovers the MIME types for documents and extracts documents from archive files. Typically, you use this pipeline as the first in your workflow.

Note
  • The Basic Preprocessing pipeline is a subset of the Default pipeline, so you shouldn't use it in a workflow that uses the Default pipeline.
  • When using both this and the Basic Extraction pipelines, place this pipeline before that one.
  • When using this pipeline, consider setting its execution mode to Preprocessing. For more information, see Pipeline execution modes.

This illustration describes the stages in this pipeline and how the pipeline functions when run as part of a workflow task.

GUID-638F7224-0B87-46D0-9F0F-6A41673F0680-low.pngGUID-1BA07050-72F5-486B-B942-D668F4C31AA1-low.pngGUID-5B144B19-73EF-40EC-B90C-383F44CBA007-low.png

Basic Extraction pipeline

This built-in pipeline gives general purpose metadata and text snippet extraction.

Note
  • Because stages in this pipeline needs the Content_Type field to exist in all documents, you should precede this pipeline with a pipeline that contains the MIME Type Detection stage (for example, the Basic Preprocessing built-in pipeline).
  • When using both this and the Basic Preprocessing pipelines, place this pipeline after that one.
  • The Basic Extraction pipeline is a subset of the default pipeline, so you shouldn't use it in a workflow that uses the Default pipeline.

This illustration describes the stages in this pipeline and how the pipeline functions when run as part of a workflow task.

GUID-684F5EC4-70F4-4489-AF64-93A713A51AA4-low.png

 

  • Was this article helpful?