Semantic index configuration for indexed sources
Summarize
Summary of Semantic index configuration for indexed sources
The AI Search generalized RAG (Retrieval-Augmented Generation) framework in ServiceNow provides a simplified approach to configuring semantic indexing for records sourced from ServiceNow AI Platform® tables. Semantic index configurations define how AI Search processes and indexes content from these sources to enable semantic vector search, which improves search relevancy by understanding the meaning behind search terms.
Show less
Key Features
- Indexed Source: References an existing indexed source containing field values or attachments to be indexed for semantic vector search.
- Embedding Models: One or more models used to encode indexed content into vector representations, allowing AI Search to find semantically similar results based on meaning rather than exact terms.
- Chunking Strategy: Defines how text is broken into manageable chunks during indexing. Chunking enhances performance and relevancy by enabling the system to process smaller, contextually coherent pieces of content.
- Semantic Index Fields: Specifies which fields or attachments from the indexed source are included in semantic indexing and the order in which they are evaluated, allowing fine-tuned control over indexing behavior.
Activation and Configuration
Semantic index configuration is enabled through the AI Search Semantic Controller plugin (com.glide.ais.semanticsearch), which activates automatically with the installation of any Now Assist application. To verify activation, navigate to All > AI Search > AI Search Index > Indexed Sources and check for the Semantic Index Configuration related list on the indexed source record.
Administrators can configure semantic indexing settings per indexed source to tailor how AI Search indexes content for semantic vector search, balancing indexing performance and search result quality. Although multiple semantic index configurations can be defined for a single indexed source, additional configurations increase indexing resource usage.
The AI Search generalized RAG (Retrieval-Augmented Generation) framework offers a streamlined way to configure semantic indexing settings for records indexed from ServiceNow AI Platform® tables.
Semantic index configuration overview
AI Search admins can configure semantic indexing settings for an indexed source. These settings specify how AI Search indexes content from the indexed source for use with semantic vector search. The group of semantic indexing settings for a particular indexed source is called a semantic index configuration.
- Indexed source
-
A reference to an existing indexed source with field values or attachments that you want indexed for semantic vector search.
For more information on indexed sources, see Indexed sources in AI Search. To learn more about semantic vector search, see Semantic vector search in AI Search.
- Embedding models
-
A list of one or more embedding models for the system to use when indexing content from the indexed source for semantic vector search.
An embedding model specifies how information found in your indexed content is encoded in a vector map. Semantic vector search uses the encoded information from the vector map to find search results that have meanings similar to those of your search terms.
- Chunking strategy and related parameters
-
A chunking strategy and related parameter values that together determine how content from the indexed source's selected fields and attachments is handled during indexing for semantic vector search.
Chunking is the process of breaking text down into smaller portions (called chunks) during indexing. By chunking your content, AI Search reduces search load and improves context and relevancy for semantic vector matches.
The following image shows how a two-paragraph block of sample field value text might be broken into chunks for semantic indexing. As shown, chunks can contain multiple sentences and may span paragraph breaks found in the original text.
- Semantic index fields
-
References to one or more semantic index fields that provide semantic indexing settings for content from the indexed source.
Each semantic index field defines semantic indexing settings for a single field from the indexed source table, or for attachments from that table. You can specify the order in which semantic index fields are evaluated when indexing content from the indexed source for semantic vector search.
Activating semantic index configuration
Semantic index configuration functionality is provided by the AI Search Semantic Controller plugin (com.glide.ais.semantic_search). This plugin is automatically activated for your instance when you install any Now Assist application.
You can verify whether semantic index configuration is activated by navigating to and selecting an indexed source record. If you see the Semantic Index Configuration related list on the Indexed Source form, the plugin is activated.