wehub-resource-sync
45 lines
1.7 KiB
Go
45 lines
1.7 KiB
Go
/*
|
|
* Copyright 2024 CloudWeGo Authors
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
* you may not use this file except in compliance with the License.
|
|
* You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software
|
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
* See the License for the specific language governing permissions and
|
|
* limitations under the License.
|
|
*/
|
|
|
|
// Package embedding defines the Embedder component interface for converting
|
|
// text into vector representations.
|
|
//
|
|
// # Overview
|
|
//
|
|
// An Embedder converts a batch of strings into dense float vectors. Semantically
|
|
// similar texts produce vectors that are close in the vector space, making
|
|
// embeddings the backbone of semantic search, RAG pipelines, and clustering.
|
|
//
|
|
// Concrete implementations (OpenAI, Ark, Ollama, …) live in eino-ext:
|
|
//
|
|
// github.com/cloudwego/eino-ext/components/embedding/
|
|
//
|
|
// # Output Format
|
|
//
|
|
// [Embedder.EmbedStrings] returns `[][]float64` where:
|
|
// - outer index corresponds to the input text at the same position
|
|
// - inner slice is the embedding vector; its length (dimensions) is fixed by
|
|
// the model and is the same for every text
|
|
//
|
|
// # Consistency Requirement
|
|
//
|
|
// The same model must be used for both indexing and retrieval. Mixing models
|
|
// produces vectors in different spaces — similarity scores become meaningless
|
|
// and semantic search breaks silently.
|
|
//
|
|
// See https://www.cloudwego.io/docs/eino/core_modules/components/embedding_guide/
|
|
package embedding
|