mirror of
https://github.com/langgenius/dify-docs.git
synced 2026-03-31 14:08:39 +07:00
385 lines
13 KiB
Plaintext
385 lines
13 KiB
Plaintext
---
|
|
title: Model Schema
|
|
---
|
|
|
|
|
|
|
|
{/*
|
|
CONTRIBUTOR NOTE:
|
|
----------------
|
|
This is a legacy document that is being deprecated.
|
|
Please DO NOT make changes to this version.
|
|
All updates should be directed to the new version at:
|
|
/plugin-dev-en/0412-model-schema
|
|
*/}
|
|
|
|
<Card title="This Documentation is Being Deprecated" icon="circle-exclamation" href="/plugin-dev-en/0412-model-schema">
|
|
<p>This page is being phased out as part of our documentation reorganization.</p>
|
|
|
|
<p><strong>Click this card</strong> to be redirected to the updated version with the most current information.</p>
|
|
|
|
<p>If you notice any discrepancies or areas needing improvement in the new documentation, please use the "Report an issue" button at the bottom of the page.</p>
|
|
</Card>
|
|
|
|
The endpoint methods and parameter descriptions that need to be implemented by the supplier and each model type are described here.
|
|
|
|
### Model Providers
|
|
|
|
Inherit from the `__base.model_provider.ModelProvider` base class, implement the following endpoint:
|
|
|
|
#### Provider Credentials Validation
|
|
|
|
```python
|
|
def validate_provider_credentials(self, credentials: dict) -> None:
|
|
"""
|
|
Validate provider credentials
|
|
You can choose any validate_credentials method of model type or implement validate method by yourself,
|
|
such as: get model list api
|
|
|
|
if validate failed, raise exception
|
|
|
|
:param credentials: provider credentials, credentials form defined in `provider_credential_schema`.
|
|
"""
|
|
```
|
|
|
|
Credentials (object): Credential information Credential parameters are defined by the provider's YAML configuration file's `provider_credential_schema`, such as passing in `api_key`. If validation fails, throw the `errors.validate.CredentialsValidateFailedError` error.
|
|
|
|
Note: Predefined models must fully implement this interface, while custom model providers can implement it simply as follows:
|
|
|
|
```python
|
|
class XinferenceProvider(Provider):
|
|
def validate_provider_credentials(self, credentials: dict) -> None:
|
|
pass
|
|
```
|
|
|
|
### Models
|
|
|
|
Models are divided into 5 different model types, each inheriting from different base classes and requiring implementation of different methods.
|
|
|
|
#### Common Interfaces
|
|
|
|
All models must uniformly implement the following 2 methods:
|
|
|
|
**Model Credential Validation**
|
|
|
|
Similar to provider credential validation, this is specifically for validating individual models.
|
|
|
|
```python
|
|
def validate_credentials(self, model: str, credentials: dict) -> None:
|
|
"""
|
|
Validate model credentials
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:return:
|
|
"""
|
|
```
|
|
|
|
Parameters:
|
|
|
|
* `model` (string): Model name
|
|
* `credentials` (object): Credential information Credential parameters are defined by the provider's YAML configuration file's `provider_credential_schema` or `model_credential_schema`, such as passing in `api_key`. If validation fails, throw the `errors.validate.CredentialsValidateFailedError` error.
|
|
|
|
**Invocation Exception Error Mapping**
|
|
|
|
When a model invocation encounters an exception, it needs to be mapped to the Runtime-specified InvokeError type to help Dify handle different errors accordingly.
|
|
|
|
Runtime Errors:
|
|
|
|
* `InvokeConnectionError`: Invocation connection error
|
|
* `InvokeServerUnavailableError`: Invocation service unavailable
|
|
* `InvokeRateLimitError`: Invocation rate limit reached
|
|
* `InvokeAuthorizationError`: Invocation authentication failed
|
|
* `InvokeBadRequestError`: Incorrect invocation parameters
|
|
|
|
```python
|
|
@property
|
|
def _invoke_error_mapping(self) -> dict[type[InvokeError], list[type[Exception]]]:
|
|
"""
|
|
Map model invoke error to unified error
|
|
The key is the error type thrown to the caller
|
|
The value is the error type thrown by the model,
|
|
which needs to be converted into a unified error type for the caller.
|
|
|
|
:return: Invoke error mapping
|
|
"""
|
|
```
|
|
|
|
You can also directly throw corresponding Errors and define them so that in subsequent calls, you can directly throw `InvokeConnectionError` and other exceptions.
|
|
|
|
### Large Language Model (LLM)
|
|
|
|
Inherit from `__base.large_language_model.LargeLanguageModel` base class, implement the following interfaces:
|
|
|
|
#### LLM Invocation
|
|
|
|
Implement the core method for LLM invocation, supporting both streaming and synchronous returns.
|
|
|
|
```python
|
|
def _invoke(self, model: str, credentials: dict,
|
|
prompt_messages: list[PromptMessage], model_parameters: dict,
|
|
tools: Optional[list[PromptMessageTool]] = None, stop: Optional[list[str]] = None,
|
|
stream: bool = True, user: Optional[str] = None) \
|
|
-> Union[LLMResult, Generator]:
|
|
"""
|
|
Invoke large language model
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:param prompt_messages: prompt messages
|
|
:param model_parameters: model parameters
|
|
:param tools: tools for tool calling
|
|
:param stop: stop words
|
|
:param stream: is stream response
|
|
:param user: unique user id
|
|
:return: full response or stream response chunk generator result
|
|
"""
|
|
```
|
|
|
|
Parameters:
|
|
|
|
* `model` (string): Model name
|
|
* `credentials` (object): Credential information Credential parameters are defined by the provider's YAML configuration file's `provider_credential_schema` or `model_credential_schema`, such as passing in `api_key`
|
|
* `prompt_messages` (array\[PromptMessage]): Prompt list
|
|
* For Completion-type models, only one UserPromptMessage element needs to be passed
|
|
* For Chat-type models, a list of SystemPromptMessage, UserPromptMessage, AssistantPromptMessage, ToolPromptMessage elements needs to be passed according to message type
|
|
* `model_parameters` (object): Model parameters defined by the model's YAML configuration's `parameter_rules`
|
|
* `tools` (array\[PromptMessageTool]) \[optional]: Tool list, equivalent to function calling functions
|
|
* `stop` (array\[string]) \[optional]: Stop sequences. Model output will stop before the defined string
|
|
* `stream` (bool): Whether to stream output, default True. Streaming returns Generator\[LLMResultChunk], non-streaming returns LLMResult
|
|
* `user` (string) \[optional]: Unique user identifier to help providers monitor and detect abuse
|
|
|
|
Return:
|
|
|
|
* Streaming returns Generator\[LLMResultChunk]
|
|
* Non-streaming returns LLMResult
|
|
|
|
#### Pre-calculate Input Tokens
|
|
|
|
If the model does not provide a pre-calculate tokens interface, directly return 0.
|
|
|
|
```python
|
|
def get_num_tokens(self, model: str, credentials: dict, prompt_messages: list[PromptMessage],
|
|
tools: Optional[list[PromptMessageTool]] = None) -> int:
|
|
"""
|
|
Get number of tokens for given prompt messages
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:param prompt_messages: prompt messages
|
|
:param tools: tools for tool calling
|
|
:return:
|
|
"""
|
|
```
|
|
|
|
#### Optional: Get Custom Model Rules
|
|
|
|
```python
|
|
def get_customizable_model_schema(self, model: str, credentials: dict) -> Optional[AIModelEntity]:
|
|
"""
|
|
Get customizable model schema
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:return: model schema
|
|
"""
|
|
```
|
|
|
|
When the vendor supports adding custom LLMs, this method can be implemented to make the model rules available to the custom model, returning None by default.
|
|
|
|
For most of the fine-tuned models under OpenAI vendor, you can get their base model by their fine-tuned model name, such as gpt-3.5-turbo-1106, and then return the predefined parameter rules of the base model, refer to the implementation of [OpenAI](https://github.com/langgenius/dify-official-plugins/tree/main/models/openai).
|
|
|
|
### Text Embedding
|
|
|
|
Inherit from `__base.text_embedding_model.TextEmbeddingModel` base class, implement the following interfaces:
|
|
|
|
#### Embedding Invocation
|
|
|
|
```python
|
|
def _invoke(self, model: str, credentials: dict,
|
|
texts: list[str], user: Optional[str] = None) \
|
|
-> TextEmbeddingResult:
|
|
"""
|
|
Invoke large language model
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:param texts: texts to embed
|
|
:param user: unique user id
|
|
:return: embeddings result
|
|
"""
|
|
```
|
|
|
|
Parameters:
|
|
|
|
* `model` (string): Model name
|
|
* `credentials` (object): Credential information Credential parameters are defined by the provider's YAML configuration file's `provider_credential_schema` or `model_credential_schema`
|
|
* `texts` (array\[string]): Text list, can be processed in batch
|
|
* `user` (string) \[optional]: Unique user identifier to help providers monitor and detect abuse
|
|
|
|
Return:
|
|
|
|
* TextEmbeddingResult entity
|
|
|
|
#### Pre-calculate Tokens
|
|
|
|
```python
|
|
def get_num_tokens(self, model: str, credentials: dict, texts: list[str]) -> int:
|
|
"""
|
|
Get number of tokens for given prompt messages
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:param texts: texts to embed
|
|
:return:
|
|
"""
|
|
```
|
|
|
|
Similar to LargeLanguageModel, this interface needs to select an appropriate tokenizer based on the model. If the model does not provide a tokenizer, it can use the `_get_num_tokens_by_gpt2(text: str)` method in the AIModel base class.
|
|
|
|
### Rerank
|
|
|
|
Inherit from `__base.rerank_model.RerankModel` base class, implement the following interfaces:
|
|
|
|
#### Rerank Invocation
|
|
|
|
```python
|
|
def _invoke(self, model: str, credentials: dict,
|
|
query: str, docs: list[str], score_threshold: Optional[float] = None, top_n: Optional[int] = None,
|
|
user: Optional[str] = None) \
|
|
-> RerankResult:
|
|
"""
|
|
Invoke rerank model
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:param query: search query
|
|
:param docs: docs for reranking
|
|
:param score_threshold: score threshold
|
|
:param top_n: top n
|
|
:param user: unique user id
|
|
:return: rerank result
|
|
"""
|
|
```
|
|
|
|
Parameters:
|
|
|
|
* `model` (string): Model name
|
|
* `credentials` (object): Credential information
|
|
* `query` (string): Search query content
|
|
* `docs` (array\[string]): List of segments to be re-ranked
|
|
* `score_threshold` (float) \[optional]: Score threshold
|
|
* `top_n` (int) \[optional]: Take top n segments
|
|
* `user` (string) \[optional]: Unique user identifier to help providers monitor and detect abuse
|
|
|
|
Return:
|
|
|
|
* RerankResult entity
|
|
|
|
### Speech2Text
|
|
|
|
Inherit from `__base.speech2text_model.Speech2TextModel` base class, implement the following interfaces:
|
|
|
|
#### Invoke Invocation
|
|
|
|
```python
|
|
def _invoke(self, model: str, credentials: dict,
|
|
file: IO[bytes], user: Optional[str] = None) \
|
|
-> str:
|
|
"""
|
|
Invoke large language model
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:param file: audio file
|
|
:param user: unique user id
|
|
:return: text for given audio file
|
|
"""
|
|
```
|
|
|
|
Parameters:
|
|
|
|
* `model` (string): Model name
|
|
* `credentials` (object): Credential information
|
|
* `file` (File): File stream
|
|
* `user` (string) \[optional]: Unique user identifier to help providers monitor and detect abuse
|
|
|
|
Return:
|
|
|
|
* Converted text string from speech
|
|
|
|
### Text2Speech
|
|
|
|
Inherit from `__base.text2speech_model.Text2SpeechModel` base class, implement the following interfaces:
|
|
|
|
#### Invoke Invocation
|
|
|
|
```python
|
|
def _invoke(self, model: str, credentials: dict, content_text: str, streaming: bool, user: Optional[str] = None):
|
|
"""
|
|
Invoke large language model
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:param content_text: text content to be translated
|
|
:param streaming: output is streaming
|
|
:param user: unique user id
|
|
:return: translated audio file
|
|
"""
|
|
```
|
|
|
|
Parameters:
|
|
|
|
* `model` (string): Model name
|
|
* `credentials` (object): Credential information
|
|
* `content_text` (string): Text content to be converted
|
|
* `streaming` (bool): Whether to stream output
|
|
* `user` (string) \[optional]: Unique user identifier to help providers monitor and detect abuse
|
|
|
|
Return:
|
|
|
|
* Audio stream converted from text
|
|
|
|
### Moderation
|
|
|
|
Inherit from `__base.moderation_model.ModerationModel` base class, implement the following interfaces:
|
|
|
|
#### Invoke Invocation
|
|
|
|
```python
|
|
def _invoke(self, model: str, credentials: dict,
|
|
text: str, user: Optional[str] = None) \
|
|
-> bool:
|
|
"""
|
|
Invoke large language model
|
|
|
|
:param model: model name
|
|
:param credentials: model credentials
|
|
:param text: text to moderate
|
|
:param user: unique user id
|
|
:return: false if text is safe, true otherwise
|
|
"""
|
|
```
|
|
|
|
Parameters:
|
|
|
|
* `model` (string): Model name
|
|
* `credentials` (object): Credential information
|
|
* `text` (string): Text content
|
|
* `user` (string) \[optional]: Unique user identifier to help providers monitor and detect abuse
|
|
|
|
Return:
|
|
|
|
* False indicates the input text is safe, True indicates otherwise
|
|
|
|
{/*
|
|
Contributing Section
|
|
DO NOT edit this section!
|
|
It will be automatically generated by the script.
|
|
*/}
|
|
|
|
---
|
|
|
|
[Edit this page](https://github.com/langgenius/dify-docs/edit/main/en/plugins/schema-definition/model/model-schema.mdx) | [Report an issue](https://github.com/langgenius/dify-docs/issues/new?template=docs.yml)
|
|
|