Reverse Invocation Model

A plugin can reverse invoke Dify’s internal LLM capabilities, including all model types and functions within the platform, such as TTS and Rerank. If you are unfamiliar with the basics of reverse invocation, first read Reverse Invocation of Dify Services.

Every model invocation takes a ModelConfig type parameter. Its structure is defined in the General Specifications Definition and varies slightly by model type.

For example, LLM type models also require completion_params and mode parameters. You can construct this structure manually or use model-selector type parameters or configurations.

Invoke LLM#

Entry Point#

    self.session.model.llm

Interface#

    def invoke(        self,        model_config: LLMModelConfig,        prompt_messages: list[PromptMessage],        tools: list[PromptMessageTool] | None = None,        stop: list[str] | None = None,        stream: bool = True,    ) -> Generator[LLMResultChunk, None, None] | LLMResult:        pass

If the model you are invoking does not have tool_call capability, the tools passed here will not take effect.

Use Case#

This example invokes OpenAI’s gpt-4o-mini model within a Tool:

from collections.abc import Generatorfrom typing import Anyfrom dify_plugin import Toolfrom dify_plugin.entities.model.llm import LLMModelConfigfrom dify_plugin.entities.tool import ToolInvokeMessagefrom dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessageclass LLMTool(Tool):    def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]:        response = self.session.model.llm.invoke(            model_config=LLMModelConfig(                provider='openai',                model='gpt-4o-mini',                mode='chat',                completion_params={}            ),            prompt_messages=[                SystemPromptMessage(                    content='you are a helpful assistant'                ),                UserPromptMessage(                    content=tool_parameters.get('query')                )            ],            stream=True        )        for chunk in response:            if chunk.delta.message:                assert isinstance(chunk.delta.message.content, str)                yield self.create_text_message(text=chunk.delta.message.content)

Note that the code passes the query parameter from tool_parameters.

Best Practice#

Avoid constructing LLMModelConfig manually. Instead, let users select the model they want in the UI by adding a model parameter to the tool’s parameter list:

identity:  name: llm  author: Dify  label:    en_US: LLM    zh_Hans: LLM    pt_BR: LLMdescription:  human:    en_US: A tool for invoking a large language model    zh_Hans: 用于调用大型语言模型的工具    pt_BR: A tool for invoking a large language model  llm: A tool for invoking a large language modelparameters:  - name: prompt    type: string    required: true    label:      en_US: Prompt string      zh_Hans: 提示字符串      pt_BR: Prompt string    human_description:      en_US: used for searching      zh_Hans: 用于搜索网页内容      pt_BR: used for searching    llm_description: key words for searching    form: llm  - name: model    type: model-selector    scope: llm    required: true    label:      en_US: Model      zh_Hans: 使用的模型      pt_BR: Model    human_description:      en_US: Model      zh_Hans: 使用的模型      pt_BR: Model    llm_description: which Model to invoke    form: formextra:  python:    source: tools/llm.py

Because the scope of the model parameter is llm, users can only select llm type models. The previous use case then becomes:

from collections.abc import Generatorfrom typing import Anyfrom dify_plugin import Toolfrom dify_plugin.entities.model.llm import LLMModelConfigfrom dify_plugin.entities.tool import ToolInvokeMessagefrom dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessageclass LLMTool(Tool):    def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]:        response = self.session.model.llm.invoke(            model_config=tool_parameters.get('model'),            prompt_messages=[                SystemPromptMessage(                    content='you are a helpful assistant'                ),                UserPromptMessage(                    content=tool_parameters.get('prompt')                )            ],            stream=True        )        for chunk in response:            if chunk.delta.message:                assert isinstance(chunk.delta.message.content, str)                yield self.create_text_message(text=chunk.delta.message.content)

Invoke Summary#

This interface summarizes a piece of text using the system model within your current workspace.

Entry Point#

    self.session.model.summary

Interface#

    def invoke(        self, text: str, instruction: str,    ) -> str:

text: The text to summarize.
instruction: Additional instructions, letting you control the style of the summary.

Invoke TextEmbedding#

Entry Point#

    self.session.model.text_embedding

Interface#

    def invoke(        self,        model_config: TextEmbeddingModelConfig,        texts: list[str],        input_type: EmbeddingInputType = EmbeddingInputType.QUERY,    ) -> TextEmbeddingResult:        pass

Invoke Rerank#

Entry Point#

    self.session.model.rerank

Interface#

    def invoke(        self, model_config: RerankModelConfig, docs: list[str], query: str    ) -> RerankResult:        pass

Invoke TTS#

Entry Point#

    self.session.model.tts

Interface#

    def invoke(        self, model_config: TTSModelConfig, content_text: str    ) -> Generator[bytes, None, None]:        pass

The bytes stream returned by the tts interface is an mp3 audio byte stream, and each iteration returns a complete audio segment. For more in-depth processing, choose an appropriate audio library.

Invoke Speech2Text#

Entry Point#

    self.session.model.speech2text

Interface#

    def invoke(        self, model_config: Speech2TextModelConfig, file: IO[bytes]    ) -> str:        pass

Here, file is an audio file encoded in mp3 format.

Invoke Moderation#

Entry Point#

    self.session.model.moderation

Interface#

    def invoke(self, model_config: ModerationModelConfig, text: str) -> bool:        pass

A return value of true indicates that the text contains sensitive content.

Reverse Invocation of Dify Services - Understand the fundamental concepts of reverse invocation
Reverse Invocation of App - Learn how to invoke Apps within the platform
Reverse Invocation of Tool - Learn how to invoke other plugins
Model Plugin Development Guide - Learn how to develop custom model plugins
Model Designing Rules - Understand the design principles of model plugins

Invoke LLM#

Entry Point#

Interface#

Use Case#

Best Practice#

Invoke Summary#

Entry Point#

Interface#

Invoke TextEmbedding#

Entry Point#

Interface#

Invoke Rerank#

Entry Point#

Interface#

Invoke TTS#

Entry Point#

Interface#

Invoke Speech2Text#

Entry Point#

Interface#

Invoke Moderation#

Entry Point#

Interface#

Related Resources#