Persistent Storage

Overview#

Most plugin tools and endpoints operate in a stateless, single-round interaction model:

Receive a request
Process data
Return a response
End the interaction

However, many real-world applications require maintaining state across multiple interactions. This is where persistent storage becomes essential.

Info

Persistent storage lets plugins keep data within the same workspace across interactions, enabling stateful applications and memory features.

Dify currently provides a key-value (KV) storage system for plugins; more flexible and powerful storage interfaces are planned based on developer needs.

Access Storage#

All storage operations are performed through the storage object available in your plugin’s session:

# Access the storage interfacestorage = self.session.storage

Storage Operations#

Store Data#

Store data with the set method:

def set(self, key: str, val: bytes) -> None:    """    Store data in persistent storage        Parameters:        key: Unique identifier for your data        val: Binary data to store (bytes)    """    pass

Warning

The value must be in bytes format. This provides flexibility to store various types of data, including files.

Example: Storing Different Data Types#

# String data (must convert to bytes)storage.set("user_name", "John Doe".encode('utf-8'))# JSON dataimport jsonuser_data = {"name": "John", "age": 30, "preferences": ["AI", "NLP"]}storage.set("user_data", json.dumps(user_data).encode('utf-8'))# File datawith open("image.jpg", "rb") as f:    image_data = f.read()    storage.set("profile_image", image_data)

Retrieve Data#

Retrieve stored data with the get method:

def get(self, key: str) -> bytes:    """    Retrieve data from persistent storage        Parameters:        key: Unique identifier for your data            Returns:        The stored data as bytes, or None if key doesn't exist    """    pass

Example: Retrieving and Converting Data#

# Retrieving string dataname_bytes = storage.get("user_name")if name_bytes:    name = name_bytes.decode('utf-8')    print(f"Retrieved name: {name}")# Retrieving JSON dataimport jsonuser_data_bytes = storage.get("user_data")if user_data_bytes:    user_data = json.loads(user_data_bytes.decode('utf-8'))    print(f"User preferences: {user_data['preferences']}")

Delete Data#

Delete stored data with the delete method:

def delete(self, key: str) -> None:    """    Delete data from persistent storage        Parameters:        key: Unique identifier for the data to delete    """    pass

Best Practices#

Use Descriptive Keys

Create a consistent naming scheme for your keys to avoid conflicts and make your code more maintainable.

Handle Missing Keys

Always check if data exists before processing it, as the key might not be found.

Serialize Complex Data

Convert complex objects to JSON or other serialized formats before storing.

Implement Error Handling

Wrap storage operations in try/except blocks to handle potential errors gracefully.

Common Use Cases#

User preferences: Store user settings and preferences between sessions.
Conversation history: Maintain context from previous conversations.
API tokens: Store authentication tokens securely.
Cached data: Store frequently accessed data to reduce API calls.
File storage: Store user-uploaded files or generated content.