Persistent Storage

Store data across plugin interactions with the built-in key-value storage

Overview

Most plugin tools and endpoints operate in a stateless, single-round interaction model:

1. Receive a request
2. Process data
3. Return a response
4. End the interaction

However, many real-world applications require maintaining state across multiple interactions. This is where persistent storage becomes essential.

Persistent storage lets plugins keep data within the same workspace across interactions, enabling stateful applications and memory features.

Dify currently provides a key-value (KV) storage system for plugins; more flexible and powerful storage interfaces are planned based on developer needs.

Access Storage

All storage operations are performed through the storage object available in your plugin’s session:

# Access the storage interface
storage = self.session.storage

Storage Operations

Store Data

Store data with the set method:

def set(self, key: str, val: bytes) -> None:
    """
    Store data in persistent storage
    
    Parameters:
        key: Unique identifier for your data
        val: Binary data to store (bytes)
    """
    pass

The value must be in bytes format. This provides flexibility to store various types of data, including files.

Example: Storing Different Data Types

# String data (must convert to bytes)
storage.set("user_name", "John Doe".encode('utf-8'))

# JSON data
import json
user_data = {"name": "John", "age": 30, "preferences": ["AI", "NLP"]}
storage.set("user_data", json.dumps(user_data).encode('utf-8'))

# File data
with open("image.jpg", "rb") as f:
    image_data = f.read()
    storage.set("profile_image", image_data)

Retrieve Data

Retrieve stored data with the get method:

def get(self, key: str) -> bytes:
    """
    Retrieve data from persistent storage
    
    Parameters:
        key: Unique identifier for your data
        
    Returns:
        The stored data as bytes, or None if key doesn't exist
    """
    pass

Example: Retrieving and Converting Data

# Retrieving string data
name_bytes = storage.get("user_name")
if name_bytes:
    name = name_bytes.decode('utf-8')
    print(f"Retrieved name: {name}")

# Retrieving JSON data
import json
user_data_bytes = storage.get("user_data")
if user_data_bytes:
    user_data = json.loads(user_data_bytes.decode('utf-8'))
    print(f"User preferences: {user_data['preferences']}")

Delete Data

Delete stored data with the delete method:

def delete(self, key: str) -> None:
    """
    Delete data from persistent storage
    
    Parameters:
        key: Unique identifier for the data to delete
    """
    pass

Best Practices

Use Descriptive Keys

Create a consistent naming scheme for your keys to avoid conflicts and make your code more maintainable.

Handle Missing Keys

Always check if data exists before processing it, as the key might not be found.

Serialize Complex Data

Convert complex objects to JSON or other serialized formats before storing.

Implement Error Handling

Wrap storage operations in try/except blocks to handle potential errors gracefully.

Common Use Cases

• User preferences: Store user settings and preferences between sessions.
• Conversation history: Maintain context from previous conversations.
• API tokens: Store authentication tokens securely.
• Cached data: Store frequently accessed data to reduce API calls.
• File storage: Store user-uploaded files or generated content.