Proposal for Abstract Encapsulation of Memory Mode Similar to LangChain's Memory

[sopaco]

Hello, the Rig.rs development team!
I'm a user and enthusiast of the Rig.rs framework, and I've been exploring its capabilities for LLM application development. I believe there's an area where Rig.rs could be further enhanced, and I'd like to propose an improvement regarding memory management.
Background
In the field of LLM application development, effective memory management is crucial. LangChain has set a great example by introducing a well - structured memory system. Their memory mode allows developers to manage the conversation history, context, and state of the application in a more organized and modular way. This not only simplifies the development process but also enhances the maintainability and extensibility of the application.
Proposal
I suggest that Rig.rs implement an abstract encapsulation of the memory mode, similar to what LangChain offers. Here are some potential benefits and considerations:
Benefits
Improved Developer Experience: An abstracted memory mode would provide a standardized interface for developers to interact with the memory system. This would reduce the learning curve and make it easier for developers to integrate memory management into their applications.
Enhanced Modularity: By encapsulating the memory logic, different memory strategies (such as buffer memory, long - term memory) can be easily implemented and switched. This would make the framework more flexible and adaptable to various application scenarios.
Better Compatibility: A standardized memory interface would make it easier to integrate Rig.rs with other tools and libraries that rely on memory management.
Considerations
Design Flexibility: The abstract memory mode should be designed to be flexible enough to support different types of memory, such as in - memory storage, disk - based storage, or even distributed memory systems.
Performance Optimization: Memory operations can be performance - critical. The design should take into account performance optimization techniques, such as caching and lazy loading.
Documentation: Along with the implementation, detailed documentation should be provided to help developers understand how to use the new memory mode effectively.
Conclusion
I believe that implementing an abstract encapsulation of the memory mode in Rig.rs would significantly enhance the framework's capabilities and make it more competitive in the LLM application development space. I'm looking forward to your thoughts on this proposal and hope to see this feature integrated in future releases.
Thank you for your time and consideration!
Best regards,

[linear]

RIG-711 Proposal for Abstract Encapsulation of Memory Mode Similar to LangChain's Memory

[0xMochan]

Hey, we greatly appreciate the issue. Memory is definitely an aspect of research the rig team is highly interested in! There's a lot of great papers that are coming out (see https://arxiv.org/abs/2502.12110) that i think would do very well in the Rig architecture.

[joshua-mo-143]

This should theoretically be unblocked now that #563 has been merged. It was previously blocked because we didn't really have a way to abstract behaviour for insertion of documents into a vector store.

At this point, I see the current solution as being somewhat similar to this:

• We implement a MemoryStrategy that allows for essentially different types of memories to be either fetched or inserted:

pub trait MemoryStrategy: Send {
     async fn fetch_memory(&self, num: usize) -> Vec<String>;
     async fn insert_memory(&self, memories: Vec<String>) -> Result<(), FooError>;
}

• Then we'd have some kind of abstraction like below (where V: VectorStoreIndex + InsertDocuments, Model: CompletionModel):

struct VectorStoreMemory<V, Model> {
     store: V,
     model: Model,
     memory_fetch_max: u8,
     similarity_threshold: f32,
}

impl<V, Model> MemoryStrategy for MemoryStore<V, Model>
where
    V: VectorStoreIndex + InsertDocuments,
    Model: CompletionModel 
    {
    // .. code goes here
}

Something like this anyway.

That way we'd still be able to basically have relatively decent defaults while basically allowing users to implement whatever they want

[joshua-mo-143]

Finally starting work on this 🥳

[rommeld]

Is it possible to add to this feature request? An integration to Redis out of the box would be a great way to manage memory.

[joshua-mo-143]

Is it possible to add to this feature request? An integration to Redis out of the box would be a great way to manage memory.

Hi, this is pretty much next on the list to be implemented alongside prompt caching!

We had some initial blockers because of not being able to directly delete vector store entries based on TTL expiry which we have now fixed and I have been unfortunately busy doing talks.

You can expect to see some progress soon assuming nothing else urgent gets in the way.