MemRL

Memory-augmented reinforcement learning system that captures coding sessions as episodes, learns from successful problem-solving patterns, and provides semantic search to retrieve relevant past experiences.

A memory-augmented reinforcement learning system for Claude Code that captures coding sessions as episodes and learns from experience to improve future task assistance. Built in Rust, it automatically captures session transcripts, extracts structured intent using LLM analysis, and stores episodes with git diffs and error resolutions in both file-based storage and vector databases using LanceDB and FastEmbed. The system implements Bellman equation-based utility propagation to spread value from helpful episodes to similar ones, uses temporal credit assignment to reward episodes that preceded successful outcomes, and provides semantic search through embeddings to retrieve relevant past experiences.