Personal RAG

31 May, 2025

Setup Overview

I’m building a setup for a LLM to be able to access my records via AppleScript like Reminders, Calendar, and other Mac apps. While also using a memory system built on chroma DB for all personal knowledge base. My goal here is to plan out the build and how to execute the build. I am starting with the Personal Knowledge base. This system will be completely locally hosted on my M3 Max 36GB Ram MacBook Pro. Im wanting to start by looking at what’s the best way to build the Chroma vector DB locally to work well with the rest of the LLM integrations. Need low Ram occupied options to make sure there’s no unneeded use of Memory

Data

• Get data into JSON
  • CSV is as easy as

- import pandas as pd
df = pd.read_csv('data.csv')
json_data = df.to_json('output.json', orient='records', lines=True)

Vectorized Database

[[Setup Personal Knowledge Base]] Chroma [[Setup Chroma APA]]

• Use chroma or one of the others to vectorize your data.
• Chroma MCP Github
• Recommends using CLI to setup Local DB

Vectorized Memory

• And Agentic feature to record certain user inputs to a vectorized DB
• Have a tier like system for what recorded memory is considered valuable enough to keep?

RAG System

[[RAG for APA]] LangChain

• Use the data to be retrieved by LLM like Gemma 3 7b
• Small enough LLM to be used with other resource extensive tasks like the VDB

Memory Notes

[[MD Memory Notes MCP]]

• There is a MCP server already build called Basic Memory Basic Machines Website
  • This does a lot of the memory ability i would like but from first glance i think it lacks fluidity.
  • You have to ask the LLM to refer to a specific note in order for it to recall that "memory" which makes it more of a CRUD Note assistant than a memory bank. Still pretty useful though.

Apple Device Interaction

[[AppleScript for APA]]

• Here is a applescript MCP server. From the research i've done applescript seems like the best/most built in way to communicate with things on a Mac. applescript-mcp
  • Run AppleScript to access Mac applications and data
  • Interact with Notes, Calendar, Contacts, Messages, and more
  • Search for files using Spotlight or Finder
  • Read/write file contents and execute shell commands
  • Remote execution support via SSH