roadmap: Jan has threads based RAG

[dan-menlo]

Goal

• Have a simple "Chat with Docs"
  • Only for new conversations
  • LanceDB
• But still support legacy RAG (let it be)

Tasks

• Sync up w/ Louis

[kalle07]

hey...
please start as simple as possible with nomic(as ambedder) and it will work well
if iam right some code you find here
https://github.com/nomic-ai/gpt4all
important that you can set embedding size and number of snippets
a fine start is "512token" and "4" snippets
create a collection and option to see my docs , how many words and how many verctors embedded (maybe live embedding)

after answer from the model, maybe an option also before (show the snippets as plain text)

next step. you can set for every LLM model and embedd-Model and document with your own settings (systempromt, token, ...all)
take a look Anything LM, they create workspaces for every settings

next step, implement more embedder models, bert-based are more or less same way, but it give some others qwen, jina and some ranking embedders...

and one nice option from LM-Studio(not opensource)
among other things they have an option for User-Poweruser-Developer (button below and instantly the layout and more option setting are changed)

[kalle07]

one options for embedding "chat" or "query" so more chat or force to keep all answer out of the doc!

and one option (small button near the chat) to keep the snippets in VRAM after first answer or delete them.

it should be set to delete by default when I ask a new question afterwards...

[kalle07]

maybe in next next step an option to send prefixes to embedder
https://www.youtube.com/watch?v=76EIC_RaDNw

and
re-ranker models (but i dont know what is the different)
one i have here
https://huggingface.co/kalle07/embedder_collection

[louis-menlo]

Thank you for sharing this, @kalle07. It's truly helpful.

[ramonpzg]

@louis-menlo -- I started working on a simple document parsing implementation with a bit of RAG using llamaindex.ts and LanceDB on a separate branch based on the Tauri branch. This is mainly for enabling users to add documents, images, videos and audio while having it all in a db.lance file for storing and querying as users need it.

@dan-menlo mentioned you have specific plans for implementing this via MCP. Can I confirm with you whether you would like me to add support for the things I mentioned or remove this as it will be done in a different way?

[louis-menlo]

cc @ramonpzg likely will drive this.