LLM – JoLoMo – AI + Software + Hardware

A Distributed RAG System with Kafka, ChromaDB, and gRPC

February 27, 2025

At JoLoMo LLC, we are building a scalable, distributed RAG system leveraging Kafka (3-node Kraft cluster), ChromaDB, and gRPC for high-performance document ingestion, storage, and retrieval. Our event-driven architecture allows us to process large workloads efficiently across multiple servers while ensuring fast, intelligent querying using embeddings from Ollama.

By choosing gRPC over REST, we ensure low-latency, high-throughput communication, with native streaming support for handling large documents. Our system is already distributed, but we’re working on further scalability improvements, including advanced query optimization, Kubernetes orchestration, and multi-tenant support.

Read the full article to learn how we’re building the future of AI-powered document retrieval and knowledge management.

Scalable RAG System

February 24, 2025

Introduction At JoLoMo LLC, we specialize in AI-driven solutions that enhance business efficiency and scalability. Our Retrieval-Augmented Generation (RAG) system leverages cutting-edge technologies to deliver real-time, context-aware data retrieval. This article explores our approach to developing and deploying a high-performance RAG system, powered by Ollama LLM, ChromaDB, Elasticsearch, Redis, and LangChain. System Architecture Overview Our…
Read more

vLLM with Open-WebUI

February 13, 2025

Setting Up vLLM with Open-WebUI Using Docker Compose Overview Leverage vLLM and Open-WebUI with Docker Compose for a streamlined, containerized deployment. This approach simplifies setup, ensures reproducibility, and offers easy scalability. Why Use Docker Compose? ✅ Simple Setup: Manage everything with one command. ✅ Reproducibility: Consistent environments across deployments. ✅ Isolation: Separate services in containers.…
Read more

AI-Powered Family Scribe

February 2, 2025

Preserving Memories & Strengthening Connections In an era where technology often distances us from genuine human interactions, we’re harnessing AI to bring families closer together. Our AI-powered Q&A texting service is designed to capture memories, spark meaningful conversations, and create a digital family historian—all through simple text messages. The Problem: Life Moves Fast, Memories Get…
Read more

RTX 4090 vs. RTX 6000 Ada for Self-Hosted vLLM Interfaces

January 25, 2025

Comparing RTX 4090 vs. RTX 6000 Ada for Self-Hosted vLLM Interfaces As the demand for self-hosted large language models (LLMs) grows, choosing the right GPU for a vLLM interface becomes a crucial decision. Two popular choices for self-hosted LLMs are the NVIDIA RTX 4090 and the NVIDIA RTX 6000 Ada. In this article, we will…
Read more

Mixture-of-Experts (MoE) Models and Neuromorphic Hardware Integration

January 11, 2025

Exploring the Future: Mixture-of-Experts (MoE) Models and Neuromorphic Hardware Integration Artificial Intelligence (AI) continues to evolve, and one of the most promising areas is the convergence of Mixture-of-Experts (MoE) models with neuromorphic hardware. This innovative approach offers exciting possibilities for creating efficient, adaptive, and scalable AI systems. Let’s explore what MoE models and neuromorphic hardware…
Read more

Vector Database Embedding

May 2, 2024

Document to Vector Database Embedding: Methods, Settings, and Configurations Explained When integrating a document into a vector database, the embedding process transforms textual content into numerical vectors, enabling semantic search, similarity comparisons, and LLM-powered interactions. Various methods and configurations impact the quality and efficiency of these embeddings. In this article, we’ll explore common embedding approaches,…
Read more

Category: LLM