This module is a stepping stone to introduce interns to the AI/LLM field, familiarizing them with the concept of Vector Databases specifically used for storing embeddings for RAG problems.

Required knowledge to grasp after the internship

  • Basic Linux & Docker: Ability to configure networks and install applications via Docker.

  • Python Language: Basic syntax, how to use pip, virtual environment (venv).

  • Embedding Concept: Basic understanding of how to transform a text segment into a sequence of numerical vectors for computers to compare similarity.

  • Proficient in basic ChromaDB.

Execution sequence

  1. Request resources: Access https://dev.mdcgroup.vn to request 01 central Linux server to deploy the vector database.

  2. Deploy ChromaDB: Install and run ChromaDB as a standalone service (Chroma Server) via Docker Compose, configuring the default port 8000.

  3. Connect Client (using Python): Create a Python virtual environment on your personal machine, install the chromadb library. Write a script to connect remotely to the Chroma Server on Linux. MacOS users should pay attention to the Python version.

  4. Create Collection and Add Data: Create a collection, experiment with loading text data. Use the default embedding function or integrate libraries from HuggingFace/OpenAI to create vectors.

  5. Perform Similarity Query (Vector Search): Write a script to search using natural language questions. Check the ability to return the most semantically similar text segments (Querying by embedding/text).

  6. Persistent Data Management: Configure storage partitions for Docker to ensure ChromaDB's vector data is safely saved to physical disk.

Result acceptance

  • Multiple-choice test: Take a short test on Vector DB concepts at the internal server.

  • Questions & Proposals: Propose how to apply ChromaDB to create a smart document search tool for MDC Group itself.

  • Report results: Submit the Python source code file demonstrating the process: Load document -> Create Vector -> Search and a screenshot of the stable system log.

  • Weekend discussion: Share experiences about Vector Database compared to traditional relational databases.