This module aims to build an internal source code generation system and text assistant to help the entire team of programmers in the company improve work productivity safely and securely.

Knowledge requirements to be grasped after the internship

  • LLMs/SLMs Concepts: Understand the number of parameters (such as 1B, 3B, 7B, 8B) and model quantization techniques (GGUF format) to optimize VRAM capacity.

  • API Knowledge: Understand how OpenAI-compatible API works (endpoints like /v1/chat/completions).

  • System Skills: Start background services, manage internal network ports.

Execution sequence

  1. Resource Registration: Go to the system https://dev.mdcgroup.vn to submit a request for resource allocation on a central server cluster equipped with mid-range GPUs (RTX 3090/4090 or A100).

  2. Install Ollama: Proceed to install Ollama on a Linux Server via a native installation script or run it through a Docker Container for easier management.

  3. Download specialized models: Use command lines to pull small but highly efficient open-source language models such as: Llama-3-8B (general text), Qwen-2.5-Coder (specialized for programming), Phi-3 (ultra-light).

  4. Configure internal network sharing: Modify the system's environment variable OLLAMA_HOST=0.0.0.0 to open service port 11434, allowing other developer computers on the company's internal network to connect.

  5. Integrate into work tools (IDE): Guide other developers to install Extensions (such as Continue.dev or Codeium) on VS Code, configure the API to point to the newly set up Ollama server to use code auto-completion and assistant chat features.

  6. Monitor disk and VRAM resources: Use the commands ollama list and nvidia-smi to monitor the amount of graphics memory occupied when multiple developers send code generation requests simultaneously.

Result acceptance

  • Quiz: Take a theoretical test on open-source LLM application technologies at the internal link.

  • Questions & Suggestions: Propose a list of minimum hardware configurations to smoothly run a 7B model for about 20 developers using it simultaneously.

  • Report results: Send a concise guide for other developers with screenshots of successful API connection configurations from their personal IDE to the Server.

  • Weekend discussion: Report summary, evaluate the usefulness and response performance (Tokens/second) of the internal AI Assistant system during the intern activity session.