This module aims to build an internal source code generation system and text assistant to help the entire team of programmers in the company improve work productivity safely and securely.
Required knowledge to grasp
LLMs/SLMs Concepts: Understand the number of parameters (such as 1B, 3B, 7B, 8B) and model quantization techniques (GGUF format) to optimize VRAM capacity.
API Knowledge: Understand how OpenAI-compatible API works (endpoints like /v1/chat/completions).
System Skills: Start background services, manage internal network ports.
Implementation sequence
Resource Registration: Go to the system https://dev.mdcgroup.vn to submit a request for resource allocation on the central server cluster equipped with mid-range GPUs (e.g., RTX 3090/4090 or A100).
Install Ollama: Proceed to install Ollama on a Linux Server via a native installation script or run it via Docker Container for easier management.
Download specialized models: Use the command line to pull small but highly efficient open-source language models such as: Llama-3-8B (general text), Qwen-2.5-Coder (specialized for programming), Phi-3 (ultra-light).
Configure internal network sharing: Edit the system's environment variable OLLAMA_HOST=0.0.0.0 to open service port 11434, allowing other programmer computers on the company's internal network to connect.
Integrate into work tools (IDE): Guide other programmers to install Extensions (such as Continue.dev or Codeium) on VS Code/Cursor, configure the API to point to the newly set up Ollama server to use code auto-completion and chat assistant features.
Monitor disk and VRAM resources: Use the commands ollama list and nvidia-smi to monitor the amount of graphics memory occupied when multiple programmers send code generation requests simultaneously.
Acceptance of results
Multiple-choice test: Take a theoretical test on open-source LLM application technologies at the internal link.
Questions & Suggestions: Propose a list of minimum hardware configurations to smoothly run a 7B model for about 20 programmers using it simultaneously.
Report results: Send a concise guide for other developers with screenshots of successful API connection configuration from personal IDE to the Server.
Weekend discussion: Report summary, evaluate the usefulness and response performance (Tokens/second) of the internal AI Assistant system during the intern activity session.