Dvirmor100 commited on
Commit
65e80b8
·
1 Parent(s): fc34b65

docs(llm,network): Added docs for ollama install and running and networking problems with WSL2 on Windows

Browse files
Files changed (2) hide show
  1. docs/llm.md +64 -0
  2. docs/network.md +39 -0
docs/llm.md ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # How to Run Deepseek Locally
2
+
3
+ This guide explains how to run the Deepseek model locally using Ollama as your server. Follow these steps to set up and use the model.
4
+
5
+ ## Prerequisites
6
+
7
+ Ensure you have Ollama installed on your machine. If not, follow the installation instructions for Windows at:
8
+
9
+ [Ollama Installation Guide](https://ollama.com/download/windows)
10
+
11
+ ## Getting Started with Deepseek Model
12
+
13
+ ### Step 1: Pull the Deepseek Model
14
+
15
+ To use models in Ollama, start by downloading the desired model using the `pull` command. Open your terminal and run:
16
+
17
+ ```bash
18
+ ollama pull <model>
19
+ ```
20
+
21
+ ### Step 2: Run the Model
22
+
23
+ After downloading, you can start the model with the following command:
24
+
25
+ ```bash
26
+ ollama run <model>
27
+ ```
28
+
29
+ This command activates the model, allowing you to enter queries or instructions, and receive responses.
30
+
31
+ ### Step 3: Using the Model with OpenAI API Specifications
32
+
33
+ Ollama is now compatible with the OpenAI Chat Completions API. This allows you to use additional tooling and applications with Ollama locally. Here's how to set it up:
34
+
35
+ ```python
36
+ from openai import OpenAI
37
+
38
+ model_name = "<model name>"
39
+ # Initialize the OpenAI client
40
+ client = OpenAI(
41
+ base_url = 'http://localhost:11434/v1',
42
+ api_key='ollama', # This is required but unused
43
+ )
44
+
45
+ # Create chat completions
46
+ response = client.chat.completions.create(
47
+ model=model_name,
48
+ messages=[
49
+ {"role": "system", "content": "You are a helpful assistant."},
50
+ {"role": "user", "content": "Who won the world series in 2020?"},
51
+ {"role": "assistant", "content": "The LA Dodgers won in 2020."},
52
+ {"role": "user", "content": "Where was it played?"}
53
+ ]
54
+ )
55
+
56
+ # Print the response
57
+ print(response.choices[0].message.content)
58
+ ```
59
+
60
+ ### Step 4: Configuring WSL2 for Local Server Access
61
+
62
+ If you are using WSL2 and wish to access the Ollama server running on your Windows machine, you need to configure WSL2 with mirror mode to share localhost between WSL2 and Windows. For more details, see:
63
+
64
+ [Network Configuration Guide](network.md)
docs/network.md ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Accessing Network Applications with WSL
2
+
3
+ This guide details how to access network applications using the Windows Subsystem for Linux (WSL) by mirroring the network configuration. This method allows you to use `localhost` to connect seamlessly between Windows and WSL.
4
+
5
+ ## Step 1: Create a WSL Configuration File
6
+
7
+ To configure network mirroring in WSL, you need to create a `.wslconfig` file in your home directory.
8
+
9
+ 1. **Open File Explorer:** Paste `%UserProfile%` in the address bar to navigate to your home directory.
10
+ 2. **Create the File:** Right-click in the folder, select "New > Text Document," and name the file `.wslconfig`.
11
+
12
+ ### Configure the File
13
+
14
+ Copy and paste the following content into the `.wslconfig` file:
15
+
16
+ ```text
17
+ [wsl2]
18
+ networkingMode=mirrored
19
+ ```
20
+
21
+ ## Step 2: Restart WSL
22
+
23
+ For the changes to take effect, you need to restart the WSL virtual machine.
24
+
25
+ Open a Command Prompt or PowerShell window and execute the following commands:
26
+
27
+ ```bash
28
+ # Shut down WSL
29
+ wsl --shutdown
30
+
31
+ # Start WSL
32
+ wsl
33
+ ```
34
+
35
+ ## Further Information
36
+
37
+ For additional details and advanced settings, refer to the official WSL Networking documentation:
38
+
39
+ [WSL Networking Documentation](https://learn.microsoft.com/en-us/windows/wsl/networking)