## 1. Create Knowledge-Base
First, let's say there're text files explaining ai. We'll build a knowledge-base with from the text files. The directory should look like below.
```
ai_tutorials/
|
*-- ai_tutorial_1.txt
|
*-- ai_tutorial_2.txt
|
*-- ai_tutorial_3.txt
|
*-- ... and many more txt files
```
Run `cd ai_tutorial; rag init`. You'll see a new directory created like below.
```
ai_tutorials/
|
*-- .ragit/
| |
| *-- chunks/
| |
| *-- configs/
| |
| *-- files/
| |
| *-- images/
| |
| *-- prompts/
| |
| *-- index.json
| |
| *-- models.json
|
*-- ai_tutorial_1.txt
|
*-- ai_tutorial_2.txt
|
*-- ai_tutorial_3.txt
|
*-- ... and many more txt files
```
`.ragit/` is like `.git/` of git repositories. It saves metadata and chunks. After `rag init`, the knowledge-base is empty. You have to add files to the staging using `rag add` command.
Run `rag add --all`. Now you're ready to build the knowledge-base. Run `rag build` to start the work. The default model is `llama3.3-70b-groq` and you need `GROQ_API_KEY` to run. If you want to run gpt-4o-mini, run `rag config --set model gpt-4o-mini`. You can see the list of the models using `rag ls-models`. You can also add models manually to `.ragit/models.json`.
```
elapsed time: 00:33
staged files: 15, processed files: 13
errors: 0
committed chunks: 39
buffered files: 8, buffered chunks: 8
flush count: 1
model: gpt-4o-mini
input tokens: 14081 (0.001$), output tokens: 1327 (0.000$)
```
`rag build` takes very long time and money (if you're using a proprietary api). It creates chunks and add title and summary to each chunk, using AI.
You can press Ctrl+C to pause the process. You can resume from where you left off by running `rag build` again. (more on [a dedicated document](./commands/build.txt))
```
ai_tutorials/
|
*-- .ragit/
| |
| *-- chunks/
| | |
| | *-- ... a lot of directories
| |
| *-- configs/
| |
| *-- files/
| |
| *-- images/
| |
| *-- prompts/
| |
| *-- index.json
| |
| *-- models.json
|
*-- ai_tutorial_1.txt
|
*-- ai_tutorial_2.txt
|
*-- ai_tutorial_3.txt
|
*-- ... and many more txt files