119cb72f0-mac

result.json

total: 34, pass: 29, fail: 5

fix typo in ragit-server

In order to iterate all the uid prefixes, the range has to be `0..256`,
but it was `0..255`. It was a fatal typo.

• commit: 119cb72f0b052ae705f0c16cc2c05bd18ef190ba
• platform: macOS-14.3.1-arm64-arm-64bit
• ragit version: ragit 0.3.0-dev
• rustc version: rustc 1.86.0-nightly (4a4309466 2025-02-02)
• cargo version: cargo 1.86.0-nightly (0e3d73849 2025-02-01)
• python version: 3.9.6
• tested at: 2025-02-13T11:03:41.308826Z (121 days ago)
• total elapsed time: 2,074,859 ms

TOC

1. cargo_tests  
2. add_and_rm  
3. recover  
4. clone  
5. clone2  
6. cli  
7. many_chunks  
8. many_jobs  
9. ls  
10. meta  
11. ii  
12. cat_file  
13. images  
14. markdown_reader  
15. csv_reader  
16. subdir  
17. tfidf  
18. merge  
19. external_bases  
20. end_to_end dummy  
21. end_to_end gpt-4o-mini  
22. prompts dummy  
23. prompts gpt-4o-mini  
24. prompts claude-3.5-sonnet  
25. empty dummy  
26. empty gpt-4o-mini  
27. images2 gpt-4o-mini  
28. images2 claude-3.5-sonnet  
29. extract_keywords dummy  
30. extract_keywords gpt-4o-mini  
31. ragit_api llama3.2-11b-groq  
32. ragit_api command-r  
33. ragit_api phi-4-14b-ollama  
34. migrate  

Cases

elapsed time: 43,759 ms

history

elapsed time: 94,473 ms

history

elapsed time: 11,681 ms

history

elapsed time: 248,778 ms

history

elapsed time: 135,470 ms

history

elapsed time: 9,965 ms

history

elapsed time: 351,165 ms

history

elapsed time: 36,112 ms

history

elapsed time: 243,398 ms

history

elapsed time: 7,367 ms

history

elapsed time: 59,700 ms

tfidf result on term 'crate 2 into stage load' is not close enough. error: `approximation[0] not in answer`, answer: ['1769f56725404827bcfa5b9098de8615930a113c7e1e262c0000000100000201', '2cb7c5d098cd9a57d4bafa02ee5840d7f99e78e96f67aa54000000010000020e', '8f32e9cbf70355959ad37eb512891b58cf7c92f7843cf0320000000100000202', '73f3b15a8d584c9252dc807e7d72f355852318257b12dac60000000100000200', 'ef45d815adb0848b8570d42062d870ec007da91cb817d5a40000000100000201', 'b4daf5642eabfaff39f0f8e908f9649af7560771a5f1f1db0000000100000200', '7af8f806193600bc6b6660c3152b08a1169e07cda1f685d80000000100000202', 'aa0f90901135b5975e581fb7252848a878657218e0aa977a0000000100000202', '0df6e45ba72b909476e0da4cc2729c887507df9ce9cc08640000000100000201', '7304fb09673f0e260670bfa74d73f584b09bb030b70a92480000000100000200'], approximation: ['86c710b60c25d2eba15531a3a6d1992dfc15220faf0b26820000000100000200', '1769f56725404827bcfa5b9098de8615930a113c7e1e262c0000000100000201', '57bc776b6c92573403d1f2bc98c1e7500fc23b2e6191135c0000000100000216', '964519d4e91635411905103bc03a40da9c6867e443bde89c0000000100000207', '6c9cd5beb2d6368f7f12477be22cf25ee127074a0a5d47110000000100000200', 'c1b08c124494c8ce9bd7d55df887151b30496aa4888f83370000000100000200', 'eb00b292fd50190e94a68e2b66bb8b3e40ed7388543f92200000000100000200', '393dc8f8a10156b5ec686ae438b883ea8a56dfbe67fa49a60000000100000200', '774dbc28ac14c24848625e6d76b8f694df61a757c35e7c010000000100000203', '39ff4225258013b524245a26e905360d147b40669928515c0000000100000210']
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ii.py", line 91, in ii
    raise AssertionError(f"approximation[{i}] not in answer")
AssertionError: approximation[0] not in answer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 358, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ii.py", line 104, in ii
    raise AssertionError(f"tfidf result on term '{term}' is not close enough. error: `{e}`, answer: {answer}, approximation: {approximation}")
AssertionError: tfidf result on term 'crate 2 into stage load' is not close enough. error: `approximation[0] not in answer`, answer: ['1769f56725404827bcfa5b9098de8615930a113c7e1e262c0000000100000201', '2cb7c5d098cd9a57d4bafa02ee5840d7f99e78e96f67aa54000000010000020e', '8f32e9cbf70355959ad37eb512891b58cf7c92f7843cf0320000000100000202', '73f3b15a8d584c9252dc807e7d72f355852318257b12dac60000000100000200', 'ef45d815adb0848b8570d42062d870ec007da91cb817d5a40000000100000201', 'b4daf5642eabfaff39f0f8e908f9649af7560771a5f1f1db0000000100000200', '7af8f806193600bc6b6660c3152b08a1169e07cda1f685d80000000100000202', 'aa0f90901135b5975e581fb7252848a878657218e0aa977a0000000100000202', '0df6e45ba72b909476e0da4cc2729c887507df9ce9cc08640000000100000201', '7304fb09673f0e260670bfa74d73f584b09bb030b70a92480000000100000200'], approximation: ['86c710b60c25d2eba15531a3a6d1992dfc15220faf0b26820000000100000200', '1769f56725404827bcfa5b9098de8615930a113c7e1e262c0000000100000201', '57bc776b6c92573403d1f2bc98c1e7500fc23b2e6191135c0000000100000216', '964519d4e91635411905103bc03a40da9c6867e443bde89c0000000100000207', '6c9cd5beb2d6368f7f12477be22cf25ee127074a0a5d47110000000100000200', 'c1b08c124494c8ce9bd7d55df887151b30496aa4888f83370000000100000200', 'eb00b292fd50190e94a68e2b66bb8b3e40ed7388543f92200000000100000200', '393dc8f8a10156b5ec686ae438b883ea8a56dfbe67fa49a60000000100000200', '774dbc28ac14c24848625e6d76b8f694df61a757c35e7c010000000100000203', '39ff4225258013b524245a26e905360d147b40669928515c0000000100000210']

history

elapsed time: 31,973 ms

history

elapsed time: 7,469 ms

history

elapsed time: 13,956 ms

history

elapsed time: 7,603 ms

history

elapsed time: 21,794 ms

history

elapsed time: 25,899 ms

history

elapsed time: 47,398 ms

history

elapsed time: 85,992 ms

history

elapsed time: 57,015 ms

history

elapsed time: 42,286 ms

Command '['cargo', 'run', '--release', '--', 'check', '--recover']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 358, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 312, in <lambda>
    ("end_to_end gpt-4o-mini", lambda: end_to_end(test_model="gpt-4o-mini")),
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/end_to_end.py", line 88, in end_to_end
    cargo_run(["check", "--recover"])
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/utils.py", line 61, in cargo_run
    result = subprocess.run(args, **kwargs)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'check', '--recover']' returned non-zero exit status 1.

history

elapsed time: 11,352 ms

history

elapsed time: 66,780 ms

history

elapsed time: 108,965 ms

history

elapsed time: 5,438 ms

history

elapsed time: 6,198 ms

history

elapsed time: 16,013 ms

history

elapsed time: 13,705 ms

history

elapsed time: 7,252 ms

history

elapsed time: 17,909 ms

history

elapsed time: 6,459 ms

Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 358, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 327, in <lambda>
    ("ragit_api llama3.2-11b-groq", lambda: ragit_api(test_model="llama3.2-11b-groq")),
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 458 ms

Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 358, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 328, in <lambda>
    ("ragit_api command-r", lambda: ragit_api(test_model="command-r")),
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 5,364 ms

Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 358, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 329, in <lambda>
    ("ragit_api phi-4-14b-ollama", lambda: ragit_api(test_model="phi-4-14b-ollama")),
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 149,321 ms

history