7912db142-linux

result.json

total: 38, pass: 31, fail: 7

update docs

• commit: 7912db1427192ea5a3721e4d1bc911073c452462
• platform: Linux-5.15.0-133-generic-x86_64-with-glibc2.35
• ragit version: ragit 0.3.0-dev
• rustc version: rustc 1.84.1 (e71f9a9a9 2025-01-27)
• cargo version: cargo 1.84.1 (66221abde 2024-11-19)
• python version: 3.10.12
• tested at: 2025-02-20T21:08:53.813099Z (113 days ago)
• total elapsed time: 2,815,153 ms

TOC

1. cargo_tests  
2. add_and_rm  
3. add_and_rm2  
4. ignore  
5. recover  
6. clone  
7. clone2  
8. cli  
9. archive  
10. many_chunks  
11. many_jobs  
12. ls  
13. meta  
14. symlink  
15. ii  
16. cat_file  
17. images  
18. markdown_reader  
19. csv_reader  
20. subdir  
21. tfidf  
22. merge  
23. external_bases  
24. end_to_end dummy  
25. end_to_end gpt-4o-mini  
26. prompts dummy  
27. prompts gpt-4o-mini  
28. prompts claude-3.5-sonnet  
29. empty dummy  
30. empty gpt-4o-mini  
31. images2 gpt-4o-mini  
32. images2 claude-3.5-sonnet  
33. extract_keywords dummy  
34. extract_keywords gpt-4o-mini  
35. ragit_api llama3.2-11b-groq  
36. ragit_api command-r  
37. ragit_api phi-4-14b-ollama  
38. migrate  

Cases

elapsed time: 120,197 ms

history

elapsed time: 26,985 ms

history

elapsed time: 6,592 ms

history

elapsed time: 2,439 ms

history

elapsed time: 2,856 ms

history

elapsed time: 1,062,202 ms

history

elapsed time: 4,720 ms

history

elapsed time: 2,846 ms

Command '['cargo', 'run', '--release', '--', 'rm', '--all']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 393, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/cli.py", line 44, in cli
    cargo_run(["rm", "--all"])
  File "/home/baehyunsol/Documents/ragit/tests/utils.py", line 65, in cargo_run
    result = subprocess.run(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'rm', '--all']' returned non-zero exit status 1.

history

elapsed time: 177,376 ms

history

elapsed time: 578,026 ms

history

elapsed time: 132,858 ms

history

elapsed time: 57,742 ms

history

elapsed time: 1,571 ms

history

elapsed time: 2,495 ms

history

elapsed time: 120,133 ms

tfidf result on term 'remov all model crate build' is not close enough. error: `approximation[2] not in answer`, answer: ['91dfe7a0e1d3df13bbb6206079ff85f5423cea98726466840000000100000212', 'c3047bec8101aacb87b8831c9960646a2dddacc317f117790000000100000201', 'd5368126b70297e71e9f1d14a62abf20e14e4b2d65a2b6b60000000100000209', '6ea382bebb6d2adbf7dbd48d7c21b62bbc17b13d8a22723e000000010000018d', 'e9d9a44d90b9887cdbb444738901ae5ee4a52abbe6e0df900000000100000201', '07edccd7468f00ef416e169bb45eeec81e91e583cce8aa8e0000000100000200', '42c9887f5422714a5f3dff5e34ac38d6e2f9845c4b529a4b0000000100000200', 'f3dac6a90605d66edc74902112d479ef4fb2ff8f4f60c20b0000000100000200', 'ea51413d5882ea717387b10bfad1acb351dec8d08ee59dc40000000100000200', '830768272cf534f8e8ed034bf4b6e4fa90e4e4af3037a0eb0000000100000200'], approximation: ['91dfe7a0e1d3df13bbb6206079ff85f5423cea98726466840000000100000212', 'd5368126b70297e71e9f1d14a62abf20e14e4b2d65a2b6b60000000100000209', 'f071e8758ddb0977861e335f747797663e610ad07ddddf77000000010000020a', '6ea382bebb6d2adbf7dbd48d7c21b62bbc17b13d8a22723e000000010000018d', '8be2584c5f1670f97f857a0a936c630367aafc6cc7a7257c0000000100000203', 'c3047bec8101aacb87b8831c9960646a2dddacc317f117790000000100000201', '87a060b5406c1901feaabe32a0d5c4e22f43e92881cf54970000000100000200', 'ddd59f2c986485ae58dcb116833260654bfd3fabce2007e70000000100000200', 'ea51413d5882ea717387b10bfad1acb351dec8d08ee59dc40000000100000200', '07edccd7468f00ef416e169bb45eeec81e91e583cce8aa8e0000000100000200']
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/ii.py", line 105, in ii_worker
    raise AssertionError(f"approximation[{i}] not in answer")
AssertionError: approximation[2] not in answer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 393, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/ii.py", line 38, in ii
    ii_worker()
  File "/home/baehyunsol/Documents/ragit/tests/ii.py", line 118, in ii_worker
    raise AssertionError(f"tfidf result on term '{term}' is not close enough. error: `{e}`, answer: {answer}, approximation: {approximation}")
AssertionError: tfidf result on term 'remov all model crate build' is not close enough. error: `approximation[2] not in answer`, answer: ['91dfe7a0e1d3df13bbb6206079ff85f5423cea98726466840000000100000212', 'c3047bec8101aacb87b8831c9960646a2dddacc317f117790000000100000201', 'd5368126b70297e71e9f1d14a62abf20e14e4b2d65a2b6b60000000100000209', '6ea382bebb6d2adbf7dbd48d7c21b62bbc17b13d8a22723e000000010000018d', 'e9d9a44d90b9887cdbb444738901ae5ee4a52abbe6e0df900000000100000201', '07edccd7468f00ef416e169bb45eeec81e91e583cce8aa8e0000000100000200', '42c9887f5422714a5f3dff5e34ac38d6e2f9845c4b529a4b0000000100000200', 'f3dac6a90605d66edc74902112d479ef4fb2ff8f4f60c20b0000000100000200', 'ea51413d5882ea717387b10bfad1acb351dec8d08ee59dc40000000100000200', '830768272cf534f8e8ed034bf4b6e4fa90e4e4af3037a0eb0000000100000200'], approximation: ['91dfe7a0e1d3df13bbb6206079ff85f5423cea98726466840000000100000212', 'd5368126b70297e71e9f1d14a62abf20e14e4b2d65a2b6b60000000100000209', 'f071e8758ddb0977861e335f747797663e610ad07ddddf77000000010000020a', '6ea382bebb6d2adbf7dbd48d7c21b62bbc17b13d8a22723e000000010000018d', '8be2584c5f1670f97f857a0a936c630367aafc6cc7a7257c0000000100000203', 'c3047bec8101aacb87b8831c9960646a2dddacc317f117790000000100000201', '87a060b5406c1901feaabe32a0d5c4e22f43e92881cf54970000000100000200', 'ddd59f2c986485ae58dcb116833260654bfd3fabce2007e70000000100000200', 'ea51413d5882ea717387b10bfad1acb351dec8d08ee59dc40000000100000200', '07edccd7468f00ef416e169bb45eeec81e91e583cce8aa8e0000000100000200']

history

elapsed time: 14,426 ms

history

elapsed time: 3,626 ms

history

elapsed time: 4,874 ms

history

elapsed time: 2,308 ms

history

elapsed time: 6,990 ms

Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 393, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/subdir.py", line 50, in subdir
    assert count_files() == (3, 0, 3)
AssertionError

history

elapsed time: 8,947 ms

history

elapsed time: 12,665 ms

history

elapsed time: 48,711 ms

history

elapsed time: 39,761 ms

history

elapsed time: 77,668 ms

history

elapsed time: 5,123 ms

history

elapsed time: 57,956 ms

history

elapsed time: 74,571 ms

history

elapsed time: 1,486 ms

history

elapsed time: 3,232 ms

history

elapsed time: 11,625 ms

history

elapsed time: 8,996 ms

history

elapsed time: 1,733 ms

history

elapsed time: 7,863 ms

history

elapsed time: 971 ms

Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 393, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 362, in <lambda>
    ("ragit_api llama3.2-11b-groq", lambda: ragit_api(test_model="llama3.2-11b-groq")),
  File "/home/baehyunsol/Documents/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 144 ms

Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 393, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 363, in <lambda>
    ("ragit_api command-r", lambda: ragit_api(test_model="command-r")),
  File "/home/baehyunsol/Documents/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 5,145 ms

Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 393, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 364, in <lambda>
    ("ragit_api phi-4-14b-ollama", lambda: ragit_api(test_model="phi-4-14b-ollama")),
  File "/home/baehyunsol/Documents/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 91,986 ms

Command '['cargo', 'run', '--release', '--', 'migrate']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 393, in <module>
    "error": None,
  File "/home/baehyunsol/Documents/ragit/tests/migrate.py", line 90, in migrate
    cargo_run(["migrate"])
  File "/home/baehyunsol/Documents/ragit/tests/utils.py", line 65, in cargo_run
    result = subprocess.run(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'migrate']' returned non-zero exit status 1.

history