5b3ae7fd7-linux

result.json

total: 39, pass: 33, fail: 6

update commit hash

• commit: 5b3ae7fd7a2f89042924b025160ccfb312f98e64
• platform: Linux-5.15.0-133-generic-x86_64-with-glibc2.35
• ragit version: ragit 0.3.0-dev
• rustc version: rustc 1.84.1 (e71f9a9a9 2025-01-27)
• cargo version: cargo 1.84.1 (66221abde 2024-11-19)
• python version: 3.10.12
• tested at: 2025-02-22T13:23:38.813172Z (112 days ago)
• total elapsed time: 2,599,938 ms

TOC

1. cargo_tests  
2. add_and_rm  
3. add_and_rm2  
4. ignore  
5. recover  
6. clone  
7. clone2  
8. cli  
9. archive  
10. many_chunks  
11. many_jobs  
12. ls  
13. meta  
14. symlink  
15. ii  
16. cat_file  
17. images  
18. markdown_reader  
19. csv_reader  
20. subdir  
21. tfidf  
22. merge  
23. external_bases  
24. end_to_end dummy  
25. end_to_end gpt-4o-mini  
26. prompts dummy  
27. prompts gpt-4o-mini  
28. prompts claude-3.5-sonnet  
29. empty dummy  
30. empty gpt-4o-mini  
31. images2 gpt-4o-mini  
32. images2 claude-3.5-sonnet  
33. extract_keywords dummy  
34. extract_keywords gpt-4o-mini  
35. ragit_api llama3.2-11b-groq  
36. ragit_api command-r  
37. ragit_api phi-4-14b-ollama  
38. migrate  
39. migrate2  

Cases

elapsed time: 120,623 ms

history

elapsed time: 33,420 ms

history

elapsed time: 8,584 ms

history

elapsed time: 2,949 ms

history

elapsed time: 2,877 ms

history

elapsed time: 94,262 ms

history

elapsed time: 4,376 ms

history

elapsed time: 3,440 ms

history

elapsed time: 188,115 ms

history

elapsed time: 586,603 ms

history

elapsed time: 130,553 ms

history

elapsed time: 58,247 ms

history

elapsed time: 1,587 ms

history

elapsed time: 2,881 ms

history

elapsed time: 128,118 ms

tfidf result on term 'chunk 1b' is not close enough. error: `answer[2] not in approximation`, answer: ['a810d2f745f9657ff762f4370d88b1a3c2d6885b1e6537a00000000100000200', 'bcb141e0a9848218d1c380a24eff14dcdbd1617ccda7e95e0000000100000224', '6dcad8bee8ba48a9e40ad1fc676ece7d5f59f0e5237576870000000100000200', '15a4fa2b2a28b3cc52da0b6bda2b16c4c5dddd0bb8d06f570000000100000200', '65bb987eda114b951d0249ae93c5f6fb702c594b0d64d9b10000000100000200', 'ece7d9ac964d374e578a3c05daf8b46245af9d2a8e76e9210000000100000200', 'c7062921979fb787f85436d9298930448db1718d47fed4930000000100000200', '66133d84b7aab03fe99667bcee59cb7aa971410ef4f60bb3000000010000020a', '1c167dbbaa87d63afa5b9e650a29c6f64f4dc19dea2249b2000000010000018e', '17ff643db0e27785a30858285dc8c3f25fd486e5a320bc57000000010000020e'], approximation: ['a810d2f745f9657ff762f4370d88b1a3c2d6885b1e6537a00000000100000200', 'bcb141e0a9848218d1c380a24eff14dcdbd1617ccda7e95e0000000100000224', '15a4fa2b2a28b3cc52da0b6bda2b16c4c5dddd0bb8d06f570000000100000200', '66133d84b7aab03fe99667bcee59cb7aa971410ef4f60bb3000000010000020a', '359d9bf3ba2dd08d78de412ecfaf53a5d90a2e3564f180370000000100000208', 'c49c7a25d092e38d5992a5a512e8711141e8051c8e8d79f20000000100000200', 'ee8176080ac63e9fe528e7542b1a3fddcb728f6b5edf4a430000000100000200', '1557ddd1b584b8dcec275d294009e627e2df8cc9860aeb570000000100000200', 'ffeb3353b505e174b8c4022e428967b5be67e7c2c9159ed4000000010000020c', 'fcd4694f8faa17dd0a9ec22b1238fd006dd3073a070bed4b0000000100000200']
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/ii.py", line 102, in ii_worker
    raise AssertionError(f"answer[{i}] not in approximation")
AssertionError: answer[2] not in approximation

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/ii.py", line 38, in ii
    ii_worker()
  File "/home/baehyunsol/Documents/ragit/tests/ii.py", line 118, in ii_worker
    raise AssertionError(f"tfidf result on term '{term}' is not close enough. error: `{e}`, answer: {answer}, approximation: {approximation}")
AssertionError: tfidf result on term 'chunk 1b' is not close enough. error: `answer[2] not in approximation`, answer: ['a810d2f745f9657ff762f4370d88b1a3c2d6885b1e6537a00000000100000200', 'bcb141e0a9848218d1c380a24eff14dcdbd1617ccda7e95e0000000100000224', '6dcad8bee8ba48a9e40ad1fc676ece7d5f59f0e5237576870000000100000200', '15a4fa2b2a28b3cc52da0b6bda2b16c4c5dddd0bb8d06f570000000100000200', '65bb987eda114b951d0249ae93c5f6fb702c594b0d64d9b10000000100000200', 'ece7d9ac964d374e578a3c05daf8b46245af9d2a8e76e9210000000100000200', 'c7062921979fb787f85436d9298930448db1718d47fed4930000000100000200', '66133d84b7aab03fe99667bcee59cb7aa971410ef4f60bb3000000010000020a', '1c167dbbaa87d63afa5b9e650a29c6f64f4dc19dea2249b2000000010000018e', '17ff643db0e27785a30858285dc8c3f25fd486e5a320bc57000000010000020e'], approximation: ['a810d2f745f9657ff762f4370d88b1a3c2d6885b1e6537a00000000100000200', 'bcb141e0a9848218d1c380a24eff14dcdbd1617ccda7e95e0000000100000224', '15a4fa2b2a28b3cc52da0b6bda2b16c4c5dddd0bb8d06f570000000100000200', '66133d84b7aab03fe99667bcee59cb7aa971410ef4f60bb3000000010000020a', '359d9bf3ba2dd08d78de412ecfaf53a5d90a2e3564f180370000000100000208', 'c49c7a25d092e38d5992a5a512e8711141e8051c8e8d79f20000000100000200', 'ee8176080ac63e9fe528e7542b1a3fddcb728f6b5edf4a430000000100000200', '1557ddd1b584b8dcec275d294009e627e2df8cc9860aeb570000000100000200', 'ffeb3353b505e174b8c4022e428967b5be67e7c2c9159ed4000000010000020c', 'fcd4694f8faa17dd0a9ec22b1238fd006dd3073a070bed4b0000000100000200']

history

elapsed time: 14,645 ms

history

elapsed time: 3,933 ms

history

elapsed time: 4,893 ms

history

elapsed time: 2,325 ms

history

elapsed time: 8,499 ms

history

elapsed time: 9,980 ms

history

elapsed time: 13,373 ms

history

elapsed time: 48,322 ms

history

elapsed time: 40,534 ms

history

elapsed time: 66,647 ms

history

elapsed time: 5,324 ms

history

elapsed time: 37,905 ms

history

elapsed time: 112,080 ms

Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 358, in <lambda>
    ("prompts claude-3.5-sonnet", lambda: prompts(test_model="claude-3.5-sonnet")),
  File "/home/baehyunsol/Documents/ragit/tests/prompts.py", line 34, in prompts
    cargo_run(["build"])
  File "/home/baehyunsol/Documents/ragit/tests/utils.py", line 66, in cargo_run
    result = subprocess.run(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.

history

elapsed time: 1,856 ms

history

elapsed time: 2,660 ms

history

elapsed time: 10,108 ms

history

elapsed time: 97,035 ms

Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 364, in <lambda>
    ("images2 claude-3.5-sonnet", lambda: images2(test_model="claude-3.5-sonnet")),
  File "/home/baehyunsol/Documents/ragit/tests/images2.py", line 30, in images2
    cargo_run(["build"])
  File "/home/baehyunsol/Documents/ragit/tests/utils.py", line 66, in cargo_run
    result = subprocess.run(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.

history

elapsed time: 1,651 ms

history

elapsed time: 7,464 ms

history

elapsed time: 1,002 ms

Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 370, in <lambda>
    ("ragit_api llama3.2-11b-groq", lambda: ragit_api(test_model="llama3.2-11b-groq")),
  File "/home/baehyunsol/Documents/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 147 ms

Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 371, in <lambda>
    ("ragit_api command-r", lambda: ragit_api(test_model="command-r")),
  File "/home/baehyunsol/Documents/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 5,151 ms

Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 372, in <lambda>
    ("ragit_api phi-4-14b-ollama", lambda: ragit_api(test_model="phi-4-14b-ollama")),
  File "/home/baehyunsol/Documents/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 92,730 ms

history

elapsed time: 619,274 ms

history