e38845246-mac

result.json

total: 34, pass: 30, fail: 4

update commit hash

• commit: e38845246bb02fba45a8f10e7bdd3929e72a662b
• platform: macOS-14.3.1-arm64-arm-64bit
• ragit version: ragit 0.3.0-dev
• rustc version: rustc 1.86.0-nightly (4a4309466 2025-02-02)
• cargo version: cargo 1.86.0-nightly (0e3d73849 2025-02-01)
• python version: 3.9.6
• tested at: 2025-02-14T10:49:06.096222Z (120 days ago)
• total elapsed time: 2,452,091 ms

TOC

1. cargo_tests  
2. add_and_rm  
3. recover  
4. clone  
5. clone2  
6. cli  
7. many_chunks  
8. many_jobs  
9. ls  
10. meta  
11. ii  
12. cat_file  
13. images  
14. markdown_reader  
15. csv_reader  
16. subdir  
17. tfidf  
18. merge  
19. external_bases  
20. end_to_end dummy  
21. end_to_end gpt-4o-mini  
22. prompts dummy  
23. prompts gpt-4o-mini  
24. prompts claude-3.5-sonnet  
25. empty dummy  
26. empty gpt-4o-mini  
27. images2 gpt-4o-mini  
28. images2 claude-3.5-sonnet  
29. extract_keywords dummy  
30. extract_keywords gpt-4o-mini  
31. ragit_api llama3.2-11b-groq  
32. ragit_api command-r  
33. ragit_api phi-4-14b-ollama  
34. migrate  

Cases

elapsed time: 236,743 ms

history

elapsed time: 96,010 ms

history

elapsed time: 11,728 ms

history

elapsed time: 252,436 ms

history

elapsed time: 74,752 ms

history

elapsed time: 10,665 ms

history

elapsed time: 478,834 ms

history

elapsed time: 37,243 ms

history

elapsed time: 246,466 ms

history

elapsed time: 7,583 ms

history

elapsed time: 61,476 ms

tfidf result on term 'crate into 2 stage usiz' is not close enough. error: `approximation[1] not in answer`, answer: ['73f3b15a8d584c9252dc807e7d72f355852318257b12dac60000000100000200', '1769f56725404827bcfa5b9098de8615930a113c7e1e262c0000000100000201', '00fc67f631a0dc1d21d5c728fa2803a26cdb6786e2ab23fc0000000100000200', '2cb7c5d098cd9a57d4bafa02ee5840d7f99e78e96f67aa54000000010000020e', 'dc63d4f1e4f897aedde3c2425642ee03506fe3f5170476050000000100000200', '4177d56a402598693d9b90ccab9ed0f61e40476aab60a6270000000100000206', 'e9d9a44d90b9887cdbb444738901ae5ee4a52abbe6e0df900000000100000201', '8f32e9cbf70355959ad37eb512891b58cf7c92f7843cf0320000000100000202', '2cce09cbd354ac736b9b66cbd27df2fe78aa620e88759b5b0000000100000200', '924b59dd00b80bb5e477e88556c4ac94f5b0a51d335a5aaa000000010000020c'], approximation: ['73f3b15a8d584c9252dc807e7d72f355852318257b12dac60000000100000200', '774dbc28ac14c24848625e6d76b8f694df61a757c35e7c010000000100000203', '57bc776b6c92573403d1f2bc98c1e7500fc23b2e6191135c0000000100000216', '964519d4e91635411905103bc03a40da9c6867e443bde89c0000000100000207', '1769f56725404827bcfa5b9098de8615930a113c7e1e262c0000000100000201', '349f430bb7db8f8592b6d6c8154b1cf36dbfddd93d0e6c900000000100000200', 'e57cfa5571576205967a7b56416b7e44f6ab4e1ac49de6b0000000010000020b', 'a6c611d385e2a82f117778af3572fddf446181e66888cf3b0000000100000208', '9818549bab19460371f48b994296a4833a2584548e504b320000000100000200', '8f32e9cbf70355959ad37eb512891b58cf7c92f7843cf0320000000100000202']
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/ii.py", line 91, in ii
    raise AssertionError(f"approximation[{i}] not in answer")
AssertionError: approximation[1] not in answer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/tests.py", line 358, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/ii.py", line 104, in ii
    raise AssertionError(f"tfidf result on term '{term}' is not close enough. error: `{e}`, answer: {answer}, approximation: {approximation}")
AssertionError: tfidf result on term 'crate into 2 stage usiz' is not close enough. error: `approximation[1] not in answer`, answer: ['73f3b15a8d584c9252dc807e7d72f355852318257b12dac60000000100000200', '1769f56725404827bcfa5b9098de8615930a113c7e1e262c0000000100000201', '00fc67f631a0dc1d21d5c728fa2803a26cdb6786e2ab23fc0000000100000200', '2cb7c5d098cd9a57d4bafa02ee5840d7f99e78e96f67aa54000000010000020e', 'dc63d4f1e4f897aedde3c2425642ee03506fe3f5170476050000000100000200', '4177d56a402598693d9b90ccab9ed0f61e40476aab60a6270000000100000206', 'e9d9a44d90b9887cdbb444738901ae5ee4a52abbe6e0df900000000100000201', '8f32e9cbf70355959ad37eb512891b58cf7c92f7843cf0320000000100000202', '2cce09cbd354ac736b9b66cbd27df2fe78aa620e88759b5b0000000100000200', '924b59dd00b80bb5e477e88556c4ac94f5b0a51d335a5aaa000000010000020c'], approximation: ['73f3b15a8d584c9252dc807e7d72f355852318257b12dac60000000100000200', '774dbc28ac14c24848625e6d76b8f694df61a757c35e7c010000000100000203', '57bc776b6c92573403d1f2bc98c1e7500fc23b2e6191135c0000000100000216', '964519d4e91635411905103bc03a40da9c6867e443bde89c0000000100000207', '1769f56725404827bcfa5b9098de8615930a113c7e1e262c0000000100000201', '349f430bb7db8f8592b6d6c8154b1cf36dbfddd93d0e6c900000000100000200', 'e57cfa5571576205967a7b56416b7e44f6ab4e1ac49de6b0000000010000020b', 'a6c611d385e2a82f117778af3572fddf446181e66888cf3b0000000100000208', '9818549bab19460371f48b994296a4833a2584548e504b320000000100000200', '8f32e9cbf70355959ad37eb512891b58cf7c92f7843cf0320000000100000202']

history

elapsed time: 33,406 ms

history

elapsed time: 7,834 ms

history

elapsed time: 14,680 ms

history

elapsed time: 7,896 ms

history

elapsed time: 22,743 ms

history

elapsed time: 26,971 ms

history

elapsed time: 48,660 ms

history

elapsed time: 87,726 ms

history

elapsed time: 60,737 ms

history

elapsed time: 90,235 ms

history

elapsed time: 12,146 ms

history

elapsed time: 85,525 ms

history

elapsed time: 100,844 ms

history

elapsed time: 5,774 ms

history

elapsed time: 6,936 ms

history

elapsed time: 13,315 ms

history

elapsed time: 14,481 ms

history

elapsed time: 7,988 ms

history

elapsed time: 16,165 ms

history

elapsed time: 3,574 ms

Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/tests.py", line 358, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/tests.py", line 327, in <lambda>
    ("ragit_api llama3.2-11b-groq", lambda: ragit_api(test_model="llama3.2-11b-groq")),
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 412 ms

Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/tests.py", line 358, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/tests.py", line 328, in <lambda>
    ("ragit_api command-r", lambda: ragit_api(test_model="command-r")),
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 5,407 ms

Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/tests.py", line 358, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/tests.py", line 329, in <lambda>
    ("ragit_api phi-4-14b-ollama", lambda: ragit_api(test_model="phi-4-14b-ollama")),
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.3.0-dev-1/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 153,940 ms

history