319b5e4fe-mac

result.json

total: 39, pass: 35, fail: 4

add `tests/migrate2.py`

• commit: 319b5e4fe94713babc102a4a26d8dcdfe8b5cd9b
• platform: macOS-14.3.1-arm64-arm-64bit
• ragit version: ragit 0.3.0-dev
• rustc version: rustc 1.86.0-nightly (4a4309466 2025-02-02)
• cargo version: cargo 1.86.0-nightly (0e3d73849 2025-02-01)
• python version: 3.9.6
• tested at: 2025-02-21T11:22:37.525002Z (113 days ago)
• total elapsed time: 3,253,961 ms

TOC

1. cargo_tests  
2. add_and_rm  
3. add_and_rm2  
4. ignore  
5. recover  
6. clone  
7. clone2  
8. cli  
9. archive  
10. many_chunks  
11. many_jobs  
12. ls  
13. meta  
14. symlink  
15. ii  
16. cat_file  
17. images  
18. markdown_reader  
19. csv_reader  
20. subdir  
21. tfidf  
22. merge  
23. external_bases  
24. end_to_end dummy  
25. end_to_end gpt-4o-mini  
26. prompts dummy  
27. prompts gpt-4o-mini  
28. prompts claude-3.5-sonnet  
29. empty dummy  
30. empty gpt-4o-mini  
31. images2 gpt-4o-mini  
32. images2 claude-3.5-sonnet  
33. extract_keywords dummy  
34. extract_keywords gpt-4o-mini  
35. ragit_api llama3.2-11b-groq  
36. ragit_api command-r  
37. ragit_api phi-4-14b-ollama  
38. migrate  
39. migrate2  

Cases

elapsed time: 43,029 ms

history

elapsed time: 97,278 ms

history

elapsed time: 25,402 ms

history

elapsed time: 11,738 ms

history

elapsed time: 11,835 ms

history

elapsed time: 466,236 ms

history

elapsed time: 6,919 ms

history

elapsed time: 15,910 ms

history

elapsed time: 368,437 ms

history

elapsed time: 708,613 ms

history

elapsed time: 47,262 ms

history

elapsed time: 245,378 ms

history

elapsed time: 7,429 ms

history

elapsed time: 8,940 ms

history

elapsed time: 73,065 ms

tfidf result on term 'let pair' is not close enough. error: `answer[2] not in approximation`, answer: ['1cb944f6e3c7a429669dfc1c444db1032ad9c89a018809590000000100000200', '2c8fdd2845af306b1d8fb7638b45d581761637d98eee8289000000010000020a', 'c551ade6d454177dc0f26f987457b2f3f10510262876f28e0000000100000200', '007205dddaab388b7f2c07841ba3e5141b234e7dec890da60000000100000214', '147c86450ff6365d46bb25d38aa800dc25f0621d1f15ace70000000100000200', 'dbbb9355736fea55d816128e21d316f0305502ac7eac5b490000000100000200', 'ead620f18e4bd15d7baa6d661f8ca1fa284450b3da0936200000000100000209', '5751fbf644c65718dd8033e67312fe1ee63999dd8e98b38b0000000100000200', '74a9821e38482554bd3e78b0f1c474a927848122a54223420000000100000212', '835e935ce4f6840254d7968171afad7c140301a23b40a10e0000000100000213'], approximation: ['1cb944f6e3c7a429669dfc1c444db1032ad9c89a018809590000000100000200', '2c8fdd2845af306b1d8fb7638b45d581761637d98eee8289000000010000020a', '2fe3a9fd86a75b6e7f8d2b81945f8ea7ef646a86ba1d1a63000000010000020c', '8aabe3a652f6ea52cac3ac88adc5ae371ab9f9fb719b86810000000100000200', 'f75435fa91ee165766a7b2c86961c401ea07dc7f90646d270000000100000200', '03f0cf96b3133470121b2ace42477a5af21bc4f78868f5ed0000000100000200', '490fba16a9451d7f9a526a7dd3f6971095416bd021310e7b0000000100000200', '06d5285ce9bf9735dc624a9084c477fe7865f50020994b8b0000000100000200', '9758b1840e76d4dc5f73ce57989365d0cd63d12cbc19af410000000100000200', '7656a74393fb169540918303494f3476d4aba4fc037ef08a0000000100000200']
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ii.py", line 102, in ii_worker
    raise AssertionError(f"answer[{i}] not in approximation")
AssertionError: answer[2] not in approximation

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ii.py", line 38, in ii
    ii_worker()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ii.py", line 118, in ii_worker
    raise AssertionError(f"tfidf result on term '{term}' is not close enough. error: `{e}`, answer: {answer}, approximation: {approximation}")
AssertionError: tfidf result on term 'let pair' is not close enough. error: `answer[2] not in approximation`, answer: ['1cb944f6e3c7a429669dfc1c444db1032ad9c89a018809590000000100000200', '2c8fdd2845af306b1d8fb7638b45d581761637d98eee8289000000010000020a', 'c551ade6d454177dc0f26f987457b2f3f10510262876f28e0000000100000200', '007205dddaab388b7f2c07841ba3e5141b234e7dec890da60000000100000214', '147c86450ff6365d46bb25d38aa800dc25f0621d1f15ace70000000100000200', 'dbbb9355736fea55d816128e21d316f0305502ac7eac5b490000000100000200', 'ead620f18e4bd15d7baa6d661f8ca1fa284450b3da0936200000000100000209', '5751fbf644c65718dd8033e67312fe1ee63999dd8e98b38b0000000100000200', '74a9821e38482554bd3e78b0f1c474a927848122a54223420000000100000212', '835e935ce4f6840254d7968171afad7c140301a23b40a10e0000000100000213'], approximation: ['1cb944f6e3c7a429669dfc1c444db1032ad9c89a018809590000000100000200', '2c8fdd2845af306b1d8fb7638b45d581761637d98eee8289000000010000020a', '2fe3a9fd86a75b6e7f8d2b81945f8ea7ef646a86ba1d1a63000000010000020c', '8aabe3a652f6ea52cac3ac88adc5ae371ab9f9fb719b86810000000100000200', 'f75435fa91ee165766a7b2c86961c401ea07dc7f90646d270000000100000200', '03f0cf96b3133470121b2ace42477a5af21bc4f78868f5ed0000000100000200', '490fba16a9451d7f9a526a7dd3f6971095416bd021310e7b0000000100000200', '06d5285ce9bf9735dc624a9084c477fe7865f50020994b8b0000000100000200', '9758b1840e76d4dc5f73ce57989365d0cd63d12cbc19af410000000100000200', '7656a74393fb169540918303494f3476d4aba4fc037ef08a0000000100000200']

history

elapsed time: 32,748 ms

history

elapsed time: 7,357 ms

history

elapsed time: 13,873 ms

history

elapsed time: 7,550 ms

history

elapsed time: 22,354 ms

history

elapsed time: 25,958 ms

history

elapsed time: 48,019 ms

history

elapsed time: 86,603 ms

history

elapsed time: 72,591 ms

history

elapsed time: 111,190 ms

history

elapsed time: 11,810 ms

history

elapsed time: 72,350 ms

history

elapsed time: 86,724 ms

history

elapsed time: 5,624 ms

history

elapsed time: 6,381 ms

history

elapsed time: 12,921 ms

history

elapsed time: 12,849 ms

history

elapsed time: 3,078 ms

history

elapsed time: 12,690 ms

history

elapsed time: 6,737 ms

Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 370, in <lambda>
    ("ragit_api llama3.2-11b-groq", lambda: ragit_api(test_model="llama3.2-11b-groq")),
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'llama3.2-11b-groq', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 480 ms

Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 371, in <lambda>
    ("ragit_api command-r", lambda: ragit_api(test_model="command-r")),
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 5,372 ms

Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 402, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 372, in <lambda>
    ("ragit_api phi-4-14b-ollama", lambda: ragit_api(test_model="phi-4-14b-ollama")),
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/ragit_api.py", line 11, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'phi-4-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 182,273 ms

history

elapsed time: 210,091 ms

history