6b1add466-linux

result.json

total: 31, pass: 24, fail: 7

fix typo in prompts

I accidentally used `{%%}` instead of `{##}` for comments.

• commit: 6b1add4661dd9e10099599ca736b1a41ed3d00d6
• platform: Linux-6.8.0-1016-aws-x86_64-with-glibc2.39
• ragit version: ragit 0.3.0-dev
• rustc version: rustc 1.82.0 (f6e511eec 2024-10-15)
• cargo version: cargo 1.82.0 (8f40fc59f 2024-08-21)
• python version: 3.12.3
• tested at: 2025-02-05T15:29:51.351113Z (129 days ago)
• total elapsed time: 3,911,333 ms

TOC

1. cargo_tests  
2. add_and_rm  
3. recover  
4. clone  
5. clone2  
6. cli  
7. many_chunks  
8. ls  
9. meta  
10. ii  
11. cat_file  
12. images  
13. markdown_reader  
14. csv_reader  
15. subdir  
16. tfidf  
17. merge  
18. external_bases  
19. end_to_end dummy  
20. end_to_end gpt-4o-mini  
21. prompts dummy  
22. prompts gpt-4o-mini  
23. prompts claude-3.5-sonnet  
24. empty dummy  
25. empty gpt-4o-mini  
26. images2 gpt-4o-mini  
27. images2 claude-3.5-sonnet  
28. ragit_api llama3.2-11b-groq  
29. ragit_api command-r  
30. ragit_api phi-3-14b-ollama  
31. migrate  

Cases

elapsed time: 1,417,237 ms

history

elapsed time: 69,874 ms

history

elapsed time: 8,711 ms

history

elapsed time: 13,762 ms

Command '['cargo', 'run', '--release', '--', 'clone', 'http://127.0.0.1/test-user/repo1']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 332, in <module>
    test()
  File "/home/ubuntu/Documents/ci/ragit/tests/clone.py", line 50, in clone
    cargo_run(["clone", "http://127.0.0.1/test-user/repo1"])
  File "/home/ubuntu/Documents/ci/ragit/tests/utils.py", line 61, in cargo_run
    result = subprocess.run(args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'clone', 'http://127.0.0.1/test-user/repo1']' returned non-zero exit status 1.

history

elapsed time: 380,372 ms

history

elapsed time: 7,416 ms

history

elapsed time: 277,546 ms

history

elapsed time: 180,703 ms

history

elapsed time: 5,465 ms

history

elapsed time: 45,783 ms

history

elapsed time: 18,146 ms

history

elapsed time: 6,059 ms

history

elapsed time: 11,099 ms

history

elapsed time: 6,001 ms

history

elapsed time: 17,057 ms

history

elapsed time: 19,722 ms

history

elapsed time: 35,577 ms

history

elapsed time: 68,811 ms

history

elapsed time: 23,332 ms

The build should have timed out
Traceback (most recent call last):
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 332, in <module>
    test()
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 288, in <lambda>
    ("end_to_end dummy", lambda: end_to_end(test_model="dummy")),
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ci/ragit/tests/end_to_end.py", line 78, in end_to_end
    raise Exception("The build should have timed out")
Exception: The build should have timed out

history

elapsed time: 36,565 ms

Command '['cargo', 'run', '--release', '--', 'check', '--recover']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 332, in <module>
    test()
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 289, in <lambda>
    ("end_to_end gpt-4o-mini", lambda: end_to_end(test_model="gpt-4o-mini")),
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ci/ragit/tests/end_to_end.py", line 80, in end_to_end
    cargo_run(["check", "--recover"])
  File "/home/ubuntu/Documents/ci/ragit/tests/utils.py", line 61, in cargo_run
    result = subprocess.run(args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'check', '--recover']' returned non-zero exit status 1.

history

elapsed time: 9,309 ms

history

elapsed time: 69,239 ms

history

elapsed time: 117,042 ms

Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 332, in <module>
    test()
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 292, in <lambda>
    ("prompts claude-3.5-sonnet", lambda: prompts(test_model="claude-3.5-sonnet")),
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ci/ragit/tests/prompts.py", line 34, in prompts
    cargo_run(["build"])
  File "/home/ubuntu/Documents/ci/ragit/tests/utils.py", line 61, in cargo_run
    result = subprocess.run(args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.

history

elapsed time: 4,354 ms

history

elapsed time: 6,087 ms

history

elapsed time: 13,350 ms

history

elapsed time: 127,093 ms

Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 332, in <module>
    test()
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 298, in <lambda>
    ("images2 claude-3.5-sonnet", lambda: images2(test_model="claude-3.5-sonnet")),
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ci/ragit/tests/images2.py", line 30, in images2
    cargo_run(["build"])
  File "/home/ubuntu/Documents/ci/ragit/tests/utils.py", line 61, in cargo_run
    result = subprocess.run(args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.

history

elapsed time: 9,025 ms

history

elapsed time: 405 ms

Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 332, in <module>
    test()
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 302, in <lambda>
    ("ragit_api command-r", lambda: ragit_api(test_model="command-r")),
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ci/ragit/tests/ragit_api.py", line 9, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'command-r', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 320 ms

Command '['cargo', 'run', '--release', '--', '--model', 'phi-3-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.
Traceback (most recent call last):
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 332, in <module>
    test()
  File "/home/ubuntu/Documents/ci/ragit/tests/tests.py", line 303, in <lambda>
    ("ragit_api phi-3-14b-ollama", lambda: ragit_api(test_model="phi-3-14b-ollama")),
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ci/ragit/tests/ragit_api.py", line 9, in ragit_api
    subprocess.run(["cargo", "run", "--release", '--', '--model', test_model, '--input', 'hello_world.pdl'], check=True)
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', '--model', 'phi-3-14b-ollama', '--input', 'hello_world.pdl']' returned non-zero exit status 101.

history

elapsed time: 500,055 ms

history