027a29d16-mac

result.json

total: 54, pass: 41, fail: 13

update `tests/svg.py`

I made the code more generic so that it's easier to add svg files.

• commit: 027a29d16171f415545d580b22a1ef3074d46410
• platform: macOS-14.3.1-arm64-arm-64bit
• ragit version: ragit 0.4.0-dev
• rustc version: rustc 1.88.0-nightly (2e6882ac5 2025-05-05)
• cargo version: cargo 1.88.0-nightly (7918c7eb5 2025-04-27)
• python version: 3.11.7
• tested at: 2025-05-14T14:26:51.020259Z (31 days ago)
• total elapsed time: 4,162,921 ms

TOC

1. cargo_tests  
2. add_and_rm  
3. add_and_rm2  
4. ignore  
5. recover  
6. clone  
7. clone_empty  
8. server  
9. server_permission  
10. cli  
11. archive  
12. many_chunks  
13. many_jobs  
14. ls  
15. meta  
16. symlink  
17. ii  
18. cat_file  
19. generous_file_reader  
20. images  
21. markdown_reader  
22. csv_reader  
23. real_repos  
24. subdir  
25. tfidf  
26. merge  
27. external_bases  
28. end_to_end dummy  
29. end_to_end llama3.3-70b  
30. audit llama3.3-70b  
31. logs llama3.3-70b  
32. prompts dummy  
33. prompts gpt-4o-mini  
34. prompts claude-3.5-sonnet  
35. empty dummy  
36. empty llama3.3-70b  
37. server_chat llama3.3-70b  
38. images2 gpt-4o-mini  
39. images3 gpt-4o-mini  
40. pdf gpt-4o-mini  
41. svg gpt-4o-mini  
42. web_images gpt-4o-mini  
43. images2 claude-3.5-sonnet  
44. extract_keywords dummy  
45. extract_keywords gpt-4o-mini  
46. orphan_process llama3.3-70b  
47. write_lock llama3.3-70b  
48. ragit_api command-r  
49. query_options llama3.3-70b  
50. models_init  
51. test_home_config_override  
52. migrate  
53. migrate2  
54. config  

Cases

elapsed time: 540,215 ms

history

elapsed time: 128,336 ms

history

elapsed time: 44,704 ms

history

elapsed time: 13,805 ms

history

elapsed time: 11,684 ms

history

elapsed time: 63,969 ms

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/clone.py", line 43, in clone
    create_user(id="test-user", password="password")
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/server.py", line 186, in create_user
    assert response.status_code == 200
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

history

elapsed time: 2,280 ms

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/clone_empty.py", line 20, in clone_empty
    create_user(id="test-user", password="87654321")
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/server.py", line 186, in create_user
    assert response.status_code == 200
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

history

elapsed time: 6,213 ms

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/server.py", line 38, in server
    create_user(id="test-user", password="12345678")
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/server.py", line 186, in create_user
    assert response.status_code == 200
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

history

elapsed time: 2,403 ms

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/server_permission.py", line 23, in server_permission
    create_user(id="test-user-2", email="sample2@email.com", password="abcdefgh")
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/server.py", line 186, in create_user
    assert response.status_code == 200
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

history

elapsed time: 24,647 ms

history

elapsed time: 406,896 ms

history

elapsed time: 413,451 ms

history

elapsed time: 54,677 ms

history

elapsed time: 275,396 ms

Command '['cargo', 'run', '--release', '--', 'ls-files', 'tmp-file', '--processed', '--json', '--name-only']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/ls.py", line 342, in ls
    assert len(json.loads(cargo_run(["ls-files", "tmp-file", "--processed", "--json", "--name-only"], stdout=True))) == 0
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/utils.py", line 70, in cargo_run
    result = subprocess.run(args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'ls-files', 'tmp-file', '--processed', '--json', '--name-only']' returned non-zero exit status 1.

history

elapsed time: 7,932 ms

history

elapsed time: 10,850 ms

history

elapsed time: 83,516 ms

tfidf result on term 'file 25' is not close enough. error: `answer[1] not in approximation`, answer: ['7fc54003b881ccdafaa1a7854a8905bbafcf2c40b2381f0800000001000001c2', 'd7d7a6856f5b3a7a92a9e57ea2f7cc021827bd0cbcef3bb20000000100000204', '318323062babb157d1e25a7e9f4ba59fd7a4d962e768706b0000000100000200', 'e83f9cbc6848572b1bfdecefe48ad859e65095be90acf88f0000000100000200', '5a6f477f07a7b59fb1bfb2209f52f374d57452ec961b7dcf0000000100000200', '01c3a5841d65abba53905b709adb16c67908c2190d7ed8810000000100000191', '03fb8fa50a27ee46979abb3af31240120f9004c18ee3babc0000000100000228', '00ed7a197725009715fa2efb71c08612ca67b6a19cc7156b000000010000020c', '9e32427659389711bfc3764e45299e7b2ff430db0da92ea70000000100000202', 'bafded8759c9e99107f67f1d03ee1addc7be61909eed58550000000100000200'], approximation: ['7fc54003b881ccdafaa1a7854a8905bbafcf2c40b2381f0800000001000001c2', 'e83f9cbc6848572b1bfdecefe48ad859e65095be90acf88f0000000100000200', '5a6f477f07a7b59fb1bfb2209f52f374d57452ec961b7dcf0000000100000200', '01c3a5841d65abba53905b709adb16c67908c2190d7ed8810000000100000191', '03fb8fa50a27ee46979abb3af31240120f9004c18ee3babc0000000100000228', '00ed7a197725009715fa2efb71c08612ca67b6a19cc7156b000000010000020c', '9e32427659389711bfc3764e45299e7b2ff430db0da92ea70000000100000202', 'bafded8759c9e99107f67f1d03ee1addc7be61909eed58550000000100000200', '2e429c8c2141c68e4c3db5db7e880721ff2b8b22eea1d13c0000000100000212', '7c7a3eaed0283c816bdba6f63a484cac70542496ce8826280000000100000200']
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/ii.py", line 100, in ii_worker
    raise AssertionError(f"answer[{i}] not in approximation")
AssertionError: answer[1] not in approximation

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/ii.py", line 36, in ii
    ii_worker()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/ii.py", line 116, in ii_worker
    raise AssertionError(f"tfidf result on term '{term}' is not close enough. error: `{e}`, answer: {answer}, approximation: {approximation}")
AssertionError: tfidf result on term 'file 25' is not close enough. error: `answer[1] not in approximation`, answer: ['7fc54003b881ccdafaa1a7854a8905bbafcf2c40b2381f0800000001000001c2', 'd7d7a6856f5b3a7a92a9e57ea2f7cc021827bd0cbcef3bb20000000100000204', '318323062babb157d1e25a7e9f4ba59fd7a4d962e768706b0000000100000200', 'e83f9cbc6848572b1bfdecefe48ad859e65095be90acf88f0000000100000200', '5a6f477f07a7b59fb1bfb2209f52f374d57452ec961b7dcf0000000100000200', '01c3a5841d65abba53905b709adb16c67908c2190d7ed8810000000100000191', '03fb8fa50a27ee46979abb3af31240120f9004c18ee3babc0000000100000228', '00ed7a197725009715fa2efb71c08612ca67b6a19cc7156b000000010000020c', '9e32427659389711bfc3764e45299e7b2ff430db0da92ea70000000100000202', 'bafded8759c9e99107f67f1d03ee1addc7be61909eed58550000000100000200'], approximation: ['7fc54003b881ccdafaa1a7854a8905bbafcf2c40b2381f0800000001000001c2', 'e83f9cbc6848572b1bfdecefe48ad859e65095be90acf88f0000000100000200', '5a6f477f07a7b59fb1bfb2209f52f374d57452ec961b7dcf0000000100000200', '01c3a5841d65abba53905b709adb16c67908c2190d7ed8810000000100000191', '03fb8fa50a27ee46979abb3af31240120f9004c18ee3babc0000000100000228', '00ed7a197725009715fa2efb71c08612ca67b6a19cc7156b000000010000020c', '9e32427659389711bfc3764e45299e7b2ff430db0da92ea70000000100000202', 'bafded8759c9e99107f67f1d03ee1addc7be61909eed58550000000100000200', '2e429c8c2141c68e4c3db5db7e880721ff2b8b22eea1d13c0000000100000212', '7c7a3eaed0283c816bdba6f63a484cac70542496ce8826280000000100000200']

history

elapsed time: 36,665 ms

history

elapsed time: 29,214 ms

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/generous_file_reader.py", line 58, in generous_file_reader
    assert "invalid/url.png" in cargo_run(["cat-file", "invalid-image-url.md"], stdout=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

history

elapsed time: 2,754 ms

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/images.py", line 50, in images
    assert "sample2.png" in stderr   # "sample2.png not found" is expected
           ^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

history

elapsed time: 14,297 ms

history

elapsed time: 7,716 ms

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/csv_reader.py", line 55, in csv_reader
    assert cargo_run(["build"], check=False) != 0
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

history

elapsed time: 309,751 ms

Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/real_repos.py", line 63, in real_repos
    cargo_run(["build"])
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/utils.py", line 70, in cargo_run
    result = subprocess.run(args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'build']' returned non-zero exit status 1.

history

elapsed time: 28,474 ms

history

elapsed time: 31,349 ms

history

elapsed time: 59,862 ms

history

elapsed time: 142,167 ms

history

elapsed time: 81,064 ms

history

elapsed time: 97,232 ms

history

elapsed time: 15,265 ms

history

elapsed time: 8,515 ms

history

elapsed time: 9,852 ms

history

elapsed time: 55,682 ms

history

elapsed time: 128,583 ms

history

elapsed time: 13,315 ms

history

elapsed time: 12,325 ms

history

elapsed time: 4,607 ms

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 547, in <lambda>
    ("server_chat llama3.3-70b", lambda: server_chat(test_model="llama3.3-70b")),
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/server_chat.py", line 32, in server_chat
    create_user(id="test-user", password="secure-password")
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/server.py", line 186, in create_user
    assert response.status_code == 200
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

history

elapsed time: 13,296 ms

history

elapsed time: 50,388 ms

history

elapsed time: 113,206 ms

Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 550, in <lambda>
    ("pdf gpt-4o-mini", lambda: pdf(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/pdf.py", line 49, in pdf
    assert any([pdf["name"] in r["source"] for r in search_result])
AssertionError

history

elapsed time: 16,821 ms

Command '['cargo', 'run', '--release', '--', 'pdl', 'test1.pdl']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 551, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/svg.py", line 122, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/utils.py", line 70, in cargo_run
    result = subprocess.run(args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'pdl', 'test1.pdl']' returned non-zero exit status 1.

history

elapsed time: 40,565 ms

history

elapsed time: 19,467 ms

history

elapsed time: 3,203 ms

history

elapsed time: 11,821 ms

history

elapsed time: 116,644 ms

history

elapsed time: 81,989 ms

history

elapsed time: 5,524 ms

history

elapsed time: 8,873 ms

history

elapsed time: 946 ms

history

elapsed time: 483 ms

history

elapsed time: 248,479 ms

history

elapsed time: 69,835 ms

history

elapsed time: 34,658 ms

history