svg gpt-4o-mini

total: 24, pass: 0, fail: 24

    svg [model]                 run `svg` test
                                It tests the svg reader.
                                You have to use a vision language model!

027a29d16-mac
adcf25624-linux
7cebb43a6-linux
ba3b6b026-linux
5ba0a12a6-mac
92df3569e-linux
3aa665352-linux
16a6c7ac3-linux
774e5c41d-linux
3c6b6ebe1-linux
f0327eaea-linux
bb69badeb-linux
29a3bf1cb-linux
7f979aa5e-mac
0526d3e20-linux
0526d3e20-windows
869568e0b-linux
a5cdea9c6-mac
3e136fdeb-mac
a24618552-mac
a0c02bb2e-linux
a0c02bb2e-mac
45d75cf09-linux
ad17ef1a2-linux

Cases

027a29d16-mac

tested at: 2025-05-14T14:26:51.020259Z (119 days ago)

elapsed time: 16,821 ms

Error

Command '['cargo', 'run', '--release', '--', 'pdl', 'test1.pdl']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 597, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 551, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/svg.py", line 122, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/utils.py", line 70, in cargo_run
    result = subprocess.run(args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--', 'pdl', 'test1.pdl']' returned non-zero exit status 1.

suite

adcf25624-linux

tested at: 2025-05-15T23:00:24.413178Z (118 days ago)

elapsed time: 8,175 ms

Error


Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 621, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 574, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/home/baehyunsol/Documents/ragit/tests/svg.py", line 123, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

7cebb43a6-linux

tested at: 2025-05-17T00:48:25.067654Z (117 days ago)

elapsed time: 8,318 ms

Error


Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 621, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 574, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/home/baehyunsol/Documents/ragit/tests/svg.py", line 123, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

ba3b6b026-linux

tested at: 2025-05-18T09:48:01.799303Z (116 days ago)

elapsed time: 8,579 ms

Error


Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 630, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 583, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/home/baehyunsol/Documents/ragit/tests/svg.py", line 123, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

5ba0a12a6-mac

tested at: 2025-05-19T10:47:10.063082Z (115 days ago)

elapsed time: 18,883 ms

Error


Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 630, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 583, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/svg.py", line 123, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

92df3569e-linux

tested at: 2025-05-20T00:59:40.946018Z (114 days ago)

elapsed time: 7,634 ms

Error


Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 650, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 603, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/home/baehyunsol/Documents/ragit/tests/svg.py", line 123, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

3aa665352-linux

tested at: 2025-05-23T00:18:04.093493Z (111 days ago)

elapsed time: 7,508 ms

Error


Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 670, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 623, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/home/baehyunsol/Documents/ragit/tests/svg.py", line 123, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

16a6c7ac3-linux

tested at: 2025-05-24T20:40:39.036733Z (109 days ago)

elapsed time: 7,873 ms

Error


Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 672, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 623, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/home/baehyunsol/Documents/ragit/tests/svg.py", line 123, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

774e5c41d-linux

tested at: 2025-05-29T21:31:38.721609Z (104 days ago)

elapsed time: 1,920 ms

Error


Traceback (most recent call last):
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 701, in <module>
    test()
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 651, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ragit/tests/svg.py", line 48, in svg
    assert stat["staged files"] == len(broken_files)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

3c6b6ebe1-linux

tested at: 2025-05-31T05:38:30.224392Z (103 days ago)

elapsed time: 15,363 ms

Error


Traceback (most recent call last):
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 709, in <module>
    test()
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 658, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ragit/tests/svg.py", line 129, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

f0327eaea-linux

tested at: 2025-06-01T18:34:31.254192Z (101 days ago)

elapsed time: 7,593 ms

Error


Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 709, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 658, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/home/baehyunsol/Documents/ragit/tests/svg.py", line 129, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

bb69badeb-linux

tested at: 2025-06-02T00:08:14.271670Z (101 days ago)

elapsed time: 7,767 ms

Error


Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 709, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 658, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/home/baehyunsol/Documents/ragit/tests/svg.py", line 129, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

29a3bf1cb-linux

tested at: 2025-06-03T01:16:18.813157Z (100 days ago)

elapsed time: 8,044 ms

Error


Traceback (most recent call last):
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 718, in <module>
    test()
  File "/home/baehyunsol/Documents/ragit/tests/tests.py", line 667, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/home/baehyunsol/Documents/ragit/tests/svg.py", line 129, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

7f979aa5e-mac

tested at: 2025-06-05T12:05:40.934810Z (98 days ago)

elapsed time: 17,565 ms

Error


Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 718, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/tests.py", line 667, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.0-dev/tests/svg.py", line 129, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

0526d3e20-linux

tested at: 2025-06-05T16:46:27.585326Z (97 days ago)

elapsed time: 16,012 ms

Error


Traceback (most recent call last):
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 720, in <module>
    test()
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 669, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ragit/tests/svg.py", line 129, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

0526d3e20-windows

tested at: 2025-06-07T08:13:40.365332Z (96 days ago)

elapsed time: 742 ms

Error

Command '['cargo', 'run', '--release', '--no-default-features', '--', 'config', '--set', 'model', 'dummy']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "C:\Users\baehy\ragit\tests\tests.py", line 720, in <module>
    test()
    ~~~~^^
  File "C:\Users\baehy\ragit\tests\tests.py", line 669, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\baehy\ragit\tests\svg.py", line 19, in svg
    cargo_run(["config", "--set", "model", "dummy"])
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\baehy\ragit\tests\utils.py", line 87, in cargo_run
    result = subprocess.run(args, **kwargs)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.1008.0_x64__qbz5n2kfra8p0\Lib\subprocess.py", line 577, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['cargo', 'run', '--release', '--no-default-features', '--', 'config', '--set', 'model', 'dummy']' returned non-zero exit status 1.

suite

869568e0b-linux

tested at: 2025-06-09T20:06:53.014494Z (93 days ago)

elapsed time: 15,399 ms

Error


Traceback (most recent call last):
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 731, in <module>
    test()
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 680, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ragit/tests/svg.py", line 129, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

a5cdea9c6-mac

tested at: 2025-06-20T11:50:38.004891Z (83 days ago)

elapsed time: 18,078 ms

Error


Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 732, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 681, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/svg.py", line 129, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

3e136fdeb-mac

tested at: 2025-06-27T11:25:09.159801Z (76 days ago)

elapsed time: 17,091 ms

Error


Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.2-dev/tests/tests.py", line 742, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.2-dev/tests/tests.py", line 691, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.2-dev/tests/svg.py", line 129, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

a24618552-mac

tested at: 2025-07-09T12:03:26.241876Z (64 days ago)

elapsed time: 17,974 ms

Error


Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 781, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/tests.py", line 730, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/baehyunsol/Documents/Rust/ragit/tests/svg.py", line 130, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

a0c02bb2e-linux

tested at: 2025-07-21T08:03:09.691900Z (52 days ago)

elapsed time: 19,707 ms

Error


Traceback (most recent call last):
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 791, in <module>
    test()
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 740, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ragit/tests/svg.py", line 130, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

a0c02bb2e-mac

tested at: 2025-07-21T11:53:02.138116Z (52 days ago)

elapsed time: 17,953 ms

Error


Traceback (most recent call last):
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.2-dev/tests/tests.py", line 791, in <module>
    test()
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.2-dev/tests/tests.py", line 740, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
  File "/Users/baehyunsol/Documents/Rust/ragit-versions/ragit-0.4.2-dev/tests/svg.py", line 130, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
AssertionError

suite

45d75cf09-linux

tested at: 2025-07-25T08:20:45.661982Z (48 days ago)

elapsed time: 19,603 ms

Error


Traceback (most recent call last):
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 802, in <module>
    test()
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 751, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ragit/tests/svg.py", line 130, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

ad17ef1a2-linux

tested at: 2025-08-01T14:20:52.926510Z (40 days ago)

elapsed time: 19,369 ms

Error


Traceback (most recent call last):
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 802, in <module>
    test()
  File "/home/ubuntu/Documents/ragit/tests/tests.py", line 751, in <lambda>
    ("svg gpt-4o-mini", lambda: svg(test_model="gpt-4o-mini")),
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/Documents/ragit/tests/svg.py", line 130, in svg
    assert "ragit" in cargo_run(["pdl", "test1.pdl"], stdout=True).lower()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

suite

svg gpt-4o-mini

TOC

Cases