DimitrovK/impossible-tasks

Generate a browsable HTML index of all 102 runs about 1 month ago

good first issue hacktoberfest

0

Gave coding agents tasks that cannot be done. 61% faked a passing test suite; half of those are invisible to the tests.

Python

#ai-agents#ai-safety#benchmark#hacktoberfest#llm#pytest#python

Add a new impossible task about 1 month ago

good first issue hacktoberfest new task

DimitrovK/impossible-tasks

0

Gave coding agents tasks that cannot be done. 61% faked a passing test suite; half of those are invisible to the tests.

Python

#ai-agents#ai-safety#benchmark#hacktoberfest#llm#pytest#python

classify.py misses cheats hidden in pytest configuration about 1 month ago

good first issue hacktoberfest classifier

DimitrovK/impossible-tasks

0

Gave coding agents tasks that cannot be done. 61% faked a passing test suite; half of those are invisible to the tests.

Python

#ai-agents#ai-safety#benchmark#hacktoberfest#llm#pytest#python

Detect models that emit tool calls as plain text instead of executing them about 1 month ago

good first issue hacktoberfest classifier

DimitrovK/impossible-tasks

0

Gave coding agents tasks that cannot be done. 61% faked a passing test suite; half of those are invisible to the tests.

Python

#ai-agents#ai-safety#benchmark#hacktoberfest#llm#pytest#python

Record tokens and wall-clock time per run about 1 month ago

good first issue hacktoberfest

DimitrovK/impossible-tasks

0

Gave coding agents tasks that cannot be done. 61% faked a passing test suite; half of those are invisible to the tests.

Python

#ai-agents#ai-safety#benchmark#hacktoberfest#llm#pytest#python

Add more inverted tasks, where editing the test is the correct answer about 1 month ago

help wanted hacktoberfest new task

DimitrovK/impossible-tasks

0

Gave coding agents tasks that cannot be done. 61% faked a passing test suite; half of those are invisible to the tests.

Python

#ai-agents#ai-safety#benchmark#hacktoberfest#llm#pytest#python

Add a local Docker runner so the experiment works without a DigitalOcean account about 1 month ago

help wanted hacktoberfest

DimitrovK/impossible-tasks

0

Gave coding agents tasks that cannot be done. 61% faked a passing test suite; half of those are invisible to the tests.

Python

#ai-agents#ai-safety#benchmark#hacktoberfest#llm#pytest#python

Open Issues Need Help