OpenAI GPT
TLDR overview
- GPT-6 Sol passes 83.09% of the 544 HumanEval and MBPP tasks with executable tests, against 85.85% for Astra.
- Sol generated 480,125 lines of code against Astra's 656,445, which is 26.9% less for the same benchmark.
- Total findings fell 21.8%, from 12,606 to 9,859. Bugs came down 17.5% and code smells 22.3%.
- Vulnerabilities went the other way. The count rose 17.1%, from 117 to 137, and density rose 60% to 285 per mLOC.
- Blocker vulnerability density is 2 per mLOC, the lowest figure we have recorded from this family. In absolute terms that is a single blocker vulnerability across 480,125 lines.
How it works
Once you click Generate, Ollama reads this article and crafts 5 comprehension questions. Your answers are graded against the article content — general knowledge won't be enough. Score 70+ to count toward your certificate.
Questions are cached — you'll always get the same 5 for this article.