OpenAI GPT-4 System Card

OpenAI published a 60-page System Card, a document that describes their due diligence and risk management efforts

OpenAI announced GPT-4 - the newest and most capable large language model. This summary from @drjimfan tells us what’s different from GPT 3.5:

  • Multimodal: API accepts images as inputs to generate captions & analyses.
  • GPT-4 scores 90th percentile on BAR exam!!! And 99th percentile with vision on Biology Olympiad! Its reasoning capabilities are far more advanced than ChatGPT.
  • 25,000 words context: allows full documents to fit within a single prompt.
  • More creative & collaborative: generate, edit, and iterate with users on writing tasks.
  • There’re already many partners testing out GPT-4: Duolingo, Be My Eyes, Stripe, Morgan Stanley, Khan Academy … even Government of Iceland!

The same week, the company published a 60-page System Card, a document that describes OpenAIs' due diligence and risk management efforts:

This system card analyzes GPT-4, the latest LLM in the GPT family of models. First, we highlight safety challenges presented by the model’s limitations (e.g., producing convincing text that is subtly false) and capabilities (e.g., increased adeptness at providing illicit advice, performance in dual-use capabilities, and risky emergent behaviors). Second, we give a high-level overview of the safety processes OpenAI adopted to prepare GPT-4 for deployment.

Look out for a summary with comments from me in a future edition.