Wyciekły szczegóły techniczne dotyczące architektury GPT4

GPT-4 is more than 10 times the size of GPT-3, with an estimated total of 1.8 trillion parameters across 120 layers. OpenAI utilized a Mixture of Experts model to help keep costs reasonable. The model would have cost around $63 million to train. Many more details are available in the text.

Źródło pierwotne: https://www.semianalysis.com/p/gpt-4-architecture-infrastructure (za paywallem)