How does ChatGPT do on the Solutions Architect Professional exam? The Solutions Architect Professional certification one of the hardest IT certifications you can get. It requires deep and broad knowledge about a lot of AWS products, furthermore it is a very long exam that requires you to switch context for every question.
If you compare the two levels of exams there is a clear difference between them:
- The associate level has two plainly wrong answers and a third answer that is wrong, if you read the question for the second time. So there is always one most answer
- The professional level has two answers that are wrong, but they are wrong if you read them twice. Then there are two answers that are both correct but one of them is more cost effective or more highly available. The question does not clearly word this context in all cases. So it might say a startup organization is looking for a solution (in this case you must pick the most cost-effective solution)
For the professional level you also need a highly detailed understanding of the AWS services. For example there might be a question where you have to choose between a storage gateway in file based mode or a storage gateway in cached based mode. In this case the whole question will point you to using the stored mode but in the question they mention that it required 2PB of storage. In this case, we cannot use stored mode because it stores a maximum of 512GB.
It is very hard to pass this exam if you have no working experience with AWS. If you do not have any experience with AWS the volume of information you need to store in your head is extreme.
You can find the transcript of the chat here.
ChatGPT 4.0
ChatGPT 4.0 got a pass mark of 832 points (of the required 750). All in all that is a great score for a language model!
ChatGPT 3.5
Unfortunately version 3.5 was not able to pass the same exam and scored 733 point, just shy of a passing score:
This shows that significant improvements were made between version 3.5 and 4.