It has been quite a long time since we have been hearing about the advent of GPT-3. It is astonishing to note that this machine model has been continuously evolving and now it is the third generation of this model. GPT-3, the shorthand of Generative Pertained Transformer-3 is widely noted for the fantastic results it produces.

OpenAI is an organization that focuses on designing artificial general intelligence devices that can compete well with humans in terms of intelligence. The organisation primarily pursues unsupervised machine learning algorithms. This new GPT-3 model was first announced in June by OpenAI although the model has not yet been released for general use due to fear about the malicious use of this technology.

How is GPT-3 different from its predecessors?

The third generation GPT-3 displays its extraordinary talent in interpreting text, answering questions, and accurately composing text. This new technology is an extremely useful language algorithm that utilizes machine learning to study the input(might be a series of words, text, and other information) and then aims to produce a suitable output for the input by analysing it thoroughly.

The predecessors of GPT-3 were GPT and GPT-2.They were most commonly known as Transformers. They had made their advent in 2017. So what did these transformers do? They used a function called attention to calculate the probability that a word will appear given surrounding words. GPT-2 was trained in 40Gb texts(1.5 billion parameters) and could effectively recognise words in its vicinity.  It was able to predict convincing streams of text in various styles when we provide an opening sentence. GPT-2 allowed experts to generate excellent and logically consistent writing.

GPT-3 is more advanced, progressed and evolved than its predecessors, GPT and GPT-2. It is one among the most powerful language models that has ever evolved. The model has 175 billion parameters compared with GPT-2’s 1.5 billion. GPT-3 can effectively process a large number of English sentences.

With incredibly powerful computer models( neural nets) that can recognize patterns, GPT-3’s language abilities are astonishing.  Various deep concepts of artificial intelligence are utilised to build this model with immense capabilities.

GPT-2	GPT-3
GPT-2 can produce artificial text in a feedback to the model when provided a random input.	GPT-3 also involves adjusted initialization, pre-normalization, and changeable tokenization in addition to GPT-2.
GPT-2 enables the user to make realistic continuation for any topic.	Apart from extraordinary performance in various NLP tasks , GPT-3 benchmarks in three distinct shots Zero-shot One-shot Some-shot environments.
1.5 billion parameters	175 billion parameters

GPT-3: Creative Potential of NLP(Natural Language Processing)

Since  GPT-3 is trained on 175 billion parameters somewhat heading over to a trillion words,it can effectively distinguish between the linguistic patterns in the input data. Minor fine-tuning can help the model to function explicit NLP tasks such as basic maths.  It can effectively do three-digits addition and subtraction. 

What does GPT-3 do?

GPT-3 can be understood as an incredibly sophisticated text indicator. It is able to achieve the so-called “meta-learning.” We provide a piece of text as input and the model tries to generate the best possible text that can appear next. Then the model takes the original input and the generated text output together as the input for the next round creating the next output till the limit has been reached. We need to feed the model with a huge amount of text until we reach a point where the model is self reliable in predicting the output.

Concepts used in GPT-3

• Various basic concepts used include Transformer and  Attention  for pre-training a dataset.  
• The model was trained  effectively on question answering tasks and was able to achieve state-of-the-art performance in it.
• The model was evaluated against various NLP benchmarks including:
  • Zero-shot
  • One-shot
  • Some-shot environments. 

The following graph displays the profits in terms of accuracy for various zero, one and few shots as a function of number of model parameters, it can be observed that huge gains are obtained due to size-scaled up.  

Graph of benefits in context of accuracy and number of examples, pic credit

 

 

 Drawbacks of GPT-3

• GPT-3 in a way lacks the presence of mind.It lacks reasoning capacity to an extent. The system will face challenges to determine what the right move will be in certain decision making scenarios.
• GPT-3 creates its output word-by-word, based on the immediately encircling information. When it comes to paragraphs or long narratives, GPT-3 might fail to arrive at a proper consensus on what the next data should be. GPT-3 in this manner can be amnesiac, constantly getting confused after a couple of sentences.

Conclusion:

GPT-3 has been a major advancement in Artificial Intelligence with 175 billion parameters( 10 times longer than its predecessor GPT-2 ). It had been said to reach the highest level of human-like intelligence through natural language processing. This technology is still evolving and there will be a lot of advancement in the near future. But, this model indeed revolutionizes the language processing abilities of existing systems. In the era of artificial intelligence, we can see this model to be a sign of immense progress in technology. This is, of course, a major leap.