Attempts to mechanize literary creativity have been made for many centuries. We can recall, for example, the logical machine of the 12th-century philosopher Raymond Lull, which is actually capable of creating combinations of concepts.
The process of mechanization was especially active in the 20th century, and the advent of computers only accelerated it. But the real breakthrough came in 1997, when LSTM (Long short-term memory) networks were created.
This is a special kind of recurrent neural networks that can learn long-term dependencies. That is, they are capable of long-term storage of information and repeated access to it. And now there are even more advanced transformer networks based on the GPT (Generative Pre-trained Transformer) algorithm.
For example, you are writing a story using GPT-2, and this network is able to remember 1024 tokens in a row, that is, parts of words, words, or individual letters that make up your text. Let’s say in the first sentence you entered a character named Vasily. If you repeat the name of this hero for the next 1024 tokens, then the network will “remember” it and understand that this is the same hero.
But when developing the experimental neural network “NeuroStanislavsky”, an even more advanced algorithm of the latest generation GPT-3 was used , capable of remembering 2048 tokens. The network was created by students of the University of Science and Technology MISIS in collaboration with the Biennale of Theater Arts.
