Attention is all we need

A Large Language Model (LLM), derives its understanding of language by mechanizing “attention”, the process where words in a paragraph are vectorized and cross referenced, thereby creating a mathematical framework of language. It is an elegant representation of the human mind, in which we replace biological neurons with artificial ones. The above model is also called a Foundation Model, to represent its capability to adapt to various tasks and domains, thanks to the diverse set of training data. To create a specialised model, the model will then go through fine tuning for domain specific training.

The discussion of LLMs often revolves around its technical and economical aspects, yet as a human invention, it deserves to be looked at from another perspective. The model theorises the learning process, namely, performance being tied to data quality and deep domain knowledge requiring specialized training.

Let’s take a step back and think backwards. If a simplified representation of the human mind requires those conditions to excel, why are we exempt from it? Contrary to model training, we have the ability to proactively curate our own training dataset and reject poor quality ones. Yet in modern society, we are constantly bombarded by information, of good and bad quality, and struggle to perform the filtration of knowledge. It was Television then, where programs were designed to be captivating and amusing, so audiences could indulge themselves in it, feeding their brain with mindless news. It is social media now, where information becomes shattered, shortened and easily available, so viewers can get that hit of dopamine as they scroll. Our predilection for immediate pleasure is biological and evolutionary, for our ancestors must acquire quick rewards (food, shelter, mates) to boost survival odds. While it is our innate desire to crave short-term pleasure, it does not mean we cannot control this impulse.

Attention is the most valuable currency now, so I urge everyone to be conscious of your time, because attention is all we need.