AI – TIDBITS
Who produced The Persistence of Memory?
A. Salvador Dalí
It was Dalí who produced The Persistence of Memory in 1931. It remains one of the most protruding examples of surrealism and is housed at the Museum of Modern Art in New York.
ChatGPT, along with text-to-image tools such as DALL-E 2 and Stable Diffusion, is part of a new wave of software called generative AI. They create works of their own by drawing on patterns they’ve identified in vast troves of existing, human-created content. This technology was pioneered at big tech companies like Google.
What is AI Vine –
A Text-to-Video generator
Vine was a beloved short-form video service purchased by Twitter in 2012 and discontinued in 2017. The beloved short-form video service, which allowed users to create and upload 6-second videos, was owned by then-Twitter shortly after its 2012 founding until its shuttering in 2017. It was the source of many early viral videos and the subject of numerous compilations of “the best Vines of all time.”
OpenAI says it’s using over a million GPUs, it’s essentially saying it has a vast digital brain made up of high-performance processors, working together to generate text, analyze images, simulate voices and much more.
About Transformer —
A transformer with decoder only trained on enormous amounts of text. Just to feed in more and more date. A new language Model – named as; “generatively pre-trained transformer” or GPT evolved.
We dive into Transformers in Deep Learning, a revolutionary architecture that powers today’s cutting-edge models like GPT and BERT. We’ll break down the core concepts behind attention mechanisms, self-attention, and how Transformers handle sequential data. We will see the limitations of RNNs, and why Transformers are so powerful. This is a 1st part of my Transformers in Deep Learning Course, providing an overview of Transformers, and its importance. Whether you’re a beginner or looking to deepen your understanding, my Transformers in Deep Learning Course playlist will guide you through the in-depth working of Transformers
[All of that data was processed by extraordinary amounts of computing power, performing quadrillions of calculations in what’s known as transformer model, a type of deep learning architecture introduced in 2017 that revolutionized how machines handle language. Transformers don’t read words in order, like humans do— thy process all the words at once, assigning each one a numerical weight based on its relationship to every other word. This allows the model to consider context at multiple levels—but not just what a word means, but what it might mean in nearly infinite combinations. It’s how ChatGPT.
The same logic works for Images, sounds, video, or anything that can be turned into data.]
GPT: Stands for Generative Pre-trained Transformer is a family of artificial intelligence (AI) deep-learning models built by OpenAI that are designed to understand, process and generate human-like text, code. And other media.
ChatGPT
All large language models (LLM) are giant prediction engines trained to guess the next word in a sentence based on everything that came before it. During training, GPT was fed hundreds of billions of words – from books, Wikipedia articles, scientific papers, appliance manuals, news articles, Reddit threads, help-desk transcripts. It also ingests billions more bits of punctuation and partial words. [Whether this material was freely available to be consumed upon is a subject that will tie up the court for many years]. What is not in dispute is that ChatGPT is a language omnivore. [ChatGPT appears to know and understand things like magic and the trick. ChatGPT doesn’t think or have opinions or a moral compass. On its own, it has no ability to assess whether what it is saying is correct or useful, which is why ChatGPT can explain a scientific law correctly in a moment, and invent fake law of the same theory the next. The model optimizes for fluency not truth. It doesn’t fact check itself because it doesn’t understand FACTS.
GPT -1:
7000 mostly self-published books found on the internet, many of them on romance or vampire fiction. A dataset downloadable…..These are now part of Large Language Models (LLM) understanding language. OpenAI feeds more and more text from the public internet into its models. Its system become increasingly life like, they were simply becoming better at making predictions about what text should come next in a sequence, based on their training data.
GPT – 2
Was trained on forty gigabytes of internet text and had about 1.5 billion parameters, making it more than ten times bigger than the first and better at generating more complex text. It also sounded more believable.
GPT – 3: was trained on 300 billion tokens (representing roughly 450 billion to 500 billion words
GPT – 4: was trained on a massive dataset of publicly available internet data, licensed third party data, and multimodal inputs using Microsoft Azure AI supercomputers. An AI model with capabilities well beyond the model. GPT- 4 – one of the most sophisticated technological creations in human history-could not be trusted with basic math. Instead, it is needed to use Python- a thirty-year old programming language famous for its clarity and precision- to do calculations.
The Model required approximately 1025 floating point operations or FLOPs—tens of trillions of trillions of individual math steps. Which is equivalent to say ~ if an average person speaks 16,000 words a day, then eight billion of us
GPT – 5: supports a large context window of up to 400,000 tokens. Tokens include internal thinking or deliberation. Critics said the model is highly verbose. According to OpenAI, GPT-5 has state-of-the-art performance across coding, math, writing, health, and visual perception, among other things, and is available to all users.
It is of no doubt that GPT-5 is a huge leap forward from the early GPT-1. The GPT-1 model, which was introduced in 2018, was the first iteration of the GPT (generative pre-trained transformer) series and the foundation of today’s ChatGPT.
Token: In artificial intelligence, a token is the basic, atomic unit of information that a large language model (LLM) reads, processes, and generates. Rather than processing entire words or raw sentences all at once, an AI breaks text down into smaller chunks —which can be a full word, part of a word (sub word), a number or even punctuation. 1 token is roughly 4 characters or about 0.75 words [~100 token + 75 words]. Every time a consumer interacts with a model like ChatGPT, the consumer entered a parallel financial universe where data is translated into a currency called Tokens. A single Token is roughly equivalent to four characters of text; …
The cost of running a prompt is calculated based on the Total number of tokens in the input (what a user sends to a model) plus the output (what the model sends back). Open AI charges for those tokens based on the model a person is using and how much context it can handle.
Example of Token setup: In early 2023, OpenAI’s published pricing for GPT – 4 was three cents (3c) per thousand tokens being input and six cents per thousand tokens being output. So, a stuffed prompt with thousand tokens worth of context, and the model responded with another thousand tokens worth of tutoring advice, that single interaction would cost nine cents (9c). This tiny 9c cost would be a gigantic bill upon scaling – and multiplying by millions of tutoring sessions. Computing power will continue to get cheaper but cheaper is still not Cheap.
From Claude – Sonet 4.6: A Sonet 4.6 also text users a 1M token context window. That is roughly 2 -3.
1 million Token ~ 3000 pages
Developer use it for high level coding, teachers use it for serious content, researchers tun it for document review, businessmen operate their day to day business. Similarly, Opus 4.6 model of Claude cost money ~ $5.0/per month. So, for Opus 4.6
1 million Token Context window = $5/per month
CPU, GPU, TPU
Central Processing Unit: The general -purpose brain of a standard computer, built for sequential tasks and everyday computing.
Graphics Processing Unit: Built for rendering images, later adapted to handle parallel AI math.
Tensor Processing Unit: a custom computer chip designed by Google to speed up artificial intelligence and machine learning work.
These handle massive matrix multiplications and tensor math, which are the building blocks of neural networks. They help large language models (LLMs) and deep learning systems learn from vast amounts of data much faster than standard chips. Also process real time requests for AI agents, search engines, and translation tools with low delay.
Google Search is a fully automated system that uses software called crawlers to explore the web, stores data in a massive database, and uses ranking algorithms to show the most relevant results in a fraction of a second, millions of times a day, everyday.
Onix, cofounded and led by a former WIRED contributor named David Bennahum, describes itself as a Substack for chatbots. Just as you subscribe to a writer on Substack, you can subscribe to an AI doppelganger of a celebrated expert, called an “Onix.” These bots are trained to conduct conversations with subscribers, delivering the provider’s expertise and advice like they would if you had a face-to-face appointment in their offices. The bots even attempt to project the unique personalities of the experts.