Tiny Language Models Come of Age

FromQuanta Science Podcast

Start listening View podcast show

Tiny Language Models Come of Age

FromQuanta Science Podcast

ratings:

Length:

21 minutes

Released:

Mar 6, 2024

Format:

Podcast episode

Description

To better understand how neural networks learn to simulate writing, researchers trained simpler versions on synthetic children’s stories. Read more at QuantaMagazine.org. Music is “Thought Bot” by Audionautix.

Released:

Mar 6, 2024

Format:

Podcast episode

Titles in the series (100)

In-depth news about mathematics, physics, biology and computer science.

Skip carousel

At Tiny Scales, a Giant Burst on Tree of Life: At tiny scales, a giant burst on tree of life. by Quanta Science Podcast
20 min listen
How Mutant Viral Swarms Spread Disease: A new understanding of viral swarms is helping researchers predict how viruses will evolve and where disease is likely to spread. by Quanta Science Podcast
28 min listen
New Letters Added to the Genetic Alphabet: Scientists hope that new genetic letters, created in the lab, will endow DNA with new powers. by Quanta Science Podcast
24 min listen
How Life and Luck Changed Earth’s Minerals: How life and luck changed Earth's minerals. by Quanta Science Podcast
34 min listen
How Humans Evolved Supersize Brains: Scientists have begun to identify the symphony of biological triggers that powered the extraordinary expansion of the human brain. by Quanta Science Podcast
37 min listen
A New Map Traces the Limits of Computation: A major advance in computational complexity reveals deep connections between the classes of problems that computers can — and can’t — possibly do. by Quanta Science Podcast
27 min listen
The Beasts That Keep the Beat: New insights from neuroscience — aided by a small zoo’s worth of dancing animals — are revealing the biological origins of rhythm. by Quanta Science Podcast
22 min listen
Math Quartet Joins Forces on Unified Theory: A new breakthrough that bridges number theory and geometry is just the latest triumph for a close-knit group of mathematicians. by Quanta Science Podcast
31 min listen
A Twisted Path to Equation-Free Prediction: Complex natural systems defy standard mathematical analysis, so one ecologist is throwing out the equations. by Quanta Science Podcast
32 min listen
A New Design for Cryptography’s Black Box: A two-year-old cryptographic breakthrough has proven difficult to put into practice. But new advances show how near-perfect computer security might be surprisingly close at hand. by Quanta Science Podcast
25 min listen
Scientists Debate Signatures of Alien Life: Searching for signs of life on faraway planets, astrobiologists must decide which telltale biosignature gases to target. by Quanta Science Podcast
25 min listen
Visions of Future Physics: Nima Arkani-Hamed is championing a campaign to build the world’s largest particle collider, even as he pursues a new vision of the laws of nature. by Quanta Science Podcast
33 min listen
Theorists Draw Closer to Perfect Coloring: A theorem for coloring a large class of “perfect” mathematical networks could ease the way for a long-sought general coloring proof. by Quanta Science Podcast
20 min listen
The Mutant Genes Behind the Black Death: Only a few genetic changes were enough to turn an ordinary stomach bug into the bacteria responsible for the plague. by Quanta Science Podcast
27 min listen
A Life in Games: The mathematician John Horton Conway’s myriad accomplishments — including the Game of Life, sprouts and the surreal numbers — are the product of a mind at play. by Quanta Science Podcast
34 min listen
The New Laws of Explosive Networks: How explosive networks form. by Quanta Science Podcast
22 min listen
How to Build Life in a Pre-Darwinian World: Perhaps chemistry played a more instrumental role in the origin of life than scientists thought. by Quanta Science Podcast
29 min listen
How the Body’s Trillions of Clocks Keep Time: Cellular clocks are almost everywhere. Clues to how they work are coming from the places that they’re not. by Quanta Science Podcast
31 min listen
Quantum Weirdness Now a Matter of Time: Bizarre quantum bonds connect distinct moments in time, suggesting that quantum links — not space-time — constitute the fundamental structure of the universe. by Quanta Science Podcast
34 min listen
A Surprise Source of Life’s Code: A surprise source of life’s code. by Quanta Science Podcast
34 min listen
Nature’s Critical Warning System: Scientists are homing in on a warning signal that arises in complex systems like ecological food webs, the brain and the Earth’s climate. Could it help prevent future catastrophes? by Quanta Science Podcast
34 min listen
The Incredible Shrinking Sex Chromosome: Nature offers species a panoply of ways to determine an organism’s sex. That flexibility suggests we need not be concerned about losing sex chromosomes, but it raises the question of why such a fundamental property is so variable. by Quanta Science Podcast
30 min listen
Mathematicians Discover Prime Conspiracy: A previously unnoticed property of prime numbers seems to violate a long-standing assumption about how they behave. by Quanta Science Podcast
26 min listen
Mongrel Microbe Tests Story of Complex Life: A newly discovered class of microbe could help to resolve one of the biggest and most controversial mysteries in evolution — how simple microbes transformed into the complex cells that produced animals, plants and fungi. by Quanta Science Podcast
30 min listen
Landmark Algorithm Breaks 30-Year Impasse: Computer scientists are abuzz over a fast new algorithm for solving one of the central problems in the field. by Quanta Science Podcast
30 min listen
Einstein’s Parable of Quantum Insanity: Einstein refused to believe in the inherent unpredictability of the world. Is the subatomic world insane, or just subtle? by Quanta Science Podcast
24 min listen
The Quantum Secret to Superconductivity: In a virtuoso experiment, physicists have revealed details of a “quantum critical point” that underlies high-temperature superconductivity. by Quanta Science Podcast
24 min listen
After Black Holes Collide, a Puzzling Flash: A satellite spotted a burst of light just as gravitational waves rolled in from the collision of two black holes. Was the flash a cosmic coincidence, or do astrophysicists need to rethink what black holes can do? by Quanta Science Podcast
22 min listen
New Clues to How the Brain Maps Time: The same brain cells that track location in space appear to also count beats in time. The research suggests that our thoughts may take place on a mental space-time canvas. by Quanta Science Podcast
26 min listen
Gravitational Waves Discovered at Long Last: This week: Long sought-after gravitational waves are found and a proposed fuzzball fix for a black hole paradox. by Quanta Science Podcast
35 min listen

More Episodes from Quanta Science Podcast

Skip carousel

New Cell Atlases Reveal Untold Variety in the Brain and Beyond: Scientists can fingerprint the function of tens of thousands or even millions of cells in a single experiment. This expansive data may fundamentally reshape our understanding of human anatomy and biology. by Quanta Science Podcast
21 min listen
Extra-Long Blasts Challenge Our Theories of Cosmic Cataclysms: Everything is made of stardust. Gamma-ray bursts may help physicists learn how each element in the periodic table is born in the cosmos. by Quanta Science Podcast
25 min listen
Meet Strange Metals: Where Electricity May Flow Without Electrons: When it comes to bizarre forms of electricity, superconductivity is just the beginning. by Quanta Science Podcast
21 min listen
In the Gut's 'Second Brain,' Key Agents of Health Emerge: Gut feelings, like the stomachache one might get from overeating, are driven by the enteric "second brain" in our digestive systems. by Quanta Science Podcast
17 min listen
During Pregnancy, a Fake 'Infection' Protects the Fetus: The placenta has a unique trick for keeping immune defenses on standby, even when no viral infections threaten a pregnancy. by Quanta Science Podcast
10 min listen
Why the Human Brain Perceives Small Numbers Better: A recent neurological study provides an unprecedented look at how number sense works in the human brain. by Quanta Science Podcast
21 min listen
Inside Scientists' Life-Saving Prediction of the Iceland Eruption: An Icelandic peninsula that hadn't seen any volcanism for 800 years has now awoken. Eruptions could continue for decades or even centuries. by Quanta Science Podcast
22 min listen
Echoes of Electromagnetism Found in Number Theory: Physics is a rich well of inspiration for mathematicians to draw from. by Quanta Science Podcast
21 min listen
Rogue Worlds Throw Planetary Ideas Out of Orbit: Astronomers have long known of rogue worlds - cosmic loners that drift through space, untethered to a star. Now JWST is spotting them in pairs. by Quanta Science Podcast
22 min listen
What Makes Life Tick? Mitochondria May Keep Time for Cells: We're all used to hearing about mitochondria as the powerhouse of the cell. It turns out that they are also the timekeepers. by Quanta Science Podcast
24 min listen
An Old Conjecture Falls, Making Spheres a Lot More Complicated: Many discoveries in mathematics make things simpler, but a recent disproof instead creates a universe of new, complicated possibilities. by Quanta Science Podcast
17 min listen
Even Synthetic Life Forms With a Tiny Genome Can Evolve: Biologists have discovered that even one of the weakest, simplest self-replicating organisms on the planet can evolve. by Quanta Science Podcast
15 min listen
Selfish, Virus-Like DNA Can Carry Genes Between Species: Genetic elements known as "Mavericks" may be responsible for transferring genes between different species throughout the history of life. by Quanta Science Podcast
13 min listen
Exoplanets Could Help Us Learn How Planets Make Magnetism: Earth has a magnetic field, but Venus, Earth's sister planet, doesn't. Why does one rocky planet have a magnetic field while the other does not? by Quanta Science Podcast
13 min listen
To Move Fast, Quantum Maze Solvers Must Forget the Past: If you've ever entered a room, and then completely forgotten how you got there, you may have something in common with a quantum computer. by Quanta Science Podcast
16 min listen
Underground Cells Make 'Dark Oxygen' Without Light: In the groundwaters beneath Alberta, Canada, researchers have found microbes that can produce abundant oxygen, even in the absence of light. by Quanta Science Podcast
14 min listen
How the Brain Protects Itself From Blood-Borne Threats: A system of sophisticated pumps and receptors protects the brain like bouncers at the door of an elite club. by Quanta Science Podcast
12 min listen
JWST Spots Giant Black Holes All Over the Early Universe: There's never been a more exciting time to study black holes. by Quanta Science Podcast
25 min listen
Is It Real or Imagined? How Your Brain Tells the Difference.: A recent neuroscience study reveals how human brains differentiate between the imagined and the observed. by Quanta Science Podcast
18 min listen
Chatbots Don't Know What Stuff Isn't: Researchers still don't understand whether machines will ever truly know the word "no." by Quanta Science Podcast
17 min listen
Global Microbiome Study Gives New View of Shared Health Risks: A recent study found that the more time you spend with someone, the more likely you are to share strains of species in your microbiome. by Quanta Science Podcast
21 min listen
Physicists Use Quantum Mechanics to Pull Energy out of Nothing: A recently demonstrated quantum protocol teleports energy from one location to another without violating any sacred physical principles. by Quanta Science Podcast
19 min listen
How Loneliness Reshapes the Brain: Loneliness doesn't just make people feel isolated. It alters their brain in ways that can hinder their ability to trust and connect to others. by Quanta Science Podcast
21 min listen
Gene Expression in Neurons Solves a Brain Evolution Puzzle: Our brains have even less in common with reptile brains than was once thought. by Quanta Science Podcast
20 min listen
Machines Learn Better if We Teach Them the Basics: To improve reinforcement learning algorithms, computer scientists are training them as if they were human. by Quanta Science Podcast
21 min listen
The Cause of Depression Is Probably Not What You Think: Better treatments for depression may require a clear-eyed look at the condition's nuance. by Quanta Science Podcast
22 min listen
Ants Live 10 Times Longer by Altering Their Insulin Responses: A tapeworm has evolved the ability to make its ant hosts live longer. Researchers recently discovered how the parasite does it. by Quanta Science Podcast
18 min listen
How the Brain Distinguishes Memories From Perceptions: The "blurriness" of remembered images may help our brains distinguish memories from current perceptions. by Quanta Science Podcast
14 min listen
What Causes Alzheimer's? Scientists Are Rethinking the Answer. (Pt 2): Researchers investigating alternatives to amyloid plaques as the possible cause of Alzheimer's disease have faced resistance, but intriguing new theories have emerged. by Quanta Science Podcast
41 min listen
What Causes Alzheimer's? Scientists Are Rethinking the Answer. (Pt. 1): For three decades, most Alzheimer's researchers have stuck by the theory that aggregations of proteins called amyloid plaques cause the disease. But unsatisfied with the slow pace of progress, some are now exploring other hypotheses. by Quanta Science Podcast
34 min listen

Related podcast episodes

Skip carousel

BI 023 Marcel van Gerven: Mind Decoding with GANs: Show notes: Donders Institute for Brain, Cognition and Behaviour Artificial Cognitive Systems on Twitter. Artificial Cognitive Systems research group. The paper we discuss: Generative adversarial networks for reconstructing natural images from brain ac by Brain Inspired
64 min listen
The brain implant that turns thoughts into text: A new neural interface lets people type with their mind, and a crafting journey into materials science. by Nature Podcast
26 min listen
Cracking the code: the neural basis of computer code comprehension: Have you ever wondered whether the brains of computer programmers are wired differently? Are there specific parts of the brain that are dedicated to computer coding? Scientists have outlined other networks in the brain, like those involved in language, b... by Science Rehashed
40 min listen
Audio long read: How does ChatGPT ‘think’? Psychology and neuroscience crack open AI large language models: To understand the 'brains' of LLMs, researchers are attempting to reverse-engineering artificial intelligence systems. by Nature Podcast
18 min listen
Computers are learning to read our minds: The home team chats with Gašper Beguš, director of the Berkeley Speech and Computation Lab, about his research into how LLMs—and humans—learn to speak. Plus: how AI is restoring a stroke survivor’s ability to talk, concern over models that pass the Turing test, and what’s going on with whale brains. by The Stack Overflow Podcast
30 min listen
Can Artificial Intelligence teach itself?: What happens if artificial intelligence starts learning from its own material? by CrowdScience
26 min listen
What is AI ”good” at (and what the heck is it, actually), with Josh Saxe by Lock and Code
45 min listen
The Naked Scientists strip down the brain in Spain: The Naked Scientists strip down the brain in Spain - attending the Federation of European Neuroscience Societies conference in Barcelona. We find out how your brain computes information, ask if watching worms can tell us about human social interactio... by Naked Neuroscience, from the Naked Scientists
10 min listen
Artificial Neural Nets Finally Yield Clues to How Brains Learn: The learning algorithm that enables the runaway success of deep neural networks doesn’t work in biological brains, but researchers are finding alternatives that could. by Quanta Science Podcast
20 min listen
BrainStuff Classics: Are Plants Conscious? by BrainStuff
6 min listen
TKC 527 Books Expert Brian O'Leary: Executive Director of the Book Industry Study Group Interview starts at 5:19 and ends at 36:00 “With respect to blockchain, there’s an opportunity to--I wouldn’t necessarily say disrupt--but to create an alternative to the closed ecosystems that... by The Kindle Chronicles
45 min listen
What are sperm whales saying? Researchers find a complex 'alphabet' by Short Wave
14 min listen
BI 124 Peter Robin Hiesinger: The Self-Assembling Brain: Support the show to get full episodes and join the Discord community. Robin and I discuss many of the ideas in his book The Self-Assembling Brain: How Neural Networks Grow Smarter. The premise is that our DNA encodes an algorithmic growth process by Brain Inspired
99 min listen
Cameron J. Buckner, "From Deep Learning to Rational Machines" (Oxford UP, 2023): An interview with Cameron J. Buckner by New Books in Science, Technology, and Society
71 min listen
Cameron J. Buckner, "From Deep Learning to Rational Machines" (Oxford UP, 2023): An interview with Cameron J. Buckner by New Books in Philosophy
71 min listen
Did AI Write This? by TechStuff
42 min listen
Do Monkeys Lack the Brains or the Vocal Anatomy for Human Speech? by BrainStuff
5 min listen
Currents 072: Ben Goertzel on Viable Paths to True AGI: Jim talks with Ben Goertzel about the ideas in his recent Substack essay "Three Viable Paths to True AGI." by The Jim Rutt Show
110 min listen
What if generative artificial intelligence became conscious? by European Parliament - EPRS Science and Technology podcasts
3 min listen
Can a machine read my mind?: How machine learning is enabling minds to start melding with machines by CrowdScience
43 min listen
BrainStuff Classics: How Can Squid 'Speak' With Their Skin? by BrainStuff
6 min listen
Do Our Brains Have Fingerprints Too? by BrainStuff
4 min listen
What’s it like to be an AI, with Anil Seth by London Futurists
44 min listen
BI 183 Dan Goodman: Neural Reckoning: Support the show to get full episodes and join the Discord community. You may know my guest as the co-founder of Neuromatch, the excellent online computational neuroscience academy, or as the creator of the Brian spiking neural network simulat by Brain Inspired
89 min listen
Can we grow a conscious brain?: With advances in stem cell research, could we grow a human brain? And would we want to? by CrowdScience
27 min listen
How Does the Wayback Machine Work? by BrainStuff
7 min listen
744 - I'm Not That Smart by Tiny Leaps, Big Changes
23 min listen
Lethal Collection: "What if we told you we could back up your mind?" That's the business pitch of Nectome, a preserve-your-brain-and-upload-it company. The catch? They have to kill you first. Its chemical solution can keep a body intact for hundreds of years, maybe thousand by We Have Concerns
21 min listen
Welcome to AI Kindergarten: A Path to Machine Genomes by The AI in Business Podcast
25 min listen
Episode 146: Gaurav Venkataraman discusses memory in DNA and RNA: Can memories be stored outside the brain? by Elucidations
40 min listen

Discover this podcast and so much more

Tiny Language Models Come of Age

Tiny Language Models Come of Age

Description

Titles in the series (100)

More Episodes from Quanta Science Podcast

Related podcast episodes