This AI finds patterns in visual brain activity and turns them into pictures
The system, called Brain-IT, uses data from functional magnetic resonance imaging (fMRI) to measure activity in the visual cortex, the part of the brain that processes visual input.
Key points
- It sorts thousands of tiny brain-scan measurements from the visual cortex into 128 groups based on the kinds of visual information they tend to carry.
- "The key idea is that every voxel, a tiny cube of brain tissue, gets its own learned vector that captures what it cares about in images," said study first author Roman Beliy, a doctoral candidate in the Department of Computer Science and Applied Mathematics at the Weizmann Institute of Science in Israel.
- In other words,it asks "based on this brain activity, what was the subject looking at?".
Which parts of this image are edges?" The Universal Brain Encoder starts with an image and predicts brain activity; Brain-IT starts with measured brain activity and reconstructs the image the person saw.
Additional details
- That model begins with an image and then predicts the fMRI response that would be expected in individual "voxels" — small 3D units — in a brain scan.
- Brain-IT surpassed prior leading methods on standard objective reconstruction metrics and produced more faithful results, the study authors wrote.
- (Image credit: Brain-IT) How scientists tested the model Brain-IT's synthetic images were compared with the images that participants saw.
- An artificial intelligence (AI) tool can use brain scans to recreate the images people looked at with eerie accuracy.
- The system then lets those groups share clues with one another, allowing it to combine, for example, information about a scene's shape, color and possible objects, to build a more complete picture of the image, Beliy explained.
How this summary was made: extractive summary — built from the sources' own reporting. It is drawn only from the reporting of the sources listed below — read the originals for full detail. How we summarize →
Advertisement
Every article behind this summary — 1 article from 1 outlet
Get stories like this in your inbox.
Subscribe — it's freeOne free email at 6am — your topics, deduplicated, five minutes. No account, no paywall.