Security Expert (Me) Says, “We Can Now Spy On Human Emotions”

Emotional surveillance has an undeniably dystopian vibe, like George Orwell’s 1984, but it’s not science fiction. Security Expert (Me) Says, “We Can Now Spy On Human Emotions”

Banks are already signing up for services that incorporate it into their analysis of behavior. A startup founded by MIT graduates called Humanyze has created a sensor-laden badge that transmits data on speech, activity, and stress patterns.

New Evidence For The Vibration Theory Of Smell

Could This Be The Digital Nose Of The Future?

Achieving Optimum Health By Understanding Biological Frequencies

Essential Oils User’s Guide

Using Smells To Boost Learning, Enhance Mood And Memory

One of these days, the walls may know when you’re happy, sad, stressed or angry by using an experimental device unveiled Tuesday by researchers at the Massachusetts Institute of Technology that uses wireless signals to recognize emotions through subtle changes in breathing and heartbeat.

Computer scientist Dina Katabi and her colleagues at the university’s Computer Science and Artificial Intelligence Lab developed a radar system for vital signs that uses reflected radio signals to track movements, moods and behavior, with potential applications for smart homes, offices and hospitals.

They posted their new research online Tuesday and plan to present their test results next month at a mobile-computing conference in New York.

These wireless signals—a thousand times less powerful than conventional Wi-Fi—are designed to bounce off anyone within range, capturing variations in vital signs that can be analyzed quickly by a computer algorithm able to detect emotional states, the researchers said.

To distinguish one mood from another, their system measures patterns of respiration, cardiac rhythms, and minute variations in the length of each individual heartbeat.

“All of us share so much in how our emotions affect our vital signs,” said Dr. Katabi. “We get an accuracy that is so high that we can look at individual heartbeats at the order of milliseconds.”

The system, which they call EQ-Radio, is 87% accurate at detecting whether a person is joyful, angry, sad or content, they said.

By providing an accurate readout of moods, the system promises to loop people more directly into wireless sensor networks, the researchers said.

While still experimental, the system could one day give buildings the capacity to respond automatically to changes in vital signs among the people living or working in them, without a need for explicit commands or a direct link to a body sensor, the researchers said.

A hospital emergency room might automatically monitor patients awaiting treatment. An amusement park might modulate special effects by monitoring the involuntary reactions of people on an exhilarating ride.

A house might one day react to a family’s stress by playing pleasant music.

“We have explored this idea of allowing a home to recognize someone’s emotions and adapt to it,” said project researcher Fadel Adib. “The idea is to enable you to seamlessly interact with your home.”

The team is already testing an earlier version of the system that tracks movements and behavior in about 15 homes in the Boston area, including that of Dr. Katabi.

She uses it to monitor her sleep patterns and eating habits. It can track movements even if the person is in another room.

“I would really like future homes to be more health aware,” she said.

In the research made public Tuesday, Dr. Katabi and her colleagues tested the wireless system on 10 women and 20 men, between 19 and 77 years old, while in a standard office setting, which contained desks, chairs, couches and computers.

During the tests, the volunteers sat from three to 10 feet away from the wireless sensors while attempting to evoke specific emotions by recalling emotion-rich memories. As a control, their vital signs during the experiment were also monitored using conventional electrocardiography and a video-based emotion recognition system that homes in on facial expressions.

All told, the researchers collected measurements of 130,000 individual heartbeats. To classify the mood changes, the computer employed a machine learning algorithm to match the waveforms within each heartbeat.

When they compared results, they found that the experimental system was almost as accurate in recognizing changes in emotion as the electrocardiograms. It was about twice as accurate as the facial cues recorded by the video system, they said.

“We use the wireless signal to obtain the changes in the vital signs and then run a machine learning algorithm to get to emotions,” she said. “The algorithm can immediately recognize the emotions of someone new.”

Wall Street Uses Technology To Spy On Traders Emotional State

The trader was in deep trouble. A millennial who had only recently been allowed to set foot on a Wall Street floor, he made bad bets, and in a panic to recoup his losses, he’d blown through risk limits, losing $4.9 million in a single afternoon.

It wasn’t a career-ending day. The trader was taking part in a simulation run by Andrew Lo, an MIT finance professor. The goal: find out if top performers can be identified based on how they respond to market volatility.

Lo had been invited into the New York-based global investment bank—he wouldn’t say which one—after giving a talk to its executives. So in 2014, unknown to the outside world, he rigged a conference room with monitors to create a lab where 57 stock and bond traders lent their bodies to science.

Banks have already set up big-data teams to harvest insights from the terabytes of customer information they possess. Now they’re looking inward to see whether they can improve operations and limit losses in their biggest cost center: employees.

Companies including JPMorgan Chase and Bank of America have had discussions with tech companies about systems that monitor worker emotions to boost performance and compliance, according to executives at the banks who didn’t want to be identified speaking about the matter.

As machines encroach on humans’ role in the markets, technology offers a way to even the fight. The devices Lo used—wristwatch sensors that measure pulse and perspiration—could warn traders to step away from their desks when their emotions run wild.

They could also be used to screen hires to find those whose physiology is best suited to risk-taking—what interested the bank that allowed the MIT study.

The most promising application, and the one with the most profound privacy issues, would be for keeping tabs on employees, Lo says. Risk managers could use it to spot problems brewing on a specific desk, such as unauthorized trading, before too much damage is done.

“Imagine if all your traders were required to wear wristwatches that monitor their physiology, and you had a dashboard that tells you in real time who is freaking out,” Lo says. “The technology exists, as does the motivation—one bad trade can cost $100 million—but you’re talking about a significant privacy intrusion.”

Emotional surveillance has an undeniably dystopian vibe, like a finance version of George Orwell’s 1984, but it’s not science fiction. Banks are already signing up for services that incorporate it into their analysis of behavior.

A startup founded by MIT graduates called Humanyze has created a sensor-laden badge that transmits data on speech, activity, and stress patterns.

Microphones and proximity sensors on the gadgets help employers understand what high-performing teams are doing differently from laggards. The Boston-based company is close to announcing a deal with a bank that’s moving some employees to new offices, according to Chief Executive Officer Ben Waber.

The bank wants to use Humanyze badges to determine seating locations for traders, asset managers, and support staff to improve productivity, he says.

Another startup, Behavox, uses machine-learning programs to scan employee communications and trading records. Emotional analysis of telephone conversations is a part of a worker’s overall behavioral picture, according to founder Erkin Adylov, a former Goldman Sachs research analyst.

When a worker deviates from established patterns—shouting at someone he’s trading with when previous conversations were calm—it could be a sign further scrutiny is warranted. “Emotion recognition and mapping in phone calls is increasingly something that banks really want from us,” says Adylov, whose company is based in London. “All the things you do as a human are driven by emotions.”

Emotions are reflexes that developed to drive behavior, scientists say, improving our prospects of seizing opportunity and surviving risk. They’re accompanied by measurable physiological changes such as increased blood pressure, sweating, and a pounding heart.

Their role in investing has been established since at least the time of economist Benjamin Graham, the father of value investing.

More recently, John Coates, a University of Cambridge neuroscientist and former derivatives trader, has studied how financial risk takers’ decisions are influenced by biology. His experiments, chronicled in a 2012 book, The Hour Between Dog and Wolf, show that hormones such as testosterone and cortisol play a part in exacerbating booms and busts.

The volunteers in Lo’s study were given a $3 million risk limit and told to make money in markets including oil, gold, stocks, currencies, and Treasuries. They came from across the bank’s fixed-income and equity desks and ranged from junior employees to veterans with 15 years of experience.

Top traders have a signature response to volatility, says Lo, who plans to publish his findings by next year. Rather than being devoid of feeling, they are emotional athletes. Their bodies swiftly respond to stressful situations and relax when calm returns, leaving them primed for the next challenge. The top performer made $1.1 million in a couple of hours of trading.

Those who fared less well, like the trader who lost almost $5 million, were hounded by their mistakes and remained emotionally charged, as measured by their heart rate and other markers such as cortisol levels, even after the volatility subsided.

Lo’s findings suggest there’s a sweet spot for emotional engagement: too much, and you’re overly aggressive or fearful; too little, and you aren’t involved enough to care. Veteran traders had more controlled responses, suggesting that training and experience count.

There are other ways to infer emotional states. Researchers led by Kellogg School of Management professor Brian Uzzi pored over 1.2 million instant messages sent by day traders over a two-year period.

They found that, as in Lo’s study, having too much or too little emotion made for poor trades. Uzzi, whose study was published this year, says he’s working with two hedge funds to design a product based on the research.

As younger traders accustomed to biometric devices like the Fitbit enter the industry, applications designed to boost performance and monitor employees will become commonplace, says Lo, who expects it to be widespread in less than 10 years.

“The more data we have, the more we’re able to characterize the emotional state of the individual,” he says. “Everybody will have to have these kinds of analytics.”

Detecting Emotions In Thin Air

One of the most writerly things a person can do is to characterize air as thick, or emotions as tangible. Sadness lingers in the air. The best dinner parties are powered by palpable tension.

The practice suggests that you are keenly attuned to your surroundings. Beyond observant, you use your senses in ways others had not thought possible. That is why people want to have sex with writers.

But if you told me that the air is actually transmitting chemical signals that influence emotions between humans, I would add you to a list that I keep in my head. It’s not a bad list, per se, but it is titled “Chumps.”

One person who would not be on that list is Jonathan Williams. An atmospheric chemist, he describes himself as “one of those wandering scientific souls,” but not in an annoying way.

He maintains a jovial British lilt after moving to Colorado to work at the National Oceanic and Atmospheric Administration, and then to Germany for a job with the Max Planck Institute (which describes itself as “Germany’s most successful research organization”). There Williams and his colleagues study air.

They focus on gases that come from vegetation in the tropics, as well as carbon industry. In doing so, the chemists use finely calibrated machines that sense the slightest changes in the contents of air. Taking measurements in the field, Williams and his colleagues always noticed that when they themselves got too close to the machines, everything went haywire.

That made sense, in that humans are bags of gas. As breathing people know, we tend to emit carbon dioxide. (Though each exhalation still contains about four times as much oxygen as carbon dioxide.) And there are many subtler ingredients in the concoctions we breathe out.

So Williams began to wonder, are these gases “significant on a global scale”? Could they be, even, contributing to climate change? Especially as the number of humans on Earth rockets toward 8 billion?

The answer was no. Just a clear, simple no. By measuring gases in soccer stadiums, the Planck chemists found no consequence of human breath. There might be some effect at a global scale, but it’s just nothing compared to the air-ravaging effects of transportation and agriculture.

But Williams didn’t come away from the stadium empty handed. As he sat and watched the fluctuating readings on the air sensors, he got an idea.

In the manner of a typical European soccer crowd, the people went through fits of elation and anger, joy and sorrow. So Williams began to wonder, as he later put it to me, “Do people emit gases as a function of their emotions?”

If we do, it wouldn’t be unprecedented. Tear some leaves off of a tree, for example, and it will emit chemical signals that may be part of a system of communication between trees. The behavior for bees and ants is clearly chemically dominated.

“We’re not like that—not like robots following chemicals,” Williams explained. “But it could be possible that we are influenced by chemicals emitted by other humans.”

The idea of airborne pheromones—chemicals that specifically influence mating behaviors— has been a source of much fascination, but the actual evidence is weak. Some small studies have suggested an effect when people put cotton balls under their armpits, and then other people smell the balls—but in minor, unreliable ways.

“I don’t know why so many previous researchers have been so into armpits,” said Williams. “A much better way to communicate would be through your breath. Because you can direct your breath, and your breath is at roughly the same height as the person you’re trying to communicate to, silently. In the dark, maybe, in your cave.”

And if these behavior-modifying volatile chemicals exist (volatile meaning anything that goes into the air), then why would they be limited to sex? Why shouldn’t we be able to signal fear or anxiety? It is true that birds seem to know that I’m afraid of them.

Williams was so intrigued by the idea of gases and emotion that he designed another experiment—something more predictable than a German soccer game. This time he used a movie theater. Unlike the open-air stadium, the theater presented fewer variables.

“You’ve got this box, the cinema, and you spool through air from outside at a continuous rate, and you have 250 people sitting there, not moving. And you show them all, simultaneously, something that should make them frightened or anxious or sad, or whatever.”

The changes in any one person’s breath might be minuscule, but a crowd of breathers could be enough to overcome the rest of the background signals.

And more importantly, unlike a soccer match, the experiment could be done with the same film again and again. This could test the reproducibility of findings, which is critical to science.

Rigging a mass spectrometer into the outflow vent of the theater, the Kino Cinestar in Mainz, Williams had a sense that the experiment as something of a lark. “I thought, we’re probably just going to get a big mixture of popcorn and perfume,” he said.

But, nonetheless, to measure relationships between scenes and gases, his team meticulously mapped out and labeled every scene in 16 films—from beginning to end. In 30 second increments, the team labeled each by its quality (kiss, pet, injury), as well as its emotional elements using a finite set of descriptors.

Their efforts were not entirely wasted. They published the findings this month in Scientific Reports, an open-access journal published by Nature.

After measuring more than 100 chemicals in the theater air from 9500 filmgoers, the team saw some changes that stood out—at the same points, in almost every showing of some films.

In Hunger Games: Catching Fire, for example, during the “suspense” scenes—when Jennifer Lawrence was in particular danger—the carbon dioxide, acetone, and isoprene levels in the theater air predictably increased.

The researchers speculate that this may have something to do with breath-holding, or stress hormone production—but it is all speculation.

The important point was that the signals occurred at exactly the same time in all four screenings of the film. They also found the reproducible changes in the air chemistry during “humor” scenes in other films.

It’s impossible to say whether the changes in the air are signals to one another, or simply byproducts of emotion-based reactions. To Williams, that’s “the billion-dollar question.”

But he is guarded in his excitement. “We have shown there is this invisible, inaudible concert of chemicals that changes regularly in the auditorium. We haven’t shown that people can detect them. But, of course, if a signal is there, then maybe it does.”

“The authors do make a very important point about the effects of stress or anxiety on human emissions,” said Ben De Lacy Costello, a senior research fellow at the University of West England. He created a catalog of all the chemicals we emit and found at least 1840.

Though it was published in The Journal of Breath Research, the list included volatile organic compounds (VOCs) coming from many parts of healthy people: 359 in saliva, 154 in blood, 256 in breast milk, 532 in skin secretions, 279 in urine, and 381 in feces—in addition to 872 in our breath.

Williams and Costello, among others who study air and perception, refer to the volatile chemicals we humans emit as the volatolome. It’s a linguistic construction akin to our genes collectively being called our genome, and our microbes making up our microbiome.

Interesting as movie theaters and pheromones might be, VOCs are potentially useful in innumerable practical ways. A 2016 study found that the breath volatolome might be helpful in making the critical medical diagnosis of pulmonary-arterial hypertension.

And some dogs have proven capable of smelling certain cancers—presumably because the metabolically altered tumor cells produce unique byproducts. Based on that, entrepreneurs have attempted to create artificial “cancer-sniffing” noses.

Costello believes that once fully understood (if it ever is), the number of chemicals in the volatolome may be in the tens of thousands.

Many will come, like the smells in our armpits and on our breath, from the huge range of microflora on and in the human body.

The possibilities for disease screening and detection are enormous.

Among healthy people, it might be possible to measure these gases to track other bodily changes not just in diseases, but normal responses to food and exercise and stress.

As Costello posits, “It could be useful to detect stress, for example, in rescue situations such as earthquakes, monitoring crowds of people, terrorists in airports, et cetera.”

Media, too, change our moods by changing the chemistry inside us, and so changing the chemistry we emit. Humans are notoriously unreliable when it comes to research that requires introspective self-report, but your gases won’t lie.

Though we’re far from it now, some day when that special person tries to tell you that the two of you just don’t have chemistry, you might be able to refute that with data.

Security Expert (Me) Says, “We Can Now Spy On Human Emotions”

Related:

Related Articles:

A List Of Top 10 African Women To Watch In Blockchain And Bitcoin

Vast Troves of Classified Info Undermine National Security, Spy Chief Says

Your Questions And Comments Are Greatly Appreciated.

Leave a Reply Cancel reply