Arvind Sathi auth. Cognitive Internet of Things Collaboration to Optimize Action

Prévia do material em texto

Cognitive (Internet of) Things 
 
 Arvind   Sathi 
 Cognitive (Internet of) 
Things 
 Collaboration to Optimize Action 
 ISBN 978-1-137-59465-5 ISBN 978-1-137-59466-2 (eBook) 
 DOI 10.1057/978-1-137-59466-2 
 Library of Congress Control Number: 2016953144 
 © Th e Editor(s) (if applicable) and Th e Author(s) 2016 
 Th is work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the 
whole or part of the material is concerned, specifi cally the rights of translation, reprinting, reuse of illustrations, 
recitation, broadcasting, reproduction on microfi lms or in any other physical way, and transmission or 
information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar 
methodology now known or hereafter developed. 
 Th e use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does 
not imply, even in the absence of a specifi c statement, that such names are exempt from the relevant protective 
laws and regulations and therefore free for general use. 
 Th e publisher, the authors and the editors are safe to assume that the advice and information in this book are 
believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors 
give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions 
that may have been made. 
 Cover illustration: Jacket image by sputnikos/iStock /Getty Images Plus 
 Printed on acid-free paper 
 Th is Palgrave Macmillan imprint is published by Springer Nature 
 Th e registered company is Nature America Inc. New York 
Th e registered company address is: 1 New York Plaza, New York, NY 10004, U.S.A.
 Arvind   Sathi 
 IBM 
 Irvine , California , USA 
 To my wife Neena, my chief historian 
 
vii
 My fi rst real introduction to artifi cial intelligence (AI) was in the Spring of 
1983, when Mark Fox gave a talk to the graduate students at Carnegie Mellon 
University’s (CMU) business school. Mark inspired me into imagining a 
future cognitive business organization and was well connected with business 
problems in manufacturing. I was searching for a Ph.D. research topic and 
eventually gathered the courage to walk up to his offi ce at CMU’s Robotics 
Institute and beg for a research position. He was at that time the director of 
Intelligent Systems Lab and was working on a number of research projects to 
explore a knowledge representation language named Schema Representation 
Language (SRL). Th e Robotics Institute and the Computer Science depart-
ment provided me with a wonderful exposure to a grand vision of Cognitive 
Th ings––including vision, mobile robots, speech analysis, and factory auto-
mation. Over the next three years, I got a chance to meet and learn from some 
of CMU’s masters in the fi eld––Herb Simon, Raj Reddy, Mark Fox, Jaime 
Carbonell, John McDermott. Pat Langley, John Anderson, and many more. 
Mark also was the one to introduce me to Distributed Artifi cial Intelligence, 
which inspired the concept of Cognitive Th ings. At that time, I was deal-
ing with Franz Lisp running on Unix machines with minimal interfaces to 
other agents. My discussions with Rick Rashid (who later founded Microsoft 
Research in 1991) and Alfred Specter (who built distributed engines for 
Transarc, and ended up working for IBM and Google) on distributed pro-
cessing were invaluable in shaping my thoughts in this area. A number of 
workshops in Distributed AI provided me with key concepts represented in 
this book, and led me to valuable information exchanges with Victor Lesser, 
Reid Smith, and Tom Malone. 
 Acknowledgements 
viii Acknowledgements
 Mark Fox asked me to teach a course on SRL at Digital Equipment in 
1984, and thus started my relationship with the Carnegie Group, an AI 
startup formed with equity capital from Digital Equipment, GSI, Texas 
Instruments, Ford Motor Company, Boeing, and U S WEST.  I had the 
opportunity to work with the AI groups at each of these equity partners, and 
also to work with a number of AI scientists, who were brave enough to work 
on industrial applications––Gary Kahn, who worked on an early version of 
the Field Service advisor, Phil Hayes (now working at IBM Watson group), 
who worked on language translation, and Vijay Saraswat (now with IBM 
Research), who introduced me to Prolog. In 1987, BusinessWeek declared AI 
winter, but Carnegie Group kept building business applications of AI, and 
eventually went through an IPO in 1995. Th is was a valuable experience in 
learning how to commercialize AI, though it was ahead of its time. 
 Watching the buzz for Cognitive Computing today is like viewing the 
sleeping beauty (frozen by the curse of AI winter back in the 1980s) awak-
ening. I am grateful to the IBM senior leadership for taking the bold move 
towards artifi cial intelligence, and facilitating the much-needed focus towards 
successful implementation. I would to thank Ravesh Lala for pulling me into 
his team and bringing my youth back. Swami Chandrasekaran and I have 
worked with each other since 2001, but never had so much fun with prod-
ucts and ideas. Many of the ideas represent our joint work and I would like 
to thank Ravesh and Swami for countless brainstorms. Th e Cognitive Value 
Assessment team at IBM brings the best of business and technology talent and 
has contributed to the use cases presented here. I would like to thank Grady 
Booch for his presentation, which helped me shape the defi nition. A number 
of my peers at IBM have helped me with this book. Th ey include, among oth-
ers, Aadhar Garg, Ayan Bhattacharya, Bart Emanuel, Bjorn Austraat, Brian 
Roumpz, Chris Loza, Christine Twiford, Dakshi Agrawal, Doug Barton, 
Eliana F Bennett, Eric Riddleberger, Janki Vora, Khalid Behairy, Manish 
Sampat, Manprit Singh, Mathews Th omas, Mikhail S Gorbatovskiy, Oliver 
Blodgett, Onye Ifemedebe, Owen Coelho, Rahul Kurane, Rajiv Phougat, 
Rich Lanahan, Sham Vaidya, Steve Trigg, Susan Visser, Tommy Eunice, and 
Shuvanker Ghosh. A number of IBM clients, educators, and business part-
ners have helped me shape ideas, including Sofi a Gomes, Anand Sekhar, Ash 
Kanagat, Dave Dunmire, Girish Varma, Jamie Williams, Joshua Koran, JP 
Benini, Manav Mishra, Maureen Little, Raj Singh, Sanjeev Dewan, Sumit 
Chowdhury, Vinodh Swaminathan, and Von McConnell. I would like to 
thank IBM senior leadership for all the publicly available media referred to 
in this book. 
 Acknowledgements ix
 In my last two books, I sought help from third party artists to draw some 
cartoons using my ideas. My cartoons were not the best, though the very idea 
of using cartoons to convey some of the ideas was appealing. Fortunately, I 
came across Sunil Agarwal , a fellow IITian, who has been creating cartoons 
for Times of India . I started subscribing to his email subscriptions, and found 
his cartoons were very much connected with technical topics. He has gra-
ciously allowed me to share some of his cartoons through my work, and I have 
included a number of his cartoons in this book. 
 I would like to thank Laurie Harting, Marcus Ballenger, and Steve 
Partridge for all their support in editorial review, and publication help, and 
to IBM Marketing––Will Reilly, Doug Barton, Susan Visser, and Gaurav 
Deshpand––for their help in accessing material for the book. I would like to 
acknowledge the timely and detailed reviews by Bjorn Austrat and Christine 
Twiford. My daughter Kinji Sathi is the most patient and detailed reviewer 
of my writing, and has provided valuable assistance in improving the writing 
quality. My son, Conal Sathi, provided a thoroughtechnical review to keep 
me honest. I would like to thank my family, Oliver, Clark, Conal, Kevin, 
Kinji, Neena, and my dad for their support during the long months of book 
writing and for giving me valuable ideas for case studies. 
 
xi
 Contents 
 1 Introduction 1 
 1.1 Background 1 
 1.2 What Are Cognitive Th ings and How Do Th ey Function? 3 
 1.3 Book Outline 9 
 1.4 Target Audience 12 
 1.5 Summary and What’s Next 12 
 2 What Is a Cognitive Device? 13 
 2.1 Background 13 
 2.2 Candidate Devices 14 
 2.3 Cognitive Device Operation 21 
 2.4 Cognitive Device Engineering 23 
 2.5 Blockchain for Transaction Management 25 
 2.6 Chapter Summary 26 
 3 Cognitive Devices as Human Assistants 29 
 3.1 Introduction 29 
 3.2 Shopping and Buying Assistant 31 
 3.3 Care Assistant 33 
 3.4 Travel and Entertainment Assistant 35 
 3.5 Administrative Assistant 37 
 3.6 Chapter Summary 39 
xii Contents
 4 Cognitive Th ings in an Organization 41 
 4.1 Introduction 41 
 4.2 Smarter Operation 44 
 4.3 Smarter Engineering 49 
 4.4 Contextual Marketing 53 
 4.5 Proactive Customer Care 55 
 4.6 Counter Fraud Management 58 
 4.7 Chapter Summary 58 
 5 Reuse and Monetization 61 
 5.1 Introduction 61 
 5.2 Weather Services 62 
 5.3 Media Viewership Services 64 
 5.4 Location Services 66 
 5.5 Location Data and Contextual Marketing 68 
 5.6 Th e New Advertising Market Place 70 
 5.7 Monetization Candidates and Criteria 73 
 5.8 Cognitive Monetization 75 
 5.9 Chapter Summary 76 
 6 Intelligent Observations 79 
 6.1 Introduction 79 
 6.2 Sense 80 
 6.3 Observe 82 
 6.4 Listen 84 
 6.5 Crawl 86 
 6.6 Visual Recognition 87 
 6.7 Identity Resolution 88 
 6.8 Chapter Summary 90 
 7 Organization of Knowledge and Problem-Solving 93 
 7.1 Introduction 93 
 7.2 Organizing Solution Space 95 
 7.3 Text Analysis 99 
 7.4 Profi le Enrichment 102 
 7.5 Automated Problem Solving 102 
 7.6 Adaptive Real-Time Decision-Making 105 
 7.7 Chapter Summary 108 
 Contents xiii
 8 Installation, Training, Maintenance, Security, 
and Infrastructure 111 
 8.1 Introduction 111 
 8.2 Installation and Maintenance 113 
 8.3 Training 114 
 8.4 Security and Privacy 116 
 8.5 Centralized or Distributed Architecture 119 
 8.6 Cloud or On-Premise Infrastructure 121 
 8.7 Chapter Summary 122 
 9 Machine-to-Machine Interfaces 125 
 9.1 Introduction 125 
 9.2 Communication Media 126 
 9.3 Identity Management 129 
 9.4 Information Governance 131 
 9.5 Negotiation 133 
 9.6 Chapter Summary 135 
 10 Man-to-Machine Interfaces 137 
 10.1 Introduction 137 
 10.2 Authentication 139 
 10.3 Cognitive Interaction 142 
 10.4 Emotions, Creativity, and Hidden Meanings 146 
 10.5 Negotiation/Disambiguation 148 
 10.6 Chapter Summary 149 
 11 Assisting in Human Communications 151 
 11.1 Introduction 151 
 11.2 Information Integration and Discovery 152 
 11.3 Alternative Generation and Prioritization 154 
 11.4 Conversation Assistance 156 
 11.5 Organization Communication 157 
 11.6 Chapter Summary 159 
 12 Balance of Power and Societal Impacts 161 
 12.1 Introduction 161 
 12.2 Displacement of Cognitive Jobs 163 
 12.3 Who Is the Winner? 166 
xiv Contents
 12.4 Regulatory Versus Consumer Privacy 167 
 12.5 Changing Role of Machines and Humans in Families 
and Organizations 169 
 12.6 Organization Design, Policy Management, 
Change Management 171 
 12.7 New Skills and Shortage Areas 172 
 12.8 Chapter Summary 174 
Index 177 
xv
Fig. 3.1 Ivy in action 37
Fig. 4.1 Ollie: Th e fi rst driverless vehicle to integrate IBM Watson 52
Fig. 4.2 Etisalat location data monetization 54
Fig. 4.3 Proactive care scenario 56
Fig. 5.1 Advertising in the broadcasting era 71
Fig. 5.2 Today’s advertising market place 71
Fig. 5.3 Advertising market participants 72
Fig. 7.1 Discover, detect, decide, and drive using advanced 
analytics platform 106
Cartoon 8.1 Dilbert’s security (Scott Adams, “Dilbert”, 
February 28, 2016, http:// dilbert.com/strip/2016-02-28, 
reprinted with 
permission) 117
Cartoon 8.2 Would you like to receive pop-up ads from your toaster? 118 
Fig. 8.1 Collaboration across distributed sources 120
Fig. 10.1 CogniToys from Elemental Path 138
Cartoon 10.1 Remember the Password 139
Cartoon 10.2 Robot with consciousness (Scott Adams, “Dilbert”, 
November 23, 2015, http://dilbert.com/strip/2015-11-23 
reprinted with permission) 146
Fig. 10.2 Watson Personality Insights 148
Fig. 12.1 Cognitive employment trends 165
Cartoon 12.1 Role of parent vs. driverless car 170
Cartoon 12.2 Interaction skills and the next generation 174
List of Figures
 
xvii
List of Tables
Table 1.1 Cognitive operators 8
Table 4.1 Sample organizational use cases 43
Table 7.1 Knowledge Representation Layers 98
Table 7.2 Sample data dimensions and data set 103
Table 9.1 OSI layers 128
Table 10.1 Comparison of authentication technologies 142
1© Th e Editor(s) (if applicable) and Th e Author(s) 2016
A. Sathi, Cognitive (Internet of ) Th ings, 
DOI 10.1057/978-1-137-59466-2_1
 1 
 Introduction 
1.1 Background 
 I have always taken pride in an active life and a balanced diet. However, my belly 
fi nally reacted to the extended time in front of a computer screen and the nearby 
kitchen pantry stocked with delicious munchies. My daughter, an internal medi-
cine doctor, was concerned. She asked me if I had a sizable decrease in exercising, 
and I gave my usual pitch about my regular exercise pattern. She was not satis-
fi ed with my response and gave me a FITBIT (R) as a Christmas gift. Within a 
week of my wearing the Fitbit , she was “taunting” me about my lack of exercise. 
 Exercising is a very personal activity, and yet it is highly infl uenced by our 
social network. People traditionally spend a lot of time making friends to exercise 
with, this network being typically limited to a geographical proximity. Fitness 
trackers have extended that concept to the wider social network, or to a world-
wide level using Internet connectivity. Friends and family members can observer 
each other’s exercise patterns and share comments with each other. Fitness track-
ers combine fi ve capabilities to build a collaborative exercising behavior.
 1. Th ey use sensors to collect event data from me automatically . As long as I 
wear the Fitbit , it is collecting data. In doing so, it is reasonably accurate 
and very user friendly. Even my 93-year-old dad is able to use a Fitbit to 
collect his movement data. 
 2. Th e tracker itself is a small low energy-consuming device connected using a 
Bluetooth®. Once charged, it lasts for a long time and gives me a fair 
 warning before running out of battery. It uses a gateway to poll the data 
into a central storage. 
 3. Th e central storage collates data about me and about my social network. It 
has the capability for storing raw data, aggregating it over time, comparing 
it to my social group and communicating the results to those in mynet-
work. Each of us may have diff erent social networks. It provides me with 
information not only about people connected to me but also to their 
friends, thereby giving me an opportunity to make new exercise friends. 
 4. My social network is able to collaborate with me in my exercising by viewing 
shared exercise data and then in turn encouraging me through comments. 
 Fitbit provides us with emoticons (“cheer” and “taunt”) to share with each 
other. In the example above, my daughter was using a “taunt” that showed 
up as a SMS text on my cell phone. 
 5. I can utilize professional consulting off ered by the software, which uses 
knowledge of medicine and fi tness activities. Under Armour’s Record TM 
app monitors my exercise, weight, calories burned, heart rate, and eating 
patterns and provides me with expert advice using IBM ’s Watson TM . 
 Potentially, this information could be shared with my primary care physi-
cian who can use it for monitoring my health, my health insurance provider 
who can give me discounts for healthy living, and with sports marketers who 
can use it for targeting campaigns for sports shoes. Many employers are subsi-
dizing the cost of acquiring a Fitbit , Apple watch or other monitoring devices, 
as they perceive the value in driving healthy living programs among their 
employees. Th us, fi tness trackers are enabling both collaborative exercising 
and health monitoring. In doing so, these Cognitive Th ings are facilitating a 
number of information processing activities—sensing, data sharing, compar-
ing, correlating, interpreting, advising, alerting—to enable and support the 
related human collaboration to optimize action . 
 Th e “Internet of Th ings” (IoT) represents a growing sophistication 
among devices. Examples of devices in the network of the IoTs include 
mobile handsets, refrigerators, cars, fi tness trackers, watches, eBooks, 
vending machines, and parking meters, and the number of types of devices 
is likely to grow exponentially over the coming years. Th ese devices are 
already gathering and communicating massive amounts of data about 
themselves, which is collated, curated, and harvested by a growing number 
of smart applications. 
 Just connecting a device to the Internet does not result in collaboration. 
Th e core theme of this book is the identifi cation of cognitive behavior among 
 IoT s. A network of Cognitive Th ings uses a new computing paradigm—
namely cognitive computing—along with the power of the Internet and the 
2 Cognitive (Internet of) Things
data available from a collection of devices to forge new collaborations and 
create new applications never imagined before. 
 Th is chapter introduces the major propositions outlined in this book. It 
provides a defi nition of “Cognitive Th ings” and the scope of devices discussed 
in this book. It introduces the concept of cognitive computing and summa-
rizes the chapters, diff erent potential reader personas and the area of focus for 
each persona. 
1.2 What Are Cognitive Things and How Do 
They Function? 
 We are on the threshold of a massive explosion of connected things. A 
 McKinsey report projects the potential business impact as $ 4 –11 trillion 
per year by 2025 using nine settings— factories (e.g., preventive mainte-
nance), cities, human (e.g., improving wellness), retail, outside (e.g., self-
driving vehicles), work sites, vehicles, homes, and offi ces . 1 Th ere are many 
other projections each defi ning IoT s and projecting their impact in the tril-
lions of dollars. 
 How do the IoT s derive such large business impacts? Let me illustrate an 
example to show how far reaching these IoTs will be in disrupting markets 
and businesses. Business Week has projected the availability of driverless cars to 
premium customers by 2025 and is also predicting driverless technology tak-
ing over taxi and ride -sharing fl eets by 2030. 2 While the driverless car includes 
a large number of sensors to collect information about the car and the road, 
the autonomous vehicle is far more than a collection of sensors connected to 
the Internet. It is actually replacing the driver ! Th rough automatic gear change, 
cruise control, automated lane detection, to automated parking, we have seen 
a number of ways in which vehicles are beginning to use this data to perform 
tasks originally performed by the drivers . We 3 will see a maturing technol-
ogy capable of making a series of cognitive decisions to drive the car without 
requiring a driver. Instead of purchasing cars, potential riders may in the future 
1 James Manyika et  al., “Th e Internet of Th ings: Mapping the Value Beyond the Hype”, Executive 
Summary of the Report titled “Unlocking the Potential of the Internet of Th ings”, McKinsey Global 
Institute, McKinesy & Company, June 2015, http://www.mckinsey.com/insights/business_technology/
the_internet_of_things_the_value_of_digitizing_the_physical_world 
2 Keith Naughton, “Can Detroit Beat Google to the Self-Driving Car”, http://www.bloomberg.com/
features/2015-gm-super-cruise-driverless-car/ 
3 I have used the term “we” to represent the humans, who are likely to be the consumers of the Cognitive 
Th ings. 
1 Introduction 3
use taxi services to move them from point A to point B, allowing for less pub-
lic parking and potentially higher vehicular speed. Th ese changes will have a 
profound impact on the auto insurance business, car dealers, taxi operations, 
rental cars, and public transportation . Moreover, they could easily translate to 
a sizable share of the $11.1 trillion business impact projected by the McKinsey 
study referenced above. 
 According to Dr. John E Kelly III, IBM’s senior vice president , cog-
nitive is the third era of computing. Th e fi rst, tabulating, began in the 
late nineteenth century and enabled such advances as the ability to con-
duct a detailed national census and the United States’ Social Security 
System. Th e next era, programmable computing, emerged in the 1940s 
and enabled everything from space exploration to the Internet. Cognitive 
systems are fundamentally diff erent. Because they learn from their inter-
actions with data and people, they continuously improve themselves. So 
cognitive systems never get outdated and only get smarter and more valu-
able with time. Th is is the most signifi cant paradigm shift in the history 
of computing. 4 
 Let me use an encounter to describe a Cognitive Th ing. As I discuss the 
concepts underlying my book with a wide spectrum of people, I get interest-
ing responses, ranging all the way from disbelief to personal encounters. Th e 
following story was recounted to me by Dan Abercrombie , CEO of Abletech. 
 Th e story covers Dan’s personal interaction with a robot developed by 
research work at the Osaka University . 5 While many of the features described 
here seem futuristic, it will likely be commonplace to fi nd robots providing 
concierge services at hotels, conferences, and musical and sport events. Th ese 
concierges will display a range of cognitive capabilities, including : 
- natural language comprehension 
- empathetic conversation 
- facial recognition 
- context retention and recall 
- knowledge of organization, products, and people  
 I was walking the expo fl oor at Semicon, and saw the banner from the booth 
of the Fujikin Corporation. One of their executive staff , a Mr. Suzuki (fi ctitious 
4 John Kelley,” Smart Machines: IBM ’s Watson and the Era of Cognitive Computing”, Columbia Business 
School Publishing, September 2013, http://www.amazon.com/Smart-Machines-Cognitive-Computing-
Publishing/dp/023116856X/ 
5 ResOU, “Leonardo da Vinci comes back as an android”, Research at Osaka University, August 21, 2015, 
 http://resou.osaka-u.ac.jp/en/research/2015/20150821_14 Cognitive (Internet of) Things
name), is a long-term industry contact and friend. I had a few minutes and 
decided to try to say hello to Mr. Suzuki. I walked into the booth and asked one of 
the (human) booth staff (in Japanese, which I speak fl uently) if Mr. Suzuki was 
there that day. To my surprise, a voice coming a couple of meters from my right 
said, “Suzuki-san desu ka? Saki made imashita yo.” (Something like, “Oh, you 
would like to see Suzuki-san? He was here until a few minutes ago.”) I turned 
around to see a distinguished gentlemen dressed as Leonardo Da Vinci sitting on a 
chair with a microphone. Upon somewhat closer inspection, it turned out that our 
Da Vinci was a chillingly lifelike robot. 
 Recovering from my initial surprise, I stepped in front of the robot and it greeted 
me cordially in English, and made comment about the booth being very busy and 
crowded, and that Mr. Suzuki would be back later. It was clear the robot had 
recognized my Caucasian appearance and decided that it should switch to English 
with me. Mildly miff ed by Da Vinci’s presumptuousness (how did it know I wasn’t 
French?), I mischievously decided to switch to Japanese, saying “Sumimasen ga, 
eigo wo wasurete shimatta.” (“Sorry, but I have forgotten how to speak English.”) 
Da Vinci’s rejoinder in a thick Osaka dialect came without hesitation, “Ah Ha Ha 
Ha Ha…Washi ha italia-go wasureta! O-taku nihonngo jouzu ya na”; something 
like “Ha, ha, ha…I have forgotten my Italian! Your Japanese is really great!” 
 By this point, I was totally off guard. In less than thirty seconds of interaction, 
the machine had overheard my initial inquiry, appropriately responded faster than 
any of the fi ve or six humans in the vicinity, adjusted its choice of language based 
on my appearance and then switched languages again when requested, making 
a natural and spontaneous joke at the same time. I bantered with the robot for 
another couple minutes and then went on my way. About fi ve hours later, I strolled 
by the Fujikin booth and Da Vinci once again, and this time Da Vinci called out 
to me as I walked in the aisle, “Ah, mata irasshita, Suzuki-san ha asoko ni imasu 
yo.” (Oh you are back. Suzuki-san is over there.) Suzuki-san, hearing his name 
mentioned, turned around in surprise, and saw me. We had a short conversation 
and well-wishes, both of us mixing in plenty of nervous laughter as Da Vinci 
continued to interject comments into our conversation, and somehow both of us 
feeling “obligated” to be polite to Da Vinci and explain that we were long-term 
business acquaintances and all. Somewhat uncomfortable with Da Vinci par-
ticipating, I made my greetings with Suzuki-san cordial, but brief, and went on 
my way again, acutely conscious of the reality that the need to deal with thinking 
machines will become routine in my lifetime. 
 Robots are beginning to perform the cognitive functions depicted in this 
encounter—recognize people, chit-chat, apply conversational humor, recall 
context in a later conversation, and be aware of the environment and the pres-
ence of others. In Chapter 10 , I will be introducing some of these cognitive 
1 Introduction 5
functions in detail, and how they are utilized in human conversation with a 
Cognitive Th ing. 
 What is cognitive computing, and how does it relate to artifi cial intelligence 
and expert systems? A cognitive system learns at scale, reasons with purpose and 
interacts with humans naturally. 6 “Cognitive Computing” refers to automated 
agents that can learn complex tasks, interact with humans via natural inter-
faces and make autonomous decisions and actions working with individual and 
groups. It represents a new generation of computing systems enabling genuine 
human–machine collaboration where the system is able to understand high-
level objectives specifi ed by humans in a natural language, autonomously learn 
how to achieve the objectives from data in the domain, report results back to 
humans, and iterate the interactions via sequential dialog until the objectives are 
achieved. To enable a natural interaction between them, cognitive computing 
systems use image and speech recognition as their eyes and ears to understand 
the world and interact more seamlessly with humans. It provides a feedback 
loop for machines and humans to learn from and teach one another. By using 
visual analytics and data visualization techniques, cognitive computers can dis-
play data in a visually compelling way that enlightens humans and helps them 
make decisions based on data. Cognitive computing systems get better over time 
as they build knowledge and learn a domain—its language and terminology, its 
processes and its preferred methods of interacting. Unlike expert systems of the 
past, which required rules to be hard coded into a system by a human expert, 
cognitive computers can process natural language and unstructured data and 
learn by experience, much in the same way humans do. While they will have 
deep domain expertise, instead of replacing human experts, cognitive computers 
will act as a decision support system and help them make better decisions based 
on the best available data, whether in healthcare, fi nance or customer service. 7 
 Th e starting point for IBM ’s cognitive journey was the Watson system with 
its fi rst major accomplishment of defeating human Jeopardy experts using a 
computer program. While it was a great accomplishment, Jeopardy actually 
represented a relatively simple cognitive activity, as it is a single question rep-
resented in the form of an answer and requires search in unstructured content 
to fi nd an answer and to pose it back as a question. Most cognitive tasks are 
far more complex, and in many cases it is extremely diffi cult, if not impos-
6 John E Kelly, “Computing, cognition, and the future of knowing”, IBM Research, October 2015, 
 http://www.research.ibm.com/software/IBMResearch/multimedia/Computing_Cognition_WhitePaper.
pdf 
7 Biplav Srivastava, Janusz Marecki, Gerald Tesauro, “2nd Workshop on Cognitive Computing and 
Applications for Augmented Human Intelligence, in conjunction with International Joint Conference on 
Artifi cial Intelligence (IJCAI)”, Buenos Aires, Argentina, July 25-31, 2015, https://sites.google.com/site/
cognitivecomputing2015/ 
6 Cognitive (Internet of) Things
sible, to replicate human function. Cognitive Th ings tend to embody selective 
cognitive functions to either augment or in some simple cases replace human 
activities. In a presentation to IBM’s Academy of Technology, Grady Booch , 
well-known co-author of Th e Unifi ed Modeling Language User Guide and now 
an IBM Fellow, introduced his defi nition of embodied cognition :
 Imagine unleashing Watson in the physical world. Give it eyes, ears, and touch, 
then let it act in that world with hands and feet and a face, not just as an action 
of force but also as an action of infl uence. Th is is embodied cognition: by plac-
ing the cognitive power of Watson in a robot, in an avatar, an object in your 
hand, or even in the walls of an operating room, conference room, or spacecraft, 
we take Watson’s ability to understand and reason and draw it closer to the natu-
ral ways in which humans live and work. In so doing, we augment individual 
human senses and abilities, giving Watson the ability to see a patient’s complete 
medical condition, feel the fl ow of a supply chain, or drive a factory like a mae-
stro before an orchestra. 8 
 So, how are automobiles getting transformed using cognitive computing? 
Driving is a cognitive activity. If the driver falls asleep or is impaired, the brain 
stops a large number of cognitive actions, each of which could be fatal to the 
car and the driver. A driver uses vision to recognize stationary obstacles, other 
movingobjects, and the curvature of the road. Th e driver also has the ability 
to use a combination of steering, breaking and acceleration actions to control 
the movement of the automobile. Th e driver keeps track of fuel consumption 
and equipment failures (for example, a burst tire) and makes decisions. In a 
typical driverless car, the driver can go to sleep, get impaired, watch a movie 
or conduct a meeting, while the car provides all these cognitive functions. In 
addition, the data collected by the car can now be used for equipment failure 
prediction, improved engineering, or can be shared with city planners, who 
will use this data to optimize traffi c fl ows. In such a case, a driverless car and 
its collection of sensors have the ability to collect the data automatically. Each 
sensor uses the car’s energy to operate, and the data can be shared with car 
components. Also, an OBDII (On-Board Diagnostics) port provides external 
access to the data. With the help of an OBDII port and Bluetooth or Wi- 
Fi technology, this data can now be shared with a number of organizations, 
including the car manufacturer, parts supplier, auto service infrastructure, 
insurance companies, city planners, etc. 
 Table 1.1 compares cognitive operators to show how computers are evolv-
ing from the automation era to the cognitive era. Th ey use evidence-based 
8 Grady Booch, presentation given to IBM ’s Academy of Technology, March 23, 2016. 
1 Introduction 7
reasoning. Th ey seek to fi nd new insights and connections from within vast 
amounts of information. Th ey change the way humans and systems interact. 
Ingestion technologies in the automation era were geared towards large-scale 
data transfers focusing on extraction, transformation, and loading of the 
data to the target platform. In the Cognitive Era, the focus is on their ability 
to observe, recognize, and identify. In the case of the driverless car, the car 
must identify moving objects, especially those which are likely to collide with 
the car. Th e car must sense a malfunctioning car wheel or a rough terrain, 
and diff erentiate it from regular conditions. Information processing in the 
Cognitive Era moves computing technologies towards out-of-box approaches 
to problem- solving, such as synthesizing a solution, which is a much harder 
problem as compared to choosing an alternative based on static decision 
trees, an automation era computing capability. Some of the related planning 
activities are already available to drivers today—such as dynamically rerouting 
based on changing conditions. Last but not least, the agent interactions are 
far more human-like with the ability to empathize and negotiate. Will a driv-
er less car yield to another car while changing lanes or decide how to reduce 
the amount of damage in an unavoidable accident, choosing saving lives over 
wrecking another driverless car without passengers?
 Table 1.1 represents many cognitive operators. As in the case of humans, 
the cognitive thing does not need to exhibit all of these operators to pass the 
“cognitive” test. Is there a cognitive IQ for the machine? Th is is a cognitive 
journey and it will take many decades to mature to the level where Cognitive 
Th ings will exhibit a majority of these operators in production systems. Th e 
race today is in fi nding a couple of nuggets that provide the biggest busi-
 Table 1.1 Cognitive operators 
 Function Automation era Cognitive era 
 Agent 
Interaction 
 Present facts, ask 
questions, retrieve 
responses, help, set 
default 
 Empathize, chit-chat, switch context, 
converse, negotiate, disambiguate, refi ne 
and change decisions, discuss and resolve 
problems 
 Solution 
Development 
 Compute, apply 
statistics, use 
decision tree, 
aggregate, search 
for key words, sort, 
select, delete, insert, 
join 
 Analyze (impact, sensitivity), reason,  design 
a solution (synthesize, confi gure, 
extrapolate, discover), fi lter, focus, learn 
(inductive, deductive, deep, knowledge- 
based), context-sensitive search, decide, 
plan, schedule, diagnose, interpret, 
abstract 
 Event Ingestion Extract, transform, 
load, stream 
 Sense , observe , listen , crawl , recognize , 
identify, reconfi gure 
8 Cognitive (Internet of) Things
ness value and use them to initiate the journey. As will be shown through 
the examples in this book, the key to success is in striking the right balance 
between man and machine to achieve optimal results in the short term, and 
adding additional capabilities in later phases, as dictated by their business 
value and appropriateness. 
1.3 Book Outline 
 In literature  and the media, we have seen many futuristic examples of smart 
refrigerators and coff ee machines and how they will simplify our daily life. 
What is our expectation of the Cognitive Th ings and how pervasive is it likely 
to get? Certainly, we are used to a large number of not-so-smart machines 
capable of supporting our day-to-day activities with an increasing level of 
automation. In the Cognitive Era , an intelligent home could be equipped 
with smart televisions, smart refrigerators, smart dish washers, smartphones, 
smart cars, and so on. In the automation era, these devices were primarily 
designed for automating the tasks in mechanical ways. As these machines 
become intelligent, they must learn how to listen, reason, perceive and change 
with our wishes. Each of these devices must be confi gured and trained to deal 
with their human counterparts. Do I like my coff ee in the early morning or 
mid-morning? Do I take it with cream and sugar? If I have to engage in train-
ing my coff ee machine each time a guest arrives and alters its daily routine, 
and retrain when the guest departs, where is the value to me as a consumer? 
Why should I use this “smart” feature and pay additional money? Is there a 
business use case for a cognitive coff ee machine, which can sense my prefer-
ences and auto - confi gure itself, and is that really feasible in the near future? 
 Th is book examines Cognitive Th ings, covering a number of important 
questions:
• What are Cognitive Th ings? 
• How do they interact with other machines and with us? 
• Can machines s upport human communication as intelligent listeners or 
aids to procure supporting facts, or evaluate alternatives? 
• Which technical components make up cognitive behavior? 
• What applications can be driven from Cognitive Th ings—today and 
tomorrow? 
• How does it redistribute the workload between humans and machines? 
• What types of data can be collected from them and shared with external 
organizations? 
1 Introduction 9
• How do they recognize and authenticate authorized users? How is the data 
safeguarded from potential theft? Who owns the data and how are data 
ownership rights enforced? 
• How do these devices collect and share data about themselves, how is this 
data collated, secured, obfuscated, and how can this data be harvested by 
smarter applications to provide new innovative capabilities for a variety of 
users? 
• What insights can be generated not only about Cognitive Th ings but also 
about the people who interact with them? 
• Does it change current corporate infrastructures for organizations? 
 Th is book covers three related directions associated with Cognitive Th ings: 
business use cases, technical capabilities, and impact on consumers and corpo-
rations. Th e business use cases describe how the Cognitive Th ings will be used 
and justifi ed in their value propositions. Th e technical capabilities discussion 
explores the technical feasibility of engineering these solutions. Th e impact 
on humans and organizations explores the integration aspects in today’s world 
and how individuals andorganizations are likely to adopt Cognitive Th ings. 
 Th e fi rst part of the book explores compelling business use cases to drive 
Cognitive Th ings. Do these use   cases relate to objects or humans, individuals 
or organizations, inside an organization or in a market place? Chapter 2 identi-
fi es how Cognitive Th ings take care of themselves, in traditional maintenance, 
operations, engineering, and reconfi guration aspects. A series of common 
examples are used to describe the business value from a cognitive thing and 
the business capabilities covered. Chapter 3 identifi es how Cognitive Th ings 
improve support for the individual customer. It examines new roles and 
responsibilities for the Cognitive Th ings and the value provided to the indi-
vidual. Chapter 4 identifi es how  Cognitive Th ings support an organization. 
Chapter 5 covers how Cognitive Th ings are adding new data sources to the 
services market place and how organizations are using new business models 
to serve their business customers using monetization of data from Cognitive 
Th ings. 
 Th e second part explores the technical dimension identifying the key tech-
nical capabilities, which are required to fulfi ll these use cases. For each tech-
nical capability, the book outlines the level of sophistication required and 
available as of today. Th e data represents extreme velocity, volume, and vari-
ety, and most of this data is unstructured. It is impossible for any computing 
environment to collect, correlate, analyze all the data being generated, and 
act upon the results in near real-time. Additionally, appropriate tools and 
10 Cognitive (Internet of) Things
techniques can isolate micro-segments and learn fi ner details relating to these 
micro-segments. Advanced analytics techniques can help identify trends, 
focus on a tiny fraction of the data, enquire and collect additional data wher-
ever needed, and decide how to act upon a situation. Cognitive Th ings will 
also speed decision-making through automated data collection and decision- 
making. Early detection and correction can result in substantial benefi ts to 
the individuals, organizations, and society. 
 Th is section is divided into four chapters. Chapter 6 focuses on data acqui-
sition aspects and how Cognitive Th ings are able to acquire data. It examines 
the diff erences provided by the cognitive function and the capabilities required 
to realize it. Chapter 7 focuses on information processes and examines how 
new cognitive information processing functions use learning to provide better 
insight and more focused action. Chapter 8 covers maintenance and support 
aspects and how Cognitive Th ings can be more easily installed, confi gured, 
monitored, trained, and secured without requiring an army of maintenance 
professionals. It covers thorny issues around data protection, and rights man-
agement. While the book touches upon technical topics, it does not provide a 
detailed explanation of the underlying technologies. A number of books have 
been published on the technical aspects, and the respective chapters provide 
some useful references to the excellent material available elsewhere. 
 Th e third part of the book explores human and machine interfaces. It looks 
at how these changes will disrupt and change our current organizations and 
how can we prepare ourselves. It addresses some of the pitfalls that must be 
avoided. Chapter 9 covers machine-to-machine interfaces and how Cognitive 
Th ings can relate to other intelligent agents in conducting their task, support-
ing their customers and negotiating a better result for their owners. Chapter 
 10 covers human-to-machine interfaces and how cognitive devices can act as 
assistants for humans. Chapter 11 covers human-to-human communication 
with the use of cognitive devices as observers and advisers. Chapter 12 covers 
impact on organizations and society. It explores the opportunities and what 
must be done to integrate them with today’s organizations. It also consid-
ers the nature of the man-machine relationship and how it changes with the 
Cognitive Th ings . It projects the changing organization structures and how 
the twenty-fi ve billion Cognitive Th ings will change the way eight billion 
humans organize and run their daily lives,  organizations and societies. 
 Th e book applies a series of public case studies to illustrate its assertions. 
Innovators and early adapters are already benefi ting from the initial explosion 
of devices and capabilities available today. However, what you see today is 
only the tip of the iceberg. Using a series of futuristic visions collated through-
out my research, the book additionally explores what is the new act of impos-
sible and where we are headed. 
1 Introduction 11
1.4 Target Audience 
 Th e book is primarily targeted towards organizations marketing, manufac-
turing and operating a network of Internet of Th ings today or planning to 
do so in the future. Most of these organizations are engaged in activities to 
apply cognitive computing and make their applications smarter. Th e book 
will provide much needed information about potential use cases, business 
and IT capabilities, and ways of mitigating failure. Th e book is written with 
the business user in mind. It will cover a number of technical topics; how-
ever, it focuses on those topics from a business perspective, describing how 
these technical topics impact business issues, rather than diving into details of 
the technologies. Universities can use this book as reading material for their 
Information System courses. I will be creating additional material for educa-
tors interested in teaching cognitive computing courses. 
1.5 Summary and What’s Next 
 In this chapter, I have laid out the purpose  and abstract of this book . Cognitive 
Th ings are capable of providing a number of key capabilities needed for col-
laboration including their ability to collect data automatically, data sharing 
across a network, and information processing for decision optimization . 
 Th e information processing is done cognitively and refl ected in a new set of 
human-like capabilities in event ingestion, solution development, and agent- 
interface. In a typical domain such as a driver less car, these are necessary capa-
bilities for a machine to think and act in a human-like way. 
 Th e key propositions included in the book are:
 1. Business use cases are driving Cognitive Th ings and how they support the 
device, the individual, the organization and the external market place. 
 2. Technical capabilities have emerged to enable cognitive capabilities to be 
added to IoTs providing the Cognitive Th ings with the power to process 
information like humans. 
 3. Th e Cognitive Th ings will have enormous impact on consumers, and orga-
nizations , creating new ways to deal with collaborative activities and 
designing new policies. 
 In the next chapter, I will discuss how Cognitive Th ings function and how 
they support maintenance, operations, design, and engineering tasks. 
12 Cognitive (Internet of) Things
13© Th e Editor(s) (if applicable) and Th e Author(s) 2016
A. Sathi, Cognitive (Internet of ) Th ings, 
DOI 10.1057/978-1-137-59466-2_2
 2 
 What Is a Cognitive Device? 
2.1 Background 
 When my wife and I relocated from Denver to Southern California, we 
decided to trash our 20-year-old bulky television and shop around for one 
of the latest models. Neither of us was in the habit of sitting and watching 
network television—we were far more interested in watching shows suited to 
our interest and convenience. We were also interested in purchasing cognitive 
appliancesfor our house and mistakenly thought a “ smart tv ” was likely to be 
a cognitive device. A number of television manufacturers were off ering what 
they proclaim to be smart TVs. As we dived into the specifi cations of smart 
TV , we found to our astonishment, that so-called smart TVs were not so smart. 
Th ey had no idea who was watching the television. Th ey could not remem-
ber what I watched last. Worst of all, when the smart TV was not working, 
it could not self-troubleshoot among the  device, Internet connectivity, and 
the content provider. Instead of taking care of its content delivery problem, 
it forced me to repair the television , integrate knowledge from many diverse 
areas, and converse with multiple technicians. A set of customer service pro-
fessionals guided me to perform random acts of hot and cold reboots, hoping 
that those reboots would remove the underlying problem. Th e experts agree 
that televisions have a long way to go before they become cognitively smart. 1 
While there are many ways a television today is far more sophisticated than 
1 Michael Miller, “Smart TVs: Viewing in a Connected World”, Que, April 2, 2015. http://www.quepub-
lishing.com/articles/article.aspx?p=2320941&seqNum=7 
our clunky 1995 television and off ers a variety of new programming using its 
Internet connectivity, the advances have very limited cognitive characteristics. 
 At the same time, I found a number of audience services, such as Netfl ix , which 
off ered many cognitive characteristics. Th ey reorganized the menus based on my 
viewing patterns, kept track of my wish list, off ered recommendations, and dif-
ferentiated among family members in the same viewing plan. Imagine a cognitive 
television, one that would be more like a server in a restaurant, that could relate to 
the person watching the television, serve based on content preferences, and take 
care of network issues instead of leaving all the troubleshooting to the owner. Like 
a good restaurant, the television would re- engineer itself around the most popular 
functions, getting rid of scarcely used menu items, simplifying the interface, mak-
ing it easier to install, operate, and troubleshoot. Last but not least, the cognitive 
television must be secure d from external hacks. Today’s smart TV with a webcam 
is a handy device for a thief to check into a house before burglary. In a presenta-
tion to IBM , Collin Dunphy and Evan Spisak , aged 12 and 10 respectively, dem-
onstrate hacks to take over household webcams for unintended access . 2 
 Th e following sections identify the characteristics of a cognitive device 
and its functions, as compared to a non-cognitive device. While many of the 
features defi ned here are in production use somewhere on some device, the 
widespread application of this use case is still a (cognitive) thing of the future. 
However, it off ers tremendous opportunity for the manufacturers to grab a 
market by off ering the fi rst truly cognitive device. 
2.2 Candidate Devices 
 Let us look at a few examples to illustrate the behavior of a cognitive device. 
Many home appliances, such as washing machines, dishwashers, and security 
systems, are becoming cognitive. My daughter and son-in-law have set up 
their home to be a cognitive house connecting lamps and video cameras to 
the Internet. Th e video cameras provide them with a useful baby monitor, giv-
ing them full access to the movement of their twin infants conveniently from 
an app on their cell phones. Th e cameras only record the video when there is 
motion or sound, thereby fi ltering out all the less important static view of sleep-
ing babies in their cribs. In this way, the camera is acting like the night watch-
man who keeps track of all activities, but only needs to report events that cross 
certain thresholds. By time-stamping the videos, it provides them with an accu-
rate account of the children’s sleeping behavior, playtime and feeding patterns. 
2 IBM Academy of Technology Channel, “Hack: Homes are compromisable by kids”, https://youtu.
be/3Q6rLQxgeLQ 
14 Cognitive (Internet of) Things
 With newborn twins, it is important to be able to track the babies’ behav-
iors. My daughter and son-in-law confi gured the app Baby Connect for all of 
their caregivers, to record the activities for each baby, including feeds, sleep 
and bowel moments. Baby Connect appeals to parents who have enough time 
and energy to record the length of their child’s naps or number of diaper 
changes. 3 Baby Connect replaces a caregiver’s handwritten notes with a col-
laboration engine that allows parents and child care centers to communicate 
easily. As soon as an event is saved, it is immediately synchronized on each 
parent and caregiver account. Everybody has access to the information in real 
time. You can not only record feedings, nursing, naps, diapers, milestones 
and pumping, but also the baby’s mood, temperature, what kind of game he’s 
playing and his GPS location. You can also attach pictures . 4 
 Most new household appliances come with Internet connectivity and 
can detect usage and troubleshooting patterns for preventive maintenance. 
 Whirlpool is now manufacturing washing machines that incorporate their 
innovative CareSync TM System . 5 Your washer and dryer can easily connect 
to any mobile device, putting control and exceptional fabric care right at 
your fi ngertips. 6 Th ese machines can track detergent use, predict defects 
and schedule repair. Th e valuable usage and performance data can now be 
fed to the engineering systems for improvements in product engineering. 
In the future, washing machines will become more cognitive, scheduling 
their workload at a time when other household water usage is minimal, 
self-diagnosing and scheduling preventive maintenance thereby avoiding 
major failures and reorganizing their menu options based on how they are 
being used. 
 How smart are the appliances and security systems in my house, and are 
they cognitive? Dogs have provided basic cognitive functions in household 
security for centuries. A well-trained dog knows how to keep an intruder away 
with a fi erce bark but equally is welcoming to members of the household and 
frequent guests. Service dogs go a step further in providing valuable company, 
3 Bob Tedeschi, “Devoting Attention to a Child and a Phone, All at Once”, New York Times, April 27, 
2011, http://www.nytimes.com/2011/04/28/technology/personaltech/28smart.html 
4 BabyConnect, “Collaborative baby care using baby connect”, YouTube, May 7, 2012, https://youtu.be/
AYLApsUDMM0 
5 Whirlpool® USA, “Whirlpool Smart Top Load Connected Laundry Pair, October 23, 2015, https://
youtu.be/2BGcGdNcSOk 
6 From Whirlpool ’s corporate website, http://www.whirlpool.com/smart-appliances/smart-top-load-
washer-dryer/ 
2 What Is a Cognitive Device? 15
guiding their masters and alerting others in case of medical attention. 7 My 
93-year-old father lives with us and we were concerned about his health and 
how to keep watch over him as he was showing signs of health issues that may 
require medical attention. Since my wife is afraid of dogs, getting a service dog 
was not an option, and we needed an option that could alert the 911 services, 
if needed. I called my home security company and a call center sales person 
promptly off ered a device my father could wear all the time. Th e device would 
alert emergency services when he presses a button or falls down. As I marveled 
through the sunny day scenario (the typical sales use case presented by the 
salesperson), my engineering mind was searching for rainy day scenarios (how 
the system could malfunction). I asked how the emergency services would 
enter the house if my father were unable to respond to the doorbell. Th e sales-person told me they would do whatever they had to, even if required breaking 
down the front door to get into the house. She also said, “If the house is wired 
with an electronic lock and the emergency services has the combination code, 
they will use it, but it is equally probable that they may end up breaking down 
the front door.” With some rigorous training, I could train my service dog to 
recognize the diff erence between a normal and emergency situation and diff er-
entiate between an unwanted intruder and an emergency care person. A cog-
nitive elderly care and security system should be able to do all of this without 
breaking down my front door every time my dad accidently drops the dongle 
on the fl oor. Th e interesting integration point that she ignored was that the 
security system has full control of the house and can welcome an emergency 
care professional while keeping intruders away. It seems like security systems 
are headed in the right direction, but for now we need both service dogs as 
well as monitoring services to get the desired features. 
 In Chapter 1 , I introduced cars as cognitive devices. Cadillac is introduc-
ing its new car with Super Cruise , the company’s most ambitious technol-
ogy foray since automatic transmission. Pairing adaptive cruise control with 
lane-centering technology, it will allow drivers (or whatever they are called in 
the future) to let the car take over only while on the highway. Google’s latest 
prototypes are already driving around Silicon Valley, where they are known as 
 Koala cars because of their bulbous shape. 8 Th e cognitive components have 
added a number of new sensors to the car’s growing list of data sources, includ-
ing front, side and back mirrors. Th ey are also equipped with plenty of pro-
7 Morieka Johnson, “5 things you don’t know about service dogs”, MNN.com, http://www.cnn.
com/2012/06/15/living/service-dogs-mnn/ 
8 Keith Naughton, “Can Detroit Beat Google to the Self-Driving Car? Inside GM’s fi ght to get to the 
future fi rst”, Oct 29, 2015, Bloomberg BusinessWeek, http://www.bloomberg.com/features/2015-gm-
super-cruise-driverless-car/ 
16 Cognitive (Internet of) Things
cessors resembling a large network of computers. Th e cars have been off ering 
an OBDII port for more than a decade, which provides a gateway to all the 
sensor data from the various car components. Th is port can be connected via 
 Wi-Fi or Bluetooth to an Internet connection point. Its data sharing enhances 
the cognitive performance of the car. All the relevant data is transmitted to a 
central computer and can be utilized in a variety of ways, including preventive 
maintenance, engineering, and operations. Cars can already follow the car 
ahead of itself in a lane. Drivers use a variety of primitive techniques to com-
municate with other drivers, including the car horn, which, if used extensively, 
may create enormous noise pollution as can be observed in the busy roads of 
Delhi. Cognitive cars should be able to electronically communicate with other 
cars without the use of light or sound and negotiate with each other for the 
right of way. What happens to all the rash drivers who are rushing to their 
appointments and are making up for the delay by driving faster than the speed 
limit? Imagine the roads of tomorrow where a series of driverless cars will be 
driving at the speed limit, in an orderly fashion with no need to watch for tail-
gaters or reckless drivers. As cars become more sophisticated, we as consumers 
have less and less ability to diagnose their problems. However, cognitive cars 
should be able to self-diagnose and schedule their own maintenance, obvi-
ously keeping in mind their owners’ convenience and preferences. In order to 
perform these functions, cars are starting to have the ability to organize their 
sensors, and to use the sensors to predict maintenance. I will discuss preven-
tive maintenance capability in more detail later in this chapter. 
 Smartphones and tablets were the fi rst natural Cognitive Th ings. Th ey have 
plenty of capabilities to locally process data, Internet connectivity to connect 
globally for voice, data, and video communication, and an ability to collabo-
rate with other devices and central servers. As a cognitive integration point for 
other IoTs, they are increasingly performing three sets of tasks. First, they are 
emerging as the universal user interface for other IoTs. As we described in the 
 Whirlpool example above, the controls for smart devices can be managed via 
smartphones and tablets. Second, they are the natural connection hub for con-
necting other IoTs to the Internet. Using Bluetooth or other technologies, they 
can collate data from other devices, buff er as needed, and upload to central serv-
ers. Th ird, they can be used as a payment hub by providing the mobile wallet 
capability. For performing these functions, they must remain ON all the time, 
and should not malfunction. Th e preventive maintenance is driven by their 
ability to collect data about their performance, including temperature, battery 
consumption, data upload/download, and many other parameters. What is 
their ability to perform preventive maintenance? One of my smartphones was 
defective. It would randomly heat up, and its battery would discharge suddenly 
2 What Is a Cognitive Device? 17
in 10 minutes or less. I found it extremely inconvenient, especially on the day 
I was running late to the airport with an electronic boarding pass on my cell 
phone. When I tried to get it fi xed, the statistics collected were not suffi ciently 
detailed enough to provide the root cause. Th e repair shop decided to play 
it safe and gave me a replacement phone. However, it is not clear what the 
problem was and whether a factory reset would have removed the problem. In 
many cases the problem may not be with the smartphones but the apps run-
ning on the phone or the network connecting those phones. Despite replacing 
the phone, the consumer may still observe the same issues. Telecom service 
providers often refer to this problem as “ No Fault Found ” referring to the pile 
of perfectly good smartphones returned by disgruntled customers. Th e returns 
are both expensive for the service provider, as well as the leading cause of Net 
Promoter Score reduction among customers. A cognitive smartphone should 
be able to alert its service provider as well as its owner about potential problems 
and fi nd ways to self-repair. A collaborating collection of Cognitive Th ings 
representing the smartphones and the network elements—wireless antennas, 
network routers, switches, and so on—can observe faults as they build up and 
help troubleshoot problems before the consumer notices the problem. 
 “ Wearables” are the next set of Cognitive Th ings. Often, as in the case of 
smart watches, they provide the mechanism for collecting physical measure-
ments from their owners. Wearables often have smaller energy consumption 
requirements and use Bluetooth and gateways to connect to the Internet. For 
example, the Fitbit described in Chapter 1 collects movement information and 
uses Bluetooth connectivity to transmit movement information to a central 
server, where a social network can observe and collaborate on exercising activi-
ties. Th e HealthBox TM from Under Armour ® is a kit that includes a chest strap 
for heart rate monitoring, a wristband, and a circular scale. Th e fi tness wrist-
band has a unique design that makes it comfortable to wear all day, even while 
sleeping. You have the option to synchronize the scale with the corresponding 
apps so that it remembers who you are each time you step onto it. Th e scale 
can also measure body fat percentage and keep track of weight loss goals. All 
of the items in the kit will communicatewith the Record app . Under Armour 
 has also released its fi rst smart shoe. It collect s movement data so wearers will 
not need another device to track workouts – the shoe does it itself. 9 
 Clothing with sensor technology is still nascent, but major apparel brands like 
 Under Armour , Ralph Lauren ® and Levi’s ® have been working to develop their 
off erings. “Th e structure of textiles is the same as the structure of touchscreens 
9 Ahiza Garcia and Hope King, “ Under Armour reveals ‘HealthBox’ 24/7 activity monitor”, CNN Money, 
Jan 5, 2016, http://money.cnn.com/2016/01/05/news/companies/under-armour-healthbox-ces-2016/ 
18 Cognitive (Internet of) Things
which we’re using in everyday mobile devices and tablets,” said Ivan Poupyrev , 
founder of Google’s Project Jacquard (which Levi’s is a part of ). A new video 
shows a biker wearing a jacket and communicating with it . 10 “ Th is means that 
if you just replace some of the threads in the textiles with conductive threads, 
you should be able to weave a textile that can recognize a variety of simple touch 
gestures, like any touch panel you have on a mobile phone.” 11 
 Let us move on to a set of Cognitive Th ings applied “in the air”. Amazon 
CEO Jeff Bezos had a big surprise for CBS correspondent Charlie Rose in 
late 2013. After their 60 Minutes interview, Bezos walked Rose into a mystery 
room at the Amazon offi ces and revealed a secret R&D project: “ Octocopter” 
drones that will fl y packages directly to your doorstep within 30 minutes. 
It’s an audacious plan that Bezos says requires more safety testing and FAA 
approvals, but he estimated that delivery-by-drone, called Amazon “Prime 
Air,” will be available to customers in as soon as four to fi ve years. Two years 
later, Amazon has released a version 2 and is now testing its safety with FAA 
with support from NASA. 12 , 13 As much as driverless cars respect the laws and 
protocols of a busy highway, drones must learn to interact with human envi-
ronment around them . Like the UPS delivery person, they must navigate to 
an address, avoid getting chased by the dogs, and be gentle to curious kids. In 
the video released by Amazon, the customer will receive a notifi cation from 
Amazon stating the package arrival time and they will be placing an Amazon 
marker to pin point where the drone will land. Drones will also be a new data 
source for valuable environmental information.  Th eir bigger counterparts, 
commercial aircraft, are already collecting a vast amount of data about them-
selves. Th ey are providing that data to the airlines, aircraft manufacturers, and 
the weather company. 
 Internet enablement has seen a lot more maturity in telecommunications 
networks where customers are demanding premium network and service 
quality, and there is fi erce competition among service providers to shine in 
front of their customers. After all, customers gravitate towards communica-
tions service providers who can provide them with fi ve bars of connectivity at 
10 Levi’s, “Levi’s® Commuter x Jacquard™ by Google Truck Jacket”, YouTube, May 20 2016, https://
youtu.be/yJ-lcdMfziwer 
11 Kristina Monllos, “5 Ways Marketers Are Already Putting Sensors to Work Next-gen ideas are sure to 
be the talk of CES”, Adweek, Jan 2, 2015, http://www.adweek.com/news/technology/5-ways-marketers-
are-already-putting-sensors-work-168777 
12 CBS News, “Amazon unveils futuristic plan: Delivery by drone”, Dec 1, 2013, http://www.cbsnews.
com/news/amazon-unveils-futuristic-plan-delivery-by-drone/ 
13 Trever, Mogg, “Tonight: Jeremy Clarkson presents the Amazon Prime Air delivery drone”, Nov 29, 
2015, http://www.digitaltrends.com/cool-tech/amazon-prime-air-delivery-drone/ 
2 What Is a Cognitive Device? 19
all their major hangouts. Service providers achieve that objective by monitor-
ing their network equipment and using this information to improve service 
quality. As communications service providers migrate to smartphones, third 
party apps, and data services, increasingly the service quality is dependent on 
a number of factors outside of their control. Telecommunication networks 
are rapidly gaining cognitive skills in reporting their performance, spotting 
performance issues, reconfi guring to meet changing demand patterns, and 
off ering insights to service and equipment providers for better design. As they 
have progressed in their cognitive journey, they have helped generate many 
important technical innovations, which can now be adopted by other indus-
tries. Th e telecommunication networks contain many intelligent devices, 
which are all connected and can provide a lot of data about their performance. 
A simple access of a Facebook page on a smartphone may generate hundreds 
of network events. Th ese events must be collected, correlated, aggregated, and 
scored close to data sources to provide meaningful insight to downstream 
monitoring systems. 
 Th ere are many other networks of intelligent devices in the business world. 
Th is includes smart equipment on the factory fl oor, corporate fl eets of trucks 
for a distribution company, or smart grids in a utilities company. As the cost 
of Internet connectivity goes down and analytics  options become technically 
and fi nancially viable, these networks are beginning to mature in their cogni-
tive journey. For example, utility companies have invested in creating smart 
meters for their consumers’ electricity consumption. In the past, meters were 
read once a month by utility company employees, who walked all over neigh-
borhoods each day physically reading these meters. With automated metering 
systems, the utilities companies can read meters much more frequently. Th is 
gave rise to a number of applications, which exploit the demand distribution 
to optimize energy consumption and distribution. 
 Smarter cities are increasingly making their resources Internet friendly. 
For example, many cities now use Internet enabled parking meters where 
money can be deposited using credit cards or phone apps. Once the parking 
meters are connected, the parking meters can inform their availability to 
intelligent cars driving around on the streets, thereby reducing the time and 
frustration associated with fi nding parking. Smarter cities can also use the 
utilization data to reorganize city parking spaces. Th e traffi c data collected 
from the cars can be utilized for a variety of planning and operational activi-
ties, including traffi c signal coordination, rerouting during disasters, street 
repairs, and so forth. 
20 Cognitive (Internet of) Things
2.3 Cognitive Device Operation 
 How do cognitive devices operate and how do they diff er from their auto-
mation era counterparts? In this section, I use a set of cognitive functions 
to compare and contrast the cognitive journey. Let us start by looking at 
the fi tness tracker example. Over fi ve hundred years ago, Leonardo Da Vinci 
came up with the concept of a pedometer . Sketches from the famous artist 
showed a gear-driven device with a pendulum arm that moved in time with 
a person’s steps. Fast-forward a few hundred years and pedometers now come 
in all shapes and sizes—mechanical, digital, or simply a smartphone app. 14 A 
 Fitbit can track physical activity using a pedometer. While pedometers use 
steps as a measure to quantify physical activity, there are many other options, 
including heart rate monitors that measure changes in heart rate, and GPS- 
enabled devices that track distance and terrain. Each of these devices provides 
a measure of physical activity which works in most situations but fails to 
work universally. For example, my Fitbit was fairly accurate in measuring my 
steps on a regular walk or jog, but was inconsistent inmeasuring physical 
activity while smooth ballroom dancing, which requires gigantic steps (as in 
the Waltz), versus rhythm dancing (as in Cha Cha) where the steps are tiny. 
In the world of Fitbit, a step is a step, and it cannot diff erentiate between 
my gigantic side step in the Waltz, which consumes a lot more energy, and 
the short step in Cha Cha. GPS is very good in measuring outside activities 
that traverse distances, but fails to properly understand a rigorous round of 
badminton in an indoor court, for example. Heart rate monitors are probably 
far better than the other two, but fail in situations where heart rates increase 
without actual physical activity, say while watching a horror movie. In each 
case, these devices use algorithms to estimate physical activity from their raw 
measurements and may in some cases adjust for age and weight diff erences. 
Despite their shortcomings, these devices are able to observe physical activ-
ity with relative ease and share insights from these observations by convert-
ing raw measurements to physical activity and then communicating those 
converted measurements to information stores where they can be aggregated, 
analyzed, and shared. Comparatively, the classic non-cognitive treadmills pro-
vided a measure using the speed of movement and displayed the result to the 
person using the machine. If I got bored jogging on the treadmill and decided 
to terminate the jog, there were no taunts from my social group. Today, the 
 Endomondo app on my iPhone tracks my location and performance during 
14 Technology, “How Pedometers work”, Runtastics, March 24, 2015, https://www.runtastic.com/blog/
en/technology/how-pedometers-work/ 
2 What Is a Cognitive Device? 21
jogs and posts it automatically to my Facebook page. It compels me to meet or 
exceed my targets and share a good jog with my friends. By using the terrain 
information, it also projects the steepness of the jogging trail and computes 
calories burned while exercising. 
 How would a cognitive weighing scale work diff erently from the classic 
scales we have used for decades? I decided to test the Under Armour Scale 
which comes equipped with the Record app that I installed on my iPhone. 
It showed up in a nice packaging and instructed me to fi rst install the app. 
As I powered on the scale, it used a Bluetooth connection with my iPhone 
to gather facts about me and greeted me by my fi rst name. My wife saw this 
shiny scale in the bathroom and decided to use it to get her weight. Th e scale 
was perplexed, as the biometrics did not match “Arvind”, and asked her, if, as 
a new user, she could introduce herself by installing the app fi rst. While this 
is a simple recognition step, the fact that the scale uses biometrics—current 
weight and body fat percentage—to diff erentiate between us, my wife and I 
do not need to manually change the settings to tell the scale which of us is 
using it. As and when I stand to measure my weight, it correctly addresses me , 
monitors and sends my biometric information to my app—using its recog-
nition to update the correct app connected with the person whose weight is 
being measured. 
 Recognition and user diff erentiation is the fi rst step in learning. Cognitive 
Th ings learn from their usage, and continually change their interactions 
with their users. In Chapter 9 , I will be discussing new toys—aptly named 
“Cognitoys TM ”—which are intended to be cognitive friends for children. 
 Cognitoys learn from interactions and keep morphing based on the interests 
and cognitive development of the child. Unlike many other classic toys, which 
off er buttons for interaction, cognitive toys do something diff erent each time 
they are used, thereby retaining their novelty and keeping the child interested 
in continuing to play with the toy. Th is learning can be a useful feature for 
any household equipment—the television, washing machine, washer, dryer, 
coff ee machine, or security system. In each case, we have patterns of use, and 
these patterns can be learned. Cognitive Th ings must recognize diff erent users 
through a variety of ways: by face, by touch, by voice, by fi ngerprints, and by 
usage, and apply past learning to customize their interactions. 
 When many relatives or friends show up and they travel together in more 
than one car, they use a variety of means to communicate with us, and they 
are always concerned about who will drive with whom, because the inter-
actions are limited to only the people in a single car. Will cognitive cars 
22 Cognitive (Internet of) Things
change how we form a fl eet of cars to travel together? 15 Maybe the cars can 
recognize each other and depending on the need, they can open up a ses-
sion to share conversations or music across many cars. In doing so, they will 
need to communicate with each other. If they are sharing a communication 
channel, they can also identify each other’s location and make it easier for 
their respective drivers to follow each other. Today, some of these functions 
can be performed by various apps on smartphones belonging to the car 
drivers or passengers. For example, using AmpMe , a fl eet of cars can start 
a party and share music. 16 Intelligent machine-to-machine communication 
is a diffi cult topic and it requires a signifi cant amount of cognitive skills to 
share goals, constraints, activities, and responses to questions. Today’s cog-
nitive machines are beginning to learn how to interact with each other in 
simple ways. I anticipate a fair amount of negotiation skills to be developed 
for driverless cars , as these cars will need to diff erentiate between friends, 
strangers, and enemies. 
 As robots start to interact with humans, they need to learn how to antici-
pate and respond to motion, such as handshake or a hug. Cognitive assistants 
must diff erentiate emotions and change their communication style based on 
emotional changes. Sensing and responding to emotion is an important cog-
nitive skill. Classic automation was emotionless and programmatic. A cog-
nitive assistant exhibits a smiley face or a cheerful voice, but also is able to 
change the conversation based on perceived tone, facial expression, or words 
used in the question. As a cognitive assistant senses anger, frustration, or sar-
casm, it must adjust to the conversation. 
2.4 Cognitive Device Engineering 
 Th e usage of a device can be collected and analyzed for valuable feedback to 
the engineering team. Th ere may be components that fail more often and if 
properly designed, could signifi cantly reduce the maintenance costs. Th ere 
may be features that are used once in a blue moon and yet clutter the user 
interface making it complex for day-to-day operation. A device may be hard 
15 Rajit Johri, Jayanthi Rao, Hai Yu, and Hongwei Zhang, “Spatiotemporal Perspectives of Connected 
and Automated Vehicles: Applications in Wireless Networking”, IEEE Intelligent Transportation Systems 
Magazine, Summer 2016, http://ieeexplore.ieee.org/document/7457387/ 
16 Ellie Zolfagharifard, “Th e ultimate party phone: Free AmpMe app lets users link handsets to play music 
together as one giant speaker”, Daily Mail, September 24, 2015, http://www.dailymail.co.uk/sciencetech/
article-3248361/The-ultimate-party-phone-Free-AmpMe-app-lets-users-link-handsets-play-music-
giant-speaker.html 
2 What Is a Cognitive Device? 23
to install and an installation wizard may signifi cantly improve the installation 
headache. How often can the engineering organization collect and analyze 
this usage feedback to improve the design. 
 Quality Function Deployment (QFD) is a method to transform quali-
tative user demands into quantitative parameters, to deploy the functions 
forming quality, and to deploy methods for achieving the design quality