IoT: Conversations with coffee machines

This article first appeared in Enterprise, The Edge Malaysia Weekly, on July 10, 2017 - July 16, 2017.

The Unified Inbox team

Not only do AI and IoT keep each other in check, they also empower each other. They push each other’s boundaries while controlling each other’s security concerns. > Ruckert

-A +A

Imagine having a conversation with your television and lamenting the fact that nothing good is on. Your television, sort of like KITT, the car of Knight Rider fame, commiserates with you, idly flipping channels looking for something that fits your specifications: “The quirky humour and scholarly dialogue of Northern Exposure with the fun of Friends. And maybe a dash of Buffy the Vampire Slayer…”

That may soon be possible as Unified Inbox takes smart appliances to the next level. With its technology, you no longer have to download an additional app or programme the appliance with a lot of complicated commands. You just have to add it as a contact on your phone or device, and send it a voice or text message.

Founder and CEO Toby Ruckert came up with the idea because he found smart appliances, well, rather stupid. “When I bought my first smart appliance, I realised that I would have to download an app and go through, literally, if you counted every one, 10 steps to get it to a point where you could consider it just connected. And while that is still not really smart, you could turn it on and off, change your programme and things like that. That means you would have to download an app, do the user registration and then so many other things that could easily get you frustrated.”

Ruckert, a concert pianist by training, wondered what it would be like if you could actually talk to your devices as if you were friends with them. “I mean, it is just really nice to have your vacuum cleaner and toaster right next to your friends and family on Facebook,” he laughs. 

“I do not know if we will exchange more messages with people or with things in the future, which is a crazy thing to say.”

How would this work? He uses the analogy of making a cup of coffee. “Coffee machines have all these sensors to grind coffee beans — how much water to put in and how much milk. What if I could say to it in natural language, ‘I need a really strong cappuccino, with only a little milk and no sugar?’ 

“You don’t know exactly what the machine will do, but it interprets what you are saying and makes your coffee. You taste it and say, ‘That was good. Please remember it for next time’ or ‘It was good, but just a dash more milk.’”

That is how your coffee machine turns into your favourite barista and produces the perfect cup of coffee, tailored to your personal preferences, every time you ask.

But why is a company that allows you to text your appliances called Unified Inbox? “We actually started out as an app for consumers to unify the different messaging channels into one,” says Ruckert. 

It seemed like a great idea at the time, but he and his partners soon found out that people were not ready for what the company had to offer. “The reality was that people actually liked the different messaging apps and were using them to keep their private and business lives separate. They did not want a single app where they get all their messages,” says Ruckert.

“So, it was not the success we envisioned. I guess it could be a different thing today as people use every channel for both private and business, and are overloaded anyway. But when we started out, the market simply was not ready.”

He and his partners put on their thinking caps to figure out what they could do with the platform they had built. They soon realised that the platform could grow massively if combined with artificial intelligence (AI) — specifically, natural language processing and the Internet of Things (IoT).

Ruckert started the business in New Zealand, where he was living at the time, but later moved to Singapore. “There is a better understanding of the need for the technology [a unified messaging platform] in that market. Everyone in Singapore uses at least four to five messaging apps as many of their friends are from Asian countries that have their own leading chat apps such as WeChat in China, Line in Japan, Kakao in South Korea and Zalo in Vietnam.”

When Ruckert moved to Singapore in 2014, he was still trying to sell his messaging app. Then, he pivoted to a human-to-machine (H2M) messaging platform. “It was really the rise of IoT [that drove this],” he says.

Ruckert went to every conference he could find on IoT to learn about the technology. It was a huge departure for him. “I am a classically trained concert pianist. I do not have a degree in business or programming. But the good thing about being a concert pianist is that it allows you to tap into something new pretty easily,” he says.

Soon, Ruckert was talking about IoT on social media. Before long, he started to be recognised as one of the thought leaders in this space. “I did not say this and there are a lot of people out there who know a lot more. But there are lists that track this and I am usually on the top 10 or 20,” he says.

With his newfound expertise in IoT and experience as a serial entrepreneur, Ruckert changed the direction of the company and the technology platform, from unified messaging to intelligent IoT messaging. “Among other factors, a partnership with IBM (using Watson, a computer system capable of answering questions posed in natural language) helped us bring the intelligence into the IoT messaging platform,” he says.

Unified Inbox already had the messaging part. When it introduced the IoT component into the mix, things started to get interesting. “That is when we plugged into enterprise systems, small cities and manufacturers of smart appliances. The idea behind it and its use is truly simple, but the execution and technology are fascinating and hard,” says Ruckert.

“Basically, we allow you to treat your things like they were a person. You can chat with them (using text or voice) on the communication channels you love and for which you already have apps such as Viber, WhatsApp, WeChat or Skype.”

It looks deceptively simple from the outside. “We apply many technologies to make this happen, from standardising the application programme interface (API) for messaging and IoT to using speech-to-text technologies, machine learning and natural language processing, as well as deep and supervised learning,” he says.

The steep learning curve and the ability to turn on a dime is what Ruckert thinks separates corporates from start-ups. “I think as a start-up, if you do not learn very fast, you are already dead.”

In fact, a start-up is in constant survival mode, he points out. “But at some point, when your vision and mission are very clear, you have to get out of that survival mode and you have to execute,” he says.

What does he mean by this? “If you keep trying out different things, you are not going to make it. You have to be very good at identifying the things that move the zeitgeist, that is, the times in which we live. 

“I was not very good at identifying the zeitgeist seven years ago because the platform we developed would be absolutely more successful today — even as a mobile app — when people get overwhelmed with messaging, social media and interruptions.”

Another thing a start-up requires is a great team. “The team is the key to success. If a start-up gets this wrong, it can die at any time, during the ups and downs,” says Ruckert.

The ups? “Yeah, when people think the company has made it and get greedy. Or when the company is about to die and you need your team to hang in there a little longer,” he says.

“I always look at whether people have a sense of purpose, whether they are curious and are keen to learn, and whether they learn from their mistakes. The moment I detect an oversized ego, I am ready to fire fast. Egos destroy the company culture, which is the energy and purpose the team feeds on.”

It took Unified Inbox a year to set a new course. “We had to change the platform and add the AI component,” says Ruckert.

His background in music came in handy when trying to figure out the best way to get people and machines to communicate. “One of the beautiful things about music is that it is a universal language. There is a reason we are broadcasting compositions of Mozart and the Beatles into space, assuming that an intelligent race will be able to pick them up and use that as a language to communicate with us.”

Ruckert says the piano has a lot to do with communication. “You communicate many voices from the musical piece to a large audience, but you have to reach people individually. You may play for an entire audience, but you also play for every single person there.”

He connects this to what Unified Inbox is doing today. “We have 20 messaging channels. Some people use Line, some use Viber, some use Twitter. At the end of the day, we reach them where they are, on their preferred channel, with their choice of communication, without forcing them to download anything or change their behaviour.”

Unified Inbox’s big break came when device and appliance manufacturers bought into the idea and started to develop paid proof-of-concepts (PoCs) with the start-up. “There is a big difference when engaging with global corporations, whether they pay you for a PoC or whether they just want to play a bit with your technology,” says Ruckert.

Last year, the company signed up its first pilot customers, which plan to release the product at key events in the coming years. Although Ruckert cannot reveal their identity, due to non-disclosure agreements, he can say that the company has worked with Bosch on its range of smart security cameras as it is already public knowledge.

The question is, how did Unified Inbox manage to demonstrate enough competence and reliability for the big boys to start a conversation with it? “We did not talk much. We proved what our technology could do. We delivered our solution and integration within a shorter time than they had expected, and with far better results than anything else they had seen on the market,” says Ruckert.

Although some big names have signed up, the companies Unified Inbox has been working with are keeping their projects under wraps for commercial reasons. The start-up appreciates the sensitivity, but it needs to get its name out there and tell the world what it can do.

“So, we came up with the idea of launching a website that showcases our technology in a virtual environment, free of charge (,” says Ruckert. This website won a Webby Award earlier this year. The Webby Awards for excellence on the internet,  presented annually by the International Academy of Digital Arts and Sciences, is kind of a big deal.

Ruckert has been running businesses (most of them quite successfully) since 1999. So, he had some money tucked away to fund the setting up of Unified Inbox. “Eventually, I raised funds from several independent sources — mostly high-net-worth individuals with networks that I felt would be of value to the company down the road.”

He was reluctant to reveal how much he raised. “Early on, I made a decision never to disclose the total funding received. I find it peculiar that start-ups place such great importance on telling the world how much funding they have. It does not really say anything about the quality of their team, the market-readiness of their product and whether they have the right way forward, beyond the time when the funding runs out.”

Ruckert may have started his first business in 1999, but he has been an entrepreneur nearly all his life. “Even when I was a pianist, I organised my own concerts and invited others to attend them. I ran my own marketing campaigns. An entrepreneur needs creativity and some management and business development ability. The rest is leadership,” he says.

In 1992, he won the first prize in a classical piano competition in Germany. It was also the year he bought his first modem. “I connected the modem to something called a bulletin board system (BBS). It was an early form of the internet and I connected several classmates at school to it as well. Those who used it also bought a modem and used the BBS to exchange electronic letters and files. I ran that in parallel to my musical career, even when I was still in school. It was great fun!”

Ruckert is very much a big picture guy and he says for the past 20 years, there have been two trends that spur each other on. “From 1995 to 2005, we saw the rise of e-commerce, facilitated by the wide adoption of internet connectivity. Without internet connectivity — if you could not order online or if the modem was slow — there would be no e-commerce. 

“From 2005 to 2015, we saw the rise of social media on the back of the rise of smartphones. You could take pictures of what you were doing, upload them and get ‘likes’ straightaway.”

He observes that the next 10 years are going to be about IoT and AI. “The whole decade will be about these two trends massively pushing and empowering each other. They accelerate each other. They innovate each other.”

The reason is simple, he says. “People are now overloaded with information, so they want AI. They are afraid of it, but they still want it.”

But how will AI work? “It needs an infrastructure where it can plug into everything, from your toaster to your smart-city camera to your passport. The infrastructure for AI is IoT and the management of IoT is AI. Not only do these two keep each other in check, they also empower each other. They push each other’s boundaries while controlling each other’s security concerns,” says Ruckert.

Unified Inbox is currently based in Singapore. Does it have any plans to do business in Malaysia? “We want to open an office in Kuala Lumpur. We partnered REDtone IoT Sdn Bhd last year and a new IoT company, Favoriot, this year. Favoriot is run by a friend, Dr Mazlan Abbas, who is one of the thought leaders in IoT.

“His idea is to use our intelligent IoT messaging to power smart cities. So, you could text — on whatever channel you use — your city council about matters that you want them to deal with,” says Ruckert.

He says Unified Inbox has a number of other partners in Malaysia. “We will be signing some MoUs there. What I can tell you is, just like you how you can talk to your appliance, you can talk to your local government. The key is to crowdsource citizens’ needs and desires — whether there is rubbish that needs clearing, a faulty traffic light or someone has parked in the wrong place. Currently, how do you provide such feedback to the authorities?”

Presently, there is no easy way. “We want to facilitate an easy way. Just insert a ‘smart city’ or a ‘smart council’ or a ‘smart something’ contact in your address book and talk to that. And it logs all these issues at the back end and assigns the matter to contractors via messaging. So, everything is clearly recorded and transparent. The key word is always transparency,” says Ruckert.