Hello and welcome to episode five of Adafruit and Digi-Key’s “All the Internet of Things”! A six-part series that covers everything you need to know about IoT. This video series has built up from the basics to more advanced topics, as we take you through the journey of IoT best practices. In previous episodes we’ve discussed the transports, protocols, and services you need to craft your IoT functionality. We’ve also showed off working example code for sending and receiving data with MQTT versus REST, and when to use WiFi, cellular and Bluetooth so you can start making the right decisions for which transports and protocols to use.

The running joke is that the ‘S’ in IoT stands for security... because it's never there. But safety and security is something you will need to think about at all steps of your design process. There are going to be billions of IoT devices on-line around the world, many of which will be connected to the internet, and almost all of them will be unmonitored. 

A 2015 survey by authentication service provider Auth0 found that 85% of IoT developers admitted to being pressured to get a product to market before adequate security could be implemented. And if you are an engineer, you’re probably used to that pressure to get a product to market, in which case selling features often get more attention than security.

With more and more of these connected devices being rushed to market, they’ve become a lucrative target. The 2016 Mirai botnet attack used unsecured CCTV cameras that were connected to the internet to launch a crippling denial of service attack. That one wasn’t even using the cameras to spy on people. The malware was just using the TCP/IP stack of the embedded Linux device to send lots of junk traffic.

And we’re not even getting into the hacks that could really threaten human safety like remote-controlled ovens or self-driving cars.

So while it might not seem like a big deal when you have an unsophisticated IoT device that has a temperature sensor and a modem, that device could be used as a launch-point for a coordinated attack. And if you do have sensors like cameras or microphones, those could be remotely enabled and turned into listening devices.

Having security as a priority for your engineering and marketing team will not just help you sleep well at night. As we’ve seen with the European GDPR regulations, privacy and security are being legislated. Having poor security will now get you fined and banned in the marketplace. It’s nearly impossible to add security after the fact, so if you want to avoid a devastating recall - listen up and take security seriously.

Before we start looking at attack and defense mechanisms. Let’s talk about why your hardware might get hacked. Knowing what people want to do with your hardware will give you a sense of the actors involved, and their motivations.

When we talk about ‘hacking’ your hardware, we mean unauthorized access and use of the device and data within it. That doesn’t necessarily mean repurposing or improving it - after all, we’re makers here at Adafruit and modifying off the shelf hardware is an engineer’s favorite pastime. So, for example, recycling a Bluetooth step counter into a doggy activity tracker is not hacking - but being able to access, read and modify that same step counter data without permission from the owner would be!

For IoT devices that connect to the internet, there are a few common reasons we see for compromising hardware. The first common type of hack is creating DDoS botnets, that is - for distributed denial of service attacks. DDoS attacks use several or even thousands of compromised or co-opted computers, also known as ‘bots’, to inundate an online service with coordinated data requests which overwhelm the target and take it offline.

This can be used as part of a revenge scheme - to punish a competitor, or as a protection racket, to demand money in exchange for calling off the attack. It’s popular because there are plenty of software packages to manage botnets, that means it doesn’t require a lot of skill to do.

The Mirai Botnet

As we mentioned earlier, the infamous Mirai malware infected thousands of IoT devices to turn them into launch platform for DDoS attacks. By automatically hacking into and taking over insecure IoT devices, the Mirai virus quickly spread through thousands of cameras connected to the internet, turning them into bots. Some time later, those bots were wielded in the DDos attack. Botnet-infected devices will have degraded performance, but are very hard to detect if you don't know what to look for.

With the proliferation of crypto-currency, some botnets are turned into crypto-farms, where your device is used to mine virtual cash. So far, this has been happening with hacked servers and desktop computers, but we’ll see this more as IoT devices become more powerful - some single board computers are now nearly desktop speed. If one is controlling thousands of them, it can add up. We suppose it’s better than attacking a third party, but that hashing software will make your device slow down and that could cause support issues when your customers are disappointed with the performance of your IoT product.

Botnet hacks are extremely common, but you’ll also see hacks that attempt to steal personal data - especially passwords and payment information. Without question, you should avoid having any way for your IoT device to interact with money, either sending or receiving, because that will make it a huge target. That goes extra if the money is untraceable such as cryptocurrency, because it's so easy to launder. Storing and managing credit card data is so incredibly dangerous that it’s beyond the scope of this video, suffice to say there are whole industries surrounding mandated PCI compliance, you will need to follow those rules if you want to process credit cards.

Usernames and passwords, if they’re extractable, are valuable on the black market - they’re used to try and log into other services with reused or shared login info, where money or compute resources may be available.

Finally, the user data, say how many steps they took per day, or video from wifi cameras, may be used to harass or abuse your customers. These sorts of hacks are sometimes one-at-a-time exposures, where a customer is tricked into giving someone their password, but sometimes an entire backend database will be downloaded! Having that data exposed will be devastating to your customers, and once your products are known as untrustworthy, its extremely hard to regain customer trust. So treat their data as if it were your own - which it kinda is.

Let’s take a closer look at the types of attacks that can be used to compromise IoT devices, so we know what to defend against.

Sun Tzu wrote “If you know the enemy and know yourself, you need not fear the result of a hundred battles” and knowing what to defend against will let you prioritize your time and energy. 

Attacks are how your opponent will be trying to crack your security, so they can gain control or extract information from your IoT device. Some attacks are easy to counter, some are extremely hard. How much time and money you spend will depend on what you’re protecting and protecting against.

Automated logins

The most common type of attack is an automated login tool, that will try to connect to every IP address on a network and login using default passwords or common passwords. For example, if an IoT camera is connected to the open internet and has a default password and login of admin/admin, that would be super easy to automate with a script. Every time a new IoT device is sold with a default login, that’s added to the hacking list, so that the script will try anything it can. These tools are incredibly effective because they can sweep through millions of IP addresses around the world, while the hacker is sound asleep.

This attack is really easy to defend against, and you must spend the effort. Most obvious is don’t have a default password - instead, have the password distributed with the product on a sticker, just like your WiFi router has. You can guess who got bit by this security flaw after many years: router manufacturers! Make sure the password is long and complex enough so it cannot be guessed - and don’t use public information in generating the password, like a MAC address.

Automated exploits

A close relative to automated logins is an automated vulnerability exploit tool. Say you have the bestest, longest password, but it turns out there’s a flaw in the firmware or operating system you’re running. That flaw could be exploited to allow an outsider access into the device even without a password. The good news is that these exploits are not very common if you’ve configured your IoT operating system well. The bad news is that when they do happen, they can be catastrophic. There’s no defense against them other than upgrading or patching the software, and there’s no way to predict or protect against it - even the most expensive and proprietary systems can have flaws.

In this case, all you can do is take steps to minimize the risk, and maximize the ability to repair. If at all possible, do not have a public-facing internet administration system. Disable SSH, ftp, telnet, webserving, as well as any other service you can do without. Run the latest operating system and have a way for users to update the firmware, or have automatic firmware updates that are fetched and installed by the system itself. If there’s an app, have it remind the user to update constantly. And, if possible, have a way of contacting customers so you can let them know if there’s an emergency update - it’s better to have it and not need it, than need it and not have it.

Sniffing, Spoofing, Replay, and Man-in-the-Middle

The next most common set of attacks, we’ll group together and call the “sniffing, spoofing, replay, and man-in-the-middle” group.

Sniffing is the ability to listen in to the communication going on inside or through your device.

Spoofing is being able to trick the device into trusting something it should not. Replay is sending data (often sniffed) over again to a device. Sometimes this lets the hacker repeat an event, sort of like spending the same quarter twice.

And Man-in-the-middle attacks are a little like a cross between sniffing and spoofing - you get ‘in the middle’ of the device communications, shuffling data back and forth but modifying the data to suit your hacking needs

These techniques are often used together. For example, let’s say you have an IoT device with a login website. You’ve got a unique password requirement. But, you forgot to require encryption for the connection. Your customer logs into the device from the coffee shop across the street and it turns out that their network has been hacked (due to a default login-password on the WiFi router) and someone listens to all the WiFi traffic and records the login and password for the IoT device, which they can then reuse - that’s a sniffing attack.

Or, say you want to defend against the Automated exploits class of attacker, so you create an over-the-air firmware update service that will send updated firmware once a week. A hacker buys one of these devices, and listens in to the network traffic during the firmware update and discovers you forgot to authenticate the connection. With some effort, she is able to craft a custom firmware upgrade, that creates a default-login backdoor, and deploys it in an automated fashion across the internet, targeting every installation of the device, for global domination. That’s a spoofing attack.

That’s not all the attacks you have to worry about, there’s a long tail of many more ways to trick computer devices - after all computers do whatever we ask, without judgement. For some companies, especially ones with health, safety or money on the line, there’s a lot of continuous research needed to keep up to date with attack vectors, and stay ahead. For most others, it’s better to follow the best practices we’ll outline than do nothing. And, as your company and product succeeds, you can invest more into improving security such as hiring specialists.

Some things you shouldn't focus all your time on is having your hardware manufacturer modify your design to add spy technology to the PCB. We’re not saying it’s impossible, just that while these sorts of attacks get a lot of attention, they are very expensive and time consuming, compared to the automated attack scripts that are your greatest risk. When you outsource your manufacturing, there are some risks involved - Your contract manufacturer is most likely to swap components on you, using lower quality or cloned components, or perhaps sell off your IP to a competitor, rather than trying to modify your hardware. And, if you’re the kind of company that has to seriously worry about state-level attackers, with vast resources, you are also the kind of company that owns their own manufacturing, or can afford to staff the CM with auditors.

That said, after you go to manufacturing, it's not a bad idea to disassemble and inspect a few random boards from your fabrication run, to catch quality and security issues such as "are those unique passwords really unique" Have your firmware images cryptographically signed or authenticated, and then verified post-manufacture by your in-house team (or a trusted auditor). 

Of course, this isn’t all the ways that you’re going to get attacked, just some of the most common ones. So, how are you going to protect your design?

“Attack surface reduction” is a security principle that you can use to guide your choices when designing an IoT product or service. The attack surface of a hardware or software environment is all of the different points where an unauthorized user can try to insert or extract data. Keeping the attack surface as small as possible is a basic but necessary security measure. 

With IoT, there’s two surfaces you’ll have to contend with. The thing itself, say an internet-connected temperature sensor, and the communicating service - whether that’s Adafruit.IO, Google Cloud, or Microsoft Azure.

Let’s start with device security, starting from the easiest first, here are ten-plus guidelines for device security. Again, this isn't everything, but it’s an excellent start.

#1 Require a login and password - this is number one because it's the bare minimum. Don’t have an open, network-accessible interface to your IoT device. You may think “oh nobody is going to guess the URL, or the port number” but that’s the first thing hackers will find out! Even if its on an intranet, require some authentication!

#2  Don’t have default logins and passwords - we mentioned this before but it bears repeating because it’s so common! Make sure your device has a unique, unguessable password by default.

#3 Two-factor authentication - In addition to a username and password, maybe have an SMS or time-based second factor. This will protect you even when if the password is sniffed or stolen. 2FA is free and pretty easy to implement these days - you no longer have to distribute a physical token, since everyone has a mobile phone.

#4 Require TLS/SSL - Whenever your users or devices connect to the internet, whether over WiFi or cellular, use the latest available version of TLS, sometimes called SSL or HTTPS. TLS will encrypt all data transmitting between the device and the service, protecting both. This will greatly reduce your risk of sniffing. A few years ago, microcontrollers were older and smaller and couldn’t effectively run a TLS stack. Nowadays, there’s no excuse to skip it.

#4.5 Authenticate Host Certificates - TLS is not just data encryption, it's also server authentication. So, if you’re using TLS, make sure your device is checking the fingerprint or certificate chain of the servers it's connecting to. We’ve seen some TLS implementations where it's possible to skip this, which makes man-in-the-middle attacks possible.

#5 Turn off any unused services - If you have an embedded Linux or RTOS for your device, make sure there are no services left on. File sharing, remote login, mail servers, etc. These days, most of these are not enabled by default, but check anyways. Sometimes these are left on during development, and are forgotten when the firmware makes it to release.

#5.5 Don’t accept any inbound connections - If you can, don’t allow any way for outside parties to connect into the device. If you have a debugging port left open that’s just another surface that can be attacked.

#6 Require physical access for important configurations - We’ve seen some WiFi cameras that can be controlled over the internet, but if you want to change the access point password, you need to plug it into a computer and change the setting over USB. This reduces the surface that can be attacked by automated scripts.

#7 Individualized/Revocable Authentication Keys - In order for your device to connect to the service, chances are it has some sort of authentication key or password. As you remember from earlier, make sure that you have a unique key or password for each device - even if the user never sees these, you shouldn’t reuse them. You’ll also need to have a way to revoke/re-instantiate keys if they’re lost, corrupted or stolen.

#8 Data Paranoia - Even though you may be only shuffling data from your IoT device to your IoT service, don’t trust that the data is well formatted. This is often forgotten in the rush to complete and ship firmware, but you should assume that attackers will try to send corrupted or malformed chunks of data to both sides of the connection to corrupt memory. Clean up and vet data fully, this will also keep your device running smoothly if the network connection is flakey.

#9 Updatable Firmware - Bootloaders are the best, and it's a good idea to have one on your device. There are many that are write-only so that the deployed firmware can’t be read back out. Being able to update firmware will help customers recover the device if it gets bricked, hacked, or if there’s an important security update. We like USB bootloaders the best, or ones where you insert an SD card with a file. Having updatable firmware increases your attack surface a bit, because it gives another access point into your device, but we think that if someone has physical access, they could connect a JTAG programmer to erase and reprogram it anyways. 

#10 Secure storage for authentication keys - Embedded Linux devices have a regular filesystem, and microcontrollers often store their code in flash memory, so even if you hard-code authentication keys in flash or EEPROM, it can be read out. Yes, even if you have a chip that has firmware-readback turned off, it’s possible to glitch chips into revealing their secrets. Your microcontroller memory should not be considered a secure storage! Instead, you may want to consider using a secure element chip. These chips are designed to withstand common decapping and glitching attacks and can be programmed with the private key at your factory. Then, it never leaves the secure chip. Instead of having the key sit in microcontroller memory where it could be read out, data that needs to be authenticated or encrypted is sent back and forth through I2C. It’s a little extra BoM cost but is a nice way to keep the secrets in a lock-box.

#11 Over the air updates - This one is a little tricky. Not having OTA is risky because then there’s no way to send important security updates. On the other hand, having OTA is risky because, if hacked, it allows the attacker to completely take over the device. We think OTA is a good idea, but you definitely need to combine it with the prior rules - firmware must be transmitted over an authenticated, encrypted connection. Having firmware be signed with public key cryptography (so the private key is not stored on the device) is a common idea, but be aware that private keys can leak out.

We’ve seen more than one company accidentally ‘brick’ their devices with a mistaken OTA update, some even required a physical recall - so if you do have OTA updates, make sure you always have a way for physical-access-rollback. And do lots of slow-rollout deployments, so you don’t end up with a mistake that affects everyone.

#12 Have a Security Contact - This is last but it’s actually super-easy. Make sure that your website has a way for folks to contact you if they think there’s a security problem. There are lots of amateur and expert security researchers who are also your customers, or potential customers. Many eyes can make flaws obvious, and many will let you know if they find one. Don’t let those emails go unread - have someone who is an assigned security contact, who will read and respond to any bug reports. Small gifts and rewards can encourage responsible disclosure and will motivate smart folks to help you out.

Next up: Service Security!  Servers have been hacked since the first ones were put online, so there’s a lot of information out there on best security practices. Many of the recommendations we made for devices still stand - require TLS, 2FA, and strong passwords, scrub data to prevent buffer overflows or cross site scripting attacks. Here’s a few more that relate specifically to IoT services

Misconfigured or glitching devices can turn them into unintentional mini-DoS attacks. If you get a device that is repeatedly sending your service bad data, make sure you have a way of throttling connectivity so other customers don’t get locked out. Contact or alert the customer to let them know about the misbehaving device.

Provide a device-health dashboard. Things like power and bandwidth usage, when reported back to the service, can help you spot if devices have turned into a 'cyber currency' coin miners. Of course, these reports can be manipulated if the device is completely taken over, so you shouldn’t only rely on them.

Help protect customers from compromised accounts.  Developers will accidentally commit their authentication key - it happens all the time. That’s why we had that earlier guideline to allow easy key revocation. A favor you can do to your users is to watch GitHub, Pastebin or other code-sharing services for credentials. Don’t actually search for key values, of course, look for your service name and words like “password” “API” or “key”.

Protect yourself from compromise - Look out for world-readable or misconfigured cloud storage buckets, git repositories, web directories. 

And of course, for both your IoT device and service… if it has a web interface, it should be protected against standard website hacking techniques like remote code execution, path traversal, cross-site request forgery, and SQL injection. There are scanning services you can run against the website as well as on the code itself to find the egregious errors.

Use monitoring services on your own service, so meta! Hosting a service makes you an open target, and especially if you have a firmware deployment service Someone taking control of your site could not only take down the site, they could take control of all the devices too! Check your daily storage, bandwidth and compute resource usage graphs, to see if anything is amiss.

This video is fairly short compared to the decades of security research. So we aren’t able to cover everything, just the big picture. But here’s the most important thing to realize:

IoT will never be completely secure.

By definition, having something be electronic and networkable means it can be hacked. And if you’re in business long enough, your device will eventually have security flaws exposed. Now, it shouldn’t be happening often, and hopefully it isn’t something obvious, but there’s just too much code involved in an IoT device so be bug-free. 

Recognizing that you will never be 100% secure is the second half of Sun Tzu’s quote about knowing the enemy and knowing yourself. And that will guide you in your IoT product design. 

You should assume that your firmware will get decompiled, and that your service database will be downloaded. So, think about how you can minimize the impact of those events. Don’t store plain-text data that, once released, is devastating. Don’t write your own cryptographic methods that, once reverse-engineered, unravel your network’s authentication scheme. If you have to store data securely, rely on experts at services that do it well, they can be partners that let you focus on the customer experience.

And most importantly, care about your customer data. To you, a leaked database is a statistic, to them it's a tragedy. Tell them that their security matters, be open about how you’re going to do it, and listen when experts try to warn you about security flaws.

When it comes to the internet and data security, your work is never done. As technology gets more connected and complex, there will be new hacking techniques discovered. We want to stress that security is not a one-and-done step of your product, but a philosophy that you’ll need to keep in mind from prototype and deployment to long term support. IoT inherits many of the security problems networked computers have had for years, so it’s important for us as hardware engineers to learn from our fellow software and network engineers.

As designers of IoT devices, you must use modern best practices in delivering secure products to end users. The reverse is also true - investigate the market you’re entering to see what security practices your competition practices or ignores. Keep tabs on the latest hacking announcements to make sure your products are not similarly exposed. Re-evaluate security on a periodic basis to review threats and revise software, access codes, and architecture.

Being transparent and open about security practices before, during, and after an issue will also demonstrate your commitment to security, and fixing things when needed, and it WILL HAPPEN.

We hope that, during this episode, you have realized how important it is to secure your IoT project, some of the best ways to go about it, and what to keep in mind when purchasing IoT devices. We hope you’ve enjoyed this video series on the growing world of the Internet of Things. When you’re ready to start getting some hands-on experience please check out all of the IoT products from Digi-Key and Adafruit, and check back for the final video in this series where we design a device with Digi-Key’s IoT Studio.