Join IoT Central | Join our LinkedIn Group | Post on IoT Central


mqtt (5)

 

MQTT (Message Queuing Telemetry Transport, Message Queuing Telemetry Transport) is a lightweight messaging protocol based on publish/subscribe method under the ISO standard, which is usually used for communication between devices and applications such as the Internet of Things and smart homes.

12157441282?profile=RESIZE_710x

The MQTT protocol consists of two parts: publisher/subscriber and message broker. As shown in Figure 1, the publisher is responsible for pushing the message to the broker, and the broker pushes the message to the matching subscriber.
Publisher: The device sends messages to subscribers through topics.
Subscriber: As a terminal device, the subscriber receives messages from the publisher through the topic.
Message broker (Broker): The server acts as a central hub and is responsible for organizational-level communication between publishers and subscribers.

There are two main versions of MQTT: v3 and v5.The principle of these two versions is basically the same, but there are some key differences between them. The following will introduce the differences between them from the following aspects.

01
Protocol format
MQTT v5 has added a Property field, which allows MQTT v5 to support more new features.In MQTT v3, MQTT has nothing to expand, which limits the possibility of MQTT expanding its functions.


02
Subject alias
Subject is the core concept in MQTT, which is used to identify the content and intent of the message.In MQTT v3, the subject is just a simple string, and its structure is composed of a series of words separated by slashes.

For example, an MQTT v3 theme can be sensors/temperature/room1, where sensors is the top-level theme, temperature is its sub-theme, and room1 is a specific device under the sub-theme.

12157441093?profile=RESIZE_710x

However, in MQTT v5, the structure of the theme has been expanded and some more advanced features have been added.Specifically, MQTT v5 introduces a new concept called topic aliasing, which allows clients to map topic strings to pre-defined topic IDs, thereby reducing network traffic and message size.
Subject aliases are maintained by the client and the server, and the life cycle and scope of scope are limited to the current connection.

12157441654?profile=RESIZE_710x

For a topic, set an alias when it is first published, and then you can use the topic alias to publish.This allows the client to send only the subject ID when sending a message, rather than having to send the complete subject string every time.This is very useful for IoT devices and environments with limited network bandwidth.

03
Subscription operation
MQTT v5 introduces a new subscription type called shared subscription.Other flags and filtering functions can be used to achieve more flexible subscriptions.As shown in the figure below, shared subscriptions allow multiple clients to share a subscription and allocate it according to certain rules.This subscription type is very useful for subscribing to high-load topics because it can balance subscription requests and reduce the load pressure on a single client.

12157441873?profile=RESIZE_710x

In addition, MQTT v5 adds the concept of subscription options. You can specify subscription options, such as QoS level, Retain As Publish, Retain Handling, message life cycle, etc., to control subscription behavior more finely.

04
Will news
Will message is the ability that MQTT provides for devices that may be accidentally disconnected to gracefully send the will to a third party.In the payload of the CONNECT message, some fields have changed, among which Will Message (will Message) becomes Will Payload (will Load).

12157442064?profile=RESIZE_710x

Will Properties (WILL Properties) is a new field in MQTT v5. Different types of packets have different attributes. For example, CONNECT packets have attributes such as maximum packet length and session expiration interval, and SUBSCRIBE packets have attributes such as subscription identifier.Moreover, compared with v3, MQTT v5 makes the content of messages more flexible, and can contain any topic and any message content.

05
Error handling
MQTT v5 supports a more detailed error handling mechanism, which can locate and solve problems through error codes and error causes.At the same time, MQTT v5 also introduces a new control message-Disconnect message, which can help clients and servers better handle error conditions.

06
Flow control
MQTT v5 introduces some new mechanisms for flow control on the basis of the v3 version, in order to better control the transmission and processing of messages, and avoid network congestion and excessive load caused by excessive message transmission speed.
Maximum Packet Size limit (Maximum Packet Size): MQTT v5 allows the client and the server to negotiate the maximum packet size when shaking hands.As shown in the figure below, this maximum packet size limit can be used to control the maximum message size transmitted between the client and the server to prevent network congestion and excessive load due to excessive transmission of messages.

12157442089?profile=RESIZE_710x

Message Queue: When the message sent by the server exceeds the speed of the client's processing, the server can store the message in the message queue and wait for the client to process it.MQTT v5 defines the queue size and timeout time of the message queue to control the size and life cycle of the message queue.

07
Performance efficiency
Compared with MQTT v3, MQTT v5 can better handle large-scale data transmission and improve the efficiency and performance of communication.For example, MQTT v5 supports functions such as Batch Publish and Message Prefetch, which can greatly reduce the overhead of MQTT communication.
In short, compared with MQTT v3, MQTT v5 has more new features and security.However, it should be noted that MQTT v5 has added many new functions and concepts. Therefore, when using MQTT v5, it is necessary to have an in-depth understanding of the new features of the MQTT protocol so that this new protocol can be better used to build reliable applications.

Chengdu Ebyte Electronic Technology Co., Ltd. specializes in the research and development and production of various frequency bands and various functional wireless data transmission modules. The products have been widely used in the Internet of Things, consumer electronics, industrial control, medical care, security alarm, field collection, smart home, highway, property management, water and electricity meter reading, power monitoring, environmental monitoring and other application scenarios. 

 

Read more…

IIoT protocols for the beginners

We all know HTTP (hypertext transfer protocol). These are the first 4 alphabets which see on any URL of a website you open in your browser. In simple terms, it is a list of rules that define do’s and don’ts of communication between web browser and web server. It is like you (web browser) going to ATM (webserver) to get some cash (request). Here the HTTP will describe the complete procedure – enter pin, amount, etc. You get your cash (result) once you follow the mentioned steps. Quite simple.

The World Wide Web (WWW) works on HTTP as it is the only protocol used there for the data transfer. However, this is not the case in the Industrial (I) IoT world. Here we have a bunch of protocols to choose depending on the type of application or so-called “use case”. The most common among them are MQTT, CoAP and of course HTTP. Before we discuss them, let us first have a look at certain networking terminologies and definitions.

Source: Pixabay

Transport layer protocols (TCP, UDP)

Transport layer protocol, as the name implies, is responsible for transportation of message or information from one computer to another. The transport of the information can be done in two ways:

  1. Connectionless protocol (UDP): This kind of protocol is preferred in cases where speed and efficiency are more important than the reliability. In this case the data is sent without establishing or waiting for a connection. This means that a bit or segment of data can get lost during transportation. A typical example of such protocol is live video streaming where sometimes bad connection results in the fragmented video. For example, imagine yourself bringing a bunch of letters to the postbox and dropping them inside. You are just dropping the letters inside the box without knowing whether they will be delivered to their recipients. This is the case with connectionless protocols. On the other hand, bringing all these letters to the post office and ordering a return receipt for them, thus ensuring their delivery, can be compared to a connection-oriented protocol.
  1. Connection-oriented protocol (TCP): Here the protocol ensures the receipt of a message at the other end without any data loss on the way, thus ensuring a reliable transport. The connection-oriented protocol needs extra overhead (discussed later) as compared to the connectionless protocol. Just like, it takes extra resources (time, money) to order a registered letter with return receipt.

Packet and Packet size 

packet contains data (payload) along with information (header) like source, destination, size etc. Just like a DHL packet that contains stuff to be shipped along with information like address, weight, dimension etc. packet size in networking, is the amount of data (in bytes) carried over the transport layer protocols.

Overhead

It is the extra information (in bytes) or features associated with the packet which ensures the reliable delivery of the data. In other terms, it is that bubble wrap foil around your shipment that is not necessarily needed but provides an extra layer of safety and reliability for a safe shipment of your parcel.

The amount of overhead associated with the packet depends on the type of transport protocol used. UDP in comparison to TCP has smaller overhead.

Bandwidth

Bandwidth is the rate (bits/MB/GB per seconds) at which the data transfer takes place. The larger the bandwidth, the more data can be transferred at a given time.

So that was a crash course on networking. Now let us try to understand the mentioned IIoT protocols using these terminologies.

Message Queue Telemetry Transport or simply MQTT is a lightweight messaging protocol for industrial and mobile applications. It is best suited for application where network bandwidth and power usage are limited, for example, small sensor, remote location applications, machine to machine communication. MQTT communicates with a server over TCP and unlike HTTP works on publish subscriber model (see figure below).

 

Fig. Example of a publish subscriber model used in MQTT

In order to understand the concept behind the MQTT, one should try to understand the underlying architecture “The publish-subscriber model”. Here a client publishes a message or a topic (temperature, humidity) to a broker that in turn sends these topics out to clients interested in subscribing to that message.

 

The publish subscriber model used in MQTT offers a couple of advantages as compared to the standard client-server model used in HTTP. Multicast, scalability and low power consumption are among the top three. These advantages are due to the fact that the publish-subscriber model overcomes some of the structural (one to one communication, tight coupling, fault sensitive) drawbacks of the traditional client-server model.

Let’s have a look at an analogy in order to understand the difference. Let us assume that MQTT and HTTP are two publishing companies: MQTT publishes magazines on various topics (sports, politics, cars, etc.) and provides them to a broker who in turn distributes them to subscribers interested in one or more topics. This way MQTT can cater many (multicast) subscribers at a given time, thus it is scalable. Since he only has to deal with a broker whom he contacts once a day, his investment (power consumption) in maintaining the business is not high.

 

HTTP, another publisher, likes to deal with one customer at a time. He highly relies on his customer and on his value chain (server to server). This, however, comes at a cost of relatively high business investment (power consumption) since he has to visit his customer each time for a handshake.

 

MQTT in contrast to HTTP is best suited for an application where bandwidth, packet size and power are at a premium. An industry generator with battery-powered temperature and humidity sensor cannot afford to maintain a connection with server each time it has to push the measured values (event or message) into the cloud. MQTT is just designed to overcome such constraints where the connection is maintained by using a very little power and the commands and events can be received with as little as 2 bytes of overhead (extra resources needed for operation).

 

Constrained Application Protocol or simply CoAP, is a UDP based protocol, which is mostly interpreted as a light version of HTTP (except the fact that HTTP works over TCP). It is specially designed to work in a constrained environment with limited bandwidth and power constraints, where communication has to be fast and ongoing. Unlike HTTP, CoAP can support one to many (multicast) requirements and is faster than other TCP based protocols which makes it a good choice for M2M.

 

It is quite common to see the device to device (D2D) or device to gateway (D2G) communication done over CoAP and the communication between gateway and cloud is HTTP job. This is due to the fact that there is a well-defined mapping between these two protocols.

So, if both MQTT and CoAP are good for the constrained environment, then what makes one better than another? The answer lies in the underlying transport layer their use. MQTT is better suited for event-based communication in a constrained environment where data needs to be sent in batches (for instance temperature and humidity values) and at regular intervals over a reliable channel.

CoAP is a better choice for continuous conditioning monitoring scenario in a constrained environment. Since it runs over UDP, CoAP offers faster communication among the devices which makes it a better option for M2M/D2D/D2G communication. CoAP is also best suited for web-based IIoT application where it has to work along with HTTP. In such a setup, you have CoAP at sensor side and HTTP running between proxy/gateway and cloud.

What about HTTP? It is on demand whenever you want to push a big chunk of data from gateway/industry modem/computer into the cloud or a web-based application without compromising on security. Here regardless of how data is collected and sent to a gateway (CoAP vs MQTT) if it comes to reliable big package delivery, then HTTP takes the front seat. Moreover, HTTP is still used as a standard protocol for devices who do not support any other protocols.

MQTT or CoAP or HTTP, it is a matter of speed vs reliability vs security, whichever suits your use case the best.

I hope you enjoyed reading the article and that it helped you to get at least a basic understanding of the major IIoT protocols. Your feedback, comments or suggestions are always welcome.

Read more…

MQTT Library Demo

About The Application

 

publisher.jpg
subscriber.jpg
 
 

To illustrate the use of the MQTT library, we have created two simple Tibbo BASIC applications called "mqtt_publisher" and "mqtt_subscriber".

In our MQTT demo, the publisher device is monitoring three buttons (Tibbits #38). This is done through the keypad (kp.) object.

The three buttons on the publisher device correspond to the red, yellow, and green LEDs (Tibbits #39) on the subscriber device.

As buttons are pushed and released, the publisher device calls mqtt_publish() with topics "LED/Red", "LED/Green", and "LED/Red". Each topic's data is either 0 for "button released" or 1 for "button pressed". The related code is in the on_kp() event handler.

The subscriber device subscribes to all three topics with a single call to mqtt_sub() and the line "LED/#". This is done once, inside callback_mqtt_connect_ok().

With every notification message received from the server, the subscriber device gets callback_mqtt_notif() invoked. The LEDs are turned on and off inside this functions's body.

Testing the MQTT demo

The demo was designed to run on our TPS3 boards, but you can easily modify it for other devices.

The easiest way to get the test hardware is to order "MQTTPublisher" and "MQTTSubscriber" TPS configurations.

You can also order all the parts separately:

  • On the publisher side:
    • TPP3 board in the TPB3 enclosure.
    • Your will need Tibbits #00-3 in sockets S1, S3, S5; and
    • Tibbits #38 in sockets S2, S4, S6;
    • You will also need some form of power, i.e. Tibbit #10 and #18, plus a suitable 12V power adaptor.
  • On the subscriber side:
    • TPP3 board in the TPB3 enclosure.
    • Your will need Tibbits #00-3 in sockets S1, S3, S5;
    • Tibbit #39-2 (red) in S2;
    • Tibbit #39-3 (yellow) in S4;
    • Tibbit #39-1 (green) in S6;
    • You will also need some form of power, i.e. Tibbit #10 and #18, plus a suitable 12V power adaptor.

Test steps

  • Install a suitable MQTT server. We suggest HiveMQ (www.hivemq.com):
    • Download the software here: www.hivemq.com/downloads/ (you will be asked to register).
    • Unzip the downloaded file.
    • Go to the "windows-service" folder and execute "installService.bat".
    • Go to the "bin" folder and launch "run.bat".
    • You do not need to configure any user names or passwords.
  • Open mqtt_publisher and mqtt_subscriber projects in two separate instances of TIDE, then correct the following in the projects' global.tbh files:
    • OWN_IP - assign a suitable unoccupied IP to the publisher and to the subscriber (you know that they will use two different IPs, right?);
    • MQTT_SERVER_HOST - set this to the address of the PC on which your run HiveMQ.
  • Select your subscriber and publisher devices as debug targets, and run corresponding demo apps on them.
  • Press buttons on the publisher to see the LEDs light up on the subscriber.
  • If you are running in debug mode you will see a lot of useful debug info printed in the output panes of both TIDE instances.
  • You can switch into the release mode to see how fast this works without the debug printing.
Read more…

Guest post by Preston Tesvich. This article originally appeared here.

Let’s say you’re in the planning phase of an IoT project. You have a lot of decisions to make, and maybe you're not sure where to start:

 

In this article, we focus on a framework of how you can think about this problem of standards, protocols, and radios. 

The framework of course depends on if your deployment is going to be internal, such as in a factory, or external, such as a consumer product. In this conversation we’ll focus on products that are launching externally to a wider audience of customers, and for that we have a lot to consider.

Let’s look at the state of the IoT right now— bottom line, there’s not a standard that’s so prolific or significant that you’re making a mistake by not using it. What we want to do, then, is pick the thing that solves the problem that we have as closely as possible and has acceptable costs to implement and scale, and not worry too much about fortune telling the future popularity of that standard.  

So, it first comes down to technical constraints:
    - What are the range and bandwidth requirements? 
    - How many nodes are going to be supported in the network?
    - What is the cost for the radio? 


That radio choice has big impacts—not only is it a hefty line item on your BOM on its own, it’s also going to determine the resources that the device needs as well. For example, if you have a WiFi radio at the end, there’s considerable CPU and memory expectations, whereas if we have BLE or some mesh network, it’ll need a lot less. There’s infrastructure scaling costs to consider as well. If we go WiFi: is there WiFi infrastructure already in place where this is being deployed? How reliable is it? If we’re starting from scratch, what’s the plan for covering a large area? That can become very costly, especially if you’re using industrial grade access points, so it’s important to consider these effects that are downstream of your decision.

Zooming in on specific standards

In our opinion, the biggest misconception we find: “Isn’t there going to be one standard to rule them all?” There’s no future of that, and it’s not just because we’re never going to all agree on stuff as an industry, it’s because in many cases different standards aren’t solving the same problems differently, they are solving different problems. So understanding that, we can now look at what each protocol attempts to solve and where they live on the OSI model, or "the stack."

 

MQTT

Some would suggest that it is a full protocol to do communication from a device to a server, but it’s not quite that. MQTT is used as a data format to communicate to something, and that payload can be sent over any transport, be it WiFi, mesh, or some socket protocol. What it tries to solve is to define a way to manipulate attributes of some thing. It centers around reading and writing properties, which lends itself very well to an IoT problem. It certainly saves development time in some regards, but depending on how strictly you’re trying to implement it, it may cost you more development time. As soon as you one-off any part of it, you have to document it really well, and at some point you approach a time and cost factor where implementing your own payload scheme may be a better option.

Is it prolific enough to where you should absolutely use it? No, it hasn't reached that level, and it won’t likely reach that level. What it is right now is a convenient standard for device-direct-to-cloud where we don’t control both ends because it gives some measure of a common language that we can agree on; however, the thing to keep in mind is that most of the time it does in fact need additional documentation—what properties are being read/written and what the exact implementation looks like—ultimately, you’re not getting out of a lot of work using MQTT.

Zigbee and Z-wave

Also starting at the network layer, Zigbee and Z-wave are the big incumbents everyone likes for mesh networking. They attempt to solve two problems: provide a reasonable specification to move packets from one place to another on a mesh network, and actually suggest how those packets should be structured; so, they both reach up higher in the stack. And that's the part hinders their futures. For example, Zigbee uses a system called profiles, which are collections of capabilities, such as the smart energy profile or the home automation profile. When a protocol gets so specific as to say ‘this is what a light bulb does’ it’s pretty difficult to implement devices that aren’t included in the profile. While there are provisions for custom data, you’re not really using a cross-compatible spec at that point—you’re basically off the standard as soon as you’re working with a device not defined in the profile.  

The other consideration with these two is that they are both routed mesh networks. We use one node to communicate with another node using intervening nodes. In other words, we can send a message from A to B to C to D, but in practice we’ve sent a message from A to D. As routed meshes, each node understands the path the message needs to take, and that has an in-memory cost associated with it. While Z-wave and Zigbee have a theoretical limit of 65,535 nodes on a network (the address space is a 16-bit integer), the practical limit is closer to few hundred nodes, because these devices are usually low power, low memory devices. The routing also has a time cost, so a large mesh network may manifest unacceptable latency for your use case. Another consideration, especially if you’re launching a cloud controlled consumer product, is that these mesh networks can’t directly connect to the internet—they require an intervening bridge (a.k.a gateway, hub, edge server) to communicate to the cloud.   

A final caveat is that Z-wave is a single source supplier—the radios are made and sold by Zensys, so you have to buy it from them. Zigbee has a certification process, and there are multiple suppliers of the radio, from Atmel to TI.

Bluetooth

You really just can’t compete with the amount of silicon being shipped based on Bluetooth. 10,000 unique SKUs were launched in Bluetooth in 2014. Other than WiFi, there’s nothing that compares in terms of adoption. Bluetooth was originally designed for  ‘personal area networks,’  with the original standard supporting 7 concurrent devices. And now we have Bluetooth low energy (BLE) which has a theoretically infinite limit. BLE did a ton to optimize around IoT challenges. They looked heavily at the amount of energy required to support a communication. They considered every facet of "low energy," not just the radio-- they looked at data format, packet size, how long the radio needed to be on to transmit those packets, how much memory was required to support it, what the power cost was for that memory, and what the protocol expects of the CPU, all while keeping overall BOM costs in mind. For example, they figured out that the radio should only be on for 1.5ms at a time. That’s a sweet spot—if you transmit for longer, the components heat up and thus require more power. They also figured out that button cell batteries are better at delivering power in short bursts as opposed to continuously. Further, they optimized it to be really durable against WiFi interference because the protocols share the same radio space (2.4GHz).

And then CSR came along and implemented a mesh standard over Bluetooth. Take all the advantages afforded with BLE, and then get all the benefits of a mesh network. The Bluetooth Mesh is flood mesh, meaning instead of specific routing to nodes, a message is sent indiscriminately across all nodes. This scales better than routed mesh because there’s no memory constraints. It’s a good solution for many problems in the IoT and at scale is probably going to be the lowest cost to implement. 

Thread

An up and coming standard that’s built on top of the same silicon that powers the Zigbee radio. It solves the problem of mesh nodes not being able to communicate directly to the cloud by adding IPv6 support, meaning that nodes on the network can make fully qualified internet requests. There’s a lot of weight behind this standard. Google seems to think it’s interesting enough to make their own protocol (known as “Weave”) on top of it. And then there’s Nest Weave which is some other version of Google Weave. As it stands, it takes a long time for a standard to really take hold-- you can immediately see how the story with Thread is a little muddier, which will not help its adoption. It’s also solving a problem that it just doesn’t seem that many devices have. Let’s take sensors as an example. Do these low power, lightweight, low cost, low memory, low processing, fairly dumb devices NEED to make internet requests directly? With Thread, each node now knows a lot more about the world—where your servers are for example, and maybe they shouldn’t be concerned with those things, because not only do the requirements of the device increase, but now the probability and frequency of having to update them in the field goes way up. When it comes to the actual sensors and other endpoints, philosophically you want minimize those responsibilities, except in special cases where offline durability, local processing and decision making is required (this is called fog computing).

When Thread announced their product certification last year, only 30 products submitted. Another thing to note about Thread's adoption is that the mesh-IPv6 problem has been solved before-- there’s actually a spec in Bluetooth 4.2 that adds IPv6 routing to Bluetooth, but very few people are using it. Although Nordic Semiconductor thought it was going to be a big deal and went ahead and implemented it first, it just hasn’t come up much in the industry—that happened Q4 2014 and no one’s talking about it.

One thing Thread does have going for it is that it steps out of defining how devices talk to each other, and how devices format their data—doing this makes it more future proof. This is where Weave comes in, because it does suppose how the data should be structured. So basically a way to look at it is that Weave + Thread = direct Zigbee/Z-wave competitor. We haven’t seen anyone outside of Google really take an initiative on Weave, other than Nest who have put a good marketing effort into making it look like they are getting traction with it.

AllJoyn

Other protocols live higher in the stack and remain agnostic at the network layer. The most well known of these is probably Qualcomm’s Alljoyn effort. They have the Allseen Alliance, although their branding is a bit murky—Allplay, AllShare, etc. We’ve seen some traction with it, but not a ton-- the biggest concern that it’s fighting is that it’s a really open ended protocol, loosely defined enough that you’re really not going to build something totally interoperable with everything else. That’s a big risk for product teams. If there aren’t enough devices in the world that speak that language, then why do I need to speak it? That said, LIFX implemented it, and it worked really well for them, especially since Windows implemented it as well. Now it’s part of Windows 10—there’s a layer specifically for AllJoyn stuff and it seems to do well. There's evidence with AllJoyn that you can bring devices to the table that don’t know anything about each other and get some kind of durable interoperability. However, at a glance, it seems complicated—the way authorization is dealt with and the way devices need to negotiate with each other. There really isn't runaway adoption

IEEE’s WiFi

They’ve ruled the roost with their 802.11 series. B then G then A, and now we have AC. 802.11 has been really good at being simple to set up and being high bandwidth. It doesn’t care about power consumption, it’s more concerned with performance because it’s meant to be a replacement for wires. Almost 2 years ago, they announced 802.11 AH which they’ve branded as HaLow, which attempts to address power, range, and pairing concerns of classic WiFi. Most WiFi devices are not headless ("headless" - no display or other input), they have a rich user interface—meaning we can login and configure them to connect to WiFi. Pairing headless devices has been a very tedious process. With HaLow, they’re solving two problems—how do we get things on easier, and how do we decrease the expectations (particularly power) of the device running the radio. It’s too early to know what type of traction this will get, but IEEE has a great track record at standards adoption.

LoRa and SIGFOX

More like: LoRa vs. SIGFOX. With these protocols we’re looking at how to connect things over fairly long distances, such as in smart city applications. LoRaWAN is an open protocol that's following a bottoms-up adoption strategy. SIGFOX is building out the infrastructure from the top down, and handing APIs to their customers. In that way, SIGFOX is more like a service. It'll be interesting to see the dance-off between these two as the IoT is adopted in these more public-type applications. 

That’s the body of standards that need to be addressed. There’s a ton more, but we don’t see them as exciting for the IoT today.

- P

Read more…

Sponsor