Arduino MQTT Broker

I am trying to make the Arduino Portenta a complete Edge Impulse Home Automation product. It seems MQTT is the standard method for subscribing and publishing Sensor, Actuator or Machine Learning data.

It seems that everyone uses the Mosquitto MQTT Broker on the Raspberry Pi, or some fancy cloud broker. As far as I can tell an MQTT Broker (other than some fancy bells and whistles) has a fixed IP and just receives an MQTT publish message from a dynamic IP and sends it to all the subscribed dynamic IP’s. That should not be very hard to get working for say 30 clients on the Arduino Portenta (Dual core with Wifi, BLE and Ethernet), however I am not finding much about an Arduino MQTT Broker.

I have just started looking in microPython but also finding nothing:

Am I missing something?

All I have found that is slightly interesting is:

Using C++
https://www.eclipse.org/lists/mosquitto-dev/msg00057.html

Using an ESP32

That’s it. Anyone any suggestions?

I put a similar topic in the Arduino Forum at

@jenny So while looking for a micropython MQTT broker is this you! I think this actual file is for a client, but I am interested in what you did.

https://os.mbed.com/teams/Cloud-Hackathon/code/MQTT-Python-Demo//file/cc36d40e9bf5/broker/broker.py/

Any words of wisdom if this would run on the Arduino PortentaH7 using OpenMV micropython? No worries since I will try it. Very cool to see your name there?

Hello @Rocksetta,

For the Arduino Portenta H7:
I just checked on the Arduino website and it seems the mqtt-client library is available: https://www.arduino.cc/reference/en/libraries/mqtt-client/ (I haven’t tried it myself)

For the ESP32:
Here is an example I wrote when I was still working at my previous company: https://github.com/luisomoreau/ESP32-cam-MQTT (using the PubSubClient library)

For the ESP8266:
Here is a lab I used to teach with some of my Engineering Students:
https://github.com/luisomoreau/IoT-Communication-Protocols-Lab-1 (also using the PubSubClient library)

And for the MQTT broker on Mbed OS, it actually seems that @jenny authored the code sample in your link :slight_smile:

And if you are new to MQTT, I’d recommend to use this lightweight software to visualise your payloads going through the broker: http://mqtt-explorer.com/
It is super easy to use and you can connect to any broker easily (just make sure you are on the same network if you use a Raspberry-Pi-base broker :wink: ).

Best regards,

Louis

Thanks @louis I am getting MQTT clients running on both Arduino and OpenMV. Also mqtt-explorer.com works great so does node-red. My interest is to get a very small MQTT broker working on the Portenta. I will have to look at the code @jenny worked on, the example page I saw was only an MQTT client.

I have a method to run C++ cross compiled programs on the arduino but not expecting that route to have much luck. Also I don’t need a full fledged Broker, just something that can handle 1-5 topics.

Presently looking into the handshake between MQTT client and broker.

So here is the full link to @jenny article

https://os.mbed.com/teams/Cloud-Hackathon/code/

I also got a standard Arduino ESP32 library MQTT broker working on an ESP32 I had lying around a Node MCU-32S Running tinyMQTT

So probably the best bet for me is to try to port this library to the Portenta.

1 Like

Finding a few MQTT and IOT HTTPS webhook sites out there, see my list here, and many charge quite a bit for monthly use ($5/month to $1000/month). This confuses me as the MQTT (port 1883) is a port allowed on most home sites and the mosquitto broker is free and static easy to write webpges can connect to these MQTT sites. See my example here.

Anyone at Edge Impulse have an MQTT broker (or HTTPS webhook) they prefer? @louis @shawn_edgeimpulse @jenny I am having some luck with:

ifttt.com,
pipedream.com,
thingSpeak,
Tagoio,
adafruit.io
and the arduino cloud,

but nothing seems a really good fit for my IOT data, like EdgeImpulse is a good fit for Machine Learning?

P.S. (I use the Helium LoRaWan Network ), free 10,000 data-credits for each email address that signs up at https://console.helium.com/ basically each data push is 1 data-credit (kind of).

Anyone know any other sites I should check out?

Hi @Rocksetta,

I have little experience with MQTT. Of the sites you mentioned:

IFTTT - used to be good, but went to a subscription model and is super wonky to use now (IMO)
ThingSpeak - owned by MathWorks, used to be an open source project. It’s decent, but there are limits on datarates (to be expected for the free version)
Adafruit.io - Free version is pretty solid

Thanks Shawn, fully agree with all three of your views.

1 Like

If I go with adafruit.io for the MQTT, or make my own home mosquito server, anyone got any suggestions for easy to use https webhooks for IOT data. The big solution is for me to go through all of the ones I have found and try them out. Really hoping someone at edgeimpulse has found something they really like.

@jenny

Thinking out loud here, is edgeimpulse interested in providing an integrated Machine Learning IOT data movement site with both MQTT and HTTPS webhook abilities? Probably not, since the space is already flooded, but since edgeimpulse already has my raw data I might be more inclined to trust them with my Edge data as well.

Here are the sites I have found. Twitter sites not shown here only webpages. See my link above for the twitter sites

https://www.digi.com/products/iot-software-services/digi-remote-manager

http://dweet.io/

https://www.edgeiq.ai/platform

https://freeboard.io/

https://www.gaonic.com/

https://docs.golioth.io/

https://www.initialstate.com/

https://www.inteliments.com/en/products-and-services/iot-data-platform

https://iotguru.cloud/

https://liveobjects.orange-business.com/#/liveobjects

https://www.hitachi.com/products/it/lumada/global/en/about/

https://www.m2mi.com/index.php/m2m-iot-application

https://mydevices.com/platform/

https://podgroup.com/

https://reekoh.com/

https://saymon.tech/

https://signomix.com/#!en

https://smart-ix.ai/

https://www.smartthings.com/

https://www.thingsmobile.com/business

https://thingspeak.com/

https://unificationengine.com/

https://ursaleo.com/

https://internetofthings.ibmcloud.com/#/

https://wolkabout.com/platform/#first

https://yaler.net/

https://test.mosquitto.org/

https://www.emqx.com/

https://litmus.io/litmus-edge-manager/

https://www.losant.com/iot-platform

https://www.afero.io/

https://aggregate.digital/

https://www.arkessa.com/

https://www.avimesa.com/

https://beebotte.com/

https://www.boltiot.com/home

https://www.clovity.com/csensornet/

https://www.connio.com/industrial-iot-industrial-iot-platform.html

https://www.softwareag.cloud/site/product/cumulocity-iot.html#/

https://www.digi.com/products/iot-software-services/digi-remote-manager

1 Like

That is an impressive list of sites, many I’d not seen before. :+1:

I’m not sure if this answers your webhooks question but adafruit io does gave an API:
https://io.adafruit.com/api/docs/?cpp#webhooks

Good catch Allen. I didn’t know adafruit.io did Webhooks as well as MQTT, that might be the service I stick with, as the MQTT works so well with the Arduino IDE ready CayenneLPP library . By the way LadyAda is one of my hero’s, even this link is interesting.

Most of this tinyML work we do is not much help unless you can publish some kind of feedback from your data on the web (unless everything you are doing is private). I wish to be able to send Arduino Portenta information using Cellular, Wifi, Ethernet and LoRaWan and then present that information on a static (websocket) webpage.

I am still really confused with MQTT or Webhook websites. I made this 45 min video where I go through the above links (I can barely stay awake).

https://www.youtube.com/watch?v=MGS7GOdAExE&list=PL57Dnr1H_egu0PkIBAbbdfQ21O_NJqJg8&index=3

I did actually find a few sites to check. Presently also looking into making my own MQTT server on Heroku. Adafruit.io seems to be winning the battle for me. Yes @LadderLogic you can use webhooks with Adafruit.io so that is cool, but I think you cannot yet receive webhooks from adafruit.io. So here is my present issue.

I love using HTML, localStorage, websockets or async/await fetch and Javascript with static webpages on Github. I should be able to get my feed or last 30 days of data off the site in JSON format, then I can make my own public dashboard, without giving up my private key. Anyone got any suggestions of sites that allow making your own public dashboard. Seems that most sites like cloudmqtt.com that were free now have a basic $5/month fee.

My work is always for teaching. I am leery of free to try sites because they often want your credit card.

Anyone other suggestions. Arduino Cloud is so close, they just have a few changes and updates to make.

.

.

.

After an adafruit search I have found this

https://io.adafruit.com/api/groups/weather/receive.json?x-aio-key=a0a0a00a0a0a0a0a00a0a0a0a00a0a0a0a0a0a0a0a

which looks like it get all past (30 days) group information. not s websocket but better than nothing. would be easy to poll it every say 30 seconds on a static webpage. The could be entered from a text box to git a bit more security. I would prefer a live websocket for each feed but this looks better than nothing.

Hello @Rocksetta,

I have been using a lot Scaleway’s IoT Hub: https://www.scaleway.com/en/iot-hub/
It is free up to 10 devices per broker (but you can create as many brokers as you want). Here is one lab I used to teach to my students using ESP32 and MQTT with Scaleway’s IoT Hub broker: https://github.com/luisomoreau/ESP32-cam-MQTT

Regards,

Louis

1 Like

That is so awesome @louis. That might fully switch me to scaleway. I never thought of sending images controlled by MQTT, great tutorial. Nice use of github markdown as well.

@louis Did you have this screen for your students? I hope there is a way around it?

Yes, what I did was to create an organisation under my name and then invite the students in my organisation (so they don’t have to go through this screen). But if they create other ressources other than the free ones, you might have to pay for it :D.
I was also making sure that all the ressources were removed at the end of the lab + remove the students from the organisation :slight_smile:.

As most of the resources are hourly billable, even for one 3 or 4-hour lab with 30 students, it does not cost more than a couple of euros/dollars when using paid resources.

Regards,

Louis

Your project @louis has shown me what can be done, but I need to find a non-credit card IOT site, and EdgeImpulse of IOT!. I get their issues with people taking advantage of them, but as a teacher I can’t be using my credit card with any potential for overcharges. Here is my latest list of sites to try, hopefully some of the Ambassadors can get involved. I am looking for the ultimate student IOT site.

These are the interesting IOT sites so far: (I should delete them from the list as issues are found)

https://thingspeak.com/login

https://docs.akenza.io/api-reference/mqtt

https://beebotte.com/overview

https://developers.mydevices.com/cayenne/docs/getting-started/#getting-started-arduino-get-started-in-minutes

https://dweet.io/play/

https://freeboard.io/#pricing

https://iotguru.cloud/signup

https://liveobjects.orange-business.com/#/liveobjects

https://ubidots.com/

2 Likes

This is a big job! At least I now know exactly what I am looking for. To teach IOT connectivity for my EdgeImpulse capable devices I need an IOT site that has:

My Student/Developer IOT Platform Criteria

  1. >= 3 free devices without a needed credit card
  2. Easy to setup MQTT connectivity for multiple devices and platforms
  3. Easy to setup HTTPS connectivity for multiple devices and platforms
  4. Easy to setup Websocket-MQTT connectivity for multiple devices and platforms
  5. Sensible free limited datastorage, say >= 10Mb, >= 2 weeks

.

Problem is that every site takes a fairbit of time to setup and get your bearings.

So far I like but they don’t have all my criteria: Arduino Cloud, Adafruit, Ubidots, thingspeak, pipedream, tagio.

Websockets seem to be the problem, and yes I am being a little stubborn here. However if you have seen the power of the Edge Impulse WASM (Machine learning output for web pages using web-Assembly) and realize that webpages are the most powerful way to reach a massive audience, you will understand the reason to try to get websocket-mqtt working with Machine Learning data.

I am now going to test Databricks and Akenza, but if anyone else has suggestions for IOT devices with potential for the above criteria I would be interested to hear about it.

.

Well that was fast: Databricks you must signup your company, Akensa only give 3 months then needs a credit card. Who’s next?

hiveMQ actually has a working websocket here

2 Likes

So this is starting to get interesting. Presently HiveMQ is my favorite. It has SSL Websocket-MQTT and I already have a vanilla Javascript static webpage working here Not userfriendly as I haven’t taught it yet. For hive you need a special hostname, username and password. To setup multiple pages you need different ClientID’s

So websockets seem to be for me the defining feature of a good IOT integration.

HiveMQ has some issues with a @Helium integration, but that might be able to be solved. This is exciting as this is one of the last things I need for my course.

If anyone knows other sites that have websocket-MQTT please tell me. I am presently trying to find out if some of the above sites have it.

1 Like

Also been trying Qubitro and thingsboard. Both have positives and a few negatives. Still open to suggestions.

AND THE WINNER IS ADAFRUIT.IO!

See the updated website about it here.

I am very impressed with Adafruit. After I emailed them about MQTT-Websockets, they contacted me to describe the issue, assigned a developer and within 24 hours had it working!

I will have an example ready soon.

2 Likes