MQTT brokers handle retained messages differently – and that can have serious implications…
\ud83d\udd38 HiveMQ Community Edition doesn\u2019t support to persist retained messages to disk. After a restart, they\u2019re gone – so it\u2019s better to go with the Enterprise Edition. Unfortunately I do not have access to the Enterprise version to perform further tests.
\ud83d\udd38 Mosquitto writes retained messages to disk periodically(!), default is 30 minutes! And during writing from memory to disk, publishing values pauses, it seems the broker gets frozen during writing to disk. Writing 1M retained messages to disk takes time… plus, I got it to crash at about 4460000 retained values (reproducible), and after the crash, it didn’t start anymore – until I removed the database file and lost all retained messages.
\ud83d\udd38 EMQX handled the test with a max of 2000 value changes per second. It was already using about 250% CPU of four cores, and I occasionally received \u201cMnesia is overloaded\u201d warnings. I don\u2019t know whether messages are being queued or dropped at that point. Writing to disk didn\u2019t seem fast. Additionally, a broker startup after storing 1 million retained messages took a while (about 30 seconds), likely because all values are being loaded into memory.
\ud83d\udd38 BifroMQ handled the test at around 8000 v\/s, using only about 20% of four CPU cores. I noticed that throughput decreases as the number of published topics increases. Interestingly, it limits wildcard subscriptions to just 10 retained messages – to protect the system, but that number feels a bit low. I couldn\u2019t find a configuration setting to change it.
\ud83d\udd38 MonsterMQ handled the test with about 7000 v\/s with 18% for the Monster plus 70% for the PostgreSQL database of four CPU cores. MonsterMQ stores retained messages in a PostgreSQL table (written in bulk). If you have a high volume of retained messages, it can lead to full queues – especially if your PostgreSQL instance can’t keep up. In that case, the broker will not persist all values.
\ud83e\udde8 Be careful when using retained messages!
\ud83d\udc49 Understand and test your broker\u2019s behavior before relying on them for data durability.
Just to note: These results are based on my own tests and specific use cases, and were performed on modest hardware. I was publishing retained messages to hierarchically structured topics (e.g., root\/0\/0\/0\/0\/0\/0\/0\/0\/0\/0), using numbers from 0 to 9, without any subscribers.<\/p>\n","protected":false},"excerpt":{"rendered":"
MQTT brokers handle retained messages differently – and that can have serious implications… \ud83d\udd38 HiveMQ Community Edition doesn\u2019t support to persist retained messages to disk. After a restart, they\u2019re gone – so it\u2019s better to go with the Enterprise Edition. … Continue reading