version 1.1.9 and release note

This reverts commit ab8ae9d0c9.

.github/ISSUE_TEMPLATE.md

@@ -11,7 +11,13 @@
 你认为正确的表现应该是什么样的。
 
 #### 补充条件
-是否是毕现的，还是偶现的？是否只有在特殊的网络/设备/平台上出现，还是所有的都出现。还有您用的版本是什么时候的，是不是最新版
+是否是必现的，还是偶现的？是否只有在特殊的网络/设备/平台上出现，还是所有的都出现。还有您用的版本是什么时候的，是不是最新版
 
 #### demo对比结果
 请用demo对比测试，demo上是什么状态。
+
+#### 检索已有问题
+除了可以检索已有issue外，还可以去野火论坛查找已有问题，论坛地址为 https://bbs.wildfirechat.cn
+
+#### star 我们
+给我们点个star，可以及时接收到我们回复信息，另外也是对我们支持人员辛苦工作的认可，激励我们更好更快地为大家服务。

README.md

@@ -4,10 +4,11 @@
 ## 功能特性
 * 极致地硬件利用率，IM服务最低128M内存即可运行，上不封顶。
 * 协议先进，采用MQTT+Protobuf组合，流量和性能极致优化。
-* 性能强大，专业版IM服务支持百万在线和集群部署。
+* 性能强大，专业版IM服务支持百万在线和集群部署，业内顶尖性能水平，详情参考野火性能报告。
 * 部署运维简单，依赖服务少，稍加配置一键启动。
 * 安全加密。网络连接AES加密。客户端数据库SqlCipher加密。安全无小事。
-* 全平台客户端，四端同时在线（移动端，pc端，web端和小程序端），数据和状态多端完美同步。
+* 全平台客户端，支持Android、iOS、Web、小程序、UniApp、Flutter、Windows、Mac OS、Linux等所有常见平台。
+* 四端同时在线（移动端，pc端，web端和小程序端），数据和状态多端完美同步。
 * 支持国产化。支持国产化操作系统、国产化芯片和国产操作系统。支持国密加密。
 * 客户端使用微信[mars](https://github.com/tencent/mars)连接库，野火IM可能是最适应中国网络国情的即时通讯服务。
 * 支持加速点加速，即可用于全球应用，也可用于政企内外双网复杂的网络环境。
@@ -15,37 +16,47 @@
 * 音视频多种解决方案，丰俭由人，可自由选择。
 * 高级音视频功能强大，支持9人以上的群组视频通话，支持1080P视频，支持会议模式，支持百人以上会议，支持服务器端录制。
 * 全私有部署，可不依赖任何第三方服务，完全内网部署。
-* 功能齐全，涵盖所有常见即时通讯功能。另外具有强大的可扩展能力。Demo成熟完善，基本可以做到开箱即用，也可把SDK嵌入其它应用。
+* 功能齐全，涵盖所有常见即时通讯功能。另外具有强大的可扩展能力。代码开源率高，二次开发简单。
+* 结构设计合理，客户端分功能库、UI库和应用三层。应用成熟完善，开箱即用，也可把SDK嵌入其它应用。
 * 拥有应用开放平台，可以开发和创建自建应用，扩展您的工作台。
 * API丰富，方便与其它服务系统的对接。
-* 拥有机器人和频道功能。
+* 拥有机器人和公众号功能，免费的频道（公众号）管理后台。
+* 超级群组功能，可以支持万人大群。
+* 社区版IM服务和移动端免费商用，可以免费构建出完备的移动互联网应用。
+* 收费项目非常便宜，野火独有的试用模式，可以先部署到客户环境试用6个月，试用满意后再购买，避免上当受骗。
+* 技术支持专业和高效，核心研发人员及时处理issue和论坛回复。
 
 ## 野火开源项目
-主要包括一下项目：
+主要包括以下项目：
 
-| [GitHub仓库地址(主站)](https://github.com/wildfirechat)      | [码云仓库地址(镜像)](https://gitee.com/wfchat)        | 说明                                                                                      
+| [GitHub仓库](https://github.com/wildfirechat)      | [码云仓库](https://gitee.com/wfchat)        | 说明                                                                                      
 | ------------------------------------------------------------ | ----------------------------------------------------- | --------------------------------------------------------------------------
-| [im-server](https://github.com/wildfirechat/im-server)       | [im-server](https://gitee.com/wfchat/im-server)          | 野火社区版IM服务，野火IM的核心服务，处理所有IM相关业务。                                                                                 |
-| [app_server](https://github.com/wildfirechat/app_server)     | [app_server](https://gitee.com/wfchat/app_server)     | Demo应用服务，模拟客户的应用服登陆处理逻辑及部分二次开发示例。                                                                                |  
-| [robot_server](https://github.com/wildfirechat/robot_server) | [robot_server](https://gitee.com/wfchat/robot_server) | 机器人服务，演示野火机器人对接其它系统的方法。                                                                              |
-| [push_server](https://github.com/wildfirechat/push_server)   | [push_server](https://gitee.com/wfchat/push_server)   | 推送服务器，可以对接所有的系统厂商推送服务或者第三方推送服务。                                                                                |
-| [wf-minio](https://github.com/wildfirechat/WF-minio)   | [wf-minio](https://gitee.com/wfchat/WF-minio)   | 私有对象存储服务，用来支持野火IM专业版的文件存储。                                                                                |
-| [wf-janus](https://github.com/wildfirechat/wf-janus  )   | [wf-janus](https://gitee.com/wfchat/wf-janus  )   | 高级音视频媒体服务。                                                                                |
-| [open-platform](https://github.com/wildfirechat/open-platform)   | [open-platform](https://gitee.com/wfchat/open-platform)   | 野火开放平台服务。                                                                                |
-| [daily-report](https://github.com/wildfirechat/daily-report)   | [daily-report](https://github.com/wildfirechat/daily-report)   | 野火开放平台日报 demo服务。                                                                                |
-| [android-chat](https://github.com/wildfirechat/android-chat) | [android-chat](https://gitee.com/wfchat/android-chat) | 野火IM Android SDK源码和App源码。
-| [ios-chat](https://github.com/wildfirechat/ios-chat)         | [ios-chat](https://gitee.com/wfchat/ios-chat)         | 野火IM iOS SDK源码和App源码。
-| [pc-chat](https://github.com/wildfirechat/vue-pc-chat)       | [pc-chat](https://gitee.com/wfchat/vue-pc-chat)       | 基于[Electron](https://electronjs.org/)的PC 端，支持Windows、Mac、Linux（包括国产化linux系统和CPU）。                                       |
-| [web-chat](https://github.com/wildfirechat/vue-chat)         | [web-chat](https://gitee.com/wfchat/vue-chat)         | 野火IM Web 端, [体验地址](https://web.wildfirechat.cn)                                     |
-| [wx-chat](https://github.com/wildfirechat/wx-chat)           | [wx-chat](https://gitee.com/wfchat/wx-chat)           | 小程序平台的Demo(支持微信、百度、阿里、字节、QQ 等小程序平台)                             |  
-| [uni-chat](https://github.com/wildfirechat/uni-chat)           | [uni-chat](https://gitee.com/wfchat/uni-chat)           | UniApp平台移动端应用(使用原生插件)                           |  
-| [uni-wfc-client](https://github.com/wildfirechat/uni-wfc-client)           | [uni-wfc-client](https://gitee.com/wfchat/uni-wfc-client)           | UniApp平台原生插件                           |  
-| [flutter_imclient](https://github.com/wildfirechat/flutter_imclient)           | [flutter_imclient](https://gitee.com/wfchat/flutter_imclient)           | Flutter平台原生插件                           |  
-| [docs](https://github.com/wildfirechat/docs)                 | [docs](https://gitee.com/wfchat/docs)                 | 野火IM相关文档，包含设计、概念、开发、使用说明，[在线查看](https://docs.wildfirechat.cn/) |
+| [im-server](https://github.com/wildfirechat/im-server)       | [im-server](https://gitee.com/wfchat/im-server)          | 野火社区版IM服务，野火IM的核心服务，处理所有IM相关业务。  |
+| [app-server](https://github.com/wildfirechat/app-server)     | [app-server](https://gitee.com/wfchat/app-server)     | Demo应用服务，模拟客户的应用服登陆处理逻辑及部分二次开发示例。 |
+| [robot_server](https://github.com/wildfirechat/robot_server) | [robot_server](https://gitee.com/wfchat/robot_server) | 机器人服务，演示野火机器人对接其它系统的方法。 |
+| [push_server](https://github.com/wildfirechat/push_server)   | [push_server](https://gitee.com/wfchat/push_server)   | 推送服务器，可以对接所有的系统厂商推送服务或者第三方推送服务。 |
+| [wf-minio](https://github.com/wildfirechat/WF-minio)   | [wf-minio](https://gitee.com/wfchat/WF-minio)   | 私有对象存储服务，用来支持野火IM专业版的文件存储。 |
+| [wf-janus](https://github.com/wildfirechat/wf-janus  )   | [wf-janus](https://gitee.com/wfchat/wf-janus  )   | 高级音视频媒体服务。 |
+| [open-platform](https://github.com/wildfirechat/open-platform)   | [open-platform](https://gitee.com/wfchat/open-platform)   | 野火开放平台服务。  |
+| [daily-report](https://github.com/wildfirechat/daily-report)   | [daily-report](https://github.com/wildfirechat/daily-report)   | 野火开放平台日报 demo服务。 |
+| [channel-platform](https://github.com/wildfirechat/channel-platform)   | [channel-platform](https://gitee.com/wfchat/channel-platform)   | 野火频道(公众号)平台服务。  |
+| [organization-platform](https://github.com/wildfirechat/organization-platform)   | [organization-platform](https://gitee.com/wfchat/organization-platform)   | 野火组织通讯录服务。  |
+| [android-chat](https://github.com/wildfirechat/android-chat) | [android-chat](https://gitee.com/wfchat/android-chat) | 野火IM Android SDK源码和App源码。 |
+| [ios-chat](https://github.com/wildfirechat/ios-chat)         | [ios-chat](https://gitee.com/wfchat/ios-chat)         | 野火IM iOS SDK源码和App源码。|
+| [pc-chat](https://github.com/wildfirechat/vue-pc-chat)       | [pc-chat](https://gitee.com/wfchat/vue-pc-chat)       | 基于[Electron](https://electronjs.org/)的PC 端，支持Windows、Mac、Linux（包括国产化linux系统和CPU）。   |
+| [web-chat](https://github.com/wildfirechat/vue-chat)         | [web-chat](https://gitee.com/wfchat/vue-chat)         | 野火IM Web 端, [体验地址](https://web.wildfirechat.cn)。  |
+| [wx-chat](https://github.com/wildfirechat/wx-chat)           | [wx-chat](https://gitee.com/wfchat/wx-chat)           | 小程序平台的Demo(支持微信、百度、阿里、字节、QQ 等小程序平台)。  |
+| [uni-chat](https://github.com/wildfirechat/uni-chat)           | [uni-chat](https://gitee.com/wfchat/uni-chat)           | UniApp平台移动端应用(使用原生插件)。   |
+| [uni-wfc-client](https://github.com/wildfirechat/uni-wfc-client)           | [uni-wfc-client](https://gitee.com/wfchat/uni-wfc-client)           | UniApp平台原生插件。已[商店](https://ext.dcloud.net.cn/plugin?id=7895)上架   |
+| [wfc_flutter_plugins](https://github.com/wildfirechat/wfc_flutter_plugins)           | [flutter_imclient](https://gitee.com/wfchat/wfc_flutter_plugins)           | Flutter平台原生插件，支持IM和音视频通话。   |
+| [docs](https://github.com/wildfirechat/docs)                 | [docs](https://gitee.com/wfchat/docs)                 | 野火IM相关文档，包含设计、概念、开发、使用说明，[在线查看](https://docs.wildfirechat.cn/)。 |
 
 ## 野火开发文档
 [在线文档](https://docs.wildfirechat.cn/)
 
+## 野火性能测试报告及测试方法
+[Github](https://github.com/wildfirechat/Performance_Test), [码云](https://gitee.com/wfchat/Performance_Test)。
+
 ## 野火Demo
 请使用微信扫码下载安装体验野火IM移动客户端
 
@@ -59,72 +70,162 @@ PC客户端点[这里](https://github.com/wildfirechat/vue-pc-chat/releases)下
 
 ![野火IM](http://static.wildfirechat.net/wx.jpg)
 
+## 快速使用
+可以按照[快速开始](https://docs.wildfirechat.cn/quick_start/)来部署应用服务和IM服务，然后打包移动端，就可以完成聊天和通话功能。
 
-## 应用截图
-![ios-demo](http://static.wildfirechat.cn/ios-demo.gif)
+之后再部署开放平台服务、频道（公众号）服务、机器人服务、推送服务、Turn服务（音视频服务）来实现完整的应用功能，详情可以详细阅读文档。
 
-<img src="http://static.wildfirechat.cn/ios-message-view.png" width = 50% height = 50% />
+## 联系我们
+商务合作请微信联系：
 
-<img src="http://static.wildfirechat.cn/ios-contact-view.png" width = 50% height = 50% />
+* 微信1：wildfirechat
+* 微信2：wfchat
 
-<img src="http://static.wildfirechat.cn/ios-discover-view.png" width = 50% height = 50% />
+## 问题交流
 
-<img src="http://static.wildfirechat.cn/ios-settings-view.png" width = 50% height = 50% />
+1. 如果大家发现bug，请在GitHub或码云提issue；如果有需求也请给我们提issue。
+2. 其他问题，请到[野火IM论坛](http://bbs.wildfirechat.cn/)进行交流学习
+3. 关注我们的公众号。我们有新版本发布或者有重大更新会通过公众号通知大家，另外我们也会不定期的发布一些关于野火IM的技术介绍。
 
-<img src="http://static.wildfirechat.cn/ios-messagelist-view.png" width = 50% height = 50% />
+<img src="http://static.wildfirechat.cn/wx_wfc_qrcode.jpg" width = 50% height = 50% />
 
-<img src="http://static.wildfirechat.cn/ios-chat-setting-view.png" width = 50% height = 50% />
-
-<img src="http://static.wildfirechat.cn/ios-takephoto-view.png" width = 50% height = 50% />
-
-<img src="http://static.wildfirechat.cn/ios-record-voice-view.png" width = 50% height = 50% />
-
-<img src="http://static.wildfirechat.cn/ios-location-view.png" width = 50% height = 50% />
-
-<img src="http://static.wildfirechat.cn/ios-voip-view.png" width = 50% height = 50% />
+我们有核心研发工程师轮流值班处理issue和论坛，会及时处理的，疑难Bug的修改和新需求的开发我们也会尽快解决。
 
 
 ## 编译
-在安装JDK1.8以上及maven的前提下，在命令行中执行```mvn clean package```，生成的目标文件在```./distribution/target/distribution-xxxx-bundle-tar.tar.gz```
-> 由于使用了一个git的maven插件，如果本地没有git信息就会编译出错，请使用```git clone```的方法下载代码，或者下载压缩包解压后在根目录创建```.git```的空目录。建议用```git clone```的方式下载代码。
+```
+mvn clean package
+```
+生成的目标文件在```./distribution/target/distribution-xxxx-bundle-tar.tar.gz```
 
 ## 配置
-解压```distribution-xxxx-bundle-tar.tar.gz```，修改解压出来的```config```目录下的```wildfirechat.conf```。修改如下部分：
-1. 修改```server.ip```为您的服务器的公网IP，或者域名。如果用域名需要做好域名解析。
-2. 可以阅读配置文件注释和文档，对其它配置项进行调整。
+解压```distribution-xxxx-bundle-tar.tar.gz```，修改解压出来的```config```目录下的```wildfirechat.conf```，可以阅读配置文件注释和文档，对其它配置项进行调整。
 
 ## 运行
-在程序目录运行如下命令：
+在解压```distribution-xxxx-bundle-tar.tar.gz```后的目录下运行如下命令：
 ```
 ./bin/wildfirechat.sh
 ```
 
 ## 验证
 1. 在浏览器中输入地址 ```http://${ip}/api/version```可以看到返回一个json文件。
-2. 部署[应用服务](应用服务)，配置和编译[Android客户端](https://github.com/wildfirechat/android-chat)和[iOS客户端](https://github.com/wildfirechat/ios-chat)进行验证。详情可参考[快速开始](https://docs.wildfirechat.cn/quick_start/)。
+2. 部署[应用服务](https://github.com/wildfirechat/app-server)，配置和编译[Android客户端](https://github.com/wildfirechat/android-chat)和[iOS客户端](https://github.com/wildfirechat/ios-chat)进行验证。详情可参考[快速开始](https://docs.wildfirechat.cn/quick_start/)。
+
+## 打包RPM格式
+打包会生成Java包和deb安装包，如果需要rpm安装包，请在```distribution/pom.xml```中取消注释生成rpm包的plugin。另外还需要本地安装有rpm，在linux或者mac系统中很容易安装，在windows系统需要安装cygwin并安装rpm，具体信息请百度查询。
+
+修改之后运行编译命令```mvn clean package```，rpm包生成在```distribution/target```目录下。
 
 ## 升级说明
 1. 从0.42 版本增加了群成员数限制，默认为2000。如果您从之前的版本升级到这个版本或以后，需要注意到群成员数的限制。升级之后超出限制的群不受影响，但不能继续加人，如果您想修改默认值，可以在升级版本之后，修改t_setting表，把默认的大小改为您期望的人数。另外修改t_group表，把已经存在的群组max_member_count改成您期望的，然后重启。
-2. 0.46和0.47版本升级到0.48及以后版本时，可能会提示flyway migrate 38错误，请执行 [修复脚本](https://github.com/wildfirechat/server/blob/wildfirechat/flyway_repaire_migrate_38.sql) 进行修复。0.46和0.47版本之外的版本不会出现此问题。
-3. 从0.54之前版本升级到0.54及以后版本时，会提示flyway migrate错误。因为0.54版本删除了sql脚本中默认敏感词的内容，flyway checksum失败。请执行```update flyway_schema_history set checksum = 0 where script = 'V17__add_default_sensitive_word.sql';```来修复。
-4. 从0.59之前的版本升级到之后的版本执行数据库升级时间比较长，请耐心等待提示运行成功，避免中途中断。
+2. 0.50版本添加了是否允许客户端发送群操作通知的配置。如果您在客户端自定义群通知，需要在服务器端配置允许，没有使用自定义群操作通知的不受影响。***
+3. 0.46和0.47版本升级到0.48及以后版本时，可能会提示flyway migrate 38错误，请执行 [修复脚本](https://github.com/wildfirechat/server/blob/wildfirechat/flyway_repaire_migrate_38.sql) 进行修复。0.46和0.47版本之外的版本不会出现此问题。
+4. 从0.54之前版本升级到0.54及以后版本时，会提示flyway migrate错误。因为0.54版本删除了sql脚本中默认敏感词的内容，flyway checksum失败。请执行```update flyway_schema_history set checksum = 0 where script = 'V17__add_default_sensitive_word.sql';```来修复。
+5. 从0.59之前的版本升级到之后的版本执行数据库升级时间比较长，请耐心等待提示运行成功，避免中途中断。
+6. 0.62/0.63 版本有严重的问题，请使用0.64及以后版本，或者0.61版。
+7. 从0.68 版本起添加了pc在线是否默认手机接收推送的开关，默认为开，与以前版本作用相反，请注意兼容（可以关掉与之前保持一致或者升级客户端）
+8. 从0.78 版本起把MySQL数据库中关键字都改为大小写敏感，另外生成id的方法也做了改变，只生成小写的id，避免出现id重复的问题，建议所有客户都升级
+9. 从0.79 版本起把log4j升级到log4j2，因为log4j已经不再维护而且还有已知的漏洞，建议所有客户都升级，升级时注意更新log4j2的配置文件
+10. 0.97版本更改了启动脚本```wildfirechat.sh```，如果是升级服务，请注意更新启动脚本。
 
+## 应用截图
+登录界面
 
-## 联系我们
+<img src="./screenshots/1_login.png" width = 50% height = 50% />
 
-商务合作请使用如下邮箱和微信联系：
+会话列表
 
-邮箱: support@wildfirechat.cn  微信1：wfchat 微信2：wildfirechat
+<img src="./screenshots/2_conversation_list.png" width = 50% height = 50% />
 
-## 问题交流
+好友列表
 
-1. 如果大家发现bug，请在GitHub提issue；如果有需求也请给我们提issue。
-2. 其他问题，请到[野火IM论坛](http://bbs.wildfirechat.cn/)进行交流学习
-3. 关注我们的公众号。我们有新版本发布或者有重大更新会通过公众号通知大家，另外我们也会不定期的发布一些关于野火IM的技术介绍。
+<img src="./screenshots/3_contacts_list.png" width = 50% height = 50% />
 
-<img src="http://static.wildfirechat.cn/wx_wfc_qrcode.jpg" width = 50% height = 50% />
+工作台
 
-> 我们有研发工程师轮流值班处理issue和论坛，一般简单问题几个小时就会回复一遍，疑难Bug的修改和新需求的开发我们也会尽快解决。
+<img src="./screenshots/4_workplatform.png" width = 50% height = 50% />
+
+发现
+
+<img src="./screenshots/5_discover.png" width = 50% height = 50% />
+
+我的页面
+
+<img src="./screenshots/6_settings.png" width = 50% height = 50% />
+
+消息聊天
+
+<img src="./screenshots/7_conversation.png" width = 50% height = 50% />
+
+动态表情
+
+<img src="./screenshots/8_conversation_sticker.png" width = 50% height = 50% />
+
+语音消息
+
+<img src="./screenshots/9_conversation_voice_msg.png" width = 50% height = 50% />
+
+位置消息
+
+<img src="./screenshots/10_conversation_location.png" width = 50% height = 50% />
+
+拍摄图片或者小视频
+
+<img src="./screenshots/11_conversation_capture.jpeg" width = 50% height = 50% />
+
+会话设置
+
+<img src="./screenshots/13_conversation_settings.png" width = 50% height = 50% />
+
+朋友圈
+
+<img src="./screenshots/14_moments.png" width = 50% height = 50% />
+
+新好友
+
+<img src="./screenshots/15_new_friends.png" width = 50% height = 50% />
+
+多人视频通话
+
+<img src="./screenshots/16_videocall.png" width = 50% height = 50% />
+
+会议主页
+
+<img src="./screenshots/17_conference_home.png" width = 50% height = 50% />
+
+会议详情
+
+<img src="./screenshots/18_conference_info.png" width = 50% height = 50% />
+
+会议界面
+
+<img src="./screenshots/19_conference.png" width = 50% height = 50% />
+
+PC客户端界面
+
+<img src="./screenshots/20_pc.png"/>
+
+移动客户端、PC客户端和Web客户端三端同框
+
+<img src="./screenshots/21_three_platform.png" />
+
+小程序客户端
+
+<img src="./screenshots/22_micro_app.jpeg" width = 50% height = 50% />
+
+公众号管理后台
+
+<img src="./screenshots/23_channel.png"/>
+
+开放平台管理后台
+
+<img src="./screenshots/24_open_platform.png" />
+
+IM服务管理后台
+
+<img src="./screenshots/25_im_admin.png" />
+
+> 更多详细信息请下载demo进行体验。
 
 ## 特别感谢
 1. [moquette](https://github.com/moquette-io/moquette) 本项目是基于此项目二次开发而来，处理MQTT相关业务。

broker/config/i18n/messages.properties

@@ -1,3 +0,0 @@
-#native2ascii -encoding UTF-8 messages.properties temp_messages.properties
-Above_Greeting_Message=\u4ee5\u4e0a\u662f\u6253\u62db\u547c\u4fe1\u606f
-Friend_Can_Start_Chat=\u4f60\u4eec\u5df2\u7ecf\u6210\u4e3a\u597d\u53cb\u4e86\uff0c\u73b0\u5728\u53ef\u4ee5\u5f00\u59cb\u804a\u5929\u4e86

broker/config/i18n/messages_en.properties

@@ -1,2 +0,0 @@
-Above_Greeting_Message=This's the greeting
-Friend_Can_Start_Chat=You're already friend, start chat now.

broker/config/i18n/messages_en_US.properties

@@ -1,2 +0,0 @@
-Above_Greeting_Message=This's the greeting
-Friend_Can_Start_Chat=You're already friend, start chat now.

broker/config/i18n/messages_zh.properties

@@ -1,2 +0,0 @@
-Above_Greeting_Message=\u4ee5\u4e0a\u662f\u6253\u62db\u547c\u4fe1\u606f
-Friend_Can_Start_Chat=\u4f60\u4eec\u5df2\u7ecf\u6210\u4e3a\u597d\u53cb\u4e86\uff0c\u73b0\u5728\u53ef\u4ee5\u5f00\u59cb\u804a\u5929\u4e86

broker/config/i18n/messages_zh_CN.properties

@@ -1,2 +0,0 @@
-Above_Greeting_Message=\u4ee5\u4e0a\u662f\u6253\u62db\u547c\u4fe1\u606f
-Friend_Can_Start_Chat=\u4f60\u4eec\u5df2\u7ecf\u6210\u4e3a\u597d\u53cb\u4e86\uff0c\u73b0\u5728\u53ef\u4ee5\u5f00\u59cb\u804a\u5929\u4e86

broker/config/wildfirechat.conf

@@ -12,7 +12,7 @@
 #服务器的接入IP。给客户端提供是${server.ip}和${http_port}。
 #客户端会从${http_port}端口获取到长链接端口。这个地址一定要改成客户端可以访问到的IP地址
 #（如果您部署云服务器上或者具有独立公网出口的服务器上，请改为对应的公网IP；如果您部署在在内网环境下，在内网使用，这个地方改成内网地址）
-server.ip 192.168.3.124
+server.ip 192.168.1.81
 
 ##原生客户端长链接端口
 port 1883
@@ -26,8 +26,8 @@ http.admin.port 18080
 ##节点ID，当集群部署时，一定不能有重复。
 node_id 1
 
-##绑定IP，保持不变即可
-host 0.0.0.0
+##绑定IP，不要打开，除非有特殊需求才可以打开
+#host 0.0.0.0
 
 ##本地绑定端口
 local_port 80
@@ -38,9 +38,13 @@ embed.db 1
 
 ## 是否自动清理历史消息记录，当为true时，IM服务会定时扫描t_messages_x和t_user_messages_x表，清除掉3年前的历史消息。
 ## 如果不开启自动清理，需要手动清理t_messages_x和t_user_messages_x表内的历史数据，保存数据不能超过3年。
+## 如果需要消息保存更长时间，可以把临近销毁的消息转储到别的数据库，客户端在拉取不到服务器消息时，再去转储的数据库拉取。
 ## 使用mongodb时次开关无效（因为mongodb设置了消息过期时间，会自动清理，仅专业版支持mongodb）。
 db.auto_clean_history_messages true
 
+## h2db数据库的路径，默认为程序目录下的，可以指定其他目录。如果使用MySQL，可以忽略此配置
+h2db.path ./h2db/wfchat
+
 ##服务器管理接口密钥。如果修改此端口需要同步修改使用管理API的地方，比如应用服务。
 http.admin.secret_key 123456
 
@@ -55,11 +59,23 @@ token.key testim
 ##token的过期时间，单位为毫秒，默认为无限期。如果需要设置无限期，客户端上一定需要加上token过期的处理。token过期的处理请参考文档的常见问题
 ##token.expire_time 2592000000
 
+##客户端（目前只有android）的应用签名（一个应用最取第一个签名），如果有多个签名(多个客户端)，用英文逗号分开。获取签名方法请参考文档：https://docs.wildfirechat.cn/blogs/签名验证.html
+#connect.client_signature_list eNkezQ1g9OsnhfLSFUY1vzKDzhs=,xykezQ1g9OsnhfLSFUY1vzKDzhs=
+
+##是否拒绝空签名的android客户端， 默认为true。当有旧的客户端需要兼容时改成false，当所有都是新客户端时，改成true。
+#connect.reject_empty_signature true
+
+##是否使用AES256加密。默认客户端和服务器都是AES128加密，如果需要开启AES256，需要客户端和IM服务同时开通。
+#encrypt.use_aes256 true
+
 ##首次登录，是否接收之前的历史消息。
-##0 不接收历史消息，只接收5分钟以内的消息，由于服务器没有保存已经收取记录，所以如果有超过5分钟之前未收取的消息也不会收取下拉；
+##0 不接收历史消息，只接收${message.compensate_time_limit}毫秒以内的消息，由于服务器没有保存已经收取记录，所以如果有超过${message.compensate_time_limit}毫秒之前未收取的消息也不会收取下拉；
 ##1接收，会接收 message.max_queue 配置的条数的历史消息
 message.roaming 0
 
+## 消息补偿时限，当${message.roaming}为0时，会同步时限以内且小于${message.max_queue}的消息，默认时限为5分钟。当${message.roaming}为1时，会接收 message.max_queue 配置的条数的历史消息。
+message.compensate_time_limit 300000
+
 ##是否开启拉取远程历史消息。如果为1，客户端在会话内如果本地消息读取完了，可以下拉继续加载在服务器上的该会话的消息；如果为0则不能。
 message.remote_history_message 0
 
@@ -72,6 +88,9 @@ message.chatroom_remote_history_message 1
 ##是否禁止陌生人聊天
 message.disable_stranger_chat false
 
+##当禁止陌生人聊天时，允许聊天的用户id，比如管理员或者文件传输助手等。用户id以英文逗号分割。
+message.allow_stranger_chat_list admin,FireRobot,wfc_file_transfer
+
 ##黑名单策略，0 发送失败，返回被拉黑的错误码；1 发送成功但消息被服务器直接丢弃
 message.blacklist.strategy 0
 
@@ -104,10 +123,22 @@ message.group_mute_exception_types 401,402,403,404,405,407,408,410,411
 ## 用户被全局禁言时允许发送的消息类型
 message.global_mute_exception_types 401,402,403,404,405,407,408,410,411
 
+## 离线用户推送过期天数，0是永不过期，建议配置为7天。
+message.push_expired_days 7
+
 ## 消息转发功能开启时，开关是否不转发server api接口消息。
 ## 当为true时，不转发server api发送的消息；当为false时，转发server api发送的消息。
 message.no_forward_admin_message false
 
+## 消息转发功能开启时，转发信息是否带上用户clientId和平台类型。
+message.forward_with_client_info false
+
+## 机器人回调信息是否带上用户clientId和平台类型。
+robot.callback_with_client_info false
+
+## 频道回调信息是否带上用户clientId和平台类型。
+channel.callback_with_client_info false
+
 ##禁止搜索用户
 friend.disable_search false
 
@@ -126,11 +157,8 @@ friend.reject_request_duration 2592000000
 ##好友请求过期时间，单位是毫秒，默认是7天，0为无限长期限。
 friend.request_expiration_duration 604800000
 
-##添加好友后会发送欢迎消息，比如"你们已经是好友了，可以聊天了"。这个开关控制是用旧的方式还是用新的方式。
-##旧的方式会发送notification消息（contenttype是80），内容会根据好友接收者的语言选择对应的语言，如果双方语言不一致会导致另外一方看到的不是本地语言。
-##新的方式会发送添加好友消息（contenttype是93）和欢迎消息（contenttype是92），需要客户端实现这两种消息。
-##如果客户端已经实现这两种消息可以打开这个开关（客户端2020.6.30添加了此消息类型）。
-friend.new_welcome_message true
+##请求添加机器人为好友时，机器人是否自动接受，默认为true
+friend.robot_auto_accept true
 
 ## 聊天室观众空闲退出时间，单位为毫秒，默认为15分钟，0为永远不退出
 chatroom.participant_idle_time 900000
@@ -141,6 +169,12 @@ chatroom.rejoin_when_active true
 ## 当一个聊天室不存在时，如果有用户加入就自动创建
 chatroom.create_when_not_exist true
 
+## 是否允许群主不受限制地撤回自己发送的消息。false可以${message.recall_time_limit}撤回自己的消息；true可以撤回任意时间自己的消息。默认为true。
+group.allow_owner_recall_self_msg true
+
+## 是否允许群管理员不受限制地撤回自己发送的消息。false可以${message.recall_time_limit}撤回自己的消息；true可以撤回任意时间自己的消息。默认为false。
+group.allow_manager_recall_self_msg false
+
 ## 是否允许客户端发送操作群的自定义群通知消息
 group.allow_client_custom_operation_notification true
 
@@ -151,6 +185,10 @@ group.allow_robot_custom_operation_notification true
 ## 0 不发送，1 退群时发送，2 踢人时发送，3 退群和踢人时都发送。
 group.visible_quit_or_kickoff_notification 0
 
+## 禁止客户端群操作。第1位是禁止创建群组，第2位是禁止销毁群组，3位禁止加入群，4位禁止退出群，5位禁止邀请群成员，6位禁止移出群成员，7位禁止转移群，8位禁止设置群管理员，9位禁止白名单处理，10位禁止群禁言，11位禁止修改群组信息，12位禁止群成员禁言。
+## Server API不受此限制。
+#group.forbidden_client_operation 0xFFF
+
 ## 隐藏用户信息属性类型，2是性别，3是电话号码，4是邮箱，5是地址，6是公司信息，7是社交账号，8是Extra信息。当需要隐藏多个时以英文逗号分开;
 ## 本配置是为了提高安全性，避免把对应信息同步到客户端，这样就能防止程序被破解从而拿到用户关键信息。仅对客户端有效，server api不受影响。
 ## 注意，修改本配置不会对已经同步的信息产生影响。
@@ -177,11 +215,6 @@ http.channel.rate_limit 1000
 ##客户端请求频率限制，默认是5秒100次。通常情况下不要修改这个值。
 client.request_rate_limit 100
 
-#*********************************************************************
-# hazelcase configuration
-#*********************************************************************
-hazelcast.configuration config/hazelcast.xml
-
 #*********************************************************************
 # Media server configuration
 #*********************************************************************
@@ -288,13 +321,23 @@ push.ios.server.address http://localhost:8085/ios/push
 ##消息转发地址
 ##注意回调是单线程回调，接收服务必须在同一内网，且异步处理快速返回，否则会有延迟问题！！
 #message.forward.url http://localhost:8888/im_event/message
-##需要转发的消息类型，当有多个时以英文逗号分割。如果转发所有消息，请注释掉配置或者设置为空
-#message.forward.types 1,2,3
+##需要转发的消息类型，当有多个时以英文逗号分割，可以用-来指定区间，注意区间不能太大，因为要把区间内的所有数字都放入到Set中。如果转发所有消息，请注释掉配置或者设置为空
+#message.forward.types 1,2,3,5-10
+##需要转发的消息类型，当有多个时以英文逗号分割，可以用-来指定区间，注意区间不能太大，因为要把区间内的所有数字都放入到Set中。
+#message.forward.exclude_types 1,2,3,5-10
 
-##提醒消息转发地址，@全体用户和@某个用户消息会回调此地址。此转发不受${message.forward.types}限制。
+##敏感消息转发地址
+##注意回调是单线程回调，接收服务必须在同一内网，且异步处理快速返回，否则会有延迟问题！！
+#message.sensitive.forward.url http://localhost:8888/im_event/message
+
+##提醒消息转发地址，@全体用户和@某个用户消息会回调此地址。
+##此转发不受${message.forward.types}和${message.forward.exclude_types}限制。
 ##注意回调是单线程回调，接收服务必须在同一内网，且异步处理快速返回，否则会有延迟问题！！
 #message.mentionmsg.forward.url http://localhost:8888/im_event/message
 
+##撤回消息转发地址，当用户撤回消息会回调此地址。
+##注意回调是单线程回调，接收服务必须在同一内网，且异步处理快速返回，否则会有延迟问题！！
+#message.recallmsg.forward.url http://localhost:8888/im_event/recall_message
 
 ##设备信息转发地址
 ##注意回调是单线程回调，接收服务必须在同一内网，且异步处理快速返回，否则会有延迟问题！！
@@ -323,10 +366,12 @@ push.ios.server.address http://localhost:8085/ios/push
 #*********************************************************************
 # Sensitive configuration
 #*********************************************************************
-## 内置文本敏感词过滤处理方法，0 发送失败；1 发送成功但消息被服务器直接丢弃；2 命中的敏感词被替换成***；3 记录并继续发送
+## 内置文本敏感词过滤处理方法，当命中敏感词后，消息会被记录到t_sensitive_messages表中，然后根据type类型做不同的处理，处理方式如下：
+## 0 发送失败；1 发送成功但消息被服务器直接丢弃；2 命中的敏感词被替换成***然后正常发送；3 正常发送。
+## 如果开启 message.sensitive.forward.url 配置，命中敏感词的消息还会被回调到这个地址
 sensitive.filter.type 0
 
-## 如果内置敏感词库无法满足您的需求，可以自己开发部署敏感词审核服务。配置如下地址和消息类型，IM服务会把指定消息类型的消息回调到指定地址
+## 如果内置敏感词库无法满足您的需求，可以开发部署独立敏感词审核服务。配置如下地址和消息类型，IM服务会把指定消息类型的消息回调到指定地址，内置的敏感词审核和敏感词转发功能就不再起作用了。
 ## 审核服务返回状态码200表示继续发送，如果需要替换内容，返回状态码200并且内容为替换后的内容。返回403表示不允许发送。其它错误继续发送。
 ## 独立审核服务地址，IM服务会post SendMessageData。如果需要修改消息内容，返回 MessagePayload。请参考应用服务中 "/message/censor" Mapping.
 #sensitive.remote_server_url http://192.168.3.202:8888/message/censor

broker/migrate/h2/V45__add_channel_menu_column.sql

@@ -0,0 +1 @@
+alter table `t_channel` add column `_menu` BLOB DEFAULT NULL;

broker/migrate/h2/V46__add_group_member_mid_index.sql

@@ -0,0 +1 @@
+alter table `t_group_member` add INDEX `group_member_mid_index` (`_mid`);

broker/migrate/h2/V47__create_super_group.sql

@@ -0,0 +1,19 @@
+alter table `t_group` add column `_super_group` tinyint NOT NULL DEFAULT 0;
+
+DROP TABLE IF EXISTS `t_group_messages`;
+CREATE TABLE `t_group_messages` (
+    `_mid` bigint(20) NOT NULL PRIMARY KEY,
+    `_sender` varchar(64) NOT NULL,
+    `_gid` varchar(64) NOT NULL,
+    `_line` int(11) NOT NULL,
+    `_client_id` varchar(64) NULL,
+    `_seq` bigint(20) NOT NULL,
+    `_persist_flag` int(11) NOT NULL,
+    `_mentioned_type` int(11) NOT NULL,
+    `_mentioned_targets` BLOB NULL,
+    `_to` BLOB NULL,
+    `_cont_type` int(11) NOT NULL DEFAULT 0,
+    `_duration` int(11) NOT NULL DEFAULT 0,
+    `_dt` bigint(20) NOT NULL DEFAULT 0,
+    INDEX `group_message_seq_uid_index` (`_gid` DESC, `_line`, `_seq` DESC)
+);

broker/migrate/mysql/V52__add_channel_menu_column.sql

@@ -0,0 +1 @@
+alter table `t_channel` add column `_menu` BLOB DEFAULT NULL;

broker/migrate/mysql/V53__add_group_member_mid_index.sql

@@ -0,0 +1 @@
+alter table `t_group_member` add INDEX `group_member_mid_index` (`_mid`);

broker/pom.xml

@@ -1,11 +1,11 @@
-<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
-         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
     <modelVersion>4.0.0</modelVersion>
     <parent>
         <relativePath>../</relativePath>
         <artifactId>wildfirechat-parent</artifactId>
         <groupId>cn.wildfirechat</groupId>
-        <version>0.89</version>
+        <version>1.1.9</version>
     </parent>
 
     <artifactId>moquette-broker</artifactId>
@@ -56,7 +56,7 @@
         <dependency>
             <groupId>mysql</groupId>
             <artifactId>mysql-connector-java</artifactId>
-            <version>8.0.19</version>
+            <version>8.0.28</version>
         </dependency>
 
         <dependency>
@@ -120,7 +120,7 @@
         <dependency>
             <groupId>commons-codec</groupId>
             <artifactId>commons-codec</artifactId>
-            <version>1.10</version>
+            <version>1.13</version>
         </dependency>
 
         <dependency>
@@ -234,6 +234,11 @@
             <version>3.4.1</version>
         </dependency>
 
+        <dependency>
+            <groupId>org.apache.tika</groupId>
+            <artifactId>tika-core</artifactId>
+            <version>2.4.1</version>
+        </dependency>
     </dependencies>
 
 

broker/src/main/java/cn/wildfirechat/push/PushMessage.java

@@ -12,9 +12,11 @@ package cn.wildfirechat.push;
 public class PushMessage {
     public String sender;
     public String senderName;
+    public String senderPortrait;
     public int convType;
     public String target;
     public String targetName;
+    public String targetPortrait;
     public String userId;
     public int line;
     public int cntType;
@@ -34,12 +36,14 @@ public class PushMessage {
     public String language;
     public long messageId;
 
-    public PushMessage(String sender, int conversationType, String target, int line, int messageContentType, long serverTime, String senderName, String targetName, int unReceivedMsg, int mentionedType, boolean isHiddenDetail, String language) {
+    public PushMessage(String sender, int conversationType, String target, int line, int messageContentType, long serverTime, String senderName, String senderPortrait, String targetName, String targetPortrait, int unReceivedMsg, int mentionedType, boolean isHiddenDetail, String language) {
         this.sender = sender;
         this.convType = conversationType;
         this.target = target;
         this.senderName = senderName;
+        this.senderPortrait = senderPortrait;
         this.targetName = targetName;
+        this.targetPortrait = targetPortrait;
         this.line = line;
         this.cntType = messageContentType;
         this.serverTime = serverTime;

broker/src/main/java/cn/wildfirechat/push/PushServer.java

@@ -11,6 +11,7 @@ package cn.wildfirechat.push;
 import cn.wildfirechat.common.IMExceptionEvent;
 import cn.wildfirechat.proto.ProtoConstants;
 import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
 import io.moquette.persistence.MemorySessionStore;
 import io.moquette.server.config.IConfig;
 import io.moquette.spi.ISessionsStore;
@@ -29,7 +30,7 @@ import static win.liyufan.im.HttpUtils.HttpPostType.POST_TYPE_Push;
 
 public class PushServer {
     private static final Logger LOG = LoggerFactory.getLogger(PushServer.class);
-
+    protected static final Gson gson = new GsonBuilder().disableHtmlEscaping().create();
     public interface PushMessageType {
         int PUSH_MESSAGE_TYPE_NORMAL = 0;
         int PUSH_MESSAGE_TYPE_VOIP_INVITE = 1;
@@ -100,7 +101,7 @@ public class PushServer {
                 url = iOSPushServerUrl;
                 pushMessage.voipDeviceToken = session.getVoipDeviceToken();
             }
-            HttpUtils.httpJsonPost(url, new Gson().toJson(pushMessage, pushMessage.getClass()), POST_TYPE_Push);
+            HttpUtils.httpJsonPost(url, gson.toJson(pushMessage, pushMessage.getClass()), POST_TYPE_Push);
         } else {
             LOG.info("Not mobile platform {}", session.getPlatform());
         }

broker/src/main/java/com/xiaoleilu/loServer/action/Action.java

@@ -9,6 +9,7 @@
 package com.xiaoleilu.loServer.action;
 
 import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
 import com.xiaoleilu.loServer.RestResult;
 import com.xiaoleilu.loServer.annotation.RequireAuthentication;
 import com.xiaoleilu.loServer.handler.Request;
@@ -39,7 +40,7 @@ import static io.moquette.BrokerConstants.*;
 abstract public class Action {
     public static IMessagesStore messagesStore = null;
     public static ISessionsStore sessionsStore = null;
-
+    protected static final Gson gson = new GsonBuilder().disableHtmlEscaping().create();
     public ChannelHandlerContext ctx;
 
     protected static RateLimiter adminLimiter = null;
@@ -129,7 +130,7 @@ abstract public class Action {
             }
 
             RestResult result = RestResult.resultOf(errorCode, errorCode.getMsg(), RestResult.resultOf(errorCode));
-            response.setContent(new Gson().toJson(result));
+            response.setContent(gson.toJson(result));
             response.send();
         }
 
@@ -145,7 +146,7 @@ abstract public class Action {
             FullHttpRequest fullHttpRequest = (FullHttpRequest) request;
             byte[] bytes = Utils.readBytesAndRewind(fullHttpRequest.content());
             String content = new String(bytes, StandardCharsets.UTF_8);//contribute by JiaRG from github
-            Gson gson = new Gson();
+            
             T t = gson.fromJson(content, cls);
             return t;
         }
@@ -154,6 +155,6 @@ abstract public class Action {
 
     protected void setResponseContent(RestResult result, Response response) {
         response.setStatus(HttpResponseStatus.OK);
-        response.setContent(new Gson().toJson(result));
+        response.setContent(gson.toJson(result));
     }
 }

broker/src/main/java/com/xiaoleilu/loServer/action/FileAction.java

@@ -90,6 +90,8 @@ public class FileAction extends Action {
         }
 
         response.setContent(file);
+        response.setHeader("Content-Disposition", "attachment; filename=\"" + file.getName() + "\"");
+
         return true;
     }
 

broker/src/main/java/com/xiaoleilu/loServer/action/IMAction.java

@@ -80,6 +80,11 @@ public class IMAction extends Action {
                 if (userId == null) {
                     sendResponse(response, ErrorCode.ERROR_CODE_TOKEN_ERROR, null);
                 } else {
+                    if(messagesStore.getUserStatus(userId) == ProtoConstants.UserStatus.Forbidden) {
+                        sendResponse(response, ErrorCode.ERROR_CODE_USER_BLOCKED, null);
+                        return true;
+                    }
+
                     ServerAPIHelper.sendRequest(userId, wrapper.getClientId(), wrapper.getRequest(), wrapper.getData().toByteArray(), new ServerAPIHelper.Callback() {
                         @Override
                         public void onSuccess(byte[] result) {

broker/src/main/java/com/xiaoleilu/loServer/action/RouteAction.java

@@ -39,17 +39,10 @@ public class RouteAction extends Action {
     @Override
     public boolean action(Request request, Response response) {
         if (request.getNettyRequest() instanceof FullHttpRequest) {
-
             response.setContentType("application/octet-stream");
             response.setHeader("Access-Control-Allow-Origin", "*");
 
             FullHttpRequest fullHttpRequest = (FullHttpRequest) request.getNettyRequest();
-            boolean b = false;
-            if ("web".equalsIgnoreCase(fullHttpRequest.headers().get("p"))) {
-                b = true;
-            }
-
-            final boolean base64Response = b;
 
             byte[] bytes = Utils.readBytesAndRewind(fullHttpRequest.content());
 
@@ -57,15 +50,20 @@ public class RouteAction extends Action {
             try {
                 bytes = Base64.getDecoder().decode(str);
             } catch (IllegalArgumentException e) {
-                sendResponse(response, ErrorCode.ERROR_CODE_INVALID_DATA, null, base64Response);
+                sendResponse(response, ErrorCode.ERROR_CODE_INVALID_DATA, null);
                 return true;
             }
 
             String cid = fullHttpRequest.headers().get("cid");
             byte[] cbytes = Base64.getDecoder().decode(cid);
-            cbytes = AES.AESDecrypt(cbytes, "", true);
+            boolean[] invalidTime = new boolean[1];
+            cbytes = AES.AESDecrypt(cbytes, "", true, invalidTime);
             if (cbytes == null) {
-                sendResponse(response, ErrorCode.ERROR_CODE_INVALID_DATA, null, base64Response);
+                if(invalidTime[0]) {
+                    sendResponse(response, ErrorCode.ERROR_CODE_TIME_INCONSISTENT, null);
+                } else {
+                    sendResponse(response, ErrorCode.ERROR_CODE_INVALID_DATA, null);
+                }
                 return true;
             }
             cid = new String(cbytes);
@@ -74,7 +72,7 @@ public class RouteAction extends Action {
             byte[] ubytes = Base64.getDecoder().decode(uid);
             ubytes = AES.AESDecrypt(ubytes, "", true);
             if (ubytes == null) {
-                sendResponse(response, ErrorCode.ERROR_CODE_INVALID_DATA, null, base64Response);
+                sendResponse(response, ErrorCode.ERROR_CODE_INVALID_DATA, null);
                 return true;
             }
             uid = new String(ubytes);
@@ -84,7 +82,7 @@ public class RouteAction extends Action {
             if (session == null) {
                 ErrorCode errorCode = sessionsStore.loadActiveSession(uid, cid);
                 if (errorCode != ErrorCode.ERROR_CODE_SUCCESS) {
-                    sendResponse(response, errorCode, null, base64Response);
+                    sendResponse(response, errorCode, null);
                     return true;
                 }
                 session = sessionsStore.sessionForClientAndUser(uid, cid);
@@ -94,16 +92,20 @@ public class RouteAction extends Action {
             if (session != null) {
                 bytes = AES.AESDecrypt(bytes, session.getSecret(), true);
             } else {
-                sendResponse(response, ErrorCode.ERROR_CODE_SECRECT_KEY_MISMATCH, null, base64Response);
+                sendResponse(response, ErrorCode.ERROR_CODE_SECRECT_KEY_MISMATCH, null);
                 return true;
             }
 
 
             if (bytes == null) {
-                sendResponse(response, ErrorCode.ERROR_CODE_SECRECT_KEY_MISMATCH, null, base64Response);
+                sendResponse(response, ErrorCode.ERROR_CODE_SECRECT_KEY_MISMATCH, null);
                 return true;
             }
 
+            if(messagesStore.getUserStatus(uid) == ProtoConstants.UserStatus.Forbidden) {
+                sendResponse(response, ErrorCode.ERROR_CODE_USER_BLOCKED, null);
+                return true;
+            }
 
             try {
                 WFCMessage.IMHttpWrapper wrapper = WFCMessage.IMHttpWrapper.parseFrom(bytes);
@@ -111,22 +113,22 @@ public class RouteAction extends Action {
                 String userId = Tokenor.getUserId(token.getBytes());
                 LOG.info("RouteAction token={}, userId={}", token, userId);
                 if (userId == null) {
-                    sendResponse(response, ErrorCode.ERROR_CODE_TOKEN_ERROR, null, base64Response);
+                    sendResponse(response, ErrorCode.ERROR_CODE_TOKEN_ERROR, null);
                 } else {
                     ServerAPIHelper.sendRequest(userId, wrapper.getClientId(), wrapper.getRequest(), wrapper.getData().toByteArray(), new ServerAPIHelper.Callback() {
                         @Override
                         public void onSuccess(byte[] result) {
-                            sendResponse(response, null, result, base64Response);
+                            sendResponse(response, null, result);
                         }
 
                         @Override
                         public void onError(ErrorCode errorCode) {
-                            sendResponse(response, errorCode, null, base64Response);
+                            sendResponse(response, errorCode, null);
                         }
 
                         @Override
                         public void onTimeout() {
-                            sendResponse(response, ErrorCode.ERROR_CODE_TIMEOUT, null, base64Response);
+                            sendResponse(response, ErrorCode.ERROR_CODE_TIMEOUT, null);
                         }
 
                         @Override
@@ -139,25 +141,21 @@ public class RouteAction extends Action {
                     return false;
                 }
             } catch (InvalidProtocolBufferException e) {
-                sendResponse(response, ErrorCode.ERROR_CODE_INVALID_DATA, null, false);
+                sendResponse(response, ErrorCode.ERROR_CODE_INVALID_DATA, null);
             }
         }
         return true;
     }
 
-    private void sendResponse(Response response, ErrorCode errorCode, byte[] contents, boolean base64Response) {
+    private void sendResponse(Response response, ErrorCode errorCode, byte[] contents) {
         response.setStatus(HttpResponseStatus.OK);
         if (contents == null) {
             ByteBuf ackPayload = Unpooled.buffer();
             ackPayload.ensureWritable(1).writeByte(errorCode.getCode());
             contents = ackPayload.array();
         }
-        if (base64Response) {
-            byte[] base64Data = Base64.getEncoder().encode(contents);
-            response.setContent(base64Data);
-        } else {
-            response.setContent(contents);
-        }
+
+        response.setContent(contents);
         response.send();
     }
 }

broker/src/main/java/com/xiaoleilu/loServer/action/UploadFileAction.java

@@ -187,6 +187,10 @@ public class UploadFileAction extends Action {
                     return false;
                 }
 
+                if(remoteFileName != null && remoteFileName.length() > 2 && remoteFileName.startsWith("'") && remoteFileName.endsWith("'")) {
+                    remoteFileName = remoteFileName.substring(1, remoteFileName.length()-1);
+                }
+
                 if (StringUtil.isNullOrEmpty(remoteFileName)) {
                     logger.warn("remoteFileName is empty!");
 
@@ -235,12 +239,17 @@ public class UploadFileAction extends Action {
                 int offset = 0; // 断点续传开始位置
 
                 Date nowTime=new Date();
-                SimpleDateFormat time=new SimpleDateFormat("yyyy/MM/dd/HH");
+                SimpleDateFormat time=new SimpleDateFormat("yyyy/MM/dd/HH/mm/ss");
                 String datePath = time.format(nowTime);
 
                 datePath = "fs/" + bucket[0] + "/" + datePath; //add bucket
 
-                String dir = "./" + MediaServerConfig.FILE_STROAGE_ROOT + "/" + datePath;
+                String dir;
+                if(MediaServerConfig.FILE_STROAGE_ROOT.startsWith("/") || MediaServerConfig.FILE_STROAGE_ROOT.startsWith("~/") || MediaServerConfig.FILE_STROAGE_ROOT.startsWith("./")) {
+                    dir = MediaServerConfig.FILE_STROAGE_ROOT + "/" + datePath;
+                } else {
+                    dir = "./" + MediaServerConfig.FILE_STROAGE_ROOT + "/" + datePath;
+                }
 
                 File dirFile = new File(dir);
                 boolean bFile  = dirFile.exists();
@@ -248,6 +257,7 @@ public class UploadFileAction extends Action {
                 if(!bFile) {
                     bFile = dirFile.mkdirs();
                     if (!bFile) {
+                        logger.error("create dirs error! {}", dir);
                         response.setStatus(HttpResponseStatus.INTERNAL_SERVER_ERROR);
                         response.setContent("服务器错误：无法创建文件");
                         return false;

broker/src/main/java/com/xiaoleilu/loServer/action/VersionAction.java

@@ -34,7 +34,7 @@ public class VersionAction extends Action {
 
             try {
 
-                response.setContent(Utility.formatJson(new Gson().toJson(GitRepositoryState.getGitRepositoryState())));
+                response.setContent(Utility.formatJson(gson.toJson(GitRepositoryState.getGitRepositoryState())));
             } catch (IOException e) {
                 e.printStackTrace();
                 response.setContent("{\"version\":\"unknown\"}");

broker/src/main/java/com/xiaoleilu/loServer/action/admin/AdminAction.java

@@ -99,7 +99,7 @@ abstract public class AdminAction extends Action {
             }
 
             RestResult result = RestResult.resultOf(errorCode, errorCode.getMsg(), data);
-            response.setContent(new Gson().toJson(result));
+            response.setContent(gson.toJson(result));
             response.send();
         }
     }
@@ -112,7 +112,7 @@ abstract public class AdminAction extends Action {
     }
 
     protected void sendApiMessage(Response response, String fromUser, String clientId, String topic, byte[] message, ApiCallback callback, boolean noAdmin) {
-        ServerAPIHelper.sendRequest(fromUser, clientId, topic, message, new ServerAPIHelper.Callback() {
+        ServerAPIHelper.sendRequest(fromUser, clientId, topic, message, callback == null ? null : new ServerAPIHelper.Callback() {
             @Override
             public void onSuccess(byte[] result) {
                 if(callback != null) {

broker/src/main/java/com/xiaoleilu/loServer/action/admin/ApplicationGetUserInfoAction.java

@@ -50,7 +50,7 @@ public class ApplicationGetUserInfoAction extends AdminAction {
             }
 
             response.setStatus(HttpResponseStatus.OK);
-            response.setContent(new Gson().toJson(result));
+            response.setContent(gson.toJson(result));
         }
         return true;
     }

broker/src/main/java/com/xiaoleilu/loServer/action/admin/BlockUserAction.java

@@ -42,7 +42,7 @@ public class BlockUserAction extends AdminAction {
                 response.setStatus(HttpResponseStatus.OK);
                 RestResult result;
                 result = RestResult.resultOf(errorCode);
-                response.setContent(new Gson().toJson(result));
+                response.setContent(gson.toJson(result));
 
                 if (inputUserBlock.getStatus() == 2) {
                     sendApiMessage(null, null, ServerAPIHelper.KICKOFF_USER_REQUEST, inputUserBlock.getUserId().getBytes(), null);

broker/src/main/java/com/xiaoleilu/loServer/action/admin/ChannelSubscriberAction.java

@@ -0,0 +1,62 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package com.xiaoleilu.loServer.action.admin;
+
+import cn.wildfirechat.common.APIPath;
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.pojos.InputChannelSubscribe;
+import cn.wildfirechat.pojos.InputCreateChannel;
+import cn.wildfirechat.pojos.InputSubscribeChannel;
+import cn.wildfirechat.pojos.OutputCreateChannel;
+import cn.wildfirechat.proto.ProtoConstants;
+import cn.wildfirechat.proto.WFCMessage;
+import com.google.gson.Gson;
+import com.xiaoleilu.loServer.RestResult;
+import com.xiaoleilu.loServer.action.channel.ChannelAction;
+import com.xiaoleilu.loServer.annotation.HttpMethod;
+import com.xiaoleilu.loServer.annotation.Route;
+import com.xiaoleilu.loServer.handler.Request;
+import com.xiaoleilu.loServer.handler.Response;
+import io.netty.buffer.ByteBuf;
+import io.netty.buffer.Unpooled;
+import io.netty.handler.codec.http.FullHttpRequest;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.netty.util.internal.StringUtil;
+import win.liyufan.im.IMTopic;
+
+@Route(APIPath.Subscribe_Channel)
+@HttpMethod("POST")
+public class ChannelSubscriberAction extends AdminAction {
+
+    @Override
+    public boolean action(Request request, Response response) {
+        if (request.getNettyRequest() instanceof FullHttpRequest) {
+            InputSubscribeChannel inputSubscribeChannel = getRequestBody(request.getNettyRequest(), InputSubscribeChannel.class);
+            if (inputSubscribeChannel != null
+                && !io.netty.util.internal.StringUtil.isNullOrEmpty(inputSubscribeChannel.getChannelId())
+                && !io.netty.util.internal.StringUtil.isNullOrEmpty(inputSubscribeChannel.getUserId())) {
+
+                WFCMessage.ListenChannel.Builder builder = WFCMessage.ListenChannel.newBuilder().setChannelId(inputSubscribeChannel.getChannelId()).setListen(inputSubscribeChannel.getSubscribe());
+                sendApiMessage(response, inputSubscribeChannel.getUserId(), IMTopic.ChannelListenTopic, builder.build().toByteArray(), result -> {
+                    ErrorCode errorCode = ErrorCode.fromCode(result[0]);
+                    if (errorCode == ErrorCode.ERROR_CODE_SUCCESS) {
+                        return new Result(ErrorCode.ERROR_CODE_SUCCESS);
+                    } else {
+                        return new Result(errorCode);
+                    }
+                });
+                return false;
+            } else {
+                setResponseContent(RestResult.resultOf(ErrorCode.INVALID_PARAMETER), response);
+            }
+
+        }
+        return true;
+    }
+}

broker/src/main/java/com/xiaoleilu/loServer/action/admin/CheckUserOnlineAction.java

@@ -36,7 +36,7 @@ public class CheckUserOnlineAction extends AdminAction {
             InputGetUserInfo inputUserId = getRequestBody(request.getNettyRequest(), InputGetUserInfo.class);
             if (inputUserId == null || !StringUtil.isNullOrEmpty(inputUserId.getUserId())) {
                 sendApiMessage(response, inputUserId.getUserId(), ServerAPIHelper.CHECK_USER_ONLINE_REQUEST, inputUserId.getUserId().getBytes(), res -> {
-                    OutputCheckUserOnline out = new Gson().fromJson(new String(res), OutputCheckUserOnline.class);
+                    OutputCheckUserOnline out = gson.fromJson(new String(res), OutputCheckUserOnline.class);
                     return new Result(ErrorCode.ERROR_CODE_SUCCESS, out);
                 });
                 return false;

broker/src/main/java/com/xiaoleilu/loServer/action/admin/CheckUserSubscribeChannelAction.java

@@ -0,0 +1,52 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package com.xiaoleilu.loServer.action.admin;
+
+import cn.wildfirechat.common.APIPath;
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.pojos.InputChannelId;
+import cn.wildfirechat.pojos.InputSubscribeChannel;
+import cn.wildfirechat.pojos.OutputBooleanValue;
+import cn.wildfirechat.pojos.OutputGetChannelInfo;
+import cn.wildfirechat.proto.WFCMessage;
+import com.xiaoleilu.loServer.RestResult;
+import com.xiaoleilu.loServer.annotation.HttpMethod;
+import com.xiaoleilu.loServer.annotation.Route;
+import com.xiaoleilu.loServer.handler.Request;
+import com.xiaoleilu.loServer.handler.Response;
+import io.netty.handler.codec.http.FullHttpRequest;
+import io.netty.util.internal.StringUtil;
+
+@Route(APIPath.Check_User_Subscribe_Channel)
+@HttpMethod("POST")
+public class CheckUserSubscribeChannelAction extends AdminAction {
+
+    @Override
+    public boolean isTransactionAction() {
+        return true;
+    }
+
+    @Override
+    public boolean action(Request request, Response response) {
+        if (request.getNettyRequest() instanceof FullHttpRequest) {
+            InputSubscribeChannel inputSubscribeChannel = getRequestBody(request.getNettyRequest(), InputSubscribeChannel.class);
+            if (inputSubscribeChannel != null
+                && !StringUtil.isNullOrEmpty(inputSubscribeChannel.getChannelId())
+                && !StringUtil.isNullOrEmpty(inputSubscribeChannel.getUserId())) {
+                boolean isInChannel = messagesStore.checkUserInChannel(inputSubscribeChannel.getUserId(), inputSubscribeChannel.getChannelId());
+                OutputBooleanValue outputBooleanValue = new OutputBooleanValue();
+                outputBooleanValue.value = isInChannel;
+                setResponseContent(RestResult.ok(outputBooleanValue), response);
+            } else {
+                setResponseContent(RestResult.resultOf(ErrorCode.INVALID_PARAMETER), response);
+            }
+        }
+        return true;
+    }
+}

broker/src/main/java/com/xiaoleilu/loServer/action/admin/CreateChatroomAction.java

@@ -39,7 +39,7 @@ public class CreateChatroomAction extends AdminAction {
             FullHttpRequest fullHttpRequest = (FullHttpRequest)request.getNettyRequest();
             byte[] bytes = Utils.readBytesAndRewind(fullHttpRequest.content());
             String content = new String(bytes);
-            Gson gson = new Gson();
+            
             InputCreateChatroom inputCreateChatroom = gson.fromJson(content, InputCreateChatroom.class);
             if (inputCreateChatroom != null
                 && !StringUtil.isNullOrEmpty(inputCreateChatroom.getTitle())) {

broker/src/main/java/com/xiaoleilu/loServer/action/admin/CreateRobotAction.java

@@ -67,7 +67,7 @@ public class CreateRobotAction extends AdminAction {
                     Utility.printExecption(LOG, e, IMExceptionEvent.EventType.ADMIN_API_Exception);
                     response.setStatus(HttpResponseStatus.OK);
                     RestResult result = RestResult.resultOf(ErrorCode.ERROR_CODE_SERVER_ERROR, e.getMessage());
-                    response.setContent(new Gson().toJson(result));
+                    response.setContent(gson.toJson(result));
                     return true;
                 }
 

broker/src/main/java/com/xiaoleilu/loServer/action/admin/DestoryChatroomAction.java

@@ -37,7 +37,7 @@ public class DestoryChatroomAction extends AdminAction {
             FullHttpRequest fullHttpRequest = (FullHttpRequest)request.getNettyRequest();
             byte[] bytes = Utils.readBytesAndRewind(fullHttpRequest.content());
             String content = new String(bytes);
-            Gson gson = new Gson();
+            
             InputDestoryChatroom inputDestoryChatroom = gson.fromJson(content, InputDestoryChatroom.class);
             if (inputDestoryChatroom != null
                 && !StringUtil.isNullOrEmpty(inputDestoryChatroom.getChatroomId())) {

broker/src/main/java/com/xiaoleilu/loServer/action/admin/GetAllUserAction.java

@@ -0,0 +1,59 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package com.xiaoleilu.loServer.action.admin;
+
+import cn.wildfirechat.common.APIPath;
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.pojos.InputGetUserInfo;
+import cn.wildfirechat.pojos.InputGetUserList;
+import cn.wildfirechat.pojos.InputOutputUserInfo;
+import cn.wildfirechat.pojos.OutputGetUserList;
+import cn.wildfirechat.proto.WFCMessage;
+import com.xiaoleilu.loServer.RestResult;
+import com.xiaoleilu.loServer.annotation.HttpMethod;
+import com.xiaoleilu.loServer.annotation.Route;
+import com.xiaoleilu.loServer.handler.Request;
+import com.xiaoleilu.loServer.handler.Response;
+import io.netty.handler.codec.http.FullHttpRequest;
+import io.netty.util.internal.StringUtil;
+
+import java.util.ArrayList;
+import java.util.List;
+
+@Route(APIPath.User_Get_All)
+@HttpMethod("POST")
+public class GetAllUserAction extends AdminAction {
+
+    @Override
+    public boolean isTransactionAction() {
+        return true;
+    }
+
+    @Override
+    public boolean action(Request request, Response response) {
+        if (request.getNettyRequest() instanceof FullHttpRequest) {
+            InputGetUserList input = getRequestBody(request.getNettyRequest(), InputGetUserList.class);
+            if (input != null && input.count > 0 && input.offset >= 0) {
+                List<WFCMessage.User> users = messagesStore.getUserInfoList(input.count, input.offset);
+                OutputGetUserList outputGetUserList = new OutputGetUserList();
+                outputGetUserList.userInfoList = new ArrayList<>();
+                for (WFCMessage.User user : users) {
+                    outputGetUserList.userInfoList.add(InputOutputUserInfo.fromPbUser(user));
+                }
+
+                RestResult result = RestResult.ok(outputGetUserList);
+                setResponseContent(result, response);
+            } else {
+                setResponseContent(RestResult.resultOf(ErrorCode.INVALID_PARAMETER), response);
+            }
+
+        }
+        return true;
+    }
+}

broker/src/main/java/com/xiaoleilu/loServer/action/admin/GetCommonGroupsAction.java

@@ -0,0 +1,55 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package com.xiaoleilu.loServer.action.admin;
+
+import cn.wildfirechat.common.APIPath;
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.pojos.InputUserId;
+import cn.wildfirechat.pojos.OutputGroupIds;
+import cn.wildfirechat.pojos.StringPairPojo;
+import com.xiaoleilu.loServer.RestResult;
+import com.xiaoleilu.loServer.annotation.HttpMethod;
+import com.xiaoleilu.loServer.annotation.Route;
+import com.xiaoleilu.loServer.handler.Request;
+import com.xiaoleilu.loServer.handler.Response;
+import io.netty.handler.codec.http.FullHttpRequest;
+import io.netty.util.internal.StringUtil;
+
+import java.util.ArrayList;
+import java.util.Set;
+
+@Route(APIPath.Get_Common_Groups)
+@HttpMethod("POST")
+public class GetCommonGroupsAction extends AdminAction {
+
+    @Override
+    public boolean isTransactionAction() {
+        return true;
+    }
+
+    @Override
+    public boolean action(Request request, Response response) {
+        if (request.getNettyRequest() instanceof FullHttpRequest) {
+            StringPairPojo input = getRequestBody(request.getNettyRequest(), StringPairPojo.class);
+            if (input != null
+                && (!StringUtil.isNullOrEmpty(input.getFirst())) && (!StringUtil.isNullOrEmpty(input.getSecond()))) {
+
+                Set<String> groupIds = messagesStore.getCommonGroupIds(input.getFirst(), input.getSecond());
+                OutputGroupIds outputGroupIds = new OutputGroupIds();
+                outputGroupIds.setGroupIds(new ArrayList<>(groupIds));
+                RestResult result = RestResult.ok(outputGroupIds);
+                setResponseContent(result, response);
+            } else {
+                setResponseContent(RestResult.resultOf(ErrorCode.INVALID_PARAMETER), response);
+            }
+
+        }
+        return true;
+    }
+}

broker/src/main/java/com/xiaoleilu/loServer/action/admin/GetGroupMemberAction.java

@@ -0,0 +1,69 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package com.xiaoleilu.loServer.action.admin;
+
+import cn.wildfirechat.common.APIPath;
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.pojos.InputGetGroup;
+import cn.wildfirechat.pojos.InputGetGroupMember;
+import cn.wildfirechat.pojos.OutputGroupMemberList;
+import cn.wildfirechat.pojos.PojoGroupMember;
+import cn.wildfirechat.proto.ProtoConstants;
+import cn.wildfirechat.proto.WFCMessage;
+import com.xiaoleilu.loServer.RestResult;
+import com.xiaoleilu.loServer.annotation.HttpMethod;
+import com.xiaoleilu.loServer.annotation.Route;
+import com.xiaoleilu.loServer.handler.Request;
+import com.xiaoleilu.loServer.handler.Response;
+import io.netty.handler.codec.http.FullHttpRequest;
+import io.netty.util.internal.StringUtil;
+
+import java.util.ArrayList;
+import java.util.List;
+
+@Route(APIPath.Group_Member_Get)
+@HttpMethod("POST")
+public class GetGroupMemberAction extends AdminAction {
+
+    @Override
+    public boolean isTransactionAction() {
+        return true;
+    }
+
+    @Override
+    public boolean action(Request request, Response response) {
+        if (request.getNettyRequest() instanceof FullHttpRequest) {
+            InputGetGroupMember input = getRequestBody(request.getNettyRequest(), InputGetGroupMember.class);
+            if (input != null
+                && (!StringUtil.isNullOrEmpty(input.getGroupId()))
+                && (!StringUtil.isNullOrEmpty(input.getMemberId()))) {
+
+                WFCMessage.GroupMember groupMember = messagesStore.getGroupMember(input.getGroupId(), input.getMemberId());
+
+                RestResult result;
+                if (groupMember == null || StringUtil.isNullOrEmpty(groupMember.getMemberId()) || groupMember.getType() == ProtoConstants.GroupMemberType.GroupMemberType_Removed) {
+                    result = RestResult.resultOf(ErrorCode.ERROR_CODE_NOT_EXIST);
+                } else {
+                    PojoGroupMember pm = new PojoGroupMember();
+                    pm.setMember_id(groupMember.getMemberId());
+                    pm.setAlias(groupMember.getAlias());
+                    pm.setType(groupMember.getType());
+                    pm.setExtra(groupMember.getExtra());
+                    pm.setCreateDt(groupMember.getCreateDt());
+                    result = RestResult.ok(pm);
+                }
+                setResponseContent(result, response);
+            } else {
+                setResponseContent(RestResult.resultOf(ErrorCode.INVALID_PARAMETER), response);
+            }
+
+        }
+        return true;
+    }
+}

broker/src/main/java/com/xiaoleilu/loServer/action/admin/GetUserAction.java

@@ -49,7 +49,7 @@ public class GetUserAction extends AdminAction {
                 }
 
                 RestResult result;
-                if (user == null) {
+                if (user == null || user.getDeleted() > 0) {
                     result = RestResult.resultOf(ErrorCode.ERROR_CODE_NOT_EXIST);
                 } else {
                     result = RestResult.ok(InputOutputUserInfo.fromPbUser(user));

broker/src/main/java/com/xiaoleilu/loServer/action/admin/GetUserGroupsByTypeAction.java

@@ -0,0 +1,55 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package com.xiaoleilu.loServer.action.admin;
+
+import cn.wildfirechat.common.APIPath;
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.pojos.InputGetUserGroupByType;
+import cn.wildfirechat.pojos.InputUserId;
+import cn.wildfirechat.pojos.OutputGroupIds;
+import com.xiaoleilu.loServer.RestResult;
+import com.xiaoleilu.loServer.annotation.HttpMethod;
+import com.xiaoleilu.loServer.annotation.Route;
+import com.xiaoleilu.loServer.handler.Request;
+import com.xiaoleilu.loServer.handler.Response;
+import io.netty.handler.codec.http.FullHttpRequest;
+import io.netty.util.internal.StringUtil;
+
+import java.util.ArrayList;
+import java.util.Set;
+
+@Route(APIPath.Get_User_Groups_By_Type)
+@HttpMethod("POST")
+public class GetUserGroupsByTypeAction extends AdminAction {
+
+    @Override
+    public boolean isTransactionAction() {
+        return true;
+    }
+
+    @Override
+    public boolean action(Request request, Response response) {
+        if (request.getNettyRequest() instanceof FullHttpRequest) {
+            InputGetUserGroupByType input = getRequestBody(request.getNettyRequest(), InputGetUserGroupByType.class);
+            if (input != null
+                && (!StringUtil.isNullOrEmpty(input.getUserId()))) {
+
+                Set<String> groupIds = messagesStore.getUserGroupIds(input.getUserId(), input.getGroupMemberTypes());
+                OutputGroupIds outputGroupIds = new OutputGroupIds();
+                outputGroupIds.setGroupIds(new ArrayList<>(groupIds));
+                RestResult result = RestResult.ok(outputGroupIds);
+                setResponseContent(result, response);
+            } else {
+                setResponseContent(RestResult.resultOf(ErrorCode.INVALID_PARAMETER), response);
+            }
+
+        }
+        return true;
+    }
+}

broker/src/main/java/com/xiaoleilu/loServer/action/admin/ModifyGroupMemberExtraAction.java

@@ -50,7 +50,7 @@ public class ModifyGroupMemberExtraAction extends AdminAction {
             } else {
                 response.setStatus(HttpResponseStatus.OK);
                 RestResult result = RestResult.resultOf(ErrorCode.INVALID_PARAMETER);
-                response.setContent(new Gson().toJson(result));
+                response.setContent(gson.toJson(result));
             }
         }
         return true;

broker/src/main/java/com/xiaoleilu/loServer/action/admin/RecallMessageAction.java

@@ -25,6 +25,9 @@ import io.netty.util.internal.StringUtil;
 import cn.wildfirechat.common.ErrorCode;
 import win.liyufan.im.IMTopic;
 
+import java.nio.charset.StandardCharsets;
+import java.util.Arrays;
+
 @Route(APIPath.Msg_Recall)
 @HttpMethod("POST")
 public class RecallMessageAction extends AdminAction {
@@ -42,10 +45,9 @@ public class RecallMessageAction extends AdminAction {
 
                 WFCMessage.INT64Buf idBuf = WFCMessage.INT64Buf.newBuilder().setId(recallMessageData.getMessageUid()).build();
                 sendApiMessage(response, recallMessageData.getOperator(), IMTopic.RecallMessageTopic, idBuf.toByteArray(), result -> {
-                    ByteBuf byteBuf = Unpooled.buffer();
-                    byteBuf.writeBytes(result);
-                    ErrorCode errorCode = ErrorCode.fromCode(byteBuf.readByte());
-                    return new Result(errorCode);
+                    ErrorCode errorCode = ErrorCode.fromCode(result[0]);
+                    byte[] data = Arrays.copyOfRange(result,1, result.length);
+                    return new Result(errorCode, new String(data, StandardCharsets.UTF_8));
                 });
                 return false;
             } else {

broker/src/main/java/com/xiaoleilu/loServer/action/channel/ApplicationGetUserInfoAction.java

@@ -52,7 +52,7 @@ public class ApplicationGetUserInfoAction extends ChannelAction {
             }
 
             response.setStatus(HttpResponseStatus.OK);
-            response.setContent(new Gson().toJson(result));
+            response.setContent(gson.toJson(result));
         }
         return true;
     }

broker/src/main/java/com/xiaoleilu/loServer/action/channel/ChannelAction.java

@@ -101,7 +101,7 @@ abstract public class ChannelAction extends Action {
         }
 
         RestResult result = RestResult.resultOf(errorCode, errorCode.getMsg(), data);
-        response.setContent(new Gson().toJson(result));
+        response.setContent(gson.toJson(result));
         response.send();
     }
 
@@ -110,7 +110,7 @@ abstract public class ChannelAction extends Action {
             FullHttpRequest fullHttpRequest = (FullHttpRequest) request;
             byte[] bytes = Utils.readBytesAndRewind(fullHttpRequest.content());
             String content = new String(bytes, StandardCharsets.UTF_8);
-            Gson gson = new Gson();
+            
             T t = gson.fromJson(content, cls);
             return t;
         }

broker/src/main/java/com/xiaoleilu/loServer/action/channel/GetIsChannelSubscriberAction.java

@@ -0,0 +1,37 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package com.xiaoleilu.loServer.action.channel;
+
+import cn.wildfirechat.common.APIPath;
+import cn.wildfirechat.pojos.InputChannelSubscribe;
+import cn.wildfirechat.pojos.InputUserId;
+import cn.wildfirechat.pojos.OutputStringList;
+import com.xiaoleilu.loServer.RestResult;
+import com.xiaoleilu.loServer.annotation.HttpMethod;
+import com.xiaoleilu.loServer.annotation.Route;
+import com.xiaoleilu.loServer.handler.Request;
+import com.xiaoleilu.loServer.handler.Response;
+import io.netty.handler.codec.http.FullHttpRequest;
+
+import java.util.ArrayList;
+
+@Route(APIPath.Channel_Is_Subscriber)
+@HttpMethod("POST")
+public class GetIsChannelSubscriberAction extends ChannelAction {
+
+    @Override
+    public boolean action(Request request, Response response) {
+        if (request.getNettyRequest() instanceof FullHttpRequest) {
+            InputUserId input = getRequestBody(request.getNettyRequest(), InputUserId.class);
+            boolean isSubscriber = messagesStore.checkUserInChannel(input.getUserId(), channelInfo.getTargetId());
+            setResponseContent(RestResult.ok(isSubscriber), response);
+        }
+        return true;
+    }
+}

broker/src/main/java/com/xiaoleilu/loServer/action/channel/GetUserAction.java

@@ -44,7 +44,7 @@ public class GetUserAction extends ChannelAction {
 
                 RestResult result;
 
-                if (user == null || StringUtil.isNullOrEmpty(user.getName())) {
+                if (user == null || StringUtil.isNullOrEmpty(user.getName()) || user.getDeleted() > 0) {
                     result = RestResult.resultOf(ErrorCode.ERROR_CODE_NOT_EXIST);
                 } else {
                     if (channelInfo.getOwner().equals(inputUserId.getUserId()) ||

broker/src/main/java/com/xiaoleilu/loServer/action/channel/SubscriberChannelAction.java

@@ -38,7 +38,7 @@ public class SubscriberChannelAction extends ChannelAction {
                 if(input.getSubscribe() > 0 && (channelInfo.getStatus() & ProtoConstants.ChannelState.Channel_State_Mask_Active_Subscribe) == 0) {
                     response.setStatus(HttpResponseStatus.OK);
                     RestResult result = RestResult.resultOf(ErrorCode.ERROR_CODE_NOT_RIGHT);
-                    response.setContent(new Gson().toJson(result));
+                    response.setContent(gson.toJson(result));
                     return true;
                 }
                 WFCMessage.ListenChannel listenChannel = WFCMessage.ListenChannel.newBuilder().setChannelId(channelInfo.getTargetId()).setListen(input.getSubscribe()).build();

broker/src/main/java/com/xiaoleilu/loServer/action/robot/ApplicationGetUserInfoAction.java

@@ -52,7 +52,7 @@ public class ApplicationGetUserInfoAction extends RobotAction {
             }
 
             response.setStatus(HttpResponseStatus.OK);
-            response.setContent(new Gson().toJson(result));
+            response.setContent(gson.toJson(result));
         }
         return true;
     }

broker/src/main/java/com/xiaoleilu/loServer/action/robot/GetGroupMemberAction.java

@@ -0,0 +1,98 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package com.xiaoleilu.loServer.action.robot;
+
+import cn.wildfirechat.common.APIPath;
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.pojos.InputGetGroup;
+import cn.wildfirechat.pojos.InputGetGroupMember;
+import cn.wildfirechat.pojos.OutputGroupMemberList;
+import cn.wildfirechat.pojos.PojoGroupMember;
+import cn.wildfirechat.proto.ProtoConstants;
+import cn.wildfirechat.proto.WFCMessage;
+import com.xiaoleilu.loServer.RestResult;
+import com.xiaoleilu.loServer.annotation.HttpMethod;
+import com.xiaoleilu.loServer.annotation.Route;
+import com.xiaoleilu.loServer.handler.Request;
+import com.xiaoleilu.loServer.handler.Response;
+import io.netty.handler.codec.http.FullHttpRequest;
+import io.netty.util.internal.StringUtil;
+
+import java.util.ArrayList;
+import java.util.List;
+
+@Route(APIPath.Robot_Group_Member_Get)
+@HttpMethod("POST")
+public class GetGroupMemberAction extends RobotAction {
+
+    @Override
+    public boolean isTransactionAction() {
+        return true;
+    }
+
+    @Override
+    public boolean action(Request request, Response response) {
+        if (request.getNettyRequest() instanceof FullHttpRequest) {
+            InputGetGroupMember input = getRequestBody(request.getNettyRequest(), InputGetGroupMember.class);
+            String robotId = robot.getUid();
+            if (input != null
+                && (!StringUtil.isNullOrEmpty(input.getGroupId()))
+                && (!StringUtil.isNullOrEmpty(input.getMemberId()))) {
+
+                List<WFCMessage.GroupMember> members = new ArrayList<>();
+                ErrorCode errorCode = messagesStore.getGroupMembers(null, input.getGroupId(), 0, members);
+                RestResult result;
+                if (errorCode != ErrorCode.ERROR_CODE_SUCCESS) {
+                    result = RestResult.resultOf(errorCode);
+                } else {
+                    WFCMessage.GroupMember groupMember = null;
+                    boolean isInGroup = false;
+                    for (WFCMessage.GroupMember member : members) {
+                        if (member.getType() == ProtoConstants.GroupMemberType.GroupMemberType_Removed) {
+                            continue;
+                        }
+                        if(member.getMemberId().equals(robotId)) {
+                            isInGroup = true;
+                            if(groupMember != null) {
+                                break;
+                            }
+                        }
+                        if(member.getMemberId().equals(input.getMemberId())) {
+                            groupMember = member;
+                            if(isInGroup) {
+                                break;
+                            }
+                        }
+                    }
+                    if(!isInGroup) {
+                        result = RestResult.resultOf(ErrorCode.ERROR_CODE_NOT_IN_GROUP);
+                    } else {
+                        if(groupMember == null) {
+                            result = RestResult.resultOf(ErrorCode.ERROR_CODE_NOT_EXIST);
+                        } else {
+                            PojoGroupMember pm = new PojoGroupMember();
+                            pm.setMember_id(groupMember.getMemberId());
+                            pm.setAlias(groupMember.getAlias());
+                            pm.setType(groupMember.getType());
+                            pm.setExtra(groupMember.getExtra());
+                            pm.setCreateDt(groupMember.getCreateDt());
+                            result = RestResult.ok(pm);
+                        }
+                    }
+                }
+
+                setResponseContent(result, response);
+            } else {
+                setResponseContent(RestResult.resultOf(ErrorCode.INVALID_PARAMETER), response);
+            }
+
+        }
+        return true;
+    }
+}

broker/src/main/java/com/xiaoleilu/loServer/action/robot/GetUserAction.java

@@ -44,7 +44,7 @@ public class GetUserAction extends RobotAction {
                 }
 
                 RestResult result;
-                if (user == null) {
+                if (user == null || user.getDeleted() > 0) {
                     result = RestResult.resultOf(ErrorCode.ERROR_CODE_NOT_EXIST);
                 } else {
                     result = RestResult.ok(InputOutputUserInfo.fromPbUser(user));

broker/src/main/java/com/xiaoleilu/loServer/action/robot/ModifyGroupMemberExtraAction.java

@@ -53,7 +53,7 @@ public class ModifyGroupMemberExtraAction extends RobotAction {
             } else {
                 response.setStatus(HttpResponseStatus.OK);
                 RestResult result = RestResult.resultOf(ErrorCode.INVALID_PARAMETER);
-                response.setContent(new Gson().toJson(result));
+                response.setContent(gson.toJson(result));
             }
         }
         return true;

broker/src/main/java/com/xiaoleilu/loServer/action/robot/RobotAction.java

@@ -102,7 +102,7 @@ abstract public class RobotAction extends Action {
         }
 
         RestResult result = RestResult.resultOf(errorCode, errorCode.getMsg(), data);
-        response.setContent(new Gson().toJson(result));
+        response.setContent(gson.toJson(result));
         response.send();
     }
 
@@ -113,7 +113,7 @@ abstract public class RobotAction extends Action {
             String content = null;
             try {
                 content = new String(bytes, "UTF-8");
-                Gson gson = new Gson();
+                
                 T t = gson.fromJson(content, cls);
                 return t;
             } catch (UnsupportedEncodingException e) {

broker/src/main/java/com/xiaoleilu/loServer/handler/ActionHandler.java

@@ -15,6 +15,7 @@ import io.netty.channel.ChannelHandlerContext;
 import io.netty.channel.SimpleChannelInboundHandler;
 import io.netty.handler.codec.http.FullHttpRequest;
 import io.netty.handler.codec.http.HttpResponseStatus;
+import io.netty.util.internal.StringUtil;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import win.liyufan.im.Utility;
@@ -34,7 +35,6 @@ abstract public class ActionHandler extends SimpleChannelInboundHandler<FullHttp
 
     @Override
 	protected void channelRead0(ChannelHandlerContext ctx, FullHttpRequest fullHttpRequest) throws Exception {
-        Logger.info("Http request whit url {}", fullHttpRequest.uri());
 		final Request request = Request.build(ctx, fullHttpRequest);
 		final Response response = Response.build(ctx, request);
 
@@ -109,14 +109,24 @@ abstract public class ActionHandler extends SimpleChannelInboundHandler<FullHttp
 	 * @param response 响应对象
 	 */
 	private void doAction(ChannelHandlerContext ctx, Request request, Response response){
-	    if(("/route".equalsIgnoreCase(request.getPath()) || request.getPath().startsWith("/fs/"))
+	    if(("/route".equalsIgnoreCase(request.getPath()) || (!StringUtil.isNullOrEmpty(request.getPath()) && request.getPath().startsWith("/fs/")))
             && "OPTIONS".equalsIgnoreCase(request.getMethod())){
 	        handleOptions(ctx, request, response);
 	        return;
         }
+
+        if ("HEAD".equalsIgnoreCase(request.getMethod())) {
+            response.setStatus(HttpResponseStatus.OK);
+            response.send();
+            return;
+        }
+
 		Action action;
 		if (isValidePath(request.getPath())) {
             action = ServerSetting.getAction(request.getPath(), request.getMethod().toUpperCase());
+            if (action != null) {
+                Logger.info("Http request whit url {}", request.getPath());
+            }
         } else {
             action = ServerSetting.getErrorAction(ServerSetting.MAPPING_ERROR);
         }
@@ -130,7 +140,11 @@ abstract public class ActionHandler extends SimpleChannelInboundHandler<FullHttp
 			action = ServerSetting.getAction(ServerSetting.MAPPING_ALL, request.getMethod());
 			if(null == action){
 				// 非Action方法，调用静态文件读取
-				action = Singleton.get(FileAction.class);
+                if(request.getPath().startsWith("/fs")) {
+                    action = Singleton.get(FileAction.class);
+                } else {
+                    action = ServerSetting.getErrorAction(ServerSetting.MAPPING_ERROR);
+                }
 			}
 		}
         action.ctx = ctx;

broker/src/main/java/com/xiaoleilu/loServer/handler/Request.java

@@ -81,18 +81,6 @@ public class Request {
 
 		// request URI parameters
 		this.putParams(new QueryStringDecoder(uri));
-		if(nettyRequest.method() != HttpMethod.GET){
-			HttpPostRequestDecoder decoder = null;
-			try {
-				decoder = new HttpPostRequestDecoder(HTTP_DATA_FACTORY, nettyRequest);
-				this.putParams(decoder);
-			} finally {
-				if(null != decoder){
-					decoder.destroy();
-					decoder = null;
-				}
-			}
-		}
 
 		// IP
 		this.putIp(ctx);

broker/src/main/java/com/xiaoleilu/loServer/handler/Response.java

@@ -41,6 +41,7 @@ import io.netty.handler.codec.http.LastHttpContent;
 import io.netty.handler.codec.http.cookie.Cookie;
 import io.netty.handler.codec.http.cookie.DefaultCookie;
 import io.netty.handler.codec.http.cookie.ServerCookieEncoder;
+import org.apache.tika.Tika;
 import org.slf4j.LoggerFactory;
 
 /**
@@ -469,7 +470,9 @@ public class Response {
 		this.setContentLength(fileLength);
 		
 		//文件类型
-		String contentType = HttpUtil.getMimeType(file.getName());
+        Tika tika = new Tika();
+        String contentType = tika.detect(file);
+//		String contentType = HttpUtil.getMimeType(file.getName());
 		if(StrUtil.isBlank(contentType)){
 			//无法识别默认使用数据流
 			contentType = "application/octet-stream";

broker/src/main/java/io/moquette/BrokerConstants.java

@@ -29,6 +29,8 @@ public final class BrokerConstants {
     public static final String HTTP_SERVER_PORT = "http_port";
     public static final String HTTP_LOCAL_PORT = "local_port";
 
+    public static final String ENCRYPT_AES256 = "encrypt.use_aes256";
+
     public static final String HTTP_ADMIN_PORT = "http.admin.port";
     public static final String DEFAULT_MOQUETTE_STORE_MAP_DB_FILENAME = "moquette_store.mapdb";
     public static final String DEFAULT_PERSISTENT_PATH = System.getProperty("user.dir") + File.separator
@@ -110,8 +112,13 @@ public final class BrokerConstants {
     public static final String TOKEN_SECRET_KEY = "token.key";
     public static final String TOKEN_EXPIRE_TIME = "token.expire_time";
 
+    public static final String CONNECT_CLIENT_SIGNATURE_LIST = "connect.client_signature_list";
+    public static final String CONNECT_REJECT_EMPTY_SIGNATURE = "connect.reject_empty_signature";
+
     public static final String EMBED_DB_PROPERTY_NAME = "embed.db";
     public static final String DB_AUTO_CLEAN_HISTORY_MESSAGES = "db.auto_clean_history_messages";
+    public static final String H2DB_PATH = "h2db.path";
+
 
     public static final String SENSITIVE_Filter_Type = "sensitive.filter.type";
 
@@ -120,9 +127,11 @@ public final class BrokerConstants {
     public static final String SENSITIVE_Remote_Fail_When_Matched = "sensitive.remote_fail_when_matched";
 
     public static final String MESSAGE_Forward_Url = "message.forward.url";
+    public static final String MESSAGE_Sensitive_Forward_Url = "message.sensitive.forward.url";
     public static final String MESSAGE_Forward_Types = "message.forward.types";
+    public static final String MESSAGE_Forward_Exclude_Types = "message.forward.exclude_types";
     public static final String MESSAGE_MentionMsg_Forward_Url = "message.mentionmsg.forward.url";
-
+    public static final String MESSAGE_RecallMsg_Forward_Url = "message.recallmsg.forward.url";
 
     public static final String SERVER_MULTI_ENDPOINT = "server.multi_endpoint";
     public static final String SERVER_MULTI_PC_ENDPOINT = "server.multi_pc_endpoint";
@@ -131,34 +140,46 @@ public final class BrokerConstants {
     public static final String SERVER_CLIENT_SUPPORT_KICKOFF_EVENT = "server.client_support_kickoff_event";
 
     public static final String MESSAGE_ROAMING = "message.roaming";
+    public static final String MESSAGE_Compensate_Time_Limit = "message.compensate_time_limit";
     public static final String MESSAGE_Remote_History_Message = "message.remote_history_message";
     public static final String MESSAGE_Remote_Chatroom_History_Message = "message.chatroom_remote_history_message";
 
     public static final String MESSAGE_Max_Queue = "message.max_queue";
 
     public static final String MESSAGE_Disable_Stranger_Chat = "message.disable_stranger_chat";
+    public static final String MESSAGE_Allow_Stranger_Chat_List = "message.allow_stranger_chat_list";
 
     public static final String MESSAGE_Blacklist_Strategy = "message.blacklist.strategy";
 
     public static final String MESSAGE_NO_Forward_Admin_Message = "message.no_forward_admin_message";
 
+    public static final String MESSAGE_Push_Expired_Days = "message.push_expired_days";
+
+    public static final String MESSAGE_Forward_With_Client_Info = "message.forward_with_client_info";
+    public static final String ROBOT_Callback_With_Client_Info = "robot.callback_with_client_info";
+    public static final String CHANNEL_Callback_With_Client_Info = "channel.callback_with_client_info";
+
     public static final String FRIEND_Disable_Search = "friend.disable_search";
     public static final String FRIEND_Disable_NickName_Search = "friend.disable_nick_name_search";
     public static final String FRIEND_Disable_Friend_Request = "friend.disable_friend_request";
     public static final String FRIEND_Repeat_Request_Duration = "friend.repeat_request_duration";
     public static final String FRIEND_Reject_Request_Duration = "friend.reject_request_duration";
     public static final String FRIEND_Request_Expiration_Duration = "friend.request_expiration_duration";
-    public static final String FRIEND_New_Welcome_Message = "friend.new_welcome_message";
+    public static final String FRIEND_Request_Robot_Auto_Accept = "friend.robot_auto_accept";
+
 
     public static final String CHATROOM_Participant_Idle_Time = "chatroom.participant_idle_time";
     public static final String CHATROOM_Rejoin_When_Active = "chatroom.rejoin_when_active";
     public static final String CHATROOM_Create_When_Not_Exist = "chatroom.create_when_not_exist";
 
 
+    public static final String GROUP_Allow_Owner_Recall_Self_Msg = "group.allow_owner_recall_self_msg";
+    public static final String GROUP_Allow_Manager_Recall_Self_Msg = "group.allow_manager_recall_self_msg";
     public static final String GROUP_Allow_Client_Custom_Operation_Notification = "group.allow_client_custom_operation_notification";
     public static final String GROUP_Allow_Robot_Custom_Operation_Notification = "group.allow_robot_custom_operation_notificatio";
 
     public static final String GROUP_Visible_Quit_Kickoff_Notification = "group.visible_quit_or_kickoff_notification";
+    public static final String GROUP_Forbidden_Client_Operation = "group.forbidden_client_operation";
 
     public static final String USER_HIDE_PROPERTIES = "user.hide_properties";
     public static final String USER_KEEP_DISPLAY_NAME_WHEN_DESTROY = "user.keep_display_name_when_destroy";

broker/src/main/java/io/moquette/imhandler/AddFriendHandler.java

@@ -30,8 +30,10 @@ public class AddFriendHandler extends GroupHandler<WFCMessage.AddFriendRequest>
                 if (user != null && user.getType() == ProtoConstants.UserType.UserType_Normal) {
                     publisher.publishNotification(IMTopic.NotifyFriendRequestTopic, request.getTargetUid(), head[0], fromUser, request.getReason());
                 } else if(user != null && user.getType() == ProtoConstants.UserType.UserType_Robot) {
-                    WFCMessage.HandleFriendRequest handleFriendRequest = WFCMessage.HandleFriendRequest.newBuilder().setTargetUid(fromUser).setStatus(ProtoConstants.FriendRequestStatus.RequestStatus_Accepted).build();
-                    mServer.onApiMessage(request.getTargetUid(), null, handleFriendRequest.toByteArray(), 0, fromUser, HandleFriendRequestTopic, requestSourceType);
+                    if(m_messagesStore.isRobotAutoAcceptFriendRequest()) {
+                        WFCMessage.HandleFriendRequest handleFriendRequest = WFCMessage.HandleFriendRequest.newBuilder().setTargetUid(fromUser).setStatus(ProtoConstants.FriendRequestStatus.RequestStatus_Accepted).build();
+                        mServer.onApiMessage(request.getTargetUid(), null, handleFriendRequest.toByteArray(), 0, fromUser, HandleFriendRequestTopic, requestSourceType);
+                    }
                 }
             }
             return errorCode;

broker/src/main/java/io/moquette/imhandler/AddGroupMember.java

@@ -24,6 +24,20 @@ public class AddGroupMember extends GroupHandler<WFCMessage.AddGroupMemberReques
     @Override
     public ErrorCode action(ByteBuf ackPayload, String clientID, String fromUser, ProtoConstants.RequestSourceType requestSourceType, WFCMessage.AddGroupMemberRequest request, Qos1PublishHandler.IMCallback callback) {
         boolean isAdmin = requestSourceType == ProtoConstants.RequestSourceType.Request_From_Admin;
+
+        if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User) {
+            int forbiddenClientOperation = m_messagesStore.getGroupForbiddenClientOperation();
+            if(request.getAddedMemberList().size() == 1 && request.getAddedMember(0).getMemberId().equals(fromUser)) {
+                if ((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Join_Group) > 0) {
+                    return ErrorCode.ERROR_CODE_NOT_RIGHT;
+                }
+            } else {
+                if ((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Invite_Group_Member) > 0) {
+                    return ErrorCode.ERROR_CODE_NOT_RIGHT;
+                }
+            }
+        }
+
         ErrorCode errorCode = m_messagesStore.addGroupMembers(fromUser, isAdmin, request.getGroupId(), request.getAddedMemberList(), request.getExtra());
         if (errorCode == ERROR_CODE_SUCCESS) {
             if (request.hasNotifyContent() && request.getNotifyContent().getType() > 0) {

broker/src/main/java/io/moquette/imhandler/ChannelListenedListHandler.java

@@ -0,0 +1,23 @@
+package io.moquette.imhandler;
+
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.proto.ProtoConstants;
+import cn.wildfirechat.proto.WFCMessage;
+import io.moquette.spi.impl.Qos1PublishHandler;
+import io.netty.buffer.ByteBuf;
+import win.liyufan.im.IMTopic;
+
+import java.util.List;
+
+@Handler(IMTopic.ListenedChannelListTopic)
+public class ChannelListenedListHandler extends IMHandler<Void> {
+    @Override
+    public ErrorCode action(ByteBuf ackPayload, String clientID, String fromUser, ProtoConstants.RequestSourceType requestSourceType, Void request, Qos1PublishHandler.IMCallback callback) {
+        List<String> channelInfoList = m_messagesStore.getListenedChannels(fromUser);
+        WFCMessage.IDListBuf.Builder builder = WFCMessage.IDListBuf.newBuilder();
+        builder.addAllId(channelInfoList);
+        byte[] data = builder.build().toByteArray();
+        ackPayload.ensureWritable(data.length).writeBytes(data);
+        return ErrorCode.ERROR_CODE_SUCCESS;
+    }
+}

broker/src/main/java/io/moquette/imhandler/CreateGroupHandler.java

@@ -21,6 +21,9 @@ import win.liyufan.im.IMTopic;
 public class CreateGroupHandler extends GroupHandler<WFCMessage.CreateGroupRequest> {
     @Override
     public ErrorCode action(ByteBuf ackPayload, String clientID, String fromUser, ProtoConstants.RequestSourceType requestSourceType, WFCMessage.CreateGroupRequest request, Qos1PublishHandler.IMCallback callback) {
+        if(request.getGroup().getGroupInfo().getType() < 0 || request.getGroup().getGroupInfo().getType() > 3) {
+            return ErrorCode.ERROR_CODE_INVALID_DATA;
+        }
 
         if (!StringUtil.isNullOrEmpty(request.getGroup().getGroupInfo().getTargetId())) {
             WFCMessage.GroupInfo existGroupInfo = m_messagesStore.getGroupInfo(request.getGroup().getGroupInfo().getTargetId());
@@ -37,6 +40,16 @@ public class CreateGroupHandler extends GroupHandler<WFCMessage.CreateGroupReque
             return ErrorCode.ERROR_CODE_NOT_RIGHT;
         }
 
+        if(!isAdmin && request.getGroup().getGroupInfo().getType() == ProtoConstants.GroupType.GroupType_Organization) {
+            return ErrorCode.ERROR_CODE_NOT_RIGHT;
+        }
+
+        if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User) {
+            int forbiddenClientOperation = m_messagesStore.getGroupForbiddenClientOperation();
+            if((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Create_Group) > 0) {
+                return ErrorCode.ERROR_CODE_NOT_RIGHT;
+            }
+        }
         WFCMessage.GroupInfo groupInfo = m_messagesStore.createGroup(fromUser, request.getGroup().getGroupInfo(), request.getGroup().getMembersList(), request.getMemberExtra(), isAdmin);
         if (groupInfo != null) {
             if(request.hasNotifyContent() && request.getNotifyContent().getType() > 0) {

broker/src/main/java/io/moquette/imhandler/DismissGroupHandler.java

@@ -26,7 +26,6 @@ public class DismissGroupHandler extends GroupHandler<WFCMessage.DismissGroupReq
             ErrorCode errorCode;
             if (groupInfo == null) {
                 errorCode = m_messagesStore.dismissGroup(fromUser, request.getGroupId(), isAdmin);
-
             } else if (isAdmin || (groupInfo.getType() == ProtoConstants.GroupType.GroupType_Normal || groupInfo.getType() == ProtoConstants.GroupType.GroupType_Restricted)
                 && groupInfo.getOwner() != null && groupInfo.getOwner().equals(fromUser)) {
 
@@ -38,6 +37,13 @@ public class DismissGroupHandler extends GroupHandler<WFCMessage.DismissGroupReq
                     return ErrorCode.ERROR_CODE_NOT_RIGHT;
                 }
 
+                if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User) {
+                    int forbiddenClientOperation = m_messagesStore.getGroupForbiddenClientOperation();
+                    if((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Dismiss_Group) > 0) {
+                        return ErrorCode.ERROR_CODE_NOT_RIGHT;
+                    }
+                }
+
                 //send notify message first, then dismiss group
                 if (request.hasNotifyContent() && request.getNotifyContent().getType() > 0) {
                     sendGroupNotification(fromUser, groupInfo.getTargetId(), request.getToLineList(), request.getNotifyContent());

broker/src/main/java/io/moquette/imhandler/GetCommonGroupsHandler.java

@@ -0,0 +1,33 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package io.moquette.imhandler;
+
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.proto.ProtoConstants;
+import cn.wildfirechat.proto.WFCMessage;
+import io.moquette.spi.impl.Qos1PublishHandler;
+import io.netty.buffer.ByteBuf;
+import win.liyufan.im.IMTopic;
+
+import java.util.Set;
+
+import static cn.wildfirechat.common.ErrorCode.ERROR_CODE_SUCCESS;
+
+@Handler(IMTopic.GetCommonGroupsTopic)
+public class GetCommonGroupsHandler extends IMHandler<WFCMessage.IDBuf> {
+    @Override
+    public ErrorCode action(ByteBuf ackPayload, String clientID, String fromUser, ProtoConstants.RequestSourceType requestSourceType, WFCMessage.IDBuf request, Qos1PublishHandler.IMCallback callback) {
+        WFCMessage.PullUserResult.Builder resultBuilder = WFCMessage.PullUserResult.newBuilder();
+        Set<String> strings = m_messagesStore.getCommonGroupIds(fromUser, request.getId());
+        WFCMessage.IDListBuf idListBuf = WFCMessage.IDListBuf.newBuilder().addAllId(strings).build();
+        byte[] data = idListBuf.toByteArray();
+        ackPayload.ensureWritable(data.length).writeBytes(data);
+        return ERROR_CODE_SUCCESS;
+    }
+}

broker/src/main/java/io/moquette/imhandler/GetGroupInfoHandler.java

@@ -21,7 +21,7 @@ import java.util.List;
 public class GetGroupInfoHandler extends IMHandler<WFCMessage.PullUserRequest> {
     @Override
     public ErrorCode action(ByteBuf ackPayload, String clientID, String fromUser, ProtoConstants.RequestSourceType requestSourceType, WFCMessage.PullUserRequest request, Qos1PublishHandler.IMCallback callback) {
-            List<WFCMessage.GroupInfo> infos = m_messagesStore.getGroupInfos(request.getRequestList());
+            List<WFCMessage.GroupInfo> infos = m_messagesStore.getGroupInfos(request.getRequestList(), fromUser, requestSourceType == ProtoConstants.RequestSourceType.Request_From_Admin);
 
             WFCMessage.PullGroupInfoResult result = WFCMessage.PullGroupInfoResult.newBuilder().addAllInfo(infos).build();
             byte[] data = result.toByteArray();

broker/src/main/java/io/moquette/imhandler/GetMyGroupsHandler.java

@@ -0,0 +1,33 @@
+/*
+ * This file is part of the Wildfire Chat package.
+ * (c) Heavyrain2012 <heavyrain.lee@gmail.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+package io.moquette.imhandler;
+
+import cn.wildfirechat.common.ErrorCode;
+import cn.wildfirechat.proto.ProtoConstants;
+import cn.wildfirechat.proto.WFCMessage;
+import io.moquette.spi.impl.Qos1PublishHandler;
+import io.netty.buffer.ByteBuf;
+import win.liyufan.im.IMTopic;
+
+import java.util.Set;
+
+import static cn.wildfirechat.common.ErrorCode.ERROR_CODE_SUCCESS;
+
+@Handler(IMTopic.GetMyGroupsTopic)
+public class GetMyGroupsHandler extends IMHandler<Void> {
+    @Override
+    public ErrorCode action(ByteBuf ackPayload, String clientID, String fromUser, ProtoConstants.RequestSourceType requestSourceType, Void request, Qos1PublishHandler.IMCallback callback) {
+        WFCMessage.PullUserResult.Builder resultBuilder = WFCMessage.PullUserResult.newBuilder();
+        Set<String> strings = m_messagesStore.getUserGroupIds(fromUser);
+        WFCMessage.IDListBuf idListBuf = WFCMessage.IDListBuf.newBuilder().addAllId(strings).build();
+        byte[] data = idListBuf.toByteArray();
+        ackPayload.ensureWritable(data.length).writeBytes(data);
+        return ERROR_CODE_SUCCESS;
+    }
+}

broker/src/main/java/io/moquette/imhandler/GetQiniuUploadTokenHandler.java

@@ -26,6 +26,7 @@ public class GetQiniuUploadTokenHandler extends IMHandler<Byte> {
         String token;
 
         WFCMessage.GetUploadTokenResult.Builder resultBuilder = WFCMessage.GetUploadTokenResult.newBuilder();
+
         if (MediaServerConfig.USER_QINIU) {
             Auth auth = Auth.create(MediaServerConfig.QINIU_ACCESS_KEY, MediaServerConfig.QINIU_SECRET_KEY);
 

broker/src/main/java/io/moquette/imhandler/HandleFriendRequestHandler.java

@@ -10,12 +10,10 @@ package io.moquette.imhandler;
 
 import cn.wildfirechat.proto.ProtoConstants;
 import cn.wildfirechat.proto.WFCMessage;
-import io.moquette.persistence.MemorySessionStore;
 import io.moquette.spi.impl.Qos1PublishHandler;
 import io.netty.buffer.ByteBuf;
 import cn.wildfirechat.common.ErrorCode;
 import io.netty.util.internal.StringUtil;
-import win.liyufan.im.I18n;
 import win.liyufan.im.IMTopic;
 import win.liyufan.im.MessageShardingUtil;
 
@@ -43,17 +41,8 @@ public class HandleFriendRequestHandler extends IMHandler<WFCMessage.HandleFrien
                             saveAndPublish(request.getTargetUid(), null, builder.build(), requestSourceType);
                         }
 
-                        MemorySessionStore.Session session = m_sessionsStore.getSession(clientID);
-                        String language = "zh_CN";
-                        if (session != null && !StringUtil.isNullOrEmpty(session.getLanguage())) {
-                            language = session.getLanguage();
-                        }
                         WFCMessage.MessageContent.Builder contentBuilder = WFCMessage.MessageContent.newBuilder();
-                        if (m_messagesStore.isNewFriendWelcomeMessage()) {
-                            contentBuilder.setType(92);
-                        } else {
-                            contentBuilder.setType(90).setContent(I18n.getString(language, "Above_Greeting_Message"));
-                        }
+                        contentBuilder.setType(92);
 
                         builder = WFCMessage.Message.newBuilder();
                         builder.setFromUser(request.getTargetUid());
@@ -64,13 +53,7 @@ public class HandleFriendRequestHandler extends IMHandler<WFCMessage.HandleFrien
                         messageId = MessageShardingUtil.generateId();
                         builder.setMessageId(messageId);
                         saveAndPublish(request.getTargetUid(), null, builder.build(), requestSourceType);
-
-                        if (m_messagesStore.isNewFriendWelcomeMessage()) {
-                            contentBuilder.setType(93);
-                        } else {
-                            contentBuilder.setContent(I18n.getString(language, "Friend_Can_Start_Chat"));
-                        }
-
+                        contentBuilder.setType(93);
                         builder.setContent(contentBuilder);
                         messageId = MessageShardingUtil.generateId();
                         builder.setMessageId(messageId);

broker/src/main/java/io/moquette/imhandler/IMHandler.java

@@ -25,6 +25,7 @@ import io.netty.util.internal.StringUtil;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import cn.wildfirechat.common.ErrorCode;
+import win.liyufan.im.GsonUtil;
 import win.liyufan.im.RateLimiter;
 import win.liyufan.im.Utility;
 
@@ -39,6 +40,8 @@ import java.util.*;
 import static cn.wildfirechat.common.ErrorCode.ERROR_CODE_OVER_FREQUENCY;
 import static cn.wildfirechat.common.ErrorCode.ERROR_CODE_SUCCESS;
 import static io.moquette.BrokerConstants.CLIENT_REQUEST_RATE_LIMIT;
+import static win.liyufan.im.IMTopic.GetUserSettingTopic;
+import static win.liyufan.im.IMTopic.PullMessageTopic;
 
 /**
  * 请求处理接口<br>
@@ -139,7 +142,7 @@ abstract public class IMHandler<T> {
 //         }
 
         //json ?
-        return (T)(new Gson().fromJson(new String(bytes), dataCls));
+        return (T)(GsonUtil.gson.fromJson(new String(bytes), dataCls));
     }
 
     public static void init(IMessagesStore ms, ISessionsStore ss, MessagesPublisher p, ThreadPoolExecutorWrapper businessExecutor, Server server) {
@@ -162,18 +165,17 @@ abstract public class IMHandler<T> {
         mLimitCounter = new RateLimiter(5, clientRateLimit);
     }
 
+    boolean isNotLimitTopic(String topic) {
+        if (PullMessageTopic.equals(topic)
+            || GetUserSettingTopic.equals(topic)) {
+            return true;
+        }
+        return false;
+    }
 
     public ErrorCode preAction(String clientID, String fromUser, String topic, Qos1PublishHandler.IMCallback callback, ProtoConstants.RequestSourceType requestSourceType) {
         LOG.info("imHandler fromUser={}, clientId={}, topic={}", fromUser, clientID, topic);
-        if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User && !mLimitCounter.isGranted(clientID + fromUser + topic)) {
-            ByteBuf ackPayload = Unpooled.buffer();
-            ackPayload.ensureWritable(1).writeByte(ERROR_CODE_OVER_FREQUENCY.getCode());
-            try {
-                callback.onIMHandled(ERROR_CODE_OVER_FREQUENCY, ackPayload);
-            } catch (Exception e) {
-                e.printStackTrace();
-                Utility.printExecption(LOG, e);
-            }
+        if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User && !isNotLimitTopic(topic) && !mLimitCounter.isGranted(clientID + fromUser + topic)) {
             return ErrorCode.ERROR_CODE_OVER_FREQUENCY;
         }
         return ErrorCode.ERROR_CODE_SUCCESS;

broker/src/main/java/io/moquette/imhandler/KickoffGroupMember.java

@@ -29,6 +29,10 @@ public class KickoffGroupMember extends GroupHandler<WFCMessage.RemoveGroupMembe
             errorCode = ErrorCode.ERROR_CODE_NOT_EXIST;
         } else {
             boolean isAdmin = requestSourceType == ProtoConstants.RequestSourceType.Request_From_Admin;
+            if (!isAdmin && groupInfo.getType() == ProtoConstants.GroupType.GroupType_Organization) {
+                return ErrorCode.ERROR_CODE_NOT_RIGHT;
+            }
+
             boolean isAllow = isAdmin;
             if (!isAllow) {
                 if (groupInfo.getOwner() != null) {
@@ -58,6 +62,13 @@ public class KickoffGroupMember extends GroupHandler<WFCMessage.RemoveGroupMembe
                     return ErrorCode.ERROR_CODE_NOT_RIGHT;
                 }
 
+                if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User) {
+                    int forbiddenClientOperation = m_messagesStore.getGroupForbiddenClientOperation();
+                    if((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Kickoff_Group_Member) > 0) {
+                        return ErrorCode.ERROR_CODE_NOT_RIGHT;
+                    }
+                }
+
                 //send notify message first, then kickoff the member
                 if (request.hasNotifyContent() && request.getNotifyContent().getType() > 0) {
                     sendGroupNotification(fromUser, groupInfo.getTargetId(), request.getToLineList(), request.getNotifyContent());

broker/src/main/java/io/moquette/imhandler/ModifyGroupInfoHandler.java

@@ -31,6 +31,19 @@ public class ModifyGroupInfoHandler extends GroupHandler<WFCMessage.ModifyGroupI
             return ErrorCode.ERROR_CODE_NOT_RIGHT;
         }
 
+        if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User) {
+            int forbiddenClientOperation = m_messagesStore.getGroupForbiddenClientOperation();
+            if(request.getType() == Modify_Group_Mute) {
+                if((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Mute_Group) > 0) {
+                    return ErrorCode.ERROR_CODE_NOT_RIGHT;
+                }
+            } else {
+                if((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Modify_Group_Info) > 0) {
+                    return ErrorCode.ERROR_CODE_NOT_RIGHT;
+                }
+            }
+        }
+
         ErrorCode errorCode= m_messagesStore.modifyGroupInfo(fromUser, request.getGroupId(), request.getType(), request.getValue(), isAdmin);
         if (errorCode == ERROR_CODE_SUCCESS) {
             if(request.hasNotifyContent() && request.getNotifyContent().getType() > 0) {

broker/src/main/java/io/moquette/imhandler/QuitGroupHandler.java

@@ -31,7 +31,14 @@ public class QuitGroupHandler extends GroupHandler<WFCMessage.QuitGroupRequest>
             return ErrorCode.ERROR_CODE_NOT_RIGHT;
         }
 
-        ErrorCode errorCode = m_messagesStore.quitGroup(fromUser, request.getGroupId());
+        if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User) {
+            int forbiddenClientOperation = m_messagesStore.getGroupForbiddenClientOperation();
+            if((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Quit_Group) > 0) {
+                return ErrorCode.ERROR_CODE_NOT_RIGHT;
+            }
+        }
+
+        ErrorCode errorCode = m_messagesStore.quitGroup(fromUser, request.getGroupId(), isAdmin);
         if (errorCode == ErrorCode.ERROR_CODE_SUCCESS) {
             if (request.hasNotifyContent() && request.getNotifyContent().getType() > 0) {
                 sendGroupNotification(fromUser, request.getGroupId(), request.getToLineList(), request.getNotifyContent());

broker/src/main/java/io/moquette/imhandler/RecallMessageHandler.java

@@ -23,7 +23,7 @@ public class RecallMessageHandler extends IMHandler<WFCMessage.INT64Buf> {
     @Override
     public ErrorCode action(ByteBuf ackPayload, String clientID, String fromUser, ProtoConstants.RequestSourceType requestSourceType, WFCMessage.INT64Buf int64Buf, Qos1PublishHandler.IMCallback callback) {
         boolean isAdmin = requestSourceType == ProtoConstants.RequestSourceType.Request_From_Admin;
-        ErrorCode errorCode = m_messagesStore.recallMessage(int64Buf.getId(), fromUser, clientID, isAdmin);
+        ErrorCode errorCode = m_messagesStore.recallMessage(int64Buf.getId(), fromUser, clientID, isAdmin, ackPayload);
 
         if(errorCode != ErrorCode.ERROR_CODE_SUCCESS) {
             return errorCode;

broker/src/main/java/io/moquette/imhandler/RouteHandler.java

@@ -40,6 +40,11 @@ public class RouteHandler extends IMHandler<WFCMessage.RouteRequest> {
             return ErrorCode.ERROR_CODE_SECRECT_KEY_MISMATCH;
         }
 
+        if(request.getPlatform() != session.getPlatform() && m_messagesStore.existSignatures()) {
+            LOG.error("Session <{}, {}> platform is {} mismatch the request {}", session.getUsername(), session.getClientID(), session.getPlatform(), request.getPlatform());
+            return ErrorCode.ERROR_CODE_SECRECT_KEY_MISMATCH;
+        }
+
         String serverIp = mServer.getServerIp();
         String longPort = mServer.getLongPort();
         String shortPort = mServer.getShortPort();

broker/src/main/java/io/moquette/imhandler/SendMessageHandler.java

@@ -9,20 +9,20 @@
 package io.moquette.imhandler;
 
 import cn.wildfirechat.pojos.MessagePayload;
+import cn.wildfirechat.pojos.OutputClient;
 import cn.wildfirechat.proto.ProtoConstants;
 import cn.wildfirechat.proto.WFCMessage;
 import com.google.gson.Gson;
 import com.hazelcast.util.StringUtil;
 import io.moquette.BrokerConstants;
+import io.moquette.persistence.MemorySessionStore;
 import io.moquette.spi.impl.Qos1PublishHandler;
 import io.netty.buffer.ByteBuf;
 import cn.wildfirechat.common.ErrorCode;
 import io.netty.buffer.Unpooled;
-import win.liyufan.im.HttpUtils;
-import win.liyufan.im.IMTopic;
-import win.liyufan.im.MessageShardingUtil;
-import win.liyufan.im.Utility;
+import win.liyufan.im.*;
 
+import java.util.Collection;
 import java.util.HashSet;
 import java.util.Set;
 
@@ -32,7 +32,9 @@ import static cn.wildfirechat.common.ErrorCode.ERROR_CODE_SUCCESS;
 public class SendMessageHandler extends IMHandler<WFCMessage.Message> {
     private int mSensitiveType = 0;  //命中敏感词时，0 失败，1 吞掉， 2 敏感词替换成*。
     private String mForwardUrl = null;
+    private String mSensitiveMessageForwardUrl = null;
     private Set<Integer> mForwardMessageTypes = new HashSet<>();
+    private Set<Integer> mForwardExcludeMessageTypes = new HashSet<>();
     private String mMentionForwardUrl = null;
     private int mBlacklistStrategy = 0; //黑名单中时，0失败，1吞掉。
     private boolean mNoForwardAdminMessage = false;
@@ -47,14 +49,11 @@ public class SendMessageHandler extends IMHandler<WFCMessage.Message> {
         mForwardUrl = mServer.getConfig().getProperty(BrokerConstants.MESSAGE_Forward_Url);
         if (!StringUtil.isNullOrEmpty(mForwardUrl)) {
             String forwardTypes = mServer.getConfig().getProperty(BrokerConstants.MESSAGE_Forward_Types);
-            if (!StringUtil.isNullOrEmpty(forwardTypes)) {
-                String[] tss = forwardTypes.split(",");
-                for (String ts:tss) {
-                    mForwardMessageTypes.add(Integer.parseInt(ts.trim()));
-                }
-            }
+            parseTypes(mForwardMessageTypes, forwardTypes);
+            String excludeTypes = mServer.getConfig().getProperty(BrokerConstants.MESSAGE_Forward_Exclude_Types);
+            parseTypes(mForwardExcludeMessageTypes, excludeTypes);
         }
-
+        mSensitiveMessageForwardUrl = mServer.getConfig().getProperty(BrokerConstants.MESSAGE_Sensitive_Forward_Url);
         mMentionForwardUrl = mServer.getConfig().getProperty(BrokerConstants.MESSAGE_MentionMsg_Forward_Url);
 
         try {
@@ -97,6 +96,30 @@ public class SendMessageHandler extends IMHandler<WFCMessage.Message> {
 
     }
 
+    private void parseTypes(Collection<Integer> collection, String types) {
+        try {
+            if (!StringUtil.isNullOrEmpty(types)) {
+                String[] tss = types.split(",");
+                for (String ts:tss) {
+                    if(ts.contains("-")) {
+                        String[] ss = ts.split("-");
+                        if(ss.length == 2) {
+                            int begin = Integer.parseInt(ss[0]);
+                            int end = Integer.parseInt(ss[1]);
+                            for (int i = begin; i <= end ; i++) {
+                                collection.add(i);
+                            }
+                        }
+                    } else {
+                        collection.add(Integer.parseInt(ts.trim()));
+                    }
+                }
+            }
+        } catch (Exception e) {
+            e.printStackTrace();
+        }
+    }
+
     @Override
     public ErrorCode action(ByteBuf ackPayload, String clientID, String fromUser, ProtoConstants.RequestSourceType requestSourceType, WFCMessage.Message message, Qos1PublishHandler.IMCallback callback) {
         ErrorCode errorCode = ErrorCode.ERROR_CODE_SUCCESS;
@@ -159,6 +182,15 @@ public class SendMessageHandler extends IMHandler<WFCMessage.Message> {
                         }
                     }
                 }
+            } else if (message.getConversation().getType() == ProtoConstants.ConversationType.ConversationType_Group) {
+                if(m_messagesStore.getGroupInfo(message.getConversation().getTarget()) == null) {
+                    return ErrorCode.ERROR_CODE_NOT_EXIST;
+                }
+            } else if (message.getConversation().getType() == ProtoConstants.ConversationType.ConversationType_Channel) {
+                WFCMessage.ChannelInfo channelData = m_messagesStore.getChannelInfo(message.getConversation().getTarget());
+                if(channelData == null || channelData.getStatus() == ProtoConstants.ChannelState.Channel_State_Mask_Deleted) {
+                    return ErrorCode.ERROR_CODE_NOT_EXIST;
+                }
             }
 
             long timestamp = System.currentTimeMillis();
@@ -171,12 +203,20 @@ public class SendMessageHandler extends IMHandler<WFCMessage.Message> {
             }
             message = message.toBuilder().setFromUser(fromUser).setMessageId(messageId).setServerTimestamp(timestamp).build();
 
-            if (!StringUtil.isNullOrEmpty(mForwardUrl) && (mForwardMessageTypes.isEmpty() || mForwardMessageTypes.contains(message.getContent().getType())) && !(isAdmin && mNoForwardAdminMessage)) {
-                publisher.forwardMessage(message, mForwardUrl);
+            OutputClient outputClient = null;
+            if(m_messagesStore.isForwardMessageWithClientInfo() && requestSourceType == ProtoConstants.RequestSourceType.Request_From_User && !StringUtil.isNullOrEmpty(clientID)) {
+                MemorySessionStore.Session session = m_sessionsStore.getSession(clientID);
+                if(session != null && session.getUsername().equals(fromUser)) {
+                    outputClient = new OutputClient(session.getPlatform(), clientID);
+                }
+            }
+
+            if (!StringUtil.isNullOrEmpty(mForwardUrl) && (mForwardMessageTypes.isEmpty() || mForwardMessageTypes.contains(message.getContent().getType())) && !(isAdmin && mNoForwardAdminMessage) && !mForwardExcludeMessageTypes.contains(message.getContent().getType())) {
+                publisher.forwardMessage(message, mForwardUrl, outputClient);
             }
 
             if(!StringUtil.isNullOrEmpty(mMentionForwardUrl) && message.hasContent() && message.getContent().getMentionedType() != 0 && !(isAdmin && mNoForwardAdminMessage)) {
-                publisher.forwardMessage(message, mMentionForwardUrl);
+                publisher.forwardMessage(message, mMentionForwardUrl, outputClient);
             }
 
             if (!isAdmin) {
@@ -184,6 +224,9 @@ public class SendMessageHandler extends IMHandler<WFCMessage.Message> {
                     Set<String> matched = m_messagesStore.handleSensitiveWord(message.getContent().getSearchableContent());
                     if (matched != null && !matched.isEmpty()) {
                         m_messagesStore.storeSensitiveMessage(message);
+                        if (!StringUtil.isNullOrEmpty(mSensitiveMessageForwardUrl)) {
+                            publisher.forwardMessage(message, mSensitiveMessageForwardUrl, outputClient);
+                        }
                         if (mSensitiveType == 0) {
                             errorCode = ErrorCode.ERROR_CODE_SENSITIVE_MATCHED;
                         } else if (mSensitiveType == 1) {
@@ -205,7 +248,7 @@ public class SendMessageHandler extends IMHandler<WFCMessage.Message> {
                                 if(StringUtil.isNullOrEmpty(content)) {
                                     saveAndPublish(fromUser, clientID, finalMsg, requestSourceType);
                                 } else {
-                                    MessagePayload payload = new Gson().fromJson(content, MessagePayload.class);
+                                    MessagePayload payload = GsonUtil.gson.fromJson(content, MessagePayload.class);
                                     if (payload != null && payload.getType() > 0) {
                                         WFCMessage.Message newMsg = finalMsg.toBuilder().setContent(payload.toProtoMessageContent()).build();
                                         saveAndPublish(fromUser, clientID, newMsg, requestSourceType);

broker/src/main/java/io/moquette/imhandler/SetGroupManagerHandler.java

@@ -30,6 +30,13 @@ public class SetGroupManagerHandler extends GroupHandler<WFCMessage.SetGroupMana
             return ErrorCode.ERROR_CODE_NOT_RIGHT;
         }
 
+        if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User) {
+            int forbiddenClientOperation = m_messagesStore.getGroupForbiddenClientOperation();
+            if((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Set_Group_Manage) > 0) {
+                return ErrorCode.ERROR_CODE_NOT_RIGHT;
+            }
+        }
+
         ErrorCode errorCode = m_messagesStore.setGroupManager(fromUser, request.getGroupId(), request.getType(), request.getUserIdList(), isAdmin);
         if (errorCode == ERROR_CODE_SUCCESS) {
             if (request.hasNotifyContent() && request.getNotifyContent().getType() > 0) {

broker/src/main/java/io/moquette/imhandler/TransferGroupHandler.java

@@ -30,6 +30,13 @@ public class TransferGroupHandler extends GroupHandler<WFCMessage.TransferGroupR
             return ErrorCode.ERROR_CODE_NOT_RIGHT;
         }
 
+        if(requestSourceType == ProtoConstants.RequestSourceType.Request_From_User) {
+            int forbiddenClientOperation = m_messagesStore.getGroupForbiddenClientOperation();
+            if((forbiddenClientOperation & ProtoConstants.ForbiddenClientGroupOperationMask.Forbidden_Transfer_Group) > 0) {
+                return ErrorCode.ERROR_CODE_NOT_RIGHT;
+            }
+        }
+
         ErrorCode errorCode = m_messagesStore.transferGroup(fromUser, request.getGroupId(), request.getNewOwner(), isAdmin);
         if (errorCode == ERROR_CODE_SUCCESS) {
             if (request.hasNotifyContent() && request.getNotifyContent().getType() > 0) {

broker/src/main/java/io/moquette/imhandler/UploadDeviceTokenHandler.java

@@ -25,9 +25,11 @@ public class UploadDeviceTokenHandler extends IMHandler<WFCMessage.UploadDeviceT
             session.setPlatform(request.getPlatform());
             session.setAppName(request.getAppName());
             if (request.getPlatform() == ProtoConstants.Platform.Platform_iOS && request.getPushType() == 2) {
+                LOG.info("Set device token, userId {}", fromUser);
                 session.setVoipDeviceToken(request.getDeviceToken());
                 m_sessionsStore.updateSessionToken(session, true);
             } else {
+                LOG.info("Set device token, userId {}, platform {}, pushType {}", fromUser, session.getPlatform(), request.getPushType());
                 session.setDeviceToken(request.getDeviceToken());
                 session.setPushType(request.getPushType());
                 m_sessionsStore.updateSessionToken(session, false);

broker/src/main/java/io/moquette/interception/messages/InterceptConnectMessage.java

@@ -72,7 +72,7 @@ public class InterceptConnectMessage extends InterceptAbstractMessage {
     }
 
     public byte[] getPassword() {
-        return msg.payload().password();
+        return msg.payload().passwordInBytes();
     }
 
     public String getWillTopic() {

broker/src/main/java/io/moquette/persistence/DatabaseStore.java

@@ -39,6 +39,7 @@ import static cn.wildfirechat.common.IMExceptionEvent.EventType.RDBS_Exception;
 import static cn.wildfirechat.proto.ProtoConstants.PersistFlag.Transparent;
 import static io.moquette.server.Constants.MAX_MESSAGE_QUEUE;
 import static cn.wildfirechat.proto.ProtoConstants.SearchUserType.*;
+import static win.liyufan.im.UserSettingScope.kUserSettingPrivacySearchable;
 
 public class DatabaseStore {
     private static final Logger LOG = LoggerFactory.getLogger(DatabaseStore.class);
@@ -161,18 +162,12 @@ public class DatabaseStore {
         }
         return null;
     }
-    //searchType
-    // 0 模糊匹配displayName, 精确匹配账户名和电话号码
-    // 1 精确匹配账户名和电话号码
-    // 2 精确匹配账户名
-    // 3 精确匹配电话号码
-    List<WFCMessage.User> searchUserFromDB(String keyword, int searchType, int page) {
+
+    List<WFCMessage.User> searchUserByNameMobile(String keyword, int searchType) {
+        ArrayList<WFCMessage.User> out = new ArrayList<>();
         Connection connection = null;
         PreparedStatement statement = null;
         ResultSet rs = null;
-
-        ArrayList<WFCMessage.User> out = new ArrayList<>();
-
         try {
             connection = DBUtil.getConnection();
             String sql = "select `_uid`, `_name`" +
@@ -185,61 +180,34 @@ public class DatabaseStore {
                 ", `_company`" +
                 ", `_social`" +
                 ", `_extra`" +
-                ", `_dt` from t_user";
-            switch (searchType) {
-                case SearchUserType_Name_Mobile:
-                    sql += " where (`_name` = ? or `_mobile` = ?) ";
-                    break;
-                case SearchUserType_Name:
-                    sql += " where `_name` = ? ";
-                    break;
-                case SearchUserType_Mobile:
-                    sql += " where `_mobile` = ? ";
-                    break;
-                case SearchUserType_General:
-                default:
-                    sql += " where (`_display_name` like ? or `_name` = ? or `_mobile` = ?) ";
-                    break;
+                ", `_dt`, `_type` from t_user";
+
+            if(searchType == SearchUserType_Name) {
+                sql += " where `_name` = ? ";
+            } else if(searchType == SearchUserType_Mobile) {
+                sql += " where `_mobile` = ? ";
+            } else {
+                sql += " where (`_name` = ? or `_mobile` = ?) ";
             }
 
+            sql += " and `_type` <> 2 and `_deleted` = 0"; //can search normal user(0) and robot(1) and admin(100), can not search things
 
-            sql += " and _type <> 2"; //can search normal user(0) and robot(1) and admin(100), can not search things
-
-            if (searchType == SearchUserType_Name_Mobile || searchType == SearchUserType_Name || searchType == SearchUserType_Mobile) {
+            if(searchType == SearchUserType_Name || searchType == SearchUserType_Mobile) {
                 sql += " limit 1";
             } else {
-                sql += " limit 20";
-            }
-            
-            if (page > 0) {
-                sql += " offset " + page * 20;
+                sql += " limit 2";
             }
 
             statement = connection.prepareStatement(sql);
-            int index = 1;
-
-            switch (searchType) {
-                case SearchUserType_Name_Mobile:
-                    statement.setString(index++, keyword);
-                    statement.setString(index++, keyword);
-                    break;
-                case SearchUserType_Name:
-                case SearchUserType_Mobile:
-                    statement.setString(index++, keyword);
-                    break;
-                case SearchUserType_General:
-                default:
-                    statement.setString(index++, "%" + keyword + "%");
-                    statement.setString(index++, keyword);
-                    statement.setString(index++, keyword);
-                    break;
+            statement.setString(1, keyword);
+            if(searchType != SearchUserType_Name && searchType != SearchUserType_Mobile) {
+                statement.setString(2, keyword);
             }
 
-
             rs = statement.executeQuery();
             while (rs.next()) {
                 WFCMessage.User.Builder builder = WFCMessage.User.newBuilder();
-                index = 1;
+                int index = 1;
 
                 String value = rs.getString(index++);
                 value = (value == null ? "" : value);
@@ -287,6 +255,9 @@ public class DatabaseStore {
                 long longValue = rs.getLong(index++);
                 builder.setUpdateDt(longValue);
 
+                int type = rs.getInt(index++);
+                builder.setType( type);
+
                 WFCMessage.User user = builder.build();
 
                 out.add(user);
@@ -298,6 +269,171 @@ public class DatabaseStore {
         } finally {
             DBUtil.closeDB(connection, statement, rs);
         }
+
+        List<WFCMessage.User> filter = new ArrayList<>();
+        for (WFCMessage.User user : out) {
+            WFCMessage.UserSettingEntry userSettingEntry = Server.getServer().getStore().messagesStore().getUserSetting(user.getUid(), kUserSettingPrivacySearchable, null);
+            if(userSettingEntry != null) {
+                int value = 0;
+                try {
+                    value = StringUtil.isNullOrEmpty(userSettingEntry.getValue())?0:Integer.parseInt(userSettingEntry.getValue());
+                } catch (NumberFormatException e) {
+                    e.printStackTrace();
+                }
+                if((value & ProtoConstants.DisableSearchMask.DisableSearchNameMask) > 0) {
+                    if (searchType == SearchUserType_Name) {
+                        filter.add(user);
+                        continue;
+                    } else if(searchType != SearchUserType_Mobile) {
+                        if(keyword.equals(user.getName())) {
+                            if(!keyword.equals(user.getMobile())) {
+                                filter.add(user);
+                                continue;
+                            } else if((value & ProtoConstants.DisableSearchMask.DisableSearchMobileMask) > 0) {
+                                filter.add(user);
+                                continue;
+                            }
+                        }
+                    }
+                }
+
+                if ((value & ProtoConstants.DisableSearchMask.DisableSearchMobileMask) > 0) {
+                    if (searchType == SearchUserType_Mobile) {
+                        filter.add(user);
+                    } else if(searchType != SearchUserType_Name) {
+                        if(keyword.equals(user.getMobile()) && !keyword.equals(user.getName())) {
+                            filter.add(user);
+                        }
+                    }
+                }
+            }
+        }
+        out.removeAll(filter);
+
+        return out;
+    }
+
+    List<WFCMessage.User> searchUserByDisplayName(String keyword, int page, List<WFCMessage.User> nameOrIdMatched) {
+        Connection connection = null;
+        PreparedStatement statement = null;
+        ResultSet rs = null;
+
+        ArrayList<WFCMessage.User> out = new ArrayList<>();
+
+        try {
+            connection = DBUtil.getConnection();
+            String sql = "select u._uid, u._name, u._display_name, u._portrait, u._mobile, u._gender, u._email, u._address, u._company, u._social, u._extra, u._dt, u._type " +
+                " from t_user u left join (select _uid, _value from t_user_setting where _scope = 27) s on u._uid = s._uid " +
+                " where u._display_name like ? and u._type <> 2 and u._deleted = 0 ";
+            if(nameOrIdMatched.size() == 1) {
+                sql += " and u._uid <> ? and u._name <> ?";
+            } else if(nameOrIdMatched.size() == 2) {
+                sql += " and u._uid <> ? and u._uid <>? and u._name <> ? and u._name <>? ";
+            }
+            sql += " and (s._value is null or (s._value <> 1 and s._value <> 3 and s._value <> 5 and s._value <> 7)) limit ";
+
+            if(page == 0) {
+                sql += (20 - nameOrIdMatched.size());
+            } else {
+                sql += 20;
+            }
+
+            if (page > 0) {
+                sql += " offset " + (page * 20 - nameOrIdMatched.size());
+            }
+
+            statement = connection.prepareStatement(sql);
+            statement.setString(1, "%" + keyword + "%");
+            if(nameOrIdMatched.size() == 1) {
+                statement.setString(2, nameOrIdMatched.toArray(new WFCMessage.User[0])[0].getUid());
+                statement.setString(3, nameOrIdMatched.toArray(new WFCMessage.User[0])[0].getUid());
+            } else if(nameOrIdMatched.size() == 2) {
+                statement.setString(2, nameOrIdMatched.toArray(new WFCMessage.User[0])[0].getUid());
+                statement.setString(3, nameOrIdMatched.toArray(new WFCMessage.User[0])[1].getUid());
+                statement.setString(4, nameOrIdMatched.toArray(new WFCMessage.User[0])[0].getUid());
+                statement.setString(5, nameOrIdMatched.toArray(new WFCMessage.User[0])[1].getUid());
+            }
+
+            rs = statement.executeQuery();
+            while (rs.next()) {
+                WFCMessage.User.Builder builder = WFCMessage.User.newBuilder();
+                int index = 1;
+
+                String value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setUid(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setName(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setDisplayName(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setPortrait(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setMobile(value);
+
+                int gender = rs.getInt(index++);
+                builder.setGender(gender);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setEmail(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setAddress(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setCompany(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setSocial(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setExtra(value);
+
+                long longValue = rs.getLong(index++);
+                builder.setUpdateDt(longValue);
+
+                int type = rs.getInt(index++);
+                builder.setType(type);
+
+                WFCMessage.User user = builder.build();
+
+                out.add(user);
+            }
+        } catch (SQLException e) {
+            // TODO Auto-generated catch block
+            e.printStackTrace();
+            Utility.printExecption(LOG, e, RDBS_Exception);
+        } finally {
+            DBUtil.closeDB(connection, statement, rs);
+        }
+        return out;
+    }
+
+    List<WFCMessage.User> searchUserFromDB(String keyword, int searchType, int page) {
+        List<WFCMessage.User> nameOrIdMatched = searchUserByNameMobile(keyword, searchType);
+        List<WFCMessage.User> out = new ArrayList<>();
+        if(page == 0) {
+            out.addAll(nameOrIdMatched);
+        }
+
+        if(searchType == SearchUserType_General) {
+            List<WFCMessage.User> general = searchUserByDisplayName(keyword, page, nameOrIdMatched);
+            general.removeAll(nameOrIdMatched);
+            out.addAll(general);
+        }
         return out;
     }
 
@@ -815,7 +951,7 @@ public class DatabaseStore {
         long beforeUid = before[0];
         String sql = "select `_mid`, `_from`, `_type`, `_target`, `_line`, `_data`, `_dt`, `_to` from " + table +" where";
         if (conversation.getType() == ProtoConstants.ConversationType.ConversationType_Private) {
-            sql += " _type = ? and _line = ? and _mid < ? and ((_target = ?  and _from = ?) or (_target = ?  and _from = ?))";
+            sql += " _type = ? and _line = ? and _mid < ? and ((_target = ?  and _from = ?) or (_target = ?  and _from = ?)) and (_to = '' or _to = ?)";
         } else if (conversation.getType() == ProtoConstants.ConversationType.ConversationType_Channel && !user.equals(channelOwner)) {
             sql += " _type = ? and _line = ? and _mid < ? and _target = ? and ((_from = ? and (_to = '' or _to = ?)) or (_from = ?))";
         } else {
@@ -853,6 +989,7 @@ public class DatabaseStore {
                 statement.setString(index++, user);
                 statement.setString(index++, user);
                 statement.setString(index++, conversation.getTarget());
+                statement.setString(index++, user);
             } else if (conversation.getType() == ProtoConstants.ConversationType.ConversationType_Channel && !user.equals(channelOwner)) {
                 statement.setString(index++, channelOwner);
                 statement.setString(index++, user);
@@ -914,7 +1051,7 @@ public class DatabaseStore {
         return false;
     }
 
-    void persistUserMessage(final String userId, final long messageId, final long messageSeq, int type, String target, int line, boolean directing, final int messageContentType) {
+    void persistUserMessage(final String userId, final  String sender, final long messageId, final long messageSeq, int type, String target, int line, boolean directing, final int messageContentType) {
         mScheduler.execute(()->{
             Connection connection = null;
             PreparedStatement statement = null;
@@ -929,7 +1066,7 @@ public class DatabaseStore {
                 statement.setString(index++, userId);
                 statement.setLong(index++, messageSeq);
                 statement.setInt(index++, type);
-                statement.setString(index++, target);
+                statement.setString(index++, getPrivateChatUserMessageTarget(type, target, sender));
                 statement.setInt(index++, line);
                 statement.setInt(index++, directing ? 1 :0);
                 statement.setInt(index++, messageContentType);
@@ -945,6 +1082,14 @@ public class DatabaseStore {
         });
     }
 
+    String getPrivateChatUserMessageTarget(int conversationType, String target, String userId) {
+        if(conversationType == ProtoConstants.ConversationType.ConversationType_Private) {
+            return target.compareTo(userId) > 0 ? (userId + "|" + target) : (target + "|" + userId);
+        } else {
+            return target;
+        }
+    }
+
     void clearUserMessage(final String userId) {
         mScheduler.execute(()->{
             String tableName = getUserMessageTable(userId);
@@ -1493,12 +1638,6 @@ public class DatabaseStore {
                 if (!strEqual(session.getAppName(), request.getApp())) {
                     sql += " `_package_name` = ?,";
                 }
-                if (session.getPlatform() != request.getPlatform()) {
-                    sql += " `_platform` = ?,";
-                }
-                if (session.getPushType() != request.getPushType()) {
-                    sql += " `_push_type` = ?,";
-                }
                 if (!strEqual(session.getDeviceName(), request.getDeviceName())) {
                     sql += " `_device_name` = ?,";
                 }
@@ -1526,12 +1665,6 @@ public class DatabaseStore {
                 if (!strEqual(session.getAppName(), request.getApp())) {
                     statement.setString(index++, request.getApp());
                 }
-                if (session.getPlatform() != request.getPlatform()) {
-                    statement.setInt(index++, request.getPlatform());
-                }
-                if (session.getPushType() != request.getPushType()) {
-                    statement.setInt(index++, request.getPushType() >= 32 ? 0 : request.getPushType());
-                }
                 if (!strEqual(session.getDeviceName(), request.getDeviceName())) {
                     statement.setString(index++, request.getDeviceName());
                 }
@@ -1712,6 +1845,74 @@ public class DatabaseStore {
         return null;
     }
 
+    Set<String> getUserGroupIds(String userId, List<Integer> memberTypes) {
+        Connection connection = null;
+        PreparedStatement statement = null;
+        ResultSet rs = null;
+        try {
+            connection = DBUtil.getConnection();
+            String sql = "select _gid from t_group_member where _mid = ? ";
+            if(!memberTypes.isEmpty()) {
+                sql += " and _type in (";
+                for (int i = 0; i < memberTypes.size(); i++) {
+                    if(i > 0) {
+                        sql += ",";
+                    }
+                    sql += memberTypes.get(i);
+                }
+                sql += ")";
+            }
+            statement = connection.prepareStatement(sql);
+
+            statement.setString(1, userId);
+
+            rs = statement.executeQuery();
+            Set<String> out = new HashSet<>();
+            while (rs.next()) {
+                String uid = rs.getString(1);
+                out.add(uid);
+            }
+            return out;
+        } catch (SQLException e) {
+            // TODO Auto-generated catch block
+            e.printStackTrace();
+            Utility.printExecption(LOG, e, RDBS_Exception);
+        } finally {
+            DBUtil.closeDB(connection, statement, rs);
+        }
+        return null;
+    }
+
+
+    Set<String> getCommonGroupIds(String userId1, String userId2) {
+        Connection connection = null;
+        PreparedStatement statement = null;
+        ResultSet rs = null;
+        try {
+            connection = DBUtil.getConnection();
+            String sql = "select t1._gid from t_group_member t1, t_group_member t2 where t1._mid = ? and t2._mid = ? and t1._type <> 4 and t2._type <> 4 and t1._gid = t2._gid";
+            statement = connection.prepareStatement(sql);
+
+            statement.setString(1, userId1);
+            statement.setString(2, userId2);
+
+            rs = statement.executeQuery();
+            Set<String> out = new HashSet<>();
+            while (rs.next()) {
+                String uid = rs.getString(1);
+                out.add(uid);
+            }
+            return out;
+        } catch (SQLException e) {
+            // TODO Auto-generated catch block
+            e.printStackTrace();
+            Utility.printExecption(LOG, e, RDBS_Exception);
+        } finally {
+            DBUtil.closeDB(connection, statement, rs);
+        }
+        return null;
+    }
+
     void updateGroupMemberDt(final String groupId, final long dt) {
         Connection connection = null;
         PreparedStatement statement = null;
@@ -2385,59 +2586,55 @@ public class DatabaseStore {
     }
 
     void deleteUserStatus(String userId) {
-        mScheduler.execute(()->{
-            Connection connection = null;
-            PreparedStatement statement = null;
-            try {
-                connection = DBUtil.getConnection();
+        Connection connection = null;
+        PreparedStatement statement = null;
+        try {
+            connection = DBUtil.getConnection();
+            String sql = "delete from t_user_status where _uid = ?";
+            statement = connection.prepareStatement(sql);
+            statement.setString(1, userId);
+            int count = statement.executeUpdate();
+            LOG.info("Update rows {}", count);
+        } catch (SQLException e) {
+            // TODO Auto-generated catch block
+            e.printStackTrace();
+            Utility.printExecption(LOG, e, RDBS_Exception);
+        } finally {
+            DBUtil.closeDB(connection, statement);
+        }
+    }
+
+    void updateUserStatus(String userId, int status) {
+        Connection connection = null;
+        PreparedStatement statement = null;
+        try {
+            connection = DBUtil.getConnection();
+            if (status == 0) {
                 String sql = "delete from t_user_status where _uid = ?";
                 statement = connection.prepareStatement(sql);
                 statement.setString(1, userId);
                 int count = statement.executeUpdate();
                 LOG.info("Update rows {}", count);
-            } catch (SQLException e) {
-                // TODO Auto-generated catch block
-                e.printStackTrace();
-                Utility.printExecption(LOG, e, RDBS_Exception);
-            } finally {
-                DBUtil.closeDB(connection, statement);
+            } else {
+                String sql = "insert into t_user_status (`_uid`, `_status`, `_dt`) values(?,?,?) ON DUPLICATE KEY UPDATE `_status` = ?";
+
+                statement = connection.prepareStatement(sql);
+                int index = 1;
+                statement.setString(index++, userId);
+                statement.setInt(index++, status);
+                statement.setLong(index++,  System.currentTimeMillis());
+                statement.setInt(index++, status);
+
+                int count = statement.executeUpdate();
+                LOG.info("Update rows {}", count);
             }
-        });
-    }
-
-    void updateUserStatus(String userId, int status) {
-        mScheduler.execute(()->{
-            Connection connection = null;
-            PreparedStatement statement = null;
-            try {
-                connection = DBUtil.getConnection();
-                if (status == 0) {
-                    String sql = "delete from t_user_status where _uid = ?";
-                    statement = connection.prepareStatement(sql);
-                    statement.setString(1, userId);
-                    int count = statement.executeUpdate();
-                    LOG.info("Update rows {}", count);
-                } else {
-                    String sql = "insert into t_user_status (`_uid`, `_status`, `_dt`) values(?,?,?) ON DUPLICATE KEY UPDATE `_status` = ?";
-
-                    statement = connection.prepareStatement(sql);
-                    int index = 1;
-                    statement.setString(index++, userId);
-                    statement.setInt(index++, status);
-                    statement.setLong(index++,  System.currentTimeMillis());
-                    statement.setInt(index++, status);
-
-                    int count = statement.executeUpdate();
-                    LOG.info("Update rows {}", count);
-                }
-            } catch (SQLException e) {
-                // TODO Auto-generated catch block
-                e.printStackTrace();
-                Utility.printExecption(LOG, e, RDBS_Exception);
-            } finally {
-                DBUtil.closeDB(connection, statement);
-            }
-        });
+        } catch (SQLException e) {
+            // TODO Auto-generated catch block
+            e.printStackTrace();
+            Utility.printExecption(LOG, e, RDBS_Exception);
+        } finally {
+            DBUtil.closeDB(connection, statement);
+        }
     }
 
     int getGeneratedId() {
@@ -2618,6 +2815,103 @@ public class DatabaseStore {
         return null;
     }
 
+    List<WFCMessage.User> getAllUsers(int count, int offset) {
+        Connection connection = null;
+        PreparedStatement statement = null;
+        ResultSet rs = null;
+        List<WFCMessage.User> outList = new ArrayList<>();
+        try {
+            connection = DBUtil.getConnection();
+            String sql = "select `_uid`, `_name`" +
+                ", `_display_name`" +
+                ", `_portrait`" +
+                ", `_mobile`" +
+                ", `_gender`" +
+                ", `_email`" +
+                ", `_address`" +
+                ", `_company`" +
+                ", `_social`" +
+                ", `_extra`" +
+                ", `_type`" +
+                ", `_deleted`" +
+                ", `_dt` from t_user where `_deleted` = 0 order by _createTime limit ? offset ?";
+            statement = connection.prepareStatement(sql);
+
+            int index = 1;
+            statement.setInt(1, count);
+            statement.setInt(2, offset);
+
+
+            rs = statement.executeQuery();
+            while (rs.next()) {
+                WFCMessage.User.Builder builder = WFCMessage.User.newBuilder();
+                index = 1;
+                String value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setUid(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setName(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setDisplayName(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setPortrait(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setMobile(value);
+
+                int gender = rs.getInt(index++);
+                builder.setGender(gender);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setEmail(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setAddress(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setCompany(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setSocial(value);
+
+                value = rs.getString(index++);
+                value = (value == null ? "" : value);
+                builder.setExtra(value);
+
+                int type = rs.getInt(index++);
+                builder.setType(type);
+
+                int deleted = rs.getInt(index++);
+                builder.setDeleted(deleted);
+
+                long longValue = rs.getLong(index++);
+                if(longValue <= 0)
+                    longValue = 1;
+                builder.setUpdateDt(longValue);
+
+                outList.add(builder.build());
+            }
+        } catch (SQLException e) {
+            // TODO Auto-generated catch block
+            e.printStackTrace();
+            Utility.printExecption(LOG, e, RDBS_Exception);
+        } finally {
+            DBUtil.closeDB(connection, statement, rs);
+        }
+        return outList;
+    }
+
     Set<String> getAllEnds() {
         Connection connection = null;
         PreparedStatement statement = null;
@@ -2972,7 +3266,8 @@ public class DatabaseStore {
                     ", `_secret`" +
                     ", `_callback`" +
                     ", `_automatic`" +
-                    ", `_dt`) values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)" +
+                    ", `_menu`" +
+                    ", `_dt`) values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)" +
                     " ON DUPLICATE KEY UPDATE `_name`=?" +
                     ", `_portrait`=?" +
                     ", `_owner`=?" +
@@ -2982,12 +3277,20 @@ public class DatabaseStore {
                     ", `_secret`=?" +
                     ", `_callback`=?" +
                     ", `_automatic`=?" +
+                    ", `_menu`=?" +
                     ", `_dt`=?";
 
                 statement = connection.prepareStatement(sql);
 
-                int index = 1;
+                WFCMessage.ChannelMenuList.Builder builder = WFCMessage.ChannelMenuList.newBuilder();
+                if (!channelInfo.getMenuList().isEmpty()) {
+                    for (WFCMessage.ChannelMenu menuBtn:channelInfo.getMenuList()) {
+                        builder.addMenu(menuBtn);
+                    }
+                }
+                byte[] menuBytes = builder.build().toByteArray();
 
+                int index = 1;
                 statement.setString(index++, channelInfo.getTargetId());
                 statement.setString(index++, channelInfo.getName());
                 statement.setString(index++, channelInfo.getPortrait());
@@ -2998,6 +3301,7 @@ public class DatabaseStore {
                 statement.setString(index++, channelInfo.getSecret());
                 statement.setString(index++, channelInfo.getCallback());
                 statement.setInt(index++, channelInfo.getAutomatic());
+                statement.setBytes(index++, menuBytes);
                 statement.setLong(index++, channelInfo.getUpdateDt() == 0 ? System.currentTimeMillis() : channelInfo.getUpdateDt());
 
                 statement.setString(index++, channelInfo.getName());
@@ -3009,6 +3313,7 @@ public class DatabaseStore {
                 statement.setString(index++, channelInfo.getSecret());
                 statement.setString(index++, channelInfo.getCallback());
                 statement.setInt(index++, channelInfo.getAutomatic());
+                statement.setBytes(index++, menuBytes);
                 statement.setLong(index++, channelInfo.getUpdateDt() == 0 ? System.currentTimeMillis() : channelInfo.getUpdateDt());
 
                 int count = statement.executeUpdate();
@@ -3061,6 +3366,7 @@ public class DatabaseStore {
                 ", `_secret`" +
                 ", `_callback`" +
                 ", `_automatic`" +
+                ", `_menu`" +
                 ", `_dt` from t_channel  where `_cid` = ?";
 
             statement = connection.prepareStatement(sql);
@@ -3111,6 +3417,21 @@ public class DatabaseStore {
                 intValue = rs.getInt(index++);
                 builder.setAutomatic(intValue);
 
+                try {
+                    byte[] bytes = null;
+                    Blob blob = rs.getBlob(index++);
+                    if (blob != null) {
+                        bytes = toByteArray(blob.getBinaryStream());
+                    }
+
+                    WFCMessage.ChannelMenuList menuButtonList = WFCMessage.ChannelMenuList.parseFrom(bytes);
+                    if (menuButtonList.getMenuCount() > 0) {
+                        builder.addAllMenu(menuButtonList.getMenuList());
+                    }
+                } catch (IOException e) {
+                    e.printStackTrace();
+                }
+
                 long longValue = rs.getLong(index++);
                 builder.setUpdateDt(longValue);
 
@@ -3289,47 +3610,45 @@ public class DatabaseStore {
     }
 
     void updateChannelListener(final String channelId, final String listener, final boolean listen) {
-        LOG.info("Database remove channel {}", channelId);
-        mScheduler.execute(()->{
-            Connection connection = null;
-            PreparedStatement statement = null;
-            try {
-                connection = DBUtil.getConnection();
-                String sql;
-                if (listen) {
-                    sql = "insert into t_channel_listener (`_cid`" +
-                        ", `_mid`" +
-                        ", `_dt`) values(?, ?, ?)" +
-                        " ON DUPLICATE KEY UPDATE " +
-                        "`_dt` = ?";
-                    statement = connection.prepareStatement(sql);
-                    int index = 1;
-                    statement.setString(index++, channelId);
-                    statement.setString(index++, listener);
-                    statement.setLong(index++, System.currentTimeMillis());
-                    statement.setLong(index++, System.currentTimeMillis());
+        LOG.info("updateChannelListener channel {}, listener {}, listen {}", channelId, listener, listen);
+        Connection connection = null;
+        PreparedStatement statement = null;
+        try {
+            connection = DBUtil.getConnection();
+            String sql;
+            if (listen) {
+                sql = "insert into t_channel_listener (`_cid`" +
+                    ", `_mid`" +
+                    ", `_dt`) values(?, ?, ?)" +
+                    " ON DUPLICATE KEY UPDATE " +
+                    "`_dt` = ?";
+                statement = connection.prepareStatement(sql);
+                int index = 1;
+                statement.setString(index++, channelId);
+                statement.setString(index++, listener);
+                statement.setLong(index++, System.currentTimeMillis());
+                statement.setLong(index++, System.currentTimeMillis());
 
-                    int count = statement.executeUpdate();
-                    LOG.info("Update rows {}", count);
-                } else {
-                    sql = "delete from t_channel_listener where `_cid`=? and `_mid`=?";
+                int count = statement.executeUpdate();
+                LOG.info("Update rows {}", count);
+            } else {
+                sql = "delete from t_channel_listener where `_cid`=? and `_mid`=?";
 
-                    statement = connection.prepareStatement(sql);
-                    int index = 1;
-                    statement.setString(index++, channelId);
-                    statement.setString(index++, listener);
+                statement = connection.prepareStatement(sql);
+                int index = 1;
+                statement.setString(index++, channelId);
+                statement.setString(index++, listener);
 
-                    int count = statement.executeUpdate();
-                    LOG.info("Update rows {}", count);
-                }
-            } catch (SQLException e) {
-                // TODO Auto-generated catch block
-                e.printStackTrace();
-                Utility.printExecption(LOG, e, RDBS_Exception);
-            } finally {
-                DBUtil.closeDB(connection, statement);
+                int count = statement.executeUpdate();
+                LOG.info("Update rows {}", count);
             }
-        });
+        } catch (SQLException e) {
+            // TODO Auto-generated catch block
+            e.printStackTrace();
+            Utility.printExecption(LOG, e, RDBS_Exception);
+        } finally {
+            DBUtil.closeDB(connection, statement);
+        }
     }
 
     List<String> getUserChannels(String userId) {

broker/src/main/java/io/moquette/persistence/MemorySessionStore.java

@@ -415,7 +415,7 @@ public class MemorySessionStore implements ISessionsStore {
                                 remove = true;
                             }
                         } else if(platform == ProtoConstants.Platform.Platform_OSX || platform == ProtoConstants.Platform.Platform_Windows || platform == ProtoConstants.Platform.Platform_LINUX) {
-                            if (s.getPlatform() == ProtoConstants.Platform.Platform_OSX || s.getPlatform() == ProtoConstants.Platform.Platform_Windows || platform == ProtoConstants.Platform.Platform_LINUX) {
+                            if (s.getPlatform() == ProtoConstants.Platform.Platform_OSX || s.getPlatform() == ProtoConstants.Platform.Platform_Windows || s.getPlatform() == ProtoConstants.Platform.Platform_LINUX) {
                                 remove = true;
                             }
                         } else {

broker/src/main/java/io/moquette/server/Server.java

@@ -232,18 +232,6 @@ public class Server {
             handlers = Collections.emptyList();
         }
         DBUtil.init(config);
-        String strKey = "0x00,0x11,0x22,0x33,0x44,0x55,0x66,0x77,0x78,0x79,0x7A,0x7B,0x7C,0x7D,0x7E,0x7F";
-        String[] strs = strKey.split(",");
-        if(strs.length != 16) {
-            LOG.error("Key error, it length should be 16");
-        }
-        byte[] keys = new byte[16];
-        for (int i = 0; i < 16; i++) {
-            keys[i] = (byte)(Integer.parseInt(strs[i].replace("0X", "").replace("0x", ""), 16));
-        }
-
-
-        AES.init(keys);
 
         LOG.info("Starting Moquette Server. MQTT message interceptors={}", getInterceptorIds(handlers));
 
@@ -369,9 +357,6 @@ public class Server {
     private String getServerIp(IConfig config) {
         String serverIp = config.getProperty(BrokerConstants.SERVER_IP_PROPERTY_NAME);
         if (serverIp == null || serverIp.equals("0.0.0.0")) {
-            String warning = "警告：无效配置，server.ip不能为空，需要配置服务的公网IP地址或者域名才能在公网使用！！！\nIM服务将使用本地地址，只能在局域网内使用！";
-            System.out.println(warning);
-            LOG.error(warning);
             serverIp = Utility.getLocalAddress().getHostAddress();
         }
         return serverIp;
@@ -381,7 +366,7 @@ public class Server {
         LOG.info("Configuring embedded Hazelcast instance");
 
         serverIp = getServerIp(config);
-        String hzConfigPath = "config/hazelcast.xml";
+        String hzConfigPath = System.getProperty("hazelcast.configuration", "config/hazelcast.xml");
         if (hzConfigPath != null) {
             boolean isHzConfigOnClasspath = this.getClass().getClassLoader().getResource(hzConfigPath) != null;
             Config hzconfig = isHzConfigOnClasspath
@@ -396,9 +381,13 @@ public class Server {
 
         longPort = config.getProperty(BrokerConstants.PORT_PROPERTY_NAME);
         shortPort = config.getProperty(BrokerConstants.HTTP_SERVER_PORT);
+        boolean aes256 = Boolean.parseBoolean(config.getProperty(ENCRYPT_AES256, "false"));
 
         MessageShardingUtil.setNodeId(1);
         Tokenor.setKey(config.getProperty(BrokerConstants.TOKEN_SECRET_KEY));
+
+        AES.useAes256(aes256);
+
         String expirTimeStr = config.getProperty(TOKEN_EXPIRE_TIME);
         if (!StringUtil.isNullOrEmpty(expirTimeStr)) {
             try {

broker/src/main/java/io/moquette/server/netty/NettyAcceptor.java

@@ -160,7 +160,7 @@ public class NettyAcceptor implements ServerAcceptor {
                                              IConfig props) throws IOException {
         LOG.info("Configuring TCP MQTT transport");
         final MoquetteIdleTimeoutHandler timeoutHandler = new MoquetteIdleTimeoutHandler();
-        String host = props.getProperty(BrokerConstants.HOST_PROPERTY_NAME);
+        String host = props.getProperty(BrokerConstants.HOST_PROPERTY_NAME, "0.0.0.0");
         String tcpPortProp = props.getProperty(PORT_PROPERTY_NAME, DISABLED_PORT_BIND);
         if (DISABLED_PORT_BIND.equals(tcpPortProp)) {
             LOG.info("Property {} has been set to {}. TCP MQTT will be disabled", BrokerConstants.PORT_PROPERTY_NAME,

broker/src/main/java/io/moquette/spi/IMessagesStore.java

@@ -110,15 +110,17 @@ public interface IMessagesStore {
     WFCMessage.GroupInfo createGroup(String operator, WFCMessage.GroupInfo groupInfo, List<WFCMessage.GroupMember> memberList, String memberExtra, boolean isAdmin);
     ErrorCode addGroupMembers(String operator, boolean isAdmin, String groupId, List<WFCMessage.GroupMember> memberList, String extra);
     ErrorCode kickoffGroupMembers(String operator, boolean isAdmin, String groupId, List<String> memberList);
-    ErrorCode quitGroup(String operator, String groupId);
+    ErrorCode quitGroup(String operator, String groupId, boolean admin);
     void clearUserGroups(String userId);
     ErrorCode dismissGroup(String operator, String groupId, boolean isAdmin);
     ErrorCode modifyGroupInfo(String operator, String groupId, int modifyType, String value, boolean isAdmin);
     ErrorCode modifyGroupMemberAlias(String operator, String groupId, String alias, String memberId, boolean isAdmin);
     ErrorCode modifyGroupMemberExtra(String operator, String groupId, String extra, String memberId, boolean isAdmin);
-    List<WFCMessage.GroupInfo> getGroupInfos(List<WFCMessage.UserRequest> requests);
+    List<WFCMessage.GroupInfo> getGroupInfos(List<WFCMessage.UserRequest> requests, String fromUser, boolean isAdmin);
     WFCMessage.GroupInfo getGroupInfo(String groupId);
     Set<String> getUserGroupIds(String userId);
+    Set<String> getUserGroupIds(String userId, List<Integer> types);
+    Set<String> getCommonGroupIds(String userId1, String userId2);
     ErrorCode getGroupMembers(String fromUser, String groupId, long maxDt, List<WFCMessage.GroupMember> members);
     WFCMessage.GroupMember getGroupMember(String groupId, String memberId);
     ErrorCode transferGroup(String operator, String groupId, String newOwner, boolean isAdmin);
@@ -127,7 +129,7 @@ public interface IMessagesStore {
     ErrorCode canSendMessageInGroup(String member, String groupId);
     Set<String> getGroupManagers(String groupId, boolean includeOwner);
 
-    ErrorCode recallMessage(long messageUid, String operatorId, String clientId, boolean isAdmin);
+    ErrorCode recallMessage(long messageUid, String operatorId, String clientId, boolean isAdmin, ByteBuf ackPayload);
     ErrorCode recallCastMessage(long messageUid, String operatorId);
 
     void clearUserMessages(String userId);
@@ -157,6 +159,8 @@ public interface IMessagesStore {
     WFCMessage.User getUserInfoByMobile(String mobile);
     List<WFCMessage.User> searchUser(String keyword, int searchType, int page);
 
+    List<WFCMessage.User> getUserInfoList(int count, int offset);
+
     boolean updateSystemSetting(int id, String value, String desc);
     SystemSettingPojo getSystemSetting(int id);
     void createChatroom(String chatroomId, WFCMessage.ChatroomInfo chatroomInfo);
@@ -179,7 +183,6 @@ public interface IMessagesStore {
 
     ErrorCode saveAddFriendRequest(String userId, WFCMessage.AddFriendRequest request, long[] head, boolean isAdmin);
     ErrorCode handleFriendRequest(String userId, WFCMessage.HandleFriendRequest request, WFCMessage.Message.Builder msgBuilder, long[] heads, boolean isAdmin);
-    boolean isNewFriendWelcomeMessage();
     ErrorCode deleteFriend(String userId, String friendUid, long[] head);
     ErrorCode blackUserRequest(String fromUser, String targetUserId, int status, long[] head);
     FriendData getFriendData(String fromUser, String targetUserId);
@@ -216,6 +219,7 @@ public interface IMessagesStore {
     ErrorCode transferChannel(String operator, String channelId, String newOwner);
     ErrorCode destroyChannel(String operator, String channelId, boolean isAdmin);
     List<WFCMessage.ChannelInfo> searchChannel(String keyword, boolean buzzy, int page);
+    List<String> getListenedChannels(String userId);
     ErrorCode listenChannel(String operator, String channelId, boolean listen);
     WFCMessage.ChannelInfo getChannelInfo(String channelId);
     boolean canSendMessageInChannel(String user, String channelId);
@@ -233,6 +237,16 @@ public interface IMessagesStore {
     boolean isAllowRobotCustomGroupNotification();
     int getVisibleQuitKickoffNotification();
 
+    int getGroupForbiddenClientOperation();
+
+    boolean isForwardMessageWithClientInfo();
+    boolean isRobotCallbackWithClientInfo();
+    boolean isChannelCallbackWithClientInfo();
+
+    boolean isRobotAutoAcceptFriendRequest();
+
+    long getPushExpiredTimes();
+
     List<Integer> getClientForbiddenSendTypes();
     List<Integer> getBlackListExceptionTypes();
     List<Integer> getGroupMuteExceptionTypes();
@@ -245,6 +259,9 @@ public interface IMessagesStore {
 
     //使用了数据库，会比较慢，仅能用户用户/群组等id的生成
     String getShortUUID();
+
+    boolean checkSignature(String signature);
+    boolean existSignatures();
     /**
      * Used to initialize all persistent store structures
      */

broker/src/main/java/io/moquette/spi/impl/MessagesPublisher.java

@@ -16,6 +16,7 @@
 
 package io.moquette.spi.impl;
 
+import cn.wildfirechat.pojos.OutputClient;
 import cn.wildfirechat.pojos.OutputMessageData;
 import cn.wildfirechat.proto.ProtoConstants;
 import cn.wildfirechat.proto.WFCMessage;
@@ -34,13 +35,11 @@ import io.moquette.spi.ISessionsStore;
 import io.netty.buffer.ByteBuf;
 import io.netty.buffer.Unpooled;
 import io.netty.handler.codec.mqtt.*;
-import win.liyufan.im.HttpUtils;
-import win.liyufan.im.IMTopic;
+import win.liyufan.im.*;
 
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-import win.liyufan.im.Utility;
 
 import java.util.*;
 import java.util.concurrent.ConcurrentHashMap;
@@ -166,6 +165,9 @@ public class MessagesPublisher {
         }
 
         WFCMessage.Message message = null;
+        String senderName = null;
+        String senderPortrait = null;
+        boolean sendNameLoaded = false;
         for (String user : receivers) {
             if (!user.equals(sender)) {
                 WFCMessage.User userInfo = m_messagesStore.getUserInfo(user);
@@ -175,10 +177,19 @@ public class MessagesPublisher {
                         if (message == null) {
                             message = m_messagesStore.getMessage(messageId);
                         }
+                        OutputClient outputClient = null;
+                        if(m_messagesStore.isRobotCallbackWithClientInfo() && !StringUtil.isNullOrEmpty(exceptClientId)) {
+                            Session session = m_sessionsStore.getSession(exceptClientId);
+                            if(session != null && session.getUsername().equals(sender)) {
+                                outputClient = new OutputClient(session.getPlatform(), exceptClientId);
+                            }
+                        }
+
                         final WFCMessage.Message finalMsg = message;
+                        OutputClient finalOutputClient = outputClient;
                         Server.getServer().getCallbackScheduler().execute(() -> {
                             try {
-                                HttpUtils.httpJsonPost(robot.getCallback(), new Gson().toJson(OutputMessageData.fromProtoMessage(finalMsg), OutputMessageData.class), HttpUtils.HttpPostType.POST_TYPE_Robot_Message_Callback);
+                                HttpUtils.httpJsonPost(robot.getCallback(), GsonUtil.gson.toJson(OutputMessageData.fromProtoMessage(finalMsg, finalOutputClient), OutputMessageData.class), HttpUtils.HttpPostType.POST_TYPE_Robot_Message_Callback);
                             } catch (Exception e) {
                                 e.printStackTrace();
                                 Utility.printExecption(LOG, e, EVENT_CALLBACK_Exception);
@@ -196,10 +207,10 @@ public class MessagesPublisher {
             }
 
             Collection<Session> sessions = m_sessionsStore.sessionForUser(user);
-            String senderName = null;
-            String targetName = null;
-            boolean nameLoaded = false;
 
+            String targetName = null;
+            String targetPortrait = null;
+            boolean targetNameLoaded = false;
 
             Collection<String> targetClients = null;
             if (pullType == ProtoConstants.PullType.Pull_ChatRoom) {
@@ -224,7 +235,7 @@ public class MessagesPublisher {
 
             for (Session targetSession : sessions) {
                 //超过7天不活跃的用户忽略
-                if(System.currentTimeMillis() - targetSession.getLastActiveTime() > 7 * 24 * 60 * 60 * 1000) {
+                if(System.currentTimeMillis() - targetSession.getLastActiveTime() > m_messagesStore.getPushExpiredTimes()) {
                     continue;
                 }
 
@@ -375,10 +386,22 @@ public class MessagesPublisher {
 
                     boolean isHiddenDetail = m_messagesStore.getUserPushHiddenDetail(user);
 
-                    if(!nameLoaded) {
-                        senderName = getUserDisplayName(sender, conversationType == ProtoConstants.ConversationType.ConversationType_Group ? target : null);
-                        targetName = getTargetName(target, conversationType);
-                        nameLoaded = true;
+                    if(!sendNameLoaded) {
+                        List<String> list = new ArrayList<>();
+                        senderName = getUserDisplayName(sender, conversationType == ProtoConstants.ConversationType.ConversationType_Group ? target : null, list);
+                        if(!list.isEmpty()) {
+                            senderPortrait = list.get(0);
+                        }
+                        sendNameLoaded = true;
+                    }
+
+                    if(!targetNameLoaded) {
+                        List<String> list = new ArrayList<>();
+                        targetName = getTargetName(target, conversationType, user, list);
+                        if(!list.isEmpty()) {
+                            targetPortrait = list.get(0);
+                        }
+                        targetNameLoaded = true;
                     }
 
                     String name = senderName;
@@ -388,7 +411,7 @@ public class MessagesPublisher {
                             name = fd.getAlias();
                         }
                     }
-                    this.messageSender.sendPush(sender, conversationType, target, line, messageId, targetSession.getClientID(), pushContent, pushData, messageContentType, serverTime, name, targetName, targetSession.getUnReceivedMsgs(), curMentionType, isHiddenDetail, targetSession.getLanguage());
+                    this.messageSender.sendPush(sender, conversationType, target, line, messageId, targetSession.getClientID(), pushContent, pushData, messageContentType, serverTime, name, senderPortrait, targetName, targetPortrait, targetSession.getUnReceivedMsgs(), curMentionType, isHiddenDetail, targetSession.getLanguage());
                 }
 
             }
@@ -450,10 +473,13 @@ public class MessagesPublisher {
         }
     }
 
-    String getUserDisplayName(String userId, String groupId) {
+    String getUserDisplayName(String userId, String groupId, List<String> portraitList) {
         WFCMessage.User user = m_messagesStore.getUserInfo(userId);
         String userName = null;
         if(user != null) {
+            if(portraitList != null && !StringUtil.isNullOrEmpty(user.getPortrait())) {
+                portraitList.add(user.getPortrait());
+            }
             userName = user.getDisplayName();
         }
         if (!StringUtil.isNullOrEmpty(groupId)) {
@@ -465,17 +491,28 @@ public class MessagesPublisher {
         return userName;
     }
 
-    String getTargetName(String targetId, int cnvType) {
+    String getTargetName(String targetId, int cnvType, String fromUser, List<String> portraitList) {
         if (cnvType == ProtoConstants.ConversationType.ConversationType_Private) {
-            return getUserDisplayName(targetId, null);
+            return getUserDisplayName(targetId, null, portraitList);
         } else if(cnvType == ProtoConstants.ConversationType.ConversationType_Group) {
             WFCMessage.GroupInfo group = m_messagesStore.getGroupInfo(targetId);
             if(group != null) {
+                if(!StringUtil.isNullOrEmpty(group.getPortrait())) {
+                    portraitList.add(group.getPortrait());
+                }
+
+                WFCMessage.UserSettingEntry entry = m_messagesStore.getUserSetting(fromUser, UserSettingScope.kUserSettingGroupRemark, targetId);
+                if(entry != null && !StringUtil.isNullOrEmpty(entry.getValue())) {
+                    return entry.getValue();
+                }
                 return group.getName();
             }
         } else if(cnvType == ProtoConstants.ConversationType.ConversationType_Channel) {
             WFCMessage.ChannelInfo channelInfo = m_messagesStore.getChannelInfo(targetId);
             if (channelInfo != null) {
+                if(!StringUtil.isNullOrEmpty(channelInfo.getPortrait())) {
+                    portraitList.add(channelInfo.getPortrait());
+                }
                 return channelInfo.getName();
             }
         }
@@ -594,10 +631,19 @@ public class MessagesPublisher {
     public void publish2Receivers(WFCMessage.Message message, Set<String> receivers, String exceptClientId, int pullType) {
         if (message.getConversation().getType() == ProtoConstants.ConversationType.ConversationType_Channel) {
             WFCMessage.ChannelInfo channelInfo = m_messagesStore.getChannelInfo(message.getConversation().getTarget());
-            if (channelInfo != null && !StringUtil.isNullOrEmpty(channelInfo.getCallback())) {
+            if (channelInfo != null && !StringUtil.isNullOrEmpty(channelInfo.getCallback()) && channelInfo.getAutomatic() == 1 && !message.getFromUser().equals(channelInfo.getOwner())) {
+                OutputClient outputClient = null;
+                if(m_messagesStore.isChannelCallbackWithClientInfo() && !StringUtil.isNullOrEmpty(exceptClientId)) {
+                    Session session = m_sessionsStore.getSession(exceptClientId);
+                    if(session != null && session.getUsername().equals(message.getFromUser())) {
+                        outputClient = new OutputClient(session.getPlatform(), exceptClientId);
+                    }
+                }
+
+                OutputClient finalOutputClient = outputClient;
                 Server.getServer().getCallbackScheduler().execute(() -> {
                     try {
-                        HttpUtils.httpJsonPost(channelInfo.getCallback() + "/message", new Gson().toJson(OutputMessageData.fromProtoMessage(message), OutputMessageData.class), HttpUtils.HttpPostType.POST_TYPE_Channel_Message_Callback);
+                        HttpUtils.httpJsonPost(channelInfo.getCallback() + "/message", GsonUtil.gson.toJson(OutputMessageData.fromProtoMessage(message, finalOutputClient), OutputMessageData.class), HttpUtils.HttpPostType.POST_TYPE_Channel_Message_Callback);
                     } catch (Exception e) {
                         e.printStackTrace();
                         Utility.printExecption(LOG, e, EVENT_CALLBACK_Exception);
@@ -632,6 +678,8 @@ public class MessagesPublisher {
                 pushContent = "[文件]";
             } else if(type == ProtoConstants.ContentType.Sticker) {
                 pushContent = "[表情]";
+            } else if (type == ProtoConstants.ContentType.Name_Card) {
+                pushContent = "[名片]";
             }
         }
 
@@ -647,10 +695,10 @@ public class MessagesPublisher {
 
     }
 
-    public void forwardMessage(final WFCMessage.Message message, String forwardUrl) {
+    public void forwardMessage(final WFCMessage.Message message, String forwardUrl, OutputClient outputClient) {
         Server.getServer().getCallbackScheduler().execute(() -> {
             try {
-                HttpUtils.httpJsonPost(forwardUrl, new Gson().toJson(OutputMessageData.fromProtoMessage(message), OutputMessageData.class), HttpUtils.HttpPostType.POST_TYPE_Forward_Message_Callback);
+                HttpUtils.httpJsonPost(forwardUrl, GsonUtil.gson.toJson(OutputMessageData.fromProtoMessage(message, outputClient), OutputMessageData.class), HttpUtils.HttpPostType.POST_TYPE_Forward_Message_Callback);
             } catch (Exception e) {
                 e.printStackTrace();
                 Utility.printExecption(LOG, e, EVENT_CALLBACK_Exception);
@@ -660,7 +708,7 @@ public class MessagesPublisher {
 
     public void forwardMessageWithCallback(final WFCMessage.Message message, String forwardUrl, HttpUtils.HttpCallback callback) {
         try {
-            HttpUtils.httpJsonPost(forwardUrl, new Gson().toJson(OutputMessageData.fromProtoMessage(message), OutputMessageData.class), new HttpUtils.HttpCallback() {
+            HttpUtils.httpJsonPost(forwardUrl, GsonUtil.gson.toJson(OutputMessageData.fromProtoMessage(message), OutputMessageData.class), new HttpUtils.HttpCallback() {
                 @Override
                 public void onSuccess(String content) {
                     Server.getServer().getImBusinessScheduler().execute(()->{
@@ -682,13 +730,13 @@ public class MessagesPublisher {
     }
 
     public void notifyChannelListenStatusChanged(WFCMessage.ChannelInfo channelInfo, String user, boolean listen) {
-        if (channelInfo == null || StringUtil.isNullOrEmpty(channelInfo.getCallback())) {
+        if (channelInfo == null || StringUtil.isNullOrEmpty(channelInfo.getCallback()) || channelInfo.getAutomatic() == 0) {
             return;
         }
 
         Server.getServer().getCallbackScheduler().execute(() -> {
             try {
-                HttpUtils.httpJsonPost(channelInfo.getCallback() + "/subscribe", new Gson().toJson(new OutputNotifyChannelSubscribeStatus(user, channelInfo.getTargetId(), listen), OutputNotifyChannelSubscribeStatus.class), HttpUtils.HttpPostType.POST_TYPE_Channel_Subscriber_Event_Callback);
+                HttpUtils.httpJsonPost(channelInfo.getCallback() + "/subscribe", GsonUtil.gson.toJson(new OutputNotifyChannelSubscribeStatus(user, channelInfo.getTargetId(), listen), OutputNotifyChannelSubscribeStatus.class), HttpUtils.HttpPostType.POST_TYPE_Channel_Subscriber_Event_Callback);
             } catch (Exception e) {
                 e.printStackTrace();
                 Utility.printExecption(LOG, e, EVENT_CALLBACK_Exception);

broker/src/main/java/io/moquette/spi/impl/PersistentQueueMessageSender.java

@@ -37,9 +37,9 @@ class PersistentQueueMessageSender {
         this.connectionDescriptorStore = connectionDescriptorStore;
     }
 
-    void sendPush(String sender, int conversationType, String target, int line, long messageId, String deviceId, String pushContent, String pushData, int messageContentType, long serverTime, String senderName, String targetName, int unReceivedMsg, int mentionType, boolean isHiddenDetail, String language) {
+    void sendPush(String sender, int conversationType, String target, int line, long messageId, String deviceId, String pushContent, String pushData, int messageContentType, long serverTime, String senderName, String senderPortrait, String targetName, String targetPortrait, int unReceivedMsg, int mentionType, boolean isHiddenDetail, String language) {
         LOG.info("Send push to {}, message from {}", deviceId, sender);
-        PushMessage pushMessage = new PushMessage(sender, conversationType, target, line, messageContentType, serverTime, senderName, targetName, unReceivedMsg, mentionType, isHiddenDetail, language);
+        PushMessage pushMessage = new PushMessage(sender, conversationType, target, line, messageContentType, serverTime, senderName, senderPortrait, targetName, targetPortrait, unReceivedMsg, mentionType, isHiddenDetail, language);
         pushMessage.pushContent = pushContent;
         pushMessage.pushData = pushData;
         pushMessage.messageId = messageId;

broker/src/main/java/io/moquette/spi/impl/ProtocolProcessor.java

@@ -43,10 +43,12 @@ import io.netty.handler.codec.mqtt.*;
 import io.netty.handler.timeout.IdleStateHandler;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
+import win.liyufan.im.GsonUtil;
 import win.liyufan.im.HttpUtils;
 import win.liyufan.im.IMTopic;
 import win.liyufan.im.Utility;
 
+import java.nio.charset.StandardCharsets;
 import java.util.*;
 import java.util.concurrent.ExecutorService;
 import java.util.concurrent.Executors;
@@ -106,7 +108,7 @@ public class ProtocolProcessor {
                 ConnectionDescriptor descriptor = connectionDescriptors.getConnection(session.getClientID());
                 try {
                     if (descriptor != null) {
-                        processDisconnect(descriptor.getChannel(), true, false);
+                        processDisconnect(descriptor.getChannel(), false, false);
                     }
                 } catch (InterruptedException e) {
                     e.printStackTrace();
@@ -264,7 +266,7 @@ public class ProtocolProcessor {
         if (!StringUtil.isNullOrEmpty(mUserOnlineStatusCallback)) {
             mServer.getCallbackScheduler().execute(() -> {
                 try {
-                    HttpUtils.httpJsonPost(mUserOnlineStatusCallback, new Gson().toJson(new UserOnlineStatus(userId, clientId, platform, status, packageName)), HttpUtils.HttpPostType.POST_TYPE_User_Online_Event_Callback);
+                    HttpUtils.httpJsonPost(mUserOnlineStatusCallback, GsonUtil.gson.toJson(new UserOnlineStatus(userId, clientId, platform, status, packageName)), HttpUtils.HttpPostType.POST_TYPE_User_Online_Event_Callback);
                 } catch (Exception e) {
                     e.printStackTrace();
                     Utility.printExecption(LOG, e, EVENT_CALLBACK_Exception);
@@ -307,7 +309,7 @@ public class ProtocolProcessor {
             }
             byte[] pwd = null;
             if (msg.variableHeader().hasPassword()) {
-                pwd = msg.payload().password();
+                pwd = msg.payload().passwordInBytes();
 
                 MemorySessionStore.Session session = m_sessionsStore.getSession(clientId);
                 if (session == null) {
@@ -338,6 +340,24 @@ public class ProtocolProcessor {
                     failedCredentials(channel);
                     return false;
                 }
+
+                if(session.getPlatform() == ProtoConstants.Platform.Platform_Android || session.getPlatform() == ProtoConstants.Platform.Platform_APad) {
+                    byte[] signature = msg.payload().signatureInBytes();
+                    String basedSign = null;
+                    if (signature != null && signature.length > 0) {
+                        signature = AES.AESDecrypt(signature, session.getSecret(), true);
+                    }
+                    if (signature != null && signature.length > 0) {
+                        basedSign = Base64.getEncoder().encodeToString(signature);
+                    }
+
+                    if (!m_messagesStore.checkSignature(basedSign)) {
+                        LOG.error("Bad signature of session <{}, {}>", session.getUsername(), session.getClientID());
+                        failedBadSignature(channel);
+                        return false;
+                    }
+                }
+
                 session.setMqttVersion(mqttVersion);
             } else {
                 LOG.error("Client didn't supply any password and MQTT anonymous mode is disabled CId={}", clientId);
@@ -445,6 +465,11 @@ public class ProtocolProcessor {
         LOG.info("Client {} failed to connect with bad username or password.", session);
     }
 
+    private void failedBadSignature(Channel session) {
+        session.writeAndFlush(connAck(CONNECTION_REFUSED_SIGNATURE_FAILURE));
+        LOG.info("Client {} failed to connect with bad signature.", session);
+    }
+
     private void setIdleTime(ChannelPipeline pipeline, int idleTime) {
         if (pipeline.names().contains("idleStateHandler")) {
             pipeline.remove("idleStateHandler");
@@ -583,8 +608,9 @@ public class ProtocolProcessor {
         channel.closeFuture();
 
         //disconnect the session
-
-        m_sessionsStore.sessionForClient(clientID).disconnect(session.getUsername(), isDup, isRetain);
+        if(session != null) {
+            m_sessionsStore.sessionForClient(clientID).disconnect(session.getUsername(), isDup, isRetain);
+        }
     }
 
 
@@ -625,12 +651,22 @@ public class ProtocolProcessor {
                 if(connectionDescriptors.removeConnection(oldConnDescr)) {
                     m_interceptor.notifyClientConnectionLost(clientID, username);
                 }
+                try {
+                    channel.close();
+                } catch (Exception e) {
+                    e.printStackTrace();
+                }
             });
         } else {
             ConnectionDescriptor oldConnDescr = new ConnectionDescriptor(clientID, channel);
             if(connectionDescriptors.removeConnection(oldConnDescr)) {
                 m_interceptor.notifyClientConnectionLost(clientID, username);
             }
+            try {
+                channel.close();
+            } catch (Exception e) {
+                e.printStackTrace();
+            }
         }
     }
 

broker/src/main/java/io/moquette/spi/impl/Qos1PublishHandler.java

@@ -57,6 +57,7 @@ import io.netty.handler.codec.mqtt.MqttFixedHeader;
 import io.netty.handler.codec.mqtt.MqttMessageType;
 import io.netty.handler.codec.mqtt.MqttPublishMessage;
 import cn.wildfirechat.common.ErrorCode;
+import win.liyufan.im.GsonUtil;
 import win.liyufan.im.IMTopic;
 import win.liyufan.im.RateLimiter;
 import win.liyufan.im.Utility;
@@ -142,7 +143,7 @@ public class Qos1PublishHandler extends QosPublishHandler {
                 out.addSession(userId, session.getClientID(), session.getPlatform(), status, session.getLastActiveTime());
             }
 
-            callback.onRouteHandled(new Gson().toJson(out).getBytes());
+            callback.onRouteHandled(GsonUtil.gson.toJson(out).getBytes());
         });
     }
 
@@ -163,7 +164,7 @@ public class Qos1PublishHandler extends QosPublishHandler {
                         } else {
                             MemorySessionStore.Session session = m_sessionStore.getSession(clientID);
                             if (session != null && session.getUsername().equals(fromUser)) {
-                                if (data.length > 7*1024 && session.getMqttVersion().protocolLevel() >= MqttVersion.Wildfire_1.protocolLevel()) {
+                                if (data.length > 7*1024 && session.getMqttVersion().protocolLevel() >= MqttVersion.MQTT_5.protocolLevel()) {
                                     ByteArrayOutputStream out = new ByteArrayOutputStream();
                                     GZIPOutputStream gzip = null;
                                     try {
@@ -242,15 +243,17 @@ public class Qos1PublishHandler extends QosPublishHandler {
         String imtopic = topic.getTopic();
         ByteBuf payload = msg.payload();
         byte[] payloadContent = readBytesAndRewind(payload);
-        if(payloadContent.length == 0) {
-            ByteBuf ackPayload = Unpooled.buffer();
-            ackPayload.ensureWritable(1).writeByte(ERROR_CODE_INVALID_DATA.getCode());
-            sendPubAck(clientID, messageID, ackPayload, ERROR_CODE_INVALID_DATA);
-            return;
+        if(payloadContent.length > 0) {
+            MemorySessionStore.Session session = m_sessionStore.getSession(clientID);
+            payloadContent = AES.AESDecrypt(payloadContent, session.getSecret(), true);
+            if(payloadContent == null) {
+                ByteBuf ackPayload = Unpooled.buffer();
+                ackPayload.ensureWritable(1).writeByte(ERROR_CODE_INVALID_DATA.getCode());
+                sendPubAck(clientID, messageID, ackPayload, ERROR_CODE_INVALID_DATA);
+                return;
+            }
         }
-        
-        MemorySessionStore.Session session = m_sessionStore.getSession(clientID);
-        payloadContent = AES.AESDecrypt(payloadContent, session.getSecret(), true);
+
         imHandler(clientID, username, imtopic, payloadContent, (errorCode, ackPayload) -> sendPubAck(clientID, messageID, ackPayload, errorCode), ProtoConstants.RequestSourceType.Request_From_User);
     }
 

broker/src/main/java/io/netty/handler/codec/mqtt/MqttCodecUtil.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -16,13 +16,30 @@
 
 package io.netty.handler.codec.mqtt;
 
+import io.netty.channel.ChannelHandlerContext;
 import io.netty.handler.codec.DecoderException;
+import io.netty.util.Attribute;
+import io.netty.util.AttributeKey;
 
 final class MqttCodecUtil {
 
     private static final char[] TOPIC_WILDCARDS = {'#', '+'};
-    private static final int MIN_CLIENT_ID_LENGTH = 1;
-    private static final int MAX_CLIENT_ID_LENGTH = 23;
+
+    static final AttributeKey<MqttVersion> MQTT_VERSION_KEY = AttributeKey.valueOf("NETTY_CODEC_MQTT_VERSION");
+
+    static MqttVersion getMqttVersion(ChannelHandlerContext ctx) {
+        Attribute<MqttVersion> attr = ctx.channel().attr(MQTT_VERSION_KEY);
+        MqttVersion version = attr.get();
+        if (version == null) {
+            return MqttVersion.MQTT_3_1_1;
+        }
+        return version;
+    }
+
+    static void setMqttVersion(ChannelHandlerContext ctx, MqttVersion version) {
+        Attribute<MqttVersion> attr = ctx.channel().attr(MQTT_VERSION_KEY);
+        attr.set(version);
+    }
 
     static boolean isValidPublishTopicName(String topicName) {
         // publish topic name must not contain any wildcard
@@ -38,20 +55,20 @@ final class MqttCodecUtil {
         return messageId != 0;
     }
 
-    static boolean isValidClientId(MqttVersion mqttVersion, String clientId) {
+    static boolean isValidClientId(MqttVersion mqttVersion, int maxClientIdLength, String clientId) {
         if (mqttVersion == MqttVersion.MQTT_3_1) {
-            return clientId != null && clientId.length() >= MIN_CLIENT_ID_LENGTH &&
-                clientId.length() <= MAX_CLIENT_ID_LENGTH;
+            return clientId != null && clientId.length() >= MqttConstant.MIN_CLIENT_ID_LENGTH &&
+                clientId.length() <= maxClientIdLength;
         }
         if (mqttVersion.protocolLevel() >= MqttVersion.MQTT_3_1_1.protocolLevel()) {
-            // In 3.1.3.1 Client Identifier of MQTT 3.1.1 specification, The Server MAY allow ClientId’s
+            // In 3.1.3.1 Client Identifier of MQTT 3.1.1 and 5.0 specifications, The Server MAY allow ClientId’s
             // that contain more than 23 encoded bytes. And, The Server MAY allow zero-length ClientId.
             return clientId != null;
         }
         throw new IllegalArgumentException(mqttVersion + " is unknown mqtt version");
     }
 
-    static MqttFixedHeader validateFixedHeader(MqttFixedHeader mqttFixedHeader) {
+    static MqttFixedHeader validateFixedHeader(ChannelHandlerContext ctx, MqttFixedHeader mqttFixedHeader) {
         switch (mqttFixedHeader.messageType()) {
             case PUBREL:
             case SUBSCRIBE:
@@ -59,6 +76,12 @@ final class MqttCodecUtil {
                 if (mqttFixedHeader.qosLevel() != MqttQoS.AT_LEAST_ONCE) {
                     throw new DecoderException(mqttFixedHeader.messageType().name() + " message must have QoS 1");
                 }
+                return mqttFixedHeader;
+            case AUTH:
+                if (MqttCodecUtil.getMqttVersion(ctx) != MqttVersion.MQTT_5) {
+                    throw new DecoderException("AUTH message requires at least MQTT 5");
+                }
+                return mqttFixedHeader;
             default:
                 return mqttFixedHeader;
         }
@@ -75,7 +98,7 @@ final class MqttCodecUtil {
             case UNSUBACK:
             case PINGREQ:
             case PINGRESP:
-            //case DISCONNECT:
+            case DISCONNECT:
                 if (mqttFixedHeader.isDup() ||
                         mqttFixedHeader.qosLevel() != MqttQoS.AT_MOST_ONCE ||
                         mqttFixedHeader.isRetain()) {

broker/src/main/java/io/netty/handler/codec/mqtt/MqttConnAckMessage.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -17,7 +17,7 @@
 package io.netty.handler.codec.mqtt;
 
 /**
- * See <a href="http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#connack">MQTTV3.1/connack</a>
+ * See <a href="https://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#connack">MQTTV3.1/connack</a>
  */
 public final class MqttConnAckMessage extends MqttMessage {
 
@@ -28,7 +28,7 @@ public final class MqttConnAckMessage extends MqttMessage {
     public MqttConnAckMessage(MqttFixedHeader mqttFixedHeader, MqttConnAckVariableHeader variableHeader, MqttConnectAckPayload payload) {
         super(mqttFixedHeader, variableHeader, payload);
     }
-
+    
     @Override
     public MqttConnAckVariableHeader variableHeader() {
         return (MqttConnAckVariableHeader) super.variableHeader();

broker/src/main/java/io/netty/handler/codec/mqtt/MqttConnAckVariableHeader.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -27,9 +27,17 @@ public final class MqttConnAckVariableHeader {
 
     private final boolean sessionPresent;
 
+    private final MqttProperties properties;
+
     public MqttConnAckVariableHeader(MqttConnectReturnCode connectReturnCode, boolean sessionPresent) {
+        this(connectReturnCode, sessionPresent, MqttProperties.NO_PROPERTIES);
+    }
+
+    public MqttConnAckVariableHeader(MqttConnectReturnCode connectReturnCode, boolean sessionPresent,
+                                     MqttProperties properties) {
         this.connectReturnCode = connectReturnCode;
         this.sessionPresent = sessionPresent;
+        this.properties = MqttProperties.withEmptyDefaults(properties);
     }
 
     public MqttConnectReturnCode connectReturnCode() {
@@ -40,6 +48,10 @@ public final class MqttConnAckVariableHeader {
         return sessionPresent;
     }
 
+    public MqttProperties properties() {
+        return properties;
+    }
+
     @Override
     public String toString() {
         return new StringBuilder(StringUtil.simpleClassName(this))

broker/src/main/java/io/netty/handler/codec/mqtt/MqttConnectMessage.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -17,7 +17,7 @@
 package io.netty.handler.codec.mqtt;
 
 /**
- * See <a href="http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#connect">MQTTV3.1/connect</a>
+ * See <a href="https://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#connect">MQTTV3.1/connect</a>
  */
 public final class MqttConnectMessage extends MqttMessage {
 

broker/src/main/java/io/netty/handler/codec/mqtt/MqttConnectPayload.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -16,6 +16,9 @@
 
 package io.netty.handler.codec.mqtt;
 
+import java.util.Arrays;
+
+import io.netty.util.CharsetUtil;
 import io.netty.util.internal.StringUtil;
 
 /**
@@ -24,33 +27,85 @@ import io.netty.util.internal.StringUtil;
 public final class MqttConnectPayload {
 
     private final String clientIdentifier;
+    private final MqttProperties willProperties;
     private final String willTopic;
-    private final String willMessage;
+    private final byte[] willMessage;
     private final String userName;
     private final byte[] password;
+    private final byte[] signature;
 
+    /**
+     * @deprecated use {@link MqttConnectPayload#MqttConnectPayload(String,
+     * MqttProperties, String, byte[], String, byte[])} instead
+     */
+    @Deprecated
     public MqttConnectPayload(
             String clientIdentifier,
             String willTopic,
             String willMessage,
             String userName,
+            String password) {
+        this(
+          clientIdentifier,
+          MqttProperties.NO_PROPERTIES,
+          willTopic,
+          willMessage.getBytes(CharsetUtil.UTF_8),
+          userName,
+          password.getBytes(CharsetUtil.UTF_8), null);
+    }
+
+    public MqttConnectPayload(
+            String clientIdentifier,
+            String willTopic,
+            byte[] willMessage,
+            String userName,
             byte[] password) {
+        this(clientIdentifier,
+                MqttProperties.NO_PROPERTIES,
+                willTopic,
+                willMessage,
+                userName,
+                password, null);
+    }
+
+    public MqttConnectPayload(
+            String clientIdentifier,
+            MqttProperties willProperties,
+            String willTopic,
+            byte[] willMessage,
+            String userName,
+            byte[] password,
+            byte[] signature) {
         this.clientIdentifier = clientIdentifier;
+        this.willProperties = MqttProperties.withEmptyDefaults(willProperties);
         this.willTopic = willTopic;
         this.willMessage = willMessage;
         this.userName = userName;
         this.password = password;
+        this.signature = signature;
     }
 
     public String clientIdentifier() {
         return clientIdentifier;
     }
 
+    public MqttProperties willProperties() {
+        return willProperties;
+    }
+
     public String willTopic() {
         return willTopic;
     }
 
+    /**
+     * @deprecated use {@link MqttConnectPayload#willMessageInBytes()} instead
+     */
+    @Deprecated
     public String willMessage() {
+        return willMessage == null ? null : new String(willMessage, CharsetUtil.UTF_8);
+    }
+
+    public byte[] willMessageInBytes() {
         return willMessage;
     }
 
@@ -58,19 +113,31 @@ public final class MqttConnectPayload {
         return userName;
     }
 
-    public byte[] password() {
+    /**
+     * @deprecated use {@link MqttConnectPayload#passwordInBytes()} instead
+     */
+    @Deprecated
+    public String password() {
+        return password == null ? null : new String(password, CharsetUtil.UTF_8);
+    }
+
+    public byte[] passwordInBytes() {
         return password;
     }
 
+    public byte[] signatureInBytes() {
+        return signature;
+    }
+
     @Override
     public String toString() {
         return new StringBuilder(StringUtil.simpleClassName(this))
             .append('[')
             .append("clientIdentifier=").append(clientIdentifier)
             .append(", willTopic=").append(willTopic)
-            .append(", willMessage=").append(willMessage)
+            .append(", willMessage=").append(Arrays.toString(willMessage))
             .append(", userName=").append(userName)
-            .append(", password=").append(password)
+            .append(", password=").append(Arrays.toString(password))
             .append(']')
             .toString();
     }

broker/src/main/java/io/netty/handler/codec/mqtt/MqttConnectReturnCode.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -16,31 +16,53 @@
 
 package io.netty.handler.codec.mqtt;
 
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.Map;
-
 /**
  * Return Code of {@link MqttConnAckMessage}
  */
 public enum MqttConnectReturnCode {
     CONNECTION_ACCEPTED((byte) 0x00),
+    //MQTT 3 codes
     CONNECTION_REFUSED_UNACCEPTABLE_PROTOCOL_VERSION((byte) 0X01),
     CONNECTION_REFUSED_IDENTIFIER_REJECTED((byte) 0x02),
     CONNECTION_REFUSED_SERVER_UNAVAILABLE((byte) 0x03),
     CONNECTION_REFUSED_BAD_USER_NAME_OR_PASSWORD((byte) 0x04),
     CONNECTION_REFUSED_NOT_AUTHORIZED((byte) 0x05),
     CONNECTION_REFUSED_UNEXPECT_NODE((byte) 0X06),
-    CONNECTION_REFUSED_SESSION_NOT_EXIST((byte) 0x07);
+    CONNECTION_REFUSED_SESSION_NOT_EXIST((byte) 0x07),
+    CONNECTION_REFUSED_SIGNATURE_FAILURE((byte) 0x08),
+    //MQTT 5 codes
+    CONNECTION_REFUSED_UNSPECIFIED_ERROR((byte) 0x80),
+    CONNECTION_REFUSED_MALFORMED_PACKET((byte) 0x81),
+    CONNECTION_REFUSED_PROTOCOL_ERROR((byte) 0x82),
+    CONNECTION_REFUSED_IMPLEMENTATION_SPECIFIC((byte) 0x83),
+    CONNECTION_REFUSED_UNSUPPORTED_PROTOCOL_VERSION((byte) 0x84),
+    CONNECTION_REFUSED_CLIENT_IDENTIFIER_NOT_VALID((byte) 0x85),
+    CONNECTION_REFUSED_BAD_USERNAME_OR_PASSWORD((byte) 0x86),
+    CONNECTION_REFUSED_NOT_AUTHORIZED_5((byte) 0x87),
+    CONNECTION_REFUSED_SERVER_UNAVAILABLE_5((byte) 0x88),
+    CONNECTION_REFUSED_SERVER_BUSY((byte) 0x89),
+    CONNECTION_REFUSED_BANNED((byte) 0x8A),
+    CONNECTION_REFUSED_BAD_AUTHENTICATION_METHOD((byte) 0x8C),
+    CONNECTION_REFUSED_TOPIC_NAME_INVALID((byte) 0x90),
+    CONNECTION_REFUSED_PACKET_TOO_LARGE((byte) 0x95),
+    CONNECTION_REFUSED_QUOTA_EXCEEDED((byte) 0x97),
+    CONNECTION_REFUSED_PAYLOAD_FORMAT_INVALID((byte) 0x99),
+    CONNECTION_REFUSED_RETAIN_NOT_SUPPORTED((byte) 0x9A),
+    CONNECTION_REFUSED_QOS_NOT_SUPPORTED((byte) 0x9B),
+    CONNECTION_REFUSED_USE_ANOTHER_SERVER((byte) 0x9C),
+    CONNECTION_REFUSED_SERVER_MOVED((byte) 0x9D),
+    CONNECTION_REFUSED_CONNECTION_RATE_EXCEEDED((byte) 0x9F);
 
-    private static final Map<Byte, MqttConnectReturnCode> VALUE_TO_CODE_MAP;
+    private static final MqttConnectReturnCode[] VALUES;
 
     static {
-        final Map<Byte, MqttConnectReturnCode> valueMap = new HashMap<Byte, MqttConnectReturnCode>();
-        for (MqttConnectReturnCode code: values()) {
-            valueMap.put(code.byteValue, code);
+        MqttConnectReturnCode[] values = values();
+        VALUES = new MqttConnectReturnCode[160];
+        for (MqttConnectReturnCode code : values) {
+            final int unsignedByte = code.byteValue & 0xFF;
+            // Suppress a warning about out of bounds access since the enum contains only correct values
+            VALUES[unsignedByte] = code;    // lgtm [java/index-out-of-bounds]
         }
-        VALUE_TO_CODE_MAP = Collections.unmodifiableMap(valueMap);
     }
 
     private final byte byteValue;
@@ -54,9 +76,16 @@ public enum MqttConnectReturnCode {
     }
 
     public static MqttConnectReturnCode valueOf(byte b) {
-        if (VALUE_TO_CODE_MAP.containsKey(b)) {
-            return VALUE_TO_CODE_MAP.get(b);
+        final int unsignedByte = b & 0xFF;
+        MqttConnectReturnCode mqttConnectReturnCode = null;
+        try {
+            mqttConnectReturnCode = VALUES[unsignedByte];
+        } catch (ArrayIndexOutOfBoundsException ignored) {
+            // no op
         }
-        throw new IllegalArgumentException("unknown connect return code: " + (b & 0xFF));
+        if (mqttConnectReturnCode == null) {
+            throw new IllegalArgumentException("unknown connect return code: " + unsignedByte);
+        }
+        return mqttConnectReturnCode;
     }
 }

broker/src/main/java/io/netty/handler/codec/mqtt/MqttConnectVariableHeader.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -32,6 +32,7 @@ public final class MqttConnectVariableHeader {
     private final boolean isWillFlag;
     private final boolean isCleanSession;
     private final int keepAliveTimeSeconds;
+    private final MqttProperties properties;
 
     public MqttConnectVariableHeader(
             String name,
@@ -43,6 +44,29 @@ public final class MqttConnectVariableHeader {
             boolean isWillFlag,
             boolean isCleanSession,
             int keepAliveTimeSeconds) {
+        this(name,
+                version,
+                hasUserName,
+                hasPassword,
+                isWillRetain,
+                willQos,
+                isWillFlag,
+                isCleanSession,
+                keepAliveTimeSeconds,
+                MqttProperties.NO_PROPERTIES);
+    }
+
+    public MqttConnectVariableHeader(
+            String name,
+            int version,
+            boolean hasUserName,
+            boolean hasPassword,
+            boolean isWillRetain,
+            int willQos,
+            boolean isWillFlag,
+            boolean isCleanSession,
+            int keepAliveTimeSeconds,
+            MqttProperties properties) {
         this.name = name;
         this.version = version;
         this.hasUserName = hasUserName;
@@ -52,6 +76,7 @@ public final class MqttConnectVariableHeader {
         this.isWillFlag = isWillFlag;
         this.isCleanSession = isCleanSession;
         this.keepAliveTimeSeconds = keepAliveTimeSeconds;
+        this.properties = MqttProperties.withEmptyDefaults(properties);
     }
 
     public String name() {
@@ -90,6 +115,10 @@ public final class MqttConnectVariableHeader {
         return keepAliveTimeSeconds;
     }
 
+    public MqttProperties properties() {
+        return properties;
+    }
+
     @Override
     public String toString() {
         return new StringBuilder(StringUtil.simpleClassName(this))

broker/src/main/java/io/netty/handler/codec/mqtt/MqttConstant.java

@@ -0,0 +1,39 @@
+/*
+ * Copyright 2021 The Netty Project
+ *
+ * The Netty Project licenses this file to you under the Apache License,
+ * version 2.0 (the "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at:
+ *
+ *   https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+package io.netty.handler.codec.mqtt;
+
+public final class MqttConstant {
+
+    private MqttConstant() {
+    }
+
+    /**
+     * Default max bytes in message
+     */
+    public static final int DEFAULT_MAX_BYTES_IN_MESSAGE = 262144; //256KB
+
+    /**
+     * min client id length
+     */
+    public static final int MIN_CLIENT_ID_LENGTH = 1;
+
+    /**
+     * Default max client id length,In the mqtt3.1 protocol,
+     * the default maximum Client Identifier length is 23
+     */
+    public static final int DEFAULT_MAX_CLIENT_ID_LENGTH = 23;
+
+}

broker/src/main/java/io/netty/handler/codec/mqtt/MqttDecoder.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -20,26 +20,22 @@ import io.netty.buffer.ByteBuf;
 import io.netty.channel.ChannelHandlerContext;
 import io.netty.handler.codec.DecoderException;
 import io.netty.handler.codec.ReplayingDecoder;
-import io.netty.handler.codec.mqtt.MqttDecoder.DecoderState;
+import io.netty.handler.codec.TooLongFrameException;
 import io.netty.util.CharsetUtil;
+import io.netty.util.internal.ObjectUtil;
 
 import java.util.ArrayList;
 import java.util.List;
 
-import static io.netty.handler.codec.mqtt.MqttCodecUtil.isValidClientId;
-import static io.netty.handler.codec.mqtt.MqttCodecUtil.isValidMessageId;
-import static io.netty.handler.codec.mqtt.MqttCodecUtil.isValidPublishTopicName;
-import static io.netty.handler.codec.mqtt.MqttCodecUtil.resetUnusedFields;
-import static io.netty.handler.codec.mqtt.MqttCodecUtil.validateFixedHeader;
-
 /**
  * Decodes Mqtt messages from bytes, following
- * <a href="http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html">
- *     the MQTT protocol specification v3.1</a>
+ * the MQTT protocol specification
+ * <a href="https://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html">v3.1</a>
+ * or
+ * <a href="https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html">v5.0</a>, depending on the
+ * version specified in the CONNECT message that first goes through the channel.
  */
-public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
-
-    private static final int DEFAULT_MAX_BYTES_IN_MESSAGE = 262144; //256K
+public final class MqttDecoder extends ReplayingDecoder<MqttDecoder.DecoderState> {
 
     /**
      * States of the decoder.
@@ -58,31 +54,42 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
     private int bytesRemainingInVariablePart;
 
     private final int maxBytesInMessage;
+    private final int maxClientIdLength;
 
     public MqttDecoder() {
-      this(DEFAULT_MAX_BYTES_IN_MESSAGE);
+      this(MqttConstant.DEFAULT_MAX_BYTES_IN_MESSAGE, MqttConstant.DEFAULT_MAX_CLIENT_ID_LENGTH);
     }
 
     public MqttDecoder(int maxBytesInMessage) {
+        this(maxBytesInMessage, MqttConstant.DEFAULT_MAX_CLIENT_ID_LENGTH);
+    }
+
+    public MqttDecoder(int maxBytesInMessage, int maxClientIdLength) {
         super(DecoderState.READ_FIXED_HEADER);
-        this.maxBytesInMessage = maxBytesInMessage;
+        this.maxBytesInMessage = ObjectUtil.checkPositive(maxBytesInMessage, "maxBytesInMessage");
+        this.maxClientIdLength = ObjectUtil.checkPositive(maxClientIdLength, "maxClientIdLength");
     }
 
     @Override
     protected void decode(ChannelHandlerContext ctx, ByteBuf buffer, List<Object> out) throws Exception {
         switch (state()) {
-            case READ_FIXED_HEADER:
-                mqttFixedHeader = decodeFixedHeader(buffer);
+            case READ_FIXED_HEADER: try {
+                mqttFixedHeader = decodeFixedHeader(ctx, buffer);
                 bytesRemainingInVariablePart = mqttFixedHeader.remainingLength();
                 checkpoint(DecoderState.READ_VARIABLE_HEADER);
                 // fall through
+            } catch (Exception cause) {
+                out.add(invalidMessage(cause));
+                return;
+            }
 
             case READ_VARIABLE_HEADER:  try {
-                if (bytesRemainingInVariablePart > maxBytesInMessage) {
-                    throw new DecoderException("too large message: " + bytesRemainingInVariablePart + " bytes");
-                }
-                final Result<?> decodedVariableHeader = decodeVariableHeader(buffer, mqttFixedHeader);
+                final Result<?> decodedVariableHeader = decodeVariableHeader(ctx, buffer, mqttFixedHeader);
                 variableHeader = decodedVariableHeader.value;
+                if (bytesRemainingInVariablePart > maxBytesInMessage) {
+                    buffer.skipBytes(actualReadableBytes());
+                    throw new TooLongFrameException("too large message: " + bytesRemainingInVariablePart + " bytes");
+                }
                 bytesRemainingInVariablePart -= decodedVariableHeader.numberOfBytesConsumed;
                 checkpoint(DecoderState.READ_PAYLOAD);
                 // fall through
@@ -94,9 +101,11 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
             case READ_PAYLOAD: try {
                 final Result<?> decodedPayload =
                         decodePayload(
+                                ctx,
                                 buffer,
                                 mqttFixedHeader.messageType(),
                                 bytesRemainingInVariablePart,
+                                maxClientIdLength,
                                 variableHeader);
                 bytesRemainingInVariablePart -= decodedPayload.numberOfBytesConsumed;
                 if (bytesRemainingInVariablePart != 0) {
@@ -129,16 +138,20 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
 
     private MqttMessage invalidMessage(Throwable cause) {
       checkpoint(DecoderState.BAD_MESSAGE);
-      return MqttMessageFactory.newInvalidMessage(cause);
+      return MqttMessageFactory.newInvalidMessage(mqttFixedHeader, variableHeader, cause);
     }
 
     /**
-     * Decodes the fixed header. It's one byte for the flags and then variable bytes for the remaining length.
+     * Decodes the fixed header. It's one byte for the flags and then variable
+     * bytes for the remaining length.
+     *
+     * @see
+     * <a href="https://docs.oasis-open.org/mqtt/mqtt/v3.1.1/errata01/os/mqtt-v3.1.1-errata01-os-complete.html#_Toc442180841">...</a>
      *
      * @param buffer the buffer to decode from
      * @return the fixed header
      */
-    private static MqttFixedHeader decodeFixedHeader(ByteBuf buffer) {
+    private static MqttFixedHeader decodeFixedHeader(ChannelHandlerContext ctx, ByteBuf buffer) {
         short b1 = buffer.readUnsignedByte();
 
         MqttMessageType messageType = MqttMessageType.valueOf(b1 >> 4);
@@ -146,6 +159,59 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
         int qosLevel = (b1 & 0x06) >> 1;
         boolean retain = (b1 & 0x01) != 0;
 
+        switch (messageType) {
+            case PUBLISH:
+                if (qosLevel == 3) {
+                    throw new DecoderException("Illegal QOS Level in fixed header of PUBLISH message ("
+                            + qosLevel + ')');
+                }
+                break;
+
+            case PUBREL:
+            case SUBSCRIBE:
+            case UNSUBSCRIBE:
+                if (dupFlag) {
+                    throw new DecoderException("Illegal BIT 3 in fixed header of " + messageType
+                            + " message, must be 0, found 1");
+                }
+                if (qosLevel != 1) {
+                    throw new DecoderException("Illegal QOS Level in fixed header of " + messageType
+                            + " message, must be 1, found " + qosLevel);
+                }
+                if (retain) {
+                    throw new DecoderException("Illegal BIT 0 in fixed header of " + messageType
+                            + " message, must be 0, found 1");
+                }
+                break;
+
+            case AUTH:
+            case CONNACK:
+            case CONNECT:
+            case DISCONNECT:
+            case PINGREQ:
+            case PINGRESP:
+            case PUBACK:
+            case PUBCOMP:
+            case PUBREC:
+            case SUBACK:
+            case UNSUBACK:
+                if (dupFlag) {
+                    throw new DecoderException("Illegal BIT 3 in fixed header of " + messageType
+                            + " message, must be 0, found 1");
+                }
+                if (qosLevel != 0) {
+                    throw new DecoderException("Illegal BIT 2 or 1 in fixed header of " + messageType
+                            + " message, must be 0, found " + qosLevel);
+                }
+                if (retain) {
+                    throw new DecoderException("Illegal BIT 0 in fixed header of " + messageType
+                            + " message, must be 0, found 1");
+                }
+                break;
+            default:
+                throw new DecoderException("Unknown message type, do not know how to validate fixed header");
+        }
+
         int remainingLength = 0;
         int multiplier = 1;
         short digit;
@@ -163,7 +229,7 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
         }
         MqttFixedHeader decodedFixedHeader =
                 new MqttFixedHeader(messageType, dupFlag, MqttQoS.valueOf(qosLevel), retain, remainingLength);
-        return validateFixedHeader(resetUnusedFields(decodedFixedHeader));
+        return MqttCodecUtil.validateFixedHeader(ctx, MqttCodecUtil.resetUnusedFields(decodedFixedHeader));
     }
 
     /**
@@ -172,50 +238,60 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
      * @param mqttFixedHeader MqttFixedHeader of the same message
      * @return the variable header
      */
-    private static Result<?> decodeVariableHeader(ByteBuf buffer, MqttFixedHeader mqttFixedHeader) {
+    private Result<?> decodeVariableHeader(ChannelHandlerContext ctx, ByteBuf buffer, MqttFixedHeader mqttFixedHeader) {
         switch (mqttFixedHeader.messageType()) {
             case CONNECT:
-                return decodeConnectionVariableHeader(buffer);
+                return decodeConnectionVariableHeader(ctx, buffer);
 
             case CONNACK:
-                return decodeConnAckVariableHeader(buffer);
+                return decodeConnAckVariableHeader(ctx, buffer);
 
-            case SUBSCRIBE:
             case UNSUBSCRIBE:
+            case SUBSCRIBE:
             case SUBACK:
             case UNSUBACK:
+                return decodeMessageIdAndPropertiesVariableHeader(ctx, buffer);
+
             case PUBACK:
             case PUBREC:
             case PUBCOMP:
             case PUBREL:
-                return decodeMessageIdVariableHeader(buffer);
+                return decodePubReplyMessage(buffer);
 
             case PUBLISH:
-                return decodePublishVariableHeader(buffer, mqttFixedHeader);
+                return decodePublishVariableHeader(ctx, buffer, mqttFixedHeader);
+
+            case DISCONNECT:
+            case AUTH:
+                return decodeReasonCodeAndPropertiesVariableHeader(buffer);
 
             case PINGREQ:
             case PINGRESP:
-            case DISCONNECT:
                 // Empty variable header
                 return new Result<Object>(null, 0);
+            default:
+                //shouldn't reach here
+                throw new DecoderException("Unknown message type: " + mqttFixedHeader.messageType());
         }
-        return new Result<Object>(null, 0); //should never reach here
     }
 
-    private static Result<MqttConnectVariableHeader> decodeConnectionVariableHeader(ByteBuf buffer) {
+    private static Result<MqttConnectVariableHeader> decodeConnectionVariableHeader(
+            ChannelHandlerContext ctx,
+            ByteBuf buffer) {
         final Result<String> protoString = decodeString(buffer);
         int numberOfBytesConsumed = protoString.numberOfBytesConsumed;
 
         final byte protocolLevel = buffer.readByte();
         numberOfBytesConsumed += 1;
 
-        final MqttVersion mqttVersion = MqttVersion.fromProtocolNameAndLevel(protoString.value, protocolLevel);
+        MqttVersion version = MqttVersion.fromProtocolNameAndLevel(protoString.value, protocolLevel);
+        MqttCodecUtil.setMqttVersion(ctx, version);
 
         final int b1 = buffer.readUnsignedByte();
         numberOfBytesConsumed += 1;
 
-        final Result<Integer> keepAlive = decodeMsbLsb(buffer);
-        numberOfBytesConsumed += keepAlive.numberOfBytesConsumed;
+        final int keepAlive = decodeMsbLsb(buffer);
+        numberOfBytesConsumed += 2;
 
         final boolean hasUserName = (b1 & 0x80) == 0x80;
         final boolean hasPassword = (b1 & 0x40) == 0x40;
@@ -223,69 +299,178 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
         final int willQos = (b1 & 0x18) >> 3;
         final boolean willFlag = (b1 & 0x04) == 0x04;
         final boolean cleanSession = (b1 & 0x02) == 0x02;
-        if (mqttVersion.protocolLevel() >= MqttVersion.MQTT_3_1_1.protocolLevel()) {
+        if (version == MqttVersion.MQTT_3_1_1 || version == MqttVersion.MQTT_5) {
             final boolean zeroReservedFlag = (b1 & 0x01) == 0x0;
             if (!zeroReservedFlag) {
                 // MQTT v3.1.1: The Server MUST validate that the reserved flag in the CONNECT Control Packet is
                 // set to zero and disconnect the Client if it is not zero.
-                // See http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc385349230
+                // See https://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc385349230
                 throw new DecoderException("non-zero reserved flag");
             }
         }
 
+        final MqttProperties properties;
+        if (version == MqttVersion.MQTT_5) {
+            final Result<MqttProperties> propertiesResult = decodeProperties(buffer);
+            properties = propertiesResult.value;
+            numberOfBytesConsumed += propertiesResult.numberOfBytesConsumed;
+        } else {
+            properties = MqttProperties.NO_PROPERTIES;
+        }
+
         final MqttConnectVariableHeader mqttConnectVariableHeader = new MqttConnectVariableHeader(
-                mqttVersion.protocolName(),
-                mqttVersion.protocolLevel(),
+                version.protocolName(),
+                version.protocolLevel(),
                 hasUserName,
                 hasPassword,
                 willRetain,
                 willQos,
                 willFlag,
                 cleanSession,
-                keepAlive.value);
+                keepAlive,
+                properties);
         return new Result<MqttConnectVariableHeader>(mqttConnectVariableHeader, numberOfBytesConsumed);
     }
 
-    private static Result<MqttConnAckVariableHeader> decodeConnAckVariableHeader(ByteBuf buffer) {
+    private static Result<MqttConnAckVariableHeader> decodeConnAckVariableHeader(
+            ChannelHandlerContext ctx,
+            ByteBuf buffer) {
+        final MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
         final boolean sessionPresent = (buffer.readUnsignedByte() & 0x01) == 0x01;
         byte returnCode = buffer.readByte();
-        final int numberOfBytesConsumed = 2;
+        int numberOfBytesConsumed = 2;
+
+        final MqttProperties properties;
+        if (mqttVersion == MqttVersion.MQTT_5) {
+            final Result<MqttProperties> propertiesResult = decodeProperties(buffer);
+            properties = propertiesResult.value;
+            numberOfBytesConsumed += propertiesResult.numberOfBytesConsumed;
+        } else {
+            properties = MqttProperties.NO_PROPERTIES;
+        }
+
         final MqttConnAckVariableHeader mqttConnAckVariableHeader =
-                new MqttConnAckVariableHeader(MqttConnectReturnCode.valueOf(returnCode), sessionPresent);
+                new MqttConnAckVariableHeader(MqttConnectReturnCode.valueOf(returnCode), sessionPresent, properties);
         return new Result<MqttConnAckVariableHeader>(mqttConnAckVariableHeader, numberOfBytesConsumed);
     }
 
-    private static Result<MqttMessageIdVariableHeader> decodeMessageIdVariableHeader(ByteBuf buffer) {
-        final Result<Integer> messageId = decodeMessageId(buffer);
-        return new Result<MqttMessageIdVariableHeader>(
-                MqttMessageIdVariableHeader.from(messageId.value),
-                messageId.numberOfBytesConsumed);
+    private static Result<MqttMessageIdAndPropertiesVariableHeader> decodeMessageIdAndPropertiesVariableHeader(
+            ChannelHandlerContext ctx,
+            ByteBuf buffer) {
+        final MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
+        final int packetId = decodeMessageId(buffer);
+
+        final MqttMessageIdAndPropertiesVariableHeader mqttVariableHeader;
+        final int mqtt5Consumed;
+
+        if (mqttVersion == MqttVersion.MQTT_5) {
+            final Result<MqttProperties> properties = decodeProperties(buffer);
+            mqttVariableHeader = new MqttMessageIdAndPropertiesVariableHeader(packetId, properties.value);
+            mqtt5Consumed = properties.numberOfBytesConsumed;
+        } else {
+            mqttVariableHeader = new MqttMessageIdAndPropertiesVariableHeader(packetId,
+                    MqttProperties.NO_PROPERTIES);
+            mqtt5Consumed = 0;
+        }
+
+        return new Result<MqttMessageIdAndPropertiesVariableHeader>(mqttVariableHeader,
+                2 + mqtt5Consumed);
     }
 
-    private static Result<MqttPublishVariableHeader> decodePublishVariableHeader(
+    private Result<MqttPubReplyMessageVariableHeader> decodePubReplyMessage(ByteBuf buffer) {
+        final int packetId = decodeMessageId(buffer);
+
+        final MqttPubReplyMessageVariableHeader mqttPubAckVariableHeader;
+        final int consumed;
+        final int packetIdNumberOfBytesConsumed = 2;
+        if (bytesRemainingInVariablePart > 3) {
+            final byte reasonCode = buffer.readByte();
+            final Result<MqttProperties> properties = decodeProperties(buffer);
+            mqttPubAckVariableHeader = new MqttPubReplyMessageVariableHeader(packetId,
+                    reasonCode,
+                    properties.value);
+            consumed = packetIdNumberOfBytesConsumed + 1 + properties.numberOfBytesConsumed;
+        } else if (bytesRemainingInVariablePart > 2) {
+            final byte reasonCode = buffer.readByte();
+            mqttPubAckVariableHeader = new MqttPubReplyMessageVariableHeader(packetId,
+                    reasonCode,
+                    MqttProperties.NO_PROPERTIES);
+            consumed = packetIdNumberOfBytesConsumed + 1;
+        } else {
+            mqttPubAckVariableHeader = new MqttPubReplyMessageVariableHeader(packetId,
+                    (byte) 0,
+                    MqttProperties.NO_PROPERTIES);
+            consumed = packetIdNumberOfBytesConsumed;
+        }
+
+        return new Result<MqttPubReplyMessageVariableHeader>(mqttPubAckVariableHeader, consumed);
+    }
+
+    private Result<MqttReasonCodeAndPropertiesVariableHeader> decodeReasonCodeAndPropertiesVariableHeader(
+            ByteBuf buffer) {
+        final byte reasonCode;
+        final MqttProperties properties;
+        final int consumed;
+        if (bytesRemainingInVariablePart > 1) {
+            reasonCode = buffer.readByte();
+            final Result<MqttProperties> propertiesResult = decodeProperties(buffer);
+            properties = propertiesResult.value;
+            consumed = 1 + propertiesResult.numberOfBytesConsumed;
+        } else if (bytesRemainingInVariablePart > 0) {
+            reasonCode = buffer.readByte();
+            properties = MqttProperties.NO_PROPERTIES;
+            consumed = 1;
+        } else {
+            reasonCode = 0;
+            properties = MqttProperties.NO_PROPERTIES;
+            consumed = 0;
+        }
+        final MqttReasonCodeAndPropertiesVariableHeader mqttReasonAndPropsVariableHeader =
+                new MqttReasonCodeAndPropertiesVariableHeader(reasonCode, properties);
+
+        return new Result<MqttReasonCodeAndPropertiesVariableHeader>(
+                mqttReasonAndPropsVariableHeader,
+                consumed);
+    }
+
+    private Result<MqttPublishVariableHeader> decodePublishVariableHeader(
+            ChannelHandlerContext ctx,
             ByteBuf buffer,
             MqttFixedHeader mqttFixedHeader) {
+        final MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
         final Result<String> decodedTopic = decodeString(buffer);
-        if (!isValidPublishTopicName(decodedTopic.value)) {
+        if (!MqttCodecUtil.isValidPublishTopicName(decodedTopic.value)) {
             throw new DecoderException("invalid publish topic name: " + decodedTopic.value + " (contains wildcards)");
         }
         int numberOfBytesConsumed = decodedTopic.numberOfBytesConsumed;
 
         int messageId = -1;
         if (mqttFixedHeader.qosLevel().value() > 0) {
-            final Result<Integer> decodedMessageId = decodeMessageId(buffer);
-            messageId = decodedMessageId.value;
-            numberOfBytesConsumed += decodedMessageId.numberOfBytesConsumed;
+            messageId = decodeMessageId(buffer);
+            numberOfBytesConsumed += 2;
         }
+
+        final MqttProperties properties;
+        if (mqttVersion == MqttVersion.MQTT_5) {
+            final Result<MqttProperties> propertiesResult = decodeProperties(buffer);
+            properties = propertiesResult.value;
+            numberOfBytesConsumed += propertiesResult.numberOfBytesConsumed;
+        } else {
+            properties = MqttProperties.NO_PROPERTIES;
+        }
+
         final MqttPublishVariableHeader mqttPublishVariableHeader =
-                new MqttPublishVariableHeader(decodedTopic.value, messageId);
+                new MqttPublishVariableHeader(decodedTopic.value, messageId, properties);
         return new Result<MqttPublishVariableHeader>(mqttPublishVariableHeader, numberOfBytesConsumed);
     }
 
-    private static Result<Integer> decodeMessageId(ByteBuf buffer) {
-        final Result<Integer> messageId = decodeMsbLsb(buffer);
-        if (!isValidMessageId(messageId.value)) {
-            throw new DecoderException("invalid messageId: " + messageId.value);
+    /**
+     * @return messageId with numberOfBytesConsumed is 2
+     */
+    private static int decodeMessageId(ByteBuf buffer) {
+        final int messageId = decodeMsbLsb(buffer);
+        if (!MqttCodecUtil.isValidMessageId(messageId)) {
+            throw new DecoderException("invalid messageId: " + messageId);
         }
         return messageId;
     }
@@ -300,13 +485,15 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
      * @return the payload
      */
     private static Result<?> decodePayload(
+            ChannelHandlerContext ctx,
             ByteBuf buffer,
             MqttMessageType messageType,
             int bytesRemainingInVariablePart,
+            int maxClientIdLength,
             Object variableHeader) {
         switch (messageType) {
             case CONNECT:
-                return decodeConnectionPayload(buffer, (MqttConnectVariableHeader) variableHeader);
+                return decodeConnectionPayload(buffer, maxClientIdLength, (MqttConnectVariableHeader) variableHeader);
 
             case SUBSCRIBE:
                 return decodeSubscribePayload(buffer, bytesRemainingInVariablePart);
@@ -317,6 +504,9 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
             case UNSUBSCRIBE:
                 return decodeUnsubscribePayload(buffer, bytesRemainingInVariablePart);
 
+            case UNSUBACK:
+                return decodeUnsubAckPayload(ctx, buffer, bytesRemainingInVariablePart);
+
             case PUBLISH:
                 return decodePublishPayload(buffer, bytesRemainingInVariablePart);
 
@@ -328,42 +518,62 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
 
     private static Result<MqttConnectPayload> decodeConnectionPayload(
             ByteBuf buffer,
+            int maxClientIdLength,
             MqttConnectVariableHeader mqttConnectVariableHeader) {
         final Result<String> decodedClientId = decodeString(buffer);
         final String decodedClientIdValue = decodedClientId.value;
         final MqttVersion mqttVersion = MqttVersion.fromProtocolNameAndLevel(mqttConnectVariableHeader.name(),
                 (byte) mqttConnectVariableHeader.version());
-        if (!isValidClientId(mqttVersion, decodedClientIdValue)) {
+        if (!MqttCodecUtil.isValidClientId(mqttVersion, maxClientIdLength, decodedClientIdValue)) {
             throw new MqttIdentifierRejectedException("invalid clientIdentifier: " + decodedClientIdValue);
         }
         int numberOfBytesConsumed = decodedClientId.numberOfBytesConsumed;
 
         Result<String> decodedWillTopic = null;
-        Result<String> decodedWillMessage = null;
+        byte[] decodedWillMessage = null;
+
+        final MqttProperties willProperties;
         if (mqttConnectVariableHeader.isWillFlag()) {
+            if (mqttVersion.protocolLevel() <= MqttVersion.MQTT_5.protocolLevel()) {
+                final Result<MqttProperties> propertiesResult = decodeProperties(buffer);
+                willProperties = propertiesResult.value;
+                numberOfBytesConsumed += propertiesResult.numberOfBytesConsumed;
+            } else {
+                willProperties = MqttProperties.NO_PROPERTIES;
+            }
             decodedWillTopic = decodeString(buffer, 0, 32767);
             numberOfBytesConsumed += decodedWillTopic.numberOfBytesConsumed;
-            decodedWillMessage = decodeAsciiString(buffer);
-            numberOfBytesConsumed += decodedWillMessage.numberOfBytesConsumed;
+            decodedWillMessage = decodeByteArray(buffer);
+            numberOfBytesConsumed += decodedWillMessage.length + 2;
+        } else {
+            willProperties = MqttProperties.NO_PROPERTIES;
         }
         Result<String> decodedUserName = null;
-        Result<byte[]> decodedPassword = null;
+        byte[] decodedPassword = null;
+        byte[] decodedSignature = null;
         if (mqttConnectVariableHeader.hasUserName()) {
             decodedUserName = decodeString(buffer);
             numberOfBytesConsumed += decodedUserName.numberOfBytesConsumed;
         }
         if (mqttConnectVariableHeader.hasPassword()) {
-            decodedPassword = decodeByte(buffer);
-            numberOfBytesConsumed += decodedPassword.numberOfBytesConsumed;
+            decodedPassword = decodeByteArray(buffer);
+            numberOfBytesConsumed += decodedPassword.length + 2;
+        }
+        if (mqttConnectVariableHeader.isWillRetain()) {
+            decodedSignature = decodeByteArray(buffer);
+            numberOfBytesConsumed += decodedSignature.length + 2;
         }
 
         final MqttConnectPayload mqttConnectPayload =
                 new MqttConnectPayload(
                         decodedClientId.value,
+                        willProperties,
                         decodedWillTopic != null ? decodedWillTopic.value : null,
-                        decodedWillMessage != null ? decodedWillMessage.value : null,
+                        decodedWillMessage,
                         decodedUserName != null ? decodedUserName.value : null,
-                        decodedPassword != null ? decodedPassword.value : null);
+                        decodedPassword,
+                        decodedSignature
+                    );
         return new Result<MqttConnectPayload>(mqttConnectPayload, numberOfBytesConsumed);
     }
 
@@ -375,9 +585,21 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
         while (numberOfBytesConsumed < bytesRemainingInVariablePart) {
             final Result<String> decodedTopicName = decodeString(buffer);
             numberOfBytesConsumed += decodedTopicName.numberOfBytesConsumed;
-            int qos = buffer.readUnsignedByte() & 0x03;
+            //See 3.8.3.1 Subscription Options of MQTT 5.0 specification for optionByte details
+            final short optionByte = buffer.readUnsignedByte();
+
+            MqttQoS qos = MqttQoS.valueOf(optionByte & 0x03);
+            boolean noLocal = ((optionByte & 0x04) >> 2) == 1;
+            boolean retainAsPublished = ((optionByte & 0x08) >> 3) == 1;
+            MqttSubscriptionOption.RetainedHandlingPolicy retainHandling = MqttSubscriptionOption.RetainedHandlingPolicy.valueOf((optionByte & 0x30) >> 4);
+
+            final MqttSubscriptionOption subscriptionOption = new MqttSubscriptionOption(qos,
+                    noLocal,
+                    retainAsPublished,
+                    retainHandling);
+
             numberOfBytesConsumed++;
-            subscribeTopics.add(new MqttTopicSubscription(decodedTopicName.value, MqttQoS.valueOf(qos)));
+            subscribeTopics.add(new MqttTopicSubscription(decodedTopicName.value, subscriptionOption));
         }
         return new Result<MqttSubscribePayload>(new MqttSubscribePayload(subscribeTopics), numberOfBytesConsumed);
     }
@@ -385,16 +607,30 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
     private static Result<MqttSubAckPayload> decodeSubackPayload(
             ByteBuf buffer,
             int bytesRemainingInVariablePart) {
-        final List<Integer> grantedQos = new ArrayList<Integer>();
+        final List<Integer> grantedQos = new ArrayList<Integer>(bytesRemainingInVariablePart);
         int numberOfBytesConsumed = 0;
         while (numberOfBytesConsumed < bytesRemainingInVariablePart) {
-            int qos = buffer.readUnsignedByte() & 0x03;
+            int reasonCode = buffer.readUnsignedByte();
             numberOfBytesConsumed++;
-            grantedQos.add(qos);
+            grantedQos.add(reasonCode);
         }
         return new Result<MqttSubAckPayload>(new MqttSubAckPayload(grantedQos), numberOfBytesConsumed);
     }
 
+    private static Result<MqttUnsubAckPayload> decodeUnsubAckPayload(
+        ChannelHandlerContext ctx,
+        ByteBuf buffer,
+        int bytesRemainingInVariablePart) {
+        final List<Short> reasonCodes = new ArrayList<Short>(bytesRemainingInVariablePart);
+        int numberOfBytesConsumed = 0;
+        while (numberOfBytesConsumed < bytesRemainingInVariablePart) {
+            short reasonCode = buffer.readUnsignedByte();
+            numberOfBytesConsumed++;
+            reasonCodes.add(reasonCode);
+        }
+        return new Result<MqttUnsubAckPayload>(new MqttUnsubAckPayload(reasonCodes), numberOfBytesConsumed);
+    }
+
     private static Result<MqttUnsubscribePayload> decodeUnsubscribePayload(
             ByteBuf buffer,
             int bytesRemainingInVariablePart) {
@@ -419,25 +655,9 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
         return decodeString(buffer, 0, Integer.MAX_VALUE);
     }
 
-    private static Result<String> decodeAsciiString(ByteBuf buffer) {
-        Result<String> result = decodeString(buffer, 0, Integer.MAX_VALUE);
-        final String s = result.value;
-        for (int i = 0; i < s.length(); i++) {
-            if (s.charAt(i) > 127) {
-                return new Result<String>(null, result.numberOfBytesConsumed);
-            }
-        }
-        return new Result<String>(s, result.numberOfBytesConsumed);
-    }
-
-    private static Result<byte[]> decodeByte(ByteBuf buffer) {
-        return decodeByte(buffer, 0, Integer.MAX_VALUE);
-    }
-
     private static Result<String> decodeString(ByteBuf buffer, int minBytes, int maxBytes) {
-        final Result<Integer> decodedSize = decodeMsbLsb(buffer);
-        int size = decodedSize.value;
-        int numberOfBytesConsumed = decodedSize.numberOfBytesConsumed;
+        int size = decodeMsbLsb(buffer);
+        int numberOfBytesConsumed = 2;
         if (size < minBytes || size > maxBytes) {
             buffer.skipBytes(size);
             numberOfBytesConsumed += size;
@@ -449,35 +669,68 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
         return new Result<String>(s, numberOfBytesConsumed);
     }
 
-    private static Result<byte[]> decodeByte(ByteBuf buffer, int minBytes, int maxBytes) {
-        final Result<Integer> decodedSize = decodeMsbLsb(buffer);
-        int size = decodedSize.value;
-        int numberOfBytesConsumed = decodedSize.numberOfBytesConsumed;
-        if (size < minBytes || size > maxBytes) {
-            buffer.skipBytes(size);
-            numberOfBytesConsumed += size;
-            return new Result<>(null, numberOfBytesConsumed);
-        }
-        byte[] s = new byte[size];
-        buffer.getBytes(buffer.readerIndex(), s, 0, size);
-        buffer.skipBytes(size);
-        numberOfBytesConsumed += size;
-        return new Result<>(s, numberOfBytesConsumed);
+    /**
+     *
+     * @return the decoded byte[], numberOfBytesConsumed = byte[].length + 2
+     */
+    private static byte[] decodeByteArray(ByteBuf buffer) {
+        int size = decodeMsbLsb(buffer);
+        byte[] bytes = new byte[size];
+        buffer.readBytes(bytes);
+        return bytes;
     }
 
-    private static Result<Integer> decodeMsbLsb(ByteBuf buffer) {
-        return decodeMsbLsb(buffer, 0, 65535);
+    // packing utils to reduce the amount of garbage while decoding ints
+    private static long packInts(int a, int b) {
+        return (((long) a) << 32) | (b & 0xFFFFFFFFL);
     }
 
-    private static Result<Integer> decodeMsbLsb(ByteBuf buffer, int min, int max) {
+    private static int unpackA(long ints) {
+        return (int) (ints >> 32);
+    }
+
+    private static int unpackB(long ints) {
+        return (int) ints;
+    }
+
+    /**
+     *  numberOfBytesConsumed = 2. return decoded result.
+     */
+    private static int decodeMsbLsb(ByteBuf buffer) {
+        int min = 0;
+        int max = 65535;
         short msbSize = buffer.readUnsignedByte();
         short lsbSize = buffer.readUnsignedByte();
-        final int numberOfBytesConsumed = 2;
         int result = msbSize << 8 | lsbSize;
         if (result < min || result > max) {
             result = -1;
         }
-        return new Result<Integer>(result, numberOfBytesConsumed);
+        return result;
+    }
+
+    /**
+     * See 1.5.5 Variable Byte Integer section of MQTT 5.0 specification for encoding/decoding rules
+     *
+     * @param buffer the buffer to decode from
+     * @return result pack with a = decoded integer, b = numberOfBytesConsumed. Need to unpack to read them.
+     * @throws DecoderException if bad MQTT protocol limits Remaining Length
+     */
+    private static long decodeVariableByteInteger(ByteBuf buffer) {
+        int remainingLength = 0;
+        int multiplier = 1;
+        short digit;
+        int loops = 0;
+        do {
+            digit = buffer.readUnsignedByte();
+            remainingLength += (digit & 127) * multiplier;
+            multiplier *= 128;
+            loops++;
+        } while ((digit & 128) != 0 && loops < 4);
+
+        if (loops == 4 && (digit & 128) != 0) {
+            throw new DecoderException("MQTT protocol limits Remaining Length to 4 bytes");
+        }
+        return packInts(remainingLength, loops);
     }
 
     private static final class Result<T> {
@@ -490,4 +743,82 @@ public final class MqttDecoder extends ReplayingDecoder<DecoderState> {
             this.numberOfBytesConsumed = numberOfBytesConsumed;
         }
     }
+
+    private static Result<MqttProperties> decodeProperties(ByteBuf buffer) {
+        final long propertiesLength = decodeVariableByteInteger(buffer);
+        int totalPropertiesLength = unpackA(propertiesLength);
+        int numberOfBytesConsumed = unpackB(propertiesLength);
+
+        MqttProperties decodedProperties = new MqttProperties();
+        while (numberOfBytesConsumed < totalPropertiesLength) {
+            long propertyId = decodeVariableByteInteger(buffer);
+            final int propertyIdValue = unpackA(propertyId);
+            numberOfBytesConsumed += unpackB(propertyId);
+            MqttProperties.MqttPropertyType propertyType = MqttProperties.MqttPropertyType.valueOf(propertyIdValue);
+            switch (propertyType) {
+                case PAYLOAD_FORMAT_INDICATOR:
+                case REQUEST_PROBLEM_INFORMATION:
+                case REQUEST_RESPONSE_INFORMATION:
+                case MAXIMUM_QOS:
+                case RETAIN_AVAILABLE:
+                case WILDCARD_SUBSCRIPTION_AVAILABLE:
+                case SUBSCRIPTION_IDENTIFIER_AVAILABLE:
+                case SHARED_SUBSCRIPTION_AVAILABLE:
+                    final int b1 = buffer.readUnsignedByte();
+                    numberOfBytesConsumed++;
+                    decodedProperties.add(new MqttProperties.IntegerProperty(propertyIdValue, b1));
+                    break;
+                case SERVER_KEEP_ALIVE:
+                case RECEIVE_MAXIMUM:
+                case TOPIC_ALIAS_MAXIMUM:
+                case TOPIC_ALIAS:
+                    final int int2BytesResult = decodeMsbLsb(buffer);
+                    numberOfBytesConsumed += 2;
+                    decodedProperties.add(new MqttProperties.IntegerProperty(propertyIdValue, int2BytesResult));
+                    break;
+                case PUBLICATION_EXPIRY_INTERVAL:
+                case SESSION_EXPIRY_INTERVAL:
+                case WILL_DELAY_INTERVAL:
+                case MAXIMUM_PACKET_SIZE:
+                    final int maxPacketSize = buffer.readInt();
+                    numberOfBytesConsumed += 4;
+                    decodedProperties.add(new MqttProperties.IntegerProperty(propertyIdValue, maxPacketSize));
+                    break;
+                case SUBSCRIPTION_IDENTIFIER:
+                    long vbIntegerResult = decodeVariableByteInteger(buffer);
+                    numberOfBytesConsumed += unpackB(vbIntegerResult);
+                    decodedProperties.add(new MqttProperties.IntegerProperty(propertyIdValue, unpackA(vbIntegerResult)));
+                    break;
+                case CONTENT_TYPE:
+                case RESPONSE_TOPIC:
+                case ASSIGNED_CLIENT_IDENTIFIER:
+                case AUTHENTICATION_METHOD:
+                case RESPONSE_INFORMATION:
+                case SERVER_REFERENCE:
+                case REASON_STRING:
+                    final Result<String> stringResult = decodeString(buffer);
+                    numberOfBytesConsumed += stringResult.numberOfBytesConsumed;
+                    decodedProperties.add(new MqttProperties.StringProperty(propertyIdValue, stringResult.value));
+                    break;
+                case USER_PROPERTY:
+                    final Result<String> keyResult = decodeString(buffer);
+                    final Result<String> valueResult = decodeString(buffer);
+                    numberOfBytesConsumed += keyResult.numberOfBytesConsumed;
+                    numberOfBytesConsumed += valueResult.numberOfBytesConsumed;
+                    decodedProperties.add(new MqttProperties.UserProperty(keyResult.value, valueResult.value));
+                    break;
+                case CORRELATION_DATA:
+                case AUTHENTICATION_DATA:
+                    final byte[] binaryDataResult = decodeByteArray(buffer);
+                    numberOfBytesConsumed += binaryDataResult.length + 2;
+                    decodedProperties.add(new MqttProperties.BinaryProperty(propertyIdValue, binaryDataResult));
+                    break;
+                default:
+                    //shouldn't reach here
+                    throw new DecoderException("Unknown property type: " + propertyType);
+            }
+        }
+
+        return new Result<MqttProperties>(decodedProperties, numberOfBytesConsumed);
+    }
 }

broker/src/main/java/io/netty/handler/codec/mqtt/MqttEncoder.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -18,19 +18,22 @@ package io.netty.handler.codec.mqtt;
 
 import io.netty.buffer.ByteBuf;
 import io.netty.buffer.ByteBufAllocator;
+import io.netty.buffer.Unpooled;
 import io.netty.channel.ChannelHandler;
 import io.netty.channel.ChannelHandlerContext;
+import io.netty.handler.codec.EncoderException;
 import io.netty.handler.codec.MessageToMessageEncoder;
-import io.netty.util.CharsetUtil;
 import io.netty.util.internal.EmptyArrays;
 
 import java.util.List;
 
-import static io.netty.handler.codec.mqtt.MqttCodecUtil.*;
+import static io.netty.buffer.ByteBufUtil.*;
 
 /**
  * Encodes Mqtt messages into bytes following the protocol specification v3.1
- * as described here <a href="http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html">MQTTV3.1</a>
+ * as described here <a href="https://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html">MQTTV3.1</a>
+ * or v5.0 as described here <a href="https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html">MQTTv5.0</a> -
+ * depending on the version specified in the first CONNECT message that goes through the channel.
  */
 @ChannelHandler.Sharable
 public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
@@ -41,49 +44,57 @@ public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
 
     @Override
     protected void encode(ChannelHandlerContext ctx, MqttMessage msg, List<Object> out) throws Exception {
-        out.add(doEncode(ctx.alloc(), msg));
+        out.add(doEncode(ctx, msg));
     }
 
     /**
      * This is the main encoding method.
      * It's only visible for testing.
      *
-     * @param byteBufAllocator Allocates ByteBuf
      * @param message MQTT message to encode
      * @return ByteBuf with encoded bytes
      */
-    static ByteBuf doEncode(ByteBufAllocator byteBufAllocator, MqttMessage message) {
+    static ByteBuf doEncode(ChannelHandlerContext ctx,
+                     MqttMessage message) {
 
         switch (message.fixedHeader().messageType()) {
             case CONNECT:
-                return encodeConnectMessage(byteBufAllocator, (MqttConnectMessage) message);
+                return encodeConnectMessage(ctx, (MqttConnectMessage) message);
 
             case CONNACK:
-                return encodeConnAckMessage(byteBufAllocator, (MqttConnAckMessage) message);
+                return encodeConnAckMessage(ctx, (MqttConnAckMessage) message);
 
             case PUBLISH:
-                return encodePublishMessage(byteBufAllocator, (MqttPublishMessage) message);
+                return encodePublishMessage(ctx, (MqttPublishMessage) message);
 
             case SUBSCRIBE:
-                return encodeSubscribeMessage(byteBufAllocator, (MqttSubscribeMessage) message);
+                return encodeSubscribeMessage(ctx, (MqttSubscribeMessage) message);
 
             case UNSUBSCRIBE:
-                return encodeUnsubscribeMessage(byteBufAllocator, (MqttUnsubscribeMessage) message);
+                return encodeUnsubscribeMessage(ctx,  (MqttUnsubscribeMessage) message);
 
             case SUBACK:
-                return encodeSubAckMessage(byteBufAllocator, (MqttSubAckMessage) message);
+                return encodeSubAckMessage(ctx, (MqttSubAckMessage) message);
 
             case UNSUBACK:
+                if (message instanceof MqttUnsubAckMessage) {
+                    return encodeUnsubAckMessage(ctx, (MqttUnsubAckMessage) message);
+                }
+                return encodeMessageWithOnlySingleByteFixedHeaderAndMessageId(ctx.alloc(), message);
+
             case PUBACK:
             case PUBREC:
             case PUBREL:
             case PUBCOMP:
-                return encodeMessageWithOnlySingleByteFixedHeaderAndMessageId(byteBufAllocator, message);
+                return encodePubReplyMessage(ctx, message);
+
+            case DISCONNECT:
+            case AUTH:
+                return encodeReasonCodePlusPropertiesMessage(ctx, message);
 
             case PINGREQ:
             case PINGRESP:
-            case DISCONNECT:
-                return encodeMessageWithOnlySingleByteFixedHeader(byteBufAllocator, message);
+                return encodeMessageWithOnlySingleByteFixedHeader(ctx.alloc(), message);
 
             default:
                 throw new IllegalArgumentException(
@@ -92,7 +103,7 @@ public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
     }
 
     private static ByteBuf encodeConnectMessage(
-            ByteBufAllocator byteBufAllocator,
+            ChannelHandlerContext ctx,
             MqttConnectMessage message) {
         int payloadBufferSize = 0;
 
@@ -101,70 +112,96 @@ public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
         MqttConnectPayload payload = message.payload();
         MqttVersion mqttVersion = MqttVersion.fromProtocolNameAndLevel(variableHeader.name(),
                 (byte) variableHeader.version());
+        MqttCodecUtil.setMqttVersion(ctx, mqttVersion);
+
+        // as MQTT 3.1 & 3.1.1 spec, If the User Name Flag is set to 0, the Password Flag MUST be set to 0
+        if (!variableHeader.hasUserName() && variableHeader.hasPassword()) {
+            throw new EncoderException("Without a username, the password MUST be not set");
+        }
 
         // Client id
         String clientIdentifier = payload.clientIdentifier();
-        if (!isValidClientId(mqttVersion, clientIdentifier)) {
+        if (!MqttCodecUtil.isValidClientId(mqttVersion, MqttConstant.DEFAULT_MAX_CLIENT_ID_LENGTH, clientIdentifier)) {
             throw new MqttIdentifierRejectedException("invalid clientIdentifier: " + clientIdentifier);
         }
-        byte[] clientIdentifierBytes = encodeStringUtf8(clientIdentifier);
-        payloadBufferSize += 2 + clientIdentifierBytes.length;
+        int clientIdentifierBytes = utf8Bytes(clientIdentifier);
+        payloadBufferSize += 2 + clientIdentifierBytes;
 
         // Will topic and message
         String willTopic = payload.willTopic();
-        byte[] willTopicBytes = willTopic != null ? encodeStringUtf8(willTopic) : EmptyArrays.EMPTY_BYTES;
-        String willMessage = payload.willMessage();
-        byte[] willMessageBytes = willMessage != null ? encodeStringUtf8(willMessage) : EmptyArrays.EMPTY_BYTES;
+        int willTopicBytes = nullableUtf8Bytes(willTopic);
+        byte[] willMessage = payload.willMessageInBytes();
+        byte[] willMessageBytes = willMessage != null ? willMessage : EmptyArrays.EMPTY_BYTES;
         if (variableHeader.isWillFlag()) {
-            payloadBufferSize += 2 + willTopicBytes.length;
+            payloadBufferSize += 2 + willTopicBytes;
             payloadBufferSize += 2 + willMessageBytes.length;
         }
 
         String userName = payload.userName();
-        byte[] userNameBytes = userName != null ? encodeStringUtf8(userName) : EmptyArrays.EMPTY_BYTES;
+        int userNameBytes = nullableUtf8Bytes(userName);
         if (variableHeader.hasUserName()) {
-            payloadBufferSize += 2 + userNameBytes.length;
+            payloadBufferSize += 2 + userNameBytes;
         }
 
-        byte[] passwordBytes = payload.password();
+        byte[] password = payload.passwordInBytes();
+        byte[] passwordBytes = password != null ? password : EmptyArrays.EMPTY_BYTES;
         if (variableHeader.hasPassword()) {
             payloadBufferSize += 2 + passwordBytes.length;
         }
 
-        // Fixed header
+        // Fixed and variable header
         byte[] protocolNameBytes = mqttVersion.protocolNameBytes();
-        int variableHeaderBufferSize = 2 + protocolNameBytes.length + 4;
-        int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
-        int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
-        ByteBuf buf = byteBufAllocator.buffer(fixedHeaderBufferSize + variablePartSize);
-        buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
-        writeVariableLengthInt(buf, variablePartSize);
+        ByteBuf propertiesBuf = encodePropertiesIfNeeded(
+                mqttVersion,
+                ctx.alloc(),
+                message.variableHeader().properties());
+        try {
+            final ByteBuf willPropertiesBuf;
+            if (variableHeader.isWillFlag()) {
+                willPropertiesBuf = encodePropertiesIfNeeded(mqttVersion, ctx.alloc(), payload.willProperties());
+                payloadBufferSize += willPropertiesBuf.readableBytes();
+            } else {
+                willPropertiesBuf = Unpooled.EMPTY_BUFFER;
+            }
+            try {
+                int variableHeaderBufferSize = 2 + protocolNameBytes.length + 4 + propertiesBuf.readableBytes();
 
-        buf.writeShort(protocolNameBytes.length);
-        buf.writeBytes(protocolNameBytes);
+                int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
+                int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
+                ByteBuf buf = ctx.alloc().buffer(fixedHeaderBufferSize + variablePartSize);
+                buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
+                writeVariableLengthInt(buf, variablePartSize);
 
-        buf.writeByte(variableHeader.version());
-        buf.writeByte(getConnVariableHeaderFlag(variableHeader));
-        buf.writeShort(variableHeader.keepAliveTimeSeconds());
+                buf.writeShort(protocolNameBytes.length);
+                buf.writeBytes(protocolNameBytes);
 
-        // Payload
-        buf.writeShort(clientIdentifierBytes.length);
-        buf.writeBytes(clientIdentifierBytes, 0, clientIdentifierBytes.length);
-        if (variableHeader.isWillFlag()) {
-            buf.writeShort(willTopicBytes.length);
-            buf.writeBytes(willTopicBytes, 0, willTopicBytes.length);
-            buf.writeShort(willMessageBytes.length);
-            buf.writeBytes(willMessageBytes, 0, willMessageBytes.length);
+                buf.writeByte(variableHeader.version());
+                buf.writeByte(getConnVariableHeaderFlag(variableHeader));
+                buf.writeShort(variableHeader.keepAliveTimeSeconds());
+                buf.writeBytes(propertiesBuf);
+
+                // Payload
+                writeExactUTF8String(buf, clientIdentifier, clientIdentifierBytes);
+                if (variableHeader.isWillFlag()) {
+                    buf.writeBytes(willPropertiesBuf);
+                    writeExactUTF8String(buf, willTopic, willTopicBytes);
+                    buf.writeShort(willMessageBytes.length);
+                    buf.writeBytes(willMessageBytes, 0, willMessageBytes.length);
+                }
+                if (variableHeader.hasUserName()) {
+                    writeExactUTF8String(buf, userName, userNameBytes);
+                }
+                if (variableHeader.hasPassword()) {
+                    buf.writeShort(passwordBytes.length);
+                    buf.writeBytes(passwordBytes, 0, passwordBytes.length);
+                }
+                return buf;
+            } finally {
+                willPropertiesBuf.release();
+            }
+        } finally {
+            propertiesBuf.release();
         }
-        if (variableHeader.hasUserName()) {
-            buf.writeShort(userNameBytes.length);
-            buf.writeBytes(userNameBytes, 0, userNameBytes.length);
-        }
-        if (variableHeader.hasPassword()) {
-            buf.writeShort(passwordBytes.length);
-            buf.writeBytes(passwordBytes, 0, passwordBytes.length);
-        }
-        return buf;
     }
 
     private static int getConnVariableHeaderFlag(MqttConnectVariableHeader variableHeader) {
@@ -189,149 +226,285 @@ public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
     }
 
     private static ByteBuf encodeConnAckMessage(
-            ByteBufAllocator byteBufAllocator,
+            ChannelHandlerContext ctx,
             MqttConnAckMessage message) {
-        int length = 4;
-        MqttConnectAckPayload payload = null;
-        if (message.payload() != null) {
-            payload = (MqttConnectAckPayload)message.payload();
-            if (payload.getData() != null && payload.getData().length > 0) {
-                length += payload.getData().length;
-            }
-        }
-        ByteBuf buf = byteBufAllocator.buffer(length);
-        buf.writeByte(getFixedHeaderByte1(message.fixedHeader()));
-        buf.writeByte(length - 2);
-        buf.writeByte(message.variableHeader().isSessionPresent() ? 0x01 : 0x00);
-        buf.writeByte(message.variableHeader().connectReturnCode().byteValue());
-        if (length > 4) {
-            buf.writeBytes(payload.getData());
-        }
+        final MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
+        ByteBuf propertiesBuf = encodePropertiesIfNeeded(mqttVersion,
+                ctx.alloc(),
+                message.variableHeader().properties());
 
-        return buf;
+        try {
+            int length = 4;
+            MqttConnectAckPayload payload = null;
+            if (message.payload() != null) {
+                payload = (MqttConnectAckPayload)message.payload();
+                if (payload.getData() != null && payload.getData().length > 0) {
+                    length += payload.getData().length;
+                }
+            }
+            ByteBuf buf = ctx.alloc().buffer(length + propertiesBuf.readableBytes());
+            buf.writeByte(getFixedHeaderByte1(message.fixedHeader()));
+            writeVariableLengthInt(buf, length - 2 + propertiesBuf.readableBytes());
+            buf.writeByte(message.variableHeader().isSessionPresent() ? 0x01 : 0x00);
+            buf.writeByte(message.variableHeader().connectReturnCode().byteValue());
+            if (length > 4) {
+                buf.writeBytes(payload.getData());
+            }
+            buf.writeBytes(propertiesBuf);
+            return buf;
+        } finally {
+            propertiesBuf.release();
+        }
     }
 
     private static ByteBuf encodeSubscribeMessage(
-            ByteBufAllocator byteBufAllocator,
+            ChannelHandlerContext ctx,
             MqttSubscribeMessage message) {
-        int variableHeaderBufferSize = 2;
-        int payloadBufferSize = 0;
+        MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
+        ByteBuf propertiesBuf = encodePropertiesIfNeeded(mqttVersion,
+                ctx.alloc(),
+                message.idAndPropertiesVariableHeader().properties());
 
-        MqttFixedHeader mqttFixedHeader = message.fixedHeader();
-        MqttMessageIdVariableHeader variableHeader = message.variableHeader();
-        MqttSubscribePayload payload = message.payload();
+        try {
+            final int variableHeaderBufferSize = 2 + propertiesBuf.readableBytes();
+            int payloadBufferSize = 0;
 
-        for (MqttTopicSubscription topic : payload.topicSubscriptions()) {
-            String topicName = topic.topicName();
-            byte[] topicNameBytes = encodeStringUtf8(topicName);
-            payloadBufferSize += 2 + topicNameBytes.length;
-            payloadBufferSize += 1;
+            MqttFixedHeader mqttFixedHeader = message.fixedHeader();
+            MqttMessageIdVariableHeader variableHeader = message.variableHeader();
+            MqttSubscribePayload payload = message.payload();
+
+            for (MqttTopicSubscription topic : payload.topicSubscriptions()) {
+                String topicName = topic.topicName();
+                int topicNameBytes = utf8Bytes(topicName);
+                payloadBufferSize += 2 + topicNameBytes;
+                payloadBufferSize += 1;
+            }
+
+            int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
+            int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
+
+            ByteBuf buf = ctx.alloc().buffer(fixedHeaderBufferSize + variablePartSize);
+            buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
+            writeVariableLengthInt(buf, variablePartSize);
+
+            // Variable Header
+            int messageId = variableHeader.messageId();
+            buf.writeShort(messageId);
+            buf.writeBytes(propertiesBuf);
+
+            // Payload
+            for (MqttTopicSubscription topic : payload.topicSubscriptions()) {
+                writeUnsafeUTF8String(buf, topic.topicName());
+                if (mqttVersion.protocolLevel() >= MqttVersion.MQTT_3_1_1.protocolLevel()) {
+                    buf.writeByte(topic.qualityOfService().value());
+                } else {
+                    final MqttSubscriptionOption option = topic.option();
+
+                    int optionEncoded = option.retainHandling().value() << 4;
+                    if (option.isRetainAsPublished()) {
+                        optionEncoded |= 0x08;
+                    }
+                    if (option.isNoLocal()) {
+                        optionEncoded |= 0x04;
+                    }
+                    optionEncoded |= option.qos().value();
+
+                    buf.writeByte(optionEncoded);
+                }
+            }
+
+            return buf;
+        } finally {
+            propertiesBuf.release();
         }
-
-        int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
-        int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
-
-        ByteBuf buf = byteBufAllocator.buffer(fixedHeaderBufferSize + variablePartSize);
-        buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
-        writeVariableLengthInt(buf, variablePartSize);
-
-        // Variable Header
-        int messageId = variableHeader.messageId();
-        buf.writeShort(messageId);
-
-        // Payload
-        for (MqttTopicSubscription topic : payload.topicSubscriptions()) {
-            String topicName = topic.topicName();
-            byte[] topicNameBytes = encodeStringUtf8(topicName);
-            buf.writeShort(topicNameBytes.length);
-            buf.writeBytes(topicNameBytes, 0, topicNameBytes.length);
-            buf.writeByte(topic.qualityOfService().value());
-        }
-
-        return buf;
     }
 
     private static ByteBuf encodeUnsubscribeMessage(
-            ByteBufAllocator byteBufAllocator,
+            ChannelHandlerContext ctx,
             MqttUnsubscribeMessage message) {
-        int variableHeaderBufferSize = 2;
-        int payloadBufferSize = 0;
+        MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
+        ByteBuf propertiesBuf = encodePropertiesIfNeeded(mqttVersion,
+                ctx.alloc(),
+                message.idAndPropertiesVariableHeader().properties());
 
-        MqttFixedHeader mqttFixedHeader = message.fixedHeader();
-        MqttMessageIdVariableHeader variableHeader = message.variableHeader();
-        MqttUnsubscribePayload payload = message.payload();
+        try {
+            final int variableHeaderBufferSize = 2 + propertiesBuf.readableBytes();
+            int payloadBufferSize = 0;
 
-        for (String topicName : payload.topics()) {
-            byte[] topicNameBytes = encodeStringUtf8(topicName);
-            payloadBufferSize += 2 + topicNameBytes.length;
+            MqttFixedHeader mqttFixedHeader = message.fixedHeader();
+            MqttMessageIdVariableHeader variableHeader = message.variableHeader();
+            MqttUnsubscribePayload payload = message.payload();
+
+            for (String topicName : payload.topics()) {
+                int topicNameBytes = utf8Bytes(topicName);
+                payloadBufferSize += 2 + topicNameBytes;
+            }
+
+            int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
+            int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
+
+            ByteBuf buf = ctx.alloc().buffer(fixedHeaderBufferSize + variablePartSize);
+            buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
+            writeVariableLengthInt(buf, variablePartSize);
+
+            // Variable Header
+            int messageId = variableHeader.messageId();
+            buf.writeShort(messageId);
+            buf.writeBytes(propertiesBuf);
+
+            // Payload
+            for (String topicName : payload.topics()) {
+                writeUnsafeUTF8String(buf, topicName);
+            }
+
+            return buf;
+        } finally {
+            propertiesBuf.release();
         }
-
-        int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
-        int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
-
-        ByteBuf buf = byteBufAllocator.buffer(fixedHeaderBufferSize + variablePartSize);
-        buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
-        writeVariableLengthInt(buf, variablePartSize);
-
-        // Variable Header
-        int messageId = variableHeader.messageId();
-        buf.writeShort(messageId);
-
-        // Payload
-        for (String topicName : payload.topics()) {
-            byte[] topicNameBytes = encodeStringUtf8(topicName);
-            buf.writeShort(topicNameBytes.length);
-            buf.writeBytes(topicNameBytes, 0, topicNameBytes.length);
-        }
-
-        return buf;
     }
 
     private static ByteBuf encodeSubAckMessage(
-            ByteBufAllocator byteBufAllocator,
+            ChannelHandlerContext ctx,
             MqttSubAckMessage message) {
-        int variableHeaderBufferSize = 2;
-        int payloadBufferSize = message.payload().grantedQoSLevels().size();
-        int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
-        int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
-        ByteBuf buf = byteBufAllocator.buffer(fixedHeaderBufferSize + variablePartSize);
-        buf.writeByte(getFixedHeaderByte1(message.fixedHeader()));
-        writeVariableLengthInt(buf, variablePartSize);
-        buf.writeShort(message.variableHeader().messageId());
-        for (int qos : message.payload().grantedQoSLevels()) {
-            buf.writeByte(qos);
-        }
+        MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
+        ByteBuf propertiesBuf = encodePropertiesIfNeeded(mqttVersion,
+                ctx.alloc(),
+                message.idAndPropertiesVariableHeader().properties());
+        try {
+            int variableHeaderBufferSize = 2 + propertiesBuf.readableBytes();
+            int payloadBufferSize = message.payload().grantedQoSLevels().size();
+            int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
+            int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
+            ByteBuf buf = ctx.alloc().buffer(fixedHeaderBufferSize + variablePartSize);
+            buf.writeByte(getFixedHeaderByte1(message.fixedHeader()));
+            writeVariableLengthInt(buf, variablePartSize);
+            buf.writeShort(message.variableHeader().messageId());
+            buf.writeBytes(propertiesBuf);
+            for (int code: message.payload().reasonCodes()) {
+                buf.writeByte(code);
+            }
 
-        return buf;
+            return buf;
+        } finally {
+            propertiesBuf.release();
+        }
+    }
+
+    private static ByteBuf encodeUnsubAckMessage(
+            ChannelHandlerContext ctx,
+            MqttUnsubAckMessage message) {
+        if (message.variableHeader() instanceof  MqttMessageIdAndPropertiesVariableHeader) {
+            MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
+            ByteBuf propertiesBuf = encodePropertiesIfNeeded(mqttVersion,
+                    ctx.alloc(),
+                    message.idAndPropertiesVariableHeader().properties());
+            try {
+                int variableHeaderBufferSize = 2 + propertiesBuf.readableBytes();
+                MqttUnsubAckPayload payload = message.payload();
+                int payloadBufferSize = payload == null ? 0 : payload.unsubscribeReasonCodes().size();
+                int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
+                int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
+                ByteBuf buf = ctx.alloc().buffer(fixedHeaderBufferSize + variablePartSize);
+                buf.writeByte(getFixedHeaderByte1(message.fixedHeader()));
+                writeVariableLengthInt(buf, variablePartSize);
+                buf.writeShort(message.variableHeader().messageId());
+                buf.writeBytes(propertiesBuf);
+
+                if (payload != null) {
+                    for (Short reasonCode : payload.unsubscribeReasonCodes()) {
+                        buf.writeByte(reasonCode);
+                    }
+                }
+
+                return buf;
+            } finally {
+                propertiesBuf.release();
+            }
+        } else {
+            return encodeMessageWithOnlySingleByteFixedHeaderAndMessageId(ctx.alloc(), message);
+        }
     }
 
     private static ByteBuf encodePublishMessage(
-            ByteBufAllocator byteBufAllocator,
+            ChannelHandlerContext ctx,
             MqttPublishMessage message) {
+        MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
         MqttFixedHeader mqttFixedHeader = message.fixedHeader();
         MqttPublishVariableHeader variableHeader = message.variableHeader();
         ByteBuf payload = message.payload().duplicate();
 
         String topicName = variableHeader.topicName();
-        byte[] topicNameBytes = encodeStringUtf8(topicName);
+        int topicNameBytes = utf8Bytes(topicName);
 
-        int variableHeaderBufferSize = 2 + topicNameBytes.length +
-                (mqttFixedHeader.qosLevel().value() > 0 ? 2 : 0);
-        int payloadBufferSize = payload.readableBytes();
-        int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
-        int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
+        ByteBuf propertiesBuf = encodePropertiesIfNeeded(mqttVersion,
+                ctx.alloc(),
+                message.variableHeader().properties());
 
-        ByteBuf buf = byteBufAllocator.buffer(fixedHeaderBufferSize + variablePartSize);
-        buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
-        writeVariableLengthInt(buf, variablePartSize);
-        buf.writeShort(topicNameBytes.length);
-        buf.writeBytes(topicNameBytes);
-        if (mqttFixedHeader.qosLevel().value() > 0) {
-            buf.writeShort(variableHeader.messageId());
+        try {
+            int variableHeaderBufferSize = 2 + topicNameBytes +
+                    (mqttFixedHeader.qosLevel().value() > 0 ? 2 : 0) + propertiesBuf.readableBytes();
+            int payloadBufferSize = payload.readableBytes();
+            int variablePartSize = variableHeaderBufferSize + payloadBufferSize;
+            int fixedHeaderBufferSize = 1 + getVariableLengthInt(variablePartSize);
+
+            ByteBuf buf = ctx.alloc().buffer(fixedHeaderBufferSize + variablePartSize);
+            buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
+            writeVariableLengthInt(buf, variablePartSize);
+            writeExactUTF8String(buf, topicName, topicNameBytes);
+            if (mqttFixedHeader.qosLevel().value() > 0) {
+                buf.writeShort(variableHeader.packetId());
+            }
+            buf.writeBytes(propertiesBuf);
+            buf.writeBytes(payload);
+
+            return buf;
+        } finally {
+            propertiesBuf.release();
         }
-        buf.writeBytes(payload);
+    }
 
-        return buf;
+    private static ByteBuf encodePubReplyMessage(ChannelHandlerContext ctx,
+                                          MqttMessage message) {
+        if (message.variableHeader() instanceof MqttPubReplyMessageVariableHeader) {
+            MqttFixedHeader mqttFixedHeader = message.fixedHeader();
+            MqttPubReplyMessageVariableHeader variableHeader =
+                    (MqttPubReplyMessageVariableHeader) message.variableHeader();
+            int msgId = variableHeader.messageId();
+
+            final ByteBuf propertiesBuf;
+            final boolean includeReasonCode;
+            final int variableHeaderBufferSize;
+            final MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
+            if (mqttVersion == MqttVersion.MQTT_5 &&
+                    (variableHeader.reasonCode() != MqttPubReplyMessageVariableHeader.REASON_CODE_OK ||
+                            !variableHeader.properties().isEmpty())) {
+                propertiesBuf = encodeProperties(ctx.alloc(), variableHeader.properties());
+                includeReasonCode = true;
+                variableHeaderBufferSize = 3 + propertiesBuf.readableBytes();
+            } else {
+                propertiesBuf = Unpooled.EMPTY_BUFFER;
+                includeReasonCode = false;
+                variableHeaderBufferSize = 2;
+            }
+
+            try {
+                final int fixedHeaderBufferSize = 1 + getVariableLengthInt(variableHeaderBufferSize);
+                ByteBuf buf = ctx.alloc().buffer(fixedHeaderBufferSize + variableHeaderBufferSize);
+                buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
+                writeVariableLengthInt(buf, variableHeaderBufferSize);
+                buf.writeShort(msgId);
+                if (includeReasonCode) {
+                    buf.writeByte(variableHeader.reasonCode());
+                }
+                buf.writeBytes(propertiesBuf);
+
+                return buf;
+            } finally {
+                propertiesBuf.release();
+            }
+        } else {
+            return encodeMessageWithOnlySingleByteFixedHeaderAndMessageId(ctx.alloc(), message);
+        }
     }
 
     private static ByteBuf encodeMessageWithOnlySingleByteFixedHeaderAndMessageId(
@@ -342,12 +515,11 @@ public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
         int msgId = variableHeader.messageId();
 
         int variableHeaderBufferSize = 2; // variable part only has a message id
-        
         ByteBuf payload = null;
         if (message.payload() != null) {
         	payload = ((ByteBuf)message.payload()).duplicate();
         	variableHeaderBufferSize += payload.readableBytes();
-		}
+		    }
         int fixedHeaderBufferSize = 1 + getVariableLengthInt(variableHeaderBufferSize);
         ByteBuf buf = byteBufAllocator.buffer(fixedHeaderBufferSize + variableHeaderBufferSize);
         buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
@@ -355,11 +527,53 @@ public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
         buf.writeShort(msgId);
         if (payload != null) {
         	buf.writeBytes(payload);
-		}
-
+		    }
         return buf;
     }
 
+    private static ByteBuf encodeReasonCodePlusPropertiesMessage(
+            ChannelHandlerContext ctx,
+            MqttMessage message) {
+        if (message.variableHeader() instanceof MqttReasonCodeAndPropertiesVariableHeader) {
+            MqttVersion mqttVersion = MqttCodecUtil.getMqttVersion(ctx);
+            MqttFixedHeader mqttFixedHeader = message.fixedHeader();
+            MqttReasonCodeAndPropertiesVariableHeader variableHeader =
+                    (MqttReasonCodeAndPropertiesVariableHeader) message.variableHeader();
+
+            final ByteBuf propertiesBuf;
+            final boolean includeReasonCode;
+            final int variableHeaderBufferSize;
+            if (mqttVersion == MqttVersion.MQTT_5 &&
+                    (variableHeader.reasonCode() != MqttReasonCodeAndPropertiesVariableHeader.REASON_CODE_OK ||
+                            !variableHeader.properties().isEmpty())) {
+                propertiesBuf = encodeProperties(ctx.alloc(), variableHeader.properties());
+                includeReasonCode = true;
+                variableHeaderBufferSize = 1 + propertiesBuf.readableBytes();
+            } else {
+                propertiesBuf = Unpooled.EMPTY_BUFFER;
+                includeReasonCode = false;
+                variableHeaderBufferSize = 0;
+            }
+
+            try {
+                final int fixedHeaderBufferSize = 1 + getVariableLengthInt(variableHeaderBufferSize);
+                ByteBuf buf = ctx.alloc().buffer(fixedHeaderBufferSize + variableHeaderBufferSize);
+                buf.writeByte(getFixedHeaderByte1(mqttFixedHeader));
+                writeVariableLengthInt(buf, variableHeaderBufferSize);
+                if (includeReasonCode) {
+                    buf.writeByte(variableHeader.reasonCode());
+                }
+                buf.writeBytes(propertiesBuf);
+
+                return buf;
+            } finally {
+                propertiesBuf.release();
+            }
+        } else {
+            return encodeMessageWithOnlySingleByteFixedHeader(ctx.alloc(), message);
+        }
+    }
+
     private static ByteBuf encodeMessageWithOnlySingleByteFixedHeader(
             ByteBufAllocator byteBufAllocator,
             MqttMessage message) {
@@ -371,6 +585,104 @@ public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
         return buf;
     }
 
+    private static ByteBuf encodePropertiesIfNeeded(MqttVersion mqttVersion,
+                                             ByteBufAllocator byteBufAllocator,
+                                             MqttProperties mqttProperties) {
+        if (mqttVersion == MqttVersion.MQTT_5) {
+            return encodeProperties(byteBufAllocator, mqttProperties);
+        }
+        return Unpooled.EMPTY_BUFFER;
+    }
+
+    private static ByteBuf encodeProperties(ByteBufAllocator byteBufAllocator,
+                                            MqttProperties mqttProperties) {
+        ByteBuf propertiesHeaderBuf = byteBufAllocator.buffer();
+        // encode also the Properties part
+        try {
+            ByteBuf propertiesBuf = byteBufAllocator.buffer();
+            try {
+                for (MqttProperties.MqttProperty property : mqttProperties.listAll()) {
+                    MqttProperties.MqttPropertyType propertyType =
+                            MqttProperties.MqttPropertyType.valueOf(property.propertyId);
+                    switch (propertyType) {
+                        case PAYLOAD_FORMAT_INDICATOR:
+                        case REQUEST_PROBLEM_INFORMATION:
+                        case REQUEST_RESPONSE_INFORMATION:
+                        case MAXIMUM_QOS:
+                        case RETAIN_AVAILABLE:
+                        case WILDCARD_SUBSCRIPTION_AVAILABLE:
+                        case SUBSCRIPTION_IDENTIFIER_AVAILABLE:
+                        case SHARED_SUBSCRIPTION_AVAILABLE:
+                            writeVariableLengthInt(propertiesBuf, property.propertyId);
+                            final byte bytePropValue = ((MqttProperties.IntegerProperty) property).value.byteValue();
+                            propertiesBuf.writeByte(bytePropValue);
+                            break;
+                        case SERVER_KEEP_ALIVE:
+                        case RECEIVE_MAXIMUM:
+                        case TOPIC_ALIAS_MAXIMUM:
+                        case TOPIC_ALIAS:
+                            writeVariableLengthInt(propertiesBuf, property.propertyId);
+                            final short twoBytesInPropValue =
+                                    ((MqttProperties.IntegerProperty) property).value.shortValue();
+                            propertiesBuf.writeShort(twoBytesInPropValue);
+                            break;
+                        case PUBLICATION_EXPIRY_INTERVAL:
+                        case SESSION_EXPIRY_INTERVAL:
+                        case WILL_DELAY_INTERVAL:
+                        case MAXIMUM_PACKET_SIZE:
+                            writeVariableLengthInt(propertiesBuf, property.propertyId);
+                            final int fourBytesIntPropValue = ((MqttProperties.IntegerProperty) property).value;
+                            propertiesBuf.writeInt(fourBytesIntPropValue);
+                            break;
+                        case SUBSCRIPTION_IDENTIFIER:
+                            writeVariableLengthInt(propertiesBuf, property.propertyId);
+                            final int vbi = ((MqttProperties.IntegerProperty) property).value;
+                            writeVariableLengthInt(propertiesBuf, vbi);
+                            break;
+                        case CONTENT_TYPE:
+                        case RESPONSE_TOPIC:
+                        case ASSIGNED_CLIENT_IDENTIFIER:
+                        case AUTHENTICATION_METHOD:
+                        case RESPONSE_INFORMATION:
+                        case SERVER_REFERENCE:
+                        case REASON_STRING:
+                            writeVariableLengthInt(propertiesBuf, property.propertyId);
+                            writeEagerUTF8String(propertiesBuf, ((MqttProperties.StringProperty) property).value);
+                            break;
+                        case USER_PROPERTY:
+                            final List<MqttProperties.StringPair> pairs =
+                                    ((MqttProperties.UserProperties) property).value;
+                            for (MqttProperties.StringPair pair : pairs) {
+                                writeVariableLengthInt(propertiesBuf, property.propertyId);
+                                writeEagerUTF8String(propertiesBuf, pair.key);
+                                writeEagerUTF8String(propertiesBuf, pair.value);
+                            }
+                            break;
+                        case CORRELATION_DATA:
+                        case AUTHENTICATION_DATA:
+                            writeVariableLengthInt(propertiesBuf, property.propertyId);
+                            final byte[] binaryPropValue = ((MqttProperties.BinaryProperty) property).value;
+                            propertiesBuf.writeShort(binaryPropValue.length);
+                            propertiesBuf.writeBytes(binaryPropValue, 0, binaryPropValue.length);
+                            break;
+                        default:
+                            //shouldn't reach here
+                            throw new EncoderException("Unknown property type: " + propertyType);
+                    }
+                }
+                writeVariableLengthInt(propertiesHeaderBuf, propertiesBuf.readableBytes());
+                propertiesHeaderBuf.writeBytes(propertiesBuf);
+
+                return propertiesHeaderBuf;
+            } finally {
+                propertiesBuf.release();
+            }
+        } catch (RuntimeException e) {
+            propertiesHeaderBuf.release();
+            throw e;
+        }
+    }
+
     private static int getFixedHeaderByte1(MqttFixedHeader header) {
         int ret = 0;
         ret |= header.messageType().value() << 4;
@@ -395,6 +707,42 @@ public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
         } while (num > 0);
     }
 
+    private static int nullableUtf8Bytes(String s) {
+        return s == null? 0 : utf8Bytes(s);
+    }
+
+    private static int nullableMaxUtf8Bytes(String s) {
+        return s == null? 0 : utf8MaxBytes(s);
+    }
+
+    private static void writeExactUTF8String(ByteBuf buf, String s, int utf8Length) {
+        buf.ensureWritable(utf8Length + 2);
+        buf.writeShort(utf8Length);
+        if (utf8Length > 0) {
+            final int writtenUtf8Length = reserveAndWriteUtf8(buf, s, utf8Length);
+            assert writtenUtf8Length == utf8Length;
+        }
+    }
+
+    private static void writeEagerUTF8String(ByteBuf buf, String s) {
+        final int maxUtf8Length = nullableMaxUtf8Bytes(s);
+        buf.ensureWritable(maxUtf8Length + 2);
+        final int writerIndex = buf.writerIndex();
+        final int startUtf8String = writerIndex + 2;
+        buf.writerIndex(startUtf8String);
+        final int utf8Length = s != null? reserveAndWriteUtf8(buf, s, maxUtf8Length) : 0;
+        buf.setShort(writerIndex, utf8Length);
+    }
+
+    private static void writeUnsafeUTF8String(ByteBuf buf, String s) {
+        final int writerIndex = buf.writerIndex();
+        final int startUtf8String = writerIndex + 2;
+        // no need to reserve any capacity here, already done earlier: that's why is Unsafe
+        buf.writerIndex(startUtf8String);
+        final int utf8Length = s != null? reserveAndWriteUtf8(buf, s, 0) : 0;
+        buf.setShort(writerIndex, utf8Length);
+    }
+
     private static int getVariableLengthInt(int num) {
         int count = 0;
         do {
@@ -404,7 +752,4 @@ public final class MqttEncoder extends MessageToMessageEncoder<MqttMessage> {
         return count;
     }
 
-    private static byte[] encodeStringUtf8(String s) {
-      return s.getBytes(CharsetUtil.UTF_8);
-    }
 }

broker/src/main/java/io/netty/handler/codec/mqtt/MqttFixedHeader.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -20,7 +20,7 @@ import io.netty.util.internal.ObjectUtil;
 import io.netty.util.internal.StringUtil;
 
 /**
- * See <a href="http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#fixed-header">
+ * See <a href="https://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#fixed-header">
  *     MQTTV3.1/fixed-header</a>
  */
 public final class MqttFixedHeader {

broker/src/main/java/io/netty/handler/codec/mqtt/MqttIdentifierRejectedException.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT

broker/src/main/java/io/netty/handler/codec/mqtt/MqttMessage.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -29,6 +29,17 @@ public class MqttMessage {
     private final Object payload;
     private final DecoderResult decoderResult;
 
+    // Constants for fixed-header only message types with all flags set to 0 (see
+    // https://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Table_2.2_-)
+    public static final MqttMessage PINGREQ = new MqttMessage(new MqttFixedHeader(MqttMessageType.PINGREQ, false,
+            MqttQoS.AT_MOST_ONCE, false, 0));
+
+    public static final MqttMessage PINGRESP = new MqttMessage(new MqttFixedHeader(MqttMessageType.PINGRESP, false,
+            MqttQoS.AT_MOST_ONCE, false, 0));
+
+    public static final MqttMessage DISCONNECT = new MqttMessage(new MqttFixedHeader(MqttMessageType.DISCONNECT, false,
+            MqttQoS.AT_MOST_ONCE, false, 0));
+
     public MqttMessage(MqttFixedHeader mqttFixedHeader) {
         this(mqttFixedHeader, null, null);
     }

broker/src/main/java/io/netty/handler/codec/mqtt/MqttMessageBuilders.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -15,10 +15,14 @@
  */
 package io.netty.handler.codec.mqtt;
 
+import static io.netty.util.internal.ObjectUtil.checkPositive;
+
 import io.netty.buffer.ByteBuf;
 import io.netty.buffer.Unpooled;
+import io.netty.util.CharsetUtil;
 
 import java.util.ArrayList;
+import java.util.Arrays;
 import java.util.List;
 
 public final class MqttMessageBuilders {
@@ -29,6 +33,7 @@ public final class MqttMessageBuilders {
         private MqttQoS qos;
         private ByteBuf payload;
         private int messageId;
+        private MqttProperties mqttProperties;
 
         PublishBuilder() {
         }
@@ -58,9 +63,15 @@ public final class MqttMessageBuilders {
             return this;
         }
 
+        public PublishBuilder properties(MqttProperties properties) {
+            this.mqttProperties = properties;
+            return this;
+        }
+
         public MqttPublishMessage build() {
             MqttFixedHeader mqttFixedHeader = new MqttFixedHeader(MqttMessageType.PUBLISH, false, qos, retained, 0);
-            MqttPublishVariableHeader mqttVariableHeader = new MqttPublishVariableHeader(topic, messageId);
+            MqttPublishVariableHeader mqttVariableHeader =
+                    new MqttPublishVariableHeader(topic, messageId, mqttProperties);
             return new MqttPublishMessage(mqttFixedHeader, mqttVariableHeader, Unpooled.buffer().writeBytes(payload));
         }
     }
@@ -73,13 +84,15 @@ public final class MqttMessageBuilders {
         private boolean hasUser;
         private boolean hasPassword;
         private int keepAliveSecs;
+        private MqttProperties willProperties = MqttProperties.NO_PROPERTIES;
         private boolean willFlag;
         private boolean willRetain;
         private MqttQoS willQos = MqttQoS.AT_MOST_ONCE;
         private String willTopic;
-        private String willMessage;
+        private byte[] willMessage;
         private String username;
         private byte[] password;
+        private MqttProperties properties = MqttProperties.NO_PROPERTIES;
 
         ConnectBuilder() {
         }
@@ -119,7 +132,16 @@ public final class MqttMessageBuilders {
             return this;
         }
 
+        /**
+         * @deprecated use {@link ConnectBuilder#willMessage(byte[])} instead
+         */
+        @Deprecated
         public ConnectBuilder willMessage(String willMessage) {
+            willMessage(willMessage == null ? null : willMessage.getBytes(CharsetUtil.UTF_8));
+            return this;
+        }
+
+        public ConnectBuilder willMessage(byte[] willMessage) {
             this.willMessage = willMessage;
             return this;
         }
@@ -129,6 +151,11 @@ public final class MqttMessageBuilders {
             return this;
         }
 
+        public ConnectBuilder willProperties(MqttProperties willProperties) {
+            this.willProperties = willProperties;
+            return this;
+        }
+
         public ConnectBuilder hasUser(boolean value) {
             this.hasUser = value;
             return this;
@@ -140,17 +167,31 @@ public final class MqttMessageBuilders {
         }
 
         public ConnectBuilder username(String username) {
-            this.hasUser = true;
+            this.hasUser = username != null;
             this.username = username;
             return this;
         }
 
+        /**
+         * @deprecated use {@link ConnectBuilder#password(byte[])} instead
+         */
+        @Deprecated
+        public ConnectBuilder password(String password) {
+            password(password == null ? null : password.getBytes(CharsetUtil.UTF_8));
+            return this;
+        }
+
         public ConnectBuilder password(byte[] password) {
-            this.hasPassword = true;
+            this.hasPassword = password != null;
             this.password = password;
             return this;
         }
 
+        public ConnectBuilder properties(MqttProperties properties) {
+            this.properties = properties;
+            return this;
+        }
+
         public MqttConnectMessage build() {
             MqttFixedHeader mqttFixedHeader =
                     new MqttFixedHeader(MqttMessageType.CONNECT, false, MqttQoS.AT_MOST_ONCE, false, 0);
@@ -164,9 +205,10 @@ public final class MqttMessageBuilders {
                             willQos.value(),
                             willFlag,
                             cleanSession,
-                            keepAliveSecs);
+                            keepAliveSecs,
+                            properties);
             MqttConnectPayload mqttConnectPayload =
-                    new MqttConnectPayload(clientId, willTopic, willMessage, username, password);
+                    new MqttConnectPayload(clientId, willProperties, willTopic, willMessage, username, password, null);
             return new MqttConnectMessage(mqttFixedHeader, mqttConnectVariableHeader, mqttConnectPayload);
         }
     }
@@ -175,36 +217,54 @@ public final class MqttMessageBuilders {
 
         private List<MqttTopicSubscription> subscriptions;
         private int messageId;
+        private MqttProperties properties;
 
         SubscribeBuilder() {
         }
 
         public SubscribeBuilder addSubscription(MqttQoS qos, String topic) {
-            if (subscriptions == null) {
-                subscriptions = new ArrayList<MqttTopicSubscription>(5);
-            }
+            ensureSubscriptionsExist();
             subscriptions.add(new MqttTopicSubscription(topic, qos));
             return this;
         }
 
+        public SubscribeBuilder addSubscription(String topic, MqttSubscriptionOption option) {
+            ensureSubscriptionsExist();
+            subscriptions.add(new MqttTopicSubscription(topic, option));
+            return this;
+        }
+
         public SubscribeBuilder messageId(int messageId) {
             this.messageId = messageId;
             return this;
         }
 
+        public SubscribeBuilder properties(MqttProperties properties) {
+            this.properties = properties;
+            return this;
+        }
+
         public MqttSubscribeMessage build() {
             MqttFixedHeader mqttFixedHeader =
                     new MqttFixedHeader(MqttMessageType.SUBSCRIBE, false, MqttQoS.AT_LEAST_ONCE, false, 0);
-            MqttMessageIdVariableHeader mqttVariableHeader = MqttMessageIdVariableHeader.from(messageId);
+            MqttMessageIdAndPropertiesVariableHeader mqttVariableHeader =
+                    new MqttMessageIdAndPropertiesVariableHeader(messageId, properties);
             MqttSubscribePayload mqttSubscribePayload = new MqttSubscribePayload(subscriptions);
             return new MqttSubscribeMessage(mqttFixedHeader, mqttVariableHeader, mqttSubscribePayload);
         }
+
+        private void ensureSubscriptionsExist() {
+            if (subscriptions == null) {
+                subscriptions = new ArrayList<MqttTopicSubscription>(5);
+            }
+        }
     }
 
     public static final class UnsubscribeBuilder {
 
         private List<String> topicFilters;
         private int messageId;
+        private MqttProperties properties;
 
         UnsubscribeBuilder() {
         }
@@ -222,21 +282,33 @@ public final class MqttMessageBuilders {
             return this;
         }
 
+        public UnsubscribeBuilder properties(MqttProperties properties) {
+            this.properties = properties;
+            return this;
+        }
+
         public MqttUnsubscribeMessage build() {
             MqttFixedHeader mqttFixedHeader =
                     new MqttFixedHeader(MqttMessageType.UNSUBSCRIBE, false, MqttQoS.AT_LEAST_ONCE, false, 0);
-            MqttMessageIdVariableHeader mqttVariableHeader = MqttMessageIdVariableHeader.from(messageId);
+            MqttMessageIdAndPropertiesVariableHeader mqttVariableHeader =
+                    new MqttMessageIdAndPropertiesVariableHeader(messageId, properties);
             MqttUnsubscribePayload mqttSubscribePayload = new MqttUnsubscribePayload(topicFilters);
             return new MqttUnsubscribeMessage(mqttFixedHeader, mqttVariableHeader, mqttSubscribePayload);
         }
     }
 
+    public interface PropertiesInitializer<T> {
+        void apply(T builder);
+    }
+
     public static final class ConnAckBuilder {
 
         private MqttConnectReturnCode returnCode;
         private boolean sessionPresent;
+        private MqttProperties properties = MqttProperties.NO_PROPERTIES;
+        private ConnAckPropertiesBuilder propsBuilder;
 
-        ConnAckBuilder() {
+        private ConnAckBuilder() {
         }
 
         public ConnAckBuilder returnCode(MqttConnectReturnCode returnCode) {
@@ -249,15 +321,402 @@ public final class MqttMessageBuilders {
             return this;
         }
 
+        public ConnAckBuilder properties(MqttProperties properties) {
+            this.properties = properties;
+            return this;
+        }
+
+        public ConnAckBuilder properties(PropertiesInitializer<ConnAckPropertiesBuilder> consumer) {
+            if (propsBuilder == null) {
+                propsBuilder = new ConnAckPropertiesBuilder();
+            }
+            consumer.apply(propsBuilder);
+            return this;
+        }
+
         public MqttConnAckMessage build() {
+            if (propsBuilder != null) {
+                properties = propsBuilder.build();
+            }
             MqttFixedHeader mqttFixedHeader =
                     new MqttFixedHeader(MqttMessageType.CONNACK, false, MqttQoS.AT_MOST_ONCE, false, 0);
             MqttConnAckVariableHeader mqttConnAckVariableHeader =
-                    new MqttConnAckVariableHeader(returnCode, sessionPresent);
+                    new MqttConnAckVariableHeader(returnCode, sessionPresent, properties);
             return new MqttConnAckMessage(mqttFixedHeader, mqttConnAckVariableHeader);
         }
     }
 
+    public static final class ConnAckPropertiesBuilder {
+        private String clientId;
+        private Long sessionExpiryInterval;
+        private int receiveMaximum;
+        private Byte maximumQos;
+        private boolean retain;
+        private Long maximumPacketSize;
+        private int topicAliasMaximum;
+        private String reasonString;
+        private final MqttProperties.UserProperties userProperties = new MqttProperties.UserProperties();
+        private Boolean wildcardSubscriptionAvailable;
+        private Boolean subscriptionIdentifiersAvailable;
+        private Boolean sharedSubscriptionAvailable;
+        private Integer serverKeepAlive;
+        private String responseInformation;
+        private String serverReference;
+        private String authenticationMethod;
+        private byte[] authenticationData;
+
+        public MqttProperties build() {
+            final MqttProperties props = new MqttProperties();
+            if (clientId != null) {
+                props.add(new MqttProperties.StringProperty(MqttProperties.MqttPropertyType.ASSIGNED_CLIENT_IDENTIFIER.value(),
+                        clientId));
+            }
+            if (sessionExpiryInterval != null) {
+                props.add(new MqttProperties.IntegerProperty(
+                        MqttProperties.MqttPropertyType.SESSION_EXPIRY_INTERVAL.value(), sessionExpiryInterval.intValue()));
+            }
+            if (receiveMaximum > 0) {
+                props.add(new MqttProperties.IntegerProperty(MqttProperties.MqttPropertyType.RECEIVE_MAXIMUM.value(), receiveMaximum));
+            }
+            if (maximumQos != null) {
+                props.add(new MqttProperties.IntegerProperty(MqttProperties.MqttPropertyType.MAXIMUM_QOS.value(), receiveMaximum));
+            }
+            props.add(new MqttProperties.IntegerProperty(MqttProperties.MqttPropertyType.RETAIN_AVAILABLE.value(), retain ? 1 : 0));
+            if (maximumPacketSize != null) {
+                props.add(new MqttProperties.IntegerProperty(MqttProperties.MqttPropertyType.MAXIMUM_PACKET_SIZE.value(),
+                        maximumPacketSize.intValue()));
+            }
+            props.add(new MqttProperties.IntegerProperty(MqttProperties.MqttPropertyType.TOPIC_ALIAS_MAXIMUM.value(),
+                    topicAliasMaximum));
+            if (reasonString != null) {
+                props.add(new MqttProperties.StringProperty(MqttProperties.MqttPropertyType.REASON_STRING.value(), reasonString));
+            }
+            props.add(userProperties);
+            if (wildcardSubscriptionAvailable != null) {
+                props.add(new MqttProperties.IntegerProperty(MqttProperties.MqttPropertyType.WILDCARD_SUBSCRIPTION_AVAILABLE.value(),
+                        wildcardSubscriptionAvailable ? 1 : 0));
+            }
+            if (subscriptionIdentifiersAvailable != null) {
+                props.add(new MqttProperties.IntegerProperty(MqttProperties.MqttPropertyType.SUBSCRIPTION_IDENTIFIER_AVAILABLE.value(),
+                        subscriptionIdentifiersAvailable ? 1 : 0));
+            }
+            if (sharedSubscriptionAvailable != null) {
+                props.add(new MqttProperties.IntegerProperty(MqttProperties.MqttPropertyType.SHARED_SUBSCRIPTION_AVAILABLE.value(),
+                        sharedSubscriptionAvailable ? 1 : 0));
+            }
+            if (serverKeepAlive != null) {
+                props.add(new MqttProperties.IntegerProperty(MqttProperties.MqttPropertyType.SERVER_KEEP_ALIVE.value(),
+                        serverKeepAlive));
+            }
+            if (responseInformation != null) {
+                props.add(new MqttProperties.StringProperty(MqttProperties.MqttPropertyType.RESPONSE_INFORMATION.value(),
+                        responseInformation));
+            }
+            if (serverReference != null) {
+                props.add(new MqttProperties.StringProperty(MqttProperties.MqttPropertyType.SERVER_REFERENCE.value(),
+                        serverReference));
+            }
+            if (authenticationMethod != null) {
+                props.add(new MqttProperties.StringProperty(MqttProperties.MqttPropertyType.AUTHENTICATION_METHOD.value(),
+                        authenticationMethod));
+            }
+            if (authenticationData != null) {
+                props.add(new MqttProperties.BinaryProperty(MqttProperties.MqttPropertyType.AUTHENTICATION_DATA.value(),
+                        authenticationData));
+            }
+
+            return props;
+        }
+
+        public ConnAckPropertiesBuilder sessionExpiryInterval(long seconds) {
+            this.sessionExpiryInterval = seconds;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder receiveMaximum(int value) {
+            this.receiveMaximum = checkPositive(value, "value");
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder maximumQos(byte value) {
+            if (value != 0 && value != 1) {
+                throw new IllegalArgumentException("maximum QoS property could be 0 or 1");
+            }
+            this.maximumQos = value;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder retainAvailable(boolean retain) {
+            this.retain = retain;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder maximumPacketSize(long size) {
+            this.maximumPacketSize = checkPositive(size, "size");
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder assignedClientId(String clientId) {
+            this.clientId = clientId;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder topicAliasMaximum(int value) {
+            this.topicAliasMaximum = value;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder reasonString(String reason) {
+            this.reasonString = reason;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder userProperty(String name, String value) {
+            userProperties.add(name, value);
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder wildcardSubscriptionAvailable(boolean value) {
+            this.wildcardSubscriptionAvailable = value;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder subscriptionIdentifiersAvailable(boolean value) {
+            this.subscriptionIdentifiersAvailable = value;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder sharedSubscriptionAvailable(boolean value) {
+            this.sharedSubscriptionAvailable = value;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder serverKeepAlive(int seconds) {
+            this.serverKeepAlive = seconds;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder responseInformation(String value) {
+            this.responseInformation = value;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder serverReference(String host) {
+            this.serverReference = host;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder authenticationMethod(String methodName) {
+            this.authenticationMethod = methodName;
+            return this;
+        }
+
+        public ConnAckPropertiesBuilder authenticationData(byte[] rawData) {
+            this.authenticationData = rawData.clone();
+            return this;
+        }
+    }
+
+    public static final class PubAckBuilder {
+
+        private int packetId;
+        private byte reasonCode;
+        private MqttProperties properties;
+
+        PubAckBuilder() {
+        }
+
+        public PubAckBuilder reasonCode(byte reasonCode) {
+            this.reasonCode = reasonCode;
+            return this;
+        }
+
+        public PubAckBuilder packetId(int packetId) {
+            this.packetId = packetId;
+            return this;
+        }
+
+        /**
+         * @deprecated use {@link PubAckBuilder#packetId(int)} instead
+         */
+        @Deprecated
+        public PubAckBuilder packetId(short packetId) {
+            return packetId(packetId & 0xFFFF);
+        }
+
+        public PubAckBuilder properties(MqttProperties properties) {
+            this.properties = properties;
+            return this;
+        }
+
+        public MqttMessage build() {
+            MqttFixedHeader mqttFixedHeader =
+                    new MqttFixedHeader(MqttMessageType.PUBACK, false, MqttQoS.AT_MOST_ONCE, false, 0);
+            MqttPubReplyMessageVariableHeader mqttPubAckVariableHeader =
+                    new MqttPubReplyMessageVariableHeader(packetId, reasonCode, properties);
+            return new MqttMessage(mqttFixedHeader, mqttPubAckVariableHeader);
+        }
+    }
+
+    public static final class SubAckBuilder {
+
+        private int packetId;
+        private MqttProperties properties;
+        private final List<MqttQoS> grantedQoses = new ArrayList<MqttQoS>();
+
+        SubAckBuilder() {
+        }
+
+        public SubAckBuilder packetId(int packetId) {
+            this.packetId = packetId;
+            return this;
+        }
+
+        /**
+         * @deprecated use {@link SubAckBuilder#packetId(int)} instead
+         */
+        @Deprecated
+        public SubAckBuilder packetId(short packetId) {
+            return packetId(packetId & 0xFFFF);
+        }
+
+        public SubAckBuilder properties(MqttProperties properties) {
+            this.properties = properties;
+            return this;
+        }
+
+        public SubAckBuilder addGrantedQos(MqttQoS qos) {
+            this.grantedQoses.add(qos);
+            return this;
+        }
+
+        public SubAckBuilder addGrantedQoses(MqttQoS... qoses) {
+            this.grantedQoses.addAll(Arrays.asList(qoses));
+            return this;
+        }
+
+        public MqttSubAckMessage build() {
+            MqttFixedHeader mqttFixedHeader =
+                    new MqttFixedHeader(MqttMessageType.SUBACK, false, MqttQoS.AT_MOST_ONCE, false, 0);
+            MqttMessageIdAndPropertiesVariableHeader mqttSubAckVariableHeader =
+                    new MqttMessageIdAndPropertiesVariableHeader(packetId, properties);
+
+            //transform to primitive types
+            int[] grantedQoses = new int[this.grantedQoses.size()];
+            int i = 0;
+            for (MqttQoS grantedQos : this.grantedQoses) {
+                grantedQoses[i++] = grantedQos.value();
+            }
+
+            MqttSubAckPayload subAckPayload = new MqttSubAckPayload(grantedQoses);
+            return new MqttSubAckMessage(mqttFixedHeader, mqttSubAckVariableHeader, subAckPayload);
+        }
+    }
+
+    public static final class UnsubAckBuilder {
+
+        private int packetId;
+        private MqttProperties properties;
+        private final List<Short> reasonCodes = new ArrayList<Short>();
+
+        UnsubAckBuilder() {
+        }
+
+        public UnsubAckBuilder packetId(int packetId) {
+            this.packetId = packetId;
+            return this;
+        }
+
+        /**
+         * @deprecated use {@link UnsubAckBuilder#packetId(int)} instead
+         */
+        @Deprecated
+        public UnsubAckBuilder packetId(short packetId) {
+            return packetId(packetId & 0xFFFF);
+        }
+
+        public UnsubAckBuilder properties(MqttProperties properties) {
+            this.properties = properties;
+            return this;
+        }
+
+        public UnsubAckBuilder addReasonCode(short reasonCode) {
+            this.reasonCodes.add(reasonCode);
+            return this;
+        }
+
+        public UnsubAckBuilder addReasonCodes(Short... reasonCodes) {
+            this.reasonCodes.addAll(Arrays.asList(reasonCodes));
+            return this;
+        }
+
+        public MqttUnsubAckMessage build() {
+            MqttFixedHeader mqttFixedHeader =
+                    new MqttFixedHeader(MqttMessageType.UNSUBACK, false, MqttQoS.AT_MOST_ONCE, false, 0);
+            MqttMessageIdAndPropertiesVariableHeader mqttSubAckVariableHeader =
+                    new MqttMessageIdAndPropertiesVariableHeader(packetId, properties);
+
+            MqttUnsubAckPayload subAckPayload = new MqttUnsubAckPayload(reasonCodes);
+            return new MqttUnsubAckMessage(mqttFixedHeader, mqttSubAckVariableHeader, subAckPayload);
+        }
+    }
+
+    public static final class DisconnectBuilder {
+
+        private MqttProperties properties;
+        private byte reasonCode;
+
+        DisconnectBuilder() {
+        }
+
+        public DisconnectBuilder properties(MqttProperties properties) {
+            this.properties = properties;
+            return this;
+        }
+
+        public DisconnectBuilder reasonCode(byte reasonCode) {
+            this.reasonCode = reasonCode;
+            return this;
+        }
+
+        public MqttMessage build() {
+            MqttFixedHeader mqttFixedHeader =
+                    new MqttFixedHeader(MqttMessageType.DISCONNECT, false, MqttQoS.AT_MOST_ONCE, false, 0);
+            MqttReasonCodeAndPropertiesVariableHeader mqttDisconnectVariableHeader =
+                    new MqttReasonCodeAndPropertiesVariableHeader(reasonCode, properties);
+
+            return new MqttMessage(mqttFixedHeader, mqttDisconnectVariableHeader);
+        }
+    }
+
+    public static final class AuthBuilder {
+
+        private MqttProperties properties;
+        private byte reasonCode;
+
+        AuthBuilder() {
+        }
+
+        public AuthBuilder properties(MqttProperties properties) {
+            this.properties = properties;
+            return this;
+        }
+
+        public AuthBuilder reasonCode(byte reasonCode) {
+            this.reasonCode = reasonCode;
+            return this;
+        }
+
+        public MqttMessage build() {
+            MqttFixedHeader mqttFixedHeader =
+                    new MqttFixedHeader(MqttMessageType.AUTH, false, MqttQoS.AT_MOST_ONCE, false, 0);
+            MqttReasonCodeAndPropertiesVariableHeader mqttAuthVariableHeader =
+                    new MqttReasonCodeAndPropertiesVariableHeader(reasonCode, properties);
+
+            return new MqttMessage(mqttFixedHeader, mqttAuthVariableHeader);
+        }
+    }
+
     public static ConnectBuilder connect() {
         return new ConnectBuilder();
     }
@@ -278,6 +737,26 @@ public final class MqttMessageBuilders {
         return new UnsubscribeBuilder();
     }
 
+    public static PubAckBuilder pubAck() {
+        return new PubAckBuilder();
+    }
+
+    public static SubAckBuilder subAck() {
+        return new SubAckBuilder();
+    }
+
+    public static UnsubAckBuilder unsubAck() {
+        return new UnsubAckBuilder();
+    }
+
+    public static DisconnectBuilder disconnect() {
+        return new DisconnectBuilder();
+    }
+
+    public static AuthBuilder auth() {
+        return new AuthBuilder();
+    }
+
     private MqttMessageBuilders() {
     }
 }

broker/src/main/java/io/netty/handler/codec/mqtt/MqttMessageFactory.java

@@ -5,7 +5,7 @@
  * version 2.0 (the "License"); you may not use this file except in compliance
  * with the License. You may obtain a copy of the License at:
  *
- *   http://www.apache.org/licenses/LICENSE-2.0
+ *   https://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
@@ -50,7 +50,8 @@ public final class MqttMessageFactory {
             case UNSUBACK:
                 return new MqttUnsubAckMessage(
                         mqttFixedHeader,
-                        (MqttMessageIdVariableHeader) variableHeader);
+                        (MqttMessageIdVariableHeader) variableHeader,
+                        (MqttUnsubAckPayload) payload);
 
             case UNSUBSCRIBE:
                 return new MqttUnsubscribeMessage(
@@ -65,17 +66,24 @@ public final class MqttMessageFactory {
                         (ByteBuf) payload);
 
             case PUBACK:
+                //Having MqttPubReplyMessageVariableHeader or MqttMessageIdVariableHeader
                 return new MqttPubAckMessage(mqttFixedHeader, (MqttMessageIdVariableHeader) variableHeader);
             case PUBREC:
             case PUBREL:
             case PUBCOMP:
+                //Having MqttPubReplyMessageVariableHeader or MqttMessageIdVariableHeader
                 return new MqttMessage(mqttFixedHeader, variableHeader);
 
             case PINGREQ:
             case PINGRESP:
-            case DISCONNECT:
                 return new MqttMessage(mqttFixedHeader);
 
+            case DISCONNECT:
+            case AUTH:
+                //Having MqttReasonCodeAndPropertiesVariableHeader
+                return new MqttMessage(mqttFixedHeader,
+                        variableHeader);
+
             default:
                 throw new IllegalArgumentException("unknown message type: " + mqttFixedHeader.messageType());
         }
@@ -85,5 +93,10 @@ public final class MqttMessageFactory {
         return new MqttMessage(null, null, null, DecoderResult.failure(cause));
     }
 
+    public static MqttMessage newInvalidMessage(MqttFixedHeader mqttFixedHeader, Object variableHeader,
+                                                Throwable cause) {
+        return new MqttMessage(mqttFixedHeader, variableHeader, null, DecoderResult.failure(cause));
+    }
+
     private MqttMessageFactory() { }
 }

broker/src/main/java/io/netty/handler/codec/mqtt/MqttMessageIdAndPropertiesVariableHeader.java

@@ -0,0 +1,52 @@
+/*
+ * Copyright 2020 The Netty Project
+ *
+ * The Netty Project licenses this file to you under the Apache License,
+ * version 2.0 (the "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at:
+ *
+ *   https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package io.netty.handler.codec.mqtt;
+
+import io.netty.util.internal.StringUtil;
+
+/**
+ * Variable Header containing, Packet Id and Properties as in MQTT v5 spec.
+ */
+public final class MqttMessageIdAndPropertiesVariableHeader extends MqttMessageIdVariableHeader {
+
+    private final MqttProperties properties;
+
+    public MqttMessageIdAndPropertiesVariableHeader(int messageId, MqttProperties properties) {
+        super(messageId);
+        if (messageId < 1 || messageId > 0xffff) {
+            throw new IllegalArgumentException("messageId: " + messageId + " (expected: 1 ~ 65535)");
+        }
+        this.properties = MqttProperties.withEmptyDefaults(properties);
+    }
+
+    public MqttProperties properties() {
+        return properties;
+    }
+
+    @Override
+    public String toString() {
+        return StringUtil.simpleClassName(this) + "[" +
+                "messageId=" + messageId() +
+                ", properties=" + properties +
+                ']';
+    }
+
+    @Override
+    MqttMessageIdAndPropertiesVariableHeader withDefaultEmptyProperties() {
+        return this;
+    }
+}