Fuzz testing documentation.

CMakeLists.txt

@@ -10,7 +10,6 @@ option (CC_MQTT5_USE_CCACHE "Use ccache on unix system" ON)
 option (CC_MQTT5_BUILD_UNIT_TESTS "Build unit tests" OFF)
 option (CC_MQTT5_BUILD_INTEGRATION_TESTS "Build integration tests which require MQTT broker on local port 1883." OFF)
 option (CC_MQTT5_WITH_DEFAULT_SANITIZERS "Build with sanitizers" OFF)
-option (CC_MQTT5_DISABLE_FALSE_POSITIVE_SANITIZERS "Disable sanitizers known as false positives" ${CC_MQTT5_WITH_DEFAULT_SANITIZERS})
 
 # Extra Configuration Variables
 # CC_MQTT5_CUSTOM_CLIENT_CONFIG_FILES - List of custom client configuration files

README.md

@@ -1,7 +1,14 @@
 # Overview
-This repository provides **single threaded**, **asynchronous**, **non-blocking**, easy to
-use, suitable for **embedded** platforms, well documented MQTT5 client library.
-It is completely generic and allows end application to have a complete control
+This repository provides well documented and easy to use MQTT5 client library.
+It is:
+
+- single threaded
+- asynchronous and non-blocking,
+- fuzz testable
+- compile time configurable (disable unneeded features and/or configure some data types)
+- suitable for **embedded** platforms with limited resources and/or without heap
+
+The library is completely generic and allows end application to have a complete control
 over the I/O link as well as perform extra manipulation on the exchanged
 raw data (such as encryption or extra framing).
 
@@ -43,6 +50,16 @@ the events loop and manage network connection(s)).
 Detailed instructions on how to build and install all the components can be
 found in [doc/BUILD.md](doc/BUILD.md) file.
 
+# How to Fuzz Test
+The provided MQTT5 client library as well as its dependencies from the
+[CommsChampion Ecosystem](https://commschamp.github.io/) have been designed with
+reliability in mind and to be able to safely handle malformed data as well as
+withstand unexpected behaviour from a MQTT broker. In order to
+verify the library's reliability it is highly recommended to perform
+[AFL++](https://github.com/AFLplusplus/AFLplusplus) based fuzz testing.
+The detailed instruction on how to fuzz test the
+library can be found in [doc/fuzz_test.md](doc/fuzz_test.md) file.
+
 # Branching Model
 This repository will follow the
 [Successful Git Branching Model](http://nvie.com/posts/a-successful-git-branching-model/).

client/afl_fuzz/Generator.cpp

@@ -79,6 +79,7 @@ void Generator::processData(const std::uint8_t* buf, unsigned bufLen)
 void Generator::handle(const Mqtt5ConnectMsg& msg)
 {
     m_logger.infoLog() << "Processing " << msg.name() << "\n";
+    m_connected = false;
     PropsHandler propsHandler;
     for (auto& p : msg.field_properties().value()) {
         p.currentFieldExec(propsHandler);
@@ -196,17 +197,21 @@ void Generator::handle(const Mqtt5AuthMsg& msg)
         return;
     }
 
+    assert(msg.field_reasonCode().doesExist());
+    assert(msg.field_properties().doesExist());
     PropsHandler propsHandler;
-    for (auto& p : m_cachedConnect->field_properties().value()) {
+    for (auto& p : msg.field_properties().field().value()) {
         p.currentFieldExec(propsHandler);
     }  
 
 
     if (msg.field_reasonCode().field().value() == Mqtt5AuthMsg::Field_reasonCode::Field::ValueType::ReAuth) {
+        m_logger.infoLog() << "Re-authentication request\n";
         sendAuth(propsHandler, Mqtt5AuthMsg::Field_reasonCode::Field::ValueType::ContinueAuth);
         return;
     }
 
+    m_logger.infoLog() << "Completing re-authentication\n";
     assert(msg.field_reasonCode().field().value() == Mqtt5AuthMsg::Field_reasonCode::Field::ValueType::ContinueAuth);
     sendAuth(propsHandler, Mqtt5AuthMsg::Field_reasonCode::Field::ValueType::Success);
 }
@@ -286,6 +291,13 @@ void Generator::sendConnack(const Mqtt5ConnectMsg& msg, const PropsHandler& prop
         auto& valueField = propBundle.field_value();        
         valueField.setValue(propsHandler.m_topicAliasMax);        
         m_topicAliasLimit = propsHandler.m_topicAliasMax;
+    }  
+
+    if (!propsHandler.m_authMethod.empty()) {
+        auto& propVar = addProp(propsField);
+        auto& propBundle = propVar.initField_authMethod();
+        auto& valueField = propBundle.field_value();        
+        valueField.setValue(propsHandler.m_authMethod);        
     }    
 
     sendMessage(outMsg);
@@ -319,25 +331,34 @@ void Generator::sendPublish(const std::string& topic, unsigned qos)
 void Generator::sendAuth(const PropsHandler& propsHandler, Mqtt5AuthMsg::Field_reasonCode::Field::ValueType reasonCode)
 {
     Mqtt5AuthMsg outMsg;
-    outMsg.field_reasonCode().field().setValue(reasonCode);
+    do {
+        if (reasonCode == Mqtt5AuthMsg::Field_reasonCode::Field::ValueType::Success) {
+            break;
+        }
 
-    auto& propsField = outMsg.field_properties().field();
-    if (!propsHandler.m_authMethod.empty()) {
-        auto& propVar = addProp(propsField);
-        auto& propBundle = propVar.initField_authMethod();
-        auto& valueField = propBundle.field_value();        
-        valueField.setValue(propsHandler.m_authMethod);        
-    }
+        outMsg.field_reasonCode().setExists();
+        outMsg.field_reasonCode().field().setValue(reasonCode);
+        outMsg.field_properties().setExists();
 
-    if (!propsHandler.m_authData.empty()) {
-        auto& propVar = addProp(propsField);
-        auto& propBundle = propVar.initField_authData();
-        auto& valueField = propBundle.field_value();        
-        valueField.setValue(propsHandler.m_authData);     
-        if ((valueField.value().size() & 0x1) != 0) {
-            valueField.value().push_back('1');
+
+        auto& propsField = outMsg.field_properties().field();
+        if (!propsHandler.m_authMethod.empty()) {
+            auto& propVar = addProp(propsField);
+            auto& propBundle = propVar.initField_authMethod();
+            auto& valueField = propBundle.field_value();        
+            valueField.setValue(propsHandler.m_authMethod);        
         }
-    }    
+
+        if (!propsHandler.m_authData.empty()) {
+            auto& propVar = addProp(propsField);
+            auto& propBundle = propVar.initField_authData();
+            auto& valueField = propBundle.field_value();        
+            valueField.setValue(propsHandler.m_authData);     
+            if ((valueField.value().size() & 0x1) != 0) {
+                valueField.value().push_back('1');
+            }
+        }    
+    } while (false);
 
     sendMessage(outMsg);
 }

client/lib/src/op/ConnectOp.cpp

@@ -774,10 +774,14 @@ void ConnectOp::handle(AuthMsg& msg)
         return; // Disconnect is sent and op is competed
     }        
 
-    if ((m_authMethod.empty()) || 
-        (msg.field_reasonCode().isMissing()) ||
+    if (m_authMethod.empty()) {
+        errorLog("The connect hasn't used extended authentication, invalid AUTH message received");
+        return; // Disconnect is sent and op is competed
+    }    
+
+    if ((msg.field_reasonCode().isMissing()) ||
         (msg.field_reasonCode().field().value() != AuthMsg::Field_reasonCode::Field::ValueType::ContinueAuth)) {
-        errorLog("Invalid reason code received received in AUTH message");
+        errorLog("Invalid reason code received received in AUTH message during connection");
         return; // Disconnect is sent and op is competed
     }
 

client/lib/test/unit/UnitTestReauth.th

@@ -14,6 +14,7 @@ public:
     void test5();
     void test6();
     void test7();
+    void test8();
 
 private:
     virtual void setUp() override
@@ -587,4 +588,73 @@ void UnitTestReauth::test7()
     // TS_ASSERT_EQUALS(respInfo.m_status, CC_Mqtt5AsyncOpStatus_BrokerDisconnected);
     // unitTestPopReauthResponseInfo();
     // TS_ASSERT(unitTestIsDisconnected());                      
+}
+
+void UnitTestReauth::test8()
+{
+    // Simple reauth from client when broker responds without reason code.
+    auto* client = unitTestAllocAndInitClient(true);
+    TS_ASSERT_DIFFERS(client, nullptr);
+
+    auto basicConfig = CC_Mqtt5ConnectBasicConfig();
+    unitTestConnectInitConfigBasic(&basicConfig);
+    basicConfig.m_clientId = __FUNCTION__;
+    basicConfig.m_cleanStart = true;
+
+    const std::string ConnectAuthMethod = "AuthMethod";
+    const UnitTestData ConnectAuthData = {0x1, 0x2, 0x3, 0x5, 0xa};
+
+    auto extraConfig = CC_Mqtt5ConnectExtraConfig();
+    unitTestConnectInitConfigExtra(&extraConfig);
+    extraConfig.m_requestProblemInfo = true;     
+
+    auto connectAuthConfig = CC_Mqtt5AuthConfig();
+    unitTestConnectInitConfigAuth(&connectAuthConfig);
+    connectAuthConfig.m_authMethod = ConnectAuthMethod.c_str();
+    connectAuthConfig.m_authData = &ConnectAuthData[0];
+    connectAuthConfig.m_authDataLen = static_cast<decltype(connectAuthConfig.m_authDataLen)>(ConnectAuthData.size());
+
+    unitTestPerformConnect(client, &basicConfig, nullptr, &extraConfig, &connectAuthConfig, nullptr);
+    TS_ASSERT(unitTestIsConnected(client));
+    TS_ASSERT(!unitTestHasSentMessage());
+
+    auto* reauth = unitTestReauthPrepare(client, nullptr);
+    TS_ASSERT_DIFFERS(reauth, nullptr);
+
+    const UnitTestData ReAuthData = {0x1, 0x2, 0x3, 0x5, 0x6};
+    auto ec = unitTestConfigReauth(reauth, std::string(), ReAuthData);
+    TS_ASSERT_EQUALS(ec, CC_Mqtt5ErrorCode_Success);    
+
+    ec = unitTestSendReauth(reauth);
+    TS_ASSERT_EQUALS(ec, CC_Mqtt5ErrorCode_Success);    
+
+    auto sentMsg = unitTestGetSentMessage();
+    TS_ASSERT(sentMsg);
+    TS_ASSERT_EQUALS(sentMsg->getId(), cc_mqtt5::MsgId_Auth);
+    auto* authMsg = dynamic_cast<UnitTestAuthMsg*>(sentMsg.get());
+    TS_ASSERT_DIFFERS(authMsg, nullptr);
+    TS_ASSERT(authMsg->field_reasonCode().doesExist());
+    TS_ASSERT_EQUALS(authMsg->field_reasonCode().field().value(), UnitTestAuthMsg::Field_reasonCode::Field::ValueType::ReAuth);
+    TS_ASSERT(authMsg->field_properties().doesExist());
+
+    UnitTestPropsHandler propsHandler;
+    for (auto& p : authMsg->field_properties().field().value()) {
+        p.currentFieldExec(propsHandler);
+    }
+
+    TS_ASSERT_DIFFERS(propsHandler.m_authMethod, nullptr);
+    TS_ASSERT_EQUALS(propsHandler.m_authMethod->field_value().value(), ConnectAuthMethod);
+
+    TS_ASSERT_DIFFERS(propsHandler.m_authData, nullptr);
+    TS_ASSERT_EQUALS(propsHandler.m_authData->field_value().value(), ReAuthData);    
+
+    TS_ASSERT(!unitTestIsReauthComplete());
+
+    UnitTestAuthMsg brokerAuth;
+    unitTestReceiveMessage(brokerAuth);
+    TS_ASSERT(unitTestIsReauthComplete());
+
+    auto& respInfo = unitTestReauthResponseInfo();
+    TS_ASSERT_EQUALS(respInfo.m_status, CC_Mqtt5AsyncOpStatus_Complete);
+    unitTestPopReauthResponseInfo();
 }

cmake/Compile.cmake

@@ -41,11 +41,6 @@ macro (cc_mqttsn_compile)
         )
     endif()
 
-    if (CC_MQTT5_DISABLE_FALSE_POSITIVE_SANITIZERS)
-        list (APPEND extra_flags_list
-            -fno-sanitize-address-use-after-scope)        
-    endif ()    
-
     string(REPLACE ";" " " extra_flags "${extra_flags_list}")
     set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${extra_flags}")
 endmacro()

doc/BUILD.md

@@ -1,7 +1,7 @@
 # How to Build
 This project uses [CMake](https://cmake.org) cross-platform build system to
 generate required build files native to the platform. Please refer to the 
-main [CMakeLists.txt](../CMakeLists.txt) file for the info on available configuraiton options and
+main [CMakeLists.txt](../CMakeLists.txt) file for the info on available configuration options and
 variables.
 
 ## External Dependencies

doc/fuzz.test.md

@@ -0,0 +1,121 @@
+# Fuzz Testing Application
+This repository provides a fuzz testing application called **cc_mqtt5_client_afl_fuzz**.
+It expects to receive a sequence of the raw bytes from the **STDIN** and intended to
+be used with fuzz testing tools like [AFL++](https://github.com/AFLplusplus/AFLplusplus).
+
+The **cc_mqtt5_client_afl_fuzz** application will try to perform the following steps:
+
+1. Initialize the library.
+2. Perform "connect" operation and wait for acknowledgement (**CONNACK**).
+3. Subscribe to all topics provided via command line arguments.
+4. Perform re-authentication if "authentication method" is provided via command line arguments.
+5. Echo back all the received messages from the broker, and proceed to the next step only after
+   the necessary number of messages has been published (configured via command line).
+6. Unsubscribe from all the previously subscribed topics
+7. Gracefully disconnect from the broker
+8. Return to step 1.
+
+Due to the nature of the fuzz testing the intended workflow from above won't be
+completed as intended due to detected malformed packets, protocol errors, and/or
+received **DISCONNECT** messages. When such scenario is detected in any of the
+steps the application will return to step 1 to try from the beginning on the remaining
+input.
+
+# How to Build
+To better understand the build instructions below please read the
+[BUILD.md](BUILD.md) document as well as refer to the
+main [CMakeLists.txt](../CMakeLists.txt) file for the info on available configuration options and
+variables.
+
+If you're not familiar with the [AFL++](https://github.com/AFLplusplus/AFLplusplus) it
+is highly recommended to read through its [instructions](https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/fuzzing_in_depth.md)
+to properly understand what is being done.
+
+To enable the fuzz testing application use **CC_MQTT5_CLIENT_AFL_FUZZ** cmake option. When
+instrumenting the binaries for the fuzz testing other applications are probably not needed
+and their build can be disabled using the **CC_MQTT5_CLIENT_APPS** cmake option.
+
+Also remember to properly set instrumenting compilers provided by the [AFL++](https://github.com/AFLplusplus/AFLplusplus).
+
+```
+CC=afl-clang-lto CXX=afl-clang-lto++ cmake /path/to/src -DCC_MQTT5_CLIENT_AFL_FUZZ=ON -DCC_MQTT5_CLIENT_APPS=OFF ...
+```
+
+It is also highly recommended to use "Debug" build to enable all the assertions.
+```
+CC=afl-clang-lto CXX=afl-clang-lto++ cmake ... -DCMAKE_BUILD_TYPE=Debug ...
+```
+
+After the "Debug" build is tested for several days without showing any crashes it also
+beneficial to rebuild the fuzzing application as a "Release" and re-run it on the same produced corpus
+(passing "-i-" to afl-fuzz) to verify that there are no unexpected pitfalls for the "Release" version.
+
+Due to the fact that [AFL++](https://github.com/AFLplusplus/AFLplusplus) compilers
+receive their configuration via environment variable before the build it is necessary
+to disable "ccache" usage in case the sources are going to be built multiple times
+with different configurations.
+
+```
+CC=afl-clang-lto CXX=afl-clang-lto++ cmake ... -DCC_MQTT5_USE_CCACHE=OFF ...
+```
+
+Enable required sanitizers before the build
+```
+export AFL_USE_ASAN=1
+export AFL_USE_UBSAN=1
+```
+
+**WARNING**: It has been noticed when too many sanitizers are enabled at the same time the
+target either fails to compile or fails to produce proper output
+("Illegal instruction" is reported) when some failure happens.
+
+As the final stage, build the fuzzing application
+```
+cmake --build . --target install
+```
+
+Note that in case [custom](custom_client_build.md) client libraries are built, the
+fuzzing application will be built for each such library and the application name will
+reflect the custom name selected for the library.
+
+The typical build sequence may look like this:
+```
+cd /path/to/cc.mqtt5.libs
+mkdir build
+cd build
+CC=afl-clang-lto CXX=afl-clang-lto++ cmake .. -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=install \
+    -DCMAKE_PREFIX_PATH=/path/to/comms/install\;/path/to/cc.mqtt5.generated/install \
+    -DCC_MQTT5_CLIENT_AFL_FUZZ=ON -DCC_MQTT5_CLIENT_APPS=OFF -DCC_MQTT5_USE_CCACHE=OFF
+export AFL_USE_ASAN=1
+export AFL_USE_UBSAN=1
+cmake --build . --target install --parallel 8
+```
+
+# How to Fuzz Test
+In order to start fuzz testing the [AFL++](https://github.com/AFLplusplus/AFLplusplus) requires
+creation of the input corpus. Please use "-h" option to list the available command line parameters:
+```
+./install/bin/cc_mqtt5_client_afl_fuzz -h
+```
+Note the presence of the "-g" option which can be used to generate a valid input sequence
+to perform a full single iteration of the **cc_mqtt5_client_afl_fuzz** described
+earlier. If you intend to use some extra command line arguments in the actual
+fuzz testing, provide them when generating the input sequence as well.
+
+For example
+```
+mkdir -p /path/to/fuzz/input
+mkdir -p /path/to/fuzz/output
+./install/bin/cc_mqtt5_client_afl_fuzz -g /path/to/fuzz/input/1.bin --auth-method myauth --min-pub-count 5
+```
+
+Now when the input corpus is created it is possible to actually start fuzz testing. Use the
+same command line option as ones used for the generation of the input (excluding the "-g" with parameter of course).
+```
+afl-fuzz -i /path/to/fuzz/input -o /path/to/fuzz/output -- ./install/bin/cc_mqtt5_client_afl_fuzz --auth-method myauth --min-pub-count 5
+```
+
+Note that "afl-fuzz" may request to change your machine configuration before being able to fuzz test.
+
+In case fuzz testing reports any crash please open an issue for this project reporting
+a build configuration and attaching the input file(s) that caused the crash.