
Description
One of the most prominent hurdles in operating manufacturing plants is to regulate the enervating heat produced by industrial processes. Therefore, an efficient industrial cooling system is the fulcrum of managing a profitable, sustainable, and robust industrial facility. There are various cooling system designs and structures to provide versatile heat regulation for different business requirements. For instance, natural draft cooling benefits the density discrepancy between the produced hot air and ambient fresh air, mechanical draft cooling utilizes sprayed hot water to transfer heat from a condenser to dry air, and water cooling uses cold water directly to reduce the targeted component temperature. When all cooling requirements are considered, water cooling options are still the most popular and budget-friendly cooling systems applicable to various cooling scenarios, including but not limited to condominiums, office buildings, and industrial facilities. Water cooling systems, also known as hydronic cooling systems, are mainly considered as the most adaptable and advantageous HVAC (heating, ventilation, and air conditioning) systems utilizing water to transfer heat from one location to another[^1]. Since hydronic HVAC systems use water to absorb and transfer heat, they are more energy efficient as compared to air-based systems since water has a higher thermal capacity. According to the applied heat transfer method and water source, water-based cooling systems provide design flexibility with low-maintenance. Nonetheless, despite the advantages of relying on water as a coolant, water-based HVAC systems still require regular inspection and maintenance to retain peak condition and avert pernicious cooling aberrations deteriorating heat regulation for industrial facilities, office buildings, or houses. Since water-based cooling equipment is a part of various demanding industrial applications[^2], including but not limited to chemicals or petrochemicals, welding, medical, pharmaceutical, automotive, data centers, and metalworking, maintaining consistent and reliable heat transfer is essential to sustain profitable business growth. Thus, to reduce production costs and increase manufacturing efficiency, mechanics should examine each cooling component painstakingly and regularly. Since hydronic HVAC systems can be intricate and multifaceted depending on the application requirements, there are plentiful malfunctions that can affect cooling efficiency and heat transfer capacity, resulting in catastrophic production downtime for industrial processes. For instance, chillers using metal tubes (copper or carbon steel) to circulate water are susceptible to corrosion and abrasion, leading to leaks and component failures. Accumulating sediment or particulates in the complex tubing systems can corrode or clog pipes, leading to inadequate heat transfer. Or, perforce, neglected electronic components can degrade and fail due to prolonged wear and tear, leading to inconsistent cooling results. Unfortunately, these HVAC system malfunctions not only deteriorate industrial process sustainability but also engender hazardous environmental impacts due to high energy loss. Water-based or not, an installed HVAC system accounts for up to 50% of the total energy consumption of an establishment, surpassing the total energy consumption of lighting, elevators, and office equipment[^3]. Thus, an unnoticed abnormality can multiply energy consumption while the HVAC system tries to compensate for the heat transfer loss. Furthermore, since HVAC systems are tightly coupled systems and operate with protracted lag and inertia, they are vulnerable even to minuscule abnormalities due to the ripple effect of a single equipment failure, whether a capacitor, pipe, or gasket. Relevant data indicates that the amount of energy waste caused by a malfunctioning cooling system and faulty control accounts for about 15%–30% of the total energy consumption of studied facilities. Thus, by running a malfunctioning cooling system, buildings became profligate energy consumers, resulting in harsh energy production demands causing excess carbon and methane emitted into the atmosphere. Therefore, applying real-time (automated) malfunction diagnosis to HVAC systems can abate excessive energy consumption and improve energy efficiency leading to savings ranging from 5% to 30% [^3]. In addition to preventing energy loss, automated HVAC fault detection can extend equipment lifespan, avoid profit loss, and provide stable heat transfer during industrial processes. In that regard, automated malfunction detection also obviates exorbitant overhaul processes due to prolonged negligence, leading to a nosedive in production quality. After perusing recent research papers on detecting component failures to automate HVAC maintenance, I noticed that there are no practical applications focusing on identifying component abnormalities of intricate water-based HVAC systems to diagnose consecutive thermal cooling malfunctions before instigating hazardous effects on both production quality and the environment. Hence, I decided to build a versatile multi-model AIoT device to detect anomalous sound emanating from cooling fans via a neural network model and to diagnose consecutive thermal cooling malfunctions based on specifically produced thermal images via a visual anomaly detection model. In addition to AI-driven features, I decided to develop a capable and feature-rich web application (dashboard) to improve user experience and make data transfer easier between development boards. As I started to work on developing my AI-powered device features, I realized that no available open-source data sets were fulfilling the purpose of multi-model HVAC malfunction diagnosis. Thus, since I did not have the resources to collect data from an industrial-level HVAC system, I decided to build a simplified HVAC system simulating the required component failures for data collection and in-field model testing. I got heavily inspired by PC (computer) water cooling systems while designing my simplified HVAC system. Similar to a closed-loop PC water cooling design, I built my system by utilizing a water pump, plastic tubing, an aluminum radiator, and aluminum blocks. As for the coolant reservoir, I decided to design a custom one and print the parts with my 3D printer. Nonetheless, since I decided to produce a precise thermal image by scanning cooling components, I still needed an additional mechanism to move a thermal camera on the targeted components — aluminum blocks. Thus, I decided to design a fully 3D-printable CNC router with the thermal camera container head to position the thermal camera, providing an automatic homing sequence. My custom CNC router is controlled by Arduino Nano and consists of a 28BYJ-48 stepper motor, GT2 pulleys, a timing belt, and gear clamps. While producing thermal images and running the visual anomaly detection model, I simply added an aquarium heater to the closed-water loop in order to instantiate aluminum block cooling malfunctions. As mentioned earlier, to provide full-fledged AIoT features with seamless integration and simplify complex data transfer procedures between development boards while constructing separate data sets and running multiple models, I decided to develop a versatile web application (dashboard) from scratch. To briefly summarize, the web dashboard can receive audio buffers via HTTP POST requests, save audio samples by given classes, communicate with the Particle Cloud to obtain variables or make Particle boards register them, produce thermal images from thermal imaging buffers to store image samples, and run the visual anomaly detection model on the generated thermal images. In the following tutorial, you can inspect all web dashboard features in detail. Since this is a multi-model AI-oriented project, I needed to construct two different data sets and train two separate machine learning models in order to build a capable device. First, I focused on constructing a valid audio data set for detecting anomalous sound originating from cooling fans. Since XIAO ESP32C6 is a compact and high-performance IoT development board providing 512KB SRAM and 4 MB Flash, I decided to utilize XIAO ESP32C6 to collect audio samples and run my neural network model for anomalous sound detection. To generate fast and accurate audio samples (buffers), I decided to use a Fermion I2S MEMS microphone. Also, I connected an SSD1306 OLED display and four control buttons to program a feature-rich on-device user interface. After collecting an audio sample, XIAO ESP32C6 transfers it to the web dashboard for data collection. As mentioned earlier, I designed my custom CNC router based on Arduino Nano due to its operating voltage. To provide seamless device operations, XIAO ESP32C6 communicates with Arduino Nano to move the thermal camera container head. After completing constructing my audio data set, I built my neural network model (Audio MFE) with Edge Impulse to detect sound-based cooling fan abnormalities. Audio MFE models employ a non-linear scale in the frequency domain, called Mel-scale, and perform well on audio data, mostly for non-voice recognition. Since Edge Impulse is nearly compatible with all microcontrollers and development boards, I have not encountered any issues while uploading and running my Audio MFE model on XIAO ESP32C6. As labels, I simply differentiated the collected audio samples by the cooling fan failure presence:- normal
- defective
- no anomaly
- anomaly

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50
Demonstration Videos
Since this HVAC malfunction detection device performs various interconnected features between different development boards and the web application (dashboard), I needed to compartmentalize consecutive processes and describe functions under the same code file separately to provide comprehensive step-by-step instructions. Thus, I highly recommend watching the demonstration videos before scrutinizing the tutorial steps to effortlessly grasp device capabilities that might look complicated in the instructions.Step 0: A brief introduction to device features and structure
As my projects became more intricate due to complex designs and multiple development board integrations, I decided to create concise illustrations to improve my tutorials, visualize the special tasks associated with each development board, and delineate the complicated data transfer procedures between different boards or complementary applications. Thus, before proceeding with the following steps, I highly recommend inspecting these illustrations to comprehend the device features and structure better. Note: Since downsizing these high-resolution illustrations is necessary for loading the tutorial page, I noticed the text on the illustrations lost legibility. Therefore, I also added the original image files below for further inspection.
51

52

53

54

55

56
Step 1.a: Designing and soldering the Kyogre-inspired PCB
As I was developing device features, I noticed that I needed to run different data collection procedures and machine learning models simultaneously. Therefore, I decided to create two separate PCB designs to run the required tasks conclusively. Since I wanted my PCB designs to represent the equilibrium of cooling fan failures and thermal (heat) malfunctions, I got inspired by two ancient rival Pokémon — Kyogre and Groudon. Their legendary fights depict the epitome of the conflict between water cooling and exuberating heat :) Before prototyping my Kyogre-inspired PCB design, I inspected the detailed pin reference of XIAO ESP32C6 and needed to prepare components requiring soldering for programming. Aside from the other components, I employed a soldering station to solder jumper wires to each leg of the micro switch in order to make it compatible with the custom switch connector on the CNC router, which will be explained in the following steps.
57

58

59

60

61

62

63

64

65

66

67

68

69

70

71
Step 1.a.1: Making connections and adjustments on the Kyogre PCB

72
Step 1.b: Designing and soldering the Groudon-inspired PCB
Before prototyping my Groudon-inspired PCB design, I inspected the detailed pin reference of Particle Photon 2 and needed to prepare components requiring soldering for programming. Then, I checked the wireless (Wi-Fi) and cloud communication quality between Photon 2, the Particle Cloud, and the web dashboard (application) while transferring and receiving data packets.
73

74

75

76

77

78

79

80

81
Step 1.b.1: Making connections and adjustments on the Groudon PCB

82
Step 2.a: Designing and printing the aluminum radiator mounts and the Kyogre PCB encasement
Since I focused on building a versatile and accessible AI-driven device that identifies the faulty cooling components via anomalous sound detection and diagnoses ensuing thermal cooling malfunctions via visual anomaly detection based on thermal images, I decided to design complementary 3D-printable parts that improve the robustness, compatibility, and capabilities of the device considering harsh operating conditions of industrial plants. First, I wanted to fix the large aluminum radiator position and integrate the Kyogre PCB as close as possible to the radiator. Thus, I designed these parts:- the main body of the right radiator mount,
- the main body of the left radiator mount,
- two tilted snap-fit joints perfectly sized for the radiator,
- four special legs (back and front) supporting the radiator mounts,
- the unique PCB encasement derived from the Kyogre PCB outline,
- the PCB encasement connector providing a buckle-shaped joint interlocking with the right radiator mount.

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102
- ePLA-Matte Tangerine

103
Step 2.a.1: Assembling the 3D-printed components
After printing all 3D models related to the aluminum radiator, I started to combine the radiator mount parts via M3 screws through the assembly-ready screw holes. Then, I fastened the unique Kyogre PCB encasement to the complementary PCB connector via M3 screws. Since the PCB connector is compatible with the right radiator mount via its buckle-shaped snap-fit joint, I was able to interlock the PCB connector with the right mount body effortlessly. Although I applied hot glue between parts while affixing them via M3 screws, it was still not enough to build a production-ready device, especially considering the harsh operating conditions of industrial HVAC systems. Thus, I employed a well-known injection molding technique to make the connections more sturdy. In this technique, a heat press mechanism is generally utilized to add threaded brass inserts between 3D-printed parts to connect them firmly. In my version, I simply used a soldering iron to embed M3 screws directly into the assembly-ready holes instead of threaded inserts to fasten the parts together.
104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123
Step 2.b: Designing and printing the CNC router moving the thermal camera and the Groudon PCB encasement
After modeling the 3D parts related to the aluminum radiator, I focused on designing a custom CNC router to move the thermal imaging camera to collect thermal scan (imaging) buffers from the predefined locations on the aluminum cooling blocks to produce an accurate thermal image. Also, I wanted to integrate the Groudon PCB as close as possible to the CNC router since the MLX90641 thermal imaging camera must be connected to Photon 2. Thus, I designed these parts:- two chamfered CNC rods,
- the micro switch connector,
- two special pins for attaching GT2 20T pulleys,
- the left CNC stand providing slots for the CNC rods, the 28BYJ-48 stepper motor, the ULN2003 driver board, and the micro switch connector,
- the right CNC stand providing slots for the CNC rods and the GT2 20T pulley pins,
- the thermal camera container head providing holes to pass CNC rods and slots for the MLX90641 thermal imaging camera, GT2 timing belt, and aluminum gear clamps,
- the unique PCB encasement derived from the Groudon PCB outline,
- the PCB encasement connector providing a buckle-shaped joint interlocking with the right CNC stand while preventing any contact with the embedded GT2 20T pulley pins.

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154
- ePLA-Matte Morandi Purple

155
Step 2.b.1: Assembling the 3D-printed components
After printing all 3D models related to the custom CNC router, I started to combine the CNC parts via M3 screws through the assembly-ready screw holes and the provided slots for the associated parts. Then, I fastened the unique Groudon PCB encasement to the complementary PCB connector via M3 screws. Since the PCB connector is compatible with the right CNC stand via its buckle-shaped snap-fit joint and avoids any contact with the GT2 20T pulley pins, I was able to interlock the PCB connector with the right CNC stand effortlessly. Although I applied hot glue between parts while affixing them via M3 screws, it was still not enough to build a production-ready device, especially for a constantly moving CNC router. Thus, I employed a well-known injection molding technique to make the connections more sturdy. In this technique, a heat press mechanism is generally utilized to add threaded brass inserts between 3D-printed parts to connect them firmly. In my version, I simply used a soldering iron to embed M3 screws directly into the assembly-ready holes instead of threaded inserts to fasten the parts together.
156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174
- a 28BYJ-48 stepper motor,
- a ULN2003 driver board,
- a GT2 60T pulley attached to the stepper motor,
- two GT2 20T pulleys attached to the special pulley pins,
- GT2 6 mm timing belt,
- two GT2 aluminum gear clamps.

175

176

177

178

179

180

181

182

183

184

185
Step 2.c: Designing and printing the water-cooled HVAC system accessories
After modeling the 3D parts related to the custom CNC router, I realized that my overall design was still lacking some of the features I wanted to implement to build an industrial-level HVAC malfunction detection device, such as an impervious custom reservoir for the simplified water cooling system. Thus, I designed these additional parts:- an aluminum cooling block holder allowing plastic tubing adjustment,
- an impermeable water reservoir compatible with the water cooling pump,
- a removable top cover for the reservoir with built-in plastic tubing fittings — IN and OUT,
- a custom case and a removable top cover for LattePanda Mu with the Lite Carrier board.

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205
- eTPU-95A Color Change by Temp

206
Step 2.c.1: Assembling the 3D-printed components
After printing all 3D models related to the additional features, I started to combine the components with their associated parts. First, I installed the special heatsink, providing thermal paste, on LattePanda Mu and attached LattePanda Mu to the Lite Carrier board via the built-in connector (slot). Since the Lite Carrier board does not support Wi-Fi connection out of the box, I connected an AC8265 wireless NIC module (WLAN expansion card) via the built-in M.2 E Key (2230).
207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223
Step 3: Setting up a simplified water-cooled HVAC system manifesting potential cooling malfunctions
As discussed earlier, I needed to build a simplified water-based HVAC system to construct data sets fulfilling the purpose of multi-model HVAC malfunction diagnosis and to conduct in-field model testing. Since I got heavily inspired by PC (computer) water cooling systems, I built my simplified system by utilizing these water cooling components, reminiscent of a closed-loop PC water cooling design:- an aluminum water cooling radiator,
- two aluminum water cooling blocks (40 x 80 mm),
- a water cooling pump (4.8 W - 240 L/H),
- 10 mm plastic tubing (hose),
- three 120 mm case fans (RGB) compatible with the radiator.

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247
Step 4: Creating an account to utilize Twilio’s SMS API
Since I decided to inform the user of the latest diagnosed cooling malfunctions via SMS after running the Audio MFE and visual anomaly detection models consecutively, I decided to utilize Twilio’s SMS API. In this regard, I was also able to transfer the prediction date and the modified resulting image name for further inspection through the web dashboard (application). Twilio provides a trial text messaging service to transfer an SMS from a virtual phone number to a verified phone number internationally. Also, Twilio supports official helper libraries for different programming languages, including PHP, enforcing its suite of APIs. :hash: First of all, sign up for Twilio and navigate to the Account page to utilize the default (first) account or create a new account. I noticed that creating free subsidiary accounts (projects) more than once may lead to the permanent suspension of a Twilio user account. So, I recommend using the default trial account or a previously created account if you have multiple iterations or did not subscribe to a paid plan.
248

249

250

251

252

253

254
Step 5.0: Setting up the XAMPP application and the required Python modules on LattePanda Mu (Ubuntu 22.04)
Before starting to develop the web dashboard (application), I needed to configure the required software and Python modules on LattePanda Mu to be able to host the web dashboard, produce thermal images for data collection, and run the FOMO-AD visual anomaly detection model. Since the web dashboard heavily relies on Python modules, especially for running the FOMO-AD model via the Edge Impulse Linux Python SDK, I set up Ubuntu as the operating system for LattePanda Mu. As I was working on this device, Ubuntu 22.04 was officially supported by LattePanda Mu. You can inspect the prioritized operating system versions here. Plausibly, the XAMPP application provides an official Linux installer. So, creating a local server with a MariaDB database to host the web dashboard (application) on LattePanda Mu becomes straightforward and effortless. :hash: First, download the XAMPP Linux installer. :hash: After downloading the XAMPP installer, change its permissions via the terminal (command line). sudo chmod 755 /home/kutluhan/Downloads/xampp-linux-x64-8.2.12-0-installer.run
255

256

257

258

259

260

261

262

263

264
Step 5: Developing a feature-rich web application to communicate w/ the Particle Cloud and process requests from XIAO ESP32C6
As discussed earlier, I decided to develop a versatile web dashboard (application) to improve the user experience and run essential device features, including but not limited to executing Python scripts. Since the web application features interconnect with data collection and model running procedures executed by different development boards, please refer to the web application code files or the following steps focusing on the device qualifications to review all of the web application capabilities thoroughly. As shown below, the web application consists of seven folders and nine code files in various programming languages:- /assets
- class.php
- dashboard_updates.php
- index.css
- index.js
- Particle_cloud_connection.php
- /generate_thermal_img
- /img_detection
- /img_sample
- generate_thermal_image_and_run_model.py
- /model
- /sample_audio_files
- /files
- convert_raw_to_wav.py
- save_audio_sample.php
- index.php

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279
Step 5.1: Converting the raw audio buffers transferred by XIAO ESP32C6 via POST requests to WAV files and transmitting the required conversion parameters as Python Arguments
As explained earlier, I needed to convert the raw audio buffers transferred by XIAO ESP32C6 to WAV audio files in order to save compatible audio samples for Edge Impulse. Therefore, I programmed a simple Python script to perform the audio conversion process. Since Python scripts can obtain parameters as Python Arguments from the terminal (shell) directly, the web dashboard (application) passes the required audio conversion variables effortlessly. 📁 convert_raw_to_wav.py ⭐ Include the required modules.
280
Step 5.2: Transferring the thermal scan (imaging) buffers obtained from the Particle Cloud as Python Arguments to generate a precise thermal image
As discussed earlier, Photon 2 is not suitable for generating thermal images, saving image samples, and running a demanding visual anomaly detection model simultaneously due to memory limitations. Therefore, I utilized the web dashboard to obtain the thermal scan (imaging) buffers registered on the Particle Cloud and programmed a Python script to perform the mentioned processes. Since Python scripts can obtain parameters as Python Arguments from the terminal (shell) directly, the web dashboard (application) passes the obtained thermal imaging buffers and the given process type effortlessly. 📁 generate_thermal_image_and_run_model.py To bundle all functions under a specific structure, I created a class named thermal_img. Please refer to the following steps to inspect all interconnected device features. ⭐ Include the required modules.
281

282
Step 5.3: Running the web application on LattePanda Mu
Since LattePanda Mu is a budget-friendly compute module providing consistent multitasking performance thanks to Intel N100 quad-core processor and 8GB LPDDR5 memory, I decided to host the web application on LattePanda Mu combined with its Lite Carrier board. :hash: After setting up the XAMPP application (lampp) on LattePanda Mu, open the phpMyAdmin tool on the browser manually to create a new database named hvac_system_updates. http://localhost/phpmyadmin/
283

284

285

286

287

288

289

290
Step 6.a: Setting up XIAO ESP32C6 on Arduino IDE
Although XIAO ESP32C6 is a production-ready and compact IoT development board, before proceeding with the following steps, I needed to set XIAO ESP32C6 on the Arduino IDE, install the required libraries, and configure some default settings. When I was setting up XIAO ESP32C6 on the Arduino IDE, the current stable release of the Arduino-ESP32 board package (2.0.15) did not support the ESP32-C6 chipset. Therefore, I utilized the latest development release (3.0.0-rc1). :hash: First, remove the Arduino-ESP32 board package if you have already installed it on the Arduino IDE.
291

292

293

294

295
Step 6.b: Setting up Particle Photon 2 on Visual Studio Code and enabling data transmission with the Particle Cloud
Even though C++ is available for programming Particle development products, the Arduino IDE is not suitable due to the additional requirements for the Particle Device OS. Fortunately, Particle officially supports Visual Studio Code (VSCode) and provides the Particle Workbench, which is an integrated development and debugging environment. Since the Particle Workbench capitalizes on the built-in IntelliSense features of VSCode, it makes programming Photon 2 straightforward and effortless. :hash: First, download Visual Studio Code (VSCode) from the official installer. :hash: After installing VS Code, go to Extensions Marketplace and search for the Particle Workbench extension. :hash: While downloading the Workbench extension, VSCode should install and build all dependencies automatically, including the device toolchain, C++ extension, Particle CLI, etc.
296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312
- MyLibrary/
- examples/
- usage/
- usage.ino
- usage/
- src/
- MyLibrary.cpp
- MyLibrary.h
- library.properties
- README.md
- LICENSE
- examples/

313

314

315

316

317
Step 6.c: Displaying images on the SSD1306 OLED screen and the ST7735 TFT display
I followed the exact same process to display images on the SSD1306 OLED screen (XIAO ESP32C6) and the ST7735 TFT display (Photon 2). :hash: To be able to display images (icons), first convert image files (PNG or JPG) to monochromatic bitmaps. Then, convert the generated bitmaps to compatible C data arrays. I decided to utilize LCD Assistant to create C data arrays. :hash: After installing LCD Assistant, upload a monochromatic bitmap and select Vertical or Horizontal, depending on the screen type. :hash: Then, save all the converted C data arrays to the logo.h file. ⭐ In the logo.h file, I defined multi-dimensional arrays to group the assigned logos and their sizes — width and height.
318

319

320
Step 7: Collecting the raw audio buffers produced by the I2S MEMS microphone
After setting up all development boards on their associated software, I started to work on improving and refining code to perform functions flawlessly. First, I focused on programming XIAO ESP32C6, which manages audio sample collection and data transmission to the web application. As explained in the previous steps, the device performs lots of interconnected features between different development boards and the web application for data collection and running advanced AI models. Thus, the described code snippets show the different aspects of the same code file. Please refer to the code files or the demonstration videos to inspect all interconnected functions in detail. 📁 HVAC_fault_diagnosis_anomalous_sound.ino ⭐ Include the required libraries.- A ➜ normal
- C ➜ defective

321

322

323

324

325
Step 7.1: Generating raw audio samples and passing the produced samples to the web application for saving them as WAV files
⚠️🔊♨️🖼️ If XIAO ESP32C6 establishes a successful connection with the given Wi-Fi network and all connected components operate as expected, the device shows the home screen on the SSD1306 OLED display.- Collect Audio
- Faulty Sound
- CNC Positioning & Thermal Buffer Collection

326

327
- A ➜ normal
- C ➜ defective

328

329

330

331

332

333

334

335

336
Step 8.a: Controlling CNC (thermal camera container head) motions via Arduino Nano
Since I decided to build a fully 3D-printable custom CNC router to position the MLX90641 thermal imaging camera, I needed to design a separate CNC control mechanism based on Arduino Nano. In this regard, I was able to move the thermal camera container head according to the CNC commands received via serial communication. After programming XIAO ESP32C6, I focused on improving and refining CNC functions performed by Arduino Nano. 📁 HVAC_thermal_camera_CNC.ino ⭐ Include the required libraries.- CW: Clockwise
- CCW: Counter-clockwise

337

338

339
Step 8.b: Communicating with Arduino Nano and the web application to initiate the four-step CNC positioning sequence for consecutive thermal imaging buffer collection via the Particle Cloud
After completing the CNC router programming, controlled by Arduino Nano, I focused on improving the remaining XIAO ESP32C6 features, including transferring commands to Arduino Nano and communicating with the web application regarding thermal imaging buffer collection. As explained in the previous steps, the device performs lots of interconnected features between different development boards and the web application for data collection and running advanced AI models. Thus, the described code snippets show the different aspects of the same code file. Please refer to the code files or the demonstration videos to inspect all interconnected functions in detail. 📁 HVAC_fault_diagnosis_anomalous_sound.ino ⭐ Define all of the required CNC commands and variables by creating a struct — _CNC — so as to organize and call them efficiently.
340

341
Step 8.c: Generating the required thermal imaging buffers via a specific color algorithm and registering the produced buffers to Particle Cloud variables
After working on the XIAO ESP32C6 data transmission procedure with the web application and the custom CNC router positioning sequence, I focused on developing and improving Particle Photon 2 functions related to thermal imaging buffer collection and registration. As discussed earlier, I set up the Particle Workbench on Visual Studio Code (VSCode) to be able to utilize the Particle Device OS to program Photon 2. You can inspect the integrated Particle Cloud transmission methods of the Device OS and their limitations from here. 📁 HVAC_fault_diagnosis_thermal_image.cpp ⭐ Include Particle Device OS APIs.
342

343

344

345

346

347

348

349
Step 8.d: Producing accurate thermal images from the registered buffers and saving them as samples via the web application
⚠️🔊♨️🖼️ If the user presses the home button, Arduino Nano homes the thermal camera container head by employing the micro switch on the CNC router. ⚠️🔊♨️🖼️ When the homing sequence starts, Arduino Nano turns the RGB LED to blue. After returning the container head to the home position (0), it turns the RGB LED to white.
350

351

352
- Scan
- Inspect

353

354

355

356

357

358

359

360

361
- Red ➡ command received via serial communication
- Yellow ➡ the positioning process is completed
- Green ➡ the coordinate update confirmation message — CNC_OK — sent (replied) to XIAO ESP32C6 via serial communication

362

363

364

365

366

367
- ‘w’ ➜ White
- ‘c’ ➜ Cyan
- ‘b’ ➜ Blue
- ‘y’ ➜ Yellow
- ‘o’ ➜ Orange
- ‘r’ ➜ Red

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385
- Red ➡ command received via serial communication
- Yellow ➡ the zeroing process is completed
- Purple ➡ the zeroing confirmation message — CNC_OK — sent (replied) to XIAO ESP32C6 via serial communication

386

387

388

389

390

391

392

393

394

395

396

397

398

399
Step 9: Building a neural network model (Audio MFE) w/ Edge Impulse Enterprise
As discussed earlier, while collecting audio samples to construct a valid audio data set, I simply differentiated the generated audio samples by the cooling fan failure presence:- normal
- defective
Step 9.1: Uploading and processing samples
After splitting my audio data set into training and testing samples, I uploaded them to my project on Edge Impulse Enterprise. :hash: First of all, to utilize the incorporated tools for advanced AI applications, sign up for Edge Impulse Enterprise. :hash: Then, create a new project under your organization.
400

401

402

403

404

405

406

407

408

409

410

411
Step 9.2: Training the model on sound-based anomalous behavior
After uploading and labeling my training and testing samples successfully, I designed an impulse and trained the model to detect anomalous sound originating from the cooling fans of the water-based HVAC system. An impulse is a custom machine learning model in Edge Impulse. I created my impulse by employing the Audio (MFE) processing block and the Classification learning block. The Audio MFE processing block extracts time and frequency features from a signal and simplifies the generated features for non-voice recognition by using a non-linear scale — Mel-scale. The Classification learning block represents a Keras neural network model. This learning block lets the user change the model settings, architecture, and layers. :hash: Go to the Create impulse page and leave Window size and Window increase parameters as default. In this case, I did not need to slice the passed audio samples since all of them have roughly one-second duration.
412

413

414

415

416

417

418

419

420
- Number of training cycles ➡ 100
- Learning rate ➡ 0.010
- Validation set size ➡ 10

421

422
Step 9.3: Evaluating the model accuracy and deploying the model
After building and training my Audio MFE neural network model, I tested its accuracy and validity by utilizing testing samples. The evaluated accuracy of the model is 100%. :hash: To validate the trained model, go to the Model testing page and click Classify all.
423

424

425

426

427

428
Step 10: Building a visual anomaly detection model (FOMO-AD) w/ Edge Impulse Enterprise
As discussed earlier, while producing thermal image samples to construct a valid image data set, I utilized the default classes to label the generated samples, required by Edge Impulse to enable the F1 score calculation:- no anomaly
- anomaly
Step 10.1: Uploading images (samples) to Edge Impulse and assigning anomaly classes
After splitting my thermal image data set into training (stable) and testing (thermal malfunction) samples, I uploaded them to my project on Edge Impulse Enterprise. :hash: First of all, to utilize the incorporated tools for advanced AI applications, sign up for Edge Impulse Enterprise. :hash: Then, create a new project under your organization.
429

430

431

432

433

434

435

436

437

438

439
Step 10.2: Training the FOMO-AD model on the thermal images showcasing stable heat transfer
After uploading and labeling my training and testing samples with the default classes successfully, I designed an impulse and trained the model to diagnose ensuing thermal cooling malfunctions after applying anomalous sound detection to the water-based HVAC system. An impulse is a custom machine learning model in Edge Impulse. I created my impulse by employing the Image processing block and the FOMO-AD (Images) learning block. The Image processing block optionally turns the input image format to grayscale or RGB and generates a features array from the passed raw image. The FOMO-AD (Images) learning block represents a machine learning algorithm that identifies anomalies based on the trained normal (stable) images by applying a Gaussian Mixture Model. In this case, I configured the input image format as RGB since distinguishing thermal cooling malfunctions based on thermal images highly relies on color differences. As stated by Edge Impulse, they empirically obtained the best anomaly detection results by applying 96x96 ImageNet weights regardless of the intended raw image input resolution. Thus, I utilized the same resolution for my visual anomaly detection model. :hash: Go to the Create impulse page and set image width and height parameters to 96. Then, select the resize mode parameter as Fit shortest axis so as to scale (resize) given training and testing image samples. :hash: Select the Image processing block.
440

441

442

443

444

445

446
- Capacity ➡ High
- MobileNetV2 0.35

447

448
Step 10.3: Evaluating the model accuracy and deploying the model
After building and training my FOMO-AD visual anomaly detection model, I tested its accuracy and validity by utilizing testing samples. In addition to validating the model during testing, Edge Impulse evaluates the F1 precision score (accuracy) and provides per region anomalous scoring results for the passed testing images. To tweak the learning block sensitivity, Edge Impulse lets the user change the suggested confidence threshold estimated based on the top anomaly scores in the training dataset. In that regard, the user can adjust the anomaly detection rate according to the expected real-world conditions. After validating my FOMO-AD model, Edge Impulse evaluated the precision score (accuracy) as 100%. Since I configured this visual anomaly detection model to conform to the produced thermal images of my simplified HVAC system components, the precision score (accuracy) is approximately 100%. Thus, I highly recommend constructing a new thermal image data set of different HVAC system components and retraining the model before running inferences to diagnose thermal cooling malfunctions. :hash: To validate the trained model, go to the Model testing page and click Classify all.
449

450

451

452

453

454

455

456

457

458
Step 11.a: Setting up the neural network model (Audio MFE) on XIAO ESP32C6
Since Edge Impulse optimizes and formats signal processing, configuration, and learning blocks into a single package while deploying models as Arduino libraries, even for complex machine learning algorithms, I was able to import my advanced model effortlessly to run inferences on XIAO ESP32C6. :hash: After downloading the model as an Arduino library in the ZIP file format, go to Sketch ➡ Include Library ➡ Add .ZIP Library… :hash: Then, include the AI-driven_HVAC_Fault_Diagnosis_Audio__inferencing.h file to import the Edge Impulse Audio MFE neural network model.- \Arduino\libraries\AI-driven_HVAC_Fault_Diagnosis_Audio__inferencing\src\edge-impulse-sdk\classifier\ei_classifier_config.h*

459

460

461

462
Step 11.b: Setting up the visual anomaly detection model (FOMO-AD) on LattePanda Mu
Since Edge Impulse optimizes and formats signal processing, configuration, and learning blocks into a single EIM file while deploying models as a Linux (x86_64) application, even for complex visual anomaly detection models, I was able to import my advanced FOMO-AD model effortlessly to run inferences in Python on LattePanda Mu (x86 Compute Module). :hash: After downloading the generated Linux (x86_64) application to the model folder under the root folder of the web application, make sure to change the file permissions via the Properties tab to be able to execute the model file. As shown earlier, you can also use the terminal (shell) to change file permissions.
463

464
Step 11.c: Running both machine learning models consecutively to detect anomalous sound originating from the cooling fans and diagnose ensuing thermal cooling malfunctions of the HVAC system and informing the user of the cooling status via SMS
⚠️🔊♨️🖼️ If the user activates the second menu option provided by XIAO ESP32C6 — Faulty Sound:
465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486
- All
- Cooling Malfunction Detections
- Thermal Image Samples
- Anomalous Sound Samples

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501
Further Discussions
By applying advanced AI-powered multi-algorithm detection methods to identify anomalous sound emanating from the cooling fans and diagnose ensuing thermal cooling malfunctions of water-based HVAC system components, we can achieve to: ⚠️🔊♨️🖼️ prevent pernicious cooling aberrations deteriorating heat regulation for industrial facilities, ⚠️🔊♨️🖼️ avert production downtime for demanding industrial processes, ⚠️🔊♨️🖼️ reduce production costs and increase manufacturing efficiency, ⚠️🔊♨️🖼️ obviate excessive energy consumption via real-time (automated) malfunction diagnosis, ⚠️🔊♨️🖼️ extend equipment lifespan by maintaining stable heat transfer, ⚠️🔊♨️🖼️ deter exorbitant overhaul processes due to prolonged negligence, leading to a nosedive in production quality.
502
Project GitHub Repository
As a developer, if you want to inspect or access code files and custom design files effortlessly, you can visit the project’s GitHub repository, providing:- Code files
- Gerber files
- STL files
- Custom libraries
- Edge Impulse machine learning models — Audio MFE and FOMO-AD
References
[^1] What is an Industrial Water-Based Cooling System and How Does it Work?, ChemREADY, https://www.getchemready.com/water-facts/what-is-an-industrial-water-based-cooling-system-and-how-does-it-work/. [^2] Industrial water cooling systems-chillers, Atlas Copco, https://www.atlascopco.com/en-uk/compressors/products/industrial-water-cooling-systems. [^3] Jian Bi, Hua Wang, Enbo Yan, Chuan Wang, Ke Yan, Liangliang Jiang, Bin Yang, AI in HVAC fault detection and diagnosis: A systematic review, Energy Reviews, Volume 3, Issue 2, 2024, https://doi.org/10.1016/j.enrev.2024.100071. [^4] Herbert W. Stanford III, HVAC Water Chillers and Cooling Towers: Fundamentals, Application, and Operation, CRC Press, 2nd edition, March 29, 2017, Page: 16-28, https://www.google.com/books/edition/HVAC_Water_Chillers_and_Cooling_Towers/KDteO_-GaLkC.Schematics

1

2

3

4

5

6

7

8

9

10

11

12

13