Internet of Things (IoT) is composed of many IoT devices connected throughout the Internet, that collect and share information to represent the environment. IoT is currently restructuring the actual manufacturing to smart manufacturing. However, inherent characteristics of IoT lead to a number of titanic challenges such as decentralization, weak interoperability, security, etc. The artificial intelligence provides opportunities to address IoT’s challenges, e.g the agent technology. This paper presents first an overview of ML and discusses some related work. Then, we briefly present the classic IoT architecture. Then we introduce our proposed Intelligent IoT (IIoT) architecture. We next concentrate on introducing the approach using multi-agent DRL in IIoT. Finally, in this promising field, we outline the open directions of future work.