[Tutorial I]: 10:40 ~ 11:30, February 22, 2023 (Wednesday)
Prof. Youn-Hee Han (KOREATECH, Republic of Korea)

Reinforcement Learning-based Task Allocation to Limited Resources: A Combinatorial Optimization Approach

In this lecture, we will focus on using the reinforcement learning algorithm DQN (Deep Q-Network) to optimize resource utilization when allocating multiple tasks to limited resources. To start, we will introduce the concept of the simple task allocation problem, which is similar to the well-known combinatorial optimization problem called the Multi-Dimensional 0-1 Knapsack Problem. This problem involves allocating a set of tasks to a set of limited resources in a way that maximizes resource utilization. We will also provide examples of accurately solving this combinatorial optimization problem using Google OR-Tools. Google OR-Tools is an open-source software suite that includes several powerful optimization algorithms. We will explain how to use Google OR-Tools to solve the task allocation problem exactly. In addition to that, we will also explain how to use DQN to quickly obtain an approximate solution for the task allocation problem. We will briefly explain the principle of DQN and introduce a simple code written in PyTorch. By running the code in this talk, we will present various performance metrics for DQN-based task allocation and also compare them with the ones of Google OR-Tools. Finally, we explain the future research direction of RL-based combinatorial optimization through Pointer Network, etc. This will give you an idea of the relative strengths and weaknesses of each approach and help you choose the best method for your specific combinatorial optimization problem.


Prof. Youn-Hee Han received his B.S. degree in MathematicsfromKorea University, Seoul, Korea in 1996, and an M.S. and Ph.D. inComputer Science and Engineeringfrom the same university in 1998 and 2002, respectively. From March 4, 2002 to February 28, 2006, he was a senior researcher in the Next Generation Network Group of Samsung Advanced Institute of Technology. Since March 2, 2006, he has been a professor in School of Computer Science and Engineering at Korea University of Technology and Education (KOREATECH), CheonAn, Korea. He also served as a visiting professor in Department of Computer Science, State University of New York (SUNY) at Albany from Sept. 2013 to Jan. 2015. His main focus of study in research is on the field of intelligent networks. He has published approximately 150 research papers on the theory and application of intelligent networks, as well as books on some practical programming languages. He has also the filed application of 30 patents on intelligent networks. He has been also interested in contributions to International Standard Organizations such as IETF, IRTF, and IEEE 802, etc. In particular, he had been an active member of the IEEE 802.21 working group. He is also an author of RFC 5181,RFC 5270, and RFC 7864. Currently, he is very interested in artificial intelligence technology, especially reinforcement learning. He is very interested in improving the performance of reinforcement learning algorithms, and continues to work on optimizing the performance by applying reinforcement learning agents to various fields, such as intelligent networking on 5G and 6G, IoT (Internet of Things), smart factory, and financial engineering.

[Tutorial II]: 11:30 ~ 12:20, February 22, 2023 (Wednesday)
Prof. Koya Sato (The University of Electro-Communications, Japan)

Decentralized Machine Learning over Wireless Networks

The growth in machine learning (ML) applications has caused issues such as data privacy and communication costs. This presentation will introduce distributed machine learning techniques, such as federated learning (FL) and its decentralized form (DFL). FL/DFL iterates training an ML model at edge nodes and combining the results. It allows us to analyze distributed big data efficiently with privacy preservation; however, its training performance is influenced by wireless channels. This tutorial first gives the basics and technical backgrounds of FL/DFL. Then, we discuss research areas in wireless networks, related works, and research directions from viewpoints of relationships between network topology and radio propagation.

Koya Sato received the B.E. degree in electrical engineering from Yamagata University, in 2013, and the M.E. and Ph.D. degrees from The University of Electro-Communications, in 2015 and 2018, respectively. From 2018 to 2021, he was an Assistant Professor at the Tokyo University of Science. He is currently an Assistant Professor at the Artificial Intelligence eXploration Research Center, The University of Electro-Communications. His current research interests include wireless communications, distributed machine learning, and spatial statistics.