Sürdürülebilir Kalkınma Amaçları
Yayınlar
- ArticleAçık Erişim2026Double Deep Q-Network-Based Solution for the Dynamic Electric Vehicle Routing Problem
Taş, Mehmet Bilge Han; Özkan, Kemal; Sarıçiçek, İnci; Yazıcı, Ahmet
The Dynamic Electric Vehicle Routing Problem (D-EVRP) presents a framework that requires electric vehicles to meet demand with limited energy capacity. When dynamic demand flows and charging requirements are considered together, traditional methods cannot provide sufficient adaptation for real-time decision-making. Therefore, a learning-based approach was chosen to ensure that decision-making processes respond quickly to changing conditions. The solution utilizes a model with a Double Deep Q-Network (DDQN) architecture and a discrete valuation structure. Prioritized Experience Replay (PER) was implemented to increase model stability, allowing infrequent but effective experiments to contribute more to the learning process. The state representation is constructed using the vehicle’s location, battery level, load status, and current customer demands. Scalability is ensured by dividing customer locations into clusters using the K-means method, with each cluster handled by an independent representative. The approach was tested with real-world road data obtained from the Meşelik Campus of Osmangazi University in Eskişehir. Experiments conducted under different demand levels and data sizes have shown that the PER-assisted DDQN structure produces more stable and shorter route lengths in dynamic scenarios, but random selection, greedy method and genetic algorithm experience significant performance losses as dynamicity increases.
RDL Kalıcı Kaydıİncele - ArticleAçık Erişim2025A Reinforcement Learning-Based Solution for the Capacitated Electric Vehicle Routing Problem from the Last-Mile Delivery Perspective
Aslan Yıldız, Özge; Sarıçiçek, İnci; Yazıcı, Ahmet
The growth of the urban population and the increase in e-commerce activities have resulted in challenges for last-mile delivery. On the other hand, electric vehicles (EVs) have been introduced to last-mile delivery as an alternative to fossil fuel vehicles. Electric vehicles (EVs) not only play a pivotal role in reducing greenhouse gas emissions and air pollution but also contribute significantly to the development of more energy-efficient and environmentally sustainable urban transportation systems. Within these dynamics, the Electric Vehicle Routing Problem (EVRP) has begun to replace the Vehicle Routing Problem (VRP) in last-mile delivery. While classic vehicle routing ignores fueling, both the location of charging stations and charging time should be included in the Electric Vehicle Routing Problem due to the long recharging time. This study addresses the Capacitated EVRP (CEVRP) with a novel Q-learning algorithm. Q-learning is a model-free reinforcement learning algorithm designed to maximize an agent’s cumulative reward over time by selecting optimal actions. Additionally, a new dataset is also published for the EVRP considering field constraints. For the design of the dataset, real geographical positions have been used, located in the province of Eskisehir, Türkiye. It also includes environmental information, such as streets, intersections, and traffic density, unlike classical EVRP datasets. Optimal solutions are obtained for each instance of the EVRP by using the mathematical model. The results of the proposed Q-learning algorithm are compared with the optimal solutions of the presented dataset. Test results show that the proposed algorithm provides remarkable advantages in obtaining routes in a shorter time for EVs.
RDL Kalıcı Kaydıİncele - ArticleAçık Erişim2025Transportation Mode Selection Using Reinforcement Learning in Simulation of Urban Mobility
Taş, Mehmet Bilge Han; Özkan, Kemal; Sarıçiçek, İnci; Yazici, Ahmet
Transportation mode selection is pivotal for navigating through cities plagued by heavy traffic congestion. This plays a crucial role in ensuring the efficient utilization of time and resources to achieve the desired objectives. Given the complex dynamics of urban mobility, strategically selecting a transportation mode can significantly mitigate delays and enhance overall productivity in densely populated areas. The objective of this study is to find the most efficient result among various transportation modes to make deliveries from different points on a university campus. To solve this problem, reinforcement learning was used and tested on the simulation environment SUMO. Traffic density was increased by using an equal number of different transportation modes, such as driving, cycling, motorbiking, and walking. Various traffic densities were generated, and different reward models were applied to select the best means of transportation. Various probability distributions were used as reward models to avoid the unfair distribution caused by how near or how far the road is when moving from random points to the destination region. As a result of the models created using the applied reward–penalty functions, it was determined that the best means of transportation in areas with a low traffic density is cycling, and in areas with high traffic density, the optimal mode of transportation is motorbiking.
RDL Kalıcı Kaydıİncele - ArticleYalnız künye2025CONFLICT-FREE PATH PLANNING FOR AUTONOMOUS TRANSPORT VEHICLES
Yıldız, Özge Aslan; Yazıcı, Ahmet; Sarıçiçek, İnci
Autonomous Transport Vehicles (ATVs) are flexible robotic systems for transportation tasks in intra-logistics. It is very important to determine the most cost-efficient routes for ATVs within factory environments. Conflicts occur if more than one ATV moves at the same point in the same time interval. Conflicts cause unwanted delays, and it is crucial to determine the conflict-free routes for ATVs on the shop floor. Efficiently planning conflict-free paths for multiple AGVs and minimizing overall task completion time are essential for assessing system performance. In the study, a NonConflict-A* (NC-A*) algorithm that is based on the A* algorithm has been proposed to solve conflict problems in multiple ATVs. Unlike studies in the literature, the proposed algorithm performs pathfinding, conflict detection, and conflict resolution simultaneously. The NC-A* algorithm detects the conflicts while obtaining the paths, chooses the conflict resolution strategy that is the least costly according to the conflict types and produces the most cost-efficient route. The proposed algorithm aims to minimize the total route duration time while ensuring system safety by detecting and resolving all potential conflicts. The algorithm is tested for different conflict types of multiple ATVs traveling at various speeds. The solutions of the algorithm show that the algorithm determines all of the conflicts, selects the appropriate solution strategy and generates conflict-free routes.
RDL Kalıcı Kaydıİncele - ArticleAçık Erişim2025A transformer-based deep reinforcement learning for the Dial-a-Ride problem
Aslan Yıldız, Özge; Sarıçiçek, İnci; Yazıcı, Ahmet
Dial-a-Ride problem (DARP) is a specialized variant of the vehicle routing problem that focuses on designing optimal routes for users who specify pick-up and delivery requests between origins and destinations. This problem involves determining the most effective and efficient routes using limited vehicles and resources to offer responsive door-to-door transportation services for users with specific needs. Logistics service providers require systems that can deliver optimal solutions to these combinatorial problems within a reasonable time. Recently, there has been a significant increase in the use of artificial intelligence optimization algorithms such as meta-heuristics or learning-based approaches to solve such problems. Among the learning-based approaches, reinforcement learning has gained prominence for routing and scheduling tasks, owing to its ability to adaptively learn from complex state spaces and dynamically changing environments. In this study, a novel transformer-based deep reinforcement learning method is proposed to solve the Dial and Ride problem for a single service vehicle. In the proposed model, we adopt a modified transformer architecture instead of employing traditional linear layers, we integrate convolutional layers. To validate our approach, we conduct comprehensive experiments comparing our method against four well-known metaheuristic algorithms and a Deep Q-Network algorithm. The results indicate that proposed approach outperforms these techniques in terms of shorter total travel distances. Additionally, the proposed method is tested on a real-world scenario generated in the Buyukdere neighborhood of Eskisehir. The results demonstrate that the proposed method makes it possible to solve the problem within a reasonable time. This study confirms that the proposed deep reinforcement learning method can effectively address the Dial-a-Ride Problems.
RDL Kalıcı Kaydıİncele - ArticleYalnız künye2023A novel conflict free routing with multi pickup delivery tasks for autonomous vehicles
Sarıçiçek, İnci; Yazıcı, Ahmet; Aslan, Özge
Purpose: This study aims to propose a novel method for the conflict detection and eradication of autonomous vehicles which has predetermined routes to establish multi pickup and delivery tasks according to task priorities and vehicle capacity status on each pickup and delivery nodes in assembly cells in the automotive production. Design/methodology/approach: In the designed system, the routing of autonomous vehicles (AVs) and scheduling of pickup and delivery tasks are established in production logistics. Gantt chart is created according to vehicle routes, and conflicts are detected using the proposed conflict-sweep algorithm. The proposed conflict-solving algorithm eliminates conflicts on intersections and roads by considering vehicle routes and task priorities. Findings: In many production systems, there is a need to obtain flexible routes in each pickup delivery task group that changes during day, week, etc. Proposed system provides remarkable advantages in obtaining conflict-free routes for pre-scheduled multi transport tasks of vehicles by considering efficiency in production systems. Originality/value: A novel method is proposed for the conflict detection and eradication of AVs. Proposed system eliminates conflicts on intersections and roads by considering pre-planned vehicle routes for a fleet of heterogeneous AVs. Unlike most of the other conflict-free algorithms, in which conflicts are solved between two points, proposed system also considers multi pickup and delivery points for AVs. This is pioneering paper that addresses conflict-free route planning with backhauls and scheduling of multi pickup and delivery tasks for AVs.
RDL Kalıcı Kaydıİncele