Description:
Interaction testing allows for determining a fault in a complex engineered system where a fault is the result of a "small" number of factors interacting. Typically, the object of study is a "covering array", which is an array of symbols (a given number of rows and columns) that encodes the tests to-be-performed: the rows are the tests, and the columns correspond to the factors/components of the system. The property of the array is that all interactions of size at most a given number (the "strength") appear in the array at least once.
One main question to ask is: for a given system, when does a covering array exist? The most common goal in interaction testing is minimizing the number of rows/tests while maintaining the "coverage" property. It can be shown that as the number of factors increases, the minimum number of rows must increase.
Showing a covering array existing is easy, but proving that one does not exist is very difficult, and often mathematicians result to case analyses to prove an array cannot exist.
This competition is focused on a "continuous" generalization of covering arrays, which appear to allow (1) a more fine-grained approach to understand how "covering" an array is, and (2) possibly a better explanation for why certain arrays do not exist.
Important Dates:
March 15, 2021: Deadline to register for the competition and submission of abstract for publication in the GECCO companion.
April 10, 2021: Deadline for submitting abstracts and solutions.
May 1, 2021: Notification of acceptance for GECCO companion.
May 10, 2021: Deadline for general registration for the competition.
June 5, 2021: Deadline for general submission of solutions and approaches.
July 10-14, 2021: GECCO 2021 conference, and announcement of winners and results.
Submission deadline:
2021-06-04
Official webpage:
Organizers:
Description:
In this competition, participants will test their single objective numerical optimization algorithms on a selected 10D and 30D problems with / without a number of transformations such as shifting, rotation, shearing, etc.
Submission deadline:
2021-04-30
Official webpage:
https://www3.ntu.edu.sg/home/epnsugan/index_files/cec-benchmarking.htm
Organizers:
Description:
The aim of the competition is to provide a common platform that encourages fair and easy comparisons across different niching algorithms. The competition allows participants to run their own niching algorithms on 20 benchmark multimodal functions with different characteristics and levels of difficulty. Researchers are welcome to evaluate their niching algorithms using this benchmark suite, and report the results by submitting a paper to the main tracks of GECCO (i.e., submitting via the online submission system of GECCO), or to hand in a GECCO short paper on their competition entry. The description of the benchmark suite, evaluation procedures, and established baselines can be found in the following technical report:
X. Li, A. Engelbrecht, and M.G. Epitropakis, ``Benchmark Functions for CEC'2013 Special Session and Competition on Niching Methods for Multimodal Function Optimization'', Technical Report, Evolutionary Computation and Machine Learning Group, RMIT University, Australia, 2013.
Competition participants may also consider submitting a short (2-page, including references) description of their method and results, for consideration for publication in the GECCO Companion. Note, the deadline for competition paper submission is considerably earlier than the competition entry deadline, and is April 12th 2021.
Submission deadline:
2021-06-24
Official webpage:
http://epitropakis.co.uk/gecco2021/
Organizers:
Description:
The use of camera networks is now common to perform various surveillance tasks. These networks can be implemented together with intelligent systems that analyze video footage, for instance, to detect events of interest, or to identify and track objects or persons. According to (7), whatever the operational needs are, the quality of service depends on the way in which the cameras are deployed in the area to be monitored (in terms of position and orientation angles). Moreover, due to the prohibitive cost of setting or modifying such a camera network, it is required to provide a priori a configuration that minimizes the number of cameras in addition to meeting the operational needs. In this context, the optimal camera placement problem (OCP) is of critical importance, and can be generically formulated as follows. Given various constraints, usually related to coverage or image quality, and an objective to optimise (typically, the cost), how can the set of positions and orientations which best (optimally) meets the requirements be determined?
More specifically, in this competition, the objective will be to determine camera locations and orientations which ensure complete coverage of the area while minimizing the cost of the infrastructure. To this aim, a discrete approach is considered here : the surveillance area is reduced to a set of three-dimensional sample points to be covered, and camera configurations are sampled into so-called candidates, each with a given set of position and orientation coordinates. A candidate can have several samples within range, and a sample can be seen by several candidates. Now, the OCP comes down to select the smallest subset of candidates which covers all the samples.
According to (5), the OCP is structurally identical to the unicost set covering problem (USCP), which is one of Karp's well-known NP-hard problems (3). The USCP can be stated as follows: given a set of elements I (rows) to be covered, and a collection of sets J (columns) such that the union of all sets in J is I, find the smallest subset C of J such that the union of all sets in C is I. In other words, identify the smallest subset of J wich covers I. As pointed out in (5), many papers dealing with the OCP use this relationship implicitly, but few works done on the USCP have been applied or adapted to the OCP, and vice versa. In very recent years however, approaches from the USCP literature have been successfully applied in the OCP context on both academic (1,2,6) and real-world (4,6) problem instances. These works suggest that bridges can be built between these two bodies of literature to improve the results obtained so far on both USCP and OCP problems.
The main goal of this competition is to encourage innovative research works in this direction, by proposing to solve OCP problem instances stated as USCP.
(1) Brévilliers M., Lepagnot J., Kritter J., and Idoumghar L. Parallel preprocessing for the optimal camera placement problem. International Journal of Modeling and Optimization, 8(1):33 – 40, 2018.
(2) Brévilliers M., Lepagnot J., Idoumghar L., Rebai M., and Kritter J. Hybrid differential evolution algorithms for the optimal camera placement problem. Journal of Systems and Information Technology, 20(4):446 – 467, 2018.
(3) Richard M. Karp. Reducibility among Combinatorial Problems, pages 85–103. Springer US, Boston, MA, 1972.
(4) J. Kritter, M. Brévilliers, J. Lepagnot, and L. Idoumghar. On the real-world applicability of state-of-the-art algorithms for the optimal camera placement problem. In 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT), pages 1103–1108, April 2019.
(5) Julien Kritter, Mathieu Brévilliers, Julien Lepagnot, and Lhassane Idoumghar. On the optimal placement of cameras for surveillance and the underlying set cover problem. Applied Soft Computing, 74:133 – 153, 2019.
(6) Weibo Lin, Fuda Ma, Zhouxing Su, Qingyun Zhang, Chumin Li, and Zhipeng Lü. Weighting-based parallel local search for optimal camera placement and unicost set covering. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion, GECCO ’20, pages 3–4, New York, NY, USA, 2020. Association for
Computing Machinery.
(7) Junbin Liu, Sridha Sridharan, and Clinton Fookes. Recent advances in camera planning for large area surveillance: A comprehensive review. ACM Comput. Surv., 49(1):6:1–6:37, May 2016.
Submission deadline:
2021-06-03
Official webpage:
http://www.mage.fst.uha.fr/brevilliers/gecco-2021-ocp-uscp-competition/
Organizers:
Description:
The Dota 2 game represents an example of a multiplayer online battle arena video game. The underlying goal of the game is to control the behaviour/ strategy for a ‘hero’ character. A hero posses certain abilities, thus resulting in different performance tradeoffs. Moreover, the hero acts with a team of ‘creeps’ who have predefined behaviours, which can be influenced by the interaction between their hero and the opposing team. In short, the hero operates collaboratively with its own creeps and defensive structures (called towers) to defeat the opponent team (kill the opponent hero twice, or destroy their tower). In addition, there is an underlying economy in which developments in the game influence the amount of wealth received by each team. As a team’s wealth increases, then the hero’s abilities improve.
This competition will assume the 1-on-1 mid lane configuration of Dota 2 using the Shadow Fiend hero. Such a configuration still includes many of the properties that have turned the game into an ‘e-sport’, but without the computational overhead of solving the task for all heroes under multi-lane settings. Specific properties that make the 1-on-1 game challenging include: 1) the need to navigate a partially observable world under ego-centric sensor information, 2) state information that is high-dimensional, but subject to variation through the ‘fog-of-war’, 3) high-dimensional action space that is both discrete, continuous valued and context specific, 4) learning hero policies that act collectively with creeps, 5) supporting real-time decision making at frame-rate, and 6) the underlying physics of the game vary with the times of day and introduce stochastic states.
Participants will create a Dota 2 Shadow Fiend hero agent based on a preset API provided by the organizers. The competition entrants will be required to engage in a 1v1 match against the built-in Shadow Fiend hero AI, where the winner is determined by number of matches won. Evaluation will be performed against the top three levels of built-in hero over multiple games.
Submission deadline:
2021-06-25
Official webpage:
Organizers:
Description:
Stacking problems are central to multiple billion-dollar industries. The container shipping industry needs to stack millions of containers every year. In the steel industry the stacking of steel slabs, blooms, and coils needs to be carried out efficiently, affecting the quality of the final product. The immediate availability of data – thanks to the continuing digitalization of industrial production processes – makes the optimization of stacking problems in highly dynamic environments feasible.
This competition extends the 2020 Dynamic Stacking competition with a second track.
The first track is identical to the 2020 competition whereby a dynamic environment is provided that represents a simplified stacking scenario. Blocks arrive continuously at a fixed arrival location from which they have to be removed swiftly. If the arrival location is full, the arrival of additional blocks is not possible. To avoid such a state, there is a range of buffer stacks that may be used to store blocks. Each block has a due date before which it should be delivered to the customer. However, blocks may leave the system only when they become ready, i.e., some time after their arrival. To deliver a block it must be put on the handover stack – which must contain only a single block at any given time. There is a single crane that may move blocks from arrival to buffer, between buffers, and from buffer to handover. The optimization must control this crane in that it reacts to changes with a sequence of moves that are to be carried out. The control does not have all information about the world. A range of performance indicators will be used to determine the winner.
The second track represents another stacking scenario that is derived from real-world scenarios. It features two cranes and two different handovers. The cranes have a capacity of larger than one which represents an additional challenge for the solver. Again the solver just provide the moves and the cranes will sort out the order in which these are performed using a simple heuristic. In this scenario, not the arrival stack is the critical part, but the handover stacks and thus the downstream process must not run empty.
The dynamic environments are implemented in form of a realtime simulation which provides the necessary change events. The simulation runs in a separate process and publishes its world state and change events via message queuing (ZeroMQ), and also listens for crane orders. Thus, control algorithms may be implemented as standalone applications using a wide range of programming languages. Exchanged messages are encoded using protocol buffers – again libraries are available for a large range of programming languages.
Submission deadline:
2021-06-26
Official webpage:
Organizers:
Description:
Following the success of the previous editions (CEC, GECCO, WCCI), we are launching a more challenging competition at major conferences in the field of computational intelligence. This GECCO 2021 competition proposes two testbeds in the energy domain:
Testbed 1) Bi-level optimization of end-users’ bidding strategies in local energy markets (LM). This test bed is constructed under the same framework of the past competitions (therefore, former competitors can adapt their algorithms to this new testbed) , representing a complex bi-level problem in which competitive agents in the upper-level try to maximize their profits, modifying and depending on the price determined in the lower-level problem (i.e., the clearing price in the LM), thus resulting in a strong interdependence of their decisions.
Testbed 2) Flexibility management of home appliances to support DSO requests. A model for aggregators flexibility provision in distribution networks that takes advantage of load flexibility resources allowing the re-schedule of shifting/real-time home-appliances to provision a request from a distribution system operator (DSO) is proposed. The problem can be modeled as a Mixed-Integer Non-Linear Programming (MINLP) in which the aggregator strives to match a flexibility request from the DSO/BRP, paying a remuneration to the households participating in the DR program according to their preferences and the modification of their baseline profile.
Note: Both testbeds are developed to run under the same framework of past competitions.
Competition goals:
Following the success of the previous editions (CEC, GECCO, WCCI), we are launching a more challenging competition at major conferences in the field of computational intelligence. This GECCO 2021 competition proposes two tracks in the energy domain:
Track 1) Bi-level optimization of end-users’ bidding strategies in local energy markets (LM). This test bed is constructed under the same framework of the past competitions (therefore, former competitors can adapt their algorithms to this new track) , representing a complex bi-level problem in which competitive agents in the upper-level try to maximize their profits, modifying and depending on the price determined in the lower-level problem (i.e., the clearing price in the LM), thus resulting in a strong interdependence of their decisions.
Track 2) Flexibility management of home appliances to support DSO requests. A model for aggregators flexibility provision in distribution networks that takes advantage of load flexibility resources allowing the re-schedule of shifting/real-time home-appliances to provision a request from a distribution system operator (DSO) is proposed. The problem can be modeled as a Mixed-Integer Non-Linear Programming (MINLP) in which the aggregator strives to match a flexibility request from the DSO/BRP, paying a remuneration to the households participating in the DR program according to their preferences and the modification of their baseline profile.
Note: Both tracks are developed to run under the same framework of past competitions.
Competition goals:
The GECCO 2021 competition on “Evolutionary Computation in the Energy Domain: Smart Grid Applications” has the purpose of bringing together and testing the more advanced Computational Intelligence (CI) techniques applied to energy domain problems, namely the optimal bidding of energy aggregators in local markets and the Flexibility management of home appliances to support DSO requests. The competition provides a coherent framework where participants and practitioners of CI can test their algorithms to solve two real-world optimization problems in the energy domain. The participants have the opportunity to evaluate if their algorithms can rank well in each independent problem since we understand the validity of the “no-free lunch theorem”, making this contest a unique opportunity worth to explore the applicability of the developed approaches in real-world problems beyond the typical benchmark and standardized CI problems.
Submission deadline:
2021-05-30
Official webpage:
http://www.gecad.isep.ipp.pt/ERM-Competitions (will be online if accepted)
Organizers:
Description:
The Game Benchmark for Evolutionary Algorithms (GBEA) is a collection of single- and multi-objective optimisation tasks that occur in applications to games research. We are proposing a competition with multiple tracks that addresses several different research questions featuring continuous and integer search spaces. The GBEA uses the COCO (COmparing Continuous Optimisers) framework for ease of integration.
The task is to find solutions of sufficient quality (as specified by a target value) as quickly as possible. The competition is available in a single- and bi-objective version for two different applications, thus resulting in 4 different tracks. Details will be available on our website.
Participants will be able to submit short algorithm descriptions as 2-page contributions to the GECCO Companion. The deadline for submissions will be in April 2021.
== Why Games? ==
Games are a very interesting topic that motivates a lot of research and have repeatedly been suggested as testbeds for AI algorithms. Key features of games are controllability, safety and repeatability, but also the ability to simulate properties of real-world problems such as measurement noise, uncertainty and the existence of multiple objectives.
The motivation for the competition setup is as follows. If an algorithm for generating content (such as a Mario level or a Top Trumps deck) is integrated in a game, the goal is usually to provide replay value by varying the content. In this context, it is not necessary to find the single solution that optimises the designer's objectives, but instead, it is important that a "good enough" solution can be found as fast as possible. "Good enough" in this case can be defined in relation to the values achieved by a baseline algorithm. Additionally, ideally, the same algorithm can find solutions across different objectives.
So which game levels can you find? Let us find out! Submit your best optimisation algorithms!
Submission deadline:
2021-05-31
Official webpage:
http://www.gm.fh-koeln.de/~naujoks/gbea/
Organizers:
Description:
The purpose of this first contest on open-endedness is to highlight the progress in algorithms that can create novel and increasingly complex artefacts. While most experiments in open-ended evolution have so far focused on simple toy domains, we believe Minecraft - with its almost unlimited possibilities - is the perfect environment to study and compare such approaches. While other popular Minecraft competitions, like MineRL, have an agent-centric focus, in this competition the goal is to directly evolve Minecraft builds.
As part of this competition, we introduce the Minecraft Mechanical Creations Environment (MMCE) API. The MMCE is implemented as a mod for Minecraft that allows clients to manipulate blocks in a running Minecraft server programmatically through an API. The framework is specifically developed to facilitate experiments in artificial evolution. The competition framework also supports the recently added "redstone" circuit components in Minecraft, which allowed players to build amazing functional structures, such as bridge builders, battle robots, or even complete CPUs. Can an open-ended algorithm running in Minecraft discover similarly complex artefacts automatically?
In contrast to the Minecraft Settlement Generation Challenge, this competition is more about - but not exclusively focused on - the evolution of mechanical/functional artefacts.
Submission deadline:
2021-05-31
Official webpage:
Organizers:
Description:
In order to promote research in black-box optimization, we organize a competition around Nevergrad (https://facebookresearch.github.io/nevergrad/index.html ) and IOHprofiler (https://iohprofiler.github.io/ ).
The competition has two tracks:
Track 1: Performance-Oriented Track: Contributors submit an optimization algorithm as a pull request in Nevergrad as detailed below. Several subtracks (“benchmark suites”) are available, covering a broad range of black-box optimization scenarios, from discrete over mixed-integer to continuous optimization, from “artificial” academic functions to real-world problems, from one-Shot Setting over sequential optimization to parallel settings.
Track 2: General Benchmarking Practices: contributions are made by pull request to Nevergrad or IOHprofiler and are accompanied by a “paper style” documentation. This track invites contributions to all aspects of benchmarking black-box optimization algorithms, e.g., by suggesting new benchmark problems, performance measures, or statistics, by extending or improving the functionalities of the benchmarking environment, etc.
Deadline: June 30 for competition entries. Authors are invited to submit a paper describing their work by April 12; such a documentation is however not required for participating in the competition.
Submission deadline:
2021-06-29
Official webpage:
http://www-ia.lip6.fr/~doerr/OpenOptimizationCompetition2021.html
Organizers:
Description:
Similar to the many previous competitions, the team of the Institute of Data Science, Engineering, and Analytics at the TH Cologne (IDE+A), hosts the 'Industrial Challenge' at the GECCO 2021.
This year’s industrial challenge is posed in cooperation with an IDE+A partner from health industry and with Bartz & Bartz GmbH.
Simulation models are valuable tools for resource usage estimation and capacity planning. Your goal is to determine improved simulation model parameters for a capacity and resource planning task for hospitals. The simulator, babsim.hospital, explicitly covers difficulties for hospitals caused by the COVID-19 pandemic. The simulator can handle many aspects of resource planning in hospitals:
- various resources such as ICU beds, ventilators, personal protection equipment, staff, pharmaceuticals
- several cohorts (based on age, health status, etc.).
The task represents an instance of an expensive, high-dimensional computer simulation-based optimization problem and provides an easy evaluation interface that will be used for the setup of our challenge. The simulation will be executed through an interface and hosted on one of our servers (similar to our last year's challenge).
The task is to find an optimal parameter configuration for the babsim.hospital simulator with a very limited budget of objective function evaluations. The best-found objective function value counts. There will be multiple versions of the babsim.hospital simulations, with slightly differing optimization goals, so that algorithms can be developed and tested before they are submitted for the final evaluation in the challenge.
The participants will be free to apply one or multiple optimization algorithms of their choice.
Thus, we enable each participant to apply his/her algorithms to a real problem from health industry, without software setup or licensing that would usually be required when working on such problems.
Submission deadline:
2021-06-30
Official webpage:
Organizers:
Description:
In this competition, participants will solve a collection of real-world multi-objective optimization problems using their algorithms. The results can be submitted as a two page long competition paper. Detailed results can be placed online and submitted to
Submission deadline:
2020-04-30
Official webpage:
https://www3.ntu.edu.sg/home/epnsugan/index_files/cec-benchmarking.htm
Organizers:
Description:
This competition aims to motivate work in the broad field of logistics. We have prepared a benchmarking framework which allows the development of multi-agent swarms to process a variety of test environments. Those can be extremely diverse, highly dynamic and variable of size. The ultimate goal of this competition is to foster comparability of multi-agent systems in logistics-related problems (e. g., in hospital logistics). Many such problems have good accessibility and are easy to comprehend, but hard to solve. Problems of different difficulty have been designed to make the framework interesting for educational purposes. However, finding efficient solutions for different a priori unknown test environments remains a challenging task for practitioners and researchers alike.
Following these ideas, in the AbstractSwarm Multi-Agent Logistics Competition, participants must develop agents that are able to cooperatively solve different a priori unknown logistics problems. A logistics problem is given as a graph containing agents and stations. An agent can interact with the graph (1) by deciding which station to visit next, (2) by communicating with other agents, and (3) by retrieving a reward for its previous decision. While simulating a scenario, a timetable in the form of a Gantt-chart is created according to the decisions of all agents. Submissions will be ranked according to the total number of idle time of all agents in several different a priori unknown problem scenarios in conjunction with the number of iterations needed to come to the solution.
Submission deadline:
2021-05-14
Official webpage:
https://abstractswarm.gitlab.io/abstractswarm_competition/