CSLE is a platform for evaluating and developing reinforcement learning agents for control problems in cyber security. It can be considered as a cyber range specifically designed for reinforcement learning agents. Everything from network emulation, to simulation and implementation of network commands have been co-designed to provide an environment where it is possible to train and evaluate reinforcement learning agents on practical problems in cyber security. The platform can also be used to combine reinforcement learning with other quantitative methods, e.g., dynamic programming, computational game theory, evolutionary methods, causal inference, and general optimization.

CSLE includes a system for emulating large scale IT infrastructures, cyber attacks, and client populations. It is based on Linux containers and can be used to collect traces and to evaluate security policies.

Note The emulation system is mainly designed to run on a distributed system, e.g., a compute cluster. It can run on a laptop as well, but then only small emulations can be created.

CSLE includes a simulation system for executing reinforcement learning algorithms and simulating Markov decision processes and Markov games. It is built in Python and can be integrated with standard machine learning libraries.

Note The simulations are compatible with OpenAI Gym/Gymnasium. Hence you can integrate the simulations with your own implementations of reinforcement learning algorithms.

CSLE includes a system for managing emulations and simulations which can be accessed either through Command-Line Interface (CLI), through a REST API, through Python libraries, or through a web interface. The management system allows a) to start/stop emulations/simulations; b) real-time monitoring of emulation and simulation processes; and c), shell access to components of emulations.

Documentation, installation instructions, and usage examples are available here. A PDF version of the documentation is available here. A video walkthrough of the installation process is available here.

Maintenance releases have a stable API and dependency tree, and receive bug fixes and critical improvements but not new features. We currently support each release for a window of 6 months.

A dataset of 6400 intrusion traces can be found here.

Contributions are very welcome. Please use GitHub issues and pull requests. See the documentation for further instructions.

Thanks go to these people!

Creative Commons (C) 2020-2026, Kim Hammar

• gym-idsgame
• gym-optimal-intrusion-response
• awesome-rl-for-cybersecurity

Made with at , , and