Lyle Wallis is my hero. Every time I visit the (awesome)
AnyLogic conference, there is one presentation that consistently puts me on the edge of my seat. It is Lyle talking about the latest toying they do with simulation at PWC. This year was no different as he shared how they try to
combine machine learning techniques with simulation modelling. One of the animations really captured my imagination: a little agent was frantically trying to reach a target on a grid world. So, I took some time to finally read up on the machine learning technique around it and reproduce it myself.
A simple reinforcement learning pathfinder simulation I built, inspired by Lyle Wallis at PWC.
Empty space, drag to resize
In the next few posts, I want to discuss machine learning and AI as part of simulation. Let’s start by thinking about why we would want to combine these worlds anyway, before I will share how you can get started practically (for example I recently built a reinforcement learning algorithm within AnyLogic without an external ML library in less than a day). Interestingly, we will see how the agent-based structure of AnyLogic can massively simplify your entry into the world of machine learning. But let’s not get ahead of ourselves…
If I tell you that simulation is extremely powerful and massively undervalued, I likely preach to the choir. If I tell you that machine-learning (ML) can be extremely successful at solving business problems, you will yawn. However,
have you noticed how they never work together? Weird, isn’t it? They both apply “models”, they both handle data, they both are good at solving certain problems. Why are they mixed so rarely?
The main reason for the divide I see is this: simulation models are build “process-centric” while ML models are built “data-centric”. What I mean is this:
- For a simulation model, the modeler needs to understand the processes going on, he needs to talk to operators and decision-makers. He needs to gather the relevant data. He needs to see and observe the system (ideally on site)
- For a ML model, the modeler needs data. More often than not, that is it. Very often, data scientists are thrown at with data and the instruction “go find something”. Very little (if any) situational awareness is required by the modeller.
Hence, simulation modelers and data scientists are a different breed with a different skillset. Since it is unlikely that the data scientists of this world will collectively decide to embrace this fringe simulation modelling stuff we love, I suggest we go the other way and embrace ML.
Why would we even want to combine ML and simulation? Well, simply put, two successful techniques will likely solve problems that we cannot solve separately thus far. Several ideas:
- A factory has no documented rules and policies to run its operations. Everyone tells you something else. It is impossible to model any heuristics with the traditional explicit simulation approach.
- Use a decision-tree model to tell you the rules by looking at historic data
- People or process-docs describe how they behave but they (intentionally or not) are not telling you their actual behavior
- Again, let a ML model figure out the actual historic heuristics
- Path-finding algorithms are not good enough to navigate through a simulation environment
- Reinforcement learning helps find optimal paths
- You struggle to match the simulation model output with real KPIs
- Let a ML model “steer” the simulation to match reality
- The real system implements advanced ML algorithms to make decisions
To simulate such a system, you need to be able to reproduce the ML algorithms within the simulation itself
There are likely many more possibilities and like with most innovations, some will only reveal themselves once we start playing. Just like Lyle at PWC.
Before sharing some concrete advice how to include ML with AnyLogic in upcoming posts, let’s think about the various modes of combining simulation models and ML
models. I can see three ways, each for specific use cases:
- ML prior to the simulation: Apply ML to process input data in order to make it usable for the simulation model. The most obvious approach would be to develop data-driven decision heuristics that agents can apply. You can run the ML outside the simulation model and decouple both.
- ML within the simulation: Make the simulation learn certain aspects and apply that learning directly. This coupled approach makes sense when you train the ML algorithms on specifics of the simulation model itself, such as pedestrians trying to find a path within the simulation model environment. I see three ways to split this further:
- Either train the ML algorithm as part of the simulation warm-up
- Re-use previously trained ML models
- Or train the ML model as the simulation goes along (i.e. for reinforcement learning)
- ML after the simulation run: Take the simulation output and feed it into an ML algorithm. I currently see little use of this on small scales currently. However, I can see this used in very advanced AI algorithm training such as autonomous driving. You train the models not only with real cars but let the AI “drive” through simulation cities as well. This takes the concept of simulation models as algorithm testbeds further than ever.
I think we currently see a very exciting time for the simulation-modelling world. As ML models mature beyond the hype cycle (weeding out where they do not deliver real value), they become usable for simulation models. This is very early days to predict where this will go but it sure will become a standard part of models in the future.
Remember 2 decades ago when agent-based modelling joined the scene? Fringe models appeared but it took a while to understand the potential for improving the traditional paradigms and modelling strategies. Nowadays, even pure discrete-event vendors claim to include agent-based functionality and it has become a normal part of many models.
Similarly, I believe we will see that ML will join simulation modelling as another technique to solve problems. Let’s find out how best to get started in future blog posts. Also, very interested to hear your views on ML and simulation: what application areas do you see? What challenges should be overcome? Do we need it at all?
