In general, I2C was not designed to support hot swapping. That is - plugging / unplugging devices while the system is powered up and code is running. This is in contrast to USB, for which hot swapping is one of its main design features.
Additionally, the convenience of STEMMA / STEMMA QT cables and connectors gives the impression that I2C does allow hot swapping. And, yes, you may even see simple product demos where this is being done. But in general, this is not a supported feature of I2C.
If your application must support hot swapping, there are potential ways to attempt it. The biggest change will likely be with your code, which must accommodate an I2C target device being disconnected / reconnected at any given time in code execution.
Additionally, augmenting the I2C hardware (which was not designed for this) with something like a TCA4307 buffer is suggested.