After a configuration change, the Jitsi Video Bridges hadn't been restarted which caused all conferences to be located to a single video bridge. Thus, Jitsi's performance dropped significanly. The bridges have been restarted and the issue is fixed.