Today we had a fire alarm in the office.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
As it gets much more attention, than I expected, here are two clarifications:
1. The Slack message was written after the person evacuated properly. It was written via phone while staying at the designated area outside the building.
2. Nobody asked AI advice explicitly. It was configured to answer automatically if it thinks it can help you. The configuration was updated after this incident. -
As it gets much more attention, than I expected, here are two clarifications:
1. The Slack message was written after the person evacuated properly. It was written via phone while staying at the designated area outside the building.
2. Nobody asked AI advice explicitly. It was configured to answer automatically if it thinks it can help you. The configuration was updated after this incident.@tagir_valeev Ahe ironic thing is that if you look at the probabilities that are represented in the training data, and likely the real world - it was very likely statistically correct. All the more reason that humans should be the arbiters of human decisions not AI or even algorithms
-
As it gets much more attention, than I expected, here are two clarifications:
1. The Slack message was written after the person evacuated properly. It was written via phone while staying at the designated area outside the building.
2. Nobody asked AI advice explicitly. It was configured to answer automatically if it thinks it can help you. The configuration was updated after this incident.@tagir_valeev phew! I’m glad the configuration is updated!
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev Having worked in the Chemical Industry all my life my first instinct is evacuate, carry out roll call and then establish whether or not it was a real fire. My colleagues and I were all fire and rescue trained but would only act on small fires we always called the fire service. Many of the sites I worked on had alarms that went to the local fire control and triggered a turn out, a turn out of 6 engines at one site plus a general alert to other brigades
-
@richlv @tagir_valeev Operators resetting/muting the alarm without understanding why it fired is a perfect example of alarm fatigue. A tragedy like that underscores why it's a Big Fuckin' Deal to avoid it.
The root cause of a failure like that is almost never the dude who did that. It's the circumstances that led to that dude thinking it was the correct thing to do.
Then people die.
My own opinion that evacuation is always the drill. Working on the alarm device, be it wiring, programming, or the noise that comes out of it, is part of working on an end-to-end system that includes people going away from the alarm.
@majick @tagir_valeev As much as I love to talk about alert noise/fatigue, in that case other factors were contributing.
A bit of pressure from the supermarket chain, a bit of leftover mentality from the russian occupation times. Similar to the "I'm not afraid of no virus, I'm not gonna mask!" etc. -
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev Charles Darwin has entered the chat.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev oh, just saw this in my feed and immediately recognized the buildings, that's just around the corner were we live! Hope it wasn't serious and everybody is alright?
-
As it gets much more attention, than I expected, here are two clarifications:
1. The Slack message was written after the person evacuated properly. It was written via phone while staying at the designated area outside the building.
2. Nobody asked AI advice explicitly. It was configured to answer automatically if it thinks it can help you. The configuration was updated after this incident.@tagir_valeev …apparently this is the start of your next book 100 LLM mistakes and how you can’t avoid them.
Actually, I might just steal that my own having written most footnoted blog post ever
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev
Have you performed the MOTHER test and ensured its priority is employee wellbeing and not corporate?Alien had some warnings about AI too
-
@tagir_valeev oh, just saw this in my feed and immediately recognized the buildings, that's just around the corner were we live! Hope it wasn't serious and everybody is alright?
@maxsz no, nothing serious!
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
-
@tagir_valeev Having worked in the Chemical Industry all my life my first instinct is evacuate, carry out roll call and then establish whether or not it was a real fire. My colleagues and I were all fire and rescue trained but would only act on small fires we always called the fire service. Many of the sites I worked on had alarms that went to the local fire control and triggered a turn out, a turn out of 6 engines at one site plus a general alert to other brigades
@aadeacon here the fire brigade is alerted automatically as well. If I understand correctly, the building owner cannot deactivate the alarm without the fire brigade.
-
Today we had a fire alarm in the office. A colleague wrote to a Slack channel 'Fire alarm in the office building', to start a thread if somebody knows any details. We have AI assistant Glean integrated into the Slack, and it answered privately to her: "today's siren is just a scheduled test and you do not need to leave your workplace". It was not a test or a drill, it was a real fire alarm. Someday, AI will kill us.
@tagir_valeev@mastodon.online Simply because it saw that a common response was to indicate it was a drill, untethered from any real signals. And of course these glorified autocomplete prediction systems are being forced everywhere
-
@majick @tagir_valeev As much as I love to talk about alert noise/fatigue, in that case other factors were contributing.
A bit of pressure from the supermarket chain, a bit of leftover mentality from the russian occupation times. Similar to the "I'm not afraid of no virus, I'm not gonna mask!" etc.@richlv @tagir_valeev Yeah, I completely agree those factors can very much influence how bad something like that turns out to become. Rarely does someone think a thing they do routinely is actually high risk.
I'd say they fall into the category of "factors" I mentioned as root causes: cultural, management-based (I've lived that life), and so on. Fatigue's one thing, for sure, not the only thing, and sometimes it's stuff like that deprioritizing the seriousness of an alarm. A different cause of fatigue/blasé attitude/misinterpretation.
Like I said, it's almost never the dude and almost always what influenced the dude to be the last link in the chain up to a horrible tragedy.
-
W wiase@ibe.social shared this topic
