NiloCK17 minutes ago
> Automation doesn't make operators more careful. It makes them forget how to be. The more reliable the system, the less ready the human.
The entire premise of a system is that it removes the need for careful attention.
system: signal lights tell me whether or not I can pass through an intersection, so that I do not have to attend to potentially high speed traffic from a variety of directions.
system: the side my knife blade sits on my arched guide fingers, so that I do not have to attend to the edge of the blade or the location of my fingers.
etc etc.
bumby2 minutes ago
>The entire premise of a system is that it removes the need for careful attention.
I think this premise is flawed or, at best, too narrow. A system is just a logical grouping of items that perform a function. Sometimes that function can be to reduce cognitive burden, but it doesn't have to be. A "vision system" like what humans use does not reduce attention, but increases/enables it, while a building automation system can reduce attention. The ability to increase/reduce attention is not the central principle of a system.
[deleted]8 minutes agocollapsed
andai14 minutes ago
> system: signal lights tell me whether or not I can pass through an intersection, so that I do not have to attend to potentially high speed traffic from a variety of directions.
You know I noticed this... I lived in a country where people obey traffic laws, and in a country where they very much don't.
I witnessed many more traffic accidents in the country where people are used to relying on the traffic lights to tell them if it's safe or not.
Whereas in the other country, everyone correctly assumes that the other drivers are completely insane, and so they stay vigilant.
infecto4 minutes ago
Other than the data of road fatalities that disproves this anecdote, my own anecdote is this is the false sense of security people get in other countries that don’t have traffic laws. Oh see the people have to look all the time so it’s much safer. When you start to live it for a long time you realize it’s not true. Many more fatalities.
Now I do think the science shows if you design roads and systems to make drivers more thoughtful it can improve outcomes. Size roads for the speed limit, roundabouts, etc. these can make a difference as it balances the system.
pjc5011 minutes ago
There's some documented studies of removing all the street clutter and lines from residential area intersections forcing drivers to be more careful, especially around pedestrians, reducing overall accidents. But this does reduce throughput slightly.
raincole11 minutes ago
Except if you look at this map: https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...
People die on road more in countries that conventionally don't follow traffic laws.
infecto7 minutes ago
Not sure why this would downvoted. Go to a less developed nation where traffic laws are not important and it’s one of those sense of false security ideas.
singpolyma334 minutes ago
I don't think we believe that the automation makes the human operators safer, but that it is overall safer than the human operator alone.
bumby17 minutes ago
I think part of the point of the article is that it also makes edge-cases more dangerous and catastrophic than if there was no autopilot at all. From the article:
>The argument for automation is that it frees up cognitive bandwidth. Fewer routine decisions means more headroom to think carefully about the ones that matter.
So if the expectation is that the human pilot is expected to pay attention to mitigate the dangerous edge cases "that matter", there is a contradiction: the tool that promises to free up the bandwidth for that attention creates a complacency that prevents that attention from being applied.
In other words, it makes the normal situations safer but the abnormal situations more dangerous.
adampunk8 minutes ago
Landing is an abnormal situation for an aircraft which we make SUBSTANTIALLY less dangerous through intense automation. Do you want to rip out automated landing systems?
bumby6 minutes ago
If you expect every aircraft to land, it seems to meet the very definition of "normal" operation.
An abnormal landing would be something like trying to land with a broken elevator surface.
adampunk3 minutes ago
And you contend that autopilot makes that situation more dangerous? Do you have any support for this?
palmotea15 minutes ago
> I don't think we believe that the automation makes the human operators safer, but that it is overall safer than the human operator alone.
If the automation is for the easy/routine stuff, then no. The automation doesn't work in exactly the most safety-critical situations, and then the human operator is thrust into fixing the situation without the full context.
kqr19 minutes ago
Well, it's differently safer. It's impossible to make categorical statements of more or less safe with automation because it depends massively on the design.
cyanydeez30 minutes ago
ijterestingly, AI tools are improving code theoughput but also elevating catastrophic error rates.
even if the average rate goes up tor net benefit, are organizations prepared for increased carastrophic failures?
supriyo-biswas34 minutes ago
Also, ironies of automation[1], etc.
[1] https://static1.squarespace.com/static/644321e78cd2dd37613af...
jonesai6 minutes ago
In this case the only reason I'm a pilot is the existence of autopilot haha
analogpixel31 minutes ago
They should get an auotpilot to fix their website to render on desktops and not just phones.
adampunk14 minutes ago
The crew and the plane are a single system. It is meaningless to imagine pilot “skill” without the plane. Further, we killed so many pilots in the 20th century through impossible workloads which we have now automated, it’s almost cruel to be wistful for it.
I know this is an analogy to AI, but I wish we would dispense with this idea that there’s some appropriate level of machinery which was reached just a hair before right now. There is no appropriate level of machinery, no point at which the nature of the system itself will unambiguously say “that’s enough.”
raincole9 minutes ago
And that's the whole freaking point. If the humans have to obtain and maintain the exact same skills as before than what's the point of automation?
[deleted]34 minutes agocollapsed
jonstewart20 minutes ago
Am I missing something? Is TFA only 2-3 paragraphs of a generic metaphor, with no actual data/research from aviation (or other fields) to back up the core thesis?
brindleth14 minutes ago
Yet another AI written slop article hitting the top of HN.
Evidence: Look at the most recent article on this blog: https://julienreszka.com/blog/difficult-conversations-don-t-...
"Memory is reconstructive. When someone recalls events differently, it feels like gaslighting. It usually isn't. Document first, then negotiate."
Bleugh
swordlucky66616 minutes ago
[dead]
kotaKat35 minutes ago
American Airlines captain Warran VanderBurgh once called this phenomenon "the children of the magenta"[1] in his talk on automation dependency in the 90s.
I wonder what we'd call the children today in hindsight and what line they're chasing now...