American Airlines Grounded by iPads

[Philippus Flavius Homovallumus]

http://www.bbc.co.uk/news/technology-32513066

More specifically, they were grounded by a non-functional iPad App they used to check their flight plans, and weren't carrying paper backups.

Let's reflect on the fact that a major freight and passenger hauler was relying on the Cloud and a personal computing company for navigation - and by glad no body was hurt when that went belly up.

[Sir Moody]

http://www.bbc.co.uk/news/technology-32513066

More specifically, they were grounded by a non-functional iPad App they used to check their flight plans, and weren't carrying paper backups.

Let's reflect on the fact that a major freight and passenger hauler was relying on the Cloud and a personal computing company for navigation - and by glad no body was hurt when that went belly up.

Not to call you a luddite but...

This kind of thing has been going on for a while and is a MASSIVE cost saving - the kind of paper backup's companies used to rely on is a) expensive and b) Labour intensive (and in the case of Airplanes where extra weight costs more fuel also HEAVY)

The real issue here is one of Software testing, speaking as an insider who has been migrating the company I work for into the cloud (Microsofts in our case), the cloud has led Software engineers to push updates out far quicker but with less testing simply because their customers are always connected and thus will all update at once - in this case the glitch was with a piece of software provided by boeing, I would hazard a guess that they had just pushed an update out which has "issues".

[Philippus Flavius Homovallumus]

Not to call you a luddite but...

This kind of thing has been going on for a while and is a MASSIVE cost saving - the kind of paper backup's companies used to rely on is a) expensive and b) Labour intensive (and in the case of Airplanes where extra weight costs more fuel also HEAVY)

The real issue here is one of Software testing, speaking as an insider who has been migrating the company I work for into the cloud (Microsofts in our case), the cloud has led Software engineers to push updates out far quicker but with less testing simply because their customers are always connected and thus will all update at once - in this case the glitch was with a piece of software provided by boeing, I would hazard a guess that they had just pushed an update out which has "issues".

You completely missed the point.

The flight plans should be on the plane's own internal computer, which should be receiving encrypted data from American Airlines' own servers this would, among other things, prevent the update issue you mentioned. The Cloud is all well and good if you're sending people weather reports or news but Civilian Airlines should have the same sort of fully internal systems as the military does, you shouldn't be using a third party like Apple to actually run the system.

[Sir Moody]

You completely missed the point.

The flight plans should be on the plane's own internal computer, which should be receiving encrypted data from American Airlines' own servers this would, among other things, prevent the update issue you mentioned. The Cloud is all well and good if you're sending people weather reports or news but Civilian Airlines should have the same sort of fully internal systems as the military does, you shouldn't be using a third party like Apple to actually run the system.

Hmmm that is a interesting point but really its just a shift of whose cloud you use - your own (in which case you are responsible for maintaining a large array of very expensive computers and a complicated network) or a third party who has dedicated a large amount of resources to maintaining an immense number of very expensive computers and is letting you rent a part of that at a fraction of the cost.

again from experience the Third party option wins out in every way - primarily due to scaling.

If you are maintaining your own cloud in order to add extra resources you need to add more computers - if you are using a third party you simply rent more resources.

If you no longer need the extra resources you are stuck with expensive computers if you are hosting and will need to go through the costly process of removing and recycling them. If you are renting you simply reduce your allocated resources...

I am not saying using third parties is flawless but generally its far more cost effective and reliable (Hosting companies will spend a lot to make sure their cloud doesn't go down- and if it does they are liable not their clients)

[Montmorency]

The flight plans should be on the plane's own internal computer, which should be receiving encrypted data from American Airlines' own servers this would, among other things, prevent the update issue you mentioned. The Cloud is all well and good if you're sending people weather reports or news but Civilian Airlines should have the same sort of fully internal systems as the military does, you shouldn't be using a third party like Apple to actually run the system.

You'd be surprised just how much third-party software and hardware the militaries of the world use.

[Kadagar_AV]

Sir Moody SCHOOLED ya'll...

[Philippus Flavius Homovallumus]

Hmmm that is a interesting point but really its just a shift of whose cloud you use - your own (in which case you are responsible for maintaining a large array of very expensive computers and a complicated network) or a third party who has dedicated a large amount of resources to maintaining an immense number of very expensive computers and is letting you rent a part of that at a fraction of the cost.

again from experience the Third party option wins out in every way - primarily due to scaling.

If you are maintaining your own cloud in order to add extra resources you need to add more computers - if you are using a third party you simply rent more resources.

If you no longer need the extra resources you are stuck with expensive computers if you are hosting and will need to go through the costly process of removing and recycling them. If you are renting you simply reduce your allocated resources...

I am not saying using third parties is flawless but generally its far more cost effective and reliable (Hosting companies will spend a lot to make sure their cloud doesn't go down- and if it does they are liable not their clients)

Yes, I'm aware of the problem - reading over your post I see that the main plank of your argument is cost. Now, cost is certainly important, even for the military, but the reality is that reliability is much more important for something life this, and the ongoing cost of renting the service is usually more than the cost of running it yourself, certainly for something like a large airline. Reliability is also not great - going back to the military thing - I once stood guard in a guardroom with a broken armoury alarm. That meant that it was impossible to know if someone was braking into the armoury, it was an old camp without CCTV. Why was it broken? Because the German-based company the British Army was renting their own camp from hadn't fixed it.

[Husar]

Why was it broken? Because the German-based company the British Army was renting their own camp from hadn't fixed it.

You really can't trust those Germans, it's becoming increasingly obvious.

[Sir Moody]

Yes, I'm aware of the problem - reading over your post I see that the main plank of your argument is cost. Now, cost is certainly important, even for the military, but the reality is that reliability is much more important for something life this, and the ongoing cost of renting the service is usually more than the cost of running it yourself, certainly for something like a large airline. Reliability is also not great - going back to the military thing - I once stood guard in a guardroom with a broken armoury alarm. That meant that it was impossible to know if someone was braking into the armoury, it was an old camp without CCTV. Why was it broken? Because the German-based company the British Army was renting their own camp from hadn't fixed it.

You really underestimate how much Clouds cost to build and operate - these are MASSIVE arrays of very powerful computers which require constant cooling.

Think about it this way for second - you run a Business which requires a large amount of computing power 5 days a week - if you buy your own machines 2 days a week you will be paying upkeep on machines you are not using. If you Rent you pay for what you use.

On reliability - if you buy the machines you have to maintain the hardware, the area that houses the hardware, the network that the machines are connected into and all software installed.
If you are renting you have a company that is reliant on the cloud working to make its money at your beck and call to make sure everything is Ok... all you need to do is make sure your internal network can contact the external cloud.

so on the Rent side of things it is both A) far cheaper and B) far more reliable.

The only benefit you can gain from controlling your own cloud is the security of knowing all data is handled in house - you would be surprised by the number of business that don't care about that... especially when you consider how difficult and expensive it is to maintain a secure network as Sony as found out...

[GrnEyedDvl]

The flight plans should be on the plane's own internal computer, which should be receiving encrypted data from American Airlines' own servers this would, among other things, prevent the update issue you mentioned.

I have a rather unique perspective on this as my wife used to work (up until last summer) for Jeppesen which is the company that produces all the charts for every airline in the world. She was actually one of their control desk managers who controlled the entire printing process for the paper charts and was involved in some of the switchover to the iPad when it first started being tested.

The first thing to understand here is what it costs to produce the paper charts. Jeppesen has a printing facility here in Colorado that probably still has 100 or more employees, and when my wife left that printing operation ran 24 hours a day 6 days a week. That is expensive, and all of that cost is transferred to you every time you buy an airline ticket. FedEx has several trucks per day that pick up charts, and then they fly them out to the various airlines who then ship them to the various airports around the world. You cannot simply email the file to an airport and let the airport print things locally. The paper these are printed on is 30 or 36 inches wide and can be up to 6-7 feet long depending on the chart. If you look at the pic in the article posted above you will see one stack that has a big rubber band around it, that is one of the charts that folds out and can be 6 feet long. These things a printed on huge printing presses like this one that cost like $50,000 each. I know when they installed 4 new ones a couple of years ago the setup time and costs were pretty crazy. So the incentive for a better solution is a real one.

When they started testing iPad stuff a few years ago, Apple built special versions of the iPad just for this. Sort of like Blackberry doing special things for Obama's phone. I have actually seen (but not touched) one of them. I am not sure of all the details but these are not retail iPads, and they work on their own cloud system.

When an airline is scheduled to go someplace, they have to have charts for the originating airport, the destination airport, and a bunch of airports in between for emergency landings or other detours (weather, whatever) that cause them to land someplace else. In addition, they have to have not just charts but all kinds of other paperwork for all these destinations. Stuff like terrorist plans and emergency procedures, airspace info and landing patterns, and probably a ton of other things I have no idea about. You can get an idea of what they need by using iflightplanner.com and just clicking around a bit. Just zoom in on anything you want and you will see a bit of what I mean. Its quite a bit of stuff, and commercial airliners have stricter requirements than private pilots do.

Its all very complicated and the more airplanes, airlines, and general aviation flights there are out there the more complicated it gets. All of these charts have to be constantly updated, printed, and shipped out. And damn near anything can trigger the requirements for charts to be updated. Airport construction, construction around an airport, a tree being planted, you name it. In fact one of reasons for the push to go digital was actually the tsunami that took down the Fukishima power plant a few years ago. Many people do not know it, but the runways at the Tokyo airport moved forcing an update of the charts.

FAA regulations prohibit commercial pilots from flying without updated charts. The reason this was a problem during the Fukishima crisis is because we were trying to ship literally tons of aid into the area. The US government actually called privately owned jets into service. There is a rule that says if the government loans your company money to buy an airplane, they can call that airplane into government service for emergencies. And they did this, they called on United and others to provide planes to ship supplies into Tokyo and support the military operations. I posted this on TWC a few years go, but the short version is that those pilots could not land in Tokyo because of the lack of updated charts and the FAA would not relax the rules for the flights. They ended up landing in Singapore and China and other places, moving the cargo to military planes, and then flying onto Tokyo because military pilots do not have the same restrictions. It was a disaster.

My wife ended up working like 36 hours straight as they updated charts, printed them by the thousands, and shipped them out. FedEx had a truck sitting on the dock running as stuff was coming off the presses and then that truck got a police escort to Denver International where it was just driven straight into a plane with priority takeoff so they could get the new charts into the hands of people needing to fly emergency supplies into Tokyo.

The digital version eliminates all of those problems since once its updated it can be loaded onto one of the iPads from just about anywhere. It may not be perfect 100% of the time but its been running since late 2012 now and I rarely hear of an issue with it, and the paper process is far from perfect as well.

[Montmorency]

Excellent post.

[Greyblades]

Well, my belief that relying on computers should never be the only option available to those operating heavy machinery has been reinforced. Tech's great, and we should use it, but until it become's fool-proof we shouldn't be completely ditching the old methods in case the new ones fail.

[Sir Moody]

Well, my belief that relying on computers should never be the only option available to those operating heavy machinery has been reinforced. Tech's great, and we should use it, but until it become's fool-proof we shouldn't be completely ditching the old methods in case the new ones fail.

What if the Heavy Machinery they are operating are essentially a computer to begin with?

Modern planes are essentially flying computers - hell most modern fighter planes cant even fly without the computer, if it breaks they literally fall out of the sky.

This is also true of modern cars - mine cant start without its onboard computer.

The simple truth is Computers are part of almost everything we do today and "clinging" to old methods with serious problems simply because they work is a really bad idea - using these old methods as a back up is still valid (and what the pilots in this case did... stupid story really)

[Montmorency]

If you want to make a real story about this, then make it about outsourcing (as opposed to "offshoring").

[Greyblades]

What if the Heavy Machinery they are operating are essentially a computer to begin with?

Modern planes are essentially flying computers - hell most modern fighter planes cant even fly without the computer, if it breaks they literally fall out of the sky.

This is also true of modern cars - mine cant start without its onboard computer.

The simple truth is Computers are part of almost everything we do today and "clinging" to old methods with serious problems simply because they work is a really bad idea - using these old methods as a back up is still valid (and what the pilots in this case did... stupid story really)

I retract my doubt of tech but I retain that there should have been a backup.

I'm a big believer in backups and safetys and these guys were caught out without one. The least they could have done is had a backup ipad, or better yet had the documents on a cheap e-reader or something.

[Sir Moody]

I retract my doubt of tech but I retain that there should have been a backup.

I'm a big believer in backups and safetys and these guys were caught out without one. The least they could have done is had a backup ipad, or better yet had the documents on a cheap e-reader or something.

I disagree - Id say the story shows they had a very rigorous safety system in place - the pilots all checked their systems before takeoff and upon finding a fault canceled the take off until paper copies could be produced (the fault in question was with software not the actual Ipad so any backup Ipads would have had the same fault)

We should be praising American Airlines for having such a well thought out and executed system instead of chiding them for reliance on technology.

The really interesting question should be to Boeing about why they pushed out a glitchy patch to a safety critical piece of software

[Husar]

I'm not sure whether I would use cloud-reliant apps on an airplane, make them work offline and give them a spare in case the first one has a failure and most scenarios should be unproblematic.

[Sir Moody]

I'm not sure whether I would use cloud-reliant apps on an airplane, make them work offline and give them a spare in case the first one has a failure and most scenarios should be unproblematic.

The software in question is not cloud reliant - it is designed to work offline (according to Boeing's website) - it seems to work by downloading the maps while the plane is at the gate and connected and then running offline while the plane is in the air - id hazard a guess that the problem was in a failure to download the maps (since the article mentions the pilots having to return to the gate to fix the issue)

[Husar]

The software in question is not cloud reliant - it is designed to work offline (according to Boeing's website) - it seems to work by downloading the maps while the plane is at the gate and connected and then running offline while the plane is in the air - id hazard a guess that the problem was in a failure to download the maps (since the article mentions the pilots having to return to the gate to fix the issue)

Well, I made a somewhat generic statement since some people mentioned that but the article doesn't indeed. It just says the app was failing due to duplicate data and they had to re-download it at the gate once the error was fixed.

This will be a non-issue once the planes are drones and fly without pilots.

[Kadagar_AV]

Well, I made a somewhat generic statement since some people mentioned that but the article doesn't indeed. It just says the app was failing due to duplicate data and they had to re-download it at the gate once the error was fixed.

This will be a non-issue once the planes are drones and fly without pilots.

Oh you DIDN'T go there...

You know, recently it's only been Germans and Arabs who willfully crash planes killing hundreds or thousands of people... Does that mean that we can be safe with humans as long as they are not Arabs or Germans?

[Sir Moody]

This will be a non-issue once the planes are drones and fly without pilots.

Its inevitable really - the planes already fly themselves with Human oversight so its only a small step to remove the pilot entirely...

I suspect they wont go all in however and the first move will be to remove the Pilot from the plane and have them instead connected like the current Reaper drone pilots (ie sitting in a secure facility in a centralised location), they still get the benefit of "trust" then - as this thread shows some people just wouldn't trust a machine without any Human oversight...